Chapter 20: The Available Standard Library Subset

Almost all of Loup's standard library is hosted code: it expects the Loup runtime, an operating system, exceptions or a heap. A freestanding image has none of those, so what it can use is small and precise. This chapter lists it:

• what the compiler provides with no import at all;
• the four standard-library modules a freestanding file can import;
• why every other module is unavailable, and what the Language Reference plans;
• the routines you write yourself, and the raw-memory intrinsics they rest on;
• what happens to C code declared with extern native.

Built into the compiler

These need no import in a @mode(freestanding) file:

The last row deserves an example. An @interchange struct declares the exact byte layout and byte order of a record, and the compiler generates heap-free methods that read it from, and write it to, an address. That is what you want for network headers and on-disk structures, which are big-endian on the wire whatever the CPU is:

@mode(freestanding)

# The first bytes of an Ethernet payload carrying IPv4 (big-endian on the wire).
@interchange(endianness: Big)
struct FrameStart {
    @field(encoding: Binary, length: 2) pub var etherType: uint16 = 0;
    @field(encoding: Binary, length: 1) pub var versionIhl: uint8 = 0;
    @field(encoding: Binary, length: 1) pub var tos: uint8 = 0;
    @field(encoding: Binary, length: 2) pub var totalLength: uint16 = 0;
}

const ETHERTYPE_IPV4: uint16 = 0x0800u16

func isIpv4(frame: uint): bool {
    let h = FrameStart.unpackFrom(frame)
    return h.etherType == ETHERTYPE_IPV4 and (h.versionIhl >> 4) == 4u8
}

func writeFrameStart(frame: uint, length: uint16): void {
    var h: FrameStart;
    h.etherType = ETHERTYPE_IPV4
    h.versionIhl = 0x45u8                 # IPv4, 20-byte header
    h.totalLength = length
    h.packInto(frame)
}

Listing 20-1: Reading and writing a big-endian header with @interchange

unpackFrom and packInto read and write memory at a caller-supplied address, so they count as raw memory access (see "Raw memory access" below). For a device register block, Volatile.view[I] gives the same unpacking with volatile accesses (chapter 15).

The freestanding modules

Four modules in the standard library are written in @mode(freestanding) and can be imported:

The debug service exists only in loup build --debug images. A release build rejects the import (E_FREESTANDING_MODULE: "debug.freestanding is available only in --debug freestanding builds"), so that a release image cannot silently contain calls that do nothing; the Language Reference names this case E_FREESTANDING_DEBUG_START_RELEASE. A kernel starts the service once and polls it from its main loop:

from debug.freestanding import (poll, start, FreestandingDebugConfig, ControlPollMode);

func startDebugService(): void {
    start(FreestandingDebugConfig(
        controlPollMode = ControlPollMode.Polling,   # poll() handles one command per call
        traceFlushThreshold = 0,                     # send each event as it happens
        traceFlushIntervalUs = 0,
        timeTravelEnabled = false,
        timeTravelRingFrames = 0))
}

func mainLoopTick(): void {
    poll()
    # ... the rest of the loop's work ...
}

Listing 20-2: Starting and polling the freestanding debug service

On the host, loup debug attach --uart <device> or loup debug attach --tcp <host:port> connects to it. The transports themselves are functions you write and mark @debug_control_channel or @debug_trace_channel; the Language Reference describes the protocol (§3-3.25).

Hosted modules cannot be imported

Every other module — core.math, system.text, io.fs, debug.assert and the rest — is hosted code, and a freestanding file cannot import it. The natural first attempt fails at the import:

@mode(freestanding)

from core.math import (sqrt);

func diagonal(width: uint64, height: uint64): uint64 {
    return sqrt(((width * width) + (height * height)) as float) as uint64
}

error[E0001]: `@mode(freestanding)` module cannot import `@mode(host.scripted)` module
 --> src/main.lpfree:3:1
  |
3 | from core.math import (sqrt);
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |

The message names the module's mode, not the fact that it lives in the standard library: the rule is the general one that freestanding code imports only freestanding code. The same applies to the native.* modules the Language Reference lists for freestanding, which do not exist yet:

@mode(freestanding)

import native.mem as mem;

func clearPage(page: uint): void {
    mem.zero(page, 4096u64)
}

error[E0001]: 'native.mem' is not a standard-library module (namespace 'native' is reserved for the standard library)
 --> src/main.lpfree:3:8
  |
3 | import native.mem as mem;
  |        ^^^^^^^^^^
  |

error[E0001]: undefined identifier `mem`
 --> src/main.lpfree:6:5
  |
6 |     mem.zero(page, 4096u64)
  |     ^^^
  |

⚠ Planned. The Language Reference's freestanding subset (§3-3.20) also lists core.math, core.text, core.complex, core.array, core.fpenv, core.interchange, native.mem (copy, zero, compare, alloc, free, tryAlloc), native.simd, native.atomic (with hwFence and MemoryOrder), native.ffi, native.dma (DmaBuffer[T]) and native.mpu (installStackGuard). None of these can be imported by freestanding code in Loup 1.25: the core.* modules that ship are hosted, native.atomic is a host.native module, and the other native.* modules do not exist. (The @interchange methods above, which core.interchange would package, are already generated by the compiler.)

The prelude's hosted functions are a trap of a different kind. println, print, exit, readText and the like are not rejected by the 1.25 checker in a freestanding file, but there is no console, process or file system behind them. Do not call them; write to your own UART (chapter 15) and stop with your own fatal (chapter 17).

Writing the pieces yourself

Kernels have always carried a small library of their own: integer helpers, formatting into fixed buffers, byte copying. In Loup these are ordinary functions over fixed-width integers and buffers:

# Integer square root: the largest r with r * r <= n (bit-by-bit method).
pub func isqrt(n: uint64): uint64 {
    var rem = n
    var root: uint64 = 0u64
    var bit: uint64 = 0x4000_0000_0000_0000u64     # the highest power of four
    while (bit > rem) {
        bit = bit >> 2
    }
    while (bit != 0u64) {
        if (rem >= root + bit) {
            rem = rem - (root + bit)
            root = (root >> 1) + bit
        } else {
            root = root >> 1
        }
        bit = bit >> 2
    }
    return root
}

pub func gcd(a: uint64, b: uint64): uint64 {
    var x = a
    var y = b
    while (y != 0u64) {
        let t = x % y
        x = y
        y = t
    }
    return x
}

# Write `value` as 16 lowercase hex digits into `out`; returns the byte count.
pub func formatHex(value: uint64, edits out: buffer[uint8, 16]): int {
    var i = 0
    while (i < 16) {
        let shift = ((15 - i) * 4) as uint64
        let nibble = ((value >> shift) & 0xFu64) as uint8
        if (nibble < 10u8) {
            out[i] = 0x30u8 + nibble          # '0'..'9'
        } else {
            out[i] = 0x57u8 + nibble          # 'a'..'f'
        }
        i = i + 1
    }
    return 16
}

Listing 20-3: A few kernel library routines

formatHex writes into a buffer the caller owns, which is the general shape of freestanding APIs: the caller provides the memory, so nothing allocates and the lifetime is obvious. A caller writes var text: buffer[uint8, 16]; and passes edits: text.

Raw memory access

Copying bytes between arbitrary addresses — a DMA buffer into a packet, a boot module into its final place — needs access by address rather than through a typed buffer. Chapter 13 introduced the raw intrinsics for this: bufGetU8(base, i) is the byte at base + i, bufGetU64(base, i) the 8-byte word at base + 8 * i, and the bufSet* forms write. Nothing checks those addresses, so mark the function that uses them:

# Copy `n` bytes from `src` to `dst`; the regions must not overlap.
@unsafe("rawpoke: callers pass mapped, non-overlapping regions of at least n bytes")
func copyBytes(dst: uint64, src: uint64, n: uint64): void {
    var i: uint64 = 0u64
    while (i < n) {
        bufSetU8(dst, i, bufGetU8(src, i))
        i = i + 1u64
    }
}

Listing 20-4: A byte copy over raw addresses

With [freestanding] rawpoke = "unsafe-only" in the manifest, the compiler insists on that annotation: a raw access outside an @unsafe function or an unsafe("reason") { … } region is E_RAWPOKE_UNSAFE_ONLY (chapter 13 shows the message). The @interchange methods packInto and unpackFrom count as raw accesses too. With the key set, grep -E '@?unsafe\(' lists every place a package touches memory by address, which is what makes the audit in chapter 23 complete.

memset and friends

An optimizing backend may turn a byte loop into a call to memset or memcpy, and a freestanding link has no C library to provide them. The toolchain supplies them, written in Loup on top of the same raw-memory intrinsics. With --backend llvm, [linker] mem_runtime chooses where they come from: "loup" (the default), "c" (an older C implementation), or "none" when your kernel defines memset, memcpy, memmove and memcmp itself. The llbe backend expands these operations inline and needs none of them.

C code: extern native

A freestanding file can declare a function implemented in C, one function per declaration (chapter 5 shows the form and the @repr(C) rule for struct arguments):

@repr(C)
struct Timespec {
    var seconds: int64;
    var nanos: int64;
}

extern native "libfwclock" func fw_clock_read(ts: Timespec): int32;

Listing 20-5: Declaring a function from a C library

The declaration type-checks, but getting the C object into the image is not handled by the toolchain yet:

⚠ Planned. The Language Reference says extern native libraries are linked directly into a freestanding image (§3-3.2). In 1.25 the manifest has no key that adds a C object or library to a freestanding link. A linker script can name extra inputs itself (INPUT(libfwclock.a) is a standard linker-script command that ld.lld accepts), but the Loup test suites do not exercise this; prefer writing the routine in Loup or in an asm block.

Porting hosted code

When you bring a hosted Loup module into a kernel, the work is usually:

1. Exceptions become Result[T, E] with ?, or try { … attempt f() … } catch Err(e) { … } (chapter 9). throw and catch (T as e) are E_FREESTANDING_EXCEPTIONS.
2. Growable collections become buffer[T, N] with an explicit count. The 1.25 checker does not stop list operations such as push in freestanding code, and the LLVM lanes silently drop them, so remove them yourself (chapter 23).
3. String building becomes formatting into caller-provided buffers, as in Listing 20-3.
4. Imports of hosted modules go: replace core.math with integer helpers, system.text with your own byte routines, io.* with your drivers.
5. Floating point usually becomes fixed point (chapter 3).
6. async becomes interrupt handlers and polling loops (chapter 16).
7. defer becomes explicit cleanup on each exit path; it is accepted but never runs in 1.25 (chapter 4).

Pure logic — parsers, checksums, data structures over buffers — often needs no change beyond the mode line, and can then also be tested on the host (chapter 22).

Summary

• Without any import, freestanding code has the core language, Option/Result, buffer, layout queries, Volatile, InterruptGuard, the atomic and raw-memory intrinsics, and @interchange pack/unpack.
• Four modules can be imported: debug.assert_fs, debug.freestanding (debug builds), debug.recording, and native.css (s390x, not yet usable).
• Hosted modules cannot be imported (… cannot import @mode(host.scripted) module); native.mem and the other planned native.* modules do not exist. Do not call hosted prelude functions such as println, even though the checker allows them.
• Write small library routines yourself over fixed-width integers and caller-provided buffers. Mark raw-memory code @unsafe, and set [freestanding] rawpoke = "unsafe-only" to make the compiler insist.
• extern native declarations check, but linking C objects into a freestanding image is not managed by the 1.25 toolchain.

Next: a complete x86_64 kernel stub.