Chapter 23: Memory Safety and Kernel Development Patterns

A kernel has memory-safety problems an application never sees. There is no allocator underneath it, devices write into its memory on their own schedule, interrupt handlers run in the middle of anything, and user programs hand it addresses it must not trust. This chapter collects patterns for those problems, built from the features of the earlier chapters:

• fixed capacity instead of growable collections;
• distinct types for distinct kinds of address;
• checking addresses that come from user space;
• handing buffers to a device with ownership the compiler tracks;
• cleanup that runs on every exit path;
• handles that detect use after close;
• data shared with interrupt handlers;
• keeping the unsafe surface small and auditable.

For each pattern, the chapter says what the 1.25 compiler checks and what remains a rule you keep. That distinction matters more here than anywhere else in the book.

Fixed capacity, never growable

A kernel cannot let a data structure grow without limit: there is no heap to grow into, and a request queue that grows under load is how a flood of interrupts turns into memory corruption. Freestanding code therefore does not use list, dict or set. The Language Reference forbids them, but the 1.25 checker does not reject them, and the generated code does not do what the source says:

@mode(freestanding)

var pending: list[uint64] = []
var pendingCount: int = 0

func queueRequest(sector: uint64): void {
    pending.push(sector)
    pendingCount = pendingCount + 1
}

Listing 23-1: A growable list in freestanding code

This compiles. The LLVM lanes silently drop the push, so pendingCount says one thing and the list holds another. There is no error to tell you: the rule is simply don't. Decide the capacity, store the elements in a buffer, and make "full" a value the caller has to handle:

const MAX_PENDING: int = 16

enum QueueError {
    Full,
}

var pending: buffer[uint64, MAX_PENDING];
var pendingCount: int = 0

func queueRequest(sector: uint64): Result[void, QueueError] {
    if (pendingCount == MAX_PENDING) {
        return Err(QueueError.Full)
    }
    pending[pendingCount] = sector
    pendingCount = pendingCount + 1
    return Ok(())
}

Listing 23-2: A bounded queue whose capacity is part of its type

The element count of a buffer can be any compile-time constant, so MAX_PENDING appears once. A caller that ignores the Result gets no warning in 1.25 (chapter 9), so check it.

Indexes that come from outside

A constant index outside a buffer is a compile error (E_BUFFER_INDEX_RANGE, chapter 13). A run-time index is not checked unless the manifest asks for it:

[freestanding]
bounds = "trap"

With that key, every run-time subscript of a buffer whose size the compiler can see is checked, and an out-of-range one is a panic (buffer index out of range, chapter 17) instead of a write into the neighbouring memory. Treat that as a backstop, not as validation: an index that arrives from a device, a user program or a disk should be checked where it enters the kernel, and turned into an error the caller handles, as the handle table later in this chapter does.

Distinct types for distinct addresses

Physical addresses, kernel virtual addresses and user addresses are all 64-bit integers, and mixing them up is one of the classic kernel bugs. A @repr(transparent) struct around a uint64 costs nothing at run time and turns that mistake into a type error (chapter 5):

@mode(freestanding)

@repr(transparent)
struct PhysAddr {
    var raw: uint64;
}

@repr(transparent)
struct VirtAddr {
    var raw: uint64;
}

# The kernel maps all physical memory at this offset.
let PHYS_MAP_BASE: uint64 = 0xFFFF_8000_0000_0000u64

func physToVirt(p: PhysAddr): VirtAddr {
    return VirtAddr(raw = p.raw + PHYS_MAP_BASE)
}

func mapPage(virt: VirtAddr, frame: PhysAddr): void {
    # ... write the page-table entry ...
}

func mapFrame(frame: PhysAddr): void {
    let v = physToVirt(frame)
    mapPage(frame, v)
}

error[E0001]: argument 1 of 'mapPage': expected 'VirtAddr', found 'PhysAddr'
  --> src/main.lpfree:26:13
   |
26 |     mapPage(frame, v)
   |             ^^^^^
   |

error[E0001]: argument 2 of 'mapPage': expected 'PhysAddr', found 'VirtAddr'
  --> src/main.lpfree:26:20
   |
26 |     mapPage(frame, v)
   |                    ^
   |

With raw uint64 parameters this call would compile and map the wrong page. With newtypes, the fix is to pass the arguments in the order the signature names them, mapPage(v, frame), and the compiler holds every later caller to the same order:

func mapFrame(frame: PhysAddr): void {
    let v = physToVirt(frame)
    mapPage(v, frame)
}

Listing 23-3: Newtypes keep physical and virtual addresses apart

Addresses from user space

A system call hands the kernel an address and a length chosen by the user program. Before touching that memory the kernel has to make sure the whole range lies in user space — otherwise a program can make the kernel read or write the kernel's own memory on its behalf. The obvious check has a hole:

const USER_TOP: uint64 = 0x0000_8000_0000_0000u64     # first address above user space

enum UserError {
    BadAddress,
}

func checkUserRange(addr: uint64, len: uint64): Result[void, UserError] {
    if (addr + len > USER_TOP) {
        return Err(UserError.BadAddress)
    }
    return Ok(())
}

Listing 23-4: A range check that overflow defeats

Fixed-width arithmetic wraps in freestanding (chapter 3). With addr = 0xFFFF_FFFF_FFFF_F000 — a kernel address — and len = 0x2000, the sum wraps around to 0x1000, which is below USER_TOP, and the check passes. Compare without adding: check the start first, then whether the length fits in what is left:

const USER_TOP: uint64 = 0x0000_8000_0000_0000u64
const MAX_COPY: uint64 = 4096u64

enum UserError {
    BadAddress,
    TooLong,
}

# A user-space address; only checkUserRange and copyFromUser look inside.
@repr(transparent)
struct UserPtr {
    var addr: uint64;
}

func checkUserRange(p: UserPtr, len: uint64): Result[void, UserError] {
    if (p.addr >= USER_TOP) {
        return Err(UserError.BadAddress)
    }
    if (len > USER_TOP - p.addr) {
        return Err(UserError.BadAddress)
    }
    return Ok(())
}

var copyBuf: buffer[uint8, 4096];

@unsafe("rawpoke: the range was checked against USER_TOP; user memory is mapped during the system call")
func copyFromUser(src: UserPtr, len: uint64): Result[uint64, UserError] {
    if (len > MAX_COPY) {
        return Err(UserError.TooLong)
    }
    checkUserRange(src, len)?
    var i: uint64 = 0u64
    while (i < len) {
        copyBuf[i as int] = bufGetU8(src.addr, i) as uint8
        i = i + 1u64
    }
    return Ok(len)
}

Listing 23-5: Checking a user range without overflow, then copying it

UserPtr makes the boundary visible in every signature: a function that takes a UserPtr cannot pass it where a kernel address is expected without unwrapping it, and a reviewer can find every unwrapping. Two problems remain that types cannot solve. A user address can be valid when checked and unmapped a moment later, so a production copyFromUser must also survive a page fault during the copy (on x86_64, a fault handler that recognizes the copy loop's instruction range and returns an error instead of panicking). And the user program can change the memory while the kernel reads it, so copy it once into kernel memory, as here, and validate the copy, never the original.

Buffers owned by a device

When the kernel hands a buffer to a device for DMA, the device writes into it on its own schedule. Until the device reports completion, the CPU must not read the buffer (it may be half-written) or reuse it. In C this is a comment; in Loup the hand-off can be an ownership transfer the compiler tracks. Make the buffer a unique class and have the function that starts the transfer take it (chapters 10 and 12):

@mode(freestanding)

# A receive buffer: the physical address and size of a DMA-able frame.
unique class RxBuffer {
    pub let phys: uint64;
    pub let len: uint64;
    new(phys: uint64, len: uint64) { this.phys = phys; this.len = len; }

    reads func byteAt(i: uint64): uint8 {
        return bufGetU8(this.phys, i) as uint8
    }
}

const NIC_RX_ADDR: uint = 0x1000_3000
const NIC_RX_GO: uint = 0x1000_3008

# Hand the buffer to the device; the caller gives it up.
func startReceive(takes buf: RxBuffer): void {
    Volatile[uint64, Little].at(NIC_RX_ADDR).write(buf.phys)
    Volatile[uint64, Little].at(NIC_RX_GO).write(1u64)
}

func receiveOne(): uint8 {
    let buf = RxBuffer(0x0080_0000u64, 2048u64)
    startReceive(takes: buf)
    return buf.byteAt(0u64)
}

error[E_USE_AFTER_TAKE]: E_USE_AFTER_TAKE: binding 'buf' was consumed by a `takes` parameter and cannot be used again
  --> src/main.lpfree:26:12
   |
26 |     return buf.byteAt(0u64)
   |            ^^^
   |

The read after startReceive is exactly the bug the transfer is meant to prevent, and the compiler rejects it. The fix is a second type that stands for "a receive is in flight": startReceive turns the buffer into a ticket, and only the function that waits for completion turns the ticket back into a buffer:

unique class RxBuffer {
    pub let phys: uint64;
    pub let len: uint64;
    new(phys: uint64, len: uint64) { this.phys = phys; this.len = len; }

    reads func byteAt(i: uint64): uint8 {
        return bufGetU8(this.phys, i) as uint8
    }
}

# Proof that a receive is in flight; only waitReceive turns it back into a buffer.
unique class RxTicket {
    pub let phys: uint64;
    pub let len: uint64;
    new(phys: uint64, len: uint64) { this.phys = phys; this.len = len; }
}

const NIC_RX_ADDR: uint = 0x1000_3000
const NIC_RX_GO: uint = 0x1000_3008
const NIC_RX_DONE: uint = 0x1000_3010

func startReceive(takes buf: RxBuffer): RxTicket {
    Volatile[uint64, Little].at(NIC_RX_ADDR).write(buf.phys)
    Volatile[uint64, Little].at(NIC_RX_GO).write(1u64)
    return RxTicket(buf.phys, buf.len)
}

func waitReceive(takes t: RxTicket): RxBuffer {
    while ((Volatile[uint64, Little].at(NIC_RX_DONE).read() as uint64) == 0u64) { }
    return RxBuffer(t.phys, t.len)
}

func receiveOne(): uint8 {
    let buf = RxBuffer(0x0080_0000u64, 2048u64)
    let ticket = startReceive(takes: buf)
    let filled = waitReceive(takes: ticket)
    return filled.byteAt(0u64)
}

Listing 23-6: A receive buffer handed to a device and back

Neither buf nor ticket can be used after it has been taken, so the only path to the data runs through waitReceive. Two limits: the compiler tracks the bindings you hold, not the device, so the ticket protocol is only as good as the functions that implement it; and in 1.25 class objects are placed by the compiler rather than allocated (chapter 7), so a design with many buffers in flight keeps their addresses in a fixed table and passes tickets that refer to it. On weakly ordered CPUs, add the device barrier of chapter 15 before the doorbell write.

⚠ Planned. The Language Reference packages this pattern as native.dma with DmaBuffer[T], DmaTransfer[T] and E_DMA_BUFFER_ACTIVE (§3-3.20). The module does not exist in 1.25; write the ownership protocol yourself as above.

Cleanup on every path

A function that claims a resource — a device, a lock, a slot in a table — must release it on every way out, including the early returns that ? makes so easy to write:

enum ProbeError {
    NoDevice,
    BadVersion,
}

var claimedDevices: uint64 = 0

func claimDevice(): void { claimedDevices = claimedDevices + 1u64 }
func releaseDevice(): void { claimedDevices = claimedDevices - 1u64 }

func readVersion(base: uint64): Result[uint32, ProbeError] {
    if (base == 0u64) {
        return Err(ProbeError.NoDevice)
    }
    return Ok(2u32)
}

func probe(base: uint64): Result[uint32, ProbeError] {
    claimDevice()
    let version = readVersion(base)?          # an Err returns here, still claimed
    releaseDevice()
    return Ok(version)
}

Listing 23-7: An early return that skips the release

The Language Reference's tool for this is defer, but in Loup 1.25 a defer block is accepted and never runs (chapter 4). Use try with a finally block, which does run. Inside try, attempt unwraps an Ok and sends an Err to the catch Err(e) arm, without any exception machinery (chapter 9):

func probe(base: uint64): Result[uint32, ProbeError] {
    var version: uint32 = 0u32
    var failure: ProbeError = ProbeError.NoDevice
    var ok = false
    claimDevice()
    try {
        version = attempt readVersion(base)
        ok = true
    } catch Err(e) {
        failure = e
    } finally {
        releaseDevice()                       # runs on both paths
    }
    if (ok) {
        return Ok(version)
    }
    return Err(failure)
}

Listing 23-8: Releasing a claim with finally

When there is only one or two exits, writing the release before each return is just as good, and simpler.

Handles instead of pointers

User programs, and other subsystems, refer to kernel objects — open files, sockets, processes — by handle. If a handle is just an index into a table, a handle kept after close silently refers to whatever object reuses the slot next: a use-after-free with no pointer involved. Store a generation number in each slot, bump it whenever the slot is reused, and put it in the handle; a stale handle then fails a check instead of reaching the new object:

const MAX_FILES: int = 32

enum HandleError {
    BadHandle,
    Stale,
    TableFull,
}

struct FileHandle {
    index: int;
    generation: uint32;
}

struct FileSlot {
    var inUse: bool;
    var generation: uint32;
    var inode: uint64;
    var offset: uint64;
}

var files: buffer[FileSlot, 32];

func openFile(inode: uint64): Result[FileHandle, HandleError] {
    var i = 0
    while (i < MAX_FILES) {
        if (not files[i].inUse) {
            files[i].inUse = true
            files[i].generation = files[i].generation + 1u32
            files[i].inode = inode
            files[i].offset = 0u64
            return Ok(FileHandle(index = i, generation = files[i].generation))
        }
        i = i + 1
    }
    return Err(HandleError.TableFull)
}

# Every operation starts here: a handle from outside is checked, not trusted.
func checkHandle(h: FileHandle): Result[int, HandleError] {
    if (h.index < 0 or h.index >= MAX_FILES) {
        return Err(HandleError.BadHandle)
    }
    if (not files[h.index].inUse or files[h.index].generation != h.generation) {
        return Err(HandleError.Stale)
    }
    return Ok(h.index)
}

func seek(h: FileHandle, offset: uint64): Result[void, HandleError] {
    let i = checkHandle(h)?
    files[i].offset = offset
    return Ok(())
}

func closeFile(h: FileHandle): Result[void, HandleError] {
    let i = checkHandle(h)?
    files[i].inUse = false
    return Ok(())
}

Listing 23-9: A handle table with generation checks

buffer[FileSlot, 32] stores each struct inline, one per slot (chapter 13); files[i].offset = … writes that slot's field in place. The index check in checkHandle comes before any subscript, so an out-of-range handle is an error value, not a trap. The generation is a uint32 that wraps after four billion reuses of one slot, which is far beyond what a stale handle survives in practice.

The same table is the answer to shared ownership without reference counting at run time: a slot with a count of holders, incremented and decremented with the atomic intrinsics (chapter 19) and released when it reaches zero, keeps every object in fixed memory and every reference checkable.

Data shared with interrupt handlers

The safest structure to share between an interrupt handler and the rest of the kernel is a single-producer, single-consumer ring. The handler only writes head, the main loop only writes tail, and each side publishes its update with a release store that the other side reads with an acquire load, so neither needs a lock or masked interrupts:

const ACQUIRE: int = 2
const RELEASE: int = 3

# Scancodes from the keyboard handler to the main loop. Head and tail count up
# forever; the slot is the count modulo 64.
var keySlots: buffer[uint64, 64];
var keyHead: uint64 = 0          # written only by the producer (the handler)
var keyTail: uint64 = 0          # written only by the consumer (the main loop)

@interruptSafe
func pushKey(code: uint64): bool {
    let head = atomicLoadU64(addressOf(keyHead), ACQUIRE)
    let tail = atomicLoadU64(addressOf(keyTail), ACQUIRE)
    if (head - tail == 64u64) {
        return false                                         # full: drop the key
    }
    keySlots[(head % 64u64) as int] = code
    atomicStoreU64(addressOf(keyHead), head + 1u64, RELEASE)   # publish the slot
    return true
}

func popKey(): Option[uint64] {
    let tail = atomicLoadU64(addressOf(keyTail), ACQUIRE)
    let head = atomicLoadU64(addressOf(keyHead), ACQUIRE)
    if (head == tail) {
        return None
    }
    let code = keySlots[(tail % 64u64) as int]
    atomicStoreU64(addressOf(keyTail), tail + 1u64, RELEASE)   # free the slot
    return Some(code)
}

Listing 23-10: A lock-free ring from an interrupt handler to the main loop

head - tail is the number of queued items even after the counters wrap, because the subtraction wraps the same way. pushKey is @interruptSafe, so the keyboard handler may call it (chapter 16). The rules this relies on — one producer, one consumer, every shared variable accessed atomically — are yours to keep; the 1.25 checker does not verify atomic-only access.

For anything more complex than one producer and one consumer, take a lock (chapter 19 or 24) with interrupts masked, or mask interrupts alone on a single core (chapter 16).

A small, auditable unsafe surface

Every place where the compiler trusts you is marked in the source: @unsafe("reason") on a declaration (naked functions, overlays, functions doing raw memory access) and unsafe("reason") { … } around a region. Make the marking complete with two manifest keys:

[freestanding]
overlays = "error"         # an unacknowledged buffer overlay is an error (chapter 13)
rawpoke  = "unsafe-only"   # raw memory access outside @unsafe code is an error (chapter 20)

With both set, one command lists every assumption the kernel makes:

grep -rnE '@?unsafe\(' src/

Review each hit the same way: is the reason precise, does the code before it establish what the reason claims, and is the code after it correct if the claim holds? Keep the reasons specific — "rawpoke: the range was checked against USER_TOP" can be checked by a reviewer; "unsafe: needed" cannot. And keep the marked regions small, so the code that depends on each claim is easy to see.

Inline asm is the one exception: in freestanding code it needs no @unsafe, so add asm to the search when you audit (grep -rnE '@?unsafe\(|asm \{' src/).

Planned features

The Language Reference describes more kernel-safety machinery than Loup 1.25 implements. Each of the following has an alternative in this chapter or an earlier one:

⚠ Planned. Not available in freestanding code in Loup 1.25:

• native.dma — DmaBuffer[T], DmaTransfer[T], E_DMA_BUFFER_ACTIVE (§3-3.20). Use a unique class hand-off (Listing 23-6).
• MaybeUninit[T] (§3-3.17) — a generic uninitialized slot. A freshly declared buffer or struct is zero-initialized; PartialInit[T] (chapter 13) checks field-by-field construction.
• reinterpret[T, U] and reinterpretChecked[T, U] (§3-3.18). Read structured data from bytes with @interchange unpackFrom (chapter 20).
• native.mpu.installStackGuard (§3-3.20). Prove a stack bound with stack_check (chapter 22) and map a guard page yourself.
• Class construction through @allocator (§3-3.16). Call your allocator directly and keep objects in fixed tables (chapter 7).
• defer (§1-6.11) is accepted but does not run. Use finally or explicit release.

Summary

• Never use growable collections in freestanding code: the 1.25 checker accepts them and the LLVM lanes drop their operations. Use buffer[T, N] with an explicit count and return an error when full.
• Constant indexes are checked at compile time; run-time indexes only with bounds = "trap". Validate indexes that come from outside where they enter.
• @repr(transparent) newtypes (PhysAddr, VirtAddr, UserPtr) make address mix-ups type errors at no run-time cost.
• Check a user range as addr < TOP and len <= TOP - addr; addr + len wraps. Copy user data once and validate the copy.
• A unique class handed to a device with takes, and returned only through a completion function, makes "do not touch it during DMA" a compile-time rule (E_USE_AFTER_TAKE).
• ? returns early; release resources with try … attempt … catch Err(e) … finally, not defer.
• Generation-checked handles turn use-after-close into an error.
• A single-producer, single-consumer ring with acquire/release atomics shares data with a handler without locks.
• overlays = "error" and rawpoke = "unsafe-only" make grep '@?unsafe\(' a complete audit list; add asm blocks to it.

Next: synchronized classes and lock ordering.