Chapter 18: Building Script Tools

What you'll learn: how to turn a script into a command-line tool: making it executable, reading arguments and options, reporting errors on standard error with the right exit status, walking directories, running other programs, reading a configuration file, and running with the permissions the tool needs. At the end, the finished tool moves to host.dual so it can be built as well as run.

The chapter builds one tool, todos, which lists the TODO: and FIXME: comments in the files under a directory. It is small enough to read in one sitting and has the parts every tool needs.

An Executable Script

On Linux and macOS, a script can carry a shebang line so you can run it like any other program. The line must come first, before @mode:

#!/usr/bin/env -S loup run
@mode(host.scripted)

println("todos: nothing to report yet");

Listing 18-1: todos.lpscr with a shebang line

chmod +x todos.lpscr
./todos.lpscr

todos: nothing to report yet

#! starts a comment as far as Loup is concerned, so the line is ignored when the file is compiled. env -S splits loup run into two words; it is available on macOS and on Linux with GNU coreutils 8.30 or later. On Windows, run the script with loup run todos.lpscr.

The shebang line can also carry loup run flags, such as the permissions described later in this chapter: #!/usr/bin/env -S loup run --deny-all --allow-fs.

Arguments

args() returns the arguments that follow the script name, as a list[string]. The script name itself is not included. runtime.args() returns the same list.

Mistake: assuming an argument is there

todos takes the directory to scan as its argument. The obvious first version reads it straight from the list:

This code throws at run time

@mode(host.scripted)

let dir = args()[0];
println(i"scanning {$ dir $}");

Uncaught IndexError: list index 0 out of range (len 0)
  at <main> (main.lpscr:3:11)

Run without an argument, the list is empty and indexing it throws. Every tool is run without arguments sooner or later, often by someone who wants to see its usage. Check the length, and fall back to a default (or print a usage message):

let argv = args();
let dir = if (argv.length() > 0) { argv[0] } else { "src" };
println(i"scanning {$ dir $}");

scanning src

Options

todos accepts two options: --json for machine-readable output and --strict to fail when any FIXME is found. Parsing them is a loop over the arguments. Returning a Result keeps the parser free of printing and exiting, so it is easy to test:

struct Options {
    dir: string;
    json: bool;
    strict: bool;
}

## Read the command line into Options, or explain what is wrong with it.
func parseOptions(argv: list[string]): Result[Options, string] {
    var dir = "src";
    var json = false;
    var strict = false;
    for (arg in argv) {
        if (arg == "--json") {
            json = true;
        } else if (arg == "--strict") {
            strict = true;
        } else if (arg.startsWith("--")) {
            return Err(i"unknown option {$ arg $}");
        } else {
            dir = arg;
        }
    }
    return Ok(Options(dir = dir, json = json, strict = strict));
}

for (argv in [["--json", "docs"], [], ["--verbose"]]) {
    match parseOptions(argv) {
        Ok(o) => println(i"dir={$ o.dir $} json={$ o.json $} strict={$ o.strict $}"),
        Err(msg) => println(i"error: {$ msg $}"),
    }
}

Listing 18-2: Parsing options into a struct

dir=docs json=true strict=false
dir=src json=false strict=false
error: unknown option --verbose

⚠ Planned. The standard library reference describes loup.cli, a declarative argument parser that generates --help, and loup.config for layered configuration. Neither module ships in Loup 1.25; parse arguments by hand as above.

Mistake: passing an option to `loup run`

With the parser in place, you try the option:

loup run todos.lpscr --json

error: unknown flag: --json

The script never ran. loup run reads every ---prefixed word after the script name as one of its own flags (Chapter 1, "Permissions"), and --json is not one of them. Put -- before the arguments meant for the script; everything after it is passed through:

loup run todos.lpscr -- --json src
./todos.lpscr -- --json src

Plain arguments that do not start with --, such as a directory name, are passed through without the separator.

Standard Error and Exit Status

A tool's output goes to standard output, so it can be piped or redirected. Everything else — warnings, error messages, usage text — goes to standard error with printErr, which writes a whole line. If an error message went to standard output instead, todos --json > notes.json would write it into the JSON file.

The exit status tells the calling shell or CI job whether the tool succeeded. A script ends with:

Status	When
0	The last statement finished
1	An exception escaped (the `Uncaught …` report is on standard error)
2	The script did not compile, or `loup run` itself was misused
n	The script called `exit(n)`

exit(code) flushes both output streams and ends the process at once; nothing after it runs. Pick statuses that do not collide with the ones loup run uses: todos exits 1 when it cannot read its input or --strict finds a FIXME, and 2 for a usage error, which matches common Unix practice.

let USAGE = "usage: todos.lpscr [--json] [--strict] [DIR]";

let parsed = parseOptions(args());
if (parsed.is_err()) {
    printErr(parsed.unwrap_err());
    printErr(USAGE);
    exit(2);
}
let opts = parsed.unwrap();
println(i"scanning {$ opts.dir $}");

Listing 18-3: A usage error goes to standard error with status 2

scanning src

loup run todos.lpscr -- --verbose; echo "status $?"

unknown option --verbose
usage: todos.lpscr [--json] [--strict] [DIR]
status 2

Files and Directories

The prelude, available in every script without an import, has the file functions you need most: readText, writeText, readBytes, writeBytes and exists. The io.fs module adds directories and more (walk, listDir, isFile, isDir, createDir, remove, removeDir, copy, rename, appendText, readLines, writeLines), and io.path handles path strings (join, basename, dirname, extname, splitExt, normalize, isAbsolute).

walk(dir) lists every file under a directory, relative to it, in sorted order. Scanning one file is a loop over its lines:

from io.fs import (walk);
from io.path import (join);

struct Note {
    file: string;
    line: int;
    tag: string;
    text: string;
}

## The TODO and FIXME notes in one file.
func scanFile(path: string): list[Note] {
    var notes: list[Note] = [];
    var lineNo = 0;
    for (line in readText(path).split("\n")) {
        lineNo = lineNo + 1;
        for (tag in ["TODO", "FIXME"]) {
            let at = line.indexOf(tag + ":");
            if (at >= 0) {
                let text = line.substring(at + tag.length() + 1, line.length()).trim();
                notes.push(Note(file = path, line = lineNo, tag = tag, text = text));
            }
        }
    }
    return notes;
}

var notes: list[Note] = [];
for (rel in walk("src")) {
    notes = notes + scanFile(join(["src", rel]));
}
for (n in notes) {
    println(i"{$ n.file $}:{$ n.line $}: {$ n.tag $} {$ n.text $}");
}

Listing 18-4: Walking a directory and scanning each file

src/main.lpscr:1: TODO read flags from a config file
src/main.lpscr:3: FIXME crashes on an empty line
src/util/text.lpscr:1: TODO handle tabs

scanFile builds its own list and returns it; the caller concatenates the lists with +. Lists are values (Chapter 5), so a helper that is handed a list and pushes onto it changes only its own copy.

Mistake: assuming the file exists

A missing or unreadable file makes readText throw an IoError. Left alone, it ends the script with a stack trace:

This code throws at run time

@mode(host.scripted)

let text = readText("notes.txt");
println(text.length());

Uncaught IoError: read_text failed on 'notes.txt': No such file or directory (os error 2)
  at <main> (main.lpscr:3:12)

That is the right outcome for a bug, but not for a mistyped path. Catch the error where you can say something useful, and report it on standard error. Here a missing file is skipped with a warning:

## The text of each file that can be read; a warning for each one that cannot.
func readAll(paths: list[string]): list[string] {
    var texts: list[string] = [];
    for (path in paths) {
        try {
            texts.push(readText(path));
        } catch (IoError as e) {
            printErr(i"warning: skipping {$ path $}: {$ e.message $}");
        }
    }
    return texts;
}

writeText("a.txt", "first\n");
let texts = readAll(["a.txt", "notes.txt"]);
println(i"read {$ texts.length() $} file(s)");

Listing 18-5: Reporting an unreadable file and carrying on

In a terminal:

loup run readall.lpscr

warning: skipping notes.txt: read_text failed on 'notes.txt': No such file or directory (os error 2)
read 1 file(s)

When the file is required, report the error and exit(1) instead. exists(path) tells you whether a path exists without throwing, which suits optional files. Chapter 10 covers exceptions in detail.

Running Other Programs

exec(argv) runs a program and waits for it. argv[0] is the program, looked up on PATH, and no shell is involved, so arguments need no quoting. The result is a ProcResult with three fields: exit_code (an int), stdout and stderr.

Mistake: treating the output as a string

This code does not compile

@mode(host.scripted)

let r = exec(["echo", "build 42 ok"]);
println(r.stdout.trim());

main.lpscr:4:9: error[E0001]: type 'bytes' has no method 'trim'

A program's output is raw bytes, because nothing guarantees it is text. decode() turns UTF-8 bytes into a string:

from io.proc import (output);

let r = exec(["echo", "build 42 ok"]);
println(r.exit_code);
println(r.stdout.decode().trim());

# io.proc.output runs a program and returns its decoded standard output.
let words = output(["echo", "one two three"]).trim().split(" ");
println(words.length());

# execShell runs a command line through the system shell. The command's own
# output goes straight to the terminal; the result is its exit status.
let status = execShell("echo from the shell; exit 3");
println(status);

Listing 18-6: Running programs with exec, io.proc and execShell

0
build 42 ok
3
from the shell
3

io.proc also has exitCode(argv), which returns just the status, and check(argv), which throws when the program exits with a non-zero status. A program that cannot be started at all — for example, because it is not installed — makes exec throw IoError: run failed on '…'. Running programs needs the proc capability, which scripts have by default.

A Configuration File

todos reads an optional todos.toml that can change the tags it looks for. data.toml parses TOML into a dict[string, Any]; as gives the values their types:

from data.toml import (parse as parseToml, TomlParseError);

## The tags to look for: from todos.toml when it exists, otherwise the defaults.
func loadTags(path: string): list[string] {
    if (not exists(path)) {
        return ["TODO", "FIXME"];
    }
    try {
        let config = parseToml(readText(path));
        return config["tags"] as list[string];
    } catch (TomlParseError as e) {
        printErr(i"todos: {$ path $}: {$ e.message $}");
        exit(2);
    }
    return [];
}

println(loadTags("todos.toml"));
println(loadTags("missing.toml"));

Listing 18-7: An optional TOML configuration file

["TODO", "FIXME", "HACK"]
["TODO", "FIXME"]

A syntax error in the file is reported with its line number, for example todos: todos.toml: TOML parse error (line 2): expected ',' or ']' in array, found end of input, and the tool exits with status 2.

For output, system.json turns lists and dicts into JSON with stringify(value) or, indented, stringifyIndented(value, 2). data.csv, data.yaml, data.ini and data.xml read the other common formats.

⚠ Planned. net.http ships only an HTTP server (serve, parseRequest, Request, Response). An HTTP client is specified but not in Loup 1.25, so a script that needs to call a web API today runs curl through exec.

Permissions

Chapter 1 introduced the capabilities: loup run grants fs, proc and entropy by default and withholds net. For a tool that only reads files, grant exactly that. In the shebang line:

#!/usr/bin/env -S loup run --deny-all --allow-fs

If the tool then tries to start a program, the attempt fails at that point:

Uncaught IoError: capability 'proc' is not available: not in permission set

A narrow permission set is a cheap safeguard for a script you run on directories you did not write. Remember that the self-hosted loup on Linux x86-64 ignores these flags; use loup-bootstrap run there.

The Complete Tool

Here are the pieces put together:

#!/usr/bin/env -S loup run
@mode(host.scripted)

# todos: list the TODO and FIXME comments in the files under a directory.

from io.fs import (walk);
from io.path import (join);
from system.json import (stringifyIndented);

let USAGE = "usage: todos.lpscr [--json] [--strict] [DIR]";

struct Options {
    dir: string;
    json: bool;
    strict: bool;
}

struct Note {
    file: string;
    line: int;
    tag: string;
    text: string;
}

## Read the command line into Options, or explain what is wrong with it.
func parseOptions(argv: list[string]): Result[Options, string] {
    var dir = "src";
    var json = false;
    var strict = false;
    for (arg in argv) {
        if (arg == "--json") {
            json = true;
        } else if (arg == "--strict") {
            strict = true;
        } else if (arg.startsWith("--")) {
            return Err(i"unknown option {$ arg $}");
        } else {
            dir = arg;
        }
    }
    return Ok(Options(dir = dir, json = json, strict = strict));
}

## The TODO and FIXME notes in one file.
func scanFile(path: string): list[Note] {
    var notes: list[Note] = [];
    var lineNo = 0;
    for (line in readText(path).split("\n")) {
        lineNo = lineNo + 1;
        for (tag in ["TODO", "FIXME"]) {
            let at = line.indexOf(tag + ":");
            if (at >= 0) {
                let text = line.substring(at + tag.length() + 1, line.length()).trim();
                notes.push(Note(file = path, line = lineNo, tag = tag, text = text));
            }
        }
    }
    return notes;
}

## The notes as text lines, or as a JSON array.
func render(notes: list[Note], asJson: bool): string {
    if (asJson) {
        var rows: list[dict[string, Any]] = [];
        for (n in notes) {
            let row: dict[string, Any] = {"file": n.file, "line": n.line, "tag": n.tag, "text": n.text};
            rows.push(row);
        }
        return stringifyIndented(rows, 2);
    }
    var lines: list[string] = [];
    for (n in notes) {
        lines.push(i"{$ n.file $}:{$ n.line $}: {$ n.tag $} {$ n.text $}");
    }
    return lines.join("\n");
}

let parsed = parseOptions(args());
if (parsed.is_err()) {
    printErr(parsed.unwrap_err());
    printErr(USAGE);
    exit(2);
}
let opts = parsed.unwrap();

var notes: list[Note] = [];
try {
    for (rel in walk(opts.dir)) {
        notes = notes + scanFile(join([opts.dir, rel]));
    }
} catch (IoError as e) {
    printErr(i"todos: {$ e.message $}");
    exit(1);
}

println(render(notes, opts.json));
let fixmes = notes.filter(func(n: Note) => n.tag == "FIXME").length();
if (opts.strict and fixmes > 0) {
    printErr(i"todos: {$ fixmes $} FIXME note(s)");
    exit(1);
}

Listing 18-8: todos.lpscr, the complete tool

src/main.lpscr:1: TODO read flags from a config file
src/main.lpscr:3: FIXME crashes on an empty line
src/util/text.lpscr:1: TODO handle tabs

The dict literal in render is annotated dict[string, Any] because its values have different types; without the annotation the compiler rejects the mixed literal. A session with the tool:

./todos.lpscr -- --strict; echo "status $?"

src/main.lpscr:1: TODO read flags from a config file
src/main.lpscr:3: FIXME crashes on an empty line
src/util/text.lpscr:1: TODO handle tabs
todos: 1 FIXME note(s)
status 1

./todos.lpscr -- --json | head -7

[
  {
    "file": "src/main.lpscr",
    "line": 1,
    "tag": "TODO",
    "text": "read flags from a config file"
  },

./todos.lpscr docs; echo "status $?"

todos: list_dir failed on 'docs': No such file or directory (os error 2)
status 1

To test the tool, move parseOptions, scanFile and render into a module and write test_ functions for them, as Chapter 16 shows; the top level that reads args() and calls exit is the only part that is awkward to test.

From Script to Program: `host.dual`

A script is the right shape while a tool is yours. When it is shared, versioned and tested, make it a project with a host.dual entry point (Chapter 1, "From Script to Program"). Create the project and move the code into src/main.loup:

loup-bootstrap init todos

The helpers and structs do not change. The top-level code moves into main(), exit(n) becomes return n, and the constant USAGE can stay at module scope because a constant initializer is allowed there:

@mode(host.dual)

# todos: list the TODO and FIXME comments in the files under a directory.

from io.fs import (walk);
from io.path import (join);

let USAGE = "usage: todos [--json] [--strict] [DIR]";

# ... Options, Note, parseOptions, scanFile and render as in Listing 18-8 ...

func main(): int {
    let parsed = parseOptions(runtime.args());
    if (parsed.is_err()) {
        printErr(parsed.unwrap_err());
        printErr(USAGE);
        return 2;
    }
    let opts = parsed.unwrap();

    var notes: list[Note] = [];
    try {
        for (rel in walk(opts.dir)) {
            notes = notes + scanFile(join([opts.dir, rel]));
        }
    } catch (IoError as e) {
        printErr(i"todos: {$ e.message $}");
        return 1;
    }

    println(render(notes));
    return 0;
}

Listing 18-9: The entry point of src/main.loup in host.dual (the JSON and --strict handling are left out here)

src/main.lpscr:1: FIXME crashes on an empty line

loup run src/main.loup -- --strict runs it exactly as before: loup run calls main() and uses its int result as the exit status. loup-bootstrap check type-checks the whole project.

If you keep the JSON output, loup-bootstrap check and loup run print a warning for each Any in a host.dual file:

warning: W_ANY_IN_AOT_MODE: use of `Any` in host.dual mode prevents AOT specialisation — consider a static type if performance is critical

It is a warning, not an error: Any works, but a compiled build cannot specialise code around it.

Mistake: `main` returning the wrong type

A main that returns its report instead of printing it is rejected:

This code does not compile

@mode(host.dual)

func main(): string {
    return "src/main.lpscr:1: FIXME crashes on an empty line";
}

main.loup:3:14: error[E_MAIN_SIGNATURE]: E_MAIN_SIGNATURE: `main` must return `void` or `int` (an `async main` `Task[void]` or `Task[int]`): its `int` result is the process exit status (§1-2.4.5, ruling MAIN-1)

The result of main is the exit status, so it can only be void (status 0) or int. Print the report and return a status:

@mode(host.dual)

func main(): int {
    println("src/main.lpscr:1: FIXME crashes on an empty line");
    return 0;
}

Building it

loup-bootstrap build compiles the project to a WebAssembly module, and loup-launcher (installed next to loup) runs it:

loup-bootstrap build
loup-launcher --run-bare build/todos.wasm

--run-bare grants every capability and passes no arguments to the program. For something you can hand to other people, build a packaged application — the launcher and the module in one directory:

loup-bootstrap build --target desktop -o pkg
./pkg/bin/main -- --strict samples

pkg/bin/main is the launcher, pkg/lib/main.wasm the module, and pkg/loup-package.json records the capabilities the program may use (fs and proc by default). As with loup run, arguments that start with -- must come after a -- separator.

⚠ Known gaps in 1.25.1 Wasm builds. The Wasm path is marked experimental, and todos does not run correctly when built. An interpolated string traps:
loup: main trap: error while executing at wasm backtrace:
    0:   0x25a0 - <module>!main

Caused by:
    call_dynamic: no exported function `RenderContext` for the function value
and a file-system error inside the built module is not caught by catch (IoError as e). Until these are fixed, run host.dual programs with loup run. On Linux x86-64, loup build --target native -o todos produces a native executable (it needs clang; macOS native builds are a preview) — see The Loup Native Book.

A Checklist for a Script Tool

A shebang line (#!/usr/bin/env -S loup run) and chmod +x on Linux and macOS.
A usage message for a missing or unknown argument, on standard error, with exit status 2.
Errors you expect (a missing file, a bad configuration) caught and reported in one line; bugs left to the stack trace.
Results on standard output, everything else on standard error with printErr.
Exit statuses that callers can rely on, documented in the usage text.
The narrowest permissions that work, in the shebang line.
The logic in functions that tests can call (Chapter 16).
When the tool is shared: a project, with a host.dual entry point.

Summary

A shebang line #!/usr/bin/env -S loup run makes a script directly executable; it can carry permission flags.
args() (or runtime.args()) returns the script's arguments. Check the length before indexing, and parse options into a struct with a function that returns a Result.
loup run claims ---prefixed words after the script name; pass script options after --.
printErr writes to standard error; exit(n) ends the script with status n. An escaping exception exits 1 and a compile error 2.
The prelude has readText, writeText and exists; io.fs and io.path handle directories and paths. Catch IoError where you can report it usefully.
exec runs a program and returns its status and raw output; decode() turns the output into a string. io.proc has shortcuts.
data.toml and system.json read configuration and write structured output.
A shared tool becomes a project with a host.dual main(): int. It runs with loup run today; Wasm builds exist but are experimental in 1.25.1.

Where to Go Next

Native programs: The Loup Native Book covers host.native, native builds, ARC and threads.
The browser: The Loup Web Book describes the web modes; their browser runtime is planned.
Bare metal: The Loup Freestanding Book covers kernels and firmware.

The language is the same in every mode; each book shows which features a mode has and how to use them.