Chapter 14: Reflection and Dynamic Compilation

⚠ Planned — not in Loup 1.25. This chapter describes the design in the Language Reference (§1-4.7 typeof and instanceof, §1-2.3 __module__ and Module, §1-8.8 annotations, §2-5.1 runtime.compile). The toolchain does not implement it yet, so its listings are design sketches and are not verified.

What you'll learn: what you can do today with values of unknown type — test them with is, convert them with as? and match — and the planned reflection design: typeof, the core.reflect module, and compiling Loup source at run time with runtime.compile.

Reflection is for the edges of a program: a serializer that walks any struct, a plugin host, a tool that reads data whose shape it does not know in advance. Most code should not need it.

What Works Today: Testing Values of Type Any

Data read from outside a program often arrives as Any: a parsed JSON document, a list of mixed settings. Loup 1.25 gives you three tools for it — the is test, the as? cast, and match with type patterns — and they work in scripts now. Unlike the design sketches later in the chapter, the listings in this section are checked with the 1.25 toolchain.

A match over an Any value can bind the value under its real type:

@mode(host.scripted)

from system.json import (parse);

func describe(value: Any): string {
    return match value {
        string(s) => i"text ({$ s.length() $} chars)",
        int(n) => i"whole number {$ n $}",
        float(x) => i"decimal {$ x $}",
        bool(b) => i"flag {$ b $}",
        _ => i"other: {$ repr(value) $}",
    };
}

let doc = parse("""{"name": "loup", "port": 8080, "ratio": 0.75, "debug": true, "tags": ["a", "b"]}""");
match doc {
    Ok(v) => {
        let fields = v as dict[string, Any];
        for (key in fields.keys()) {
            println(i"{$ key $}: {$ describe(fields[key]) $}");
        }
    },
    Err(msg) => { println(i"bad JSON: {$ msg $}"); },
}

Listing 14-1: Describing the values of a JSON document by their runtime type

name: text (4 chars)
port: whole number 8080
ratio: decimal 0.75
debug: flag true
tags: other: ["a", "b"]

int(n) matches when the value is an int and binds it to n with type int, so n can be used in arithmetic. _ matches anything else, and repr renders any value for display.

value is T answers a yes-or-no question, and value as? T returns Some(value) when the value has type T and None otherwise:

@mode(host.scripted)

func kind(value: Any): string {
    if (value is string) {
        return "string";
    }
    if (value is int) {
        return "int";
    }
    if (value is list[Any]) {
        return "list";
    }
    return "other";
}

let settings: list[Any] = ["app.toml", 8080, [1, 2], 2.5];
for (s in settings) {
    println(kind(s));
}

let port = (settings[1] as? int) ?? 80;
println(port + 1);

Listing 14-2: is tests and an as? cast with a default

string
int
list
other
8081

For a collection, is and as? check only the container: value is list[int] is true for any list, because a list value does not record its element type (§1-4.3).

instanceof. The Language Reference defines x instanceof T as a synonym for x is T. The 1.25 parser does not support it — it stops with an internal error (exit status 101) instead of a diagnostic. Write is.

Mistake: expecting is to narrow an Any

The Language Reference says that inside if (value is string) { … }, value has type string. For a value of type Any, the 1.25 type checker does not narrow:

@mode(host.scripted)

func firstWord(value: Any): string {
    if (value is string) {
        let text: string = value;
        return text.split(" ")[0];
    }
    return "";
}

println(firstWord("hello there"));

main.lpscr:5:9: error[E0001]: cannot assign 'Any' to binding 'text' of type 'string'

Bind the converted value instead, with match or as?:

@mode(host.scripted)

func firstWord(value: Any): string {
    return match value {
        string(text) => text.split(" ")[0],
        _ => "(not text)",
    };
}

println(firstWord("hello there"));
println(firstWord(42));

hello
(not text)

Narrowing does work on a union type: with value: int | string, if (value is string) { let s: string = value; … } compiles. When you know the possible types in advance, a union (Chapter 7) is the better parameter type.

Mistake: calling methods on an Any without checking

Member access on an Any value is not checked at compile time. If the value turns out to have another type, the call fails at run time:

@mode(host.scripted)

func shout(value: Any): string {
    return value.toUpperCase();
}

println(shout("ready"));
println(shout(42));

READY
Uncaught RuntimeError: internal interpreter limitation: unsupported instruction: CallVirtual toUpperCase on Int(42)

The message is the interpreter's, not a friendly one; the cause is that 42 has no toUpperCase. Check the type before you use it:

@mode(host.scripted)

func shout(value: Any): string {
    return match value {
        string(s) => s.toUpperCase(),
        _ => repr(value),
    };
}

println(shout("ready"));
println(shout(42));

READY
42

Mistake: trusting as to check

The Language Reference says value as T throws TypeError when the value is not a T. In Loup 1.25 it does not check; the wrong value passes through and fails later, somewhere else:

@mode(host.scripted)

func nextPort(setting: Any): int {
    let port = setting as int;
    return port + 1;
}

println(nextPort(8080));
println(nextPort("8080"));

8081
Uncaught RuntimeError: internal interpreter limitation: type error: Add expects same numeric type, got Str("8080") + Int(1)

Until as checks, use as only where the type is already known and prefer match or as? for data from outside:

@mode(host.scripted)

func nextPort(setting: Any): int {
    let port = match setting {
        int(n) => n,
        string(s) => s.toInt(),
        _ => 80,
    };
    return port + 1;
}

println(nextPort(8080));
println(nextPort("8080"));
println(nextPort(true));

8081
8081
81

The Planned Design

None of the following compiles with Loup 1.25. The listings show the design in the Language Reference.

typeof and core.reflect

typeof expr (§1-4.7) returns a Type value describing the runtime type of expr; it is sugar for core.reflect.typeOf(expr). The core.reflect module adds structural queries: the fields and methods of a type, its generic parameters, and its annotations.

@mode(host.scripted)

from core.reflect import (typeOf, fields);

struct User {
    id: int;
    name: string;
    var email: string;
}

let u = User(id = 42, name = "Ada", email = "ada@example.com");
let t = typeOf(u);
println(i"{$ t.name $} ({$ t.kind $})");      # User (struct)

for (f in fields(t)) {
    println(i"  {$ f.name $}: {$ f.fieldType.name $}");
}

Listing 14-3: Listing a struct's fields with core.reflect (design sketch)

In 1.25, typeof 42 is a syntax error (expected expression) and importing core.reflect fails with 'core.reflect' is not a standard-library module.

Annotations on a declaration are readable through core.reflect.annotations(target), which returns a dict[string, Any] from annotation names to their arguments (§1-8.8). That is how a framework would find, say, every method marked @route("/users") and register it.

__module__

Every named type and function is planned to carry a __module__ property (§1-2.3) — a Module value with __name__, __file__ and __execmode__ — so code can ask where a declaration came from. Today Encoder.__module__ is type 'Encoder' has no member '__module__'. The module-level variables __name__, __file__ and __execmode__ do work; Chapter 1 shows what they print.

runtime.compile

runtime.compile(source) (§2-5.1) compiles a string of Loup source through the normal pipeline — parse, resolve, type-check — and returns a Module. The module's top-level statements are not run; you call its exported functions by name:

@mode(host.scripted)

let mod = runtime.compile("""
    pub func greet(name: string): string {
        return i"Hello, {$ name $}!";
    }

    pub func add(a: int, b: int): int {
        return a + b;
    }
""");

println(mod.exports());                     # ["greet", "add"]
println(mod.call("greet", ["world"]));      # Hello, world!
println(mod.call("add", [1, 2]));           # 3

Listing 14-4: Compiling and calling a module at run time (design sketch)

The Module type (§1-2.3) provides exports(), has(name), get(name) and call(name, args). An optional CompileOptions value sets the module's name (shown in stack traces) and whether it is type-checked. A compile error in the source surfaces as an exception at the runtime.compile call, so a program can report it and try again.

Two rules from the specification matter for anyone planning to run generated code:

• runtime.compile requires the eval capability. It is not available in web.compiled or freestanding.
• The compiled module inherits the caller's capabilities and resolves its imports against the caller's module graph. It is not a sandbox: source compiled this way can do whatever the calling script can. Run generated code in a separate loup run --deny-all --allow-… process if you need to limit it.

In 1.25 the call is rejected: type 'Runtime' has no method 'compile'.

A plugin loader would check a module's exports before using it:

@mode(host.scripted)

class PluginError : Exception {}

func loadPlugin(source: string): Module {
    let mod = runtime.compile(source);
    for (required in ["init", "handle"]) {
        if (not mod.has(required)) {
            throw PluginError(i"plugin is missing '{$ required $}'");
        }
    }
    return mod;
}

Listing 14-5: Validating a compiled plugin's exports (design sketch)

Summary

• Values of type Any can be examined today with match type patterns (int(n) => …), is tests and as? casts; Listing 14-1 walks a parsed JSON document this way.
• is does not narrow an Any in Loup 1.25 — bind the value with match or as? instead. Narrowing does work on union types.
• Member access on Any is unchecked and fails at run time on the wrong type; as does not check either. Check first.
• instanceof crashes the 1.25 parser; use is.
• Planned: typeof, the core.reflect module (typeOf, fields, methods, annotations), the __module__ property, and runtime.compile returning a callable Module. A compiled module inherits the caller's capabilities; it is not a sandbox.

Try It Yourself

1. Extend Listing 14-1 so a list value is described as list of N items: convert it with value as? list[Any] and match the Option.
2. Write func toInt(value: Any): Option[int] that accepts an int or a numeric string and returns None for anything else.
3. Rewrite firstWord to take value: int | string and use is narrowing directly.