Volume 8 — Time-Travel Debugging in Production — contents
  1. Chapter 1 — Why Time-Travel Debugging
  2. Chapter 2 — Pod Instrumentation
  3. Chapter 3 — The Trace-Event Protocol
  4. Chapter 4 — Server-Side Ingestion
  5. Chapter 5 — Remote Debug Control
  6. Chapter 6 — Session Management at Scale
  7. Chapter 7 — Case Studies
  8. Chapter 8 — Performance and Storage Trade-offs

Chapter 2 — Pod Instrumentation

Abstract. The pod's contract with the application is a pair of context managers — TimeTravelGateway at the top and TimeTravelPortal("name") at every interesting scope — and a process-wide registry that decides which portal names are active at any moment. This chapter walks the entry points the application sees, the state machine the pod runs through when a portal opens, the configuration object that controls recording fidelity, and the static-vs-dynamic instrumentation trade-off the design forces on the operator. All claims are anchored on codecraftai-timetravel-pod.

2.1 The two context managers

The application surface of the pod is two classes. Everything else — the trace hook, the recorder chain, the serialiser, the backend — is reached through them.

TimeTravelGateway (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/gateway.py:22) is the application-scope context manager. It is entered exactly once, near the top of the application's lifetime; it owns the monotonic clock that all timestamps are measured against, holds the registry of currently-active recorders, and broadcasts the two boundary events that bracket every recording — a GatewayStartEvent on entry, a GatewayStopEvent on exit.

TimeTravelPortal (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/portal.py:28) is the per-scope context manager. It is entered every time control flows into a portal scope, regardless of whether that portal is currently activated. The cost of entering an inactive portal is three early-exit conditional branches and one thread-local lookup; the cost of entering an active portal is the installation of sys.settrace, the construction of a recorder chain, and an event broadcast.

The asymmetry is deliberate. Portal entry is on the hot path of the application — every request, every scheduled task, every agent step might cross a portal boundary. Gateway entry is once per process. The framework optimises the common, hot, cheap case (portal entry while inactive) and accepts higher cost on the rare, cold, expensive case (gateway start, portal activation).

A minimal usage looks like this:

from ccai.timetravel.pod import TimeTravelGateway, TimeTravelPortal

def main() -> None:
    with TimeTravelGateway(backend_uri="file:///var/log/mysvc/trace.bin"):
        run_service()

def handle_request(req):
    with TimeTravelPortal("http.request"):
        return process(req)

The gateway is not a daemon; it is a with block that brackets the recording window. In a long-lived service the gateway is typically entered in main() and never exited until the process terminates. In a CLI command, the gateway brackets the command's body. In a unit test, the gateway brackets the test body.

2.2 The startup configuration

TimeTravelConfig (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/configuration.py:19) collects every knob the pod exposes into a single object. Its 13 __slots__ fields cover four concerns:

  • What to record. active_portals (a list of names to activate at startup), trace_calls, trace_lines, trace_variables, record_inner_portals (the four trace- granularity booleans).
  • Where to record it. backend_uri (a file://, tcp://, or unix:// URI; resolved by the backend registry).
  • How to be remote-controlled. remote_enabled, remote_host, remote_port, remote_credential (for the JSON-lines control server).
  • How to be agent-controlled. mcp_enabled, mcp_transport, mcp_host, mcp_port, mcp_pipe (for the Model Context Protocol bridge — see ch. 5).

The bootstrap function configure_timetravel (configuration.py:259-304) reads a TimeTravelConfig and produces the runtime side-effects:

def configure_timetravel(config: TimeTravelConfig) -> dict:
    servers = {}

    # Activate portals
    registry = get_portal_registry()
    if config.active_portals:
        registry.activate_portals(config.active_portals)

    # Create backend (if URI specified)
    if config.backend_uri:
        backend_registry = create_default_registry()
        backend_registry.create_backend(config.backend_uri)

    # Start remote control server (if enabled)
    if config.remote_enabled:
        remote_server = RemoteControlServer(
            host=config.remote_host,
            port=config.remote_port,
            credential=config.remote_credential
        )
        remote_server.start()
        servers["remote"] = remote_server

    # Start MCP server (if enabled)
    if config.mcp_enabled:
        mcp_server = McpServer()
        if config.mcp_transport == "tcp":
            mcp_server.start_background_tcp(config.mcp_host, config.mcp_port)
        elif config.mcp_transport == "pipe" and config.mcp_pipe:
            mcp_server.start_background_pipe(config.mcp_pipe)
        else:
            mcp_server.start_background_stdio()
        servers["mcp"] = mcp_server

    return servers

The function returns a dict of running servers keyed by "remote" and "mcp", so the application can keep references for shutdown. Calling configure_timetravel does not enter the gateway; the gateway is still entered by the application at the appropriate scope. The function is purely about bootstrapping the side-channels: the portal registry's initial state, the backend, the control plane.

The choice to keep the gateway out of configure_timetravel is deliberate and worth understanding. The gateway controls the recording-window timestamp origin (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/traceclock.py). A long-running service that wants the gateway alive for the process lifetime enters it as a with block in main(). A short-lived task that wants the gateway alive for the task's extent enters it inside the task's body. The application decides; configure_timetravel does not enforce.

2.3 The gateway's invariants

TimeTravelGateway.__enter__ does three things, in order (gateway.py):

  1. Records the current wall-clock time and starts the monotonic clock — so every event timestamp is an offset from gateway start.
  2. Sets the process-wide _gateway_active flag to True (tracingstate.py:71-79); this is the flag every portal reads first to decide whether to do anything.
  3. Emits a GatewayStartEvent to all registered recorders.

TimeTravelGateway.__exit__ (lines 73-101) is more intricate because it has to land safely even if a recording in flight has errored:

def __exit__(self, exc_type, exc_val, exc_tb):
    try:
        # Emit gateway stop event
        stop_event = GatewayStopEvent(
            timestamp_ns=self._clock.timestamp(),
            thread_id=threading.get_ident(),
            wall_clock_time=time.time(),
        )
        self._broadcast_event(stop_event)

        # Flush and close all active recorders
        for recorder in self._active_recorders:
            recorder.flush()
            recorder.close()
        self._active_recorders.clear()
    finally:
        self._clock.stop()
        set_gateway_active(False)
        set_active_gateway(None)
    return False

Three observations.

First, the try body emits the stop event before flushing — so the stop boundary appears in the recorded stream even if a flush fails afterwards. This is the same boundary-marker discipline JSOS uses (Volume 1 ch. 5): the marker is broadcast, then the buffered data is committed.

Second, the finally block always runs, regardless of any exception in the try. The clock is stopped, the gateway flag is cleared, the active-gateway singleton is unset. Even if a recorder's flush() or close() raises, the next gateway attempt on the same process will not see stale state.

Third, the __exit__ returns False — meaning any exception that propagated into __exit__ (an error in the with body, or in the recorder cleanup) is not suppressed. The pod is not in the business of swallowing the application's exceptions. This is the same convention every interop layer in the portfolio honours (Volume 5: control reverts via typed exceptions).

2.4 The portal's decision tree

TimeTravelPortal.__enter__ (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/portal.py:60-141) is the heart of the pod. It is run on every entry into every portal scope; its cost dominates the application's quiescent overhead. The decision tree it implements is, in order:

1. Is the gateway active?            → no  → return self
2. Is this thread already tracing?   → yes → bump depth, return self
3. Is this portal's name activated?  → no  → return self
4. Does the activation have an
   expression?                       → if so, eval it; if false,
                                       return self
5. Construct recorder chain.
6. Install sys.settrace hook.
7. Mark thread as tracing,
   depth = 1.

Steps 1-4 are guards; steps 5-7 are activations. In the typical production case — most portals not currently active — only steps 1 and 3 actually evaluate, and both fall through to return self.

The activation block (5-7) is more substantive. Two flavours exist depending on whether a backend URI is configured:

def _create_record_callback(self, gateway, capture_id):
    backend_uri = capture_id or gateway.backend_uri

    if backend_uri:
        # Real recorder chain
        backend_registry = create_default_registry()
        backend = backend_registry.create_backend(backend_uri)
        serializer = TraceSerializer()
        writer = BufferedTraceWriter(backend, max_buffer_size=65536)

        class PortalRecorder(TraceRecorder):
            def record(self, event):
                data = self.serializer.serialize(event)
                self.writer.write(data)
            def flush(self):
                self.writer.flush()
            def close(self):
                self.writer.close()

        recorder = PortalRecorder(writer, serializer)
        gateway.register_recorder(recorder)
        ...

The chain is backend → serializer → buffered writer → recorder. Each layer has a single responsibility; each is swappable. The 65,536-byte buffer (auto-flush at full) is the default chosen at this site; a different recording profile could override it without changing the recorder ABC.

The fall-through case — no backend URI configured — is a list collector:

else:
    # Fallback: in-memory collector for testing
    collected_events = []
    def record_callback(event):
        collected_events.append(event)
    state.recorder = _ListRecorder(collected_events)

This second branch is what unit tests use. A test enters a portal, runs code, exits the portal, and inspects the in-memory event list. Production deployments configure a backend URI.

2.5 The portal registry

PortalRegistry (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/portalregistry.py:12) is the structure that holds which named portals are active right now. It is a singleton (get_portal_registry() at line 133) keyed by portal name, with each value a dict of active, expression, once, and capture_id.

Every public method takes a threading.Lock before reading or writing the dictionary (portalregistry.py:25-41):

def activate_portal(self, name: str, expression: str = None,
                    once: bool = False, capture_id: str = None) -> None:
    with self._lock:
        self._portals[name] = {
            "active": True,
            "expression": expression,
            "once": once,
            "capture_id": capture_id,
        }
    return

The get_portal_info method (portalregistry.py:71-83) explicitly returns a copy of the internal dict so a caller cannot mutate registry state by mutating the returned object:

def get_portal_info(self, name: str) -> dict:
    with self._lock:
        rtnval = self._portals.get(name, {})
        # Make a copy to avoid exposing the internal structure
        rtnval = dict(rtnval)
    return rtnval

This is the kind of small discipline that prevents action-at-a- distance bugs three years later — the operator-tool client queries portal info, sets active=False on the returned dict by accident, and the registry is silently corrupted. The copy defends against that.

The lock is uncontended in the common case: it is taken on every portal-entry's is_portal_active check, but the check is microseconds long. The lock is contended when remote-control commands flood the registry — a debug session that bulk- activates 50 portals at once will serialise through this lock — but those events are infrequent compared to portal entries.

2.6 The thread-local state

The decision tree in §2.4 does not actually take the registry's lock at step 2 ("is this thread already tracing?"). Instead it reads thread-local state. The TracingState (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/tracingstate.py:15) holds four fields per thread:

  • is_tracing — bool, true while a portal is open and the trace hook is installed.
  • recorder — the active recorder, used by the hook callback to publish events.
  • portal_depth — int, the current portal nesting level on this thread.
  • active_portal_name — str, the name of the outermost currently-active portal.

The lookup helper (tracingstate.py:44-55) lazy-initialises:

_thread_local = threading.local()

def get_tracing_state() -> TracingState:
    state = getattr(_thread_local, "tracing_state", None)
    if state is None:
        state = TracingState()
        _thread_local.tracing_state = state
    return state

There is no global lock here — threading.local() is the standard library's per-thread storage primitive, and the lazy initialisation pattern is idempotent on a per-thread basis.

The reason the design splits into a thread-local state and a process-wide registry is the classic what changes vs what is read axis:

  • The registry is read from many threads on every portal entry, but written rarely (only when an activation command arrives). A Lock is the right tool: it protects the occasional write against the frequent reads.
  • The state is read and written on every portal entry on a particular thread, but only that thread. A threading.local is the right tool: no lock needed, because no other thread is reading or writing the same fields.

2.7 Nested portals

When a thread already inside an active portal enters a second portal, the second portal does not install a second sys.settrace. It increments state.portal_depth and (if record_inner_portals is set in the trace configuration) emits a PortalEnterEvent so the trace contains the boundary marker.

if state.is_tracing:
    state.portal_depth += 1
    if config.record_inner_portals:
        record(PortalEnterEvent(timestamp=clock.timestamp(),
                                portal_name=self._name))
    return self

The exit path mirrors the entry: state.portal_depth -= 1, and if the record_inner_portals flag is set, a PortalExitEvent. The hook is uninstalled only when state.portal_depth drops to zero (i.e. the outermost portal exits).

This single-hook-many-portals design has two consequences.

The first is correctness. sys.settrace is a per-thread setting — Python only allows one trace function at a time per thread. If portals stacked their hooks, the inner hook would replace the outer hook, the outer hook's events would stop being recorded for the inner scope, and reinstalling the outer hook on inner-portal exit would either work (only because Python's sys.settrace semantics happen to support this) or produce gaps. The single-hook approach sidesteps the issue entirely.

The second is annotation. When the off-line tooling reconstructs the timeline, the PortalEnterEvent / PortalExitEvent markers are the boundaries it uses to filter the recording to a portal scope. Volume 8 ch. 4 walks tracedata/portal_filter.py (server-side) and pod/portalfilter.py (pod-side; same algorithm) — both are tiny state machines that turn the markers into scope membership.

2.8 Conditional activation

Activation can carry an expression string:

registry.activate_portal(
    name="invoice.charge.attempt",
    expression="invoice.amount > 10000",
    once=True,
    capture_id="big-invoice-anomaly",
)

When a portal entry encounters this activation, it evaluates the expression in the caller's frame (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/portal.py:99-112):

expression = portal_info.get("expression")

if expression is not None:
    try:
        frame = sys._getframe(1)
        result = eval(expression, frame.f_globals, frame.f_locals)
        if not result:
            return self
    except Exception:
        return self

If the expression evaluates falsy, the portal returns without activating tracing — the recording is reserved for the calls that match the predicate. If evaluating the expression raises (a NameError because invoice was not in scope, a TypeError because invoice.amount was None), the exception is swallowed and the portal does not activate. This last behaviour is a quiet override of the portfolio's "errors should never pass silently" rule, justified by the use case: a malformed predicate must not crash the production application that is using the recording as a debugging tool. The cost is that operators submitting bad predicates see no activation and no error feedback — the feedback path is logging.

This is a tractable design point worth pushing on. Two options were available:

  • Swallow. As implemented. The portal silently does not activate. Pros: no production risk. Cons: silent failure; the operator does not learn that their predicate is wrong.
  • Surface. Raise to the operator-tool client that submitted the activation. Pros: visible error. Cons: requires a channel for asynchronous notification of failed activations, because the predicate is evaluated later, not at activation time.

The implemented choice is the conservative one. Volume 8 notes the trade-off; the source comment is honest about the swallow (portal.py:108-112).

2.9 The trace configuration

TraceConfiguration (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/traceconfiguration.py:10-63) is an immutable, four-boolean object that controls what the trace hook records on every Python event:

class TraceConfiguration:
    __slots__ = ("_trace_calls", "_trace_lines", "_trace_variables", "_record_inner_portals")

    def __init__(
        self,
        trace_calls: bool = True,
        trace_lines: bool = False,
        trace_variables: bool = False,
        record_inner_portals: bool = False,
    ) -> None:
        self._trace_calls = trace_calls
        self._trace_lines = trace_lines
        self._trace_variables = trace_variables
        self._record_inner_portals = record_inner_portals

The __slots__ plus read-only properties enforce immutability — there is no setter for _trace_calls etc. The configuration is a value in the strict sense: two configurations with the same booleans are interchangeable.

The four knobs cover four growing recording densities:

  • calls only (trace_calls=True, others false): a CallEvent for each function entered, a ReturnEvent for each function exited. A function-graph trace. Cheap.
  • calls + lines (trace_lines=True added): on every line executed inside a recorded function, a LineEvent. About 3- 10x the volume of calls-only, depending on the code's loop density.
  • calls + lines + variables (trace_variables=True added): on every line executed, a VariableSnapshotEvent carrying the current frame.f_locals as a dict. An order of magnitude more volume than calls + lines.
  • + inner portals (record_inner_portals=True added): PortalEnterEvent and PortalExitEvent markers for nested portals. Negligible additional volume, but enables off-line filtering by portal scope.

Volume 8 ch. 8 measures the cost. The numbers, briefly: calls-only is a few-percent overhead on a typical Python service. Lines is 5-10x. Variables is 50-100x and a flood of data. The design intent is that calls is the production default, lines is the targeted-window default, and variables is the last-resort, narrow-scope, this-bug-now default.

2.10 The event model

TraceEvent (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/traceevents.py:29) is the base class of nine concrete event types (traceevents.py:12-20):

class EventType:
    CALL = 1
    RETURN = 2
    LINE = 3
    VARIABLE_SNAPSHOT = 4
    PORTAL_ENTER = 5
    PORTAL_EXIT = 6
    GATEWAY_START = 7
    GATEWAY_STOP = 8
    EXCEPTION = 9

Each event uses __slots__ for memory efficiency, has a nanosecond-precision timestamp_ns field (an integer offset from gateway start), a thread_id, and a per-type payload.

A representative event — the variable snapshot — looks like:

class VariableSnapshotEvent(TraceEvent):
    __slots__ = ("filename", "lineno", "local_variables")

    def __init__(self, timestamp_ns: int, thread_id: int,
                 filename: str, lineno: int, local_variables: dict) -> None:
        super().__init__(EventType.VARIABLE_SNAPSHOT, timestamp_ns, thread_id)
        self.filename = filename
        self.lineno = lineno
        self.local_variables = local_variables

The local_variables field is a literal dict snapshot of frame.f_locals at the point of the event. This is the most expensive event in the model — it serialises an arbitrary dict, including whatever the application happened to have on the stack. It is also the most informative, because it is the answer to the on-call engineer's "what was in scope when this fired?" question.

Boundary events — GatewayStartEvent, GatewayStopEvent — also carry a wall_clock_time (float, seconds since epoch) so the recording can be correlated with external systems (application logs, APM traces, alerting timestamps). All other events are nanosecond offsets only.

2.11 The trace hook

The actual sys.settrace callback lives in TraceHookManager._trace_function (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/tracehookmanager.py:63-137). It dispatches on the Python event type (the second argument Python passes to a trace function), constructs the appropriate TraceEvent subclass, and pushes it through the record_callback:

def _trace_function(self, frame, event, arg):
    timestamp_ns = self._clock.timestamp()
    record = self._record_callback

    if event == "call":
        if self._trace_calls:
            call_event = CallEvent(
                timestamp_ns=timestamp_ns,
                thread_id=self._thread_id,
                filename=frame.f_code.co_filename,
                lineno=frame.f_lineno,
                function_name=frame.f_code.co_name,
            )
            record(call_event)
        return self._trace_function

    elif event == "line":
        if self._trace_lines:
            line_event = LineEvent(...)
            record(line_event)
        if self._trace_variables:
            var_event = VariableSnapshotEvent(
                timestamp_ns=timestamp_ns,
                thread_id=self._thread_id,
                filename=frame.f_code.co_filename,
                lineno=frame.f_lineno,
                local_variables=dict(frame.f_locals),
            )
            record(var_event)

    return self._trace_function

Two performance details are worth the attention.

First, self._trace_calls, self._trace_lines, self._trace_variables are cached as instance attributes (tracehookmanager.py:42-44) instead of being read out of the TraceConfiguration on every event. Attribute lookup is faster than property access; on a hot trace, the difference matters.

Second, the trace function returns itself on every event. Python's sys.settrace semantics require this — the returned function is what gets installed as the line trace function for the frame that just received a call event. Returning None turns line tracing off for that frame; returning self._trace_function keeps the hook installed. The pod always returns self._trace_function, ensuring the recording continues across nested calls.

The hook does not catch exceptions raised in record(...). If the recorder fails — a backend disconnect, a serialiser bug, an OOM — the exception propagates back up through the trace hook and out of the recorded code. This is consistent with the gateway's __exit__ returning False (§2.3): the pod does not pretend to handle errors it cannot resolve.

2.12 The decorator

For functions whose entire body is the portal scope, @trace_portal("name") (codecraftai-timetravel-pod/source/packages/ccai/timetravel/pod/decorators.py:19-46) is the convenience form:

def trace_portal(name: str):
    def decorator(func):
        if inspect.iscoroutinefunction(func):
            @functools.wraps(func)
            async def async_wrapper(*args, **kwargs):
                with TimeTravelPortal(name):
                    rtnval = await func(*args, **kwargs)
                return rtnval
            return async_wrapper
        else:
            @functools.wraps(func)
            def sync_wrapper(*args, **kwargs):
                with TimeTravelPortal(name):
                    rtnval = func(*args, **kwargs)
                return rtnval
            return sync_wrapper

    return decorator

The decorator does nothing the explicit with form does not do — it just wraps the function body in a with TimeTravelPortal(name). The branch on inspect.iscoroutinefunction is what allows the same decorator to work on both sync and async functions; the async wrapper awaits the coroutine inside the portal scope, so the recording covers the entire async lifetime.

Decorators in the portfolio are constrained by the project's coding standards. The rule (in codecraftai-* CLAUDE.md): "Decorators are only allowed for adding metadata or for registering a function with a mapping system. Do NOT use behavior-modifying decorators." @trace_portal is on the edge — it modifies behaviour by wrapping the call in a context manager. The justification, taken on its own terms, is that the wrapped behaviour is purely additive and purely declarative: it does not change the function's semantics; it adds a recording boundary. Even so, the explicit with form remains the dominant pattern in the portfolio's own code; the decorator is offered for ergonomics on functions whose body is the portal.

2.13 Static vs dynamic instrumentation

The pod's portals are static, not dynamic. They are written into the application source as with blocks or @trace_portal decorators, before the code is built and deployed. The operator's runtime control is which static portals to activate, not where to add new portals.

The alternative — dynamic instrumentation — would let the operator say, at runtime, "trace mymodule.somefunc as if it were wrapped in a portal." Tools in the broader Python ecosystem do this: sys.settrace itself is the underlying mechanism, and a debugger-like tool that takes a fully-qualified function path could install a hook at runtime.

The pod chose static instrumentation for three reasons.

First, scope discipline. A static portal expresses an intent: "this scope is one I expect to need to debug." The intent is encoded in the source review — a code reviewer can see that the developer thought http.request was a debugging-worthy boundary. A dynamic tool with no source-side annotation can trace anywhere, which makes the activation policy harder to reason about and harder to audit.

Second, performance predictability. A static portal's cost is bounded — the inactive cost is three thread-local conditional branches (§2.4); the active cost is sys.settrace'd Python inside a known scope. A dynamic tool that activates by qualified name has to install hooks at function entry, return, or line boundaries it does not know in advance; the cost surface is larger.

Third, operator vocabulary. "Activate the invoice.charge.attempt portal" is a sentence the on-call engineer can type without consulting the source. "Install a trace hook on ccai.invoicing.service.InvoiceService.charge at line 412" is not. The named-portal abstraction trades some flexibility for a vocabulary the production operator already shares with the developer.

The trade-off is real and worth naming. Static instrumentation forecloses the bug whose right scope nobody anticipated. If the failing call is in code that has no portal, the operator cannot record it without a code change, a build, and a deploy — which can be a long road in production. The pod accepts that foreclosure as the price of the three benefits above.

A practical mitigation: the portfolio's own coding-standards document encourages portals at every architecturally meaningful boundary — request entry, request exit, RPC call, RPC reply, DB transaction, agent step, scheduled-task body, message-queue consumer. In a fully-instrumented service the coverage of static portals approaches the operator's likely debug surface, even if the exact line is not directly inside one.

2.14 Where the portals belong

Using the portal pattern well is mostly a question of where to place portals. The portfolio's own code provides several patterns:

  • At every external entry point. Every HTTP handler, every message-queue consumer, every CLI subcommand opens a portal named for the entry point (http.request, kafka.consumer.invoice-events, cli.invoke.deploy).
  • At every external call. Every database query, every RPC, every third-party API call opens a portal named for the destination (db.query, rpc.payment-service.charge, http.client.stripe).
  • At every long-running task body. Every scheduled task, every background worker, every agent step opens a portal named for the task type (task.daily-rollup, worker.invoice-batch, agent.step).
  • At every "I expect this to be hard to debug" scope. The scope the senior developer imagined would be the locus of pain — the cache-invalidation block, the retry loop, the branch that handles the rare schema variant — opens a portal named for that intent (cache.invalidate.user, retry.flaky-third-party, legacy.migration.v1-to-v2).

A typical service ends up with on the order of 10-50 named portals. The number is small because the named unit is a scope of debug interest, not a function or a line. Many functions live inside a single portal.

The cost of a static portal in inactive state — three conditional branches and a thread-local lookup — is small enough that this density is supportable. Volume 8 ch. 8 quotes the measured numbers.

2.15 Cross-references and the next chapter

What the application surface looks like is now in place. The next layer down is what the recording looks like on the wire: the envelope format the serialiser emits, the layout of the buffered writer's output, and the contract the server is built to parse on the other end.

The next chapter walks the trace-event protocol — including a documented mismatch between the pod-side serialiser and the server-side deserialiser as the source stands today. That mismatch is honest engineering signal worth surfacing: the protocol has evolved, the two ends have not yet been re- synchronised, and the book does not pretend otherwise.

→ Continue to 03-trace-event-protocol.md.

← Back to 01-why-time-travel.md.