Chapter 4 — Memory Architecture

An agent's memory is what makes it more than a stateless request/response. The SDK's memory layer has three distinct concerns: conversation history (the dialogue the agent is having now), persistent memory (facts the agent should remember across runs), and retrieval (finding the relevant subset of memory for the current moment). This chapter walks each.

The memory layer has three loosely-coupled subsystems:

• Conversation history via LLMConversation — short-term, in-process, truncates as needed.
• Persistent memory via MemoryStore and its implementations — long-term, scope-aware, optionally embedded for semantic search.
• Summarisation and retrieval — ContextSummarizer for compressing conversation; MemoryIndex for finding relevant entries.

The chapter walks each.

Conversation history — LLMConversation

File: llmconversation.py:15–84

class LLMConversation:
    """
        An ordered, mutable sequence of LLMMessage objects
        representing a conversation history.
    """

    def truncate(self, max_messages: int) -> None:
        """
            Removes the oldest non-system messages to fit
            within the limit, preserving all system messages.
        """
        if len(self._messages) <= max_messages:
            return
        system_messages = [m for m in self._messages if m.role == "system"]
        non_system_messages = [m for m in self._messages if m.role != "system"]
        available_slots = max_messages - len(system_messages)
        kept_non_system = non_system_messages[-available_slots:] if available_slots > 0 else []
        self._messages = system_messages + kept_non_system

The class is an ordered list of messages, mutable, with the framework's truncation discipline. Three observations:

System messages are pinned

A typical agent conversation has one or more system messages at the start (the agent's instructions). The truncation rule preserves all system messages and removes oldest non-system messages first.

This matters because system messages are the agent's shape — without them, the LLM's responses drift away from the agent's intended behaviour. Truncation that removed system messages would produce subtly broken behaviour over long conversations.

Truncation is demand-driven

The LLMConversation doesn't automatically truncate. Application code calls truncate(max_messages) when it decides the conversation is too long. This separation lets applications choose their truncation policy:

• Truncate before every LLM call (cap at N messages).
• Truncate on token-count threshold.
• Truncate after specific events (a "task completed" message clears the queue).
• Don't truncate (let the conversation grow indefinitely; the LLM provider may complain when context is exceeded).

Each agent's start() or step() method decides.

Mutability is structured

The class is mutable (you can append, truncate, remove) but the messages themselves are immutable. An LLMMessage(role="user", content="hello") constructed once is constant for its lifetime; the conversation is the ordered collection of such immutables.

This mirrors the framework's broader "data is immutable; collections are mutable" pattern.

Persistent memory — MemoryStore

File: memorystore.py:25–127

class MemoryStore(abc.ABC):
    """
        Abstract base class for memory storage backends.
        Provides a common interface for storing, retrieving,
        searching, updating, and deleting MemoryEntry objects.
    """
    # ... store / retrieve / search / update / delete abstract methods ...

The store is backend-agnostic. The SDK ships FileMemoryStore (chapter 4 worked example below); other backends — Redis, PostgreSQL, vector databases — are implemented as subclasses.

Three concepts compose the persistent memory model:

MemoryEntry

File: memoryentry.py:17–48

A single piece of remembered information:

• identifier — UUID or string key.
• content — the actual data.
• tags — list of strings for categorical filtering.
• metadata — dict for free-form attributes.
• scope — MemoryScope enum value (below).
• created_at, updated_at — timestamps.
• embedding — optional vector for semantic search.
• expiration — optional ExpirationPolicy for TTL.

Entries are immutable — modification means constructing a new entry with the same identifier and update()-ing.

MemoryScope

File: memoryscope.py:13–24

class MemoryScope(Enum):
    """
        Visibility scope for memory entries.
    """
    AGENT = "agent"      # visible only to the owning agent
    SHARED = "shared"    # visible to agents using the same store
    GLOBAL = "global"    # visible to all agents in the runtime

Three scopes. Each agent's memory queries default to AGENT scope (only its own entries); a query with scope=MemoryScope.SHARED returns entries from any agent sharing the store; GLOBAL returns entries from all agents in the runtime.

The scope distinction matters for multi-agent systems. Two agents collaborating on a task can share information through SHARED entries; private agent state stays AGENT-scoped.

EmbeddingProvider

File: embeddingprovider.py:15–41

class EmbeddingProvider(abc.ABC):
    """
        Abstract base for embedding providers — services that
        turn text into fixed-length vectors for semantic search.
    """

    @abc.abstractmethod
    def embed(self, text: str) -> list[float]:
        ...

    @abc.abstractmethod
    def batch_embed(self, texts: list[str]) -> list[list[float]]:
        ...

When an EmbeddingProvider is configured on a store, new entries get embeddings computed at storage time. Queries with semantic_text parameters use cosine similarity through MemoryIndex (below).

The provider is optional. Stores without an embedding provider support exact-match and tag-based queries; semantic queries return empty.

FileMemoryStore — the worked implementation

File: filememorystore.py:32–195

class FileMemoryStore(MemoryStore):
    """
        Built-in MemoryStore implementation that persists
        MemoryEntry objects to JSON files on the local file
        system.  Each entry is stored as a separate JSON file
        using the entry identifier as the file name.
    """

The simplest persistent backend. Each entry becomes a JSON file at <store_root>/<identifier>.json. Reads, writes, deletes are filesystem operations. No database; no schema migrations; no indexing beyond what MemoryIndex (in-memory) computes.

Limitations:

• Scaling. Tens of thousands of entries become slow. Production agents with large memories use a different backend (Redis, PostgreSQL, etc.).
• Concurrency. Multiple agents writing to the same store risk file-level race conditions. The SDK's file-locking is best-effort.

For development and small-scale production agents, FileMemoryStore is enough. For larger agents, the abstract MemoryStore is the integration point — write a custom backend, register it.

Querying — MemoryQuery

File: memoryquery.py:15–45

class MemoryQuery:
    def __init__(self,
                 tags: list = None,
                 metadata: dict = None,
                 scope: MemoryScope = None,
                 start_time: datetime = None,
                 end_time: datetime = None,
                 semantic_text: str = None,
                 max_results: int = None,
                 min_relevance: float = None) -> None:

A query is a bundle of filters:

• tags — entries must have all listed tags.
• metadata — entries must have matching metadata key/value pairs.
• scope — entries must be in the given scope.
• start_time / end_time — temporal bounds on created_at.
• semantic_text — for similarity search; the store's EmbeddingProvider embeds this and MemoryIndex ranks entries by cosine similarity.
• max_results — limit the result set.
• min_relevance — for semantic queries, only return entries above a relevance threshold.

The store applies each filter; the result is the intersection of entries that match all filters.

MemoryIndex — semantic similarity

File: memoryindex.py:17–52

class MemoryIndex:
    """
        In-memory cosine-similarity index over embedded entries.
    """

    def search(self, query_vector: list[float],
               max_results: int = 10,
               min_relevance: float = 0.0) -> list[ScoredEntry]:
        """
            Returns entries ranked by cosine similarity to
            query_vector.
        """
        # ... cosine similarity ranking ...

The index is in-memory. For each query:

1. Compute the query's embedding (via the EmbeddingProvider).
2. Rank stored entries by cosine similarity.
3. Filter to top-K above threshold.

The implementation is straightforward — dot-product divided by L2 norms. Stores can override with their own indexing (a vector-database-backed store would push similarity search into the database).

Summarisation — ContextSummarizer

File: contextsummarizer.py:22–108

def summarize_messages(self, messages: list) -> LLMMessage:
    """
        Summarizes a list of LLMMessage objects into a single
        summary message.
    """
    if messages is None or len(messages) == 0:
        errmsg = "Cannot summarize an empty list of messages."
        raise SemanticError(errmsg)

    content_parts = []
    for msg in messages:
        content_parts.append("[{}] {}".format(msg.role, msg.content))
    combined_content = "\n".join(content_parts)

    prompt_content = ("Summarize the following conversation "
                      "concisely, preserving key information:\n\n"
                      "{}").format(combined_content)
    # ... call LLM provider with this prompt; return the response as
    # a system-role LLMMessage ...

Summarisation is another LLM call. The summariser:

1. Formats the messages with role prefixes.
2. Constructs a system prompt asking for a concise summary.
3. Calls the configured LLM provider.
4. Returns a single system-role LLMMessage with the summary.

The cost is one LLM call's worth of tokens. For long conversations, this is far cheaper than carrying the full history in every subsequent call.

When to summarise

The SDK doesn't impose a summarisation policy. Common patterns:

• Threshold-based. When the conversation exceeds N messages, summarise the oldest M messages and replace them with the summary.
• Token-based. When the token count approaches the context window, summarise to fit.
• Phase-based. After a logical phase (a multi-step task completes), summarise the phase and start fresh.

Application code decides. A typical agent run has explicit summarisation calls between work units.

Checkpointing — CheckpointManager

File: checkpointmanager.py:22–75

class CheckpointManager:
    """
        Persists and restores WorkflowExecution state at
        configurable checkpoint boundaries using a MemoryStore
        backend.
    """

    def save_checkpoint(self, execution_id: str,
                        context: WorkflowContext,
                        completed_step_ids: set,
                        current_step_id: str) -> None:
        """
            Persists a WorkflowCheckpoint to the MemoryStore.
        """
        try:
            context_state = context.as_dict()
            workflow_name = context.get_metadata("workflow_name", "unknown")
            checkpoint = WorkflowCheckpoint(
                workflow_name=workflow_name,
                execution_id=execution_id,
                completed_step_ids=completed_step_ids,
                context_state=context_state,
                current_step_id=current_step_id,
            )
        # ... store as MemoryEntry ...

Checkpoints are how agents survive crashes. When a long-running workflow's process dies, a fresh process can:

1. Load the most recent checkpoint via manager.load_checkpoint(execution_id).
2. Reconstruct the workflow context from the checkpoint's context_state.
3. Resume from the current_step_id rather than starting over.

The checkpoint is itself a MemoryEntry, stored through the same MemoryStore that persists other agent memory. This means checkpoints inherit the store's properties — on disk for FileMemoryStore, durable for a database-backed store, distributed for a Redis-backed store.

Checkpoints are tagged with the workflow name for efficient lookup; manager.list_checkpoints(workflow_name) returns all checkpoints for a given workflow.

Chapter 5 walks the workflow layer that uses checkpoints.

Memory's interaction with cost tracking

Every memory operation that reads or writes goes through the store. The store can (but doesn't have to) record costs:

• A FileMemoryStore has zero per-operation cost — just filesystem I/O.
• A vector-database-backed store has per-query cost (database CPU, network I/O).
• A store that embeds at write time via an EmbeddingProvider has per-write cost (the embedding call's tokens).

When the embedding step uses an LLM-style provider, those calls do hit the agent's cost tracker. An agent that writes 10,000 entries to memory and embeds each one pays for 10,000 embedding requests. The cost tracker shows the breakdown by provider; an SDET reviewing costs can see "embedding-provider-1" charged separately from "main-llm-provider."

What memory deliberately doesn't do

A few patterns the SDK leaves to the application:

No automatic summarisation

The summariser is a tool the agent calls when it decides. The SDK doesn't watch the conversation length and trigger summarisation automatically.

This is for predictability. An automated summarisation that fires unexpectedly mid-conversation can produce surprising behaviour (the LLM "forgets" things it said two turns ago). Explicit calls let the application decide when the cost / behaviour-change is acceptable.

No automatic memory expiration

Entries can carry an ExpirationPolicy, but expiration is evaluated lazily — entries are removed on the next query or write that touches them. No background sweeper.

This avoids the operational complexity of a sweeper thread that needs to be scheduled, monitored, and tuned.

No cross-agent message passing

The SDK's memory is for state, not messaging. Agents that need to communicate use a separate message-routing layer (the SDK ships a MessageRouter — chapter 5 covers).

What's next

Chapter 5 walks workflow orchestration for agents. The SDK's WorkflowExecutor traverses a DAG of steps with conditional branches, parallel execution, checkpointing (via the CheckpointManager from this chapter), and human-in-the-loop approval gates. Chapter 5 also covers CircuitBreaker for handling repeated failures gracefully.