Skip to content
apomesh

Context plane

Every call to a language model has a hard limit — the context window — and a long session accumulates more history than that window holds. The raw history is never quite the prompt you want to send anyway: it needs a persona, the operator’s instructions, few-shot examples, a response-format directive. The context plane governs what a model call actually sees — how a session’s growing working memory is trimmed to fit, and how the trimmed context is compiled into the exact messages the provider receives. If you came looking for the prompt plane, this is it: prompt composition lives here.

The plane exists so that what a model sees is governed, inspectable, and reproducible — not ad-hoc string concatenation scattered across the codebase. Two durable events record every decision, so a replay can rebuild the exact prompt a call saw and verify it bit-for-bit.

What it governs

Section titled “What it governs”
  • Bounding working memory to a token runway, so a session’s history never overruns the model’s window.
  • Composing the prompt — turning a session’s decided inputs (system text, task turns, persona, few-shot examples, operator prompt, response format, sampling, reasoning) into the concrete messages a call carries.
  • Recording both decisions as events — CompactionApplied (the trim) and PromptComposed (the compile) — so the context a call saw is replayable.
  • What it does not govern: which model or provider runs and what the agent is configured to be (the agent-configuration plane), or long-term recall across sessions (the memory plane).

How it works

Section titled “How it works”

The plane is deliberately split into two sub-planes with opposite natures. Compaction is budget-driven, stateful, and may call a provider — an impure decision about what context survives. Composition is a pure, synchronous function that compiles whatever context it is handed — deterministic, provider-free, and therefore replay-stable. Keeping them apart is what lets the compile stay reproducible: a decision that could fail or vary (compaction) never contaminates the step whose determinism the replay guarantee depends on (composition).

Plane 1 bounds the history to a runway; plane 2 compiles the typed context into the call. The planner runs both; every worker dispatch runs a safety-bound plane 1.

Plane 1 — token-runway compaction

Section titled “Plane 1 — token-runway compaction”

As a session runs, two places accumulate conversation that can overflow a model window: the coordinator’s cross-iteration planner history (each plan appends the planner’s output), and a worker’s tool-use loop (every assistant turn and every tool result — fetched pages, command output — piles onto the conversation). Before a call is assembled, the Compactor estimates the token cost with a cheap, deterministic character-ratio heuristic and, if it exceeds the runway, trims to a CompactionStrategy:

StrategyWhat it keepsProvider call
SlidingWindow (default)pinned leading system messages + the most-recent turns that fit; drops the oldestnone — pure recency
RunningSummarythe recent tail, plus one rolling LLM summary of the dropped prefixa summarizer dispatch
SemanticTruncationthe messages an LLM judge scores most relevant to the goal, admitted greedily by scorea relevance-judge dispatch

The two LLM-backed strategies route a CompactionSummarizer at a configured model tier (a cheap tier by default); SlidingWindow calls nothing. Every strategy keeps at least the most recent turn, even if it alone exceeds the runway — dropping the live turn would defeat the loop.

Two consumers, two runways. The planner runs the full, configured strategy (with the summarizer and the fail-soft ladder below), gated by the session’s compaction config. Each worker dispatch runs an always-on, pure SlidingWindow pass at a large fixed safety runway sized to the model context window — a hard floor that stops accumulated tool results from tripping a “prompt too long” provider error, independent of any strategy the operator chose. Both emit a CompactionApplied event tagged with their scope (planner or worker:<id>) whenever they actually trim.

Fail-soft, never fail-open. An LLM-backed strategy that fails does not fall back to the raw unbounded history — it degrades to the pure SlidingWindow, so a summarizer hiccup can never reintroduce overflow:

The planner compaction ladder: an LLM-backed strategy that fails degrades to pure sliding-window, never to the raw history

Plane 2 — the pure prompt transformer

Section titled “Plane 2 — the pure prompt transformer”

A PromptContext is the typed, fully-decided input to a prompt: its system_template and task_turns, the variables for placeholder interpolation, the already-compacted prior_turns, plus persona, exemplars, reasoning, operator_prompt, response_format, sampling, and the open-vocabulary extensions data slot. The PromptTransformer runs an ordered, closed PromptProcessor pipeline over that context and returns a CompiledPayload — the message list plus the response-format, sampling, and reasoning knobs that ride onto the call:

ProcessorContribution
Personaa role/persona system message at index 0
Interpolationresolves {name} placeholders; establishes the base system + task turns
OperatorPromptthe operator-authored text, composed as labelled user-role turns (untrusted content, never substrate control structure)
FewShotexemplars as structured user/assistant turn pairs
StructuredOutputsets the response-format directive
Samplingcopies the per-session sampling knobs onto the payload
ReasoningTriggernative reasoning effort, or a chain-of-thought instruction when the provider has no native knob
Extensiona consumer-registered pure step, keyed by an id, resolved in a boot registry (the M24 open seam)

Each processor is a no-op when its context field is empty, so a primitive is selected per session just by populating the field. The transform is synchronous, holds no live state, and makes no provider call — the purity is structural, not a convention.

The vocabulary is closed on purpose. The apply match over PromptProcessor is exhaustive, so a new composition step cannot ship without a compiled arm — the same closed-enum discipline the rest of the substrate relies on, which is what makes a replay’s pipeline record trustworthy. The one exception is the tier-4 Extension variant (added in M24): its variant stays closed and canary-guarded, but the vocabulary inside it is open — a consumer registers a pure processor in the boot-global registry and supplies its per-session data through the context’s extensions slot, without a substrate PR per signal. An unresolved id is a no-op; the base ships zero handlers. Because the registry is the same pinned version-set the session bootstraps against, an extension processor composes reproducibly across a replay.

Composing the system prompt

Section titled “Composing the system prompt”

What a model sees as its system context is composed, not hard-coded. A skill contributes system-prompt fragments, tool grants, verification scopes, and a response format through a ComposedSkillState, folded deterministically by skill id and layered manifest → per-role. Those fragments become the planner’s system context, and the manifest’s own fragments fold into the base worker system template.

Crucially, every role composes through the one transformer. The planner composes in-process; the Worker and Synthesizer roles compose orchestrator-side too — the orchestrator builds their PromptContext, runs the same PromptTransformer, and hands the worker a finished ComposedPrompt it executes verbatim rather than reassembling. One composition seam, one place the model’s input is decided. What that shape is at session start is pinned, so the same published agent composes the same prompt every run — see Coordination loops and Sessions, events & durability.

Per-role generation

Section titled “Per-role generation”

Generation knobs are resolved per capability tier (per-role, falling back to a session-uniform default) and ride this plane. A role’s sampling and reasoning feed the PromptContext, where the Sampling and ReasoningTrigger processors copy them onto the compiled call. Two related knobs ride beside the prompt: max_output_tokens shapes generation length (not the compaction runway), and the model override is a routing choice applied at the worker’s request — the PromptContext carries no model field. Where each is configured is the agent-configuration plane; this plane only consumes the resolved values.

Guarantees

Section titled “Guarantees”
  • The context window is never overrun. Compaction bounds the history to the runway before every dispatch, and always keeps at least the last turn. The worker’s always-on safety pass makes this true even when no strategy is configured.
  • Fail-soft, never fail-open. An LLM-backed strategy that fails degrades to the pure sliding-window — never to the raw unbounded history.
  • Composition is pure and reproducible. The transformer makes no provider call and holds no state; the same PromptContext and pipeline always compile to the same messages, verified by a SHA-256 content digest.
  • The context decision is inspectable and replayable. CompactionApplied and PromptComposed are on the durable event log, so a replay reconstructs the exact prompt and checks it against the recorded digest — consistent with the event-sourcing invariant.

Where this shows up

Section titled “Where this shows up”