Replay & history
The event log keeps a session inspectable forever, but a finished session needs two things a raw log does not give directly: a fast way to find it among thousands, and a way to reproduce the run rather than just read it. This page covers the completed-session index the console browses, deterministic replay via cassettes, and the reproducibility posture the whole event-sourced design delivers.
Completed sessions
Section titled “Completed sessions”When a session reaches a terminal outcome, the orchestrator writes one summary row to a per-tenant completed-session index — a denormalized record distinct from both the live session registry (which evicts on terminate) and the full event log (which stays queryable but needs a known session id). The row carries what a history list needs at a glance: the root goal, the coordination kind, the terminal status and — for a failure — its failure class, the total cost, the agent count, and the start/finish timestamps. The write is best-effort: if it fails, the durable log still carries the full history, so the index can be reconciled and never becomes a second source of truth.
The index is queried with keyset pagination, not offset — the cursor is the
(completed_at, session_id) pair, ordered newest-first, so paging stays stable and cheap
as the table grows. Queries filter by status, manifest, test-run flag, and time window,
with a bounded page size. This surface is gRPC-only — REST is deliberately a
human-facing subset — and the console consumes it in the Sessions rail’s History tab,
loaded lazily on first view. Opening one completed session reconstructs its full spawn
tree from the durable log and joins a per-agent cost rollup, so a single request answers
config, bindings, cost, and memory provenance for a run that is no longer live.
Deterministic replay with cassettes
Section titled “Deterministic replay with cassettes”The event log makes a run inspectable; cassettes make it reproducible. A cassette records a run’s LLM interactions once and replays them with zero live provider calls — the classic record-then-replay (VCR) pattern, applied one layer up from the HTTP wire, at the provider abstraction. Because it wraps the provider trait rather than the transport, it works against any provider — HTTP, gRPC, or an in-process fake.
Matching is by request identity: a cassette entry is keyed by a digest over only the semantically-identifying request fields — the messages, model hint, tools, and response format — deliberately excluding incidental tuning like sampling temperature, so a run matches on what it asked, not how it was tuned. A digest miss is a hard error, never a silent fall-through to a live call, so a drift in request shape surfaces loudly instead of quietly costing money and determinism. That buys reproducible tests, re-runnable incidents, and multi-configuration bake-offs recorded from a single live pass and replayed independently. It is a substrate primitive, not a test-only trick — the orchestrator’s compute-matched harness and the Deep Research evaluation harness both build on it.
A separate, operator-facing tool keeps a smaller set of hand-authored provider cassettes
honest against real provider wire shapes: it re-records each against the live endpoint,
classifies genuine structural drift (a field added or removed, a type changed) apart from
incidental value churn, and opens an issue when a provider’s shape moves. It never
rewrites the test assertions itself — a human decides whether a shape change is acceptable.
The reader-level walkthrough is on Monitoring, and the tool’s own
doc is in the repository at
docs/cassette-refresh.md.
The reproducibility posture
Section titled “The reproducibility posture”Reproducibility is one of the properties that separates an open stack from a closed platform, and here it is a mechanism rather than a claim. Transparency comes from event-sourcing: every decision a session made is on the log, inspectable, not hidden behind an API. Reproducibility comes from two layers on top of it — cassettes replay a run’s model calls deterministically, and a session-bootstrap reproducibility manifest pins the versions of the extension handlers that ran, so a later inspector can tell whether a run is still fully re-derivable or only structurally auditable. The manifest’s honesty is worth stating plainly: the guarantee is audit-preservation and registry-pinned re-derivation, not bit-identical replay of arbitrary model output — it records which handler code ran and flags when a pinned version has drifted or gone missing, rather than promising the universe was frozen. Grounded, not marketed: the open stack’s edge is that these are inspectable and runnable on your own infrastructure, not that any run is magically deterministic.
Where this shows up
Section titled “Where this shows up”- Operate: The operator console browses the completed-session index; Monitoring covers the cassette-refresh workflow.
- Build: Deep Research is a worked domain whose evaluation harness runs on replayed cassettes.
- Reference: the completed-session store, the provider replay layer, and the reproducibility types under Reference.