Deep Research

Deep Research is the reference vertical: web search + fetch producing a research report in which every claim carries a fetched citation. The research domain — its framing, web tools, output shape, and citation verification — rides the substrate’s skill plane, so the same domain runs under a coordination strategy you pick at session start. This page is the worked example: what a run does, the two credentials it needs, the three ways to launch it, what you can tune, and how to read what comes back.

What a run does end to end

At reader altitude a run is a five-move pipeline, driven by the coordinator you select:

1. Plan. The planner decomposes your goal into a set of sub-topics. The subtopic_breadth knob sets how many it aims for and the hard cap enforced at plan conversion.
2. Research in parallel. One worker per sub-topic runs the RETRIEVE→REASON pattern — web_search to find sources, web_fetch to read them, then it extracts claims that the fetched sources actually support. A claim that can’t be grounded in a fetched source is dropped, never padded.
3. Validate citations. Each worker is paired with a validator that re-fetches the cited URLs and runs an LLM judge (NLI-style entailment) over every claim. The worker passes only if every claim is grounded — the field-standard claim-decomposition → per-claim entailment → binary groundedness gate.
4. Judge coverage. A coverage gate checks how many planned sub-topics were actually covered. Below the coverage_strictness bar, the loop replans and researches the gaps; at or above it, the run proceeds to synthesis.
5. Synthesize. The synthesizer merges the validated per-sub-topic claims into a structured report — sections grouped by sub-topic, each claim carrying its citation URLs verbatim, plus an executive summary.

Prerequisites — two credentials

A real run needs two credentials, and they live in the same one per-tenant CredentialStore (there is no parallel secret store — see the credential plane). The substrate resolves each one server-side per tenant and injects it at the sandbox boundary; a sandboxed tool never reads process env.

• An LLM provider credential drives the planner / worker / validator / synthesizer model calls. This is the provider plane’s own credential (e.g. anthropic), declared in deploy/local/providers.yaml.
• A web-search provider credential is the tool credential the workers’ web_search needs. The native provider defaults to Brave (BRAVE_API_KEY); set [substrate] native_search_provider — or the worker’s APOMESH_SEARCH_PROVIDER env var — to serper to use Serper (SERPER_API_KEY, or the neutral alias APOMESH_SEARCH_API_KEY).

The search key rides the same credentials: block in providers.yaml as your provider keys — its id must match the provider’s required-credential name (BRAVE_API_KEY / SERPER_API_KEY), and its secret.ref names the env var (in ~/.apomesh/env) the value resolves from at import. Declare only the provider whose key you actually set; local import is strict, so a declared credential whose secret is unresolvable aborts boot. See the local stack for the bootstrap flow.

Caution

Without a search credential the workers’ web_search surfaces a typed config error at dispatch — the substrate never crashes and no secret is logged, but the workers have nothing to retrieve, so the run yields an empty or near-empty report. Set the key before launching a real run.

The two shipped bundles

The daemon seeds two fully pre-configured, strategy-locked published agents into the dev tenant at boot. Both bind the same base Deep Research manifest and the same per-role research plane; they differ only in the coordinator they lock:

Published agent	Locked strategy	Coordinator
io.descoped/deep-research@1.0.0	io.descoped/deep-research-vmao@1.0.0	VMAO loop — plan → execute → verify → replan
io.descoped/deep-research-mad@1.0.0	io.descoped/deep-research-debate@1.0.0	Multi-agent debate — governed lanes over rounds, majority vote

Both are published dev-private (shared to other tenants only by an explicit operator act). The strategy configs they lock are seeded alongside them, so the launch surface offers real coordinator choices out of the box rather than an empty picker. The source of truth for these shapes is catalog.rs.

VMAO or MAD? Reach for the VMAO bundle for the standard research loop — plan, research, verify, replan the gaps. Reach for the MAD (debate) bundle when you want multiple independent research lanes to argue toward a consensus answer rather than converge through replanning. The coordinators are explained in VMAO and Debate; this page does not re-explain them.

The role plane

Both bundles bind the same three research skills across the coordinator’s roles; the planner stays unbound because it runs in-process with no tool-use loop, so a bound skill’s tools would be inert:

Role	Bound skill	What it carries
Worker	io.descoped/research-worker	RETRIEVE→REASON framing + the web_search / web_fetch tools
Validator	io.descoped/research-validator	the citation-grounding rubric (an LLM-judge over every claim)
Synthesizer	io.descoped/research-synthesizer	synthesis framing + the report output schema
Planner	(unbound)	—

These are bound per role, not through the manifest’s flat default_skills — per-role binding is what reaches every coordinator’s worker lanes. The VMAO bundle keeps deep-research’s hardcoded prompts (its per-role skills stay empty, a zero-behaviour-change path); the debate bundle sets the per-role skills so the debate lanes receive the research framing the debate skeleton does not hardcode.

Launch it

From the console

The catalog’s Configure & Launch surface offers the coordination-strategy picker: choose Deep Research — VMAO or Deep Research — MAD (debate), set your goal, and launch. Alternatively, start directly from one of the two published bundles (io.descoped/deep-research or io.descoped/deep-research-mad) on the published-agent launch surface — that path uses the sealed, strategy-locked shape as-is. See published agents for what a sealed shape locks.

From the Python SDK

apomesh.deep_research.run drives a research session end-to-end and returns a typed DeepResearchReport. Its signature is run(goal_text, endpoint, *, budget_dollars="10.00", tenant_id="", project_id="", config=None):

import asyncio

from apomesh.deep_research import DeepResearchConfig, SynthesisDepth, run


async def main() -> None:
    report = await run(
        "What are the leading approaches to durable agent state?",
        "127.0.0.1:50051",
        budget_dollars="10.00",
        tenant_id="dev",
        config=DeepResearchConfig(synthesis_depth=SynthesisDepth.COMPREHENSIVE),
    )

    # DeepResearchReport is fully typed — walk it directly.
    print(report.title)
    for section in report.sections:
        print(f"\n## {section.heading}")
        for claim in section.claims:
            print(f"- {claim.statement}")
            for url in claim.citations:
                print(f"    ↳ {url}")
    print(f"\n{report.summary}")


asyncio.run(main())

Every field on DeepResearchReport, ReportSection, and ReportClaim is exported from apomesh.deep_research, so you consume the result as typed data — render it to Markdown, PDF, or a UI as a downstream concern. See the Python SDK guide for install and connection detail.

The demo CLIs

Three runnable modules exercise the vertical from a driver’s seat against a running daemon — a good first end-to-end check:

# Research goal in, structured report + per-provider cost breakdown out.
python -m apomesh.deep_research.demos.driving --daemon localhost:50051

demos.verified_citation observes the recovery path when a worker fabricates a citation, and demos.coverage_replan observes the bounded coverage-replan loop. Each runs in stub mode by default (a fake-LLM daemon) or against live providers with --real.

Over REST

A deep-research dispatch is a plain POST /api/v1/sessions with "sessionKind": "deepResearch", then a tail of GET /api/v1/events. The REST + SSE guide has the full request body and curl flow.

Tuning knobs at launch

Launch-time tuning rides an opaque per-launch config the substrate ferries to the vertical unread (the vertical_config bytes). From the SDK, build a DeepResearchConfig; the daemon deserializes it into its own typed config. Every field is optional and falls back to the vertical’s shipped default — an absent, empty, partial, or even malformed config all resolve to defaults rather than failing the run:

Knob	Effect	Default
synthesis_depth	Report depth: CONCISE, STANDARD, or COMPREHENSIVE	STANDARD
coverage_strictness	Fraction of plan sub-topics that must be covered to pass the coverage gate	3/4
citation_strictness	Fraction of a claim’s citations that must ground it	1/2
subtopic_breadth	Planner sub-topic target range + hard cap (broader → longer report)	target 3–7, cap 10
min_claims_per_subtopic	Thoroughness floor a worker is asked to meet per sub-topic	none

A degenerate ratio or breadth (a zero denominator, target_min > target_max, …) is not rejected — the vertical clamps it back to its default at resolve time, so the SDK stays a faithful carrier.

For the long, detailed report, pair SynthesisDepth.COMPREHENSIVE with a raised synthesizer max_output_tokens on the role config, and pin per-role models through the agent-configuration plane. The manifest ships a default model with a small allowed set; per-role model pins are validated against that set at dispatch.

Reading the results

The synthesized DeepResearchReport is a typed JSON shape: a title, a list of sections (each a heading plus cited claims), and a summary. Each claim carries its statement and the citations — the source URLs, preserved verbatim so cite-check traceability survives synthesis.

In the console, the Output facet renders the run’s deliverable — the terminal verdict, the full synthesized report (with Copy as Markdown), its citations, and reproducibility provenance; see the console walkthrough. Citation-verification outcomes surface through the run’s events and coverage audits: the deep-research projection crate supplies the dependency-light consumer surfaces — a pure handler that enriches the coverage-audit events and a sandboxed schema-check unit — that let the light console and worker consume those signals without pulling the heavy vertical runtime.

Budget posture

The Deep Research manifest ships a default cost budget, and a run is confirmation-gated because it is a multi-agent loop. That budget covers everything the run spends: the LLM token cost across the planner, every parallel worker, the per-worker validators, the coverage judge, and the synthesizer, plus the per-query cost of each web_search call. A broader subtopic_breadth, a stricter coverage bar, or COMPREHENSIVE synthesis all spend more against it. See budgets & cost for how the budget caps a run and how spend is attributed.

Where this shows up

• Python SDK — the deep_research.run driver.
• REST + SSE — the transport-neutral dispatch.
• VMAO and Debate — the two coordinators the bundles lock.
• Source: verticals/deep-research and its README.