apomesh-substrate-llm

apomesh-substrate-llm is a substrate implementation plane: the concrete provider plugins that back the LLMProvider, EmbeddingProvider, and GenerativeMediaProvider ports defined in apomesh-substrate-types. It owns the vendor-specific machinery — HTTP calls, SSE streaming, AWS SigV4 signing, model catalogs, and pricing — that the dependency-light vocabulary crate deliberately does not carry. It does not own routing or tenancy: the runtime constructs these providers and injects them behind the ports. You touch this crate when a provider’s wire behaviour, a model catalog, or the provider-config stores change — never to add a routing or per-tenant policy.

Position in the workspace

• Layer: substrate implementation plane. See Context plane for the model-invocation narrative and Credential plane for how per-tenant credentials reach a provider.
• Depends on: apomesh-substrate-types — the provider ports and the provider-agnostic request/response/streaming vocabulary — plus apomesh-substrate-net, the shared webpki-roots reqwest client every provider composes for HTTP egress.
• Consumed by: apomesh-substrate — the runtime that owns ProviderRouter / EmbeddingProviderRouter, per-tenant wiring, and credential resolution, and re-exports these modules as a flat facade — and apomesh-substrate-tools, which composes providers for tool-side model calls.

What it owns

• The concrete LLM plugins — AnthropicProvider, BedrockProvider, GeminiProvider, OpenAIChatProvider, OpenRouterProvider, and the deterministic FakeLlmProvider, each implementing the -types LLMProvider port. The seam stays open: a new vendor is a new plugin, never a change to the port.
• The embedding and generative-media plugins — OpenAIEmbeddingProvider, GeminiEmbeddingProvider, VoyageEmbeddingProvider, the three Bedrock embedding providers, and OpenAIImageProvider / OpenAIAudioProvider behind the EmbeddingProvider / GenerativeMediaProvider ports.
• The model catalog and pricing plane — ModelSpecCatalog, ModelSpec, ModelRate, the curated seed catalogs, and ModelCatalogCache: the models a provider defaults to and prices against, operator-overlayable at config.
• The provider-config persistence contracts — LlmConfigStore, CredentialStore, and ProviderProfileStore (with in-memory adapters), provenance-layered by ConfigSource so an operator edit is never clobbered by a boot re-seed.
• Deterministic replay, rate control, and streaming — the cassette record-then-replay VCR over LLMProvider, RateControlledProvider plus the per-provider token buckets, and the SSE aggregation shared across plugins.
• Typed auth adapters — AnthropicAuth and the Bedrock SigV4 signer turn a resolved ProviderCredential into a provider-native auth shape; the auth types redact their secret bytes in Debug.

Public surface

• AnthropicProvider — the production LLM plugin; from_env reads deployment auth, from_profile builds from a stored profile + resolved per-tenant credential.
• BedrockProvider / GeminiProvider / OpenAIChatProvider / OpenRouterProvider — the other concrete LLMProvider plugins, each with its own auth and wire shape.
• FakeLlmProvider — the deterministic in-process provider for fixtures and tests, no network.
• cassette::RecordingProvider / ReplayProvider — the record-then-replay VCR over any LLMProvider, so a harness reproduces a run deterministically without depending on a vertical.
• AnthropicAuth — the credential → Anthropic-native auth adapter (TryFrom<&ProviderCredential>); a mismatched credential is a typed Configuration error, never a panic.
• ModelSpecCatalog / ModelSpec / ModelRate — the model catalog and per-unit pricing; seed_catalog builds the curated default set, with_overlay layers an operator’s config.
• LlmConfigStore / CredentialStore / ProviderProfileStore — the provider-config persistence ports (model rows, credentials, profiles), each with an in-memory adapter and a provenance-guarded upsert.
• RateControlledProvider — the adaptive-concurrency limiter the runtime wraps each constructed provider in.
• StreamAggregator / parse_sse_stream — the SSE streaming-chunk parser and aggregator the streaming plugins share.

Usage

The provider plane reads its own boot config from the environment for the substrate’s own model calls — a carve-out distinct from the tool-credential forwarding path. from_env returns a typed error when auth is unset, and the auth type redacts in Debug, so a token never reaches a log line. The runtime then wraps and injects the constructed provider; a dispatch site never names a concrete provider.

use apomesh_substrate_llm::llm_provider::{AnthropicProvider, LLMProviderError};

fn build_anthropic_provider() -> Result<AnthropicProvider, LLMProviderError> {
    // Reads ANTHROPIC_API_KEY (production) or, as a dev fallback,
    // CLAUDE_CODE_OAUTH_TOKEN. Both unset is an operator-fixable startup
    // error — a typed Configuration failure, never a panic.
    let provider = AnthropicProvider::from_env()?;

    // The runtime (apomesh-substrate) wraps this in RateControlledProvider and
    // injects it behind the LLMProvider port via ProviderRouter.
    Ok(provider)
}

Related

• Concepts: Context plane (model invocation and the streaming vocabulary these plugins serve) and Credential plane (how a resolved per-tenant credential becomes provider-native auth).
• Reference: apomesh-substrate-types (the ports and vocabulary this plane implements) and apomesh-substrate (the runtime that routes and wires these providers).