Provider Routing
How requests are routed to LLM providers — the operator-authored inventory, driver-declared sovereignty, complexity/PII/entitlement gating, model preference, circuit breakers, and fallback. Grounded in crates/platform/hoziron-core/src/provider/ (inventory.rs, preference.rs, config.rs) and crates/platform/hoziron-core/src/routing/ (gateway.rs, tier.rs, entitlement.rs, pii_policy.rs, provider.rs), plus ADR-034, ADR-053, ADR-054, ADR-055, ADR-056.
This module has been substantially reworked from an earlier honor-system design (a [providers.*] map keyed by an arbitrary string, IP-heuristic "is this local" detection, and a hardcoded LicenceSourced/CarrierConfigured split) to a code-authoritative, closed inventory with structurally-determined sovereignty. The routing fabric evaluates four dimensions in sequence — each can only route more restrictively than the one before it.
1. The Provider/Model Inventory Is a Closed, Operator-Authored Set (ADR-053)
There is no auto-scan of the kernel's builtin ~130-model catalog and no runtime API that mutates the inventory. The set of usable providers and models is declared explicitly and is the only set the gateway will ever consider:
[[provider]]
id = "ollama.west" # arbitrary, operator-chosen, unique instance id
driver = "ollama" # must match a kernel-shipped driver — closed set
url = "http://10.4.2.7:11434/v1"
[[provider.model]]
id = "llama3.2" # validated against the catalog for this driver
max_tokens = 2048
temperature = 0.1
[[provider]]
id = "big.licence"
driver = "anthropic"
api_key_env = "ANTHROPIC_API_KEY"
[[provider.model]]
id = "claude-sonnet-4-5-20260929"
Provider identity is split into instance (id, arbitrary) and driver (driver, a closed set of kernel-shipped implementations) — two Ollama boxes at different URLs are both representable, each with their own id, both using the ollama driver. A new model is a config entry; a new provider type requires a new kernel-shipped driver, never a config-only addition. The builtin model catalog is a metadata/validation source — consulted to verify and decorate config-declared models, never to supply them.
2. Sovereignty Is Driver-Declared, Not Operator-Typed (ADR-053)
Whether a provider instance counts as Local or Cloud for POPIA §72 sovereignty purposes is resolved from the driver's compile-time DriverDisposition, not from heuristics over the configured URL (the pre-ADR-053 approach — matching 127.0.0.1/192.168.x.x/10.x.x.x — rested entirely on an operator correctly typing a private-looking address, with no code-level guarantee):
| Driver disposition | Sovereignty | SovereigntyReason |
|---|---|---|
AlwaysLocal (e.g. Ollama, vLLM) | Local | NativeLocal — structurally local, no attestation needed |
Ambiguous, with a carrier sovereignty_attestation | Local | AttestedLocal — operator has explicitly attested this endpoint is in-boundary |
Ambiguous, no attestation | Cloud | DefaultedCloud — fails closed: an unattested ambiguous driver is treated as Cloud, never assumed safe |
AlwaysCloud (e.g. Anthropic, OpenAI) | Cloud | Cloud |
The SovereigntyReason is retained specifically so an auditor can distinguish "structurally local by driver type" from "operator attested in-boundary" — a materially different compliance claim. The local/cloud sovereignty promise rests on code, not on config-file string matching.
3. Provider Identity Collapses to One Meaningful Distinction (ADR-035, ADR-055)
The historical three-way RequestedProvider::{Local, HozironCloud, Byo} taxonomy — and the later two-way LicenceSourced/CarrierConfigured split that resolved through two entirely separate functions — is collapsed. There is one concept: a registered model in the inventory. The only commercially meaningful distinction is:
- Licence-sourced — the carrier licence's locked Anthropic tier models (
hoziron.cloud, a reserved pseudo-provider-id, never a real[[provider]]entry a carrier can declare — the inventory loader hard-errors on a collision). Metered, carrier-immutable; credentials and budget come from the signed licence. - Everything else — ordinary carrier-managed configuration (local models, BYO cloud endpoints), no licence opinion involved.
Both resolve through the same ResolvedModelTarget type and the same preference-ranking mechanism (§4) within whichever pool (Local/Cloud) they belong to — a carrier can rank a BYO cloud provider above the licence-sourced Anthropic entry within the Cloud pool (ADR-055).
Force-local resolution is cloud-impossible by construction. pii_policy:force_local and Tier::Local resolve through a dedicated local-only resolution function that has no branch capable of returning a Cloud variant — this is a return-type invariant, not a config setting a caller could get wrong.
4. Preference: Which Eligible Model Serves a Call (ADR-054)
Pool resolution (Local vs. Cloud) is unchallengeable once decided — nothing below it can cross the boundary. Within the chosen pool, a ranked preference list picks the specific model:
One shape (PreferenceEntry { provider, model }, array order = rank), three consumers with different strictness:
| Client | Config | Required? |
|---|---|---|
| Gateway | [routing_gateway].local_preference / cloud_preference | Boot error if empty for a pool the gateway may route to — the gateway cannot function without one |
| Agent | [model_preference].local / .cloud in the agent manifest | Optional; empty → gateway ranking stands. Cannot promote a Cloud model into a Local pool decision — that's a load/boot error, not a silent no-op |
| Competency manager | [competency_manager].preference | Optional and soft-degrading: an unresolvable or empty manager preference falls back to static competency order rather than failing the invocation — deliberately asymmetric with the gateway, since one operator typo here must not take down every invocation for every agent |
Every preference entry is validated against the ADR-053 inventory; a dangling reference or a cross-pool entry in the gateway's or an agent's ranking is a boot/load error (the competency manager's is a skip-and-warn instead, per its soft-degrade rule).
Provider Authentication
[providers.anthropic]
api_key_env = "ANTHROPIC_API_KEY" # name of env var, resolved lazily via resolve_env_secret
| Env Var State | Auth Status | Effect |
|---|---|---|
| Set, non-empty (vault or process env) | Configured | Provider usable |
| Unset or whitespace | Missing | Excluded from available_models() |
N/A (no api_key_env, e.g. local Ollama) | NotRequired | Always usable |
Keys are checked at call time, not at startup — adding or removing a credential takes effect without a daemon restart. Provider credentials are Zeroizing-wrapped end-to-end from vault decrypt through ResolvedModelTarget to the driver; ResolvedModelTarget carries a manual redacting Debug so a derived impl cannot leak the key.
Circuit Breaker and Fallback
Unchanged in mechanism from earlier iterations of the routing layer:
[health]
failure_threshold = 5
recovery_cooldown_secs = 60
When the primary preference entry's circuit is open, the gateway walks the next entry in the same pool's preference ranking (never crossing pools) rather than an unranked "any registered provider for this model" search — this is a direct consequence of routing through the preference mechanism rather than a separate fallback list.
Local-Only Echo Provider
echo (echo-1) is an always-present, deterministic test/fallback provider — deliberately not a member of the ADR-053 inventory (excluded from local_pool()/cloud_pool(), preference resolution, and agent [model_preference] validation). It exists purely so the platform has a guaranteed-usable driver when no real [[provider]] is configured, mirroring the pre-ADR-053 register_echo_provider behavior without folding it into the closed, operator-authored set.
Related:
- ../architecture/data-flow.md — provider resolution in the context of a full request
- pii-data-protection.md — the PII-dimension gate that can force local routing
- workflow-engine.md — billing keyed on outbound writes, not model choice
docs/decisions/034-llm-routing-fabric.md,035-unified-model-routing.md,053-provider-model-inventory-and-driver-disposition-sovereignty.md,054-model-preference-over-eligibles.md,055-unify-licence-and-carrier-routing.md,056-unified-activation-gate.md