LLM Providers
Providers are how Hoziron connects to the language models that power your agents. The platform is model-agnostic — you can use cloud APIs (OpenAI, Anthropic, Azure), self-hosted models (Ollama, vLLM), or a mix of both. Complexity-based routing lets you send simple tasks to fast, cheap models and reserve expensive ones for work that needs them.
This design means you're never locked into a single vendor. Swap providers, add fallbacks, or migrate to local models without changing your agents or competencies.
What you'll find here
| Guide | Description |
|---|---|
| Configuring providers | Register providers in config.toml, set API keys, verify connectivity |
| Local models | Set up Ollama or vLLM, handle Docker networking, manage model pulls |
| Complexity routing | Define thresholds for automatic model selection based on task complexity |
Common tasks
- Declare a cloud provider → Configuring providers § Config shape
- Connect to Ollama running on the host → Local models § Ollama setup
- Size-classify requests into local/economy/standard tiers → Complexity routing § The three tiers
- List the live provider/model inventory → Configuring providers § Listing the inventory
- Check what the dashboard's Local/Cloud pool view shows → Complexity routing § Gateway pools
- Understand why there's no "add provider" API → Configuring providers § The inventory is boot-time config
How routing works
Requests classify into one of three tiers (local/economy/standard) purely by size, then resolve against that tier's ranked provider preference — filtered by live circuit-breaker availability. The carrier's PII policy can independently force a request into the local tier's resolution path regardless of size. See Complexity routing for the full mechanism.
Key configuration
The provider/model inventory is a closed, boot-time-only set declared in config.toml — there is no runtime API to add a provider (ADR-053):
[[provider]]
id = "big.licence"
driver = "anthropic" # closed set of kernel-shipped drivers
api_key_env = "ANTHROPIC_API_KEY"
[[provider.model]]
id = "claude-sonnet-4-5-20260929"
[[provider]]
id = "ollama.west"
driver = "ollama" # AlwaysLocal — structurally in-boundary
url = "http://localhost:11434/v1"
[[provider.model]]
id = "llama3.2"
[routing_gateway.standard]
preference = [{ provider = "big.licence", model = "claude-sonnet-4-5-20260929" }]
API keys are read from environment variables at request time — they're never stored in the config file itself. See Configuring providers for the full schema and the closed driver list.
Related
- Agents — each agent is assigned to (or routed between) providers
- Deployment § Docker Compose — networking between Hoziron and local model servers
- Security — protecting API keys and provider credentials