LLM Providers
Providers are how Hoziron connects to the language models that power your agents. The platform is model-agnostic — you can use cloud APIs (OpenAI, Anthropic, Azure), self-hosted models (Ollama, vLLM), or a mix of both. Complexity-based routing lets you send simple tasks to fast, cheap models and reserve expensive ones for work that needs them.
This design means you're never locked into a single vendor. Swap providers, add fallbacks, or migrate to local models without changing your agents or competencies.
What you'll find here
| Guide | Description |
|---|---|
| Configuring providers | Register providers in config.toml, set API keys, verify connectivity |
| Local models | Set up Ollama or vLLM, handle Docker networking, manage model pulls |
| Complexity routing | Define thresholds for automatic model selection based on task complexity |
Common tasks
- Add an OpenAI provider → Configuring providers § Cloud providers
- Connect to Ollama running on the host → Local models § Ollama setup
- Route simple queries to a small model → Complexity routing § Defining thresholds
- Test that a provider is reachable → Configuring providers § Verifying connectivity
- Use Azure OpenAI with custom endpoints → Configuring providers § Azure configuration
- Set a default model for all agents → Configuring providers § Default model
How routing works
The router evaluates each request against your configured thresholds. If the primary provider for a complexity tier is unavailable, the circuit breaker kicks in and routes to the next healthy provider.
Key configuration
Provider configuration lives in your config.toml:
[providers.openai]
api_key_env = "OPENAI_API_KEY"
default_model = "gpt-4o"
[providers.ollama]
base_url = "http://localhost:11434"
default_model = "llama3.1"
API keys are read from environment variables at request time — they're never stored in the config file itself.
Related
- Agents — each agent is assigned to (or routed between) providers
- Deployment § Docker Compose — networking between Hoziron and local model servers
- Security — protecting API keys and provider credentials