Local Models

What you'll accomplish: Set up Ollama as a structurally-local inference provider, handle Docker networking, and understand exactly what "local" guarantees Hoziron gives you.

Why Ollama is different from an attested local endpoint

ollama (along with vllm, lmstudio, lemonade, and the built-in echo test provider) is an AlwaysLocal driver in the kernel's compiled-in disposition table (hoziron-runtime::drivers::driver_disposition). That means a provider entry with driver = "ollama" resolves to Sovereignty::Local structurally — no sovereignty_attestation, no config string, no operator claim is involved. This is the strongest of the three sovereignty guarantees the platform offers (the other two: Ambiguous drivers need an explicit attestation, and AlwaysCloud drivers like anthropic/openai can never be local, full stop — see Configuring providers § The closed driver set).

Practically: if your carrier's local_only PII policy forces a request local (see PII Engine), and your [routing_gateway.local] preference list points at an ollama provider, that request is guaranteed — by the driver's compiled-in disposition, not by trusting a string — to never carry an API key or reach a third-party endpoint.

Ollama setup

[[provider]]
id     = "ollama.west"
driver = "ollama"
url    = "http://127.0.0.1:11434/v1"

  [[provider.model]]
  id = "llama3.2:1b"

[routing_gateway.local]
preference = [{ provider = "ollama.west", model = "llama3.2:1b" }]

[routing_gateway.standard]
preference = [{ provider = "ollama.west", model = "llama3.2:1b" }]

No api_key_env is needed — Ollama's driver has NotRequired auth status by default (its auth_status is only Missing/Configured when you've explicitly set api_key_env on the entry).

standard must have a non-empty preference whenever the inventory has any provider at all (boot-validated) — for an all-local deployment, point it at the same Ollama entry as local so every tier resolves.

Verify

hoziron-cli providers list         # confirm the entry is Configured/NotRequired
curl http://localhost:4200/gateway/pools -H "Authorization: Bearer hzk_..."
# → "local": [{ "id": "ollama.west", "driver": "ollama", "disposition": "AlwaysLocal", ... }]

Network accessibility

Ollama binds to 127.0.0.1 by default. If Hoziron connects from a different host or container:

OLLAMA_HOST=0.0.0.0 ollama serve

Docker networking (Hoziron in a container, Ollama on the host)

docker run --add-host=host.docker.internal:host-gateway ...

[[provider]]
id     = "ollama.west"
driver = "ollama"
url    = "http://host.docker.internal:11434/v1"

Verify from inside the container:

wget -qO- http://host.docker.internal:11434/api/tags

Docker Compose (both in containers)

services:
  hoziron:
    image: ghcr.io/hozironos/hoziron:latest
    depends_on:
      - ollama
  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"

[[provider]]
id     = "ollama.west"
driver = "ollama"
url    = "http://ollama:11434/v1"

Attested-local endpoints (in-house OpenAI-compatible servers)

A self-hosted server that speaks the OpenAI-compatible protocol (vLLM, LM Studio behind a proxy, an in-house inference box) is not automatically AlwaysLocal unless it uses one of the dedicated local-runtime drivers (vllm, lmstudio, lemonade). If you're fronting it with the generic openai-compatible driver, it resolves as Ambiguous and needs an explicit attestation to be treated as local:

[[provider]]
id                      = "inhouse.vllm"
driver                  = "openai-compatible"
url                     = "http://10.4.2.7:8000/v1"
sovereignty_attestation = "carrier confirms 10.4.2.7 is in-boundary"

  [[provider.model]]
  id = "meta-llama/Llama-3.1-70B-Instruct"

Read Security § Sovereignty attestation is a trust assumption before relying on this for local_only PII — an attestation is an operator claim, not a technical proof, and the platform only runs a best-effort, false-positive-prone RFC-1918 heuristic to catch accidental misattestation.

Air-gapped deployments

For a fully self-hosted carrier, omit every AlwaysCloud provider entirely — there is no enabled = false flag to toggle (ADR-053 removed it); absence from config.toml is disablement. Point every tier's preference at your local provider(s):

[routing_gateway.local]
preference = [{ provider = "ollama.west", model = "llama3.2:1b" }]

[routing_gateway.standard]
preference = [{ provider = "ollama.west", model = "llama3.2:1b" }]

Next steps

Related: