ollama-gateway

A lightweight authenticated HTTP proxy that sits in front of a single Ollama instance. It enforces API token authentication, tracks per-token token usage in memory, and exposes a /usage endpoint.

Architecture

Client
  │
  │  Authorization: Bearer <token>
  ▼
┌─────────────────────────────┐
│       ollama-gateway        │
│                             │
│  ┌─────────────────────┐    │
│  │  Auth middleware    │    │
│  │  (Bearer token)     │    │
│  └────────┬────────────┘    │
│           │                 │
│    ┌──────┴──────┐          │
│    │             │          │
│  /usage      all other      │
│  handler     requests       │
│    │             │          │
│    │         ┌───┴──────┐   │
│    │         │  Reverse │   │
│    │         │  Proxy   │   │
│    │         └───┬──────┘   │
│    │             │          │
│  Usage      Inspect body    │
│  Store  ◄── for token usage │
│             │               │
└─────────────┼───────────────┘
              │  plain HTTP
              ▼
        Ollama instance
        (OLLAMA_BASE_URL)

Prerequisites

• Go 1.21 or later
• A running Ollama instance

Environment Variables

Variable	Required	Default	Description
OLLAMA_BASE_URL	Yes	—	Base URL of the Ollama backend (e.g. http://localhost:11434)
API_TOKENS	Yes	—	Comma-separated list of valid Bearer tokens (e.g. token-a,token-b)
PORT	No	11434	Port the gateway listens on
HTTPS	No	false	Set to true to enable TLS on the client-facing listener. Must be true or false; any other value is a fatal error.
HTTPS_CERTIFICATE	When HTTPS=true	/app/cert.pem	Path to the PEM certificate file. The file must exist when HTTPS=true.
HTTPS_PRIVATE_KEY	When HTTPS=true	/app/key.pem	Path to the PEM private key file. The file must exist when HTTPS=true.

Quick Start

Build:

make build
# or
CGO_ENABLED=0 go build -o ollama-gateway .

Run:

export OLLAMA_BASE_URL=http://localhost:11434
export API_TOKENS=my-secret-token,another-token
export PORT=11434          # optional

./ollama-gateway
# ollama-gateway listening on :11434, proxying to http://localhost:11434

Use it exactly like the Ollama API — just add the Authorization header:

# List models
curl https://localhost:11434/api/tags \
  -H "Authorization: Bearer my-secret-token"

# Generate (non-streaming)
curl https://localhost:11434/api/generate \
  -H "Authorization: Bearer my-secret-token" \
  -d '{"model":"llama3","prompt":"Why is the sky blue?","stream":false}'

# Generate (streaming)
curl https://localhost:11434/api/generate \
  -H "Authorization: Bearer my-secret-token" \
  -d '{"model":"llama3","prompt":"Why is the sky blue?"}'

Without a valid token — 401:

curl -i https://localhost:11434/api/tags
# HTTP/1.1 401 Unauthorized

Usage Endpoint

GET /usage returns a JSON snapshot of accumulated token usage, grouped by date then token. The same authentication requirement applies.

curl https://localhost:11434/usage \
  -H "Authorization: Bearer my-secret-token"

{
  "usage": {
    "2026-04-16": {
      "my-secret-token": {
        "requests": 5,
        "prompt_tokens": 120,
        "completion_tokens": 340,
        "total_tokens": 460
      },
      "another-token": {
        "requests": 2,
        "prompt_tokens": 40,
        "completion_tokens": 95,
        "total_tokens": 135
      }
    }
  }
}

Usage is tracked in memory and resets when the gateway restarts.

How Token Usage is Captured

Ollama responses include token counts in JSON fields prompt_eval_count and eval_count. The gateway inspects every response body without buffering:

• Streaming responses (NDJSON): each chunk is flushed to the client immediately; the final "done": true object's counts are recorded.
• Non-streaming responses: the single JSON object is inspected on close.

The request counter is incremented unconditionally for every proxied request, regardless of whether the response contains usage fields.

More details here - https://docs.ollama.com/api/usage

Invoking the endpoint

You will need to provide an API key to access the gateway. Here are some sample curl commands:

Fetch usage

curl -k https://localhost:11434/usage

Fetch usage with filters (date, apikey, model)

curl -k https://localhost:11434/usage/2026-04-21/
curl -k https://localhost:11434/usage/2026-04-21/abcdefghij/
curl -k https://localhost:11434/usage/2026-04-21/abcdefghij/qwen3.5:0.8b

Perform a request

curl -k https://localhost:11434/api/generate \
  -d '{
    "model": "qwen3.5:0.8b",
    "prompt": "What is 1 + 4?",
    "stream": false
  }'

Client Image

mountainpass/ollama-gateway-client is a lightweight nginx-based Docker image that acts as a local proxy to a remote ollama-gateway server. It automatically injects the required API key on every request, so local tools (e.g. Ollama clients, Open WebUI) can talk to the gateway without needing to know about authentication, or accepting self signed certificates.

Environment Variables

Variable	Required	Description
TARGET	Yes	URL of the remote ollama-gateway server (e.g. https://202.111.222.123:11434)
APIKEY	Yes	Bearer token to authenticate with the gateway

The container exits immediately with a descriptive error message if either variable is missing.

Usage

docker run -d \
  -e TARGET=https://202.111.222.123:11434 \
  -e APIKEY=my-secret-token \
  -p 11434:11434 \
  --name ollama-gateway-client \
  mountainpass/ollama-gateway-client

Once running, any Ollama-compatible client pointed at http://localhost:11434 will have its requests transparently proxied and authenticated.

Build

docker build -f Dockerfile.client -t mountainpass/ollama-gateway-client .

Development

make test    # run tests with race detector
make build   # produce ./ollama-gateway binary
make run     # build and run with example env vars