nexus-mock-provider

A faithful, high-performance mock of the three mainstream LLM API specs — OpenAI, Gemini, and Anthropic — for load- and performance-testing gateways, proxies, and clients. It speaks each provider's real wire format (streaming SSE and non-streaming), echoes your request back, and reports plausible token usage — so traffic exercises your whole pipeline (routing, auth, accounting, streaming) without a real, paid, rate-limited provider behind it.

Built to sustain tens of thousands of concurrent connections: no per-request logging, a fast JSON path (go_json), zero-goroutine precomputed streaming, container-aware GOMAXPROCS, and a tuned HTTP server.

load generator ─▶ your gateway/proxy ─▶ nexus-mock-provider ─▶ echoed response
                  (the thing you're      (this project: free,
                   actually measuring)    instant, deterministic)

It's a test fixture, not a secure service. It does not validate API keys (any credential is accepted), echoes request content, and enables permissive CORS. Run it on loopback or a trusted network. See SECURITY.md.

---

Supported specs & endpoints

Select which to serve with --spec (default all mounts every spec; their path namespaces don't collide except GET /v1/models, which is dispatched by the anthropic-version header).

Spec	Chat	Embeddings	Models	Streaming
OpenAI	POST /v1/chat/completions · POST /v1/responses (+ GET/DELETE /v1/responses/{id})	POST /v1/embeddings	GET /v1/models, /v1/models/{id}	chat: data:{chunk}…[DONE]; responses: response.created…response.completed (no [DONE])
Gemini	POST /v1beta/models/{m}:generateContent · :streamGenerateContent (?alt=sse or JSON-array) · :countTokens	:embedContent · :batchEmbedContents	GET /v1beta/models, /v1beta/models/{m}	data: {GenerateContentResponse} (no [DONE])
Anthropic	POST /v1/messages · POST /v1/messages/count_tokens	— (no native embeddings API)	GET /v1/models, /v1/models/{id}	event: <name> + data: {…} (message_start … message_stop, no [DONE])
Gemini OpenAI-compat	POST /v1beta/openai/chat/completions	POST /v1beta/openai/embeddings	GET /v1beta/openai/models	OpenAI framing

Plus operational endpoints: GET /healthz (liveness), GET /version (build metadata plus started_at, the process start instant — sample it at both ends of a measurement window to detect a mid-window restart, which build metadata alone cannot reveal since a restarted process reports the same commit).

Each spec's GET /v1/models-style endpoint returns that provider's real, current model ids in its native envelope (OpenAI gpt-5.5/gpt-5.4-mini/ text-embedding-3-*; Gemini gemini-3.5-flash/gemini-2.5-pro/gemini-embedding-2; Anthropic claude-fable-5/claude-opus-4-8/claude-sonnet-4-6). Add a prefix with --model-prefix (e.g. mock → mock-gpt-5.5), or replace the whole list with --models.

Quick start

make run                 # builds with go_json, serves :3062, --spec all
# or directly:
cd src && go run -tags go_json ./cmd --server-port 3062 --spec all

Build / Docker:

make build               # -> bin/nexus-mock-provider (host platform, version-stamped)
make build-linux         # -> static linux/amd64
docker build -t nexus-mock-provider src && \
  docker run --rm -p 3062:3062 nexus-mock-provider --server-port 3062 --spec all

Smoke test (all three specs)

# OpenAI
curl -s localhost:3062/v1/chat/completions -H 'content-type: application/json' \
  -d '{"model":"gpt-5.5","messages":[{"role":"user","content":"hi"}]}'
curl -s localhost:3062/v1/models | head -c 120

# Gemini
curl -s 'localhost:3062/v1beta/models/gemini-3.5-flash:generateContent' \
  -H 'content-type: application/json' \
  -d '{"contents":[{"parts":[{"text":"hi"}]}]}'

# Anthropic
curl -s localhost:3062/v1/messages \
  -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"claude-opus-4-8","max_tokens":100,"messages":[{"role":"user","content":"hi"}]}'

The mock echoes your last user message back as the completion.

Configuration

Every setting is available as a flag or an environment variable (the env value becomes the flag's default; an explicit flag wins). This keeps it ergonomic under both systemd (env) and the command line (flags).

Flag	Env var	Default	Description
--server-port	MOCK_SERVER_PORT	3000	TCP port to bind (all interfaces).
--spec	MOCK_SPEC	all	Spec(s) to mount: all or comma list of openai,gemini,anthropic.
--stream-delay-ms	MOCK_STREAM_DELAY_MS	0	Per-frame streaming delay (ms); 0 = instant. Simulates a slow provider.
--pprof-addr	MOCK_PPROF_ADDR	(off)	Start net/http/pprof on this address (e.g. 127.0.0.1:6060).
--models	MOCK_MODELS	(per-spec defaults)	Comma-separated override for every spec's model catalog.
--model-prefix	MOCK_MODEL_PREFIX	(none)	Prefix added to advertised model ids, e.g. mock → mock-gpt-5.5.
(n/a)	GIN_MODE	release	Set to debug for gin debug mode.

Examples:

# Only the OpenAI surface, on :8080, advertising mock-prefixed ids
nexus-mock-provider --spec openai --server-port 8080 --model-prefix mock

# Anthropic + Gemini, custom catalog, 20ms/token streaming, pprof on
MOCK_SPEC=anthropic,gemini MOCK_MODELS=foo,bar MOCK_STREAM_DELAY_MS=20 \
  MOCK_PPROF_ADDR=127.0.0.1:6060 nexus-mock-provider

Performance & high concurrency

Designed to disappear into the background of a load test even at very high RPS:

• No per-request logging — access logging is off; only panics are logged.
• Fast JSON — gin's bind/render and the codecs' stream-frame marshaling both use goccy/go-json (-tags go_json, wired into Makefile/Docker/CI/release).
• Zero-goroutine streaming — SSE frames are precomputed and written through a single shared writer; no goroutine-per-stream. (Anthropic's named-event frames use typed structs, not maps, to stay allocation-light.)
• Container-aware — go.uber.org/automaxprocs sets GOMAXPROCS from the cgroup CPU quota, avoiding scheduler thrashing under CPU limits.
• Tuned http.Server — ReadHeaderTimeout (slowloris guard) + IdleTimeout (reclaim idle keep-alives); no WriteTimeout so long SSE streams aren't cut.
• Graceful shutdown — SIGINT/SIGTERM drains in-flight requests.

Tips: raise LimitNOFILE (the systemd unit sets 1048576); for max-throughput / gateway-overhead numbers prefer non-streaming requests (the mock returns instantly, so you measure your gateway, not it); profile under load with --pprof-addr and go tool pprof.

Behaviour & fidelity notes

• Echo semantics: the last user message is echoed back, capped to the request's max-output-tokens (default 256); empty input → "ok". Usage scales with input size.
• Deterministic: same input → same output (including embedding vectors, which are L2-normalized so ‖v‖≈1).
• Response ids: OpenAI chat completions return a unique id per completion, chatcmpl-mock-<base36 process nonce>-<base36 counter>. The chatcmpl-mock- prefix is stable and is the contract — assert on the prefix, never on the whole string. The id is deliberately not constant: clients that upsert per-request records keyed by the upstream response id (e.g. LiteLLM's SpendLogs) collapse every request into one row when it is. The nonce is derived from the process start time, so it is stable for the life of one mock process and changes on restart — useful for detecting a mid-run restart. Anthropic messages (msg_mock) and the OpenAI Responses item id (msg_mock) are still fixed; only the chat-completions id is unique.
• No auth enforcement: any/no credential is accepted (it's a test fixture).
• Routing is by endpoint, not by model name. The response format is determined by the URL you call, not the model field — which is only echoed back. So a cross-spec model on the wrong endpoint (e.g. claude-… to /v1/chat/completions, or gpt-… to /v1/messages), an unknown model, or a typo all return 200 in that endpoint's format with the model echoed — never a model_not_found/404 (the catalog is configurable, and the mock deliberately doesn't get in the way).
• Tools/function-calling, images, and thinking blocks are accepted and ignored; the mock echoes text. Anthropic has no embeddings endpoint (faithful to the real API, which recommends Voyage AI).

Deployment

• Systemd + nginx: DEPLOY.md, deploy/.
• Behind a gateway as a provider: CONFIGURE-NEXUS.md.
• Behind LiteLLM / Bifrost: CONFIGURE-LITELLM-BIFROST.md.

Development

make check      # gofmt + go vet + go test
make cover      # tests + aggregate coverage (~97%)
make bench      # benchmarks
make lint       # golangci-lint

Architecture: a spec-agnostic core (pkg/core: echo + token + vector engine) and one SSE writer (pkg/sse) under three thin per-spec codecs (pkg/spec/{openai,gemini,anthropic}) that translate to/from each wire format. Adding a provider = adding one codec. The Go module lives under src/. See CONTRIBUTING.md.

License

Released under the Apache-2.0 License.