Next-Gen High-Performance AI Router & Token Saver for Coding Agents
Connect Claude Code, Cursor, Codex, OpenCode, Cline, and Copilot to AI providers with session-aware routing and granular multi-tenant access control.
Forked from the official decolua/9router (v0.5.75 Core)
Enhanced with enterprise-grade features from VansRouter and intelligent combo routing from OmniRoute.
Standard AI routers often suffer from three major shortcomings when driving long-running coding agents:
- Broken Prompt Caching: Naive round-robin can switch accounts between turns, causing upstream prompt-cache misses and avoidable cost or latency.
- Missing Access Control: Anyone with an API key can access every connected model and provider without restrictions.
- Agentic Loops & Degeneration: Reasoning models (like Kimi K2.6/K2.7) frequently enter degenerate overthinking loops or leak unparsed raw markup into client outputs.
RcRouter solves all of this out-of-the-box:
- β Context-Relay (Cache-Aware Routing): Reuses the same eligible upstream account across successful turns when a session ID is supplied, improving the chance of upstream prompt-cache hits.
- β Upstream v0.5.75 Core: Fully aligned with the latest official 9router core (1M context window auto-compact, 4-marker Claude cache budget caps, Kiro protocol updates).
- β
Granular Tri-State ACL: Per-API-key restrictions across service kinds, providers, combos, and specific models (
null= permit all,[]= deny all, array = whitelist). - β Bounded read caching: 5-second settings and 2-second connection caches, with explicit invalidation on mutation, reduce repeated SQLite reads within a process.
- β
Account Capacity Guard: Per-connection concurrency caps (default 3) with FIFO queueing and cancellation-safe slot release; configure
maxConcurrencyin a connection's provider-specific settings. - β
In-Memory Circuit Breaker: Keyed per
provider:proxyHashthat automatically isolates failing upstreams while excluding HTTP 429 rate limits. - β Token Savers (Ponytail + Caveman + RTK): Injects senior-developer YAGNI rules (Lite, Full, Ultra) and terse formatting into system prompts; output reduction depends on the prompt and provider.
- β
Anti-Loop Hardening: Built-in
LoopGuardterminates repetitive planning loops and converts Kimi-native<|tool_calls_section_begin|>into standard OpenAItool_calls. - β
Seamless Auto-Migration: Automatically migrates configurations and accounts from
~/.9routerto~/.rcrouteron initial startup without manual setup.
Modern LLMs (Anthropic Claude 3.5/3.7, Google Gemini 2.5/3, DeepSeek V3) feature Prompt Caching. When large contexts (50kβ150k tokens of repository code) are re-sent in multi-turn conversations:
- Cache Hit: A provider may discount cached input and return it faster according to its own cache policy.
- Cache Miss: The provider may bill and process the full context again; the exact impact depends on provider pricing and workload.
In naive routers with round-robin or random rotation:
Turn 1: Developer sends 60k tokens of code ββ> Sent to Account A (Cache created on Account A)
Turn 2: Developer asks follow-up question ββ> Rotated to Account B (CACHE MISS! Re-billed 60k tokens)
Turn 3: Developer asks to edit file X ββ> Rotated to Account C (CACHE MISS! Re-billed 60k tokens)
Every rotation wipes out the prompt cache.
When a combo uses the context-relay strategy, RcRouter extracts a session identifier from x-session-id, x-conversation-id, session-id, body.session_id, body.conversation_id, or body.user. The binding is tenant-scoped, expires after 30 minutes of inactivity, and is held in memory (so a restart clears it):
Turn 1: Request arrives (Session ID: "sess-abc")
βββ> Account A succeeds; RcRouter anchors the session to Account A.
Turn 2: Follow-up request arrives (Session ID: "sess-abc")
βββ> RcRouter prefers Account A if eligible; the upstream may reuse its cache.
Turn 3: Account A hits rate-limit (HTTP 429) or upstream error (5xx)
βββ> RcRouter invalidates the failed binding and tries an eligible fallback.
βββ> The session re-anchors to Account B only if that request succeeds.
Key Advantages:
- Cache continuity: Reusing an eligible account can preserve upstream prompt-cache locality; actual savings depend on the provider and request.
- Failure safety: A failed request does not create an affinity binding.
- Resilient failover: Retryable account failures can switch to another account or target, with a new binding after success.
RcRouter supports 6 specialized strategies per combo:
| Strategy | Description | Best For |
|---|---|---|
context-relay |
Anchors sessions to the same target for prompt cache hits; fails over on error | Claude Code, Cursor, long multi-turn sessions |
p2c |
Power of Two Choices: picks 2 random candidates, routes to the least loaded | High-concurrency teams, latency reduction |
reset-aware |
Prioritizes accounts whose free monthly quota resets soonest (< 48h) | Maximizing free tier tokens before expiration |
fallback |
Tries targets in strictly configured sequence; moves to next on error | Simple primary/backup chains |
round-robin |
Sequential cyclic rotation across all healthy targets | Distributing uniform load across accounts |
fusion |
Queries all candidates in parallel; judge model synthesizes one optimal answer | Highest possible reasoning and code quality |
The virtual auto, auto/coding, auto/fast, auto/cheap, and
auto/reasoning models apply their policy only after candidate validation.
Disabled connections, model locks, cached quota exhaustion, circuit-breaker
cooldowns, and API-key ACL denials are excluded before routing. If no eligible
target remains, the API returns 503 No eligible models available for auto combo; it never sends the synthetic auto/fallback model upstream.
RcRouter implements strict tri-state access control per API key:
null/undefined: Allow all (unrestricted key).[](empty array): Deny all (total restriction).["item1", "item2"]: Whitelist (only listed items permitted).
- Service Kind:
allowedKinds(llm,embedding,image,tts,stt,video,webSearch). - Provider Whitelist:
allowedProviders(anthropic,openai,glm,kiro, etc.). - Combo Whitelist:
allowedCombos(combo/coding,combo/fast). - Model Whitelist:
allowedModels(restricts specific model IDs). - Catalog Discovery:
GET /v1/modelsautomatically filters visible models according to the caller's key. - Internal Trust Bypass: Constant-time token verification supporting both
x-rc-cli-tokenand legacyx-9r-cli-tokenheaders.
Reduce upstream token usage without modifying client code:
- Ponytail (Senior Dev YAGNI Ruleset): Injects concise design ladder instructions directly into system prompts:
lite: Standard implementation with a one-line mention of simpler alternatives.full: Enforced ladder: standard library first, native features second, shortest diff.ultra: Aggressive YAGNI: favors deletion over addition, prefers one-line stdlib solutions.
- Caveman Mode: Enforces terse, jargon-free output format.
- RTK (Request Token Killer): In-place compression of bulky
tool_resultcontents (such as git diffs, file searches, directory listings). - LoopGuard: Enabled by default; detects repeated tool calls or text-only planning loops and injects a stop-and-summarize hint. Disable it with
PATCH /api/settings({"loopGuardEnabled":false}) when a provider intentionally repeats context.
- ZCode (Z.ai GLM-5.2): Integrates ZCode Plan GLM models with automated Aliyun CAPTCHA solving via headless Playwright.
- Local SearXNG: Self-hosted web search provider (
http://127.0.0.1:8888/search) with zero API keys required. - Indonesian & Southeast Asian TTS: Pre-registered neural voices (
id-ID-ArdiNeural,id-ID-GadisNeural,th-TH-PremwadeeNeural,ms-MY-YasminNeural,tl-PH-BlessicaNeural). - Multi-Account Compatible Nodes: Allows operators to add unlimited connection keys to a single custom OpenAI or Anthropic compatible node.
# npm
npm install -g rcrouter
# pnpm
pnpm i -g rcrouter
rcrouterThe CLI bundles the production server and opens the dashboard at
http://localhost:20128. Use rcrouter --no-browser on headless machines.
cd RcRouter
# Install dependencies
pnpm install
# Start development server (port 20128)
pnpm run devDashboard is accessible at http://localhost:20128/dashboard.
API endpoint is available at http://localhost:20128/v1.
cd RcRouter
# Production standalone build (includes auto DB backup & static asset copying)
pnpm run build
# Start with PM2 on default port 20128:
PORT=20128 pm2 start server.js --name rcrouter
# Or specify custom port for Nginx reverse proxy (e.g. 3003):
PORT=3003 pm2 start server.js --name rcrouter
# Save PM2 state across system reboots
pm2 saveCreate ~/.config/systemd/user/rcrouter.service:
[Unit]
Description=rcrouter - Local AI Model Proxy Gateway (port 20128)
After=network.target network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/node /path/to/RcRouter/server.js
WorkingDirectory=/path/to/RcRouter
Environment="PORT=20128"
Environment="HOSTNAME=0.0.0.0"
Environment="NODE_ENV=production"
Restart=always
RestartSec=5
[Install]
WantedBy=default.targetEnable and start the service:
systemctl --user daemon-reload
systemctl --user enable --now rcrouter.servicepnpm install --frozen-lockfile
pnpm test # regression gate against reviewed baseline
pnpm test:raw # full Vitest output
pnpm test:focused tests/unit/capabilities.test.js
pnpm lint # lint regression gate against reviewed baseline
pnpm lint:raw # full ESLint output, including known debt
pnpm check # lint + regression gate
pnpm run build
pnpm run release:cloud-smoke # Cloudflare Worker embeddings fixtureThe offline suite is green: all runnable tests pass. pnpm test still compares
against the historical reviewed baseline in tests/__baseline__/README.md so
future regressions remain visible; do not refresh that snapshot merely to hide
a new failure. Live/real-provider tests remain explicitly gated on credentials
or RUN_LIVE_TESTS=1.
The opt-in provider lane is available from GitHub Actions β Live provider tests.
Set the protected RCROUTER_LIVE_TESTS=1 repository secret before dispatching it;
the default quality workflow never contacts external providers. The optional real
provider and RTK jobs require a runner with a seeded DATA_DIR/API key and are
enabled explicitly through workflow inputs.
The optional Cloudflare Worker slice lives in cloud/. It shares the
open-sse core and exposes /health plus OpenAI-compatible /v1/embeddings.
pnpm run release:cloud-smoke runs its deterministic fixture and is part of
pnpm run release:verify.
Account capacity is process-local. Set a connection's maxConcurrency to a
positive number to cap in-flight upstream requests, or 0/null to bypass
the cap. Multi-process deployments need a shared coordinator if limits must
apply across workers.
GET /api/metrics is protected by dashboard authentication (or the machine-bound
CLI token). It returns bounded, process-local counters, gauges, latency summaries,
circuit-breaker states, and redacted semaphore identifiers; prompts, credentials,
and raw session identifiers are not included. Chat responses carry an
x-rc-correlation-id header that can be matched with request-detail diagnostics.
export ANTHROPIC_BASE_URL="http://localhost:20128/v1"
export ANTHROPIC_AUTH_TOKEN="<your-rcrouter-api-key>"
claude- OpenAI API Key:
<your-rcrouter-api-key> - OpenAI Base URL:
http://localhost:20128/v1 - Models:
combo/glm-5.2,combo/claude-opus-4.8,kr/claude-sonnet-4.5
In ~/.codex/config.toml:
model = "combo/glm-5.2"
model_provider = "RcRoute"
[model_providers.RcRoute]
name = "RcRouter"
base_url = "http://localhost:20128/v1"
wire_api = "responses"- Upstream Project: decolua/9router
- License: MIT License (Open Source)