Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,8 @@ chidori chat agent.ts

`chidori model-login` opens your browser, signs you in with OpenRouter, and saves a
key to `~/.chidori/credentials.json` — the zero-setup way to try things out.
Prefer your own provider key? Set `ANTHROPIC_API_KEY` (or `OPENAI_API_KEY`)
Prefer your own provider key? Set `ANTHROPIC_API_KEY` (or `OPENAI_API_KEY`,
or `ORCAROUTER_API_KEY` for the OrcaRouter routing gateway)
instead; explicit keys always take precedence over the OpenRouter fallback.

Then ask it things like *"What is a host call?"* or *"How do I write a tool?"*.
Expand Down Expand Up @@ -215,11 +216,23 @@ the per-file directive). See the
```bash
# The OpenRouter sign-in from step 1 is all you need. Prefer your own key?
# export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY=...
# export ORCAROUTER_API_KEY=sk-orca-... # or use the OrcaRouter routing gateway

chidori run summarizer.ts \
--input document="Rust is a systems programming language..."
```

**OrcaRouter is a named multi-provider routing gateway** — one endpoint in
front of Anthropic, OpenAI, Google, DeepSeek, and more, plus smart routing
(`orcarouter/auto`). Set `ORCAROUTER_API_KEY` and pick the model with `--model`
(or `CHIDORI_MODEL`):

```bash
export ORCAROUTER_API_KEY=sk-orca-...
chidori run summarizer.ts --model orcarouter/auto \
--input document="Rust is a systems programming language..."
```

**Any OpenAI-compatible provider (DeepSeek, Groq, Ollama, vLLM, LiteLLM…).**
Point Chidori at any endpoint that speaks the OpenAI chat-completions
protocol, and pick the model with `--model` (or the `CHIDORI_MODEL` env var —
Expand Down
13 changes: 7 additions & 6 deletions crates/chidori/src/init.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ Or call it with a fixed list of messages:
chidori run agent.ts --input '{"messages": ["Hi, who are you?"]}'

Every turn is a durable host call, so replaying the whole conversation costs
zero tokens. Set a provider key first (e.g. `ANTHROPIC_API_KEY` or
`OPENAI_API_KEY`) — or just run `chidori model-login` to sign in with OpenRouter and
skip the env var entirely.
zero tokens. Set a provider key first (e.g. `ANTHROPIC_API_KEY`,
`OPENAI_API_KEY`, or `ORCAROUTER_API_KEY`) — or just run `chidori model-login`
to sign in with OpenRouter and skip the env var entirely.
"#;

const WORKER_README: &str = r#"# Chidori worker agent
Expand All @@ -75,9 +75,9 @@ terminal (the ask-by-default policy for running unfamiliar code). Add
where gated effects fail closed.

Add your own tools with more `defineTool({...})` handles and register them in
the agent's `toolbox` map. Set a provider key first (e.g. `ANTHROPIC_API_KEY`
or `OPENAI_API_KEY`) — or just run `chidori model-login` to sign in with
OpenRouter and skip the env var entirely.
the agent's `toolbox` map. Set a provider key first (e.g. `ANTHROPIC_API_KEY`,
`OPENAI_API_KEY`, or `ORCAROUTER_API_KEY`) — or just run `chidori model-login`
to sign in with OpenRouter and skip the env var entirely.
"#;

/// The docs-chat agent: an offline-friendly RAG-lite assistant that answers
Expand Down Expand Up @@ -149,6 +149,7 @@ Sign in once with OpenRouter (opens your browser — no API key to manage):
…or set a provider key instead:

export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY=...
# export ORCAROUTER_API_KEY=sk-orca-... # or the OrcaRouter routing gateway

Then chat:

Expand Down
4 changes: 3 additions & 1 deletion crates/chidori/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1179,6 +1179,7 @@ fn cmd_demo() -> Result<()> {
println!("or set one of:");
println!(" export ANTHROPIC_API_KEY=sk-ant-...");
println!(" export OPENAI_API_KEY=sk-...");
println!(" export ORCAROUTER_API_KEY=sk-orca-... # OrcaRouter routing gateway");
println!(" # any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, LiteLLM...):");
println!(" export CHIDORI_OPENAI_COMPAT_URL=https://api.deepseek.com");
println!(" export CHIDORI_OPENAI_COMPAT_KEY=sk-...");
Expand Down Expand Up @@ -1266,6 +1267,7 @@ fn confirm_start_server(port: u16) -> Result<bool> {
fn has_llm_provider() -> bool {
std::env::var_os("ANTHROPIC_API_KEY").is_some()
|| std::env::var_os("OPENAI_API_KEY").is_some()
|| std::env::var_os("ORCAROUTER_API_KEY").is_some()
|| std::env::var_os("CHIDORI_OPENAI_COMPAT_URL").is_some()
|| std::env::var_os("LITELLM_API_URL").is_some()
|| providers::openrouter::saved_api_key().is_some()
Expand Down Expand Up @@ -1313,7 +1315,7 @@ fn ensure_llm_provider_interactive() -> bool {
println!();
println!(
"No LLM provider key found (ANTHROPIC_API_KEY / OPENAI_API_KEY / \
CHIDORI_OPENAI_COMPAT_URL)."
ORCAROUTER_API_KEY / CHIDORI_OPENAI_COMPAT_URL)."
);
println!("You can sign in with OpenRouter to try this out — no API key setup needed.");
if !providers::openrouter::confirm_login() {
Expand Down
22 changes: 20 additions & 2 deletions crates/chidori/src/providers/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
pub mod anthropic;
pub mod openai;
pub mod openrouter;
pub mod orcarouter;
pub mod rate_limit;

use std::sync::atomic::{AtomicUsize, Ordering};
Expand Down Expand Up @@ -368,6 +369,23 @@ impl ProviderRegistry {
registry.register(Box::new(p));
}

// OrcaRouter is a named OpenAI-compatible routing gateway
// (<https://www.orcarouter.ai>) — one endpoint in front of Anthropic,
// OpenAI, Google, DeepSeek, and more, plus smart routing. An explicit
// `ORCAROUTER_API_KEY` registers it ahead of the OpenRouter fallback
// (so it wins whenever both are configured); like OpenRouter it
// matches every model and only handles requests no explicit provider
// above claimed. See [`orcarouter`] for the model-id translation.
if let Ok(api_key) = std::env::var(orcarouter::ORCAROUTER_API_KEY_ENV) {
if !api_key.trim().is_empty() {
let mut p = orcarouter::OrcaRouterProvider::new(api_key);
if let Some(rpm) = rpm_env("CHIDORI_ORCAROUTER_RPM") {
p = p.with_rate_limit(rpm);
}
registry.register(Box::new(p));
}
}

// OpenRouter is the zero-config fallback: an `OPENROUTER_API_KEY`, or a
// key saved by a prior `chidori model-login` / demo OAuth sign-in. Registered
// last and matching every model, so it only handles requests no
Expand All @@ -391,7 +409,7 @@ impl ProviderRegistry {
}
}
bail!(
"No provider found for model '{}'. Set ANTHROPIC_API_KEY or OPENAI_API_KEY, point CHIDORI_OPENAI_COMPAT_URL + CHIDORI_OPENAI_COMPAT_KEY at any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, ...), or run `chidori model-login` to sign in with OpenRouter.",
"No provider found for model '{}'. Set ANTHROPIC_API_KEY or OPENAI_API_KEY, point CHIDORI_OPENAI_COMPAT_URL + CHIDORI_OPENAI_COMPAT_KEY at any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, ...), set ORCAROUTER_API_KEY for OrcaRouter, or run `chidori model-login` to sign in with OpenRouter.",
request.model
);
}
Expand All @@ -409,7 +427,7 @@ impl ProviderRegistry {
}
}
bail!(
"No provider found for model '{}'. Set ANTHROPIC_API_KEY or OPENAI_API_KEY, point CHIDORI_OPENAI_COMPAT_URL + CHIDORI_OPENAI_COMPAT_KEY at any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, ...), or run `chidori model-login` to sign in with OpenRouter.",
"No provider found for model '{}'. Set ANTHROPIC_API_KEY or OPENAI_API_KEY, point CHIDORI_OPENAI_COMPAT_URL + CHIDORI_OPENAI_COMPAT_KEY at any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, ...), set ORCAROUTER_API_KEY for OrcaRouter, or run `chidori model-login` to sign in with OpenRouter.",
request.model
);
}
Expand Down
3 changes: 2 additions & 1 deletion crates/chidori/src/providers/openrouter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,8 @@ pub fn to_openrouter_slug(model: &str) -> String {

/// Rewrite a trailing `-<digits>-<digits>` version segment with a dot:
/// `claude-sonnet-4-6` → `claude-sonnet-4.6`. Leaves anything else unchanged.
fn hyphen_version_to_dot(s: &str) -> String {
/// Shared with [`super::orcarouter`], whose catalog uses the same dot format.
pub(crate) fn hyphen_version_to_dot(s: &str) -> String {
let segs: Vec<&str> = s.split('-').collect();
let n = segs.len();
if n >= 2
Expand Down
166 changes: 166 additions & 0 deletions crates/chidori/src/providers/orcarouter.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
//! OrcaRouter provider — a named OpenAI-compatible routing gateway.
//!
//! OrcaRouter (<https://www.orcarouter.ai>) is a multi-provider routing
//! gateway: one OpenAI-compatible endpoint in front of Anthropic, OpenAI,
//! Google, DeepSeek, xAI, and more, plus smart routing (`orcarouter/auto`).
//! Setting `ORCAROUTER_API_KEY` registers it, and Chidori then routes every
//! model the explicit providers above didn't claim through it — the same
//! catch-all role the OpenRouter fallback plays, but as a named gateway a
//! user opts into with their own key rather than the OAuth sign-in.
//!
//! The wire format is OpenAI chat-completions, so like OpenRouter this is a
//! thin wrapper over [`OpenAiProvider`] pointed at OrcaRouter's base URL. The
//! only extra work is translating Chidori's model ids (`claude-sonnet-4-6`)
//! into OrcaRouter's namespaced catalog ids (`anthropic/claude-sonnet-4.6`)
//! on the way out.

use anyhow::Result;

use super::openai::OpenAiProvider;
use super::openrouter::hyphen_version_to_dot;
use super::{LlmProvider, LlmRequest, LlmResponse, TokenSink};

/// OrcaRouter's OpenAI-compatible chat endpoint.
const ORCAROUTER_CHAT_URL: &str = "https://api.orcarouter.ai/v1/chat/completions";

/// Env var carrying an OrcaRouter key directly (mirrors the other providers).
pub const ORCAROUTER_API_KEY_ENV: &str = "ORCAROUTER_API_KEY";

/// An OrcaRouter-backed LLM provider. Acts as a catch-all (`supports_model`
/// always true), so it slots in behind any explicit provider and ahead of the
/// OpenRouter fallback — an explicit `ORCAROUTER_API_KEY` wins whenever both
/// gateways are configured.
pub struct OrcaRouterProvider {
inner: OpenAiProvider,
}

impl OrcaRouterProvider {
pub fn new(api_key: String) -> Self {
Self {
// A single empty prefix makes the inner OpenAI provider match every
// model; we own routing via `supports_model` below.
inner: OpenAiProvider::with_base_url(
api_key,
ORCAROUTER_CHAT_URL.to_string(),
vec![String::new()],
),
}
}

pub fn with_rate_limit(mut self, rpm: u32) -> Self {
self.inner = self.inner.with_rate_limit(rpm);
self
}
}

#[async_trait::async_trait]
impl LlmProvider for OrcaRouterProvider {
fn supports_model(&self, _model: &str) -> bool {
true
}

async fn send(&self, request: &LlmRequest) -> Result<LlmResponse> {
let mut req = request.clone();
req.model = to_orcarouter_slug(&request.model);
self.inner.send(&req).await
}

async fn stream(&self, request: &LlmRequest, on_delta: &mut TokenSink) -> Result<LlmResponse> {
let mut req = request.clone();
req.model = to_orcarouter_slug(&request.model);
self.inner.stream(&req, on_delta).await
}
}

/// Translate a Chidori model id into an OrcaRouter catalog id.
///
/// - Anything already containing `/` is assumed to be an OrcaRouter catalog id
/// and passes through untouched (`orcarouter/auto`, `deepseek/deepseek-chat`).
/// - Claude ids are canonicalized via the Anthropic alias table, then the
/// trailing `-<major>-<minor>` version is rewritten with a dot to match
/// OrcaRouter (`claude-sonnet-4-6` → `anthropic/claude-sonnet-4.6`).
/// - OpenAI ids (`gpt*`, `o1*`, `o3*`, `o4*`) are prefixed with `openai/`.
/// - The bare `auto` router alias is namespaced to `orcarouter/auto` — the
/// OrcaRouter backend keys its routing channels on the namespaced id.
/// - Anything else passes through so an explicit catalog id always wins.
pub fn to_orcarouter_slug(model: &str) -> String {
if model.contains('/') {
return model.to_string();
}
let canonical = super::anthropic::resolve_alias(model);
let lower = canonical.to_ascii_lowercase();
if lower == "auto" {
return "orcarouter/auto".to_string();
}
if lower.starts_with("claude") {
return format!("anthropic/{}", hyphen_version_to_dot(canonical));
}
if lower.starts_with("gpt")
|| lower.starts_with("o1")
|| lower.starts_with("o3")
|| lower.starts_with("o4")
{
return format!("openai/{canonical}");
}
canonical.to_string()
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn maps_default_claude_model_to_orcarouter_slug() {
assert_eq!(
to_orcarouter_slug("claude-sonnet-4-6"),
"anthropic/claude-sonnet-4.6"
);
assert_eq!(
to_orcarouter_slug("claude-opus-4-7"),
"anthropic/claude-opus-4.7"
);
assert_eq!(
to_orcarouter_slug("claude-haiku-4-5"),
"anthropic/claude-haiku-4.5"
);
}

#[test]
fn maps_claude_aliases_before_slugging() {
assert_eq!(
to_orcarouter_slug("claude-sonnet"),
"anthropic/claude-sonnet-4.6"
);
assert_eq!(
to_orcarouter_slug("claude-3-5-sonnet"),
"anthropic/claude-sonnet-4.6"
);
}

#[test]
fn maps_openai_models() {
assert_eq!(to_orcarouter_slug("gpt-4o"), "openai/gpt-4o");
assert_eq!(to_orcarouter_slug("gpt-4.1-mini"), "openai/gpt-4.1-mini");
assert_eq!(to_orcarouter_slug("o3-mini"), "openai/o3-mini");
}

#[test]
fn namespaces_bare_auto_router_alias() {
assert_eq!(to_orcarouter_slug("auto"), "orcarouter/auto");
assert_eq!(to_orcarouter_slug("Auto"), "orcarouter/auto");
}

#[test]
fn passes_through_explicit_catalog_ids_and_unknowns() {
assert_eq!(to_orcarouter_slug("orcarouter/auto"), "orcarouter/auto");
assert_eq!(
to_orcarouter_slug("anthropic/claude-sonnet-4.6"),
"anthropic/claude-sonnet-4.6"
);
assert_eq!(
to_orcarouter_slug("deepseek/deepseek-chat"),
"deepseek/deepseek-chat"
);
assert_eq!(to_orcarouter_slug("some-local-model"), "some-local-model");
}
}
5 changes: 3 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ A high-level map of the runtime.
- **TypeScript runtime** transpiles `.ts` agents and exposes a deterministic `chidori` host API.
- **Host functions** are the only way agents touch the outside world.
- **Call-log / replay engine** records every host call and replays the journal for deterministic, zero-LLM-call resume.
- **LLM providers** (Anthropic, OpenAI, LiteLLM-compatible, OpenRouter as the `chidori model-login` fallback) are swappable via `reqwest`.
- **LLM providers** (Anthropic, OpenAI, OrcaRouter, LiteLLM-compatible, OpenRouter as the `chidori model-login` fallback) are swappable via `reqwest`.
- **Template engine** uses `minijinja` for Jinja2 prompt templates.
- **HTTP server** (`axum`) powers the `serve` command and session API.

Expand Down Expand Up @@ -60,7 +60,8 @@ chidori/
│ │ │ │ ├── mod.rs # Provider registry, model routing
│ │ │ │ ├── anthropic.rs # Anthropic Messages API
│ │ │ │ ├── openai.rs # OpenAI-compatible (incl. LiteLLM)
│ │ │ │ └── openrouter.rs # OpenRouter OAuth fallback (`chidori model-login`)
│ │ │ │ ├── openrouter.rs # OpenRouter OAuth fallback (`chidori model-login`)
│ │ │ │ └── orcarouter.rs # OrcaRouter routing gateway (`ORCAROUTER_API_KEY`)
│ │ │ └── tools/
│ │ │ └── mod.rs # Tool discovery + JSON schema generation
│ │ └── tests/ # CLI integration tests
Expand Down
7 changes: 5 additions & 2 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,11 @@ verification ([Package Management](./package-management.md)).
### Which model providers work?

Anthropic (`ANTHROPIC_API_KEY`), OpenAI (`OPENAI_API_KEY`, redirectable
via `OPENAI_BASE_URL`), any OpenAI-compatible endpoint — DeepSeek, Groq,
Ollama, vLLM, LiteLLM — via `CHIDORI_OPENAI_COMPAT_URL`, and a zero-setup
via `OPENAI_BASE_URL`), OrcaRouter (`ORCAROUTER_API_KEY` — a named
multi-provider routing gateway that fronts Anthropic, OpenAI, Google,
DeepSeek, and more behind one OpenAI-compatible endpoint), any
OpenAI-compatible endpoint — DeepSeek, Groq, Ollama, vLLM, LiteLLM — via
`CHIDORI_OPENAI_COMPAT_URL`, and a zero-setup
OpenRouter fallback via `chidori model-login`. All can coexist; requests
route by model name. Details:
[Providers & model selection](./host-api.md#providers--model-selection).
Expand Down
1 change: 1 addition & 0 deletions docs/host-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,7 @@ route by model name, first match wins):
| `ANTHROPIC_API_KEY` | Anthropic (`claude-*` models). |
| `OPENAI_API_KEY` | OpenAI; `OPENAI_BASE_URL` redirects it at any OpenAI-compatible endpoint and widens it to match all model names. |
| `CHIDORI_OPENAI_COMPAT_URL` + `CHIDORI_OPENAI_COMPAT_KEY` | Any OpenAI-compatible endpoint (DeepSeek, Groq, Ollama, vLLM, LiteLLM…), matching all model names. `/v1` and bare hosts both work. |
| `ORCAROUTER_API_KEY` | OrcaRouter — a named multi-provider routing gateway ([orcarouter.ai](https://www.orcarouter.ai)) that fronts Anthropic, OpenAI, Google, DeepSeek, and more behind one OpenAI-compatible endpoint (plus `orcarouter/auto` smart routing), matching all model names. |
| `chidori model-login` | Zero-setup OpenRouter fallback. |

The default model for prompts that don't set `model` in code is
Expand Down
4 changes: 2 additions & 2 deletions docs/your-first-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ You need the `chidori` binary ([install](./getting-started.md)) and,
ideally, a provider key exported in your shell:

```bash
export ANTHROPIC_API_KEY=sk-ant-... # or another provider — see below
export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / ORCAROUTER_API_KEY — see below
```

Any Anthropic, OpenAI, or OpenAI-compatible key works
Any Anthropic, OpenAI, OrcaRouter, or OpenAI-compatible key works
([provider setup](./host-api.md#providers--model-selection)), and
`chidori model-login` gives you a zero-setup OpenRouter fallback.

Expand Down