README.md

CodeyBox documentation

CodeyBox runs coding agents in throwaway VMs, reviews what they produce, and merges it. These pages describe how that works and how to run it.

New here? getting-started.md takes you from a clean host to a merged change. Then read concepts/architecture.md for the shape of the system, and concepts/security.md before you point it at anything that matters.

Concepts — how it works

Page	What it covers
architecture	components, trust boundaries, the state machine, plugin points
pipeline	what work, audit, rework, merge and push do at the git level
work items	dependencies, cancellation, check-and-act, refactor items, replay
projects	per-project repository, auditors, upstream, budgets, credentials
agents	the agent CLI contract, the built-in eight, adding a ninth
agent classes	routing across agents, quality scores, quota-aware fallback
sandboxes	Incus, Multipass, remote Multipass, Sprites, Bubblewrap, Process
security	threat model, mitigations, sharp edges, known gaps
questions and suggestions	how an agent asks you something, or flags adjacent work

Quality — the gate before merge

Page	What it covers
audit	the audit phase, capability-grouped sandboxes, the rework loop
audit reports	how findings are stored, queried, and de-duplicated
presets	language detection and audit-type prompts, and who may override them
mutation rigor	the per-item gate that checks tests actually catch bugs
test cases	test cases as a first-class artifact on a work item
E2E execution	replaying committed E2E artifacts on cheap CPU-only VMs

Operating — running it

Page	What it covers
running	host setup, starting the service, the failure modes you will meet
host firewall	host-side nftables egress enforcement, profiles, troubleshooting
worker pool	concurrency sizing, stuck-agent detection, queue and per-agent pause
recovery	crash recovery, where each state resumes, the restart window
agent-turn checkpoints	resuming a partial agent turn — the deep end of recovery
sandbox reliability	leaked VMs, smoke probes, surviving suspend/resume
quota	probes, floors, the burn gate, the observed-failure breaker
costs	what each run cost, and where the rates come from
spend limits	per-agent budgets and per-project budget alerts
logging	the structured audit log: files, common properties, event names
observability	OpenTelemetry traces, metrics, Prometheus scrape
pipeline metrics	per-step timings and the transition-health score
agent streams	capturing agent stdout as NDJSON, and what the analyser derives
supervision	watching and injecting into a live agent session
releases	release branches with a deep audit, and changelog automation

Reference — look it up

Page	What it covers
configuration	every CodeyBox:* key, defaults, hot-reload behaviour
API	REST endpoints, auth, the work-item record, SignalR
webhooks	outbound events, payloads, HMAC signing, delivery semantics
events	the versioned event envelope and its evolution rules
CLI	the codeybox client and the project-config wizard
knobs	per-item directives and how to add one
external IDs	addressing work items by your tracker's identifier
agent quirks	per-CLI binary names, auth layouts, flags, traps
agent sessions	the opt-in session contract and the Claude session worker
sandbox baselines	bake recipes for C#, Python, Node, Go, Rust, agent CLIs, security tools

Extending — plugins

Page	What it covers
plugin SDK	the plugin contract, allowlist, API-version rules, threat model
auditor plugins	shipping a custom auditor
upstream plugins	shipping a forge integration
credential plugins	shipping a credential provider
sandbox provider plugins	shipping a sandbox backend and its trust model
statistics plugin	the bundled quota-history and capacity plugin
file-size-limits auditor	a small worked example of a deterministic auditor

Developing CodeyBox itself

Page	What it covers
build environment	provisioning prerequisites that break the build when missing
manual UAT	operator checklists for what automated tests cannot cover
AGENTS.md	the engineering contract every change is graded against

Evaluations — spikes

Page	What it covers
caveman evaluation	spike: evaluating the caveman output-compression skill as a token-saving layer over existing agents — capture/parse compatibility gate, measured savings, per-agent opt-in design

The other clients

The admin dashboard (tools/CodeyBox.Admin/) is a Blazor Server UI — queue, diffs, findings, cost and timing charts:

dotnet run --project tools/CodeyBox.Admin/src/CodeyBox.Admin.Web

The CLI (tools/CodeyBox.Cli/) is a typed client for the same API:

codeybox configure
codeybox queue add --project myapp --title "healthz" --prompt-file ./prompt.md
codeybox queue ls --state Queued,Working
codeybox queue watch <id>

Both talk to the orchestrator over REST only, and share no code with it.