feat(furrow): reach a run's live workspace from anywhere

[AbirAbbas]

A build's workspace is currently only reachable if you happen to be on the same machine as the node. This makes it reachable from anywhere, so the harness that started a run can read the files while the run is still going — including the uncommitted edits, untracked files and dev state a git push would never carry.

Mirroring is opt-in: set SWE_FURROW_ENABLED=1 (or true/yes/on) to turn it on. Unset, 0, or anything unrecognised leaves it off, and the manifest ships the default as 0. A mirror is a byte-exact second copy of the whole build workspace — untracked files and any secrets the coder wrote there included — so an operator has to ask for it and accept the disk it costs.

With it on, nothing else has to know it is there. Availability is discovered, never configured: a build's result carries a workspace_handle when the mirror is working and simply does not carry one when it is not. Every unavailable path — feature off, no binary, no .git, attach or publish failing — degrades to exactly today's behaviour.

What is here

internal/furrow — resolves a vendored furrow binary the same way internal/pro resolves the engine, attaches a build's clone, republishes at each DAG-level boundary and at completion, and keeps a registry mapping the control-plane run ID to the workspace on disk. That mapping is the thing that was missing: SWE-AF knew its workspaces only by a locally generated build ID, so nothing outside the process could name one. An hourly sweeper bounds what keeping them costs, because furrow never prunes a remote on its own:

• Age — SWE_FURROW_TTL_HOURS, default 72. A mirror whose last publish is older is retired.
• Disk — SWE_FURROW_MAX_GB, default 20, covering the client store and every run's remote together. Over budget, the sweeper retires the least recently published mirrors, and Attach refuses to start new ones until there is room. SWE_FURROW_MAX_GB=0 means unlimited — no cap, no budget eviction. Budget eviction never touches a mirror that published within the last hour, so reclaiming space cannot delete a running build's workspace; it degrades to refusing new mirrors instead.

cmd/furrowd + cmd/furrow-dial — the transport. furrowd serves one run's encrypted blob store over TLS by spawning furrow __remote behind a token check; furrow-dial stands in as FURROW_SSH_COMMAND on the caller's side so a stock furrow clone reaches it with no patches to furrow.

Deliberately not sshd: the cloud image runs the control plane and its agent nodes in one container as root, so an ssh key there is whole-box access to every run and every secret on the volume. A furrowd token unlocks a single run's data root, and the payload is ciphertext either way.

get_workspace_handle — returns a run's handle on demand, or {"available": false}. It is registered only on a node where mirroring is actually on.

It performs no per-caller authorization: anything that can reach the node and name a run gets an answer. So the two secrets in a handle are withheld by default and the result says "secrets_redacted": true — the recovery key (which decrypts that workspace) and the transport token (which authenticates to furrowd read-write). What remains is the remote, namespace, handle version and on-node repo path: enough for a caller that already shares the filesystem, and enough to see that a mirror exists. SWE_FURROW_EXPOSE_SECRETS=1 returns them in full, and is appropriate only on a single-tenant cluster where every caller is already trusted with the workspace contents.

Three bugs that only a real binary could find

The unit tests inject a fake exec and assert argv, which pins what we mean to run. These are all things a fake happily accepted:

• furrow snap takes -m; there is no --label.
• The recovery key comes back as key_hex, not key.
• .furrowpolicy lines must be exclude <subtree>. Bare names make furrow reject the file, watch then exits non-zero, and the mirror is silently off for every build with one debug line to show for it — the worst failure mode a feature like this can have.

internal/furrow/integration_test.go now drives the real binary end to end (attach → publish → materialize elsewhere → diff) and skips when furrow is not installed, so the CLI contract is checked rather than assumed. The exact policy bytes are pinned in a unit test too, since CI has no furrow binary.

.furrowpolicy also joins .artifacts/ and .worktrees/ in .git/info/exclude — it lands in the repository the agent is working in, and without that it shows up in git status and any acceptance criterion about a clean tree becomes unsatisfiable.

Verified

• gofmt, go build ./..., go vet ./..., go test -race -count=1 ./... — all green, nothing regressed.
• Real furrow binary, end to end: a workspace with an uncommitted edit, an untracked .env and a file written after attach all materialize byte-exact on a separate store, with a usable .git.
• Real transport, end to end: furrow clone ssh://… through furrow-dial → TLS → furrowd → furrow __remote delivers the workspace; a wrong token is rejected.
• Live node against a real control plane: furrow discovered via sibling-binary resolution, get_workspace_handle registered, and an unknown run answers available:false rather than erroring. (Measured before mirroring became opt-in; a default node now registers 31 reasoners and mounts get_workspace_handle only with SWE_FURROW_ENABLED on.)

Not yet exercised: a full LLM-backed build (needs a provider key in the node's environment) and the cloud deployment (its node is still not_configured).

Notes for review

• Freshness is bounded by publish cadence — level boundaries and completion — not a background watcher. watch --no-daemon is deliberate: no orphan daemons inside a container. A periodic publish while a run is in flight is the obvious next increment.
• The namespace on the wire is furrow's blinded name, so furrowd cannot compare it to the registry's; the token is what scopes a connection, and the namespace is charset-validated and passed through. Comparing them rejected every real clone.
• Attaching is restricted to a workspace whose .git is a real directory. A git worktree's .git is a file pointing outside the tree, so mirroring one would produce a clone with no object database.

Review hardening (second pass)

A follow-up review pass added one commit:

• Honest handles: a public ssh:// handle was minted from FURROW_PUBLIC_ADDR alone, even when furrowd never resolved, failed to bind, or exhausted its restart budget — a valid-looking address nothing listens on. The supervisor now exposes a cached process+TCP health signal and the manager falls back to the run's dir: handle (one warning logged) when it is down.
• Durable stores: store/remotes roots defaulted under the workspace tree — ephemeral in cloud containers, so a restart deleted every published store and the TLS identity. When AGENTFIELD_HOME is set (cloud images set /data) the defaults now live under it; explicit env vars still win.
• Sanitized paths: the remote-store dir was joined from the raw run ID while only the namespace was sanitized; ..-shaped IDs could land — and later be swept with RemoveAll — outside the root. The dir now uses the sanitized namespace, with ./.. survivors collapsing to the fallback name.
• Manifest honesty: bin resolution order corrected (/usr/local/bin first, vendored sibling second), and the client-side vars ssh:// consumers actually need (FURROW_DIAL_TOKEN, FURROW_DIAL_INSECURE, FURROWD_TLS_CERT/KEY) are now declared.

go build ./..., go vet ./..., go test -race -count=1 ./... — all green.

Correctness and trust-boundary pass (third)

A deep review with empirical repros found four real defects. All four are fixed here, each with a regression test that was confirmed to fail against the previous code.

• Empty run ID shared one mirror between builds. build() read the run ID straight off the execution context, where it can be empty, and sanitization turned "" into the namespace run. Two builds that hit that case landed on one registry row, so build B's Attach returned build A's workspace path, recovery key and token. Reproduced. Fixed on both sides: build() and execute() now use the same RunID-else-RootWorkflowID fallback planning.Scout already uses for scoped credentials (which also means the deferred ClearScopedCredentials finally clears the scope Scout wrote), and Manager.Attach refuses an empty run ID outright — it is the registry key and the remote directory name, so an empty one is a key two builds share, not a label one is missing.

• SWE_FURROW_MAX_GB=0 wiped every live mirror, hourly. Three of the four readers already treated 0 as unlimited; Sweep gated its eviction loop on maxBytes >= 0 and so read the no-cap setting as a zero-byte cap — walking the store every tick and retiring mirrors until the registry was empty. The most destructive setting available was the one that asked for no limits. Non-positive now skips budget eviction entirely; age expiry is unaffected.

• Budget eviction deleted mirrors of runs that were still building. retire() documents that it "re-checks staleness under the lock so a run that became active in the meantime is left alone", but the budget path called it with maxAge 0, which disabled that check on the one path where the victim is chosen by size rather than age. retire now takes an explicit eligibility predicate, so "no rule" cannot be expressed by accident, and the budget path requires the candidate's last publish to be both unchanged since it was measured and at least an hour old.

• get_workspace_handle handed recovery keys to any caller with a run ID. See the reasoner section above — redacted by default now, opt-in via SWE_FURROW_EXPOSE_SECRETS.

Alongside them:

• Opt-in, for real. The manifest shipped SWE_FURROW_ENABLED="1", the node built the manager unconditionally, and New() only bailed on the literal string "0" — so this PR's own claim that the default was off was wrong. Default is now 0, the parse accepts 1/true/yes/on as on and treats everything else (including a typo) as off, and get_workspace_handle is gated on mirroring actually being enabled, mirroring the if pro.Available() gate beside it.
• furrowd no longer parses recovery keys. It decoded registry.json into furrow.Entry, whose Key field is the run's recovery key, on every AUTH line — before any authentication succeeded. It now decodes a narrow local struct with only the token, namespace and store dir, so the one network-facing process in this feature never holds a recovery key at all.
• Sweeper deletion is bounded. entry.StoreDir came off disk and went straight to os.RemoveAll. A row naming anything outside the remotes root (or the root itself, or the empty string) now drops the row and deletes nothing.
• furrowd's namespace validation rejects a leading -. The namespace is attacker-supplied text that becomes an argv element of furrow __remote <namespace>; - is in the permitted charset, so a leading one arrives looking like a flag. Refused rather than assuming furrow's parser honours --.
• A furrowd restart loop is no longer silent. The child's stdout and stderr went nowhere, so a daemon that could not bind its port restarted every backoff interval and left one line after five failures naming an exit status, never the reason. Both streams now reach the node's log, prefixed.
• Two doc comments corrected. Attacher claimed callers never need a nil check (a nil *Manager is fine; a nil interface panics, which is why orch.Deps nil-checks it), and Detach claimed to stop mirroring when it only reports whether the run is still known.

Scope trim. This branch also carried an unrelated git checkout -B worktree-reclaim fallback in internal/issue/gitops.go plus the LC_ALL=C pin it needs, and their test file. Nothing in the furrow work depends on internal/issue, so they are reverted to main here. They are worth landing — separately, and with their own argument about whether "git's stderr does not contain a branch named" is a sound signal for "this leftover branch is mine to force-reset", given issue branches carry commits that outlive their build.

gofmt -l . empty, go build ./..., go vet ./..., go test -race -count=1 ./... — all green against the SDK at CI's pinned ref with GOWORK=off. No Python files touched.

[AbirAbbas]

Reviewer note on the transport's threat model, since the client skips certificate verification and that deserves an explicit answer rather than a comment in passing.

What crosses the furrowd connection: the per-run token, then furrow's framed protocol carrying encrypted objects. Object contents, object ids and the workspace name are all encrypted or blinded by furrow before they reach the transport.

What does not: the recovery key. It travels in the workspace_handle inside the reasoner result — over the control plane's own TLS — and is never sent to furrowd. furrowd never has it either; it only shuttles bytes to furrow __remote, which stores ciphertext it cannot read.

So an attacker who fully MITMs the furrowd connection gets a token and a stream of ciphertext. The token lets them pull that one run's objects, which stay unreadable without the recovery key they do not have. They cannot read the workspace, cannot forge a snapshot the real client would accept (objects are BLAKE3-verified against their ids on restore), and cannot reach any other run — the token pins one data root.

That is why the default self-signed certificate is tolerable and why FURROW_DIAL_INSECURE exists. It is not a claim that certificate verification is worthless: an operator who sets FURROWD_TLS_CERT/FURROWD_TLS_KEY to a real certificate gets verification by leaving that variable unset, and that is the better posture on a shared network. The default is chosen so the feature works with no setup, not because the stronger option was unavailable.

The part I would push back on in review is that the token is sent before the client has authenticated the server, so a MITM harvests tokens even though they cannot use them for much. If we want that closed, the options are pinning furrowd's certificate fingerprint in the handle (the handle is already a confidential channel, so it can carry one) or a challenge-response that never puts the token on the wire. Neither is in this PR; the fingerprint would be a small addition if we want it before this is used across untrusted networks.

[AbirAbbas]

Two corrections to my threat-model comment above. Both were wrong in the direction of making the transport look safer than it was.

1. "furrowd never has it either" — the recovery key — was false. lookupEntry decoded registry.json into furrow.Entry, and Entry.Key is the run's recovery key. That decode ran on every AUTH line, before any authentication succeeded, so an unauthenticated connection was enough to make the network-facing daemon load every recovery key on the node into its address space. The key was never sent to furrowd over the wire, which is what I was thinking of — but it was already sitting on the same disk furrowd reads, and furrowd was reading all of it.

Fixed in this push: furrowd now decodes a narrow local struct carrying only the token, namespace and store dir. The key is not a field there, so it is never parsed and never resident.

2. "The token lets them pull that one run's objects" understated it. The token authenticates to furrow __remote, which is the remote server side — push as well as pull, and object deletion. So a harvested token buys write access to that run's remote, not just a read of it.

The confidentiality half of the original statement stands: the objects are ciphertext, the attacker still has no recovery key, so they cannot read the workspace. The integrity and availability half does not. A token holder can push objects into the run's store or delete what is there. Restore-side BLAKE3 verification means they cannot make the real client accept forged content — a tampered object fails its id check — but they can corrupt or empty the store, which turns a clone into a failure rather than a lie. Availability, not confidentiality, is the exposure.

Neither changes the conclusion about certificate pinning: the token is still sent before the client has authenticated the server, so a MITM still harvests it. That is now worth a bit more to the attacker than I said, which strengthens the case for putting furrowd's certificate fingerprint in the handle before this is used across an untrusted network. Still not in this PR.

Separately, and related to who can obtain a token in the first place: get_workspace_handle was returning both the recovery key and the token to any caller that could reach the node and name a run, with no authorization anywhere on that path. Both are now redacted by default ("secrets_redacted": true), with SWE_FURROW_EXPOSE_SECRETS=1 as the single-tenant opt-in.