Skip to content

fix(agent): price a turn's progress instead of counting its rounds - #8404

Open
esengine wants to merge 1 commit into
main-v2from
fix/progress-guard-false-stop
Open

fix(agent): price a turn's progress instead of counting its rounds#8404
esengine wants to merge 1 commit into
main-v2from
fix/progress-guard-false-stop

Conversation

@esengine

@esengine esengine commented Aug 11, 2026

Copy link
Copy Markdown
Owner

The no-progress guard was stopping honest investigation. Measured on the real
tool loop before this change:

turn shape (every round asked something new) old behaviour
grep one package, ten different patterns told "produced no new evidence", ordered to answer at round 7
page through one long file (10 windows) same, round 7
read a file never read before, every round same, round 12
reads interleaved with go test never fired (control)

Two causes. Read novelty was keyed on the path alone, so a new grep pattern over
an already-searched package and the next window of an already-opened file both
scored as repeats from the second round on. And a hard exploration cutoff zeroed
every gain after six look-only rounds, which made a read-only turn reach the
stop tier by arithmetic.

Fixed thresholds were the wrong shape for the job. They fire at a particular
round however well the turn had been going, and a single incidental new read
resets them to zero — so they cut off real investigation while a loop that
touches something new every third round slips past.

What replaces them

One per-turn account (internal/evidence/runway.go). Every round costs the same
and what it produced pays part of that back: an observation that could have
refuted the plan buys rounds of slack, a landed change covers its own round, a
round that found something new costs a quarter of one that found nothing. The
tuned quantities are in look-only rounds — what a fresh turn opens with (24) and
the most a productive one banks (40).

  • a turn that keeps verifying its work never runs out (36 rounds, untouched)
  • a turn that keeps landing changes never runs out either; that those changes
    still owe verification is real, and already owned by the EBM trigger — pricing
    it here too would stop working turns
  • a turn that keeps repeating itself is stopped in six, as before
  • there is no cliff at any particular round number

Signals reach the model through the round arbiter added in #8257's line of work:
the runway raises verdictAdvise while the balance drains and verdictLand on
the round that empties it, and applyInterventions folds it into the single
round tail alongside the storm breaker and the evidence nudge. One scoring pass
now serves them all — the runway, the EBM trigger, the reasoning governor and
the trajectory record read the same sample.

The host also stops giving orders. While the balance drains it states what it
measured — dry rounds, rounds without acting, budget left at this rate — and on
the single round that empties the account it states that it has stopped
requiring further receipts. The model decides what to do with either. Earning
the balance back out of the low band closes the episode, so a turn that recovers
and later stalls again gets its own statement and stand-down.

This also collapses the duplicate round scorers: ProgressTracker and its
LegacyGain shadow are gone and OutcomeTracker is the turn's only scorer,
feeding the runway, the EBM trigger, the reasoning governor and the trajectory
record from one pass. cmd/e2ebench sourced two metrics from legacy_gain, so
ProgressRounds / FalseProgressRounds now derive from the decomposition
instead; bookkeeping rounds, which the retired scorer credited, correctly no
longer claim progress.

Every scored round stamps the balance, the dry and idle counts and the spent
transition onto its sample, so runway, runway_dry, runway_idle and
runway_spent land in the trajectory. Without them a pause cannot be explained
after the fact and the prices below stay guesswork; with them the next
adjustment can be argued from recorded turns.

What this does not fix

The scorer is unified; the policies are not. Fixed counts still govern the storm
breaker (3 identical host failures), the todo progress lease (8 then 16 rounds)
and recovery.MaxEpisodeFailures (6). They are the next candidates for the same
account, and this PR should not be read as having retired thresholds generally.

The account is also anchored on rounds, not on spend: every round costs the same
whether it was a 200-token grep or a 40k-token read. Spend is the thing actually
worth bounding, but charging by tokens makes a long, productive turn drain faster
purely because its prompt grew, so it needs the recorded balances above to
calibrate against before it is worth doing.

Measured on real runs, not only tests

Two real reasonix run --trajectory sessions against this repo, read back from
the recorded balances (they need #8411, without which no audit record reaches a
trajectory at all).

A research turn — 14 rounds reading and searching one package:

rd  expl churn disc  runway dry idle
 1     2     0    0      23   0    1
 …
14     1     0    0       …   0   14

dry is 0 on every round: fourteen rounds of grepping and paging one package,
and not one was miscounted as a repeat. That is the fix working on real
traffic — the same shape used to be told it had produced no new evidence and
ordered to answer at round 7.

It also falsified the first pricing I shipped here. At 12 look-only rounds the
account ran out at round 12 of a turn that was still learning something every
single round, which in Goal mode would have paused a productive investigation —
the same false stop this PR exists to remove, at a new number. Looking now costs
a quarter of a round that finds nothing, so a research turn gets 24 rounds while
a repeating one is still gone in six.

A working turn — fix two functions, run the test:

rd  expl churn disc  runway
 1     2     0    0      23     read
 3     0     1    0      23     the edit: covers its own round
 4     1     1    1      31     edit + test run: refills

Zero interventions, balance never fell. That half of the pricing holds.

Verification

Regression tests cover the four turn shapes above, the economics themselves, and
the openings that must never be punished: a first round that fails, a failure
followed by recovery, and short turns re-reading one path. An all-failing turn
still pauses on the storm breaker, not the runway.

Root module: every package run, green except two failures that reproduce
identically on unmodified main-v2internal/cli
TestClearMCPAuthenticationUsesControllerWorkspace (unknown kind "anthropic")
and cmd/e2ebench
TestNoSolutionCorpusGradesTheInverseContract/nosol-authoritative-wrong-test.
Both predate this branch and are tracked separately. The desktop module builds,
vets and tests clean. gofmt, go vet ./..., repolint and golangci-lint at the
pinned version are all clean.

Cache-impact: none - the only agent.go change is a struct field comment, and the host observation rides tool results in the turn tail, so the system-prompt prefix stays byte-identical.
Cache-guard: go test ./internal/boot/ (provider_request golden) and internal/agent/cachehit_e2e_test.go, both unchanged and green.
Documentation-impact: updated - docs/SPEC.md, docs/GUIDE.md, docs/GUIDE.zh-CN.md and docs/GOAL_ENFORCEMENT.zh-CN.md now describe the runway account instead of the retired round thresholds.

@esengine
esengine requested a review from SivanCola as a code owner August 11, 2026 13:04
@github-actions github-actions Bot added v2 Go rewrite (1.x) — main-v2 branch, active development agent Core agent loop (internal/agent, internal/control) labels Aug 11, 2026
@esengine
esengine force-pushed the fix/progress-guard-false-stop branch from 466ab73 to 96e2e77 Compare August 11, 2026 13:51
@esengine
esengine force-pushed the fix/progress-guard-false-stop branch from 96e2e77 to c9902dd Compare August 11, 2026 14:38
The no-progress guard fired on honest investigation. It scored a round's
novelty by path alone, so the two most common research moves — grepping one
package for a second symbol, paging through a long file — scored as repeats
from the second round on, and a hard exploration cutoff zeroed every gain
after six look-only rounds. Measured on the real loop: a turn grepping one
package with ten different patterns was told it had "produced no new evidence"
and was ordered to answer at round 7; paging one file, round 7; reading a new
file every round, round 12.

Fixed thresholds were the wrong shape for the job. They fire at a particular
round however well the turn had been going, and a single incidental new read
resets them to zero — so they cut off real investigation while a loop that
touches something new every third round slips past.

Replace the ladders with one per-turn account. Every round costs the same and
what it produced pays part of that back: an observation that could have refuted
the plan buys rounds of slack, new information or an unverified change buys
back half a round, a round that produced nothing pays full price. A turn that
keeps verifying its work never runs out; one that keeps repeating itself is
stopped in six rounds; and there is no cliff at any particular round number.

The host also stops giving orders. While the balance drains it states what it
measured — dry rounds, rounds without acting, budget left at this rate — and
on the single round that empties the account it states that it has stopped
requiring further receipts. The model decides what to do with either.

This also collapses the duplicate round scorers: ProgressTracker and its
LegacyGain shadow are gone, and OutcomeTracker is the turn's only scorer,
feeding the runway, the EBM trigger, the reasoning governor and the
trajectory record from one pass.
@esengine
esengine force-pushed the fix/progress-guard-false-stop branch from c9902dd to 43b2d5c Compare August 11, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Core agent loop (internal/agent, internal/control) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant