fix(agent): hand a forced turn back the tools compaction emptied, once - #2148
Open
AlexLiu190625 wants to merge 7 commits into
Open
fix(agent): hand a forced turn back the tools compaction emptied, once#2148AlexLiu190625 wants to merge 7 commits into
AlexLiu190625 wants to merge 7 commits into
Conversation
A ReAct turn forced back to final_answer alone can lose the tool observations that answer was meant to rest on, because the same turn's compaction removes them after the decision to answer was already made. The turn still asked for an answer "using the accumulated conversation and tool results", which on that turn is a request to invent them. The forced instruction now has a second form. When this turn's compaction destroyed tool evidence, it names the removed observations, forbids reconstructing, estimating or illustrating a removed value, and withdraws outcome=completed for the turn. The ordinary wording is replaced rather than appended to, so the two cannot contradict each other. Choosing between the two forms needs to know why the turn was forced, so every site that forces one now records its reason alongside the flag, and the reason, the recovery budget and the follow-up marker all travel through checkpoints. Their reload defaults differ on purpose: an unreadable reason or marker does less, while a missing recovery counter reads as unspent so runs checkpointed before it existed keep the path. Refusing to restore tools is recorded in pattern state as well, so an interrupt between the refusal and the answer resumes into a turn that still remembers to be honest.
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to handle forced-answer turns in the ReAct reasoning loop when the underlying tool evidence has been removed by context compaction. It defines explicit reasons for forced answers, tracks compaction recovery attempts within a budget, and either temporarily restores the dropped tools for a single recovery turn or instructs the LLM to honestly report the missing evidence rather than hallucinating or estimating it. Additionally, it updates state serialization to support these new fields across checkpoints and adds a comprehensive test suite to validate the recovery and fallback behaviors. As there are no review comments, no feedback is provided.
…-dropped turn The forced-answer gate decided, per turn, whether to send the honest "evidence was dropped" instruction via a loop-local variable, but only persisted that decision to the recovery-followup marker on the branch where recovery was declined. A turn where the gate found dropped evidence but recovery had not yet been declined left the marker unset, so an interrupt between the decision and the LLM call resumed into a forced turn that had forgotten the evidence was ever dropped. The marker write now happens at the gate itself, alongside the loop-local it mirrors, so both halves of the same decision are set together regardless of which branch runs afterward. The decline branch no longer duplicates the write.
When compaction destroys the observations a forced turn was going to answer from, saying so is the fallback, not the goal. If the run can still afford it, the better move is to undo the forcing for one turn and hand back exactly the tools whose results went missing, plus final_answer so the model can still finish. Handing tools back is bounded on every side. It happens at most once per pattern, never for a forcing that means "stop and report blocked", never for the turn a previous recovery already produced, and never without two iterations left -- one to re-fetch and one to answer -- so recovering can never cost the run its answer. Tools that lost nothing stay out, and so do the two that talk to the user rather than read anything. A system message names the missing observations, states that fetching them again outranks the standing advice against repeating tool work, limits re-runs to reading tools, and withholds the final answer until the values are back or reported missing. Its tool-name list mirrors the bounds the compaction notice already applies. Whether the re-fetch worked is read from the turn's own tool call ids rather than tool names. The dropped observation was itself a successful call of one of those names and its ledger entry outlives the compaction, so a name match would always claim the evidence came back. The verdict decides which instruction the following turn gets.
A tool-protocol repair rebuilds the whole request from scratch, and on a turn whose evidence was destroyed it is the call that actually produces the user's answer. It was rebuilding the prompt without knowing that, handing back the same "answer from the accumulated results" sentence the first call had replaced, so the repair undid the honest wording. The repair path now receives that fact along with the tool set the turn is entitled to. Ordinary and forced turns are unchanged: the repair still hands back the run's whole tool set, and its wording is byte-identical. A turn that narrowed its own tools gets that set minus the two tools that contact the user rather than read anything, and the "re-decide with the complete tool set" line becomes an accurate one, since the repair set is wider than what the turn first offered but still not everything the run has. Keeping the repair set wider than the narrowed one is deliberate. A model that needs some other tool to reach the dropped values would otherwise name it twice and fail the run outright, which the narrowed turn made reachable and no earlier behaviour did.
A recovery turn undoes the forcing and hands back the tools whose observations compaction dropped. The marker that carries the same decision across a checkpoint was left behind, so a pause inside a recovery turn resumed into a forced answer that reported the evidence unavailable instead of continuing to fetch it. Both halves of the decision are now undone together, before the recovery checkpoint that would otherwise persist the stale marker. The resume case for a recovery turn had no coverage, so the test that paused a cleared turn now asserts the recovery half rather than the refusal it used to assert on a build that could not recover.
AlexLiu190625
force-pushed
the
fix/react-forced-answer-recovery
branch
from
September 6, 2026 17:44
d9cb40f to
af4198f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on
fix/react-forced-answer-honest-instruction(#2147); merge that one first. Until that PR merges, this diff also shows its commits.Saying that the evidence is gone is the fallback, not the goal. When this turn's
compaction destroys the observations a forced turn was going to answer from, and
the run can still afford it, this change undoes the forcing for exactly one turn
and hands back only the reading tools whose results went missing, plus
final_answerso the model can still finish. The turn after that one is alwaysthe forced answer turn: if the re-fetch brought the values back it gets the
ordinary instruction, and if it did not it gets the honest one. A tool-protocol
repair inside the same turn now rebuilds the request from that turn's own tool
set and keeps the honest wording instead of restoring the sentence the first call
had already replaced.
Behavior changes
for one turn, and receives the dropped reading tools plus
final_answer.that contact the user rather than read anything.
fetching them again outranks the standing advice against repeating tool work,
limits re-runs to reading tools, and withholds the final answer until the
values are back or reported missing.
final_answer, with the instruction chosen by whether the evidence actuallyreturned.
from tool names. The dropped observation was itself a successful call of one of
those names and its ledger entry outlives the compaction, so a name match would
always claim the evidence came back.
fact that evidence was dropped. Ordinary and forced turns are unchanged: the
repair still offers the run's whole tool set and its wording is byte-identical.
A turn that narrowed its own tools gets that narrowed set widened only by the
tools it is still entitled to, and the "re-decide with the complete tool set"
line becomes accurate for it.
test_a_recovery_turn_resumes_into_a_recovery_turnfails without that clearing line.Bounds and residual risk
checkpoint, so a pause and resume cannot buy a second one.
so recovering can never cost the run its answer; a run with room for a single
call therefore never recovers.
turn a previous recovery already produced, which is what stops recovery from
chaining into itself.
can contain a tool that writes as well as reads. The limit to reading is stated
in the prompt, not enforced by the tool layer. Marking tools read-only in the
tool protocol, so this set can be filtered rather than described, is the
follow-up that closes it.
model that needs some other tool to reach the dropped values would otherwise
name it twice and fail the run outright.
Not in this PR
which land in the branch this one stacks on.
strict xfail test carried by the first branch still records that gap.
Verification
tests/core/agent/test_react.py,tests/core/agent/test_grounding.pyandtests/core/agent/test_react_forced_answer_compaction.py: 314 passed,1 xfailed. This branch adds 38 cases to that file on top of the branch it
stacks on, which alone runs 39 passed and 1 xfailed there.
re-fetch verdict match by tool name instead of the turn's own call ids, giving
the recovery turn the run's whole tool set, dropping the recovery reason from
the set exempt from recovery, clearing the follow-up marker as soon as it is
read, folding every control tool into the restored set, rebuilding the repair
request without the dropped-evidence fact, and rebuilding it from the run's
whole tool set. Each turned red, caught by
test_an_older_successful_call_does_not_pass_for_a_recovered_one,test_recovery_hands_back_the_tools_whose_results_were_dropped,test_the_recovery_budget_is_spent_once_per_pattern, the recovery-chaintests,
test_the_retry_that_produces_the_answer_is_honest_tooandtest_the_protocol_retry_uses_this_turns_own_tool_set.test_an_evidence_loss_still_ends_the_run_with_an_answerwalks the outcomematrix and asserts every one of them ends the run with an answer and never at
the iteration ceiling.
pre-commit run --fileson both changed files: ruff check, ruff format, mypy,isort, codespell and the whitespace hooks all pass.
Part of #2146