Skip to content

Preserve Pydantic AI context across validator retries - #511

Open
hertznsk wants to merge 8 commits into
microsoft:mainfrom
hertznsk:fix/pydantic-validator-session
Open

Preserve Pydantic AI context across validator retries#511
hertznsk wants to merge 8 commits into
microsoft:mainfrom
hertznsk:fix/pydantic-validator-session

Conversation

@hertznsk

@hertznsk hertznsk commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Preserve Pydantic AI context across validator retries

Summary

  • preserve the completed Pydantic AI message history when a semantic validator rejects Claude or OpenAI output
  • send validation feedback as the next user turn instead of rebuilding the original prompt from scratch
  • keep existing stateless validator retry behavior for providers that do not expose continuation state

Why

Validator retries previously called the primary agent again with a reconstructed prompt. For the Pydantic AI providers, that discarded the model response and any tool-call/tool-result exchanges from the first attempt. The correction now uses Pydantic AI message history so the model can revise its existing work with the validator feedback in context.

Testing

  • make check
  • uv run pytest tests/test_engine/test_validator_integration.py tests/test_providers/test_pydantic_ai_interrupt.py tests/test_providers/test_pydantic_ai_provider.py tests/test_providers/test_openai.py -q
  • make test (8610 passed, 32 skipped; 4 unrelated environment/performance failures: two POSIX unreadable-path assertions and the existing Claude performance thresholds)

Notes

  • continuation state is provider-opaque and in-memory only; workflow checkpoints and serialized outputs are unchanged
  • Pydantic AI receives the prior all_messages() as message_history and the validator feedback as a new user prompt
  • external providers are not passed the new keyword unless continuation state exists
  • AGENTS.md validator sections updated to describe the continuation behavior; a fail-open regression test covers a failing continuation re-run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant