Skip to content

Add programmatic deep-research harness - #76

Merged
xqlin98 merged 1 commit into
mainfrom
feat/deep-research-harness
Oct 8, 2026
Merged

xqlin98 merged 1 commit into
mainfrom
feat/deep-research-harness

Conversation

@xqlin98

@xqlin98 xqlin98 commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Models can now compose web searches, page reads, and summaries in a single run_research_program MCP call. This adds a research-primitives server using the existing function-calling agent, with provider clients separated from program execution, a reusable research prompt, and a portable benchmark example and CLI.

Each program executes in a fresh subprocess with a parent wall-clock limit and bounded returned output. The documentation explains the AST restrictions and their limits. The CLI persists traces and evaluator results and supports resuming completed tasks. A small benchmark-runner fix keeps cached results in the same shape as newly evaluated results; the end-to-end test exercises both paths. Two existing cache-quota comparisons are simplified without changing behavior so the Pylint version installed by CI can proceed to the tests.

This is one commit based on current upstream main, scoped to the deep-research/PTC harness. Training, internal experiment configurations, and EnterpriseOps are outside this PR.

Validation:

  • GitHub CI passes on Python 3.10 and 3.13, including full-package lint and the test suite. SAST and credential scanning also pass.
  • 34 local tests passed: 21 harness tests covering mocked provider recovery, real subprocess timeouts, real MCP execution with a scripted model, and benchmark resume; plus 13 existing environment-pool manager tests.
  • Full-package Pylint 4.1.2 passes (10.00/10), matching the version installed by GitHub CI.
  • Wheel build succeeded; an installed-wheel smoke test outside the checkout executed a research program and verified the packaged prompt.
  • External provider calls and live benchmark accuracy runs were not performed; the provider tests use mocks.

@xqlin98
xqlin98 force-pushed the feat/deep-research-harness branch from 09b1400 to b840cb5 Compare October 8, 2026 01:10
@xqlin98
xqlin98 requested a review from StevenShi-23 October 8, 2026 01:28
@xqlin98
xqlin98 marked this pull request as ready for review October 8, 2026 01:28
@xqlin98
xqlin98 merged commit 0df6c42 into main Oct 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant