ci(governance): rearm the OpenAPI workflow's route/SDK exit integrity [Tier 4] - #9719
Merged
Conversation
… [Tier 4] VAL-CDG-012: the OpenAPI workflow's authoritative validation steps must propagate failure, and their verdicts must be method-aware and bound to the exact execution SHA. Workflow rearm (.github/workflows/openapi.yml): - Bind the route authority to the execution's exact resolved commit SHA: run scripts/validate_openapi_routes.py with --ref set to the resolved HEAD (method-aware VAL-CDG-011 operation plane) and require the plane to echo the same binding back, before the baseline-gated coverage check. - Remove the unconditional-success guard from the contract-matrix pytest step: its exit code is now the verdict. - Arm pipefail in the operationId audit pipe so a crashed producer fails the step instead of vanishing into the pipe. - Make both drift-summary steps require parseable CURRENT-run inputs (route-coverage.json, openapi_generated.json, contract-drift-summary.json) before presenting any verdict. - Justify the two remaining continue-on-error seams in place: the setup-python runner-resilience seam (closed fail-closed by the next step) and the PR-lane best-effort Live types twin (strict twin covers push/schedule/dispatch). Contract tests (tests/scripts/test_openapi_workflow_contract.py): the 12 VAL-CDG-012 named tests: census (single active workflow, single live check-name emitter), triggers, method-aware route/SDK authorities, reference selection rules (unfiltered pagination newest-by (run_started_at, run_id, run_attempt), attempt-specific jobs/checks, run-level artifacts, movement-restart), failure propagation (behavioral bash -e simulation of the real run blocks), pipefail census, summary-input preconditions, gate aggregator exhaustive case table, and conclusion-rewrite exclusion. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Contributor
|
OpenAPI Spec Update The OpenAPI specification has changed. Please review the generated spec in the workflow artifacts. |
Collaborator
Author
Claude independent model reviewReviewer: claude (anthropic) — independent adversarial model review via the Aragora Claude reviewer, grounded on the exact PR head. Verdict: PASS Verification performed (all at head d773453 in this worktree):
Findings:
dogfood: yes |
Collaborator
Author
OpenAI independent model reviewReviewer: openai (openai) — independent adversarial model review via Codex CLI OpenAI harness, grounded on the exact PR head. Verdict: PASS
dogfood: yes |
scarmani
marked this pull request as ready for review
August 6, 2026 20:31
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VAL-CDG-012: rearm the OpenAPI workflow's route/SDK exit integrity [Tier 4]
Milestone
cdg-bootstrap-route-truth, featurecdg-openapi-rearm-impl. Sequenced after merged #9672 (fail-closed aggregators, base of this branch) and merged #9717 (method-aware route-core plane,9148ba2934). Prepared under the operator ledger line-388 narrow overlap lift: this PR'sopenapi.ymlhunks are disjoint from open #9690's single setup-node hunk (L169–176) and do not touch theSet up Node.jsstep.Workflow rearm (
.github/workflows/openapi.yml)Validate handler route coveragenow first runsscripts/validate_openapi_routes.py --ref "$(git rev-parse HEAD)"(the VAL-CDG-011 operation plane, exact-40-hex enforced by_require_exact_ref) and refuses to continue unless the plane echoes the identical SHA back; the baseline-gated--fail-on-missingcoverage check then runs unchanged as the drift ratchet. Verified locally: plane binds and echoes622fc3df8f…, 652 served operations, step exit 0; a failing authority propagates through the armed pipefail.|| truethat discarded its exit code is removed (live main runs showed1 passedunder the guard — the guard was masking, not carrying, the verdict).set -o pipefailbefore the producer|consumer pipe: the count stays advisory, but a crashed producer now fails the step.if: always()drift-summary steps require parseable current-run inputs (/tmp/route-coverage.json+ generated spec; then/tmp/contract-drift-summary.json) before invoking their generators. Missing or truncated inputs fail the step (verified in both directions).continue-on-errorcensus — remove-or-justify resolved as justify-in-place (2 seams), both now annotated in the workflow:Set up Python 3.11(L124): runner-resilience seam only; the immediately followingEnsure python toolchain is availablestep re-verifies the interpreter fail-closed (set -euo pipefail,exit 1), so a masked setup failure cannot reach any validation step.Generate Live API types (best-effort on PR)(L329): PR-lane frontend-types twin, not a route/SDK authority; the strict twin runs fail-closed on every push/schedule/dispatch execution.Comment on PRandsync-jobCommit updated specare notification/manual-dispatch-publish paths, outside the validation plane; censused by the tests.)|| trueguards kept (checkout self-repair, ensurepip recovery re-verified fail-closed, npm cache ownership, sync publish path) — censused and bounded by the contract tests so no authoritative step can ever join that set.SDK-authority question — resolved: current extraction already satisfies method-awareness
VAL-CDG-012 requires the workflow to invoke "method-aware route and SDK authorities". For the SDK plane, no new ref-bound mode is needed:
scripts/verify_sdk_contracts.pyextracts method-bearing(method, path)operation pairs from both SDKs via(?P<method>…)named-group regexes and compares them against the spec's per-method operations — the comparison is method-aware by construction, and--strict --baselinemakes it fail-closed against its pinned baseline. Same forcheck_sdk_namespace_parity.py --strict --baseline. Exact-SHA binding for the execution is provided once by the route plane's--refecho in the same job on the same checkout; duplicating a ref-bound mode inside the SDK scripts would add surface without adding evidence. The contract test pins the method-bearing extraction ((?P<method>present) so silent regression to path-only comparison fails CI.The 12 VAL-CDG-012 contract tests (
tests/scripts/test_openapi_workflow_contract.py)Census (exactly one active OpenAPI workflow, live workflow ID 226588000; single emitter of the live
Generate & Validatecheck name), trigger topology, method-aware authorities, reference selection rules mirrored from the CDG helpers (unfiltered per_page=100 pagination newest-by(run_started_at, run_id, run_attempt); attempt-specific jobs/check URLs; run-level SHA-bound artifacts; movement-in-(main, run, attempt, conclusion)restart), behavioral failure-propagation simulations of the real run blocks underbash -ewith command stubs (route authority masked-by-tee, contract-matrix exit, SDK verify), pipefail-before-first-pipe census, summary-input preconditions (missing/truncated inputs fail), exhaustive 12-case gate aggregator table, and conclusion-rewrite exclusion (gate reads onlyneeds.*; summaries/uploads/comment cannot write checks, statuses, or job outputs).Verification
pytest tests/scripts/test_openapi_workflow_contract.pypytest tests/scripts/test_validate_openapi_routes.py tests/ci/test_required_aggregator_fail_closed.py tests/server/openapi/test_contract_matrix.py+ contract fileValidate handler route coveragerun block (bash -e)make lint,actionlint openapi.yml, pre-commit hooksRed-first: the contract test file did not exist at base (pytest exit 4 captured before implementation).
Tier 4
Draft +
operator-review-required. Touches a required-check workflow (Generate & Validate); merge only via exact-head operator settlement (scripts/settle_tier4_pr.py). Base622fc3df8f87b23dc4d499c79b5638842b0fbd65(merged #9672).