fix(server): pre-populate storyboard-declared test kits on hosted storyboard runs - #6756
Open
fgranata wants to merge 1 commit into
Open
fix(server): pre-populate storyboard-declared test kits on hosted storyboard runs#6756fgranata wants to merge 1 commit into
fgranata wants to merge 1 commit into
Conversation
…ryboard runs The hosted engine's withHostedAuthTestKit substituted the graded run's own bearer into test_kit.auth.api_key, so from_test_kit steps ran as the seller's normal principal instead of the storyboard-declared kit credential - comply_controller_mode_gate failed deterministically against conformant sellers (NOT_FOUND instead of the contracted FORBIDDEN; adcp#6735, 7-of-8 hosted runs in the issue's audit-log evidence). Per the triage brief: resolve the declared kit via getTestKitForStoryboard and pre-populate options.test_kit at all four withHostedStoryboardRunOptions call sites (registry-api step run, conformance-socket run, Addie run_storyboard + run_storyboard_step). The existing !nextAuth.api_key guard in withHostedAuthTestKit then correctly no-ops the bearer substitution when the storyboard carried its own kit; storyboards that declare no kit keep the substitution. withHostedComplianceRunOptions (suite-wide comply()) untouched. Tests: getTestKitForStoryboard resolves the mode-gate kit WITH its credential; the substitution guard holds over a pre-populated declared kit. Docs: prerequisites.test_kit documented as a runner loading directive (storyboard-troubleshooting).
Contributor
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — server-only auth fix, no blocking or medium findings.
What I checked:
- Scope is server-only (
server/**+ one troubleshooting doc). No touch tostatic/schemas/source/**,docs/reference/**,dist/**, task definitions, or any published protocol surface — so no changeset is required, and no schema↔docs / oneOf / immutable-artifact concerns apply. - The fix pre-populates the storyboard-declared test kit into
options.test_kitat all four hostedwithHostedStoryboardRunOptionscall sites, sofrom_test_kitsteps authenticate with the storyboard's declared credential rather than the seller's run bearer. Mechanism relies on the existing!nextAuth.api_keyguard inwithHostedAuthTestKit, which no-ops the substitution — that load-bearing guard is covered by a test. - No REST self-report-score route (certification remains Addie-gated). No production migration changes.
High-risk flag: true only because server/src/addie/** and server/src/routes/** were modified. Both are (modified), not deleted, and the reviewer surfaced no medium-or-higher concerns — presumed contract-preserving. Not escalation-worthy on the flag alone.
Gate check: gated_paths is false, so row 2 does not apply despite review_decision: REVIEW_REQUIRED. No no-auto-approve team match. No prior decision. Zero findings → rows 1–8 do not fire → row 9 approve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Server-side half of #6735 (P0), built to the triage brief and the maintainer go-ahead in the issue thread.
withHostedAuthTestKitsubstitutes the graded run's own bearer intotest_kit.auth.api_keywhen no kit is configured — so on hosted runs,from_test_kitsteps authenticate as the seller's normal principal instead of the storyboard-declared kit credential.comply_controller_mode_gatethen fails deterministically against conformant sellers (NOT_FOUNDinstead of the contractedFORBIDDEN; 7 of 8 hosted runs in the issue's audit-log evidence).Change (per the brief)
getTestKitForStoryboardand pre-populateoptions.test_kitat all fourwithHostedStoryboardRunOptionscall sites:registry-api.tsstep run,run-storyboard-via-ws.tsconformance-socket run, and both Addiemember-tools.tssites (run_storyboard,run_storyboard_step).!nextAuth.api_keyguard inwithHostedAuthTestKitthen correctly no-ops the bearer substitution when the storyboard carried its own kit. Storyboards that declare no kit keep the substitution — its legitimate purpose.withHostedComplianceRunOptions(suite-widecomply()) untouched, per the scope guard in the brief.storyboard-troubleshooting.mdxnow states thatprerequisites.test_kitis a runner loading directive, not decoration.Testing
server/tests/unit/storyboards.test.ts(47/47): new cases —getTestKitForStoryboard('comply_controller_mode_gate')resolves the kit with itsauth.api_key, and the substitution guard holds over a pre-populated declared kit (run bearer does not clobber it).npm run typecheckclean.validator.test.ts/training-agent-webhook-fetch.test.ts(DNS/public-hostname fetches unavailable in my sandbox); untouched by this diff — CI is authoritative.Changeset
The triage brief asked for a
patchchangeset with a3.1.xcherry-pick, but the repo's push policy rejects changesets for server-only changes ("Protocol changesets are only allowed when the PR also changes protocol schemas, compliance assets, normative reference docs…"). Deferring to the policy — happy to re-add if you classify this as compliance-assets-adjacent, and the3.1.xcherry-pick works either way.Pairs with the runner half: adcontextprotocol/adcp-client#2643.
🤖 Generated with Claude Code