Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/building/operating/storyboard-troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ The `mechanism_required` phase found no contribution to `auth_mechanism_verified

**The `--auth TOKEN` distinction:** The `--auth TOKEN` flag you pass to the runner is the runner's own session credential — it authorizes the runner's own requests to your agent. It is entirely separate from `test_kit.auth.api_key` or `test_kit.auth.basic`, which are the specific credentials the static-credential phases send during positive and invalid-credential probes. These are not the same token and are not interchangeable.

**`prerequisites.test_kit` is a loading directive, not decoration:** when a storyboard declares `prerequisites.test_kit` (for example `comply_controller_mode_gate` declares `test-kits/acme-outdoor-live.yaml`), the runner loads that kit from the active compliance cache and its credentials flow into the storyboard's `from_test_kit` and `$test_kit.*` references without any CLI flag. An explicitly supplied kit (`options.test_kit` / hosted-run configuration) still wins. A step whose `auth.from_test_kit` resolves no credential fails with an explicit configuration error rather than sending an unauthenticated probe — a probe with no credential cannot test a credential-keyed contract.

**Fix for Bearer API key agents:** Every default AdCP brand test kit declares its probe API key under `auth.api_key` using the `demo-<kit>-v1` naming convention. The default test kit (`acme-outdoor`) uses `demo-acme-outdoor-v1`. Configure your agent to accept the kit's probe key as a valid compliance-testing credential alongside your production key. The `demo-<kit>-` prefix is the AdCP conformance handle — accept any Bearer token matching the prefix for the kits you run against (the suffix can rotate across spec versions, the prefix stays stable):

```typescript
Expand Down
10 changes: 10 additions & 0 deletions server/src/addie/mcp/member-tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ import {
} from '@adcp/sdk/testing';
import { AuthenticationRequiredError } from '@adcp/sdk';
import { renderAllHintFixPlans } from '../services/storyboard-fix-plan.js';
import { getTestKitForStoryboard } from '../../services/storyboards.js';
import {
hostedComplianceTarget,
hostedComplianceOptions,
Expand Down Expand Up @@ -5308,10 +5309,16 @@ export function createMemberToolHandlers(

try {
const authProbeTask = await inferHostedAuthProbeTask(resolved.resolvedUrl, authOption, runTarget);
// adcp#6735 — pre-populate the storyboard-declared test kit so
// `from_test_kit` steps run with the credential the storyboard was
// authored against; the run-auth bearer substitution no-ops when the
// kit already carries auth.
const declaredTestKit = getTestKitForStoryboard(storyboardId);
const result = await runStoryboard(
resolved.resolvedUrl,
sb,
withSdkSafeTransport(withHostedStoryboardRunOptions({
...(declaredTestKit && { test_kit: declaredTestKit }),
...(authOption && { auth: authOption }),
}, runTarget, authProbeTask)),
);
Expand Down Expand Up @@ -5525,11 +5532,14 @@ export function createMemberToolHandlers(

try {
const authProbeTask = await inferHostedAuthProbeTask(resolved.resolvedUrl, authOption, runTarget);
// adcp#6735 — same declared-kit pre-population as run_storyboard.
const declaredStepTestKit = getTestKitForStoryboard(storyboardId);
const result: StoryboardStepResult = await runStoryboardStep(
resolved.resolvedUrl,
sb,
resolvedStepId,
withSdkSafeTransport(withHostedStoryboardRunOptions({
...(declaredStepTestKit && { test_kit: declaredStepTestKit }),
context,
...(authOption && { auth: authOption }),
}, runTarget, authProbeTask)),
Expand Down
8 changes: 7 additions & 1 deletion server/src/conformance/run-storyboard-via-ws.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import { AgentClient } from '@adcp/sdk';
import { runStoryboard } from '@adcp/sdk/testing';
import type { StoryboardResult, StoryboardRunOptions } from '@adcp/sdk/testing';
import { conformanceSessions } from './session-store.js';
import { getStoryboard } from '../services/storyboards.js';
import { getStoryboard, getTestKitForStoryboard } from '../services/storyboards.js';
import { createLogger } from '../logger.js';
import { hostedComplianceTarget, withHostedStoryboardRunOptions } from '../services/hosted-compliance-version.js';

Expand Down Expand Up @@ -99,7 +99,13 @@ export async function runStoryboardViaConformanceSocket(
// `getOrCreateClient` but isn't on the public type. Cast through the
// narrow runOptions shape rather than `as any` so unrelated typos still
// get caught.
// adcp#6735 — pre-populate the storyboard-declared test kit so
// `from_test_kit` steps run with the credential the storyboard was
// authored against (the bearer substitution in `withHostedAuthTestKit`
// no-ops when the kit already carries auth).
const declaredTestKit = getTestKitForStoryboard(storyboardId);
const runOptions = withHostedStoryboardRunOptions({
...(declaredTestKit && { test_kit: declaredTestKit }),
_client: agentClient,
test_session_id: testSessionId,
timeout_ms: options.timeoutMs ?? 60_000,
Expand Down
6 changes: 6 additions & 0 deletions server/src/routes/registry-api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8346,11 +8346,17 @@ export function createRegistryApiRouters(config: RegistryApiConfig): { router: R
return res.status(400).json({ error: "context too large" });
}

// adcp#6735 — pre-populate the storyboard-declared test kit so
// `from_test_kit` steps run with the credential the storyboard was
// authored against; `withHostedAuthTestKit`'s `!nextAuth.api_key`
// guard then no-ops the run-auth bearer substitution.
const declaredTestKit = getTestKitForStoryboard(storyboard.id);
const result = await runStoryboardStep(
agentUrl,
storyboard,
req.params.stepId,
withSdkSafeTransport(withHostedStoryboardRunOptions({
...(declaredTestKit && { test_kit: declaredTestKit }),
...(sdkAuth && { auth: sdkAuth }),
...(context && { context }),
}, runTarget, authProbeTask)),
Expand Down
28 changes: 28 additions & 0 deletions server/tests/unit/storyboards.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,17 @@ describe('getTestKitForStoryboard', () => {
}
throw new Error('Expected at least one storyboard to declare prerequisites.test_kit');
});

// adcp#6735 — the mode-gate storyboard's declared kit must resolve WITH its
// auth block: the hosted call sites pre-populate options.test_kit from this
// resolver, and withHostedAuthTestKit's !nextAuth.api_key guard relies on
// the kit carrying the credential to no-op the run-auth bearer substitution.
it('resolves the mode-gate kit with its live credential', () => {
const kit = getTestKitForStoryboard('comply_controller_mode_gate');
expect(kit).toBeDefined();
const auth = kit!.auth as { api_key?: string } | undefined;
expect(auth?.api_key).toBeTruthy();
});
});

describe('wrapper contract', () => {
Expand Down Expand Up @@ -322,6 +333,23 @@ describe('wrapper contract', () => {
expect(options.test_kit?.auth?.probe_task).toBe('list_creatives');
});

// adcp#6735 — the load-bearing guard for declared-kit pre-population: when
// the call site placed the storyboard-declared kit (with its own api_key)
// into options.test_kit, the run-auth bearer must NOT substitute over it.
it('does not substitute the run bearer over a pre-populated declared kit credential', () => {
const declaredKit = getTestKitForStoryboard('comply_controller_mode_gate');
expect(declaredKit).toBeDefined();
const options = withHostedAuthTestKit({
test_kit: declaredKit,
auth: { type: 'bearer', token: 'seller-run-bearer' },
});

const declaredKey = (declaredKit!.auth as { api_key?: string }).api_key;
expect(declaredKey).toBeTruthy();
expect(options.test_kit?.auth?.api_key).toBe(declaredKey);
expect(options.test_kit?.auth?.api_key).not.toBe('seller-run-bearer');
});

it('threads hosted static fixture auth into the runtime test kit when no operator auth is supplied', () => {
const apiKey = hostedStaticApiKeyForProfile({
tools: ['get_adcp_capabilities', 'get_products'],
Expand Down
Loading