diff --git a/README.md b/README.md index 02671e1..7bbe20f 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,10 @@ The approved agent-integration contract is documented in the [decision record](d Relay is a local task sidecar for human–AI workflows. The current MVP is usable through its local web UI and through five safe local stdio MCP task tools. +## Agent skills + +The canonical [Relay Capture](skills/relay-capture/SKILL.md) and [Relay Session Review](skills/relay-session-review/SKILL.md) files guide agent behaviour; they do not implement persistence or protocol handlers. MCP is preferred and the CLI JSON output is the fallback; one adapter is retained through one workflow unless unavailable. The caller supplies the agent name and exact active session ID, Relay owns `createdByType` and autonomous `INBOX` status, and the exact-session lookup always occurs before final completion, including when its authoritative result is empty. See [agent skill guidance](docs/agent-skills.md) and the authoritative [MCP](docs/mcp-tools.md), [CLI](docs/cli-reference.md), and [session](docs/session-semantics.md) contracts. + ## Prerequisites and setup - Node.js `24.x` (see `.nvmrc`) diff --git a/docs/agent-skills.md b/docs/agent-skills.md new file mode 100644 index 0000000..7c93eaa --- /dev/null +++ b/docs/agent-skills.md @@ -0,0 +1,9 @@ +# Relay agent skills + +Relay capabilities live in the [MCP contracts](mcp-tools.md) and [CLI reference](cli-reference.md); behavioural policy lives only in the canonical [Relay Capture](../skills/relay-capture/SKILL.md) and [Relay Session Review](../skills/relay-session-review/SKILL.md) skills. + +MCP is preferred for supported interactive clients. The CLI is the JSON-only fallback for unsupported clients, scripts, debugging, or explicit one-shot use. Both use the same database and contracts, and one workflow retains one adapter unless it becomes unavailable. + +The caller supplies the agent name and exact active session ID where required. Relay owns adapter provenance (`createdByType`) and autonomous capture status (`INBOX`). Before final completion, the exact active-session lookup always occurs; an empty authoritative result is valid. Concurrent sessions remain isolated; see [session semantics](session-semantics.md). + +Fixtures in `skills/fixtures/` are deterministic policy examples validated by `validateSkillAssets`; they are not live-model tests. Vendor integrations may reference or mechanically copy the canonical content, but may not independently alter policy. Vendor packaging, setup workflows, and live-LLM testing remain deferred. diff --git a/docs/superpowers/plans/2026-07-28-issue-23-canonical-relay-skills.md b/docs/superpowers/plans/2026-07-28-issue-23-canonical-relay-skills.md index 25e2eba..b109c9b 100644 --- a/docs/superpowers/plans/2026-07-28-issue-23-canonical-relay-skills.md +++ b/docs/superpowers/plans/2026-07-28-issue-23-canonical-relay-skills.md @@ -77,12 +77,15 @@ Create a temporary fixture root helper that writes the six expected files. Each Expected: ACCEPT ### Scenario + A regression gap is discovered while implementing session expiry. ### Agent action + Capture a concise follow-up task and continue the original work. ### Reason + The work is concrete, actionable, and safely deferred. ``` diff --git a/docs/superpowers/plans/2026-07-28-pr-32-review-remediation.md b/docs/superpowers/plans/2026-07-28-pr-32-review-remediation.md new file mode 100644 index 0000000..5f80acf --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-pr-32-review-remediation.md @@ -0,0 +1,117 @@ +# PR #32 Review Remediation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Address all eight actionable remediation tasks from PR #32 while preserving issue #23 scope and strengthening deterministic skill-asset validation. + +**Architecture:** Keep policy in the two canonical skill files, examples in the four fixture files, and enforcement in `scripts/validate-skill-assets.ts`. Add focused validator tests that use canonical required fixture IDs and narrow affirmative-permission regex rules; update docs only where policy wording changed. + +**Tech Stack:** TypeScript, Vitest, pnpm, Markdown policy assets, Prettier, ESLint, TypeScript build. + +## Global Constraints + +- Remain within issue #23 scope: canonical skills, deterministic fixtures, validators/tests, supporting docs, and verification. +- Do not add MCP/CLI implementation, lifecycle logic, vendor packaging, marketplace work, live-LLM tests, or a policy engine. +- Do not weaken coverage or quality gates. +- Do not add CI-only bypasses, ignored warnings, or content-based test-mode escapes. +- Do not post GitHub comments, resolve threads, push, or create a PR in this session. + +--- + +### Task 1: Capture provenance wording + +**Files:** + +- Modify: `skills/relay-capture/SKILL.md` +- Test: `tests/unit/scripts/validate-skill-assets.test.ts` + +- [ ] Add a failing regression test that removes caller-owned provenance guidance and expects validation to fail. +- [ ] Run the focused test and confirm failure is caused by missing `createdByName`, exact active `sessionId`, adapter-owned `createdByType`, and Relay-owned `INBOX)/status concepts. +- [ ] Replace the ambiguous Capture procedure wording with precise caller-owned and adapter-owned guidance linked to `docs/mcp-tools.md`. +- [ ] Run the focused test and confirm it passes. + +### Task 2: Unconditional session review + +**Files:** + +- Modify: `skills/relay-session-review/SKILL.md` +- Modify: `skills/fixtures/session-review-negative.md` +- Test: `tests/unit/scripts/validate-skill-assets.test.ts` + +- [ ] Add a failing regression test for conditional pre-completion lookup wording. +- [ ] Add `REVIEW-SKIP-EMPTY-006` as a deterministic negative fixture without removing existing IDs. +- [ ] Rewrite Prohibited behaviour to require exact-session lookup before final completion, including when the result is empty. +- [ ] Add validator assertions for unconditional lookup and empty-result authority. +- [ ] Run focused tests and confirm they pass. + +### Task 3: Required fixture coverage + +**Files:** + +- Modify: `scripts/validate-skill-assets.ts` +- Modify: `tests/unit/scripts/validate-skill-assets.test.ts` + +- [ ] Add a failing test with syntactically valid `CASE-*` entries and no canonical required IDs. +- [ ] Remove the `CASE-*` early return from `validateFixtureCoverage`. +- [ ] Update the temporary-root helper to generate every required issue-specific ID for the matching fixture file. +- [ ] Add a positive test proving the minimum canonical set passes. +- [ ] Run focused tests and confirm both negative and positive cases pass. + +### Task 4: Forbidden-policy guardrails + +**Files:** + +- Modify: `scripts/validate-skill-assets.ts` +- Test: `tests/unit/scripts/validate-skill-assets.test.ts` + +- [ ] Add table-driven failing tests for affirmative autonomous archive, conditional empty-review skip, decorative CLI parsing, and storing full source/secrets despite safe wording. +- [ ] Add a small `ForbiddenPolicyRule` structure and `validateForbiddenPolicies` function with inspectable regex, label, and skill applicability. +- [ ] Apply capture and review rules only to their relevant canonical skill. +- [ ] Ensure prohibition wording such as “Never archive autonomously” remains valid. +- [ ] Run focused tests and confirm contradiction cases fail for the unsafe sentence while positive controls pass. + +### Task 5: Fixture coverage + +**Files:** + +- Modify: `skills/fixtures/capture-positive.md` +- Modify: `skills/fixtures/capture-negative.md` +- Modify: `skills/fixtures/session-review-positive.md` +- Modify: `skills/fixtures/session-review-negative.md` + +- [ ] Compare all four files against issue #23’s required positive and negative behaviors. +- [ ] Add only concise deterministic cases needed for missing coverage, including skipped empty review. +- [ ] Run the validator tests and confirm all fixture IDs and expected outcomes are valid. + +### Task 6: Policy-aligned documentation + +**Files:** + +- Modify: `docs/agent-skills.md` +- Modify: `README.md` only if a policy statement is inconsistent. + +- [ ] Update documentation to state MCP preference, CLI JSON fallback, adapter consistency, caller-supplied name/session, Relay-owned provenance/status, unconditional exact-session review, and authoritative empty results. +- [ ] Keep detailed schemas and lifecycle rules in the canonical skills/contracts instead of duplicating them. +- [ ] Run formatting and asset validation checks. + +### Task 7: Verification failure + +**Files:** + +- Modify only files justified by the failing verification and preceding tasks. + +- [ ] Re-run the individual verification commands in package.json order after the changes. +- [ ] Identify the deterministic cause of the current `validate:assets` failure and fix the underlying parser/asset issue. +- [ ] Run `pnpm verify` from the repository checkout. +- [ ] Confirm no tracked files are mutated by verification and no quality gate is weakened. + +### Task 8: Final self-review + +**Files:** + +- Review all changed canonical skills, fixtures, validator/tests, and docs. + +- [ ] Read both canonical skills for provenance, unconditional review, privacy, adapter selection, and scope compliance. +- [ ] Confirm no vendor-specific canonical policy source or duplicated schema was introduced. +- [ ] Confirm all eight local task items are satisfied. +- [ ] Run fresh full verification and report exact evidence. diff --git a/scripts/validate-repository-assets.ts b/scripts/validate-repository-assets.ts index 23a0268..59832d8 100644 --- a/scripts/validate-repository-assets.ts +++ b/scripts/validate-repository-assets.ts @@ -1,6 +1,7 @@ import { existsSync, readFileSync, readdirSync } from 'node:fs'; import { isAbsolute, join, relative, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; +import { validateSkillAssets } from './validate-skill-assets.js'; function fail(msg: string): never { throw new Error(`[ASSET VALIDATION FAILURE] ${msg}`); @@ -31,32 +32,42 @@ function walkFiles(rootDir: string, startDir = rootDir): string[] { return files; } -function validateMarkdownLinks(markdownPath: string, content: string): void { - const linkPattern = /\[[^\]]+\]\(([^)]+)\)/g; +function extractMarkdownLinkTargets(content: string): readonly string[] { + const withoutCode = content.replace(/```[\s\S]*?```/g, '').replace(/`[^`\r\n]*`/g, ''); + return [...withoutCode.matchAll(/\[[^\]]+\]\(([^)]+)\)/g)] + .map((match) => match[1]?.trim()) + .filter((target): target is string => Boolean(target)); +} - for (const match of content.matchAll(linkPattern)) { - const rawTarget = match[1]; - if (!rawTarget) { - continue; - } +function normalizeMarkdownLinkTarget(rawTarget: string): string | undefined { + let target = rawTarget.trim(); + if (target.startsWith('<')) { + const closingBracket = target.indexOf('>'); + target = closingBracket >= 0 ? target.slice(1, closingBracket) : target; + } else { + target = target.split(/\s+(?=["'])/)[0] ?? target; + } + const cleanTarget = target.split('#')[0]?.split('?')[0]; + return cleanTarget?.replaceAll('\\', '/').replace(/^\.\//, '') || undefined; +} + +function validateMarkdownLinks(markdownPath: string, content: string): void { + for (const rawTarget of extractMarkdownLinkTargets(content)) { + const normalizedTarget = normalizeMarkdownLinkTarget(rawTarget); + if (!normalizedTarget) continue; if ( - rawTarget.startsWith('http://') || - rawTarget.startsWith('https://') || - rawTarget.startsWith('mailto:') || - rawTarget.startsWith('#') + normalizedTarget.startsWith('http://') || + normalizedTarget.startsWith('https://') || + normalizedTarget.startsWith('mailto:') || + normalizedTarget.startsWith('#') ) { continue; } - const cleanTarget = rawTarget.split('#')[0]?.split('?')[0]; - if (!cleanTarget) { - continue; - } - - const resolvedTarget = isAbsolute(cleanTarget) - ? cleanTarget - : resolve(markdownPath, '..', cleanTarget); + const resolvedTarget = isAbsolute(normalizedTarget) + ? normalizedTarget + : resolve(markdownPath, '..', normalizedTarget); if (!existsSync(resolvedTarget)) { fail(`README local link does not resolve: ${rawTarget}`); @@ -124,6 +135,13 @@ export function validateRepositoryAssets(options: ValidateRepositoryAssetsOption 'docs/mcp-tools.md', 'docs/cli-reference.md', 'docs/session-semantics.md', + 'docs/agent-skills.md', + 'skills/relay-capture/SKILL.md', + 'skills/relay-session-review/SKILL.md', + 'skills/fixtures/capture-positive.md', + 'skills/fixtures/capture-negative.md', + 'skills/fixtures/session-review-positive.md', + 'skills/fixtures/session-review-negative.md', 'tests/fixtures/contracts/capture-success.json', 'tests/fixtures/contracts/capture-duplicate-warning.json', 'tests/fixtures/contracts/validation-error.json', @@ -190,9 +208,25 @@ export function validateRepositoryAssets(options: ValidateRepositoryAssetsOption if (!readme.includes('dist/cli/main.js') || !cliReference.includes('dist/cli/main.js')) { fail('README.md and docs/cli-reference.md must document the built CLI invocation.'); } + const readmeLinkTargets = new Set( + extractMarkdownLinkTargets(readme) + .map(normalizeMarkdownLinkTarget) + .filter((target): target is string => Boolean(target)), + ); + for (const requiredLink of [ + 'docs/agent-skills.md', + 'skills/relay-capture/SKILL.md', + 'skills/relay-session-review/SKILL.md', + ]) { + if (!readmeLinkTargets.has(requiredLink)) { + fail(`README.md must link to ${requiredLink}.`); + } + } const allFiles = walkFiles(rootDir); + validateSkillAssets({ rootDir }); + validateJsonFiles(allFiles); validatePlaceholders(allFiles); validateMarkdownLinks( @@ -200,8 +234,8 @@ export function validateRepositoryAssets(options: ValidateRepositoryAssetsOption readFileSync(join(rootDir, 'README.md'), 'utf-8'), ); - // 4. No SKILL.md or agent configs in #1 - const forbidden = ['SKILL.md', 'agent/skills', 'agent/mcp']; + // 4. No legacy agent integration roots + const forbidden = ['agent/skills', 'agent/mcp']; for (const f of forbidden) { if (existsSync(join(rootDir, f))) { fail(`Forbidden asset for Issue #1 present: ${f}`); diff --git a/scripts/validate-skill-assets.ts b/scripts/validate-skill-assets.ts new file mode 100644 index 0000000..ae95eee --- /dev/null +++ b/scripts/validate-skill-assets.ts @@ -0,0 +1,357 @@ +import { existsSync, readFileSync, readdirSync } from 'node:fs'; +import { join, relative, resolve } from 'node:path'; + +const canonicalSkillPaths = [ + 'skills/relay-capture/SKILL.md', + 'skills/relay-session-review/SKILL.md', +] as const; + +const fixturePaths = [ + 'skills/fixtures/capture-positive.md', + 'skills/fixtures/capture-negative.md', + 'skills/fixtures/session-review-positive.md', + 'skills/fixtures/session-review-negative.md', +] as const; + +interface SkillFixtureCase { + readonly id: string; + readonly expected: 'ACCEPT' | 'REJECT'; + readonly scenario: string; + readonly agentAction: string; + readonly reason: string; +} + +interface ForbiddenPolicyRule { + readonly label: string; + readonly pattern: RegExp; + readonly skill: 'capture' | 'review'; +} + +export interface ValidateSkillAssetsOptions { + readonly rootDir?: string; +} + +function fail(message: string): never { + throw new Error(`[SKILL ASSET VALIDATION FAILURE] ${message}`); +} + +function requiredSection(caseContent: string, heading: string, fixturePath: string): string { + const escapedHeading = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + const match = + new RegExp(`^### ${escapedHeading}\\r?\\n([\\s\\S]*?)(?=^### |^## )`, 'm').exec(caseContent) ?? + new RegExp(`^### ${escapedHeading}\\r?\\n([\\s\\S]*)$`, 'm').exec(caseContent); + const value = match?.[1]?.trim(); + if (!value) { + fail(`${fixturePath} case is missing a non-empty ${heading} section.`); + } + return value; +} + +function parseFixtureCases(fixturePath: string, content: string): readonly SkillFixtureCase[] { + const headings = [...content.matchAll(/^## ([^\r\n#]+)\r?$/gm)]; + if (headings.length === 0) { + fail(`${fixturePath} contains zero fixture cases.`); + } + + return headings.map((heading, index) => { + const id = heading[1]?.trim(); + if (!id) { + fail(`${fixturePath} contains a fixture case without an ID.`); + } + const start = (heading.index ?? 0) + heading[0].length; + const end = headings[index + 1]?.index ?? content.length; + const caseContent = content.slice(start, end); + const expectedMatches = [...caseContent.matchAll(/^Expected: (ACCEPT|REJECT)\r?$/gm)]; + if (expectedMatches.length !== 1 || !expectedMatches[0]?.[1]) { + fail( + `${fixturePath} case ${id} must contain exactly one Expected: ACCEPT or Expected: REJECT line.`, + ); + } + + return { + id, + expected: expectedMatches[0][1] as 'ACCEPT' | 'REJECT', + scenario: requiredSection(caseContent, 'Scenario', fixturePath), + agentAction: requiredSection(caseContent, 'Agent action', fixturePath), + reason: requiredSection(caseContent, 'Reason', fixturePath), + }; + }); +} + +function validateFixtureCoverage(fixturePath: string, cases: readonly SkillFixtureCase[]): void { + const required = fixturePath.endsWith('capture-positive.md') + ? ['CAPTURE-ACTIONABLE-001', 'CAPTURE-DUPLICATE-002', 'CAPTURE-CLI-FALLBACK-003'] + : fixturePath.endsWith('capture-negative.md') + ? [ + 'CAPTURE-SENSITIVE-002', + 'CAPTURE-MUTATION-003', + 'CAPTURE-SESSION-005', + 'CAPTURE-ADAPTER-006', + ] + : fixturePath.endsWith('session-review-positive.md') + ? ['REVIEW-ACTIVE-SESSION-001', 'REVIEW-EXPLICIT-ACTIONS-002', 'REVIEW-UNRESOLVED-003'] + : [ + 'REVIEW-OMITTED-001', + 'REVIEW-WRONG-SESSION-002', + 'REVIEW-SILENT-MUTATION-003', + 'REVIEW-TIMER-005', + 'REVIEW-SKIP-EMPTY-006', + 'REVIEW-GENERIC-MUTATION-007', + ]; + for (const id of required) { + if (!cases.some((fixtureCase) => fixtureCase.id === id)) + fail(`${fixturePath} is missing required coverage: ${id}.`); + } +} + +function validateContains(content: string, pattern: RegExp, label: string): void { + if (!pattern.test(content)) { + fail(`Canonical skill is missing required policy: ${label}.`); + } +} + +const forbiddenPolicyRules: readonly ForbiddenPolicyRule[] = [ + { + skill: 'capture', + label: 'autonomous mutation of an existing task', + pattern: + /\b(?:may|can)\s+(?:silently\s+)?(?:autonomously\s+)?(?:edit|triage|move|start|complete|archive|delete|merge)\b/i, + }, + { + skill: 'capture', + label: 'moving a new autonomous capture out of INBOX', + pattern: + /(?:may|can|should)\b[^.\n]*\b(?:autonomous|new)\b[^.\n]*\b(?:move|remove|take)\b[^.\n]*\b(?:out of|from)\s+INBOX\b/i, + }, + { + skill: 'capture', + label: 'storing sensitive or oversized context', + pattern: + /\b(?:may|can|should)\s+(?:store|include|attach|copy)\b[^.\n]*\b(?:prompts?|transcripts?|source files?|secrets?|credentials?|tokens?|large stack traces?|logs?|oversized)\b/i, + }, + { + skill: 'capture', + label: 'reusing a session ID across unrelated sessions', + pattern: /\b(?:may|can|should)\b[^.\n]*\breuse\b[^.\n]*\bsession ID\b[^.\n]*\bunrelated\b/i, + }, + { + skill: 'capture', + label: 'unjustified adapter switching', + pattern: + /\b(?:may|can|should)\b[^.\n]*\bswitch\b[^.\n]*\b(?:MCP|CLI)\b[^.\n]*\b(?:without|regardless of)\b[^.\n]*(?:failure|unavailable|reason|debug)/i, + }, + { + skill: 'capture', + label: 'parsing decorative CLI output', + pattern: + /\b(?:may|can|should)\b[^.\n]*\bparse\b[^.\n]*\b(?:decorative|human|terminal)\b[^.\n]*(?:output|text)/i, + }, + { + skill: 'review', + label: 'mutation without explicit user direction', + pattern: + /\b(?:may|can|should)\b[^.\n]*\b(?:mutate|change|update)\b[^.\n]*(?:without|no)\b[^.\n]*\buser direction\b/i, + }, + { + skill: 'review', + label: 'generic status mutation', + pattern: + /\b(?:may|can|should)\b[^.\n]*\b(?:generic|unrestricted)\b[^.\n]*\bstatus\b[^.\n]*\b(?:mutation|update|change)\b/i, + }, + { + skill: 'review', + label: 'skipping the exact-session lookup', + pattern: + /(? /^(name|description): .+/.test(line))) { + fail(`Canonical skill ${expectedName} frontmatter may contain only name and description.`); + } + const values = Object.fromEntries(lines.map((line) => line.split(/: (.+)/, 2))) as Record< + string, + string + >; + if (values.name !== expectedName || !values.description?.startsWith('Use when')) { + fail( + `Canonical skill ${expectedName} must have its canonical name and a description beginning Use when.`, + ); + } +} + +function validateContractLinks(content: string): void { + for (const link of ['docs/mcp-tools.md', 'docs/cli-reference.md', 'docs/session-semantics.md']) { + if (!content.includes(link)) fail(`Canonical skill must link to ${link}.`); + } +} + +function validateCaptureSkill(content: string): void { + for (const section of [ + 'Purpose', + 'When to capture', + 'Adapter selection', + 'Session and provenance', + 'Capture procedure', + 'Duplicate handling', + 'Context safety', + 'Autonomy boundaries', + 'Do not capture', + ]) { + validateContains(content, new RegExp(`^## ${section}$`, 'mi'), section); + } + for (const [pattern, label] of [ + [/concrete,? actionable follow-up/i, 'concrete actionable follow-up'], + [/MCP.*preferred/i, 'MCP preference'], + [/CLI.*fallback/i, 'CLI fallback'], + [/--output json/i, 'CLI JSON output'], + [/same adapter|one adapter/i, 'one adapter per workflow'], + [/session ID/i, 'session ID'], + [/createdByName/i, 'caller-owned createdByName'], + [/exact active session ID/i, 'exact active session ID'], + [/createdByType/i, 'adapter-owned createdByType'], + [/Relay.*(?:INBOX|status)|(?:INBOX|status).*Relay/i, 'Relay-owned capture status'], + [/INBOX/i, 'INBOX capture'], + [/duplicate.*advisory/i, 'advisory duplicate handling'], + [/continue.*original work/i, 'continue original work'], + [/must not.*(?:edit|triage|start|complete|archive)/i, 'autonomy boundary'], + ] as const) { + validateContains(content, pattern, label); + } + validateForbiddenPolicies(content, 'capture'); +} + +function validateReviewSkill(content: string): void { + for (const section of [ + 'Purpose', + 'When to review', + 'Session lookup', + 'Review presentation', + 'User-directed actions', + 'Unresolved captures', + 'Adapter selection', + 'Prohibited behaviour', + ]) + validateContains(content, new RegExp(`^## ${section}$`, 'mi'), section); + for (const [pattern, label] of [ + [ + /always.*exact active[- ]session.*before final completion/i, + 'unconditional pre-completion review', + ], + [/exact active session ID/i, 'exact session ID'], + [/empty.*authoritative|authoritative.*empty/i, 'authoritative empty result'], + [/completed.*archived|archived.*completed/i, 'all-status review'], + [/explicit user direction/i, 'explicit user direction'], + [/intent-specific/i, 'intent-specific actions'], + [/unresolved.*INBOX/i, 'unresolved INBOX'], + [/never infer.*(?:timer|inactivity|process exit)/i, 'no timer inference'], + [/never.*(?:mix|another).*session/i, 'session isolation'], + ] as const) + validateContains(content, pattern, label); + validateForbiddenPolicies(content, 'review'); +} + +function validateCanonicalSources(rootDir: string, currentDir = rootDir): void { + for (const entry of readdirSync(currentDir, { withFileTypes: true })) { + if (['.git', 'node_modules', 'dist', 'coverage'].includes(entry.name)) continue; + const fullPath = join(currentDir, entry.name); + if (entry.isDirectory()) { + validateCanonicalSources(rootDir, fullPath); + continue; + } + if (entry.name !== 'SKILL.md') continue; + const path = relative(rootDir, fullPath).replaceAll('\\', '/'); + if (canonicalSkillPaths.includes(path as (typeof canonicalSkillPaths)[number])) continue; + const content = readFileSync(fullPath, 'utf-8'); + const candidates = canonicalSkillPaths.filter((canonicalPath) => { + const skillName = canonicalPath.includes('relay-capture') + ? /relay-capture|Relay Capture/i + : /relay-session-review|Relay Session Review/i; + return skillName.test(content); + }); + if (candidates.length === 0) continue; + if (candidates.length > 1) { + fail(`Vendor-specific Relay policy must identify one canonical source: ${path}.`); + } + const canonical = candidates[0]; + if (!canonical) { + fail(`Vendor-specific Relay policy must identify a canonical source: ${path}.`); + } + const canonicalName = canonical.includes('relay-capture') + ? 'relay-capture' + : 'relay-session-review'; + const sourcePattern = new RegExp( + `${canonicalName}/SKILL\\.md|${canonical.replace(/[.*+?^${}()|[\\]\\]/g, '\\$&')}`, + 'i', + ); + if (!sourcePattern.test(content)) { + fail(`Vendor-specific Relay policy must explicitly reference its canonical source: ${path}.`); + } + } +} + +export function validateSkillAssets(options: ValidateSkillAssetsOptions = {}): void { + const rootDir = options.rootDir ? resolve(options.rootDir) : process.cwd(); + + for (const path of [...canonicalSkillPaths, ...fixturePaths]) { + if (!existsSync(join(rootDir, path))) { + fail(`Required canonical skill asset missing: ${path}`); + } + } + + const capture = readFileSync(join(rootDir, canonicalSkillPaths[0]), 'utf-8'); + const review = readFileSync(join(rootDir, canonicalSkillPaths[1]), 'utf-8'); + validateFrontmatter(capture, 'relay-capture'); + validateFrontmatter(review, 'relay-session-review'); + validateContractLinks(capture); + validateContractLinks(review); + validateCaptureSkill(capture); + validateReviewSkill(review); + validateCanonicalSources(rootDir); + + const seenIds = new Set(); + for (const fixturePath of fixturePaths) { + const cases = parseFixtureCases(fixturePath, readFileSync(join(rootDir, fixturePath), 'utf-8')); + validateFixtureCoverage(fixturePath, cases); + const expected = fixturePath.endsWith('-positive.md') ? 'ACCEPT' : 'REJECT'; + for (const fixtureCase of cases) { + if (fixtureCase.expected !== expected) { + fail(`${fixturePath} case ${fixtureCase.id} must be ${expected}.`); + } + if (seenIds.has(fixtureCase.id)) { + fail(`Duplicate fixture case ID: ${fixtureCase.id}.`); + } + seenIds.add(fixtureCase.id); + } + } +} diff --git a/skills/fixtures/capture-negative.md b/skills/fixtures/capture-negative.md new file mode 100644 index 0000000..9ef9163 --- /dev/null +++ b/skills/fixtures/capture-negative.md @@ -0,0 +1,95 @@ +## CAPTURE-SPECULATION-001 + +Expected: REJECT + +### Scenario + +The agent has a possible future enhancement idea. + +### Agent action + +Capture every thought as a Relay task. + +### Reason + +Speculation is not concrete follow-up work. + +## CAPTURE-SENSITIVE-002 + +Expected: REJECT + +### Scenario + +A diagnostic includes a credential and a large stack trace. + +### Agent action + +Attach the prompt, source file, credential, and stack trace as source context. + +### Reason + +Sensitive and oversized context is forbidden. + +## CAPTURE-MUTATION-003 + +Expected: REJECT + +### Scenario + +A possible duplicate exists. + +### Agent action + +Silently complete or archive the existing task. + +### Reason + +Autonomous existing-task mutation is forbidden. + +## CAPTURE-TRIAGE-004 + +Expected: REJECT + +### Scenario + +An autonomous capture succeeds. + +### Agent action + +Move it from INBOX to ACTIVE. + +### Reason + +Autonomous captures cannot leave INBOX. + +## CAPTURE-SESSION-005 + +Expected: REJECT + +### Scenario + +Two unrelated concurrent sessions run in one workspace. + +### Agent action + +Reuse one session ID for both. + +### Reason + +Concurrent sessions must remain isolated. + +## CAPTURE-ADAPTER-006 + +Expected: REJECT + +### Scenario + +MCP is available throughout capture. + +### Agent action + +Use MCP for lookup and CLI for capture without a failure or explicit reason. + +### Reason + +One workflow retains one adapter. diff --git a/skills/fixtures/capture-positive.md b/skills/fixtures/capture-positive.md new file mode 100644 index 0000000..ef44fc7 --- /dev/null +++ b/skills/fixtures/capture-positive.md @@ -0,0 +1,63 @@ +## CAPTURE-ACTIONABLE-001 + +Expected: ACCEPT + +### Scenario + +A missing regression test is discovered while implementing session expiry. + +### Agent action + +Use `task_find_similar`, capture a concise INBOX task with the active session ID, then continue the original work. + +### Reason + +The regression gap is concrete, actionable, and safely deferred. + +## CAPTURE-DUPLICATE-002 + +Expected: ACCEPT + +### Scenario + +Relay returns a possible duplicate warning. + +### Agent action + +Keep the successful capture and record the warning as advisory without mutating the existing task. + +### Reason + +Duplicate candidates never silently suppress capture. + +## CAPTURE-CLI-FALLBACK-003 + +Expected: ACCEPT + +### Scenario + +MCP is unavailable for a script. + +### Agent action + +Use `task find-similar --output json` and `task capture --output json` through the same CLI adapter. + +### Reason + +CLI JSON is the deterministic fallback. + +## CAPTURE-CONTEXT-004 + +Expected: ACCEPT + +### Scenario + +A capture needs source context. + +### Agent action + +Store `session expiry integration tests` as the source context. + +### Reason + +The reference is concise and does not include code or transcripts. diff --git a/skills/fixtures/session-review-negative.md b/skills/fixtures/session-review-negative.md new file mode 100644 index 0000000..0a1cf74 --- /dev/null +++ b/skills/fixtures/session-review-negative.md @@ -0,0 +1,111 @@ +## REVIEW-OMITTED-001 + +Expected: REJECT + +### Scenario + +The agent is ready to finish. + +### Agent action + +Finish without querying captured tasks. + +### Reason + +Final review must not be omitted. + +## REVIEW-WRONG-SESSION-002 + +Expected: REJECT + +### Scenario + +Another agent ran concurrently. + +### Agent action + +Query its session or merge both session IDs. + +### Reason + +Sessions must remain isolated. + +## REVIEW-SILENT-MUTATION-003 + +Expected: REJECT + +### Scenario + +Several captures remain unresolved. + +### Agent action + +Move all to Active or archive them without choices. + +### Reason + +Disposition requires explicit user direction. + +## REVIEW-INBOX-ONLY-004 + +Expected: REJECT + +### Scenario + +Relay returns DONE and ARCHIVED captures. + +### Agent action + +Hide them and present only INBOX tasks. + +### Reason + +Review includes all returned statuses. + +## REVIEW-TIMER-005 + +Expected: REJECT + +### Scenario + +The user has been inactive. + +### Agent action + +Infer wrap-up from timer or process exit. + +### Reason + +Completion is never inferred. + +## REVIEW-SKIP-EMPTY-006 + +Expected: REJECT + +### Scenario + +The agent believes it captured nothing and is ready to finish. + +### Agent action + +Skip `session_captures_list` before final completion because the expected result is empty. + +### Reason + +The exact active-session lookup is mandatory and an empty authoritative result is valid. + +## REVIEW-GENERIC-MUTATION-007 + +Expected: REJECT + +### Scenario + +The user requests a status change. + +### Agent action + +Use a generic update command. + +### Reason + +Only intent-specific capabilities are allowed. diff --git a/skills/fixtures/session-review-positive.md b/skills/fixtures/session-review-positive.md new file mode 100644 index 0000000..11927f1 --- /dev/null +++ b/skills/fixtures/session-review-positive.md @@ -0,0 +1,79 @@ +## REVIEW-ACTIVE-SESSION-001 + +Expected: ACCEPT + +### Scenario + +The agent is about to give final completion. + +### Agent action + +Call `session_captures_list` with the exact active session ID and report INBOX, DONE, and ARCHIVED results. + +### Reason + +Exact-session review includes every status returned by Relay. + +## REVIEW-EXPLICIT-ACTIONS-002 + +Expected: ACCEPT + +### Scenario + +The user selects Active, Complete, and Archive actions. + +### Agent action + +Use only `task_triage`, `task_complete`, and `task_archive` for the selected IDs. + +### Reason + +Mutations require explicit intent-specific direction. + +## REVIEW-UNRESOLVED-003 + +Expected: ACCEPT + +### Scenario + +The user chooses one of several captures. + +### Agent action + +Leave every unselected capture unchanged in INBOX. + +### Reason + +Unresolved work is not silently mutated. + +## REVIEW-PREEXISTING-004 + +Expected: ACCEPT + +### Scenario + +A duplicate warning points to an older task. + +### Agent action + +Present it separately from tasks returned by the active-session query. + +### Reason + +Duplicate candidates are pre-existing unless captured in the exact session. + +## REVIEW-CLI-FALLBACK-005 + +Expected: ACCEPT + +### Scenario + +MCP is unavailable. + +### Agent action + +Use `session captures --session --output json` and JSON mutation commands. + +### Reason + +CLI is the deterministic fallback. diff --git a/skills/relay-capture/SKILL.md b/skills/relay-capture/SKILL.md new file mode 100644 index 0000000..79656e8 --- /dev/null +++ b/skills/relay-capture/SKILL.md @@ -0,0 +1,47 @@ +--- +name: relay-capture +description: Use when concrete follow-up work is discovered while performing another task and Relay is available through MCP or its deterministic CLI. +--- + +# Relay Capture + +## Purpose + +Capture a concrete, actionable follow-up without derailing the current activity. Relay capabilities perform persistence and validation; this skill governs agent behaviour. + +## When to capture + +Capture work discovered during another task when it is concrete, independently actionable, safely deferrable, and should persist beyond the current response. Do not capture speculative ideas, vague reminders, work already being completed, or status notes. + +## Adapter selection + +MCP is preferred for supported interactive clients. CLI is the deterministic fallback for unsupported clients, scripts, debugging, or explicit one-shot use. Keep the same adapter for one workflow unless it fails or becomes unavailable. In CLI mode, use `--output json` and parse only JSON, never decorative terminal output. MCP and CLI use the same Relay database and contracts. + +## Session and provenance + +Generate one valid opaque session ID for the active agent session or reuse its already-established ID. Retain that exact active session ID for every capture and final review; never reuse it across unrelated concurrent agents or shells. Provide a concise title, agent name as `createdByName`, exact active session ID, workspace when known, and limited source context. + +## Capture procedure + +1. Decide that the follow-up is concrete and actionable. +2. When practical, use `task_find_similar` or the matching CLI command before capture. +3. Use `task_capture`, or the documented source-checkout CLI invocation for `task capture --output json`. The agent supplies `title`, `createdByName`, the exact active `sessionId`, and concise optional `workspace` and `sourceContext` when available; supply other optional fields only when allowed by the authoritative contract. Do not supply adapter-owned `createdByType`, `status`, lifecycle timestamps, or other forbidden fields; Relay sets autonomous captures to `INBOX`. See [MCP tool contracts](../../docs/mcp-tools.md) for the authoritative schema. +4. Retain the returned task ID and warnings, then continue the original work without separately interrupting the user after every capture. + +## Duplicate handling + +Duplicate candidates are advisory. Do not suppress capture solely because a candidate exists, and do not merge or mutate existing tasks unless the user explicitly directs it. + +## Context safety + +Store only concise source context that identifies where or why the work was found. Never store prompts, transcripts, source files, secrets, credentials, tokens, large stack traces, logs, or oversized copied context. + +## Autonomy boundaries + +An agent may autonomously create only a new Relay task in `INBOX`. It must not edit, triage, start, complete, archive, delete, merge, or move any task, including a new capture, without explicit user direction in the active conversation. + +## Do not capture + +Do not capture every thought, casual ideas, or anything that cannot be acted on later without reconstructing the conversation. + +See [MCP tool contracts](../../docs/mcp-tools.md), [CLI contract reference](../../docs/cli-reference.md), and [session semantics](../../docs/session-semantics.md). diff --git a/skills/relay-session-review/SKILL.md b/skills/relay-session-review/SKILL.md new file mode 100644 index 0000000..c2db2c8 --- /dev/null +++ b/skills/relay-session-review/SKILL.md @@ -0,0 +1,40 @@ +--- +name: relay-session-review +description: Use when preparing final completion or when the user asks to wrap up, review, or show Relay tasks captured in the active agent session. +--- + +# Relay Session Review + +## Purpose + +Before final completion, review every Relay task captured with the exact active session ID. Present captures compactly, distinguish pre-existing duplicate candidates, and mutate only on explicit user direction. + +## When to review + +Always perform the exact active-session lookup before final completion, even when the agent believes no captures exist. A user-triggered wrap-up or review is an additional trigger, not a replacement. Never infer completion from a timer, inactivity, or process exit. + +## Session lookup + +Use `session_captures_list` with the exact active session ID, or the documented source-checkout CLI `session captures --session --output json` fallback. Relay's ordered result is authoritative: an empty result is valid, and completed and archived captures must be included alongside INBOX tasks. Never query a guessed session or mix tasks from another session ID. + +## Review presentation + +Present returned captures with ID, title, and current status. Label duplicate candidates separately as pre-existing unless the exact-session query also returned them. Do not reconstruct captures from remembered IDs or timestamps. + +## User-directed actions + +Obtain explicit user direction for each selected task. Use only intent-specific capabilities: `task_edit`, `task_triage`, `task_start`, `task_complete`, or `task_archive`. Report `NO_CHANGE`, conflicts, archived-task restrictions, and errors accurately; never invent success. + +## Unresolved captures + +Do not mutate unselected tasks. Unresolved captures remain in `INBOX`. + +## Adapter selection + +Keep the adapter used for capture unless it is concretely unavailable. MCP is preferred; CLI fallback always uses `--output json` and parses structured output only. + +## Prohibited behaviour + +Never omit the exact-session lookup before final completion, even when the agent believes no captures exist. Treat an empty authoritative result as valid. Never query a guessed session, reconstruct captures from memory or timestamps, silently apply dispositions, use a generic status mutation, hide completed or archived captures returned by Relay, or infer completion from a timer, inactivity, or process exit. + +See [MCP tool contracts](../../docs/mcp-tools.md), [CLI contract reference](../../docs/cli-reference.md), and [session semantics](../../docs/session-semantics.md). diff --git a/tests/unit/scripts/validate-repository-assets.test.ts b/tests/unit/scripts/validate-repository-assets.test.ts index af8f68b..efc63f5 100644 --- a/tests/unit/scripts/validate-repository-assets.test.ts +++ b/tests/unit/scripts/validate-repository-assets.test.ts @@ -1,4 +1,4 @@ -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { afterEach, describe, expect, it } from 'vitest'; @@ -14,6 +14,9 @@ function createFixtureRoot(): string { 'src/interfaces/http', 'src/interfaces/cli', 'web/src', + 'skills/relay-capture', + 'skills/relay-session-review', + 'skills/fixtures', ]; for (const dir of requiredDirs) { @@ -43,7 +46,7 @@ function createFixtureRoot(): string { ); writeFileSync( join(rootDir, 'README.md'), - '# Relay\n\n[Decision](docs/decisions/0001-product-and-architecture.md)\n\n`node dist/cli/main.js`\n', + '# Relay\n\n[Decision](docs/decisions/0001-product-and-architecture.md)\n\n`node dist/cli/main.js`\n\n[Agent skills](docs/agent-skills.md)\n\n[Capture skill](skills/relay-capture/SKILL.md)\n\n[Review skill](skills/relay-session-review/SKILL.md)\n', ); writeFileSync(join(rootDir, 'src/application/health/get-health.ts'), 'export {};\n'); writeFileSync(join(rootDir, 'src/database/connection.ts'), 'export {};\n'); @@ -65,6 +68,61 @@ function createFixtureRoot(): string { '# CLI reference\n\n`node dist/cli/main.js`\n', ); writeFileSync(join(rootDir, 'docs/session-semantics.md'), '# Session semantics\n'); + writeFileSync(join(rootDir, 'docs/agent-skills.md'), '# Agent skills\n'); + writeFileSync( + join(rootDir, 'skills/relay-capture/SKILL.md'), + `## Purpose\n\nCapture a concrete, actionable follow-up.\n\n## When to capture\n\nUse it for a concrete, actionable follow-up.\n\n## Adapter selection\n\nMCP is preferred. CLI is the fallback with --output json and one adapter.\n\n## Session and provenance\n\nThe agent supplies createdByName and the exact active session ID. Relay supplies createdByType: AGENT and status: INBOX.\n\n## Capture procedure\n\nContinue the original work.\n\n## Duplicate handling\n\nA duplicate is advisory.\n\n## Context safety\n\nKeep context concise.\n\n## Autonomy boundaries\n\nAn agent must not edit, triage, start, complete, or archive tasks. Leave captures in INBOX.\n\n## Do not capture\n\nDo not capture speculation.\n`, + ); + writeFileSync( + join(rootDir, 'skills/relay-session-review/SKILL.md'), + `## Purpose\n\nReview before final completion.\n\n## When to review\n\nAlways perform the exact active session lookup before final completion.\n\n## Session lookup\n\nUse the exact active session ID. Include completed and archived tasks; never mix sessions. An empty authoritative result is valid.\n\n## Review presentation\n\nPresent captures.\n\n## User-directed actions\n\nRequire explicit user direction and intent-specific actions.\n\n## Unresolved captures\n\nLeave unresolved tasks in INBOX.\n\n## Adapter selection\n\nUse the same adapter.\n\n## Prohibited behaviour\n\nNever infer completion from timer, inactivity, or process exit.\n`, + ); + for (const [path, name, description] of [ + ['skills/relay-capture/SKILL.md', 'relay-capture', 'Use when testing capture.'], + ['skills/relay-session-review/SKILL.md', 'relay-session-review', 'Use when testing review.'], + ] as const) { + const filePath = join(rootDir, path); + writeFileSync( + filePath, + `---\nname: ${name}\ndescription: ${description}\n---\n\n${readFileSync(filePath, 'utf-8')}\n../../docs/mcp-tools.md ../../docs/cli-reference.md ../../docs/session-semantics.md\n`, + ); + } + for (const [index, filename] of [ + 'capture-positive.md', + 'capture-negative.md', + 'session-review-positive.md', + 'session-review-negative.md', + ].entries()) { + const expected = filename.includes('positive') ? 'ACCEPT' : 'REJECT'; + const requiredIds = [ + ['CAPTURE-ACTIONABLE-001', 'CAPTURE-DUPLICATE-002', 'CAPTURE-CLI-FALLBACK-003'], + [ + 'CAPTURE-SENSITIVE-002', + 'CAPTURE-MUTATION-003', + 'CAPTURE-SESSION-005', + 'CAPTURE-ADAPTER-006', + ], + ['REVIEW-ACTIVE-SESSION-001', 'REVIEW-EXPLICIT-ACTIONS-002', 'REVIEW-UNRESOLVED-003'], + [ + 'REVIEW-OMITTED-001', + 'REVIEW-WRONG-SESSION-002', + 'REVIEW-SILENT-MUTATION-003', + 'REVIEW-TIMER-005', + 'REVIEW-SKIP-EMPTY-006', + 'REVIEW-GENERIC-MUTATION-007', + ], + ][index]; + if (!requiredIds) throw new Error(`Missing required fixture IDs for ${filename}`); + writeFileSync( + join(rootDir, 'skills/fixtures', filename), + requiredIds + .map( + (id) => + `## ${id}\n\nExpected: ${expected}\n\n### Scenario\nScenario\n\n### Agent action\nAction\n\n### Reason\nReason\n`, + ) + .join('\n'), + ); + } mkdirSync(join(rootDir, 'src/interfaces/contracts'), { recursive: true }); for (const filename of [ 'contract-version.ts', @@ -150,4 +208,42 @@ describe('validateRepositoryAssets', () => { expect(() => validateRepositoryAssets({ rootDir })).toThrow(/CLI executable|dist\/cli/i); }); + + it('accepts canonical skills but rejects legacy agent policy roots', () => { + const rootDir = createFixtureRoot(); + createdRoots.push(rootDir); + mkdirSync(join(rootDir, 'agent/skills'), { recursive: true }); + + expect(() => validateRepositoryAssets({ rootDir })).toThrow(/legacy|agent\/skills/i); + }); + + it('requires README links to canonical skill guidance', () => { + const rootDir = createFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync(join(rootDir, 'README.md'), '# Relay\n\n`node dist/cli/main.js`\n'); + + expect(() => validateRepositoryAssets({ rootDir })).toThrow(/README.*agent-skills/i); + }); + + it('does not count plain text or fenced code mentions as README links', () => { + const rootDir = createFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync( + join(rootDir, 'README.md'), + '# Relay\n\n`node dist/cli/main.js`\n\n```md\n[Agent skills](docs/agent-skills.md)\n[Capture skill](skills/relay-capture/SKILL.md)\n[Review skill](skills/relay-session-review/SKILL.md)\n```\n', + ); + + expect(() => validateRepositoryAssets({ rootDir })).toThrow(/README.*agent-skills/i); + }); + + it('accepts normalized Markdown links to canonical skill guidance', () => { + const rootDir = createFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync( + join(rootDir, 'README.md'), + '# Relay\n\n`node dist/cli/main.js`\n\n[Agent skills](./docs/agent-skills.md#overview)\n[Capture skill](./skills/relay-capture/SKILL.md?source=readme)\n[Review skill](./skills/relay-session-review/SKILL.md)\n', + ); + + expect(() => validateRepositoryAssets({ rootDir })).not.toThrow(); + }); }); diff --git a/tests/unit/scripts/validate-skill-assets.test.ts b/tests/unit/scripts/validate-skill-assets.test.ts new file mode 100644 index 0000000..c539af4 --- /dev/null +++ b/tests/unit/scripts/validate-skill-assets.test.ts @@ -0,0 +1,228 @@ +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; +import { validateSkillAssets } from '../../../scripts/validate-skill-assets.js'; + +const fixtureFiles = [ + 'skills/fixtures/capture-positive.md', + 'skills/fixtures/capture-negative.md', + 'skills/fixtures/session-review-positive.md', + 'skills/fixtures/session-review-negative.md', +] as const; + +function fixtureCase(id: string, expected: 'ACCEPT' | 'REJECT'): string { + return `## ${id}\n\nExpected: ${expected}\n\n### Scenario\nA concrete scenario.\n\n### Agent action\nA bounded agent action.\n\n### Reason\nA deterministic policy reason.\n`; +} + +function createValidSkillFixtureRoot(): string { + const rootDir = mkdtempSync(join(tmpdir(), 'relay-skill-validator-')); + + for (const path of ['skills/relay-capture', 'skills/relay-session-review', 'skills/fixtures']) { + mkdirSync(join(rootDir, path), { recursive: true }); + } + + writeFileSync( + join(rootDir, 'skills/relay-capture/SKILL.md'), + `## Purpose\n\nCapture a concrete, actionable follow-up.\n\n## When to capture\n\nUse it for a concrete, actionable follow-up.\n\n## Adapter selection\n\nMCP is preferred. CLI is the fallback with --output json and one adapter.\n\n## Session and provenance\n\nThe agent supplies createdByName and the exact active session ID. Relay supplies createdByType: AGENT and status: INBOX.\n\n## Capture procedure\n\nContinue the original work.\n\n## Duplicate handling\n\nA duplicate is advisory.\n\n## Context safety\n\nKeep context concise.\n\n## Autonomy boundaries\n\nAn agent must not edit, triage, start, complete, or archive tasks. Leave captures in INBOX.\n\n## Do not capture\n\nDo not capture speculation.\n`, + ); + writeFileSync( + join(rootDir, 'skills/relay-session-review/SKILL.md'), + `## Purpose\n\nReview before final completion.\n\n## When to review\n\nAlways perform the exact active session lookup before final completion.\n\n## Session lookup\n\nUse the exact active session ID. Include completed and archived tasks; never mix sessions. An empty authoritative result is valid.\n\n## Review presentation\n\nPresent captures.\n\n## User-directed actions\n\nRequire explicit user direction and intent-specific actions.\n\n## Unresolved captures\n\nLeave unresolved tasks in INBOX.\n\n## Adapter selection\n\nUse the same adapter.\n\n## Prohibited behaviour\n\nNever infer completion from timer, inactivity, or process exit.\n`, + ); + for (const [path, name, description] of [ + ['skills/relay-capture/SKILL.md', 'relay-capture', 'Use when testing capture.'], + ['skills/relay-session-review/SKILL.md', 'relay-session-review', 'Use when testing review.'], + ] as const) { + const filePath = join(rootDir, path); + writeFileSync( + filePath, + `---\nname: ${name}\ndescription: ${description}\n---\n\n${readFileSync(filePath, 'utf-8')}\n../../docs/mcp-tools.md ../../docs/cli-reference.md ../../docs/session-semantics.md\n`, + ); + } + const requiredFixtureIds = [ + ['CAPTURE-ACTIONABLE-001', 'CAPTURE-DUPLICATE-002', 'CAPTURE-CLI-FALLBACK-003'], + ['CAPTURE-SENSITIVE-002', 'CAPTURE-MUTATION-003', 'CAPTURE-SESSION-005', 'CAPTURE-ADAPTER-006'], + ['REVIEW-ACTIVE-SESSION-001', 'REVIEW-EXPLICIT-ACTIONS-002', 'REVIEW-UNRESOLVED-003'], + [ + 'REVIEW-OMITTED-001', + 'REVIEW-WRONG-SESSION-002', + 'REVIEW-SILENT-MUTATION-003', + 'REVIEW-TIMER-005', + 'REVIEW-SKIP-EMPTY-006', + 'REVIEW-GENERIC-MUTATION-007', + ], + ] as const; + for (const [index, path] of fixtureFiles.entries()) { + const ids = requiredFixtureIds[index]; + if (!ids) throw new Error(`Missing required fixture IDs for ${path}`); + writeFileSync( + join(rootDir, path), + ids.map((id) => fixtureCase(id, path.includes('positive') ? 'ACCEPT' : 'REJECT')).join('\n'), + ); + } + + return rootDir; +} + +describe('validateSkillAssets', () => { + const createdRoots: string[] = []; + + afterEach(() => { + for (const rootDir of createdRoots) { + rmSync(rootDir, { recursive: true, force: true }); + } + createdRoots.splice(0, createdRoots.length); + }); + + it('requires both canonical skill files and four fixture files', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + rmSync(join(rootDir, 'skills/relay-capture/SKILL.md')); + + expect(() => validateSkillAssets({ rootDir })).toThrow(/relay-capture\/SKILL\.md/i); + }); + + it('rejects malformed fixture cases', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync( + join(rootDir, 'skills/fixtures/capture-positive.md'), + '## CAPTURE-BROKEN-001\n\nExpected: ACCEPT\n', + ); + + expect(() => validateSkillAssets({ rootDir })).toThrow(/Scenario|Agent action|Reason/i); + }); + + it('requires the Capture policy sections and invariants', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync(join(rootDir, 'skills/relay-capture/SKILL.md'), '# Relay Capture\n'); + + expect(() => validateSkillAssets({ rootDir })).toThrow( + /frontmatter|Purpose|When to capture|MCP|INBOX/i, + ); + }); + + it('rejects vendor-specific canonical policy files', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + mkdirSync(join(rootDir, 'integrations/codex/relay-capture'), { recursive: true }); + writeFileSync(join(rootDir, 'integrations/codex/relay-capture/SKILL.md'), '# Relay Capture\n'); + + expect(() => validateSkillAssets({ rootDir })).toThrow(/canonical|vendor-specific/i); + }); + + it('accepts vendor skills that explicitly reference either canonical policy source', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + mkdirSync(join(rootDir, 'integrations/codex/relay-capture'), { recursive: true }); + mkdirSync(join(rootDir, 'integrations/codex/relay-session-review'), { recursive: true }); + writeFileSync( + join(rootDir, 'integrations/codex/relay-capture/SKILL.md'), + '# Relay Capture\n\nSee [canonical source](../../../../skills/relay-capture/SKILL.md).\n', + ); + writeFileSync( + join(rootDir, 'integrations/codex/relay-session-review/SKILL.md'), + '# Relay Session Review\n\nSee [canonical source](../../../../skills/relay-session-review/SKILL.md).\n', + ); + + expect(() => validateSkillAssets({ rootDir })).not.toThrow(); + }); + + it('requires caller-owned capture provenance and Relay-owned fields', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + const skillPath = join(rootDir, 'skills/relay-capture/SKILL.md'); + writeFileSync( + skillPath, + readFileSync(skillPath, 'utf-8').replace( + /The agent supplies createdByName and the exact active session ID\. Relay supplies createdByType: AGENT and status: INBOX\./, + 'Provide provenance for the capture.', + ), + ); + + expect(() => validateSkillAssets({ rootDir })).toThrow( + /createdByName|session ID|createdByType|INBOX|status/i, + ); + }); + + it('requires session review before completion even when the result is empty', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + const skillPath = join(rootDir, 'skills/relay-session-review/SKILL.md'); + writeFileSync( + skillPath, + readFileSync(skillPath, 'utf-8').replace( + 'Always perform the exact active session lookup before final completion.', + 'Review before final completion when captures may exist.', + ), + ); + + expect(() => validateSkillAssets({ rootDir })).toThrow( + /conditional|final completion|session lookup/i, + ); + }); + + it('rejects valid-looking fixtures that omit required issue coverage', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + writeFileSync( + join(rootDir, 'skills/fixtures/capture-positive.md'), + fixtureCase('CASE-001', 'ACCEPT'), + ); + + expect(() => validateSkillAssets({ rootDir })).toThrow( + /required coverage|CAPTURE-ACTIONABLE-001/i, + ); + }); + + it.each([ + ['capture', 'An agent may autonomously archive low-priority tasks.', /forbidden|archive/i], + [ + 'review', + 'Skip the exact session lookup when the agent believes there are no captures.', + /forbidden|lookup|skip/i, + ], + [ + 'capture', + 'The agent may parse decorative CLI output instead of JSON.', + /forbidden|decorative|JSON/i, + ], + [ + 'capture', + 'The agent may store full source files and secrets as context.', + /forbidden|source|secret/i, + ], + ])('rejects contradictory unsafe %s policy: %s', (skill, unsafePolicy, errorPattern) => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + const skillPath = join( + rootDir, + skill === 'capture' + ? 'skills/relay-capture/SKILL.md' + : 'skills/relay-session-review/SKILL.md', + ); + writeFileSync(skillPath, `${readFileSync(skillPath, 'utf-8')}\n${unsafePolicy}\n`); + + expect(() => validateSkillAssets({ rootDir })).toThrow(errorPattern); + }); + + it('allows canonical prohibition wording without treating it as permission', () => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + expect(() => validateSkillAssets({ rootDir })).not.toThrow(); + }); + + it.each([ + 'Do not skip the exact session lookup before final completion.', + 'The agent must not omit the exact session lookup before final completion.', + ])('allows exact-session prohibitions: %s', (prohibition) => { + const rootDir = createValidSkillFixtureRoot(); + createdRoots.push(rootDir); + const skillPath = join(rootDir, 'skills/relay-session-review/SKILL.md'); + writeFileSync(skillPath, `${readFileSync(skillPath, 'utf-8')}\n${prohibition}\n`); + + expect(() => validateSkillAssets({ rootDir })).not.toThrow(); + }); +});