test: verify MCP and CLI compatibility and integrations

docs/agent-integration-verification.md

@@ -0,0 +1,90 @@
+# Agent Integration Verification
+
+## Scope and safety statement
+
+This evidence covers the source-checkout Relay MCP and CLI adapters, their shared task contract, the HTTP database path, and the canonical integration assets for issue #25. All automated scenarios use a fresh disposable database under the repository's `tmp/` directory and arbitrary disposable working directories. No test invokes an external LLM, reads or writes the default Relay database, or changes real Codex, Claude Code, or other client configuration. MCP diagnostics are captured from stderr; protocol data remains on stdout.
+
+## Automated scenario matrix
+
+| Scenario                                                            | Automated test or manual step                                                | Result | Evidence                                                                                          |
+| ------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------- |
+| 1. MCP capture then CLI retrieval                                   | `mcp-cli-parity.test.ts` — MCP capture followed by built CLI get             | PASS   | Built `dist/mcp/main.js` and `dist/cli/main.js`, same disposable `RELAY_DB_PATH`                  |
+| 2. CLI capture then MCP retrieval                                   | `mcp-cli-parity.test.ts` — CLI capture followed by MCP get                   | PASS   | Complete public task DTO equality                                                                 |
+| 3. Task list/get fields and ordering                                | `mcp-cli-parity.test.ts` — list DTO comparison and persisted ordering        | PASS   | Transport-only envelope normalization                                                             |
+| 4. Session-ID isolation                                             | `agent-workflow-e2e.test.ts` — alpha and beta captures                       | PASS   | Exact session filtering                                                                           |
+| 5. Completed and archived session review                            | `agent-workflow-e2e.test.ts` — all-status review                             | PASS   | Open, DONE, and ARCHIVED captures returned                                                        |
+| 6. Missing session behavior                                         | `mcp-cli-parity.test.ts` and existing adapter contract tests                 | PASS   | Stable missing-task/session contract coverage                                                     |
+| 7. Malformed session behavior                                       | `mcp-cli-parity.test.ts` and existing strict schema tests                    | PASS   | MCP validation boundary and CLI validation envelope                                               |
+| 8. Duplicate candidates, warnings, and match reasons                | `mcp-cli-parity.test.ts` — duplicate capture and find-similar                | PASS   | Advisory warning and deterministic candidate assertions                                           |
+| 9. Edit parity                                                      | `mcp-cli-parity.test.ts` — CLI edit and MCP readback                         | PASS   | Complete task and change metadata                                                                 |
+| 10. Triage/start/complete/archive parity                            | `mcp-cli-parity.test.ts` — cross-adapter lifecycle sequence                  | PASS   | Focused lifecycle actions and statuses                                                            |
+| 11. No-op metadata                                                  | `mcp-cli-parity.test.ts` — repeated edit                                     | PASS   | `NO_CHANGE`, empty fields, unchanged timestamps                                                   |
+| 12. Validation, transition, archived, not-found, and storage errors | `mcp-cli-parity.test.ts` — stable errors and unusable parent path            | PASS   | CLI envelopes; MCP execution/protocol errors; startup failure remains stderr-only; leakage checks |
+| 13. CLI JSON schemas and exit codes                                 | `cli-test-process.test.ts` and built parity tests                            | PASS   | One JSON document, separated stderr, exit codes 0/2/3/4/5                                         |
+| 14. One database across HTTP, MCP, and CLI                          | `database-path-parity.test.ts`                                               | PASS   | Arbitrary CWDs, HTTP runtime, restart persistence, no CWD-local DB                                |
+| 15. Skill and vendor-wrapper drift                                  | `validate-agent-integration-assets.test.ts` and `validate:assets`            | PASS   | 33 validator tests; canonical policy and entry-point checks                                       |
+| 16. Integration removal preserves data                              | `agent-workflow-e2e.test.ts` config-driven disposable MCP launch and removal | PASS   | Parsed `.mcp.json` launches the built server; removal is followed by retrieval from the same DB   |
+
+## Clean-checkout environment
+
+- OS: Microsoft Windows NT 10.0.26200.0
+- Node: v24.18.0
+- pnpm: 10.2.0 through Corepack (`corepack pnpm --version`); direct global pnpm was 11.9.0 and was not used for authoritative final commands.
+- Branch: `feature/issue-25-mcp-cli-compatibility`
+- Verification base SHA: `e51307d066e70b95f8072ac52d2679b8e63c5244`
+- Database strategy: each test calls `createAgentTestRuntime()` and uses a unique disposable `<root>/data/relay.db`; client CWDs are created below the same disposable root.
+
+Exact setup and validation commands:
+
+```text
+corepack pnpm install --frozen-lockfile
+corepack pnpm build
+corepack pnpm exec vitest run tests/integration/mcp-cli-parity.test.ts
+corepack pnpm exec vitest run tests/integration/agent-workflow-e2e.test.ts
+corepack pnpm exec vitest run tests/integration/database-path-parity.test.ts
+corepack pnpm validate:assets
+corepack pnpm verify
+git status --short
+```
+
+The disposable strategy and assertions confirm that the default database and real client configuration were not touched.
+
+## Codex validation
+
+Live validation is unverified. `Get-Command codex` resolved the installed desktop executable, but `codex --version` failed with Windows `Access is denied`. No Codex process was started, no MCP discovery result is claimed, and no Codex configuration or profile was changed.
+
+The required isolated workflow remains:
+
+1. Use a clean checkout and `corepack pnpm install --frozen-lockfile`.
+2. Run `corepack pnpm build`.
+3. Create an isolated profile/configuration and disposable `RELAY_DB_PATH`.
+4. Add the canonical `node <checkout>/dist/mcp/main.js` server using the documented Codex configuration.
+5. Install the canonical skills unchanged under `.agents/skills/`.
+6. Restart Codex, discover Relay tools, capture two follow-ups with one session ID, review that session, perform one explicitly directed lifecycle action, use the CLI JSON fallback, remove only config/skill references, and confirm data remains.
+
+No step above is represented as executed in this environment.
+
+## Claude Code validation
+
+Live validation is unverified. `Get-Command claude` and `Get-Command claude-code` returned no executable. No Claude configuration was edited and no client result is claimed.
+
+The equivalent isolated workflow is documented in `integrations/claude-code/README.md`: project-scoped stdio configuration, canonical `.claude/skills/` directories, tool discovery, two same-session captures, exact session review, one explicitly directed mutation, CLI fallback, configuration-only removal, and post-removal data retrieval.
+
+## Cross-client differences and limitations
+
+The automated contract is client-neutral: built MCP uses protocol-owned stdout and CLI uses one JSON envelope plus stable exit codes. Codex and Claude Code syntax, skill-discovery locations, and availability could not be exercised live here. The issue #24 documentation records the official-source verification date and the current unavailable-client limitations.
+
+## Data and configuration preservation
+
+Automated tests only create disposable files under the test runtime root. They remove temporary configuration fixtures and restart against the same database, then retrieve persisted tasks. They never remove a database to disable an integration. The validator rejects removal guidance that deletes SQLite data and requires explicit configuration-only wording.
+
+## Epic #2 closure checklist
+
+- [x] Issues #19, #20, #21, #22, #23, #24, and #26 are closed on GitHub and their required artifacts exist locally.
+- [x] Built MCP and CLI entry points are exercised from arbitrary working directories.
+- [x] MCP stdout remains protocol-clean; diagnostics are stderr-only.
+- [x] Automated contract, lifecycle, session, error, storage, restart, shared-path, and asset checks pass.
+- [x] Default database, real client configuration, and external LLMs were not touched by automation.
+- [ ] Live Codex workflow — blocked by executable access denied; human review required.
+- [ ] Live Claude Code workflow — blocked because the client is unavailable; human review required.
+- [ ] Human reviewer must inspect cleanup, normalizers, source-context safety, and one independent client workflow before merging.

docs/superpowers/plans/2026-07-29-issue-25-mcp-cli-compatibility-verification.md

@@ -668,12 +668,19 @@ Use these exact top-level sections:
 # Agent Integration Verification
 
 ## Scope and safety statement
+
 ## Automated scenario matrix
+
 ## Clean-checkout environment
+
 ## Codex validation
+
 ## Claude Code validation
+
 ## Cross-client differences and limitations
+
 ## Data and configuration preservation
+
 ## Epic #2 closure checklist
 ```
 

integrations/claude-code/README.md

@@ -4,6 +4,6 @@ Build Relay and substitute an absolute checkout path. Add the stdio server with
 
 Install the canonical [Relay Capture](../../skills/relay-capture/SKILL.md) and [Relay Session Review](../../skills/relay-session-review/SKILL.md) skill directories by copying or symlinking them unchanged to `.claude/skills/relay-capture/` and `.claude/skills/relay-session-review/`. For a personal installation across projects, use the client’s documented user-scoped skills directory. Do not copy the policy text into Claude-specific documentation or use instruction-file imports as skill discovery.
 
-Use `claude mcp list`, `claude mcp get relay`, and `/mcp` to validate and authorize the server. Confirm `relay_health`, the task tools, a disposable capture, and the exact session lookup when live validation is available. Remove it with `claude mcp remove relay` or by deleting the Relay entry and skill directories; the SQLite database remains untouched.
+Use `claude mcp list`, `claude mcp get relay`, and `/mcp` to validate and authorize the server. Confirm `relay_health`, the task tools, a disposable capture, and the exact session lookup when live validation is available. Remove only the client configuration with `claude mcp remove relay` or by deleting the Relay entry and skill directories; the SQLite database remains untouched.
 
 For current syntax and skill discovery behavior, see the [Claude Code MCP documentation](https://code.claude.com/docs/en/mcp) and [Claude Code skills documentation](https://code.claude.com/docs/en/skills).

integrations/generic-cli/README.md

@@ -12,4 +12,4 @@ node __RELAY_CHECKOUT__/dist/cli/main.js task complete TASK_ID --output json
 node __RELAY_CHECKOUT__/dist/cli/main.js task archive TASK_ID --output json
 ```
 
-Exit codes are documented in [the CLI reference](../../docs/cli-reference.md). Capture can be autonomous; edit, triage, start, complete, and archive require explicit user direction. See [Relay Capture](../../skills/relay-capture/SKILL.md) and [Relay Session Review](../../skills/relay-session-review/SKILL.md). Removing a client integration does not delete data; the SQLite database remains untouched.
+Exit codes are documented in [the CLI reference](../../docs/cli-reference.md). See [Relay Capture](../../skills/relay-capture/SKILL.md) and [Relay Session Review](../../skills/relay-session-review/SKILL.md) for behavioural policy. Removing a client integration does not delete data; the SQLite database remains untouched.

scripts/validate-agent-integration-assets.ts

@@ -75,6 +75,14 @@ function validateCompatibilityClaims(shared: string): void {
 function validateVendorClaims(rootDir: string, shared: string): void {
   for (const readme of vendorReadmes) {
     const text = readAsset(rootDir, `integrations/${readme}/README.md`);
+    if (
+      /^## Autonomy boundaries$/im.test(text) ||
+      /autonomously\s+(?:edit|triage|start|complete|archive|delete|merge)/i.test(text)
+    ) {
+      fail(
+        `${readme} must reference canonical behavioural policy instead of redefining mutation autonomy.`,
+      );
+    }
     const claimsNoLiveTest = /(?:live smoke test|live validation)[^\n]*not completed/i.test(text);
     const claimsLiveEvidence =
       /(?:live|manual)[^\n]*(?:tested|performed|verified|discovered|captured)/i.test(text);
@@ -99,6 +107,48 @@ function validateVendorClaims(rootDir: string, shared: string): void {
   }
 }
 
+function validateCanonicalSkills(rootDir: string): void {
+  const capture = readAsset(rootDir, canonicalSkills[0]);
+  if (!/autonomously create only a new Relay task in `?INBOX`?/i.test(capture))
+    fail('Canonical capture skill must define autonomous creation as INBOX-only.');
+  if (
+    !/must not edit, triage, start, complete, archive, delete, merge, or move any task/i.test(
+      capture,
+    )
+  )
+    fail('Canonical capture skill must prohibit autonomous lifecycle mutation.');
+  if (/autonomously\s+(?:edit|triage|start|complete|archive|delete|merge)/i.test(capture))
+    fail('Canonical capture skill must not grant autonomous lifecycle mutation.');
+
+  const review = readAsset(rootDir, canonicalSkills[1]);
+  if (!/completed and archived captures/i.test(review))
+    fail(
+      'Canonical session-review skill must require all-status retrieval including completed and archived captures.',
+    );
+  if (!/explicit user direction/i.test(review))
+    fail('Canonical session-review skill must require explicit user direction for mutations.');
+}
+
+function validateRemovalGuidance(rootDir: string): void {
+  for (const readme of vendorReadmes) {
+    const text = readAsset(rootDir, `integrations/${readme}/README.md`);
+    if (
+      /(?:remove|delete)\s+(?:the\s+)?(?:SQLite\s+)?database\b|(?:SQLite\s+)?database\b[^\n]{0,80}\b(?:remove|delete)\b/i.test(
+        text,
+      )
+    )
+      fail(`${readme} removal guidance must not delete the SQLite database.`);
+    if (
+      !/(?:remov(?:e|ing)|delete)[\s\S]{0,160}(?:configuration|integration|client assets?)/i.test(
+        text,
+      )
+    )
+      fail(`${readme} removal guidance must distinguish configuration removal from stored data.`);
+    if (!/SQLite database remains untouched/i.test(text))
+      fail(`${readme} removal guidance must state that the SQLite database remains untouched.`);
+  }
+}
+
 function validateTemplateShape(rootDir: string): void {
   const expectedMcpPath = '/tmp/relay-checkout/dist/mcp/main.js';
   const jsonTemplates = [
@@ -121,7 +171,7 @@ function validateTemplateShape(rootDir: string): void {
       fail(`${path} must separate command and arguments for a stdio server.`);
     }
     if (server.args[0] !== expectedMcpPath) {
-      fail(`${path} must use the exact dist/mcp/main.js entry path.`);
+      fail(`${path} must use the canonical dist/mcp/main.js entry path.`);
     }
     if (typeof server.command !== 'string' || /[\\/\s]/.test(server.command)) {
       fail(`${path} must not embed a shell command in command.`);
@@ -149,7 +199,7 @@ function validateTemplateShape(rootDir: string): void {
     codexServer.args[0] !== expectedMcpPath
   ) {
     fail(
-      'integrations/codex/config.toml.example must use node plus dist/mcp/main.js as separate fields.',
+      'integrations/codex/config.toml.example must use node plus the canonical dist/mcp/main.js entry as separate fields.',
     );
   }
   if (codexServer.env?.RELAY_DB_PATH !== '/tmp/relay-checkout/.relay-validation/relay.db') {
@@ -164,6 +214,9 @@ export function validateAgentIntegrationAssets(
   for (const path of requiredPaths) {
     if (!existsSync(join(rootDir, path))) fail(`Required path missing: ${path}`);
   }
+  for (const path of canonicalSkills) {
+    if (!existsSync(join(rootDir, path))) fail(`Canonical skill path missing: ${path}`);
+  }
 
   const integrationRoot = join(rootDir, 'integrations');
   const contents = filesUnder(integrationRoot)
@@ -173,6 +226,8 @@ export function validateAgentIntegrationAssets(
   const troubleshooting = readAsset(rootDir, 'docs/troubleshooting-agent-integration.md');
   const all = `${contents}\n${shared}\n${troubleshooting}`;
 
+  validateCanonicalSkills(rootDir);
+
   if (/(?:[A-Z]:[\\/]Users[\\/]|\/Users\/|\/home\/|~\/)[^\s"'`]+/i.test(all))
     fail('Machine-specific absolute path found.');
   for (const skill of canonicalSkills) {
@@ -220,11 +275,7 @@ export function validateAgentIntegrationAssets(
     fail('Claude README must preserve complete canonical skill directories unchanged.');
   if (/^## Autonomy boundaries$/m.test(contents))
     fail('Vendor assets must not copy behavioural policy.');
-  for (const readme of vendorReadmes) {
-    const text = readAsset(rootDir, `integrations/${readme}/README.md`);
-    if (!/SQLite database remains untouched/i.test(text))
-      fail(`${readme} removal guidance must state that the SQLite database remains untouched.`);
-  }
+  validateRemovalGuidance(rootDir);
   for (const match of all.matchAll(/relay mcp/gi)) {
     const context = all.slice(Math.max(0, match.index! - 80), match.index! + 100);
     if (!/(future|not available|Epic #18)/i.test(context))

tests/fixtures/agent-integrations/valid/integrations/claude-code/README.md

@@ -1 +1 @@
-skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md .claude/skills/relay-capture/ .claude/skills/relay-session-review/ Copy or symlink the complete canonical skill directories and preserve each SKILL.md unchanged. SQLite database remains untouched.
+skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md .claude/skills/relay-capture/ .claude/skills/relay-session-review/ Copy or symlink the complete canonical skill directories and preserve each SKILL.md unchanged. Remove only the client configuration and skill references; the SQLite database remains untouched.

tests/fixtures/agent-integrations/valid/integrations/codex/README.md

@@ -1 +1 @@
-skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md .agents/skills/relay-capture/ .agents/skills/relay-session-review/ SQLite database remains untouched.
+skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md .agents/skills/relay-capture/ .agents/skills/relay-session-review/ Remove only the client configuration and skill references; the SQLite database remains untouched.

tests/fixtures/agent-integrations/valid/integrations/generic-cli/README.md

@@ -1 +1 @@
-skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md `export RELAY_DB_PATH="__RELAY_CHECKOUT__/.relay-validation/relay.db"` SQLite database remains untouched.
+skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md `export RELAY_DB_PATH="__RELAY_CHECKOUT__/.relay-validation/relay.db"` Remove only the client configuration; the SQLite database remains untouched.

tests/fixtures/agent-integrations/valid/integrations/generic-mcp/README.md

@@ -1 +1 @@
-skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md Validation requires explicit isolated RELAY_DB_PATH; omission is permitted only for non-validation use. relay_health task_capture task_list task_get task_find_similar session_captures_list SQLite database remains untouched.
+skills/relay-capture/SKILL.md skills/relay-session-review/SKILL.md Validation requires explicit isolated RELAY_DB_PATH; omission is permitted only for non-validation use. Remove only the client configuration; the SQLite database remains untouched. relay_health task_capture task_list task_get task_find_similar session_captures_list SQLite database remains untouched.