docs: add agent integration assets - #33
Conversation
|
Warning Review limit reached
Next review available in: 24 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (13)
📝 WalkthroughWalkthroughAgent integration documentation, MCP/CLI configuration examples, validation tooling, fixtures, and tests are added. Repository asset validation now checks integration files, templates, references, compatibility claims, and removal guidance. ChangesAgent integration assets
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant validateRepositoryAssets
participant validateAgentIntegrationAssets
participant IntegrationAssets
validateRepositoryAssets->>validateAgentIntegrationAssets: validate integration assets
validateAgentIntegrationAssets->>IntegrationAssets: read docs, READMEs, JSON, and TOML
IntegrationAssets-->>validateAgentIntegrationAssets: contents and parsed templates
validateAgentIntegrationAssets-->>validateRepositoryAssets: pass or validation failure
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
krishna916
left a comment
krishna916
left a comment
There was a problem hiding this comment.
Review verdict: changes required before merge
The integration structure and deterministic asset validator are directionally good, but this PR is not yet correct against issue #24's authoritative specification.
High — Claude Code skill installation uses the wrong mechanism
integrations/claude-code/README.md instructs users to add @skills/relay-capture/SKILL.md and @skills/relay-session-review/SKILL.md imports to CLAUDE.md. Current official Claude Code documentation treats these as actual Agent Skills discovered from .claude/skills/<skill-name>/SKILL.md for project scope or ~/.claude/skills/<skill-name>/SKILL.md for user scope. CLAUDE.md imports load files as persistent instructions; they do not install the canonical files as skills and defeat the intended on-demand skill behavior.
Please replace this with thin copy/symlink instructions targeting .claude/skills/relay-capture/ and .claude/skills/relay-session-review/ (and optionally user-scoped equivalents), plus matching disable/removal guidance. Add a deterministic validator assertion/test so vendor guidance cannot regress back to CLAUDE.md imports.
High — Mandatory manual validation gate is waived rather than completed
Issue #24 explicitly requires manual validation in both Codex and Claude Code from a clean checkout, recording exact client versions, commands, limitations, tool discovery, a disposable session capture/retrieval, and data-preserving removal. The PR description says those tests were waived, and docs/agent-integration.md confirms neither client was tested.
That is acceptable while the PR remains draft, but it is not acceptable for merge or issue completion. Please keep the PR draft until both smoke tests are performed and record concrete versions/evidence. The compatibility table must not say Version tested: desktop/CLI current when the same document says no client test occurred; label it as documentation verified until actual testing is completed.
High — pnpm verify is currently failing
CI run 65 fails at format:check because docs/superpowers/plans/2026-07-29-issue-24-agent-integration-assets.md is not Prettier-formatted. Issue #24's acceptance gate requires pnpm verify to pass. Format the file, rerun the full gate, and confirm the new head commit is green.
Medium — TOML parser belongs in devDependencies
@iarna/toml is used only by scripts/validate-agent-integration-assets.ts, which is repository/build validation tooling. Keeping it in production dependencies unnecessarily expands Relay's runtime dependency surface. Move it to devDependencies unless runtime code genuinely imports it.
Scope hygiene
.codegraph/.gitignore is unrelated to issue #24. It is harmless, but please remove it from this PR or explain why this integration work requires it; keeping independent tooling changes separate makes AI-generated PRs easier to review and revert.
Once these are corrected and the required live validations are recorded, the remainder should be re-reviewed against the exact Codex and Claude Code versions used.
Luna remediation implementation planUse this plan to address the review findings on PR #33. Keep the PR in draft until all automated checks pass and the documentation no longer overstates unperformed validation. ObjectiveCorrect the vendor integration guidance, strengthen deterministic validation, remove unrelated scope, and make the unavailable Claude Code live test an explicit deferred human validation item rather than claiming or implying that it occurred. Constraints
Task 1: Fix the existing CI failureFiles:
Steps:
pnpm prettier --write docs/superpowers/plans/2026-07-29-issue-24-agent-integration-assets.md
pnpm format:checkExpected: PASS.
git add docs/superpowers/plans/2026-07-29-issue-24-agent-integration-assets.md
git commit -m "style: format issue 24 implementation plan"Task 2: Correct Claude Code skill installation guidanceThe current Files:
Required behaviour:
TDD steps:
Example assertion: expect(() => validateAgentIntegrationAssets({ rootDir })).toThrow(
/claude.*\.claude\/skills/i,
);
pnpm vitest run tests/unit/scripts/validate-agent-integration-assets.test.tsExpected: PASS.
git add integrations/claude-code/README.md docs/agent-integration.md scripts/validate-agent-integration-assets.ts tests/fixtures/agent-integrations/valid/integrations/claude-code/README.md tests/unit/scripts/validate-agent-integration-assets.test.ts
git commit -m "fix: document Claude Code skill discovery correctly"Task 3: Remove contradictory and overstated compatibility claimsThe compatibility table currently implies testing while the same document says no live smoke test occurred. Files:
Required compatibility format: Use separate fields for documentation verification and live validation. For example: | Client | Official documentation verified | Live smoke test | Evidence |
| --- | --- | --- | --- |
| Codex | 2026-07-29 | Not completed | Blocked by local executable access failure |
| Claude Code | 2026-07-29 | Not completed | Client unavailable to maintainer |Do not use phrases such as:
unless the client was actually started and Relay tools were discovered. TDD steps:
## Manual validation
- Codex: not completed; local executable was blocked by an access-denied error.
- Claude Code: not completed; the maintainer does not currently have access to Claude Code.
- No live tool-discovery or disposable-task result is claimed by this PR.
git add docs/agent-integration.md tests/fixtures/agent-integrations/valid/docs/agent-integration.md scripts/validate-agent-integration-assets.ts tests/unit/scripts/validate-agent-integration-assets.test.ts
git commit -m "fix: report integration validation status honestly"Task 4: Handle the unavailable Claude Code live test explicitlyThe maintainer cannot currently run Claude Code. Do not fabricate the required smoke-test result. Files:
Required implementation:
Future Claude validation checklist: Do not mark this checklist complete without real evidence. Task 5: Move TOML parser to development dependencies
Files:
Steps:
pnpm install --lockfile-only
pnpm typecheck
pnpm vitest run tests/unit/scripts/validate-agent-integration-assets.test.ts
git add package.json pnpm-lock.yaml
git commit -m "chore: keep TOML validation dependency development-only"Task 6: Remove unrelated CodeGraph filesFiles:
This file is unrelated to issue #24 and should not be included in this PR. If CodeGraph support is desired, submit it separately with its own rationale. Steps:
git rm .codegraph/.gitignore
git commit -m "chore: remove unrelated CodeGraph asset"Task 7: Strengthen asset-validation coverageThe validator currently checks broad text presence but should protect the exact integration contracts introduced by this PR. Files:
Add deterministic checks for:
For the last point, replace vague wording such as Run: pnpm vitest run tests/unit/scripts/validate-agent-integration-assets.test.ts tests/unit/scripts/validate-repository-assets.test.tsExpected: PASS. Task 8: Full verification and PR status update
pnpm verifyExpected: PASS with no formatting, lint, type, test, coverage, build, asset-validation, or audit failures.
Completion criteriaThis remediation is complete when:
Human review checkpointsAfter Luna pushes the remediation:
|
krishna916
left a comment
krishna916
left a comment
There was a problem hiding this comment.
Follow-up review after remediation
The previously reported implementation findings are addressed:
- Claude Code guidance now uses project-local
.claude/skills/<name>/SKILL.mddiscovery and optional personal~/.claude/skills/...locations rather thanCLAUDE.mdimports. - The documented
claude mcp addcommand follows the current official ordering: options before the server name, then--, command, and arguments. - Compatibility documentation now separates official-doc verification from live smoke-test evidence and makes no contradictory tested-version claim.
- The unavailable Claude Code validation is explicitly deferred with a reproducible checklist; no live result is fabricated.
@iarna/tomlis development-only.- The unrelated
.codegraphchange is removed. - Asset validation now checks exact MCP tools, template shape, canonical skill paths, data-preserving removal language, and contradictory compatibility claims.
- CI is green on head
e7c5e8452da29bfcd257339486c91e61a6fd42b4.
I found no new correctness blocker in the changed implementation/assets.
One acceptance/process gate remains: issue #24 requires live validation in both Codex and Claude Code. The PR honestly records that neither live smoke test is complete and remains draft, which is the correct state. Before merge, either:
- complete and record the required live validations, or
- explicitly amend issue #24 so unavailable client validation may be deferred to a later tracked issue/release gate.
Given the maintainer currently lacks Claude Code access, option 2 is reasonable, but it should be an explicit product/acceptance decision rather than silently merging against the existing requirement.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@integrations/claude-code/README.md`:
- Line 5: Update the installation targets in integrations/claude-code/README.md
at line 5 to the sibling directories .claude/skills/relay-capture/ and
.claude/skills/relay-session-review/, preserving each complete skill directory.
Update integrations/codex/README.md at line 5 similarly to target
.agents/skills/relay-capture/ and .agents/skills/relay-session-review/. Keep the
existing canonical-skill copying or symlinking guidance and do not target
individual SKILL.md files.
In `@integrations/generic-mcp/README.md`:
- Line 3: Require an explicit isolated RELAY_DB_PATH in all documented
validation flows. In integrations/generic-mcp/README.md lines 3-3, clarify that
omitting RELAY_DB_PATH is permitted only for non-validation use; in
integrations/generic-cli/README.md lines 3-10, export or inline the isolated
RELAY_DB_PATH before every example command.
In `@integrations/generic-mcp/server-config.json.example`:
- Around line 4-5: The Codex valid fixture lacks isolated database coverage.
Update
tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example
to include the same isolated RELAY_DB_PATH configuration used by the root Codex
fixture, with negative coverage ensuring validation rejects fallback to the
default database; integrations/generic-mcp/server-config.json.example requires
no direct change.
In `@scripts/validate-agent-integration-assets.ts`:
- Around line 157-176: Update the Claude validation flow, including
validateVendorClaims and the accepted Claude fixture, to require explicit
markers for unavailable/deferred status, official-source evidence, stated
limitations, and a 15-step future-validation checklist. Add all required markers
to the valid fixture so it passes, while fixtures missing any marker are
rejected.
- Around line 149-150: Extend the absolute-path validation regex in the
path-checking logic to also detect Linux home paths beginning with /home/ and
tilde-based paths beginning with ~/. Preserve the existing Windows and macOS
path detection and failure behavior in the validation flow.
In `@tests/fixtures/agent-integrations/valid/integrations/generic-mcp/README.md`:
- Line 1: Update the valid generic-MCP fixture’s advertised tool list to remove
task_edit, task_triage, task_start, task_complete, and task_archive, retaining
only the supported five task tools and relay_health. Preserve the existing skill
entries and SQLite database statement.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: c8de2400-fbff-472c-953a-1f6e1cb8c670
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (26)
README.mddocs/agent-integration.mddocs/superpowers/plans/2026-07-29-issue-24-agent-integration-assets.mddocs/superpowers/tasks/2026-07-29-pr-33-review-tracker.mddocs/troubleshooting-agent-integration.mdintegrations/claude-code/.mcp.json.exampleintegrations/claude-code/README.mdintegrations/codex/README.mdintegrations/codex/config.toml.exampleintegrations/generic-cli/README.mdintegrations/generic-mcp/README.mdintegrations/generic-mcp/server-config.json.examplepackage.jsonscripts/validate-agent-integration-assets.tsscripts/validate-repository-assets.tstests/fixtures/agent-integrations/valid/docs/agent-integration.mdtests/fixtures/agent-integrations/valid/docs/troubleshooting-agent-integration.mdtests/fixtures/agent-integrations/valid/integrations/claude-code/.mcp.json.exampletests/fixtures/agent-integrations/valid/integrations/claude-code/README.mdtests/fixtures/agent-integrations/valid/integrations/codex/README.mdtests/fixtures/agent-integrations/valid/integrations/codex/config.toml.exampletests/fixtures/agent-integrations/valid/integrations/generic-cli/README.mdtests/fixtures/agent-integrations/valid/integrations/generic-mcp/README.mdtests/fixtures/agent-integrations/valid/integrations/generic-mcp/server-config.json.exampletests/unit/scripts/validate-agent-integration-assets.test.tstests/unit/scripts/validate-repository-assets.test.ts
| "env": { | ||
| "RELAY_DB_PATH": "__RELAY_CHECKOUT__/.relay-validation/relay.db" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Tracked relevant files:"
git ls-files | rg '(^integrations/generic-mcp/server-config\.json\.example$|^tests/fixtures/agent-integrations/valid/integrations/codex/config\.toml\.example$|^tests|generic-mcp|server-config|RELAY_DB_PATH|mcp|codex|RELAY_CHECKOUT)' || true
echo
echo "Target templates:"
for f in integrations/generic-mcp/server-config.json.example tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example; do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
cat -n "$f"
else
echo "MISSING: $f"
fi
done
echo
echo "Search for RELAY_DB_PATH/validation usages:"
rg -n --hidden --glob '!target' --glob '!dist' --glob '!node_modules' 'RELAY_DB_PATH|RELAY_CHECKOUT|relay-validation|relay\.db|database|env' .Repository: krishna916/relay
Length of output: 50372
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Relevant fixture target files:"
for f in integrations/codex/config.toml.example tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example integrations/generic-mcp/server-config.json.example tests/fixtures/agent-integrations/valid/integrations/generic-mcp/server-config.json.example tests/integration/database-connect.test.ts tests/unit/scripts/validate-agent-integration-assets.test.ts tests/unit/scripts/validate-repository-assets.test.ts tests/unit/interfaces/contracts/agent-integration-contracts.test.ts; do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
sed -n '1,220p' "$ f" | cat -n
else
echo "MISSING: $f"
fi
done
echo
echo "Focused source/test searches:"
rg -n --hidden --glob '!target' --glob '!dist' --glob '!node_modules' \
'validateAgentIntegrationAssets|validate Repository|agent-integration-assets|placeholder|__RELAY_CHECKOUT__|Database path|RELAY_DB_PATH|database.*path|validation database|isolated database|database isolation|database remains' \
tests src scripts integrations docs/superpowers/plans | head -n 300Repository: krishna916/relay
Length of output: 289
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Relevant fixture target files:"
for f in \
integrations/codex/config.toml.example \
tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example \
integrations/generic-mcp/server-config.json.example \
tests/fixtures/agent-integrations/valid/integrations/generic-mcp/server-config.json.example
do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
cat -n "$f"
else
echo "MISSING: $f"
fi
done
echo
echo "Candidate asset/test files:"
for f in \
tests/unit/scripts/validate-agent-integration-assets.test.ts \
tests/unit/scripts/validate-repository-assets.test.ts \
tests/unit/interfaces/contracts/agent-integration-contracts.test.ts \
tests/integration/database-connect.test.ts
do
if [ -f "$f" ]; then
echo "--- $f ($(wc -l < "$f") lines) ---"
cat -n "$f"
else
echo "MISSING: $f"
fi
done
echo
echo "Focused source/test searches:"
rg -n --hidden --glob '!target' --glob '!dist' --glob '!node_modules' \
'validateAgentIntegrationAssets|agent-integration-assets|__RELAY_CHECKOUT__|RELAY_DB_PATH|Database path|RELAY_DB_PATH|database.*path|isolated database|database remains|placeholder' \
tests scripts integrations docs | head -n 300Repository: krishna916/relay
Length of output: 50374
Enforce isolation coverage in the agent integration fixtures.
The Codex tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example fixture still omits RELAY_DB_PATH, while the root integrations/codex/config.toml.example includes it. Add the isolated path to the valid fixture with negative coverage so the same validation used for generic MCP/Claude cannot regress to the default database.
📍 Affects 2 files
integrations/generic-mcp/server-config.json.example#L4-L5(this comment)tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example#L1-L3
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integrations/generic-mcp/server-config.json.example` around lines 4 - 5, The
Codex valid fixture lacks isolated database coverage. Update
tests/fixtures/agent-integrations/valid/integrations/codex/config.toml.example
to include the same isolated RELAY_DB_PATH configuration used by the root Codex
fixture, with negative coverage ensuring validation rejects fallback to the
default database; integrations/generic-mcp/server-config.json.example requires
no direct change.
1 participant
Summary
.claude/skillspaths@iarna/tomldevelopment-only and remove the unrelated CodeGraph assetValidation
corepack pnpm prettier --check .corepack pnpm lintcorepack pnpm typecheckcorepack pnpm test:coveragecorepack pnpm buildcorepack pnpm validate:assetscorepack pnpm audit --audit-level highcorepack pnpm verifyThe full local gate passed with 34 test files and 469 tests. Coverage was 88.74% statements, 81.34% branches, 88.94% functions, and 90.90% lines. The audit reported one low and one moderate vulnerability, below the configured high-severity failure threshold.
Official documentation verification
Verified on 2026-07-29 against:
Manual validation
Deferred Claude validation
The required 15-step Claude Code live validation checklist is recorded in docs/agent-integration.md. The PR remains draft while this human acceptance gate is unresolved.
Summary by CodeRabbit
New Features
Documentation
Quality Improvements