Tighten the security reviewer and stop the Rails expert hanging on the console - #141
Draft
mattmenefee wants to merge 2 commits into
Draft
mattmenefee wants to merge 2 commits into
mattmenefee wants to merge 2 commits into
Conversation
The reviewer tagged its categories with OWASP Top 10:2021 numbers, which the 2025 edition reshuffled, so it was attaching the wrong standards reference to real findings. It also had no category for server-side request forgery, a common class in Rails apps that fetch user-supplied URLs for webhooks, link previews and file generators. Its report format was underspecified in two ways that made output drift between runs. Only critical issues were given the Location, Vulnerability, Risk and Remediation fields, leaving it unclear whether lower severities used them. And a single critical issue satisfied both NEEDS CHANGES and BLOCKED, with nothing saying which to emit. Category headings now carry names only, which do not go stale with each edition, and an SSRF category covers outbound requests built from user input, including those that reach internal or cloud metadata addresses. Every finding uses the same four fields, the approval statuses are disjoint so the most severe finding alone decides, and the severity icons lead their headings to match /doc-review.
The agent told itself to "use Rails console effectively" when debugging. Without a TTY the console waits for input that never arrives, so following that instruction hangs the tool call, which the global instructions already warn against. It also listed version-gated features such as load_async, strict loading and horizontal sharding with no reminder that a project may run an older Rails, unlike the PostgreSQL expert, which checks the version before recommending anything version-dependent. Debugging now starts from bin/rails runner or a one-off script, and the agent checks the app's Rails and Ruby versions before recommending a version-gated feature.
mattmenefee
force-pushed
the
fix-security-and-rails-agent-docs
branch
from
September 17, 2026 03:26
ec43145 to
e5b94ec
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/doc-review.bin/rails runneror a one-off script instead of the interactive Rails console, which hangs without a TTY. The agent now checks the app's Rails and Ruby versions before recommending version-gated features, as the PostgreSQL expert already does.These come from a documentation review of the Claude configuration files. More fixes from that review, mostly in
update-deps.md, may land on this branch before it leaves draft.Test plan
YAML.safe_loadon the frontmatter block)OWASP A0xtags remain insecurity-reviewer.md, and no "Rails console effectively" instruction remains inruby-on-rails-expert.mdbin/rails runnerrather than the console