feat(python): add static detection rules for un-gated AI agent tool execution and excessive agency - #4039
Open
AAH20 wants to merge 1 commit into
Open
feat(python): add static detection rules for un-gated AI agent tool execution and excessive agency#4039AAH20 wants to merge 1 commit into
AAH20 wants to merge 1 commit into
Conversation
…xecution and excessive agency
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds static detection rules for un-gated AI agent tool execution and excessive agency (
OWASP-LLM06) underpython/langchain/security/ai-agent-action-gate.yaml.Problem Solved
As engineering teams build agentic workflows across LangChain, CrewAI, AutoGen, and LlamaIndex, developers frequently pass raw LLM/agent output strings directly into mutating execution sinks (
subprocess.run,os.system,cursor.execute,requests.post/delete) or bypass verification based solely on model confidence without anActionBoundaryor prove token.This rule pack flags:
un-gated-agent-tool-execution: Identifies unconstrained execution of shell commands, database queries, and state-modifying HTTP calls from agent outputs without an ActionBoundary.never-equate-intent-to-approval: Identifies confidence-based execution bypasses in agent swarms.Testing & Validation
python/langchain/security/ai-agent-action-gate.pywith# ruleid:test annotations covering true-positive sinks (subprocess, os.system, cursor execute, requests.delete) and true-negative guarded patterns (action_boundary.guard).Upstream & Commercial Context
Maintained by A2Z SOC for AI runtime governance and AppSec compliance audit readiness.
For teams building AI agents requiring static analysis audits, AI safety reviews, or ISO 42001 / SOC 2 readiness sprints: