feat(platform): dry-run execution mode with LLM block simulation - #12483
Merged
Conversation
majdyz
requested review from
0ubbe and
Bentlybro
and removed request for
a team
March 19, 2026 14:27
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds an LLM-driven dry-run simulator for block execution, a Changes
Sequence DiagramsequenceDiagram
participant Client as Client
participant Tool as RunBlockTool
participant Exec as execute_block
participant Sim as simulate_block
participant LLM as LLM
participant Block as Block
Client->>Tool: Invoke RunBlockTool(dry_run=true)
Tool->>Exec: execute_block(..., dry_run=true)
alt Dry-run
Exec->>Sim: simulate_block(block, inputs)
Sim->>LLM: Chat completion (prompt with schemas + inputs)
LLM-->>Sim: JSON response
Sim->>Sim: Parse, validate, retry if needed
Sim-->>Exec: yield (pin, value) tuples
Exec-->>Client: BlockOutputResponse (message contains "[DRY RUN]", outputs aggregated)
else Live execution
Exec->>Block: block.execute(...)
Block-->>Exec: real outputs
Exec-->>Client: BlockOutputResponse (real run)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early. 🔴 Merge Conflicts DetectedThe following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.
🟢 Low Risk — File Overlap OnlyThese PRs touch the same files but different sections (click to expand)
Summary: 2 conflict(s), 0 medium risk, 3 low risk (out of 5 PRs with file overlap) Auto-generated on push. Ignores: |
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py (1)
233-243: Verify the patch target matches the lazy import behavior.The patch targets
backend.executor.simulator.simulate_block, whilehelpers.pydoes a lazy import insideexecute_block. Since the import happens fresh each time the function runs, patching at the source module level should work correctly. However, if the import behavior changes (e.g., module-level import), this test would break silently.Consider adding a comment explaining why patching the source module works here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py` around lines 233 - 243, The test patches backend.executor.simulator.simulate_block while execute_block performs a lazy import inside helpers.py; add a short comment above the patch (or near the test setup) explaining that execute_block imports simulate_block at call time so patching the source module is intentional and will intercept the lazy import, and note that if execute_block later moves to a module-level import the test must be updated to patch the new import site; reference simulate_block, execute_block, helpers.py, and fake_simulate in the comment for clarity.autogpt_platform/backend/backend/executor/simulator.py (1)
28-36: Consider extending truncation to nested structures.The
_truncate_input_valuesfunction only truncates top-level string values. Deeply nested structures with large strings won't be truncated, potentially causing prompt size issues.This is acceptable for the initial implementation, but consider extending this in the future if users encounter prompt-size issues with complex nested inputs.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/executor/simulator.py` around lines 28 - 36, _truncate_input_values currently only trims top-level strings; update it to recursively walk nested structures (dicts, lists/tuples, sets) and truncate any string longer than _MAX_INPUT_VALUE_CHARS, preserving non-string types and keys. Implement a helper (e.g., _truncate_value) that checks types, truncates long strings, maps dict values by calling itself, iterates and rebuilds lists/tuples/sets with truncated items, and return the original value for unsupported types; call this helper from _truncate_input_values and keep using the _MAX_INPUT_VALUE_CHARS constant.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py`:
- Around line 345-346: The assertion currently checks the same substring twice;
update the assertion in test_dry_run.py to verify both common patterns instead
of duplicating: ensure it asserts that either 'kwargs.get("dry_run")' or
"kwargs.get('dry_run')" appears in source (replace the duplicated
`'kwargs.get("dry_run"' or-or with the two distinct patterns) so the test
catches both double-quoted and single-quoted usages.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py`:
- Around line 233-243: The test patches
backend.executor.simulator.simulate_block while execute_block performs a lazy
import inside helpers.py; add a short comment above the patch (or near the test
setup) explaining that execute_block imports simulate_block at call time so
patching the source module is intentional and will intercept the lazy import,
and note that if execute_block later moves to a module-level import the test
must be updated to patch the new import site; reference simulate_block,
execute_block, helpers.py, and fake_simulate in the comment for clarity.
In `@autogpt_platform/backend/backend/executor/simulator.py`:
- Around line 28-36: _truncate_input_values currently only trims top-level
strings; update it to recursively walk nested structures (dicts, lists/tuples,
sets) and truncate any string longer than _MAX_INPUT_VALUE_CHARS, preserving
non-string types and keys. Implement a helper (e.g., _truncate_value) that
checks types, truncates long strings, maps dict values by calling itself,
iterates and rebuilds lists/tuples/sets with truncated items, and return the
original value for unsupported types; call this helper from
_truncate_input_values and keep using the _MAX_INPUT_VALUE_CHARS constant.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 35b3ff00-6416-4e40-b92a-8b3743697d5e
📒 Files selected for processing (7)
autogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.pyautogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/executor/utils.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: Seer Code Review
- GitHub Check: end-to-end tests
- GitHub Check: test (3.11)
- GitHub Check: setup
- GitHub Check: type-check (3.12)
- GitHub Check: type-check (3.13)
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.12)
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
- GitHub Check: Analyze (typescript)
🧰 Additional context used
📓 Path-based instructions (7)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/backend/backend/data/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
All data access in backend requires user ID checks; verify this for any 'data/*.py' changes
Files:
autogpt_platform/backend/backend/data/execution.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/**/data/*.py
📄 CodeRabbit inference engine (AGENTS.md)
For changes touching
data/*.py, validate user ID checks or explain why not needed
Files:
autogpt_platform/backend/backend/data/execution.py
autogpt_platform/backend/**/*test*.py
📄 CodeRabbit inference engine (AGENTS.md)
Run
poetry run testfor backend testing (runs pytest with docker based postgres + prisma)
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
🧠 Learnings (17)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/data/execution.pyautogpt_platform/backend/backend/executor/utils.pyautogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Implement 'run' method with proper error handling in backend blocks
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.pyautogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-10T08:38:29.078Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:29.078Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review is session-scoped: it uses graph_exec_id (copilot-session-{session_id}) combined with node_id (copilot-node-{block_id}). If the user enables the auto-approve toggle for a CoPilot session, approving a block type in that session automatically approves all future invocations of that same block type within the same session. If the user does not toggle auto-approve (default), each individual invocation requires its own approval. This behavior should be clearly documented in code and comments, and tested to ensure the toggle reliably toggles between session-scoped auto-approval and per-invocation approval.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-10T08:38:30.834Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:349-370
Timestamp: 2026-03-10T08:38:30.834Z
Learning: In the AutoGPT CoPilot HITL flow, the review card presents with editable: false, so the input payload cannot be modified. Therefore credentials resolved before is_block_exec_need_review() remain valid and do not need to be recomputed after the review step. Ensure run_block.py uses the original input_data for credential validation and that any credential caching relies on the pre-review input. If there is any logic that would recompute credentials post-review, guard it against input_data mutations and rely on the UI to enforce immutability. This guideline is specific to autogpt_platform/backend/backend/copilot/tools/run_block.py.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-17T10:57:10.126Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_agent.pyautogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Write tests alongside block implementation when adding new blocks in backend
Applied to files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/backend/blocks/*.py : When adding new blocks, analyze block interfaces to ensure inputs and outputs tie well together for productive graph-based editor connections
Applied to files:
autogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-10T08:38:33.249Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-15T15:30:02.282Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:02.282Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, inside execute_block: when InsufficientBalanceError occurs after post-execution credit charging (i.e., balance drained concurrently after pre-check passed), treat as a non-fatal billing leak. Log at ERROR level with structured JSON: {"billing_leak": True, "user_id": ..., "cost": ...} for monitoring/alerting, then return BlockOutputResponse normally (do not discard the output). Do not perform a second get_user_credit_model call; reuse the credit_model obtained during the pre-execution balance check (guarded by if cost > 0 and credit_model:). This guidance improves UX by not discarding results and provides observable billing leak signals.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/**/test/**/*.py : Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Always review snapshot changes with `git diff` before committing when updating snapshots with `poetry run pytest --snapshot-update`
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Colocate test files with source files using the `*_test.py` naming convention
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
🔇 Additional comments (11)
autogpt_platform/backend/backend/data/execution.py (1)
92-92: LGTM!The
dry_runfield is well-placed in the safety settings section with a clear inline comment explaining its purpose. TheFalsedefault ensures backward compatibility.autogpt_platform/backend/backend/executor/utils.py (1)
905-931: LGTM!The dry-run propagation logic is correct:
- When a partial
ExecutionContextis provided (e.g., onlydry_run=True), the flag is extracted and forwarded to the newly constructed full context.- When a complete context with
graph_exec_idis provided, it's used as-is, preserving itsdry_runvalue.The broadened condition on line 908 properly handles the case where callers like
run_agent.pypassExecutionContext(dry_run=True)without other fields.autogpt_platform/backend/backend/copilot/tools/run_block.py (2)
84-91: LGTM!The
dry_runparameter is well-defined with a clear description explaining its purpose. Keeping it out of therequiredlist makes it optional with a sensiblefalsedefault.
121-121: LGTM!The
bool()coercion safely handles various truthy/falsy values that LLMs might return, and the parameter is correctly forwarded toexecute_block.Also applies to: 399-399
autogpt_platform/backend/backend/copilot/tools/helpers.py (1)
72-97: LGTM!The dry-run implementation is well-structured:
- Lazy import avoids potential import issues
- Output aggregation pattern matches the real execution path
- Error handling provides useful debugging info with
exc_info=True- The
[DRY RUN]marker clearly distinguishes simulated from real outputs- Correctly bypasses workspace setup, credential injection, and credit charging
autogpt_platform/backend/backend/copilot/tools/run_agent.py (2)
164-171: LGTM!The tool schema clearly describes the dry-run behavior and correctly notes that each block in the graph is LLM-simulated.
468-474: LGTM!The conditional
ExecutionContext(dry_run=dry_run) if dry_run else Noneis a clean approach—it only creates a partial context when needed, allowingadd_graph_executionto build the full context while preserving the dry-run flag.autogpt_platform/backend/backend/executor/simulator.py (2)
116-163: LGTM with a note on exception handling.The retry logic is well-structured:
- JSON parsing errors trigger retries with warning logs
- Non-recoverable exceptions (network failures, API errors) immediately break the loop with error logging
- After exhausting retries, a clear error tuple is yielded
The distinction between retryable (JSON parse) and non-retryable (other) exceptions is appropriate for this use case.
63-84: System prompt is well-crafted.The prompt effectively constrains the LLM:
- Clear instruction to assume valid credentials prevents auth failure simulation
- Explicit output pin names reduce hallucinated keys
- The
errorpin default behavior is clearly specifiedautogpt_platform/backend/backend/copilot/tools/test_dry_run.py (2)
1-23: Good test organization with comprehensive coverage.The lazy import pattern for
simulate_blockandbuild_simulation_promptis a pragmatic solution to avoid Pyright import resolution issues while keeping tests isolated.
282-316: LGTM!This test correctly verifies that
dry_run=Falsefollows the real execution path by:
- Confirming
simulate_blockis not called- Verifying the code attempts real execution (which fails on
workspace_db())The non-local
simulate_calledflag is a clean way to verify the mock wasn't invoked.
14 tasks
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
autogpt_platform/backend/backend/executor/simulator.py (2)
40-48: Consider handling nested structures for truncation.The truncation logic only handles top-level string values. If
input_datacontains nested dictionaries or lists with long strings, those won't be truncated and could still cause prompt size issues.For now this is likely acceptable since most block inputs are relatively flat, but consider extending this in the future if nested large inputs become common.
♻️ Optional: recursive truncation helper
def _truncate_input_values(input_data: dict[str, Any]) -> dict[str, Any]: """Truncate long string values so the prompt doesn't blow up.""" def _truncate(v: Any) -> Any: if isinstance(v, str) and len(v) > _MAX_INPUT_VALUE_CHARS: return v[:_MAX_INPUT_VALUE_CHARS] + "... [TRUNCATED]" elif isinstance(v, dict): return {k: _truncate(val) for k, val in v.items()} elif isinstance(v, list): return [_truncate(item) for item in v] return v return {k: _truncate(v) for k, v in input_data.items()}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/executor/simulator.py` around lines 40 - 48, The current _truncate_input_values function only truncates top-level string values which misses nested strings in dicts/lists; update _truncate_input_values to recurse into nested dicts and lists (and apply truncation when encountering str longer than _MAX_INPUT_VALUE_CHARS) by adding a helper like _truncate(v: Any) used for dict/list/str cases and return the transformed structure so all nested long strings get truncated.
139-142: Consider guarding against emptychoicesarray.If the LLM API returns an empty
choicesarray (which can happen in edge cases like content filtering),response.choices[0]will raise anIndexError. While this is caught by the generic exception handler and breaks the loop, an explicit check would provide a clearer error message.♻️ Optional: explicit choices check
) - raw = response.choices[0].message.content or "" + if not response.choices: + raise ValueError("LLM returned empty choices array") + raw = response.choices[0].message.content or ""🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/executor/simulator.py` around lines 139 - 142, The code assumes response.choices[0] exists which can raise IndexError for empty choices; update the block that reads response.choices to explicitly check that response.choices is a non-empty sequence (e.g., if not response.choices: raise a clear ValueError or log and continue) before accessing response.choices[0], then proceed to set raw = response.choices[0].message.content or "" and json.loads(raw) as currently done; reference the response.choices access and the variables raw and parsed so you only add the guard and a descriptive error if choices is empty.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@autogpt_platform/backend/backend/executor/simulator.py`:
- Around line 40-48: The current _truncate_input_values function only truncates
top-level string values which misses nested strings in dicts/lists; update
_truncate_input_values to recurse into nested dicts and lists (and apply
truncation when encountering str longer than _MAX_INPUT_VALUE_CHARS) by adding a
helper like _truncate(v: Any) used for dict/list/str cases and return the
transformed structure so all nested long strings get truncated.
- Around line 139-142: The code assumes response.choices[0] exists which can
raise IndexError for empty choices; update the block that reads response.choices
to explicitly check that response.choices is a non-empty sequence (e.g., if not
response.choices: raise a clear ValueError or log and continue) before accessing
response.choices[0], then proceed to set raw =
response.choices[0].message.content or "" and json.loads(raw) as currently done;
reference the response.choices access and the variables raw and parsed so you
only add the guard and a descriptive error if choices is empty.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6bde7994-2026-494e-adfc-e00aaf197cff
📒 Files selected for processing (3)
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: check API types
- GitHub Check: end-to-end tests
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.13)
- GitHub Check: test (3.11)
- GitHub Check: lint
- GitHub Check: test (3.12)
- GitHub Check: type-check (3.13)
- GitHub Check: type-check (3.12)
- GitHub Check: Seer Code Review
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (typescript)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
🧠 Learnings (16)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Implement 'run' method with proper error handling in backend blocks
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-10T08:38:33.249Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.py
📚 Learning: 2026-03-16T16:30:30.764Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:30.764Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, explicit try/except in the `run()` method is NOT required for standard error handling. The block framework's `_execute()` method in `_base.py` catches unhandled exceptions and re-raises them as `BlockExecutionError` or `BlockUnknownError`. Additionally, when a block yields `("error", message)`, `_execute()` immediately raises `BlockExecutionError` — so the `error` output port never propagates downstream. Explicit try/except is only needed when partial output must be controlled (e.g., attachment blocks that must skip yielding `content_base64` on failure).
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-16T16:32:29.430Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:29.430Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, the Block base class `execute()` method in `backend/blocks/_base.py` already wraps `run()` in a try/except that converts uncaught exceptions into `BlockExecutionError`/`BlockUnknownError`. Therefore, explicit try/except in individual block `run()` methods is redundant and not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Exception: blocks like attachment blocks that need to distinguish between success and error yield paths within the generator use explicit try/except for branching control, not for the framework's error routing.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-10T08:38:36.655Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:349-370
Timestamp: 2026-03-10T08:38:36.655Z
Learning: In the AutoGPT CoPilot HITL (Human-In-The-Loop) flow (`autogpt_platform/backend/backend/copilot/tools/run_block.py`), the review card presented to users sets `editable: false`, meaning reviewers cannot modify the input payload. Therefore, credentials resolved before `is_block_exec_need_review()` remain valid and do not need to be recomputed after the review step — the original `input_data` is unchanged through the review lifecycle.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-10T08:39:22.025Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-17T10:57:10.126Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/continue_run_block.pyautogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-15T15:30:02.282Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:02.282Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, inside execute_block: when InsufficientBalanceError occurs after post-execution credit charging (i.e., balance drained concurrently after pre-check passed), treat as a non-fatal billing leak. Log at ERROR level with structured JSON: {"billing_leak": True, "user_id": ..., "cost": ...} for monitoring/alerting, then return BlockOutputResponse normally (do not discard the output). Do not perform a second get_user_credit_model call; reuse the credit_model obtained during the pre-execution balance check (guarded by if cost > 0 and credit_model:). This guidance improves UX by not discarding results and provides observable billing leak signals.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.py
🔇 Additional comments (6)
autogpt_platform/backend/backend/copilot/tools/continue_run_block.py (1)
143-153: LGTM!The explicit
dry_run=Falseparameter with a clarifying comment is a good defensive practice. It makes the intent clear that post-approval execution always runs for real, preventing any accidental dry-run behavior in the human review flow.autogpt_platform/backend/backend/copilot/tools/helpers.py (1)
72-98: LGTM!The dry-run implementation is well-structured:
- The lazy import avoids circular dependencies and unnecessary loading.
- The comment explicitly documents that HITL review is skipped (as requested in prior review).
- Error handling logs with
exc_info=Truefor proper debugging context.- The output collection pattern mirrors the real execution path, maintaining consistency.
autogpt_platform/backend/backend/executor/simulator.py (4)
1-22: LGTM!The module documentation clearly explains the simulator's purpose and grounding strategy. The import of
AsyncIteratorfromcollections.abccorrectly addresses the past review feedback about the return type annotation.
24-37: LGTM!The
_simulator_model()function addresses the past review feedback by dynamically obtaining the model fromChatConfig.title_modelwith a sensible fallback. The broad exception handling is appropriate here sinceChatConfigmay not be importable in all contexts.
63-101: LGTM!The prompt construction is well-designed:
- Clear separation of system and user prompts.
- Structured guidance with explicit rules for the LLM.
- The
json.dumps(output_properties)ensures the LLM knows the exact pin names to include.- Safe fallbacks for block name and description.
164-181: LGTM!The error handling strategy is well-designed:
- Retries only on recoverable JSON parse errors.
- Breaks immediately on API/network failures to avoid wasting retries.
- The
[SIMULATOR ERROR — NOT A BLOCK FAILURE]prefix clearly distinguishes simulator issues from actual block errors, which is helpful for debugging.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py (2)
332-349: Avoid module-wide source-string assertions for forwarding checks.The Line 347 check scans the entire module source, which can pass from unrelated text/comments and is brittle to refactors. At minimum, scope inspection to
_execute; ideally assert actual forwarded kwargs via a mockedexecute_block.Minimal hardening (scope to method source)
- source_execute = inspect.getsource(run_block_module) + source_execute = inspect.getsource(run_block_module.RunBlockTool._execute) # Verify dry_run is passed through to execute_block call assert "dry_run=dry_run" in source_execute🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py` around lines 332 - 349, The test currently inspects the entire module source to assert dry_run is forwarded which is brittle; instead scope the check to RunBlockTool._execute or (preferably) mock the execute_block call and assert it was invoked with dry_run in its kwargs. Update test_run_block_tool_dry_run_calls_execute to either use inspect.getsource(run_block_module.RunBlockTool._execute) to look for the forwarding token or, better, patch backend.copilot.tools.run_block.execute_block with a MagicMock, call RunBlockTool._execute with dry_run=True and then assert the mock was called with a kwargs containing 'dry_run': True (reference RunBlockTool._execute and execute_block to locate code).
131-151: Assert retry budget in the exhausted-retries test.This test validates the final error tuple, but it doesn’t prove all retry attempts were consumed. Add a call-count assertion to lock in retry behavior.
Suggested tightening
async def test_simulate_block_all_retries_exhausted(): @@ assert len(outputs) == 1 name, data = outputs[0] assert name == "error" assert "[SIMULATOR ERROR" in data + assert mock_client.chat.completions.create.call_count == 5🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py` around lines 131 - 151, The test test_simulate_block_all_retries_exhausted should also assert that the OpenAI call was invoked for each retry attempt; after collecting outputs from simulate_block (from _get_simulate_block) add an assertion that mock_client.chat.completions.create.call_count equals the expected retry budget (use the module constant if one exists, otherwise the literal expected count, e.g., 3) so the test verifies all retry attempts were consumed when simulate_block exhausts retries.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py`:
- Around line 298-303: The test patches are mocking simulate_block in the wrong
module; update all patches that currently target
"backend.executor.simulator.simulate_block" to mock the name where it's imported
(helpers) by using "backend.copilot.tools.helpers.simulate_block" instead so the
fake_simulate intercepts calls; ensure the patches around the execute_block call
(and the other occurrences at lines 233 and 262) are changed accordingly while
leaving the workspace_db patch (helpers.workspace_db) and the
fake_simulate/execute_block usage intact.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/copilot/tools/test_dry_run.py`:
- Around line 332-349: The test currently inspects the entire module source to
assert dry_run is forwarded which is brittle; instead scope the check to
RunBlockTool._execute or (preferably) mock the execute_block call and assert it
was invoked with dry_run in its kwargs. Update
test_run_block_tool_dry_run_calls_execute to either use
inspect.getsource(run_block_module.RunBlockTool._execute) to look for the
forwarding token or, better, patch backend.copilot.tools.run_block.execute_block
with a MagicMock, call RunBlockTool._execute with dry_run=True and then assert
the mock was called with a kwargs containing 'dry_run': True (reference
RunBlockTool._execute and execute_block to locate code).
- Around line 131-151: The test test_simulate_block_all_retries_exhausted should
also assert that the OpenAI call was invoked for each retry attempt; after
collecting outputs from simulate_block (from _get_simulate_block) add an
assertion that mock_client.chat.completions.create.call_count equals the
expected retry budget (use the module constant if one exists, otherwise the
literal expected count, e.g., 3) so the test verifies all retry attempts were
consumed when simulate_block exhausts retries.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 030400ce-360d-47de-8a28-d1394630065c
📒 Files selected for processing (2)
autogpt_platform/backend/backend/copilot/tools/test_dry_run.pyautogpt_platform/backend/backend/executor/simulator.py
🚧 Files skipped from review as they are similar to previous changes (1)
- autogpt_platform/backend/backend/executor/simulator.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: check API types
- GitHub Check: test (3.13)
- GitHub Check: test (3.11)
- GitHub Check: type-check (3.11)
- GitHub Check: type-check (3.12)
- GitHub Check: test (3.12)
- GitHub Check: type-check (3.13)
- GitHub Check: Seer Code Review
- GitHub Check: end-to-end tests
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (5)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
autogpt_platform/backend/**/*test*.py
📄 CodeRabbit inference engine (AGENTS.md)
Run
poetry run testfor backend testing (runs pytest with docker based postgres + prisma)
Files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
🧠 Learnings (15)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/**/test/**/*.py : Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Write tests alongside block implementation when adding new blocks in backend
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Always review snapshot changes with `git diff` before committing when updating snapshots with `poetry run pytest --snapshot-update`
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Use pytest with snapshot testing for API responses in test files
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Implement 'run' method with proper error handling in backend blocks
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:51.495Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-04T16:50:51.495Z
Learning: Applies to autogpt_platform/backend/**/*test*.py : Run `poetry run test` for backend testing (runs pytest with docker based postgres + prisma)
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-17T10:57:10.126Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Run 'poetry run test' before committing backend changes to ensure all tests pass
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-04T16:50:20.508Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/CLAUDE.md:0-0
Timestamp: 2026-02-04T16:50:20.508Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Colocate test files with source files using the `*_test.py` naming convention
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
🔇 Additional comments (1)
autogpt_platform/backend/backend/copilot/tools/test_dry_run.py (1)
84-197: Good simulator behavior coverage across core scenarios.The success path, invalid-JSON retries, missing pin defaults, and no-client fallback are all covered cleanly here.
majdyz
commented
Mar 19, 2026
majdyz
commented
Mar 19, 2026
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
autogpt_platform/backend/backend/copilot/tools/helpers.py (1)
74-80: Coerce dry-run inputs before simulating.Line 159 normalizes the real execution payload against
block.input_schema, but dry-run sends the rawinput_datastraight into the prompt. That makes the preview drift on common cases like"42"vs42, booleans, and enum/default coercions.♻️ Proposed change
if dry_run: try: + from copy import deepcopy from backend.executor.simulator import simulate_block # lazy import + sim_input_data = deepcopy(input_data) + coerce_inputs_to_schema(sim_input_data, block.input_schema) outputs: dict[str, list[Any]] = defaultdict(list) - async for output_name, output_data in simulate_block(block, input_data): + async for output_name, output_data in simulate_block( + block, sim_input_data + ): outputs[output_name].append(output_data)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/tools/helpers.py` around lines 74 - 80, Dry-run currently passes raw input_data into simulate_block causing type/enum/default mismatches; before the async for simulate_block(block, input_data) call, coerce/normalize input_data against block.input_schema using the same normalization/validation logic used for real execution (the code that normalizes the payload against block.input_schema) and pass the normalized payload into simulate_block so previews match real execution (ensure numbers, booleans, enums and defaults are handled).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/tools/helpers.py`:
- Around line 81-103: The current dry-run handling only treats outputs["error"]
starting with "[SIMULATOR ERROR" as a real failure, but any non-empty error pin
should be treated as a block failure; update the branch in helpers.py (the code
that reads sim_error = outputs.get("error", [])) to return an ErrorResponse
(rather than a successful BlockOutputResponse) whenever sim_error is non-empty
and contains an error message (not just the sentinel). Ensure the returned
ErrorResponse mirrors what execute_block()/_execute() would propagate on a real
failure (use the same session_id, message and error fields, and avoid
success=True), so callers see the failure consistently with BlockExecutionError
behavior.
In `@autogpt_platform/backend/backend/executor/simulator.py`:
- Around line 56-64: _truncate_input_values currently only truncates top-level
strings; make it recursive so nested structures are also bounded. Update
_truncate_input_values to walk input_data and for dicts call itself on values,
for lists/tuples/map over elements, and for strings truncate at
_MAX_INPUT_VALUE_CHARS appending "... [TRUNCATED]"; leave other scalar types
unchanged and preserve original container types (dict, list, tuple). Ensure you
reference and use the same _MAX_INPUT_VALUE_CHARS and function name
_truncate_input_values so all nested branches apply the same limit.
- Around line 34-45: The _simulator_model() logic incorrectly requires
secrets.open_router_api_key to be false before stripping an "openai/..." prefix;
get_openai_client() prefers a direct OpenAI client whenever
secrets.openai_internal_api_key is set regardless of open_router_api_key, so
update the condition inside the try block that inspects Settings().secrets to
strip the prefix whenever secrets.openai_internal_api_key is truthy (i.e.,
remove the "and not secrets.open_router_api_key" check), ensuring model (the
local variable) is normalized when "/" in model; refer to get_openai_client(),
_simulator_model(), Settings, secrets.openai_internal_api_key, and model to
locate the change.
---
Nitpick comments:
In `@autogpt_platform/backend/backend/copilot/tools/helpers.py`:
- Around line 74-80: Dry-run currently passes raw input_data into simulate_block
causing type/enum/default mismatches; before the async for simulate_block(block,
input_data) call, coerce/normalize input_data against block.input_schema using
the same normalization/validation logic used for real execution (the code that
normalizes the payload against block.input_schema) and pass the normalized
payload into simulate_block so previews match real execution (ensure numbers,
booleans, enums and defaults are handled).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 271e6c49-3d5b-4a92-905b-be80a4bc0a21
📒 Files selected for processing (3)
autogpt_platform/backend/backend/copilot/tools/helpers.pyautogpt_platform/backend/backend/copilot/tools/test_dry_run.pyautogpt_platform/backend/backend/executor/simulator.py
🚧 Files skipped from review as they are similar to previous changes (1)
- autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: end-to-end tests
- GitHub Check: type-check (3.11)
- GitHub Check: test (3.13)
- GitHub Check: type-check (3.13)
- GitHub Check: test (3.11)
- GitHub Check: test (3.12)
- GitHub Check: type-check (3.12)
- GitHub Check: Analyze (python)
- GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
🧠 Learnings (15)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
📚 Learning: 2026-03-17T10:57:10.126Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:10.126Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/executor/simulator.pyautogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-10T08:38:33.249Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-15T15:30:02.282Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:02.282Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, inside execute_block: when InsufficientBalanceError occurs after post-execution credit charging (i.e., balance drained concurrently after pre-check passed), treat as a non-fatal billing leak. Log at ERROR level with structured JSON: {"billing_leak": True, "user_id": ..., "cost": ...} for monitoring/alerting, then return BlockOutputResponse normally (do not discard the output). Do not perform a second get_user_credit_model call; reuse the credit_model obtained during the pre-execution balance check (guarded by if cost > 0 and credit_model:). This guidance improves UX by not discarding results and provides observable billing leak signals.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Implement 'run' method with proper error handling in backend blocks
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-16T16:30:30.764Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:30.764Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, explicit try/except in the `run()` method is NOT required for standard error handling. The block framework's `_execute()` method in `_base.py` catches unhandled exceptions and re-raises them as `BlockExecutionError` or `BlockUnknownError`. Additionally, when a block yields `("error", message)`, `_execute()` immediately raises `BlockExecutionError` — so the `error` output port never propagates downstream. Explicit try/except is only needed when partial output must be controlled (e.g., attachment blocks that must skip yielding `content_base64` on failure).
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-16T16:32:29.430Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:29.430Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, the Block base class `execute()` method in `backend/blocks/_base.py` already wraps `run()` in a try/except that converts uncaught exceptions into `BlockExecutionError`/`BlockUnknownError`. Therefore, explicit try/except in individual block `run()` methods is redundant and not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Exception: blocks like attachment blocks that need to distinguish between success and error yield paths within the generator use explicit try/except for branching control, not for the framework's error routing.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-10T08:39:22.025Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-10T08:38:36.655Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:349-370
Timestamp: 2026-03-10T08:38:36.655Z
Learning: In the AutoGPT CoPilot HITL (Human-In-The-Loop) flow (`autogpt_platform/backend/backend/copilot/tools/run_block.py`), the review card presented to users sets `editable: false`, meaning reviewers cannot modify the input payload. Therefore, credentials resolved before `is_block_exec_need_review()` remain valid and do not need to be recomputed after the review step — the original `input_data` is unchanged through the review lifecycle.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/helpers.py
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/tools/run_block.py`:
- Line 242: Replace the hardcoded ":" in the dry-run node_exec_id construction
with the existing COPILOT_NODE_EXEC_ID_SEPARATOR constant to match the
non-dry-run path; update the expression that builds node_exec_id (currently
using COPILOT_NODE_PREFIX, block_id and uuid.uuid4().hex) to concatenate those
parts using COPILOT_NODE_EXEC_ID_SEPARATOR instead of ":" so both branches use
the same separator.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bdeb16d3-6a2a-402a-89a0-8ab0b2031e0c
📒 Files selected for processing (1)
autogpt_platform/backend/backend/copilot/tools/run_block.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: Check PR Status
- GitHub Check: test (3.11)
- GitHub Check: test (3.13)
- GitHub Check: test (3.12)
- GitHub Check: end-to-end tests
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
Files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
autogpt_platform/backend/**/*.{py,txt}
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use
poetry runprefix for all Python commands, including testing, linting, formatting, and migrations
Files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
autogpt_platform/backend/backend/**/*.py
📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)
Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings
Files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
autogpt_platform/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
🧠 Learnings (14)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:33.249Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review uses graph_exec_id (copilot-session-{session_id}) + node_id (copilot-node-{block_id}). This is intentional: approving a block type within a CoPilot session auto-approves all future invocations of that same block type within the same session, mirroring how auto-approve works in normal graph execution. The user explicitly opts into this session-scoped behavior via an auto-approve toggle. Without the toggle (default), each individual invocation requires its own approval.
📚 Learning: 2026-03-16T16:32:29.430Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:32:29.430Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, the Block base class `execute()` method in `backend/blocks/_base.py` already wraps `run()` in a try/except that converts uncaught exceptions into `BlockExecutionError`/`BlockUnknownError`. Therefore, explicit try/except in individual block `run()` methods is redundant and not the established pattern (e.g., Gmail, Slack, Todoist blocks omit it). Exception: blocks like attachment blocks that need to distinguish between success and error yield paths within the generator use explicit try/except for branching control, not for the framework's error routing.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-16T16:30:30.764Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/pods.py:62-74
Timestamp: 2026-03-16T16:30:30.764Z
Learning: In autogpt_platform/backend/backend/blocks/**/*.py, explicit try/except in the `run()` method is NOT required for standard error handling. The block framework's `_execute()` method in `_base.py` catches unhandled exceptions and re-raises them as `BlockExecutionError` or `BlockUnknownError`. Additionally, when a block yields `("error", message)`, `_execute()` immediately raises `BlockExecutionError` — so the `error` output port never propagates downstream. Explicit try/except is only needed when partial output must be controlled (e.g., attachment blocks that must skip yielding `content_base64` on failure).
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/backend/blocks/**/*.py : Implement 'run' method with proper error handling in backend blocks
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-15T15:30:09.706Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/tools/helpers.py:149-185
Timestamp: 2026-03-15T15:30:09.706Z
Learning: In autogpt_platform/backend/backend/copilot/tools/helpers.py, within execute_block, when InsufficientBalanceError occurs after post-execution credit charging (concurrent balance drain after pre-check passed), this is treated as a non-fatal billing leak: log at ERROR level with structured JSON fields `{"billing_leak": True, "user_id": ..., "cost": ...}` for monitoring/alerting, then return BlockOutputResponse normally. Discarding the output would worsen UX since the block already executed with potential side effects. Reuse the credit_model obtained during the pre-execution balance check (guarded by `if cost > 0 and credit_model:`) for the post-execution charge; do not perform a second get_user_credit_model call.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-16T16:30:20.657Z
Learnt from: Abhi1992002
Repo: Significant-Gravitas/AutoGPT PR: 12417
File: autogpt_platform/backend/backend/blocks/agent_mail/threads.py:80-102
Timestamp: 2026-03-16T16:30:20.657Z
Learning: In autogpt_platform/backend/backend/blocks/agent_mail/ (and other blocks under autogpt_platform/backend/backend/blocks/), the block executor framework (backend/executor/manager.py lines ~708-733) automatically catches all uncaught exceptions from a block's `run()` method and emits them on the `"error"` output. Explicit try/except blocks within `run()` are therefore not required for standard error propagation — they are only needed when partial output behaviour must be controlled (e.g., preventing some outputs from being yielded on failure, as in attachment blocks). This is the standard pattern across the codebase.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-10T08:38:29.078Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:297-300
Timestamp: 2026-03-10T08:38:29.078Z
Learning: In autogpt_platform/backend/backend/copilot/tools/run_block.py, the auto-approval key for sensitive block HITL review is session-scoped: it uses graph_exec_id (copilot-session-{session_id}) combined with node_id (copilot-node-{block_id}). If the user enables the auto-approve toggle for a CoPilot session, approving a block type in that session automatically approves all future invocations of that same block type within the same session. If the user does not toggle auto-approve (default), each individual invocation requires its own approval. This behavior should be clearly documented in code and comments, and tested to ensure the toggle reliably toggles between session-scoped auto-approval and per-invocation approval.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-10T08:39:22.025Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-10T08:38:30.834Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/tools/run_block.py:349-370
Timestamp: 2026-03-10T08:38:30.834Z
Learning: In the AutoGPT CoPilot HITL flow, the review card presents with editable: false, so the input payload cannot be modified. Therefore credentials resolved before is_block_exec_need_review() remain valid and do not need to be recomputed after the review step. Ensure run_block.py uses the original input_data for credential validation and that any credential caching relies on the pre-review input. If there is any logic that would recompute credentials post-review, guard it against input_data mutations and rely on the UI to enforce immutability. This guideline is specific to autogpt_platform/backend/backend/copilot/tools/run_block.py.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-04T12:19:39.243Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12279
File: autogpt_platform/backend/backend/copilot/tools/base.py:184-188
Timestamp: 2026-03-04T12:19:39.243Z
Learning: In autogpt_platform/backend/backend/copilot/tools/, ensure that anonymous users always pass user_id=None to tool execution methods. The anon_ prefix (e.g., anon_123) is used only for PostHog/analytics distinct_id and must not be used as an actual user_id. Use a simple truthiness check on user_id (e.g., if user_id: ... else: ... or a dedicated is_authenticated flag) to distinguish anonymous from authenticated users, and review all tool execution call sites within this directory to prevent accidentally forwarding an anon_ user_id to tools.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/tools/run_block.py
🔇 Additional comments (3)
autogpt_platform/backend/backend/copilot/tools/run_block.py (3)
84-91: LGTM!The
dry_runparameter is well-defined with a clear description explaining its purpose and behavior. The default offalseis appropriate for maintaining backward compatibility.
121-121: LGTM!The
bool()coercion is a good defensive practice given thatkwargsfrom OpenAI function calls may contain varying types (as shown inbase.py:execute()which passes them through without coercion).
405-414: LGTM!The
dry_runflag is correctly propagated toexecute_block(). Whiledry_runwill always beFalseat this point (sinceTruetriggers the early return at line 235), explicitly passing the parameter maintains clarity and consistency with the function signature.
github-actions
Bot
added
the
conflicts
Contributor
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
Integrate dry_run feature with dev's prepare_block_for_execution helper.
github-actions
Bot
removed
the
conflicts
Contributor
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
When dry_run=True, prepare_block_for_execution now skips the missing credentials check since simulation never calls real services.
ntindle
approved these changes
Mar 24, 2026
github-project-automation
Bot
moved this from 🚧 Needs work
to 👍🏼 Mergeable
in AutoGPT development kanban
github-project-automation
Bot
moved this from 👍🏼 Mergeable
to ✅ Done
in AutoGPT development kanban
majdyz
added a commit
that referenced
this pull request
Mar 25, 2026
- useResetRateLimit now accepts onCreditChange callback, invoked after a successful reset to re-fetch the credit balance - Restores reset_cost and /usage/reset endpoint in openapi.json lost during dev merge, plus dry_run fields from PR #12483 - Regenerates API client to match
Issues attributed to commits in this pull requestThis pull request was merged and Sentry observed the following issues:
|
This was referenced Mar 26, 2026
5 tasks
Bentlybro
pushed a commit
that referenced
this pull request
Apr 4, 2026
#12507) ## Summary - Adds `/pr-test` skill for automated E2E testing of PRs using docker compose, agent-browser, and API calls - Covers full environment setup (copy .env, configure copilot auth, ARM64 Docker fix) - Includes browser UI testing, direct API testing, screenshot capture, and test report generation - Has `--fix` mode for auto-fixing bugs found during testing (similar to `/pr-address`) - **Screenshot uploads use GitHub Git API** (blobs → tree → commit → ref) — no local git operations, safe for worktrees - **Subscription mode improvements:** - Extract subscription auth logic to `sdk/subscription.py` — uses SDK's bundled CLI binary instead of requiring `npm install -g @anthropic-ai/claude-code` - Auto-provision `~/.claude/.credentials.json` from `CLAUDE_CODE_OAUTH_TOKEN` env var on container startup — no `claude login` needed in Docker - Add `scripts/refresh_claude_token.sh` — cross-platform helper (macOS/Linux/Windows) to extract OAuth tokens from host and update `backend/.env` ## Test plan - [x] Validated skill on multiple PRs (#12482, #12483, #12499, #12500, #12501, #12440, #12472) — all test scenarios passed - [x] Confirmed screenshot upload via GitHub Git API renders correctly on all 7 PRs - [x] Verified subscription mode E2E in Docker: `refresh_claude_token.sh` → `docker compose up` → copilot chat responds correctly with no API keys (pure OAuth subscription) - [x] Verified auto-provisioning of credentials file inside container from `CLAUDE_CODE_OAUTH_TOKEN` env var - [x] Confirmed bundled CLI detection (`claude_agent_sdk._bundled/claude`) works without system-installed `claude` - [x] `poetry run pytest backend/copilot/sdk/service_test.py` — 24/24 tests pass
This was referenced Apr 8, 2026
Merged
Merged
Merged
9 tasks
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Agent generation and building needs a way to test-run agents without requiring real credentials or producing side effects. Currently, every execution hits real APIs, consumes credits, and requires valid credentials — making it impossible to debug or validate agent graphs during the build phase without real consequences.
Summary
Adds a
dry_runexecution mode to the copilot'srun_blockandrun_agenttools. Whendry_run=True, every block execution is simulated by an LLM instead of calling the real service — no real API calls, no credentials consumed, no side effects.Inspired by Significant-Gravitas/agent-simulator.
How it works
backend/executor/simulator.py(new):simulate_block()builds a prompt from the block's name, description, input/output schemas, and actual input values, then callsgpt-4o-minivia the existing OpenRouter client with JSON mode. Retries up to 5 times on JSON parse failures. Missing output pins are filled withNone(or""for theerrorpin). Long inputs (>20k chars) are truncated before sending to the LLM.ExecutionContext: Addeddry_run: bool = Falsefield; threaded throughadd_graph_execution()so graph-level dry runs propagate to every block execution.execute_block()helper: Whendry_run=True, the function short-circuits before any credential injection or credit checks, callssimulate_block(), and returns a[DRY RUN]-prefixedBlockOutputResponse.RunBlockTool: Newdry_runboolean parameter.RunAgentTool: Newdry_runboolean parameter; passesExecutionContext(dry_run=True)to graph execution.Tests
11 tests in
backend/copilot/tools/test_dry_run.py:SIMULATOR ERRORNone/""execute_block(dry_run=True)skips realblock.execute()[DRY RUN]message,success=Truedry_run=Falseunchanged (real path)RunBlockToolparameter presencedry_runkwarg forwardingTest plan
pytest backend/copilot/tools/test_dry_run.py -v— all 11 passrun_blockwithdry_run=truein copilot; verify no real API calls occur and output contains[DRY RUN]run_agentwithdry_run=true; verify execution is created withdry_run=Truein context