refactor(backend/copilot): unified transcript context — extract_context_messages, mode-gated --resume, compaction-aware gap-fill - #12804
Merged
Conversation
…subsequent turns When a user switches from baseline (fast) mode to SDK (extended_thinking) mode mid-session, the first SDK turn has has_history=True (prior baseline messages in DB) but no CLI session file in storage. The old code gated session_id on `not has_history`, so mode-switch T1 never received a session_id — the CLI generated a random ID that wasn't uploaded under the expected key. Every subsequent SDK turn would fail to restore the CLI session and run without --resume, injecting the full compressed history on each turn, causing model confusion. Fix: set session_id whenever not using --resume (the `else` branch), covering T1 fresh, mode-switch T1, and T2+ fallback turns. The retry path is updated to use `"session_id" in sdk_options_kwargs` as the discriminator (instead of `not has_history`) so mode-switch T1 retries also keep the session_id while T2+ retries (where T1 restored a session file via restore_cli_session) still remove it to avoid "Session ID already in use".
…on context loss The Claude Code CLI auto-compacts its native session JSONL when the context approaches the model's token limit (~200K for Sonnet). After compaction the detailed conversation history is replaced by a ~27K-token summary, causing the silent context loss users see as memory failures in long sessions. Root cause identified from production logs for session 93ecf7c9: - T6 CLI session: 233KB / ~207K tokens (near Sonnet limit) - T7 CLI compacted session -> ~167KB / ~47K tokens (PreCompact hook missed) - T12 second compaction -> ~176KB / ~27K tokens (just system prompt + summary) - T14-T21: cache_read=26714 constantly -- only system prompt visible to Claude The same stripping we already apply to our transcript (stale thinking blocks, progress/metadata entries) now also runs on the CLI native session file. At ~2x the size of the stripped transcript, unstripped sessions routinely hit the compaction threshold within 6-10 turns of a heavy Opus/thinking session. After stripping: - same-pod turns reuse the stripped local file (no compaction trigger) - cross-pod turns restore the stripped GCS file (same benefit)
…location Before this change the SDK turn cycle made two separate GCS downloads and two uploads per turn: chat-transcripts/ (our stripped JSONL + message_count meta) and cli-sessions/ (raw CLI session for --resume). The chat-transcripts/ path was introduced before --resume existed; cli-sessions/ was added in PR #12777 to enable cross-pod resume, but chat-transcripts/ was never removed. This refactoring eliminates chat-transcripts/ from the SDK path entirely: - message_count watermark moves to a companion cli-sessions/.meta.json, uploaded and downloaded in the same asyncio.gather as the session file — no window for divergence between them. - TranscriptBuilder is now seeded from the restored CLI session content (strip_for_upload applied in-memory), replacing the separate transcript download. - restore_cli_session returns CliSessionRestore | None (content + message_count) instead of bool, combining the two previous download operations into one. - upload_cli_session accepts message_count and writes the companion meta. - The same-pod early-return optimisation is removed (cross-pod fix): the local file may be stale from an older turn that ran on this pod while a newer turn ran on a different pod and uploaded to GCS. upload_transcript / download_transcript are kept for the baseline service which has its own separate context management path.
majdyz
requested review from
Bentlybro and
Pwuts
and removed request for
a team
April 15, 2026 18:43
github-project-automation
Bot
moved this to 🆕 Needs initial review
in AutoGPT development kanban
Contributor
|
This PR targets the Automatically setting the base branch to |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughReplace JSONL transcript storage with a bytes-based CLI session model: introduce typed TranscriptDownload with Changes
Sequence DiagramsequenceDiagram
participant SDK as SDK Service
participant Trans as Transcript module
participant GCS as GCS Storage
participant Builder as TranscriptBuilder
SDK->>Trans: download_transcript(user_id, session_id)
par fetch session + meta
Trans->>GCS: GET <session_id>.jsonl
Trans->>GCS: GET <session_id>.meta.json
end
GCS-->>Trans: bytes + meta (or missing)
Trans-->>SDK: TranscriptDownload(content=bytes, message_count, mode) or None
SDK->>SDK: decode UTF‑8, strip_for_upload(), validate_transcript()
alt valid restore
SDK->>Builder: load_previous(stripped_jsonl)
SDK->>SDK: set use_resume, transcript_msg_count
else missing/invalid
SDK->>SDK: build transcript from DB messages
end
SDK->>SDK: read CLI session bytes from disk
SDK->>Trans: upload_transcript(user_id, session_id, content=bytes, message_count, mode)
Trans->>GCS: PUT <session_id>.jsonl (bytes)
Trans->>GCS: PUT <session_id>.meta.json (json with message_count, mode, uploaded_at)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
size/xl
platform/backend
Contributor
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
Contributor
🔍 PR Overlap DetectionThis check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early.
|
majdyz
commented
Apr 15, 2026
majdyz
commented
Apr 15, 2026
majdyz
commented
Apr 15, 2026
- Replace redundant elif guard with plain else in service.py restore path - Use isinstance(…, Exception) instead of BaseException in gather error checks for upload_cli_session (BaseException swallows KeyboardInterrupt) - Use explicit list side_effect in test_returns_none_when_file_not_found to document the two-call contract of the concurrent retrieve gather
majdyz
commented
Apr 15, 2026
…tion test Make two-call contract of asyncio.gather explicit: RuntimeError for session retrieve and FileNotFoundError for meta retrieve, matching the pattern already applied to test_returns_none_when_file_not_found_in_storage.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@autogpt_platform/backend/backend/copilot/transcript.py`:
- Around line 833-841: The meta parsing currently assumes
json.loads(meta_result.decode(...)) yields a dict and that
meta.get("message_count") is a non-negative int; update the block handling
meta_result to: decode and json.loads inside a try/except (catch
JSONDecodeError, TypeError, ValueError) and log a debug message on failure,
treat non-dict results (None, list, string) as an empty dict, then read value =
meta.get("message_count", 0) and coerce/validate it as an integer with
message_count = max(0, int(value)) (guarding int() with a try/except to fall
back to 0 on invalid types); keep existing logging via logger and variables like
meta_result, log_prefix, and message_count.
- Around line 758-773: If the session upload (session_result) fails but the
companion metadata upload (meta_result) succeeded, delete/rollback the companion
metadata to avoid inconsistent state; after detecting isinstance(session_result,
BaseException) and that meta_result is not an exception, call the storage
deletion API for the metadata using the same identifiers used in the store call
(e.g., storage.delete or equivalent with workspace_id=mwid and file_id=mfid or
filename=mfname), handle and log any deletion errors, and only then
return—ensure deletion uses the same unique symbols (mwid, mfid, mfname,
meta_result) so stale meta isn't left behind.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0698b19f-8c70-43f6-889a-1fc7ecf019f5
📒 Files selected for processing (5)
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: check API types
- GitHub Check: Seer Code Review
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (typescript)
- GitHub Check: end-to-end tests
- GitHub Check: type-check (3.13)
- GitHub Check: test (3.11)
- GitHub Check: test (3.13)
- GitHub Check: test (3.12)
- GitHub Check: Check PR Status
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
📓 Path-based instructions (3)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript_test.py
🧠 Learnings (22)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12797
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1991-2021
Timestamp: 2026-04-15T13:44:31.808Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/service.py` (`_run_stream_attempt`), the pre-create block (PR `#12797`) intentionally does NOT call `state.transcript_builder.append_assistant(...)` when inserting the empty assistant placeholder into `ctx.session.messages`. The transcript is left ending at the `tool_result` entry (N entries) while `message_count` metadata is N+1. This mismatch is benign and deliberate: on the next `--resume`, the SDK sees the transcript ending at `tool_result` and correctly regenerates the assistant response. Pre-appending the assistant turn to the transcript would suppress regeneration while leaving `session.messages[-1].content = ""` permanently (worse outcome). On the gap-fallback path, `transcript_msg_count (N+1) >= msg_count-1 (N)` means no gap is injected for the empty placeholder, which is correct because injecting an empty assistant message as context would mislead the SDK. Do NOT flag this transcript/message_count discrepancy as a bug.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/copilot/pending_messages.py:52-64
Timestamp: 2026-04-14T14:36:22.396Z
Learning: In `autogpt_platform/backend/backend/copilot` (PR `#12773`, commit d7bced0c6): when draining pending messages into `session.messages`, each message's text is sanitized via `strip_user_context_tags` before persistence to prevent user-controlled `<user_context>` injection from bypassing the trusted server-side context prefix. Additionally, if `upsert_chat_session` fails after draining, the drained `PendingMessage` objects are requeued back to Redis to avoid silent message loss. Do NOT flag the drain-then-requeue pattern as redundant — it is the intentional failure-resilience strategy for the pending buffer.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12445
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1071-1072
Timestamp: 2026-03-17T06:48:26.471Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), the AI SDK enforces `z.strictObject({type, errorText})` on SSE `StreamError` responses, so additional fields like `retryable: bool` cannot be added to `StreamError` or serialized via `to_sse()`. Instead, retry signaling for transient Anthropic API errors is done via the `COPILOT_RETRYABLE_ERROR_PREFIX` constant prepended to persisted session messages (in `ChatMessage.content`). The frontend detects retryable errors by checking `markerType === "retryable_error"` from `parseSpecialMarkers()` — no SSE schema changes and no string matching on error text. This pattern was established in PR `#12445`, commit 64d82797b.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/blocks/autopilot.py:631-638
Timestamp: 2026-04-14T07:35:09.273Z
Learning: In `autogpt_platform/backend/backend/copilot/executor/utils.py`, `CoPilotExecutionEntry` includes a `permissions: CopilotPermissions | None` field (added in PR `#12773` / commit a0184c87b9). `enqueue_copilot_turn` accepts and serializes this field into the queue entry, `_enqueue_for_recovery` in `autopilot.py` accepts and forwards `permissions` to `enqueue_copilot_turn`, and `_execute_async` in `processor.py` restores `entry.permissions` and passes it into `stream_chat_completion_sdk`/`stream_chat_completion_baseline` via `set_execution_context`. This ensures recovered sub-agent turns respect the same tool/block permission ceiling as the original in-process execution (mirroring `_merge_inherited_permissions`). Do NOT flag recovered turns as losing their permission ceiling — it is now fully propagated through the queue.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/transcript.py:47-53
Timestamp: 2026-04-03T11:14:25.653Z
Learning: In `autogpt_platform/backend/backend/copilot/transcript.py`, `TranscriptDownload` is intentionally a Python `dataclass` (not a Pydantic model). It is a simple internal data container with 3 fields (`content: str`, `message_count: int`, `uploaded_at: float`), has no validation logic, and does not cross API boundaries. Do not flag it for conversion to Pydantic — the overhead is not justified for this use case.
📚 Learning: 2026-04-15T13:44:31.808Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12797
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1991-2021
Timestamp: 2026-04-15T13:44:31.808Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/service.py` (`_run_stream_attempt`), the pre-create block (PR `#12797`) intentionally does NOT call `state.transcript_builder.append_assistant(...)` when inserting the empty assistant placeholder into `ctx.session.messages`. The transcript is left ending at the `tool_result` entry (N entries) while `message_count` metadata is N+1. This mismatch is benign and deliberate: on the next `--resume`, the SDK sees the transcript ending at `tool_result` and correctly regenerates the assistant response. Pre-appending the assistant turn to the transcript would suppress regeneration while leaving `session.messages[-1].content = ""` permanently (worse outcome). On the gap-fallback path, `transcript_msg_count (N+1) >= msg_count-1 (N)` means no gap is injected for the empty placeholder, which is correct because injecting an empty assistant message as context would mislead the SDK. Do NOT flag this transcript/message_count discrepancy as a bug.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-04-01T04:17:41.600Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12632
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-01T04:17:41.600Z
Learning: When reviewing AutoGPT Copilot tool implementations, accept that `readOnlyHint=True` (provided via `ToolAnnotations`) may be applied unconditionally to *all* tools—even tools that have side effects (e.g., `bash_exec`, `write_workspace_file`, or other write/save operations). Do **not** flag these tools for having `readOnlyHint=True`; this is intentional to enable fully-parallel dispatch by the Anthropic SDK/CLI and has been E2E validated. Only flag `readOnlyHint` issues if they conflict with the established `ToolAnnotations` behavior (e.g., missing/incorrect propagation relative to the intended annotation mechanism).
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/transcript.pyautogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-17T10:57:12.953Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-26T07:00:03.405Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12574
File: autogpt_platform/backend/backend/copilot/sdk/transcript.py:980-990
Timestamp: 2026-03-26T07:00:03.405Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/transcript.py`, `_rechain_tail` intentionally rewrites `parentUuid` for **all** tail entries (not just the first), because a single assistant turn can span multiple consecutive JSONL entries sharing the same `message.id` (e.g., a thinking entry + a tool_use entry). Their original `parentUuid` values may reference entries that were absorbed into the compressed prefix, so sequential rechaining of the entire tail is required to maintain a valid parent→child graph. The test `test_chains_multiple_tail_entries` validates this: the second tail entry's `parentUuid` is rewritten from its original value to the uuid of the first tail entry.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-17T06:48:26.471Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12445
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1071-1072
Timestamp: 2026-03-17T06:48:26.471Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), the AI SDK enforces `z.strictObject({type, errorText})` on SSE `StreamError` responses, so additional fields like `retryable: bool` cannot be added to `StreamError` or serialized via `to_sse()`. Instead, retry signaling for transient Anthropic API errors is done via the `COPILOT_RETRYABLE_ERROR_PREFIX` constant prepended to persisted session messages (in `ChatMessage.content`). The frontend detects retryable errors by checking `markerType === "retryable_error"` from `parseSpecialMarkers()` — no SSE schema changes and no string matching on error text. This pattern was established in PR `#12445`, commit 64d82797b.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-03T11:14:25.653Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/transcript.py:47-53
Timestamp: 2026-04-03T11:14:25.653Z
Learning: In `autogpt_platform/backend/backend/copilot/transcript.py`, `TranscriptDownload` is intentionally a Python `dataclass` (not a Pydantic model). It is a simple internal data container with 3 fields (`content: str`, `message_count: int`, `uploaded_at: float`), has no validation logic, and does not cross API boundaries. Do not flag it for conversion to Pydantic — the overhead is not justified for this use case.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/**/test/**/*.py : Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-04-13T14:19:19.341Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12740
File: autogpt_platform/frontend/src/app/api/openapi.json:0-0
Timestamp: 2026-04-13T14:19:19.341Z
Learning: Repo: Significant-Gravitas/AutoGPT — autogpt_platform
When adding new CoPilot tool response models (e.g., ScheduleListResponse, ScheduleDeletedResponse), update backend/api/features/chat/routes.py to include them in the ToolResponseUnion so the frontend’s autogenerated openapi.json dummy export (/api/chat/schema/tool-responses) exposes them for codegen. Do not hand-edit frontend/src/app/api/openapi.json.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-08T17:28:23.439Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/AGENTS.md:0-0
Timestamp: 2026-04-08T17:28:23.439Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : Mock at boundaries — mock where the symbol is **used**, not where it's **defined**; after refactoring, update mock targets to match new module paths
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.py
📚 Learning: 2026-04-03T11:14:45.569Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/baseline/service.py:0-0
Timestamp: 2026-04-03T11:14:45.569Z
Learning: In `autogpt_platform/backend/backend/copilot/baseline/service.py`, `transcript_builder.append_user(content=message)` is called unconditionally even when the message is a duplicate that was suppressed by the `is_new_message` guard. This is intentional: the downloaded transcript may be stale (uploaded before the previous attempt persisted the message), so always appending the current user turn prevents a malformed assistant-after-assistant transcript structure. The `is_user_message` flag is still checked (`if message and is_user_message:`), so assistant-role inputs are excluded. Do NOT flag this as a bug.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-03T11:14:16.378Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/transcript_builder.py:30-34
Timestamp: 2026-04-03T11:14:16.378Z
Learning: In `autogpt_platform/backend/backend/copilot/transcript_builder.py` (and its re-export shim at `sdk/transcript_builder.py`), `TranscriptEntry.parentUuid` is typed `str` (not `str | None`) and root entries use `parentUuid=""` (empty string) to match the canonical `_messages_to_transcript` JSONL format. `_parse_entry`, `append_user`, and `append_assistant` all coerce `None` to `""`. Do NOT flag `parentUuid=""` as incorrect — it is the correct root marker. This was fixed in PR `#12623`, commit b753cb7d0b.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-14T14:36:22.396Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/copilot/pending_messages.py:52-64
Timestamp: 2026-04-14T14:36:22.396Z
Learning: In `autogpt_platform/backend/backend/copilot` (PR `#12773`, commit d7bced0c6): when draining pending messages into `session.messages`, each message's text is sanitized via `strip_user_context_tags` before persistence to prevent user-controlled `<user_context>` injection from bypassing the trusted server-side context prefix. Additionally, if `upsert_chat_session` fails after draining, the drained `PendingMessage` objects are requeued back to Redis to avoid silent message loss. Do NOT flag the drain-then-requeue pattern as redundant — it is the intentional failure-resilience strategy for the pending buffer.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.py
📚 Learning: 2026-04-14T07:35:09.273Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/blocks/autopilot.py:631-638
Timestamp: 2026-04-14T07:35:09.273Z
Learning: In `autogpt_platform/backend/backend/copilot/executor/utils.py`, `CoPilotExecutionEntry` includes a `permissions: CopilotPermissions | None` field (added in PR `#12773` / commit a0184c87b9). `enqueue_copilot_turn` accepts and serializes this field into the queue entry, `_enqueue_for_recovery` in `autopilot.py` accepts and forwards `permissions` to `enqueue_copilot_turn`, and `_execute_async` in `processor.py` restores `entry.permissions` and passes it into `stream_chat_completion_sdk`/`stream_chat_completion_baseline` via `set_execution_context`. This ensures recovered sub-agent turns respect the same tool/block permission ceiling as the original in-process execution (mirroring `_merge_inherited_permissions`). Do NOT flag recovered turns as losing their permission ceiling — it is now fully propagated through the queue.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-23T06:51:32.535Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:0-0
Timestamp: 2026-03-23T06:51:32.535Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, the dedicated REST endpoint `/api/import/workflow` (and its earlier form `/api/import/competitor-workflow`) for external workflow import was removed across commits 30f801a5e, 374c8cfdb, and 732960e2d. The final architecture is a frontend-only flow: file upload is handled client-side, and URL fetching (for n8n templates) uses a Next.js server action (`fetchWorkflowFromUrl`) — there is no dedicated backend HTTP endpoint and no CoPilot tool invocation for workflow import. Do not expect or require a standalone HTTP endpoint, route-level integration tests, or a CoPilot `import_workflow` tool for workflow import in this repository.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-08T17:28:23.439Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/AGENTS.md:0-0
Timestamp: 2026-04-08T17:28:23.439Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : When creating snapshots in tests, use `poetry run pytest path/to/test.py --snapshot-update`; always review snapshot changes with `git diff` before committing
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.py
🔇 Additional comments (3)
autogpt_platform/backend/backend/copilot/sdk/service.py (3)
2429-2432: Good use ofasyncio.gather()after removing transcript I/O.This keeps the remaining independent startup work parallelized, so the refactor does not give back the latency win from dropping the redundant transcript fetch.
2456-2532: Nice consolidation to a single authoritative restore path.Using
restore_cli_session(...)for both native--resumestate andTranscriptBuilderseeding, with DB-message reconstruction as the fallback, keeps the SDK resume flow coherent in cross-pod cases.
3379-3386: Persisting the watermark with the CLI session upload is the right coupling.Passing
message_count=len(session.messages)ties the companion metadata to the exact session snapshot that was just persisted, which is what the new gap-fill restore flow needs.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/copilot/sdk/service.py (1)
3337-3399:⚠️ Potential issue | 🟠 MajorCatch
CancelledErrorhere and re-raise only after releasing the lock.In Python 3.11,
asyncio.CancelledErroris a subclass ofBaseException, notException. Whenasyncio.shield()raisesCancelledErrorin the awaiting task, this block'sexcept Exceptionclause does not catch it. The exception propagates immediately, skipping both_cleanup_sdk_tool_results()and the subsequentlock.release()in the finally block. This strands the per-session stream lock until its TTL expires.Suggested fix
+ upload_cancelled = False if ( config.claude_agent_use_resume and user_id and sdk_cwd and session is not None @@ try: await asyncio.shield( upload_cli_session( user_id=user_id, session_id=session_id, sdk_cwd=sdk_cwd, message_count=len(session.messages), log_prefix=log_prefix, ) ) + except asyncio.CancelledError: + upload_cancelled = True except Exception as cli_upload_err: logger.warning( "%s CLI session upload failed in finally: %s", log_prefix, cli_upload_err, ) try: if sdk_cwd: await _cleanup_sdk_tool_results(sdk_cwd) except Exception: logger.warning("%s SDK cleanup failed", log_prefix, exc_info=True) finally: # Release stream lock to allow new streams for this session await lock.release() + if upload_cancelled: + raise🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/sdk/service.py` around lines 3337 - 3399, The except Exception around the asyncio.shield(upload_cli_session(...)) block misses asyncio.CancelledError (a BaseException in Py3.11), which can skip _cleanup_sdk_tool_results and the lock.release in the finally; modify the try/except to catch BaseException (or explicitly asyncio.CancelledError) as cli_upload_err, perform the same cleanup and ensure lock.release is executed, then re-raise CancelledError (only for that type) after cleanup; update the upload_cli_session/asyncio.shield handling and the surrounding finally to guarantee _cleanup_sdk_tool_results and lock.release always run before re-raising.
♻️ Duplicate comments (2)
autogpt_platform/backend/backend/copilot/transcript.py (2)
758-779:⚠️ Potential issue | 🟠 MajorRollback the companion
.meta.jsonwhen the session upload fails.These two writes are concurrent, not atomic. If the
.jsonlupload fails but the.meta.jsonupload succeeds, the next restore can pair stale session bytes with a newermessage_count, which makes the gap-fill logic think more history is covered than actually exists.Suggested fix
session_result, meta_result = await asyncio.gather( storage.store(workspace_id=wid, file_id=fid, filename=fname, content=content), storage.store( workspace_id=mwid, file_id=mfid, filename=mfname, content=meta_encoded ), return_exceptions=True, ) if isinstance(session_result, Exception): + if not isinstance(meta_result, Exception): + try: + meta_path = _build_path_from_parts((mwid, mfid, mfname), storage) + await storage.delete(meta_path) + except Exception as cleanup_err: + logger.warning( + "%s Failed to roll back CLI session meta: %s", + log_prefix, + cleanup_err, + ) logger.warning( "%s Failed to upload CLI session file: %s", log_prefix, session_result ) return🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/transcript.py` around lines 758 - 779, When the concurrent uploads via storage.store for the session (.jsonl) and meta (.meta.json) complete, if session_result is an Exception but meta_result succeeded, delete/rollback the uploaded meta file to avoid mismatched state; specifically, in the branch that checks isinstance(session_result, Exception) (and before returning) call the storage deletion API (e.g., storage.delete or storage.remove) for the meta upload using the same identifiers used for the meta.store call (mwid, mfid, mfname), catch and log any deletion errors via logger.warning, then return.
825-841:⚠️ Potential issue | 🟠 MajorHarden restore error handling and
.meta.jsonparsing.This path still treats
BaseExceptionas a normal retrieve failure, soCancelledError/shutdown signals can be swallowed into a silentNonerestore. It also assumes the decoded metadata is a dict with a usable integermessage_count; a malformed or non-dict payload turns a best-effort watermark into an avoidable restore failure.Suggested fix
- if isinstance(content_result, BaseException): + if isinstance(content_result, Exception): logger.warning( "%s Failed to download CLI session: %s", log_prefix, content_result ) return None @@ - elif isinstance(meta_result, BaseException): + elif isinstance(meta_result, Exception): logger.debug("%s Failed to load CLI session meta: %s", log_prefix, meta_result) else: - meta = json.loads(meta_result.decode("utf-8"), fallback={}) - message_count = meta.get("message_count", 0) + try: + meta = json.loads(meta_result.decode("utf-8"), fallback={}) + except (UnicodeDecodeError, TypeError, ValueError): + meta = {} + if isinstance(meta, dict): + raw_count = meta.get("message_count", 0) + try: + message_count = max(0, int(raw_count)) + except (TypeError, ValueError): + message_count = 0As per coding guidelines, use
max(0, value)guards for computed values that should never be negative.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@autogpt_platform/backend/backend/copilot/transcript.py` around lines 825 - 841, The restore path must not swallow shutdown/cancellation exceptions and must robustly parse the .meta.json blob: when content_result or meta_result is an exception, re-raise critical signals (e.g., asyncio.CancelledError, KeyboardInterrupt, SystemExit) instead of returning None; for other exceptions log as before using logger and log_prefix. For meta_result, decode and parse safely inside a try/except around json.loads (don’t rely on a non-existent fallback parameter), ensure the parsed value is a dict before reading "message_count", coerce message_count = max(0, int(parsed.get("message_count", 0))) with safe fallback to 0 on type/parse errors, and keep the existing FileNotFoundError behavior for missing meta. Use the existing variable names content_result, meta_result, message_count, logger and log_prefix to locate where to apply these changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@autogpt_platform/backend/backend/copilot/sdk/service.py`:
- Around line 3337-3399: The except Exception around the
asyncio.shield(upload_cli_session(...)) block misses asyncio.CancelledError (a
BaseException in Py3.11), which can skip _cleanup_sdk_tool_results and the
lock.release in the finally; modify the try/except to catch BaseException (or
explicitly asyncio.CancelledError) as cli_upload_err, perform the same cleanup
and ensure lock.release is executed, then re-raise CancelledError (only for that
type) after cleanup; update the upload_cli_session/asyncio.shield handling and
the surrounding finally to guarantee _cleanup_sdk_tool_results and lock.release
always run before re-raising.
---
Duplicate comments:
In `@autogpt_platform/backend/backend/copilot/transcript.py`:
- Around line 758-779: When the concurrent uploads via storage.store for the
session (.jsonl) and meta (.meta.json) complete, if session_result is an
Exception but meta_result succeeded, delete/rollback the uploaded meta file to
avoid mismatched state; specifically, in the branch that checks
isinstance(session_result, Exception) (and before returning) call the storage
deletion API (e.g., storage.delete or storage.remove) for the meta upload using
the same identifiers used for the meta.store call (mwid, mfid, mfname), catch
and log any deletion errors via logger.warning, then return.
- Around line 825-841: The restore path must not swallow shutdown/cancellation
exceptions and must robustly parse the .meta.json blob: when content_result or
meta_result is an exception, re-raise critical signals (e.g.,
asyncio.CancelledError, KeyboardInterrupt, SystemExit) instead of returning
None; for other exceptions log as before using logger and log_prefix. For
meta_result, decode and parse safely inside a try/except around json.loads
(don’t rely on a non-existent fallback parameter), ensure the parsed value is a
dict before reading "message_count", coerce message_count = max(0,
int(parsed.get("message_count", 0))) with safe fallback to 0 on type/parse
errors, and keep the existing FileNotFoundError behavior for missing meta. Use
the existing variable names content_result, meta_result, message_count, logger
and log_prefix to locate where to apply these changes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 254d2145-4d9e-4318-9f9d-3f5318240c9b
📒 Files selected for processing (3)
autogpt_platform/backend/backend/copilot/sdk/service.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/transcript_test.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (3)
autogpt_platform/backend/**/*.py
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development
autogpt_platform/backend/**/*.py: Usepoetry run ...command for executing Python package dependencies
Use top-level imports only — avoid local/inner imports except for lazy imports of heavy optional dependencies likeopenpyxl
Use absolute imports withfrom backend.module import ...for cross-package imports; single-dot relative imports are acceptable for sibling modules within the same package; avoid double-dot relative imports
Do not use duck typing — avoidhasattr/getattr/isinstancefor type dispatch; use typed interfaces/unions/protocols instead
Use Pydantic models over dataclass/namedtuple/dict for structured data
Do not use linter suppressors — no# type: ignore,# noqa,# pyright: ignore; fix the type/code instead
Prefer list comprehensions over manual loop-and-append patterns
Use early return with guard clauses first to avoid deep nesting
Use%sfor deferred interpolation indebuglog statements for efficiency; use f-strings elsewhere for readability (e.g.,logger.debug("Processing %s items", count)vslogger.info(f"Processing {count} items"))
Sanitize error paths by usingos.path.basename()in error messages to avoid leaking directory structure
Be aware of TOCTOU (Time-Of-Check-Time-Of-Use) issues — avoid check-then-act patterns for file access and credit charging
Usetransaction=Truefor Redis pipelines to ensure atomicity on multi-step operations
Usemax(0, value)guards for computed values that should never be negative
Keep files under ~300 lines; if a file grows beyond this, split by responsibility (extract helpers, models, or a sub-module into a new file)
Keep functions under ~40 lines; extract named helpers when a function grows longer
...
Files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/{backend,autogpt_libs}/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Format Python code with
poetry run format
Files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/backend/**/*_test.py
📄 CodeRabbit inference engine (autogpt_platform/backend/AGENTS.md)
autogpt_platform/backend/**/*_test.py: Use pytest with snapshot testing for API responses
Colocate test files with source files using*_test.pynaming convention
Mock at boundaries — mock where the symbol is used, not where it's defined; after refactoring, update mock targets to match new module paths
UseAsyncMockfromunittest.mockfor async functions in tests
When writing tests, use Test-Driven Development (TDD): write failing tests marked with@pytest.mark.xfailbefore implementation, then remove the marker once the implementation is complete
When creating snapshots in tests, usepoetry run pytest path/to/test.py --snapshot-update; always review snapshot changes withgit diffbefore committing
Files:
autogpt_platform/backend/backend/copilot/transcript_test.py
🧠 Learnings (26)
📓 Common learnings
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12797
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1991-2021
Timestamp: 2026-04-15T13:44:31.808Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/service.py` (`_run_stream_attempt`), the pre-create block (PR `#12797`) intentionally does NOT call `state.transcript_builder.append_assistant(...)` when inserting the empty assistant placeholder into `ctx.session.messages`. The transcript is left ending at the `tool_result` entry (N entries) while `message_count` metadata is N+1. This mismatch is benign and deliberate: on the next `--resume`, the SDK sees the transcript ending at `tool_result` and correctly regenerates the assistant response. Pre-appending the assistant turn to the transcript would suppress regeneration while leaving `session.messages[-1].content = ""` permanently (worse outcome). On the gap-fallback path, `transcript_msg_count (N+1) >= msg_count-1 (N)` means no gap is injected for the empty placeholder, which is correct because injecting an empty assistant message as context would mislead the SDK. Do NOT flag this transcript/message_count discrepancy as a bug.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/copilot/pending_messages.py:52-64
Timestamp: 2026-04-14T14:36:22.396Z
Learning: In `autogpt_platform/backend/backend/copilot` (PR `#12773`, commit d7bced0c6): when draining pending messages into `session.messages`, each message's text is sanitized via `strip_user_context_tags` before persistence to prevent user-controlled `<user_context>` injection from bypassing the trusted server-side context prefix. Additionally, if `upsert_chat_session` fails after draining, the drained `PendingMessage` objects are requeued back to Redis to avoid silent message loss. Do NOT flag the drain-then-requeue pattern as redundant — it is the intentional failure-resilience strategy for the pending buffer.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12445
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1071-1072
Timestamp: 2026-03-17T06:48:26.471Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), the AI SDK enforces `z.strictObject({type, errorText})` on SSE `StreamError` responses, so additional fields like `retryable: bool` cannot be added to `StreamError` or serialized via `to_sse()`. Instead, retry signaling for transient Anthropic API errors is done via the `COPILOT_RETRYABLE_ERROR_PREFIX` constant prepended to persisted session messages (in `ChatMessage.content`). The frontend detects retryable errors by checking `markerType === "retryable_error"` from `parseSpecialMarkers()` — no SSE schema changes and no string matching on error text. This pattern was established in PR `#12445`, commit 64d82797b.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/blocks/autopilot.py:631-638
Timestamp: 2026-04-14T07:35:09.273Z
Learning: In `autogpt_platform/backend/backend/copilot/executor/utils.py`, `CoPilotExecutionEntry` includes a `permissions: CopilotPermissions | None` field (added in PR `#12773` / commit a0184c87b9). `enqueue_copilot_turn` accepts and serializes this field into the queue entry, `_enqueue_for_recovery` in `autopilot.py` accepts and forwards `permissions` to `enqueue_copilot_turn`, and `_execute_async` in `processor.py` restores `entry.permissions` and passes it into `stream_chat_completion_sdk`/`stream_chat_completion_baseline` via `set_execution_context`. This ensures recovered sub-agent turns respect the same tool/block permission ceiling as the original in-process execution (mirroring `_merge_inherited_permissions`). Do NOT flag recovered turns as losing their permission ceiling — it is now fully propagated through the queue.
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12284
File: autogpt_platform/frontend/src/app/api/openapi.json:11897-11900
Timestamp: 2026-03-04T23:58:18.476Z
Learning: Repo: Significant-Gravitas/AutoGPT — PR `#12284`
Backend/frontend OpenAPI codegen convention: In backend/api/features/store/model.py, the StoreSubmission and StoreSubmissionAdminView models define submitted_at: datetime | None, changes_summary: str | None, and instructions: str | None with no default. This is intentional to produce “required but nullable” fields in OpenAPI (properties appear in required[] and use anyOf [type, null]). This matches Prisma’s submittedAt DateTime? and changesSummary String?. Do not flag this as a required/nullable mismatch.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12574
File: autogpt_platform/backend/backend/copilot/sdk/transcript.py:980-990
Timestamp: 2026-03-26T07:00:03.405Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/transcript.py`, `_rechain_tail` intentionally rewrites `parentUuid` for **all** tail entries (not just the first), because a single assistant turn can span multiple consecutive JSONL entries sharing the same `message.id` (e.g., a thinking entry + a tool_use entry). Their original `parentUuid` values may reference entries that were absorbed into the compressed prefix, so sequential rechaining of the entire tail is required to maintain a valid parent→child graph. The test `test_chains_multiple_tail_entries` validates this: the second tail entry's `parentUuid` is rewritten from its original value to the uuid of the first tail entry.
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12740
File: autogpt_platform/frontend/src/app/api/openapi.json:0-0
Timestamp: 2026-04-13T14:19:19.341Z
Learning: Repo: Significant-Gravitas/AutoGPT — autogpt_platform
When adding new CoPilot tool response models (e.g., ScheduleListResponse, ScheduleDeletedResponse), update backend/api/features/chat/routes.py to include them in the ToolResponseUnion so the frontend’s autogenerated openapi.json dummy export (/api/chat/schema/tool-responses) exposes them for codegen. Do not hand-edit frontend/src/app/api/openapi.json.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/baseline/service.py:0-0
Timestamp: 2026-04-03T11:14:45.569Z
Learning: In `autogpt_platform/backend/backend/copilot/baseline/service.py`, `transcript_builder.append_user(content=message)` is called unconditionally even when the message is a duplicate that was suppressed by the `is_new_message` guard. This is intentional: the downloaded transcript may be stale (uploaded before the previous attempt persisted the message), so always appending the current user turn prevents a malformed assistant-after-assistant transcript structure. The `is_user_message` flag is still checked (`if message and is_user_message:`), so assistant-role inputs are excluded. Do NOT flag this as a bug.
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12356
File: autogpt_platform/backend/backend/copilot/constants.py:9-12
Timestamp: 2026-03-10T08:39:22.025Z
Learning: In Significant-Gravitas/AutoGPT PR `#12356`, the `COPILOT_SYNTHETIC_ID_PREFIX = "copilot-"` check in `create_auto_approval_record` (human_review.py) is intentional and safe. The `graph_exec_id` passed to this function comes from server-side `PendingHumanReview` DB records (not from user input); the API only accepts `node_exec_id` from users. Synthetic `copilot-*` IDs are only ever created server-side in `run_block.py`. The prefix skip avoids a DB lookup for a `AgentGraphExecution` record that legitimately does not exist for CoPilot sessions, while `user_id` scoping is enforced at the auth layer and on the resulting auto-approval record.
📚 Learning: 2026-04-15T13:44:31.808Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12797
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1991-2021
Timestamp: 2026-04-15T13:44:31.808Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/service.py` (`_run_stream_attempt`), the pre-create block (PR `#12797`) intentionally does NOT call `state.transcript_builder.append_assistant(...)` when inserting the empty assistant placeholder into `ctx.session.messages`. The transcript is left ending at the `tool_result` entry (N entries) while `message_count` metadata is N+1. This mismatch is benign and deliberate: on the next `--resume`, the SDK sees the transcript ending at `tool_result` and correctly regenerates the assistant response. Pre-appending the assistant turn to the transcript would suppress regeneration while leaving `session.messages[-1].content = ""` permanently (worse outcome). On the gap-fallback path, `transcript_msg_count (N+1) >= msg_count-1 (N)` means no gap is injected for the empty placeholder, which is correct because injecting an empty assistant message as context would mislead the SDK. Do NOT flag this transcript/message_count discrepancy as a bug.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-02-04T16:49:42.490Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2026-02-04T16:49:42.490Z
Learning: Applies to autogpt_platform/backend/**/test/**/*.py : Use snapshot testing with '--snapshot-update' flag in backend tests when output changes; always review with 'git diff'
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-04-08T17:28:23.439Z
Learnt from: CR
Repo: Significant-Gravitas/AutoGPT PR: 0
File: autogpt_platform/backend/AGENTS.md:0-0
Timestamp: 2026-04-08T17:28:23.439Z
Learning: Applies to autogpt_platform/backend/**/*_test.py : When creating snapshots in tests, use `poetry run pytest path/to/test.py --snapshot-update`; always review snapshot changes with `git diff` before committing
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-03-17T10:57:12.953Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/copilot/workflow_import/converter.py:0-0
Timestamp: 2026-03-17T10:57:12.953Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, `autogpt_platform/backend/backend/copilot/workflow_import/converter.py` was fully rewritten (commit 732960e2d) to no longer make direct LLM/OpenAI API calls. The converter now builds a structured text prompt for AutoPilot/CoPilot instead. There is no `response.choices` access or any direct LLM client usage in this file. Do not flag `response.choices` access or LLM client initialization patterns as issues in this file.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-26T07:00:03.405Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12574
File: autogpt_platform/backend/backend/copilot/sdk/transcript.py:980-990
Timestamp: 2026-03-26T07:00:03.405Z
Learning: In `autogpt_platform/backend/backend/copilot/sdk/transcript.py`, `_rechain_tail` intentionally rewrites `parentUuid` for **all** tail entries (not just the first), because a single assistant turn can span multiple consecutive JSONL entries sharing the same `message.id` (e.g., a thinking entry + a tool_use entry). Their original `parentUuid` values may reference entries that were absorbed into the compressed prefix, so sequential rechaining of the entire tail is required to maintain a valid parent→child graph. The test `test_chains_multiple_tail_entries` validates this: the second tail entry's `parentUuid` is rewritten from its original value to the uuid of the first tail entry.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.py
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-01T04:17:41.600Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12632
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-01T04:17:41.600Z
Learning: When reviewing AutoGPT Copilot tool implementations, accept that `readOnlyHint=True` (provided via `ToolAnnotations`) may be applied unconditionally to *all* tools—even tools that have side effects (e.g., `bash_exec`, `write_workspace_file`, or other write/save operations). Do **not** flag these tools for having `readOnlyHint=True`; this is intentional to enable fully-parallel dispatch by the Anthropic SDK/CLI and has been E2E validated. Only flag `readOnlyHint` issues if they conflict with the established `ToolAnnotations` behavior (e.g., missing/incorrect propagation relative to the intended annotation mechanism).
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-05T15:42:08.207Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12297
File: .claude/skills/backend-check/SKILL.md:14-16
Timestamp: 2026-03-05T15:42:08.207Z
Learning: In Python files under autogpt_platform/backend (recursively), rely on poetry run format to perform formatting (Black + isort) and linting (ruff). Do not run poetry run lint as a separate step after poetry run format, since format already includes linting checks.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-16T16:35:40.236Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:54-63
Timestamp: 2026-03-16T16:35:40.236Z
Learning: Avoid using the word 'competitor' in public-facing identifiers and text. Use neutral naming for API paths, model names, function names, and UI text. Examples: rename 'CompetitorFormat' to 'SourcePlatform', 'convert_competitor_workflow' to 'convert_workflow', '/competitor-workflow' to '/workflow'. Apply this guideline to files under autogpt_platform/backend and autogpt_platform/frontend.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-31T15:37:38.626Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/tools/agent_generator/fixer.py:37-47
Timestamp: 2026-03-31T15:37:38.626Z
Learning: When validating/constructing Anthropic API model IDs in Significant-Gravitas/AutoGPT, allow the hyphen-separated Claude Opus 4.6 model ID `claude-opus-4-6` (it corresponds to `LlmModel.CLAUDE_4_6_OPUS` in `autogpt_platform/backend/backend/blocks/llm.py`). Do NOT require the dot-separated form in Anthropic contexts. Only OpenRouter routing variants should use the dot separator (e.g., `anthropic/claude-opus-4.6`); `claude-opus-4-6` should be treated as correct when passed to Anthropic, and flagged only if it’s used in the OpenRouter path where the dot form is expected.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-15T02:43:36.890Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:0-0
Timestamp: 2026-04-15T02:43:36.890Z
Learning: When reviewing Python exception handlers, do not flag `isinstance(e, X)` checks as dead/unreachable if the caught exception `X` is a subclass of the exception type being handled. For example, if `X` (e.g., `VirusScanError`) inherits from `ValueError` (directly or via an intermediate class) and it can be raised within an `except ValueError:` block, then `isinstance(e, X)` inside that handler is reachable and should not be treated as dead code.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript_test.pyautogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-03T11:14:25.653Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/transcript.py:47-53
Timestamp: 2026-04-03T11:14:25.653Z
Learning: In `autogpt_platform/backend/backend/copilot/transcript.py`, `TranscriptDownload` is intentionally a Python `dataclass` (not a Pydantic model). It is a simple internal data container with 3 fields (`content: str`, `message_count: int`, `uploaded_at: float`), has no validation logic, and does not cross API boundaries. Do not flag it for conversion to Pydantic — the overhead is not justified for this use case.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-17T06:48:26.471Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12445
File: autogpt_platform/backend/backend/copilot/sdk/service.py:1071-1072
Timestamp: 2026-03-17T06:48:26.471Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), the AI SDK enforces `z.strictObject({type, errorText})` on SSE `StreamError` responses, so additional fields like `retryable: bool` cannot be added to `StreamError` or serialized via `to_sse()`. Instead, retry signaling for transient Anthropic API errors is done via the `COPILOT_RETRYABLE_ERROR_PREFIX` constant prepended to persisted session messages (in `ChatMessage.content`). The frontend detects retryable errors by checking `markerType === "retryable_error"` from `parseSpecialMarkers()` — no SSE schema changes and no string matching on error text. This pattern was established in PR `#12445`, commit 64d82797b.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-03T11:14:45.569Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/baseline/service.py:0-0
Timestamp: 2026-04-03T11:14:45.569Z
Learning: In `autogpt_platform/backend/backend/copilot/baseline/service.py`, `transcript_builder.append_user(content=message)` is called unconditionally even when the message is a duplicate that was suppressed by the `is_new_message` guard. This is intentional: the downloaded transcript may be stale (uploaded before the previous attempt persisted the message), so always appending the current user turn prevents a malformed assistant-after-assistant transcript structure. The `is_user_message` flag is still checked (`if message and is_user_message:`), so assistant-role inputs are excluded. Do NOT flag this as a bug.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-14T07:35:09.273Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/blocks/autopilot.py:631-638
Timestamp: 2026-04-14T07:35:09.273Z
Learning: In `autogpt_platform/backend/backend/copilot/executor/utils.py`, `CoPilotExecutionEntry` includes a `permissions: CopilotPermissions | None` field (added in PR `#12773` / commit a0184c87b9). `enqueue_copilot_turn` accepts and serializes this field into the queue entry, `_enqueue_for_recovery` in `autopilot.py` accepts and forwards `permissions` to `enqueue_copilot_turn`, and `_execute_async` in `processor.py` restores `entry.permissions` and passes it into `stream_chat_completion_sdk`/`stream_chat_completion_baseline` via `set_execution_context`. This ensures recovered sub-agent turns respect the same tool/block permission ceiling as the original in-process execution (mirroring `_merge_inherited_permissions`). Do NOT flag recovered turns as losing their permission ceiling — it is now fully propagated through the queue.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-14T14:36:22.396Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12773
File: autogpt_platform/backend/backend/copilot/pending_messages.py:52-64
Timestamp: 2026-04-14T14:36:22.396Z
Learning: In `autogpt_platform/backend/backend/copilot` (PR `#12773`, commit d7bced0c6): when draining pending messages into `session.messages`, each message's text is sanitized via `strip_user_context_tags` before persistence to prevent user-controlled `<user_context>` injection from bypassing the trusted server-side context prefix. Additionally, if `upsert_chat_session` fails after draining, the drained `PendingMessage` objects are requeued back to Redis to avoid silent message loss. Do NOT flag the drain-then-requeue pattern as redundant — it is the intentional failure-resilience strategy for the pending buffer.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.pyautogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-17T07:24:34.302Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12385
File: autogpt_platform/backend/backend/copilot/rate_limit.py:0-0
Timestamp: 2026-03-17T07:24:34.302Z
Learning: In `autogpt_platform/backend/backend/copilot/rate_limit.py`, all fail-open `except` blocks catch `(RedisError, ConnectionError, OSError)` specifically — not bare `except Exception`. This applies to `_session_reset_from_ttl`, `get_usage_status`, `check_rate_limit`, and `record_token_usage`. The narrowed tuple ensures only genuine Redis/network failures are swallowed; unexpected exceptions propagate normally.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.py
📚 Learning: 2026-04-15T02:06:38.113Z
Learnt from: ntindle
Repo: Significant-Gravitas/AutoGPT PR: 12780
File: autogpt_platform/backend/backend/util/workspace.py:198-221
Timestamp: 2026-04-15T02:06:38.113Z
Learning: In `autogpt_platform/backend/backend/util/workspace.py`, the TOCTOU gap between the `get_workspace_total_size()` pre-check and the subsequent storage write + DB insert in `WorkspaceManager.write_file()` is an accepted trade-off. Reasons: (1) the workspace is single-user-scoped, so a true race requires precise concurrent timing from the same user; (2) the REST upload route (`autogpt_platform/backend/backend/api/features/workspace/routes.py`) already has a post-write quota check with soft-delete rollback as a safety net for that path; (3) adding the same rollback inside `write_file()` would couple the manager to HTTP semantics. The pre-write check catches the overwhelming majority of cases. Do NOT flag this as a blocking TOCTOU issue.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.py
📚 Learning: 2026-04-14T06:33:59.422Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12774
File: autogpt_platform/backend/backend/copilot/tools/e2b_sandbox.py:0-0
Timestamp: 2026-04-14T06:33:59.422Z
Learning: In `autogpt_platform/backend/backend/copilot/tools/e2b_sandbox.py`, the `asyncio.wait_for()` retry loop around `AsyncSandbox.create()` (introduced in PR `#12774`) can leak up to `_SANDBOX_CREATE_MAX_RETRIES - 1` (≤2) orphaned E2B sandboxes per hang incident because `wait_for` cancels only the client-side wait while E2B may complete server-side provisioning. With the default `on_timeout="pause"` lifecycle, leaked orphaned sandboxes are **paused** (not killed) when their original `end_at` is reached and persist indefinitely until explicitly killed — there is NO automatic E2B project-level cleanup. Operators must manage these manually or via their own cleanup jobs. The sandbox_id is not accessible from the timed-out coroutine, so recovery via `AsyncSandbox.connect(sandbox_id)` is not possible at timeout. This is an intentionally accepted trade-off; a proper fix is deferred to a follow-up PR. Do NOT flag the retry loop as a blocking issue.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.py
📚 Learning: 2026-03-09T10:50:43.907Z
Learnt from: Bentlybro
Repo: Significant-Gravitas/AutoGPT PR: 0
File: :0-0
Timestamp: 2026-03-09T10:50:43.907Z
Learning: Repo: Significant-Gravitas/AutoGPT — File: autogpt_platform/backend/backend/blocks/llm.py
For xAI Grok models accessed via OpenRouter, the API returns `null` for `max_completion_tokens`. The convention in this codebase is to use the model's context window size as the `max_output_tokens` value in ModelMetadata. For example, Grok 3 uses 131072 (128k) and Grok 4 uses 262144 (256k). Do not flag these as incorrect max output token values.
Applied to files:
autogpt_platform/backend/backend/copilot/transcript.py
📚 Learning: 2026-03-23T06:51:32.535Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12440
File: autogpt_platform/backend/backend/api/features/workflow_import.py:0-0
Timestamp: 2026-03-23T06:51:32.535Z
Learning: In Significant-Gravitas/AutoGPT PR `#12440`, the dedicated REST endpoint `/api/import/workflow` (and its earlier form `/api/import/competitor-workflow`) for external workflow import was removed across commits 30f801a5e, 374c8cfdb, and 732960e2d. The final architecture is a frontend-only flow: file upload is handled client-side, and URL fetching (for n8n templates) uses a Next.js server action (`fetchWorkflowFromUrl`) — there is no dedicated backend HTTP endpoint and no CoPilot tool invocation for workflow import. Do not expect or require a standalone HTTP endpoint, route-level integration tests, or a CoPilot `import_workflow` tool for workflow import in this repository.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-13T14:19:19.341Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12740
File: autogpt_platform/frontend/src/app/api/openapi.json:0-0
Timestamp: 2026-04-13T14:19:19.341Z
Learning: Repo: Significant-Gravitas/AutoGPT — autogpt_platform
When adding new CoPilot tool response models (e.g., ScheduleListResponse, ScheduleDeletedResponse), update backend/api/features/chat/routes.py to include them in the ToolResponseUnion so the frontend’s autogenerated openapi.json dummy export (/api/chat/schema/tool-responses) exposes them for codegen. Do not hand-edit frontend/src/app/api/openapi.json.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-03T11:14:16.378Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12623
File: autogpt_platform/backend/backend/copilot/transcript_builder.py:30-34
Timestamp: 2026-04-03T11:14:16.378Z
Learning: In `autogpt_platform/backend/backend/copilot/transcript_builder.py` (and its re-export shim at `sdk/transcript_builder.py`), `TranscriptEntry.parentUuid` is typed `str` (not `str | None`) and root entries use `parentUuid=""` (empty string) to match the canonical `_messages_to_transcript` JSONL format. `_parse_entry`, `append_user`, and `append_assistant` all coerce `None` to `""`. Do NOT flag `parentUuid=""` as incorrect — it is the correct root marker. This was fixed in PR `#12623`, commit b753cb7d0b.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-04-01T14:54:01.937Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12636
File: autogpt_platform/backend/backend/copilot/sdk/service.py:0-0
Timestamp: 2026-04-01T14:54:01.937Z
Learning: In Significant-Gravitas/AutoGPT (autogpt_platform), `claude_agent_max_transient_retries` (default=3) in `ChatConfig` counts **total attempts including the initial one**, not the number of extra retries. With the pre-incremented `transient_retries >= max_transient` guard in `service.py`, a value of 3 yields 3 total stream attempts (initial + 2 retries with exponential backoff: 1s, 2s). Do NOT flag this as an off-by-one — the `>=` check is intentional.
Applied to files:
autogpt_platform/backend/backend/copilot/sdk/service.py
majdyz
added a commit
that referenced
this pull request
Apr 15, 2026
…complete sdk re-exports - Use len(source) instead of len(prior) in _build_query_message fallback warning so the logged count reflects the actual source being compressed - Add comment explaining retry path intentionally omits prior_messages and falls back to full DB context (authoritative, overhead acceptable) - Add missing cli_session_path, extract_context_messages, projects_base to sdk/transcript.py re-export for complete public API surface
Contributor
Author
|
Fixed in 33a7b83 — changed to |
Contributor
Author
|
Addressed in 33a7b83 — added explicit comment on the retry call explaining it intentionally uses full DB context rather than transcript+gap for the retry path. |
majdyz
added a commit
that referenced
this pull request
Apr 16, 2026
…e %-logging - _process_cli_restore: fix 'lines stripped' log metric — was reporting remaining line count; now correctly computes original_lines - remaining_lines - _restore_cli_session_for_turn: narrow broad 'except Exception' to (UnicodeDecodeError, ValueError, OSError) so unexpected programming errors in strip_for_upload / validate_transcript are not silently masked - _compress_messages: convert f-string logger.info to %-style lazy formatting to avoid unnecessary string interpolation when the log level is disabled
Contributor
Author
Round 3 review findings, addressed in add4f83Bug fix: Should Fix: Nit: |
Contributor
Author
E2E Test ReportBranch: fix/copilot-single-session-store Standard Tests
Invasive Tests
Edge Cases
Negative / Regression
Unit Tests
Key Behaviors Verified
Log EvidenceNote on BuildInitial test found stale container images (code without mode-gated Screenshots
|
…ng tool_call_id In both _session_messages_to_transcript (SDK path) and _append_gap_to_builder (baseline path), silently skipping a tool message with no tool_call_id can leave the TranscriptBuilder missing a tool_result entry, which would corrupt the JSONL conversation tree used by --resume. Replace the silent drop with an explicit warning so the issue is visible in logs, making it easier to diagnose data corruption rather than discovering it as a broken --resume session later.
Contributor
Author
Round 4 review findings, addressed in 8b9b2e0Should Fix: Both |
…ntent_blocks The unknown block type warning was using an f-string which forces string interpolation even when the log level is disabled. Convert to %-style lazy formatting for consistency with the rest of the logging in the module.
Contributor
Author
Round 5 review findings, addressed in 44e070cNit: All 5 rounds complete. Full commit trail: b05846d (round 1), 33a7b83 (round 2), add4f83 (round 3), 8b9b2e0 (round 4), 44e070c (round 5). |
_load_prior_transcript was returning (False, None) for missing/invalid transcripts, preventing the upload guard from firing. The intent was to protect against overwriting a *newer* GCS version — but a missing or corrupt file has nothing worth protecting. Only download errors (unknown GCS state) should suppress upload now. Root cause of the session 7803bde1 bug: the baseline turn ran with 23 session messages, no transcript existed in GCS (first baseline turn in that session), _load_prior_transcript returned (False, None), and should_upload_transcript gated the upload to False. The SDK's subsequent turn found no baseline JSONL and fell back to full DB reconstruction. Also renames transcript_covers_prefix → transcript_upload_safe throughout to accurately reflect the flag's semantics.
…n baseline Accessing session.messages[-1] raised IndexError when a session had no messages (e.g. tool-result submission with no message param). Changed to conditional expression so an empty session safely produces an empty messages_for_context list rather than crashing.
…s spurious gap-fills on tool-use turns
github-project-automation
Bot
moved this from 🆕 Needs initial review
to ✅ Done
in AutoGPT development kanban
majdyz
added a commit
that referenced
this pull request
Apr 16, 2026
Conflicts in baseline/service.py, baseline/transcript_integration_test.py, and transcript.py arose because dev-only commit 0cd0a76 (baseline upload fix) overlapped with the same fix in PR #12804 which landed in master. Took master's version for all three files — it is the complete, reviewed implementation.
Merged
3 tasks
majdyz
added a commit
that referenced
this pull request
Apr 16, 2026
… Pyright warnings (#12812) ## Why After PR #12804 was squashed into dev, two module-level helper functions in `backend/copilot/sdk/service.py` remained private (`_`-prefixed) while being directly imported by name in `sdk/transcript_test.py`. Pyright reports `reportAttributeAccessIssue` when tests (even those excluded from CI lint) import private symbols from outside their defining module. ## What Rename two helpers to remove the underscore prefix: - `_process_cli_restore` → `process_cli_restore` - `_read_cli_session_from_disk` → `read_cli_session_from_disk` Update call sites in `service.py` and imports/calls/docstrings in `sdk/transcript_test.py`. ## How Pure rename — no logic change. Both functions were already module-level helpers with no reason to be private; the underscore was convention carried over during the refactor but they are directly unit-tested and should be public. All 66 `sdk/transcript_test.py` tests pass after the rename. ## Checklist - [x] Tests pass (`poetry run pytest backend/copilot/sdk/transcript_test.py`) - [x] No `_`-prefixed symbols imported across module boundaries - [x] No linter suppressors added
majdyz
added a commit
that referenced
this pull request
Apr 16, 2026
Conflicts in baseline/service.py, baseline/transcript_integration_test.py, and transcript.py arose because dev-only commit 0cd0a76 (baseline upload fix) overlapped with the same fix in PR #12804 which landed in master. Took master's version for all three files — it is the complete, reviewed implementation.
Otto-AGPT
pushed a commit
that referenced
this pull request
Apr 16, 2026
…xt_messages, mode-gated --resume, compaction-aware gap-fill (#12804) **Why:** The copilot had two separate GCS paths (`cli-sessions/` and `chat-transcripts/`), redundant function names (`upload_cli_session`/`restore_cli_session`), and no shared context strategy between modes. When switching from baseline→SDK or SDK→baseline, the receiving mode discarded the stored transcript and fell back to full DB reconstruction — loading all raw messages instead of the compacted form — causing inflated context, wasted tokens, and loss of CLI compaction summaries. **What:** - Single GCS path (`cli-sessions/`) for both modes — `chat-transcripts/` removed - Unified public API: `upload_transcript` / `download_transcript` / `TranscriptDownload` - `TranscriptMode = Literal["sdk", "baseline"]` persisted in `.meta.json` — SDK skips `--resume` when `mode != "sdk"` (baseline-written JSONL has stripped fields / synthetic IDs) - `extract_context_messages(download, session_messages)` — shared context primitive used by **both SDK and baseline**: reads compacted transcript content + fills only the DB gap (messages after watermark), so CLI compaction summaries are preserved across mode switches - Watermark fix: `_jsonl_covered = transcript_msg_count + 2` when a real transcript is present, preventing false gap detection after `--resume` - Baseline gap-fill: `_append_gap_to_builder` converts `ChatMessage` → JSONL entries; no more silently discarded stale transcripts **How:** ``` SDK turn (mode="sdk" transcript available): ──► --resume [full CLI session restored natively] ──► inject gap prefix if DB has messages after watermark SDK turn (mode="baseline" transcript available): ──► cannot --resume (synthetic CLI IDs) ──► extract_context_messages(download, session_messages): returns transcript JSONL (compacted, isCompactSummary preserved) + gap excludes session_messages[-1] (current turn — caller injects it separately) ──► format as <conversation_history> + "Now, the user says: {current}" Baseline turn (any transcript): ──► _load_prior_transcript → TranscriptDownload ──► extract_context_messages(download, session_messages) + session_messages[-1] replaces full session.messages DB read ──► LLM messages: [compacted history + gap] + [current user turn] Transcript unavailable — both SDK (use_resume=False) and baseline: ──► extract_context_messages(None, session_messages) returns session_messages[:-1] (all prior DB messages except the current user turn at [-1]) ──► graceful fallback — no crash, no empty context ──► covers: first turn, GCS error, corrupt JSONL, missing .meta.json ──► next successful response uploads a fresh transcript ``` `extract_context_messages` is the shared primitive — both modes call the same function, which handles: - `download=None` (first turn, GCS unavailable) → falls back to `session_messages[:-1]` - Empty/corrupt content → falls back to `session_messages[:-1]` - `bytes` content (raw GCS) or `str` content (pre-decoded baseline path) - `isCompactSummary=True` entries → preserved so CLI compaction survives mode switches - Missing/corrupt `.meta.json` → `message_count` defaults to `0`, `mode` defaults to `"sdk"` **Why `[:-1]` and not all messages?** `session_messages[-1]` is always the current user turn being handled right now. Both callers inject it separately — SDK wraps it as `"Now, the user says: ..."`, baseline appends it as the final message in the LLM array. Returning it inside `extract_context_messages` would double-inject it. - **`transcript.py`**: `CliSessionRestore` → `TranscriptDownload` + `mode` field; `upload_cli_session` → `upload_transcript`; `restore_cli_session` → `download_transcript`; add `TranscriptMode`, `detect_gap`, `extract_context_messages`; import `ChatMessage` via relative path to match `service.py` style - **`sdk/service.py`**: mode-check before `--resume`; `_RestoreResult` carries `baseline_download` + `context_messages` + `transcript_content`; `_build_query_message` accepts `prior_messages` override; `_restore_cli_session_for_turn` populates `context_messages` via `extract_context_messages` and sets `transcript_content` to prevent duplicate DB reconstruction; watermark fix (`_jsonl_covered = transcript_msg_count + 2`) - **`baseline/service.py`**: `_load_prior_transcript` returns `(bool, TranscriptDownload | None)`; LLM context replaced with `extract_context_messages(download, messages)`; `_append_gap_to_builder` + `detect_gap` call; `upload_transcript(mode="baseline")` - **`sdk/transcript.py`**: updated re-exports, old aliases removed - **`scripts/download_transcripts.py`**: updated for `bytes | str` content type - **Test files**: 179 tests total; `transcript_test.py`, `baseline/transcript_integration_test.py`, `sdk/service_helpers_test.py`, `sdk/test_transcript_watermark.py`, `test/copilot/test_transcript_watermark.py` all updated/added - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] 179 unit tests pass — `transcript_test`, `baseline/transcript_integration_test`, `sdk/service_helpers_test`, `sdk/test_transcript_watermark` - [x] pyright 0 errors on all changed files - [x] SDK `--resume` path still works when `mode="sdk"` transcript is present - [x] SDK fallback uses `extract_context_messages` (compacted baseline content + gap) when `mode="baseline"` transcript is stored — no more full DB reconstruction - [x] Baseline uses `extract_context_messages` per turn instead of full `session.messages` DB read - [x] `isCompactSummary=True` entries preserved across mode switches - [x] Watermark (`_jsonl_covered`) fix prevents false gap detection after `--resume` - [x] Baseline gap detection no longer silently discards stale transcripts - [x] `TranscriptDownload.content` accepts `bytes | str` — backward compatible - [x] Transcript unavailable (GCS error, first turn, corrupt file) gracefully falls back to `session_messages[:-1]` without crash — applies to both SDK and baseline paths --------- Co-authored-by: chernistry <73943355+chernistry@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Otto-AGPT
pushed a commit
that referenced
this pull request
Apr 16, 2026
… Pyright warnings (#12812) ## Why After PR #12804 was squashed into dev, two module-level helper functions in `backend/copilot/sdk/service.py` remained private (`_`-prefixed) while being directly imported by name in `sdk/transcript_test.py`. Pyright reports `reportAttributeAccessIssue` when tests (even those excluded from CI lint) import private symbols from outside their defining module. ## What Rename two helpers to remove the underscore prefix: - `_process_cli_restore` → `process_cli_restore` - `_read_cli_session_from_disk` → `read_cli_session_from_disk` Update call sites in `service.py` and imports/calls/docstrings in `sdk/transcript_test.py`. ## How Pure rename — no logic change. Both functions were already module-level helpers with no reason to be private; the underscore was convention carried over during the refactor but they are directly unit-tested and should be public. All 66 `sdk/transcript_test.py` tests pass after the rename. ## Checklist - [x] Tests pass (`poetry run pytest backend/copilot/sdk/transcript_test.py`) - [x] No `_`-prefixed symbols imported across module boundaries - [x] No linter suppressors added
7 tasks
This was referenced May 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why / What / How
Why: The copilot had two separate GCS paths (
cli-sessions/andchat-transcripts/), redundant function names (upload_cli_session/restore_cli_session), and no shared context strategy between modes. When switching from baseline→SDK or SDK→baseline, the receiving mode discarded the stored transcript and fell back to full DB reconstruction — loading all raw messages instead of the compacted form — causing inflated context, wasted tokens, and loss of CLI compaction summaries.What:
cli-sessions/) for both modes —chat-transcripts/removedupload_transcript/download_transcript/TranscriptDownloadTranscriptMode = Literal["sdk", "baseline"]persisted in.meta.json— SDK skips--resumewhenmode != "sdk"(baseline-written JSONL has stripped fields / synthetic IDs)extract_context_messages(download, session_messages)— shared context primitive used by both SDK and baseline: reads compacted transcript content + fills only the DB gap (messages after watermark), so CLI compaction summaries are preserved across mode switches_jsonl_covered = transcript_msg_count + 2when a real transcript is present, preventing false gap detection after--resume_append_gap_to_builderconvertsChatMessage→ JSONL entries; no more silently discarded stale transcriptsHow:
extract_context_messagesis the shared primitive — both modes call the same function, which handles:download=None(first turn, GCS unavailable) → falls back tosession_messages[:-1]session_messages[:-1]bytescontent (raw GCS) orstrcontent (pre-decoded baseline path)isCompactSummary=Trueentries → preserved so CLI compaction survives mode switches.meta.json→message_countdefaults to0,modedefaults to"sdk"Why
[:-1]and not all messages?session_messages[-1]is always the current user turn being handled right now. Both callers inject it separately — SDK wraps it as"Now, the user says: ...", baseline appends it as the final message in the LLM array. Returning it insideextract_context_messageswould double-inject it.Changes 🏗️
transcript.py:CliSessionRestore→TranscriptDownload+modefield;upload_cli_session→upload_transcript;restore_cli_session→download_transcript; addTranscriptMode,detect_gap,extract_context_messages; importChatMessagevia relative path to matchservice.pystylesdk/service.py: mode-check before--resume;_RestoreResultcarriesbaseline_download+context_messages+transcript_content;_build_query_messageacceptsprior_messagesoverride;_restore_cli_session_for_turnpopulatescontext_messagesviaextract_context_messagesand setstranscript_contentto prevent duplicate DB reconstruction; watermark fix (_jsonl_covered = transcript_msg_count + 2)baseline/service.py:_load_prior_transcriptreturns(bool, TranscriptDownload | None); LLM context replaced withextract_context_messages(download, messages);_append_gap_to_builder+detect_gapcall;upload_transcript(mode="baseline")sdk/transcript.py: updated re-exports, old aliases removedscripts/download_transcripts.py: updated forbytes | strcontent typetranscript_test.py,baseline/transcript_integration_test.py,sdk/service_helpers_test.py,sdk/test_transcript_watermark.py,test/copilot/test_transcript_watermark.pyall updated/addedChecklist 📋
For code changes:
transcript_test,baseline/transcript_integration_test,sdk/service_helpers_test,sdk/test_transcript_watermark--resumepath still works whenmode="sdk"transcript is presentextract_context_messages(compacted baseline content + gap) whenmode="baseline"transcript is stored — no more full DB reconstructionextract_context_messagesper turn instead of fullsession.messagesDB readisCompactSummary=Trueentries preserved across mode switches_jsonl_covered) fix prevents false gap detection after--resumeTranscriptDownload.contentacceptsbytes | str— backward compatiblesession_messages[:-1]without crash — applies to both SDK and baseline paths