Skip to content

feat(copilot): E2B cloud sandbox — unified file tools, persistent execution, output truncation - #12212

Merged
majdyz merged 39 commits into
devfrom
feat/improve-copilot-file
Mar 3, 2026
Merged

feat(copilot): E2B cloud sandbox — unified file tools, persistent execution, output truncation#12212
majdyz merged 39 commits into
devfrom
feat/improve-copilot-file

Conversation

@majdyz

@majdyz majdyz commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • E2B file tools: New MCP tools (read_file/write_file/edit_file/glob/grep) that operate directly on the E2B sandbox filesystem (/home/user). When E2B is active, these replace SDK built-in Read/Write/Edit/Glob/Grep so all tools share a single coherent filesystem with bash_exec — no sync needed.
  • E2B sandbox lifecycle: New e2b_sandbox.py manages sandbox creation and reconnection via Redis, with stale-key cleanup on reconnection failure.
  • E2B enabled by default: use_e2b_sandbox defaults to True; set CHAT_USE_E2B_SANDBOX=false to disable.
  • Centralized output truncation: All MCP tool outputs are truncated via _truncating wrapper and stashed (_pending_tool_outputs) to bypass SDK's head-truncation for the frontend.
  • Frontend tool display: GenericTool.tsx now renders bash stdout/stderr, file content, edit diffs (old/new), todo lists, and glob/grep results with category-specific icons and status text.
  • Workspace file tools + E2B: read_workspace_file's save_to_path and write_workspace_file's source_path route to E2B sandbox when active.

Files changed

Area Files What
E2B file tools sdk/e2b_file_tools.py, sdk/e2b_file_tools_test.py MCP file tool handlers + tests
E2B sandbox tools/e2b_sandbox.py Sandbox lifecycle (create/reconnect/Redis)
Tool adapter sdk/tool_adapter.py MCP server, truncation, stash, path validation
Service sdk/service.py E2B integration, prompt supplements
Security sdk/security_hooks.py, sdk/security_hooks_test.py Path validation for E2B mode
Bash exec tools/bash_exec.py E2B execution path
Workspace files tools/workspace_files.py, tools/workspace_files_test.py E2B-aware save/source paths
Config copilot/config.py E2B config fields (default on)
Truncation util/truncate.py Middle-out truncation fix
Frontend GenericTool.tsx Tool-specific display rendering

Test plan

  • security_hooks_test.py — 43 tests (path validation, tool access, deny messages)
  • e2b_file_tools_test.py — 19 tests (path resolution, local read safety)
  • workspace_files_test.py — 17 tests (ephemeral path validation)
  • CI green (backend 3.11/3.12/3.13, lint, types, e2e)

The tool supplement previously used a static `/tmp/copilot-<session>/`
placeholder, so the agent had no idea what its real working directory was
and wasted turns probing wrong paths before getting an error. Now the
exact cwd is pre-computed and formatted into the prompt so the agent
knows immediately where it can read and write.
@majdyz
majdyz requested a review from a team as a code owner February 26, 2026 14:07
@majdyz
majdyz requested review from Pwuts and kcze and removed request for a team February 26, 2026 14:07
@github-project-automation github-project-automation Bot moved this to 🆕 Needs initial review in AutoGPT development kanban Feb 26, 2026

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@github-actions github-actions Bot added the platform/backend AutoGPT Platform - Back end label Feb 26, 2026
@github-actions

github-actions Bot commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

🔍 PR Overlap Detection

This check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early.

🔴 Merge Conflicts Detected

The following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.

🟢 Low Risk — File Overlap Only

These PRs touch the same files but different sections (click to expand)

Summary: 2 conflict(s), 0 medium risk, 6 low risk (out of 8 PRs with file overlap)


Auto-generated on push. Ignores: openapi.json, lock files.

@coderabbitai

coderabbitai Bot commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds optional E2B sandbox support to CoPilot: E2B-aware tool selection and disallowed lists, sandbox lifecycle with Redis persistence, two-way workspace sync, E2B-backed MCP file tools and E2B execution path for bash_exec, config flags/validators, async workspace-file handling, tests and compose entries for end-to-end validation.

Changes

Cohort / File(s) Summary
SDK service (streaming & prompts)
autogpt_platform/backend/backend/copilot/sdk/service.py
Make tool supplement E2B-aware via _build_sdk_tool_supplement(..., use_e2b=...); initialize/persist sandbox when configured; pass sandbox/use_e2b through prompt/tool flows; use dynamic tool getters.
Tool adapter / MCP registration
autogpt_platform/backend/backend/copilot/sdk/tool_adapter.py
Add sandbox ContextVar, set_execution_context(..., sandbox=...), get_current_sandbox(); create_copilot_mcp_server(use_e2b=...); add get_copilot_tool_names and get_sdk_disallowed_tools; adjust tool registration for E2B.
E2B sandbox lifecycle & sync
autogpt_platform/backend/backend/copilot/tools/e2b_sandbox.py
New Redis-backed sandbox lifecycle (get_or_create_sandbox) and sync primitives (sync_from_sandbox, sync_to_sandbox) with size limits and per-file resilience.
E2B file-backed MCP tools
autogpt_platform/backend/backend/copilot/sdk/e2b_file_tools.py
New MCP file-tool implementations routing file ops to sandbox (read/write/edit/glob/grep); exports E2B_FILE_TOOLS and E2B_FILE_TOOL_NAMES.
bash_exec tool (dual execution)
autogpt_platform/backend/backend/copilot/tools/bash_exec.py
Add E2B remote execution path (_execute_on_e2b) with timeout handling and shell quoting; retain local bubblewrap fallback; add _E2B_WORKDIR.
Workspace file helpers & tests
autogpt_platform/backend/backend/copilot/tools/workspace_files.py, autogpt_platform/backend/backend/copilot/tools/workspace_files_test.py
Make _resolve_write_content async and sandbox-aware; add _read_source_path and _resolve_sandbox_path; update tests to async/await.
Config (E2B flags & validators)
autogpt_platform/backend/backend/copilot/config.py
Add use_e2b_sandbox, e2b_api_key, e2b_sandbox_template, e2b_sandbox_timeout plus env-aware pre-validators for sandbox flags and API key resolution.
Integration test & compose
autogpt_platform/backend/test_e2b_integration.py, autogpt_platform/docker-compose.platform.yml, autogpt_platform/docker-compose.yml
Add end-to-end E2B integration test and e2b_integration_test service; make copilot_executor unbuffered in compose.
Dockerfile comment
autogpt_platform/backend/Dockerfile
Clarify comment about bash_exec fallback when E2B is not configured; no behavior change.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant SDK_Service as SDK Service
  participant Redis
  participant E2B as E2B Sandbox
  participant LocalFS as Local Filesystem
  participant Tool as bash_exec/MCP

  Client->>SDK_Service: stream_chat_completion_sdk(session_id, ...)
  SDK_Service->>Redis: get_or_create_sandbox(session_id, api_key, template, timeout) [if configured]
  Redis-->>SDK_Service: sandbox_id / metadata
  SDK_Service->>E2B: initialize AsyncSandbox (get_or_create_sandbox)
  SDK_Service->>LocalFS: ensure sdk_cwd exists
  SDK_Service->>E2B: sync_from_sandbox(sandbox, sdk_cwd)
  SDK_Service->>SDK_Service: build system prompt + _build_sdk_tool_supplement(cwd, use_e2b)
  SDK_Service->>Tool: set_execution_context(user_id, session, sandbox) and invoke tools
  alt sandbox present
    Tool->>E2B: _execute_on_e2b(command, workdir, env, timeout)
    E2B-->>Tool: execution result / files
  else no sandbox
    Tool->>LocalFS: run bubblewrap-local command
    LocalFS-->>Tool: execution result / files
  end
  SDK_Service->>E2B: sync_to_sandbox(sandbox, sdk_cwd)  /* persist outputs */
  SDK_Service-->>Client: stream completion / tool results
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

Possible security concern, Review effort 4/5

Suggested reviewers

  • Pwuts
  • ntindle
  • 0ubbe

Poem

🐰 I found a cloud, a snug sandbox den,
I sync my carrots, then hop back again,
Commands that wander, files that roam,
Turn by turn they always come home,
Hooray — our workspace has a cozy den! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 48.94% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main focus: introducing E2B cloud sandbox integration with unified file tools and persistent execution, which aligns with the primary objectives of the PR.
Description check ✅ Passed The description comprehensively covers the key changes (E2B file tools, sandbox lifecycle, workspace integration, truncation, config defaults) and provides context on test coverage and implementation approach.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/improve-copilot-file

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@autogpt_platform/backend/backend/copilot/sdk/service.py`:
- Around line 461-465: Compute the SDK working directory once with
_precomputed_cwd = _make_sdk_cwd(session_id) and use that same variable
everywhere instead of recomputing cwd later (remove the second call that
re-derives cwd around the code that currently uses it), and ensure any
exceptions from _make_sdk_cwd propagate into the main error-handling path
(either by awaiting it where exceptions bubble up or by catching and
re-raising/logging via the existing stream error handler). Update references to
the recomputed cwd to use _precomputed_cwd and keep the call to
_build_sdk_tool_supplement(_precomputed_cwd) as the single source-of-truth so
the prompt and execution directory cannot drift.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 29ca034 and aa4ed81.

📒 Files selected for processing (1)
  • autogpt_platform/backend/backend/copilot/sdk/service.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: types
  • GitHub Check: test (3.12)
  • GitHub Check: test (3.13)
  • GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/backend/**/*.{py,txt}

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use poetry run prefix for all Python commands, including testing, linting, formatting, and migrations

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/backend/backend/**/*.py

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
🧬 Code graph analysis (1)
autogpt_platform/backend/backend/copilot/sdk/service.py (1)
autogpt_platform/backend/backend/copilot/service.py (1)
  • _build_system_prompt (211-242)
🔇 Additional comments (1)
autogpt_platform/backend/backend/copilot/sdk/service.py (1)

86-139: Good improvement to tool supplement clarity and deterministic cwd injection.

This segment cleanly replaces the placeholder path with a concrete cwd and adds a clearer “Working directory” section for the agent.

Comment thread autogpt_platform/backend/backend/copilot/sdk/service.py Outdated
@github-actions github-actions Bot added the platform/frontend AutoGPT Platform - Front end label Feb 26, 2026
- Wrap _make_sdk_cwd() in try/except so a ValueError yields a clean
  StreamError rather than propagating outside the stream error path
- Reuse _precomputed_cwd inside the try block instead of recomputing,
  ensuring the system prompt and execution directory cannot drift
- Reset frontend files (useCredits.ts, client.ts) to dev state — they
  were carried in from a master hotfix that is not yet in dev
@github-actions github-actions Bot removed the platform/frontend AutoGPT Platform - Front end label Feb 26, 2026
makedirs can raise OSError which was previously outside the stream
error-handling path. Move it into the early try/except alongside
_make_sdk_cwd, widen the catch to (ValueError, OSError), and remove
the now-redundant sdk_cwd = "" initialiser — sdk_cwd is assigned
directly from _precomputed_cwd before the main try block.
_precomputed_cwd was a pointless intermediate. sdk_cwd is now set
directly from _make_sdk_cwd() in the early try/except and used
everywhere, eliminating the redundant variable.
…execution

bash_exec now routes to E2B sandbox.commands.run() when CHAT_USE_E2B_SANDBOX=true,
giving full internet access, persistent /home/user across turns, and pre-installed
packages — without requiring any special container capabilities.

Workspace sync approach (replaces FUSE/sshfs which can't reach E2B's HTTP-proxy
infrastructure): at turn start, files are downloaded from E2B /home/user → sdk_cwd
via E2B's files API so SDK tools see the previous turn's state; at turn end,
local files are uploaded back so bash_exec sees SDK-tool writes next turn.

Config: set CHAT_USE_E2B_SANDBOX=true and E2B_API_KEY in backend/.env.

New files:
- backend/copilot/tools/e2b_sandbox.py  — sandbox lifecycle + sync helpers
- backend/test_e2b_integration.py       — docker-compose e2e test

Changes:
- config.py            — added use_e2b_sandbox, e2b_api_key, template, timeout fields
- tool_adapter.py      — added _current_sandbox context var + get/set helpers
- bash_exec.py         — E2B execution path via get_current_sandbox()
- service.py           — E2B setup (connect/create + sync_from) and teardown (sync_to)
- Dockerfile           — updated comment (no new packages needed)
- docker-compose.yml   — added e2b_integration_test service (profile: e2b-test)
@majdyz majdyz changed the title fix(copilot): inject actual working directory into SDK system prompt feat(copilot): E2B cloud sandbox integration for persistent bash execution Feb 26, 2026
@majdyz
majdyz marked this pull request as draft February 26, 2026 16:02
auto-merge was automatically disabled February 26, 2026 16:02

Pull request was converted to draft

Comment thread autogpt_platform/backend/backend/copilot/tools/bash_exec.py Outdated
Covers the asyncio.TimeoutError path when E2B API calls exceed
the 10s wait_for timeout in kill_sandbox.

@autogpt-reviewer autogpt-reviewer left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #12212 — feat(copilot): E2B cloud sandbox — unified file tools, persistent execution, output truncation
Author: majdyz | RE-REVIEW v9 (bc5767241da564) | Delta: 1 commit, 1 file changed (+16 lines) | CI: ✅ All green

🎯 Verdict: APPROVE

What Changed Since Last Approval (v8)

Single commit 41da5646test(copilot): add kill_sandbox timeout test

Adds test_kill_timeout_returns_false to TestKillSandbox in e2b_sandbox_test.py. The test:

  • Patches asyncio.wait_for to raise asyncio.TimeoutError
  • Asserts kill_sandbox() returns False (graceful degradation, not crash)
  • Asserts Redis key is still cleaned up despite timeout

This was the sole remaining should-fix from our v8 review. It is now addressed. ✅

Specialist Findings (8/8 reported)

🛡️ Security ✅ — No security concerns. Test-only change with mocked dependencies. Validates Redis cleanup on timeout (prevents stale sandbox references). Path traversal protections intact from v8.

🏗️ Architecture ✅ — Clean test, correct abstraction boundary. Patches asyncio.wait_for at module level rather than reaching into E2B internals. Follows established _mock_redis/_patch_redis patterns.

Performance ✅ — Zero production code changes. No performance implications.

🧪 Testing ✅ — Well-structured test covering the last uncovered branch in kill_sandbox. The TestKillSandbox class now has 6 tests covering all branches: no key, creating state, happy path, connect failure, bytes Redis value, and timeout. Full branch coverage achieved.

📖 Quality ✅ — Excellent naming, docstring present, follows existing test patterns consistently. Proper arrange-act-assert structure.

📦 Product ✅ — No user-facing changes. Strictly positive test coverage improvement.

📬 Discussion ✅ — All review threads resolved. Sentry bot findings are pre-existing patterns already accepted at v8. No new human reviewer comments. CI fully green (all 20+ checks).

🔎 QA ✅ — Frontend loads successfully (login page renders). Backend responsive. No regressions observed.

QA Screenshots

Landing
After Load

Blockers

None.

Risk Assessment

Merge risk: LOW (feature-flagged, safely falls back to bubblewrap) | Rollback: EASY (set CHAT_USE_E2B_SANDBOX=false)

@ntindle Single commit adds the timeout test we flagged as "should-fix" in v8. All 8 specialists approve unanimously. 9th iteration — all feedback from 8 rounds fully incorporated. This is ready. Recommend merge.

@majdyz
majdyz added this pull request to the merge queue Mar 3, 2026
@github-project-automation github-project-automation Bot moved this from 🆕 Needs initial review to 👍🏼 Mergeable in AutoGPT development kanban Mar 3, 2026
@github-actions github-actions Bot mentioned this pull request Mar 3, 2026
12 tasks
Merged via the queue into dev with commit 29da8db Mar 3, 2026
28 checks passed
@majdyz
majdyz deleted the feat/improve-copilot-file branch March 3, 2026 21:52
@github-project-automation github-project-automation Bot moved this to Done in Frontend Mar 3, 2026
@github-project-automation github-project-automation Bot moved this from 👍🏼 Mergeable to ✅ Done in AutoGPT development kanban Mar 3, 2026
@sentry

sentry Bot commented Mar 6, 2026

Copy link
Copy Markdown

Issues attributed to commits in this pull request

This pull request was merged and Sentry observed the following issues:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

platform/backend AutoGPT Platform - Back end platform/frontend AutoGPT Platform - Front end size/m size/xl

Projects

Status: ✅ Done
Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants