Skip to content

feat(copilot): add Langfuse tracing to baseline LLM path - #12281

Merged
majdyz merged 1 commit into
devfrom
feat/copilot-baseline-tracing
Mar 5, 2026
Merged

feat(copilot): add Langfuse tracing to baseline LLM path#12281
majdyz merged 1 commit into
devfrom
feat/copilot-baseline-tracing

Conversation

@majdyz

@majdyz majdyz commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Depends on #12276 (baseline code).

  • Swap shared OpenAI client to langfuse.openai.AsyncOpenAI — auto-captures all LLM calls (token usage, latency, model, prompts) as Langfuse generations when configured
  • Add propagate_attributes() context in baseline streaming for user_id/session_id attribution, matching the SDK path's OTEL tracing
  • No-op when Langfuse is not configured — langfuse.openai.AsyncOpenAI falls back to standard openai.AsyncOpenAI behavior

Observability parity

Aspect SDK path Baseline path (after this PR)
LLM call tracing OTEL via configure_claude_agent_sdk() langfuse.openai.AsyncOpenAI auto-instrumentation
User/session context propagate_attributes() propagate_attributes()
Langfuse prompts Shared _build_system_prompt() Shared _build_system_prompt()
Token/cost tracking Via OTEL spans Via Langfuse generation objects

Test plan

  • poetry run format passes (pyright, ruff, black, isort)
  • Verify Langfuse traces appear for baseline path with CHAT_USE_CLAUDE_AGENT_SDK=false
  • Verify SDK path tracing is unaffected

@majdyz
majdyz requested a review from a team as a code owner March 4, 2026 14:16
@majdyz
majdyz requested review from 0ubbe and kcze and removed request for a team March 4, 2026 14:16
@github-project-automation github-project-automation Bot moved this to 🆕 Needs initial review in AutoGPT development kanban Mar 4, 2026
@github-actions github-actions Bot added the conflicts Automatically applied to PRs with merge conflicts label Mar 4, 2026
@github-actions

github-actions Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions Bot added platform/backend AutoGPT Platform - Back end size/xl labels Mar 4, 2026
@coderabbitai

coderabbitai Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds Langfuse tracing: replaces OpenAI AsyncOpenAI with Langfuse AsyncOpenAI, wraps baseline streaming flow in Langfuse trace propagation/enter/teardown, and patches the OpenTelemetry TracerProvider Resource to include Langfuse environment.

Changes

Cohort / File(s) Summary
Baseline streaming trace propagation
autogpt_platform/backend/backend/copilot/baseline/service.py
Import and use propagate_attributes; create/enter a Langfuse trace context (user_id, session_id, trace_name "copilot-baseline", tag "baseline") around stream_chat_completion_baseline; ensure safe trace teardown in finally.
Langfuse OpenAI client swap
autogpt_platform/backend/backend/copilot/service.py
Replace openai.AsyncOpenAI client with langfuse.openai.AsyncOpenAI (Langfuse wrapper) for LLM calls; keep existing get_client() usage.
OTel TracerProvider resource patching
autogpt_platform/backend/backend/copilot/sdk/service.py
Add _patch_tracer_provider_resource(environment: str) and call it from _setup_langfuse_otel; remove reliance on OTEL_RESOURCE_ATTRIBUTES env patching and merge Langfuse environment into the active TracerProvider Resource.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant CopilotService
    participant Langfuse
    participant OpenAI
    participant StreamConsumer

    Client->>CopilotService: Request baseline stream (user_id, session_id)
    CopilotService->>Langfuse: propagate_attributes -> enter trace ("copilot-baseline", tag="baseline")
    CopilotService->>OpenAI: Stream request via LangfuseAsyncOpenAI
    OpenAI-->>CopilotService: Streaming chunks
    CopilotService->>StreamConsumer: yield StreamStart then stream chunks
    StreamConsumer-->>CopilotService: acknowledge/consume
    CopilotService->>Langfuse: exit / teardown trace (on complete/error)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • 0ubbe
  • ntindle
  • Swiftyos
  • kcze

Poem

🐰 I hop through traces, light and keen,

Session threads in fields of green.
Streams of tokens, snug and fine,
Langfuse ties each hop to shine.
🥕✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: adding Langfuse tracing to the baseline LLM path, which is the core objective of this PR.
Description check ✅ Passed The description provides relevant context about the changes, explaining the swap to langfuse.openai.AsyncOpenAI, the addition of propagate_attributes() context, and observability parity details aligned with the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/copilot-baseline-tracing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot mentioned this pull request Mar 4, 2026
8 tasks
@majdyz
majdyz force-pushed the feat/copilot-baseline-tracing branch from 201f8d0 to c9d55ae Compare March 4, 2026 14:26
@github-actions github-actions Bot removed the conflicts Automatically applied to PRs with merge conflicts label Mar 4, 2026
@github-actions

github-actions Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@github-actions github-actions Bot added the size/m label Mar 4, 2026
@github-actions

github-actions Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

🔍 PR Overlap Detection

This check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early.

🟢 Low Risk — File Overlap Only

These PRs touch the same files but different sections (click to expand)

Summary: 0 conflict(s), 0 medium risk, 3 low risk (out of 3 PRs with file overlap)


Auto-generated on push. Ignores: openapi.json, lock files.

@majdyz
majdyz force-pushed the feat/copilot-baseline-tracing branch from c9d55ae to 9d593f1 Compare March 4, 2026 14:32

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
autogpt_platform/backend/backend/copilot/baseline/service.py (1)

202-216: ⚠️ Potential issue | 🟡 Minor

Consider moving the try block to start immediately after entering the trace context.

The trace context is entered at line 210, but the try block doesn't begin until line 216. If an exception were to occur in lines 212-215 (unlikely but possible), the trace context would remain open since the finally block wouldn't execute.

🛠️ Suggested restructuring
     # Propagate user/session context to Langfuse so all LLM calls within
     # this request are grouped under a single trace with proper attribution.
     _trace_ctx = propagate_attributes(
         user_id=user_id,
         session_id=session_id,
         trace_name="copilot-baseline",
         tags=["baseline"],
     )
     _trace_ctx.__enter__()
-
-    assistant_text = ""
-    text_block_id = str(uuid.uuid4())
-    text_started = False
-    step_open = False
     try:
+        assistant_text = ""
+        text_block_id = str(uuid.uuid4())
+        text_started = False
+        step_open = False
         for _round in range(_MAX_TOOL_ROUNDS):
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@autogpt_platform/backend/backend/copilot/baseline/service.py` around lines
202 - 216, Move the try block to begin immediately after entering the Langfuse
trace context so the trace is always exited in the finally handler;
specifically, after calling _trace_ctx = propagate_attributes(...) and
_trace_ctx.__enter__(), start the try that currently begins later so any
exception raised between propagate_attributes/_trace_ctx.__enter__() and the
original try's start still flows through the existing finally which should call
_trace_ctx.__exit__ (or close the trace). Update the placement around the
variables assistant_text, text_block_id, text_started, and step_open so they
remain initialized inside the new try scope without changing their names.
🧹 Nitpick comments (1)
autogpt_platform/backend/backend/copilot/baseline/service.py (1)

398-403: Consider passing exception info to __exit__ for richer trace metadata.

Currently __exit__(None, None, None) is always called, even when an exception occurred. Passing the actual exception info would allow Langfuse to mark the trace as errored, improving observability.

♻️ Proposed enhancement
+    except Exception as e:
+        error_msg = str(e) or type(e).__name__
+        logger.error("[Baseline] Streaming error: %s", error_msg, exc_info=True)
+        # Close any open text/step before emitting error
+        if text_started:
+            yield StreamTextEnd(id=text_block_id)
+        if step_open:
+            yield StreamFinishStep()
+        yield StreamError(errorText=error_msg, code="baseline_error")
+        # Store exception info for trace context
+        _exc_info = (type(e), e, e.__traceback__)
+        # Still persist whatever we got
     finally:
         # Close Langfuse trace context
         try:
-            _trace_ctx.__exit__(None, None, None)
+            _trace_ctx.__exit__(*_exc_info if '_exc_info' in dir() else (None, None, None))
         except Exception:
             logger.warning("[Baseline] Langfuse trace context teardown failed")

Alternatively, use sys.exc_info() within the finally block if still in exception context.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@autogpt_platform/backend/backend/copilot/baseline/service.py` around lines
398 - 403, The finally block always calls _trace_ctx.__exit__(None, None, None)
which hides real exception information; change it to pass the actual exception
info to __exit__ (e.g., use sys.exc_info() or capture the exception tuple) so
Langfuse can mark traces as errored, and keep the existing try/except around
_trace_ctx.__exit__ to catch teardown failures and still call logger.warning if
teardown fails.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Around line 202-216: Move the try block to begin immediately after entering
the Langfuse trace context so the trace is always exited in the finally handler;
specifically, after calling _trace_ctx = propagate_attributes(...) and
_trace_ctx.__enter__(), start the try that currently begins later so any
exception raised between propagate_attributes/_trace_ctx.__enter__() and the
original try's start still flows through the existing finally which should call
_trace_ctx.__exit__ (or close the trace). Update the placement around the
variables assistant_text, text_block_id, text_started, and step_open so they
remain initialized inside the new try scope without changing their names.

---

Nitpick comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Around line 398-403: The finally block always calls _trace_ctx.__exit__(None,
None, None) which hides real exception information; change it to pass the actual
exception info to __exit__ (e.g., use sys.exc_info() or capture the exception
tuple) so Langfuse can mark traces as errored, and keep the existing try/except
around _trace_ctx.__exit__ to catch teardown failures and still call
logger.warning if teardown fails.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 575f6749-e93d-41e4-baa8-ac90c13e269d

📥 Commits

Reviewing files that changed from the base of the PR and between 0215332 and c9d55ae.

📒 Files selected for processing (2)
  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Seer Code Review
  • GitHub Check: types
  • GitHub Check: test (3.12)
  • GitHub Check: test (3.11)
  • GitHub Check: test (3.13)
  • GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
autogpt_platform/backend/**/*.{py,txt}

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use poetry run prefix for all Python commands, including testing, linting, formatting, and migrations

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
autogpt_platform/backend/backend/**/*.py

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
autogpt_platform/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
🧠 Learnings (2)
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.

Applied to files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.

Applied to files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
🔇 Additional comments (2)
autogpt_platform/backend/backend/copilot/service.py (1)

15-15: LGTM! Langfuse-wrapped AsyncOpenAI integration looks correct.

The langfuse.openai.AsyncOpenAI wrapper is a drop-in replacement that auto-instruments OpenAI API calls when Langfuse credentials are configured, falling back to standard behavior otherwise. The # type: ignore[attr-defined] comment is appropriate since the module doesn't explicitly export this in __all__.

Also applies to: 29-29

autogpt_platform/backend/backend/copilot/baseline/service.py (1)

16-16: LGTM!

Import for propagate_attributes is appropriate for establishing Langfuse trace context.

@majdyz
majdyz requested review from Swiftyos and ntindle March 4, 2026 14:45
@majdyz
majdyz enabled auto-merge March 4, 2026 14:45
@majdyz
majdyz force-pushed the feat/copilot-baseline-tracing branch from 9d593f1 to b3fd6e3 Compare March 4, 2026 14:50
Comment thread autogpt_platform/backend/backend/copilot/baseline/service.py Outdated
- Swap shared OpenAI client to langfuse.openai.AsyncOpenAI which
  auto-captures all LLM calls as Langfuse generations (token usage,
  latency, model, prompts) when Langfuse is configured
- Add propagate_attributes() context in baseline streaming for
  user_id/session_id attribution, matching the SDK path's tracing
@majdyz
majdyz force-pushed the feat/copilot-baseline-tracing branch from b3fd6e3 to c349149 Compare March 4, 2026 15:16

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
autogpt_platform/backend/backend/copilot/baseline/service.py (1)

202-214: Track whether __enter__ succeeded to avoid calling __exit__ on an un-entered context.

If propagate_attributes() succeeds but __enter__() raises, _trace_ctx will be non-None. The finally block will then call __exit__() on a context manager that was never entered, violating the context manager protocol.

While the try/except in finally handles this gracefully, it's cleaner to track entry success explicitly.

♻️ Suggested fix
     # Propagate user/session context to Langfuse so all LLM calls within
     # this request are grouped under a single trace with proper attribution.
     _trace_ctx: Any = None
+    _trace_entered = False
     try:
         _trace_ctx = propagate_attributes(
             user_id=user_id,
             session_id=session_id,
             trace_name="copilot-baseline",
             tags=["baseline"],
         )
         _trace_ctx.__enter__()
+        _trace_entered = True
     except Exception:
         logger.warning("[Baseline] Langfuse trace context setup failed")

Then in the finally block (lines 403-408):

         # Close Langfuse trace context
-        if _trace_ctx is not None:
+        if _trace_ctx is not None and _trace_entered:
             try:
                 _trace_ctx.__exit__(None, None, None)
             except Exception:
                 logger.warning("[Baseline] Langfuse trace context teardown failed")
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@autogpt_platform/backend/backend/copilot/baseline/service.py` around lines
202 - 214, propagate_attributes() may return a context manager but __enter__ can
fail, leaving _trace_ctx non-None and causing __exit__ to be called on an
un-entered context; modify the block that calls propagate_attributes() (the
_trace_ctx setup in service.py) to track entry success with a boolean (e.g.,
entered = False), set entered = True only after _trace_ctx.__enter__ completes,
and in the finally block only call _trace_ctx.__exit__ if _trace_ctx is not None
and entered is True (wrap the __exit__ call in a try/except as before for
safety).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Line 16: The current manual context management of Langfuse via
propagate_attributes and direct __enter__/__exit__ calls can call __exit__ on an
un-entered context; modify the trace setup in the function that creates
_trace_ctx (using propagate_attributes) to track enter success with a boolean
(e.g., _trace_ctx_entered = False), set it to True only after
_trace_ctx.__enter__() completes, wrap the __enter__() call in try/except to log
via logger.warning on failure, and in the finally/teardown block only call
_trace_ctx.__exit__(...) if _trace_ctx_entered is True, wrapping that call in
its own try/except to log teardown failures; reference the existing _trace_ctx,
propagate_attributes and logger symbols when making these changes.

---

Nitpick comments:
In `@autogpt_platform/backend/backend/copilot/baseline/service.py`:
- Around line 202-214: propagate_attributes() may return a context manager but
__enter__ can fail, leaving _trace_ctx non-None and causing __exit__ to be
called on an un-entered context; modify the block that calls
propagate_attributes() (the _trace_ctx setup in service.py) to track entry
success with a boolean (e.g., entered = False), set entered = True only after
_trace_ctx.__enter__ completes, and in the finally block only call
_trace_ctx.__exit__ if _trace_ctx is not None and entered is True (wrap the
__exit__ call in a try/except as before for safety).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ddd7101c-eb1b-4fa8-a078-87bcb6b2863b

📥 Commits

Reviewing files that changed from the base of the PR and between b3fd6e3 and c349149.

📒 Files selected for processing (2)
  • autogpt_platform/backend/backend/copilot/baseline/service.py
  • autogpt_platform/backend/backend/copilot/service.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • autogpt_platform/backend/backend/copilot/service.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: Seer Code Review
  • GitHub Check: types
  • GitHub Check: test (3.11)
  • GitHub Check: test (3.13)
  • GitHub Check: test (3.12)
  • GitHub Check: Analyze (python)
  • GitHub Check: Analyze (typescript)
  • GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
autogpt_platform/backend/**/*.{py,txt}

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use poetry run prefix for all Python commands, including testing, linting, formatting, and migrations

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
autogpt_platform/backend/backend/**/*.py

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
autogpt_platform/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
🧠 Learnings (2)
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.

Applied to files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.

Applied to files:

  • autogpt_platform/backend/backend/copilot/baseline/service.py
🔇 Additional comments (1)
autogpt_platform/backend/backend/copilot/baseline/service.py (1)

402-408: LGTM on the teardown pattern.

The finally block correctly ensures trace context cleanup, with proper null check and defensive exception handling to avoid blocking the main flow.

Comment thread autogpt_platform/backend/backend/copilot/baseline/service.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@autogpt_platform/backend/backend/copilot/sdk/service.py`:
- Around line 82-108: The current _patch_tracer_provider_resource function
mutates private attributes (_proxy, _resource) on the global provider and runs
at import time which is fragile; instead, detect when the global provider is an
SDKTracerProvider via get_tracer_provider()/SDKTracerProvider, build a new
Resource by merging existing resource + {"langfuse.environment": environment}
using opentelemetry.sdk.resources.Resource, create a new SDK TracerProvider (or
clone via public constructor) with that merged resource, and install it with
opentelemetry.trace.set_tracer_provider() at a safe time (e.g., defer until
first use or hook into app lifecycle in _setup_langfuse_otel) so you do not
access private attributes or mutate provider internals. Ensure you no longer
reference _proxy or _resource and perform replacement only when you know the
framework has finished initializing the global TracerProvider.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 33b1b9ff-ca0e-4384-8979-0d7b6461f739

📥 Commits

Reviewing files that changed from the base of the PR and between c349149 and d0dc18a.

📒 Files selected for processing (1)
  • autogpt_platform/backend/backend/copilot/sdk/service.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: types
  • GitHub Check: Seer Code Review
  • GitHub Check: test (3.12)
  • GitHub Check: test (3.13)
  • GitHub Check: test (3.11)
  • GitHub Check: Analyze (python)
  • GitHub Check: Check PR Status
🧰 Additional context used
📓 Path-based instructions (4)
autogpt_platform/backend/**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

autogpt_platform/backend/**/*.py: Use Python 3.11 (required; managed by Poetry via pyproject.toml) for backend development
Always run 'poetry run format' (Black + isort) before linting in backend development
Always run 'poetry run lint' (ruff) after formatting in backend development

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/backend/**/*.{py,txt}

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use poetry run prefix for all Python commands, including testing, linting, formatting, and migrations

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/backend/backend/**/*.py

📄 CodeRabbit inference engine (autogpt_platform/backend/CLAUDE.md)

Use Prisma ORM for database operations in PostgreSQL with pgvector for embeddings

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
autogpt_platform/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Format Python code with poetry run format

Files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
🧠 Learnings (2)
📚 Learning: 2026-02-26T17:02:22.448Z
Learnt from: Pwuts
Repo: Significant-Gravitas/AutoGPT PR: 12211
File: .pre-commit-config.yaml:160-179
Timestamp: 2026-02-26T17:02:22.448Z
Learning: Keep the pre-commit hook pattern broad for autogpt_platform/backend to ensure OpenAPI schema changes are captured. Do not narrow to backend/api/ alone, since the generated schema depends on Pydantic models across multiple directories (backend/data/, backend/blocks/, backend/copilot/, backend/integrations/, backend/util/). Narrowing could miss schema changes and cause frontend type desynchronization.

Applied to files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
📚 Learning: 2026-03-04T08:04:35.881Z
Learnt from: majdyz
Repo: Significant-Gravitas/AutoGPT PR: 12273
File: autogpt_platform/backend/backend/copilot/tools/workspace_files.py:216-220
Timestamp: 2026-03-04T08:04:35.881Z
Learning: In the AutoGPT Copilot backend, ensure that SVG images are not treated as vision image types by excluding 'image/svg+xml' from INLINEABLE_MIME_TYPES and MULTIMODAL_TYPES in tool_adapter.py; the Claude API supports PNG, JPEG, GIF, and WebP for vision. SVGs (XML text) should be handled via the text path instead, not the vision path.

Applied to files:

  • autogpt_platform/backend/backend/copilot/sdk/service.py
🔇 Additional comments (1)
autogpt_platform/backend/backend/copilot/sdk/service.py (1)

143-147: Cleaner approach than env-var patching; timing caveat applies.

Calling _patch_tracer_provider_resource after configure_claude_agent_sdk() is logically correct—LangSmith's helper should have installed its TracerProvider by then. The inline comment explains the rationale well.

The caveat from the previous comment (framework OTEL init timing) still applies. If manual verification confirms the TracerProvider is stable at this point, this is good to merge.

Comment thread autogpt_platform/backend/backend/copilot/sdk/service.py Outdated
Comment thread autogpt_platform/backend/backend/copilot/sdk/service.py
@majdyz
majdyz force-pushed the feat/copilot-baseline-tracing branch from a38ed94 to c349149 Compare March 5, 2026 09:27
@majdyz
majdyz added this pull request to the merge queue Mar 5, 2026
@github-project-automation github-project-automation Bot moved this from 🆕 Needs initial review to 👍🏼 Mergeable in AutoGPT development kanban Mar 5, 2026
Merged via the queue into dev with commit ce1675c Mar 5, 2026
40 of 42 checks passed
@majdyz
majdyz deleted the feat/copilot-baseline-tracing branch March 5, 2026 10:07
@github-project-automation github-project-automation Bot moved this from 👍🏼 Mergeable to ✅ Done in AutoGPT development kanban Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

platform/backend AutoGPT Platform - Back end size/m size/xl

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

2 participants