Skip to content

feat: return annotation creation outcome - #4412

Open
AmatyaJoshi wants to merge 2 commits into
traceloop:mainfrom
AmatyaJoshi:fix/annotation-feedback-outcome
Open

feat: return annotation creation outcome#4412
AmatyaJoshi wants to merge 2 commits into
traceloop:mainfrom
AmatyaJoshi:fix/annotation-feedback-outcome

Conversation

@AmatyaJoshi

@AmatyaJoshi AmatyaJoshi commented Aug 11, 2026

Copy link
Copy Markdown

Summary

Adds structured outcomes for annotation creation so callers can distinguish between successful delivery, server refusal, and transport failures.

Changes

  • Added AnnotationCreateStatus with DELIVERED, REFUSED, and UNREACHABLE states.
  • Added AnnotationCreateResult containing status, response payload, HTTP status code, and error details.
  • Updated BaseAnnotation.create() to return structured results.
  • Updated UserFeedback.create() to expose the result.
  • Added raise_on_error support to HTTPClient.post() while preserving existing behavior by default.
  • Added tests covering successful feedback creation, HTTP errors, transport errors, and HTTP client behavior.

Validation

  • npx nx run traceloop-sdk:lint — passed
  • Contribution-specific tests — 14 passed
  • mypy traceloop/sdk — passed
  • Full test suite — 440 passed, 3 pre-existing/environment-specific failures

The three full-suite failures are unrelated to these changes:

  • Two Windows temporary-file cleanup failures (WinError 32)
  • One timestamp assertion failure in test_manual_report

Summary by CodeRabbit

New Features

  • Annotation and user feedback submissions now return clear delivery statuses: delivered, refused, or unreachable.
  • Submission results include available response details, HTTP status codes, and error information.
  • HTTP POST operations can optionally propagate request errors to calling applications.

Bug Fixes

  • Improved handling of successful empty responses, non-JSON responses, HTTP failures, and network errors.
  • Improved compatibility for dataset publishing and experiment task creation when requests return no response.

@CLAassistant

CLAassistant commented Aug 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3a82c0e1-9ada-4912-a124-f129e9c1b33d

📥 Commits

Reviewing files that changed from the base of the PR and between 1bc37d1 and c3444b4.

📒 Files selected for processing (2)
  • packages/traceloop-sdk/tests/test_http_client_post.py
  • packages/traceloop-sdk/traceloop/sdk/client/http.py

📝 Walkthrough

Walkthrough

The SDK now reports annotation delivery outcomes through AnnotationCreateResult. HTTP POST supports optional exception propagation. Tests cover successful delivery, HTTP refusal, unreachable services, response payloads, and compatibility behavior.

Changes

Annotation delivery outcomes

Layer / File(s) Summary
HTTP POST error control
packages/traceloop-sdk/traceloop/sdk/client/http.py, packages/traceloop-sdk/tests/test_http_client_post.py
HTTPClient.post accepts keyword-only raise_on_error. Tests cover JSON, empty, and text responses, default failure handling, opt-in exception propagation, and existing None failure behavior.
Annotation result model and handling
packages/traceloop-sdk/traceloop/sdk/annotation/base_annotation.py
BaseAnnotation.create returns delivered, refused, or unreachable results. HTTP errors include status codes, extracted payloads, and exceptions.
User feedback result contract
packages/traceloop-sdk/traceloop/sdk/annotation/user_feedback.py, packages/traceloop-sdk/tests/test_user_feedback.py
UserFeedback.create returns AnnotationCreateResult. Tests validate success, HTTP errors, transport errors, payloads, status codes, and error fields.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant UserFeedback
  participant BaseAnnotation
  participant HTTPClient
  UserFeedback->>BaseAnnotation: create annotation
  BaseAnnotation->>HTTPClient: POST with raise_on_error=True
  HTTPClient-->>BaseAnnotation: response or exception
  BaseAnnotation-->>UserFeedback: AnnotationCreateResult
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: returning a structured outcome from annotation creation.
Linked Issues check ✅ Passed The changes satisfy issue #4404 by distinguishing delivered, refused, and unreachable annotation writes without crashing callers.
Out of Scope Changes check ✅ Passed The HTTP client changes, annotation result types, and tests directly support the linked issue objectives.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/traceloop-sdk/traceloop/sdk/client/http.py`:
- Around line 36-37: Update the response-handling method containing
raise_on_error so that after raise_for_status(), ValueError from response.json()
is caught and the method returns response.text or None instead of re-raising.
Preserve HTTP error propagation when raise_for_error applies, and add a
regression test covering a successful response with an empty or non-JSON body.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 086ab2e4-57b2-44a1-8483-2d172fe17d3f

📥 Commits

Reviewing files that changed from the base of the PR and between 62e24c2 and 1bc37d1.

📒 Files selected for processing (5)
  • packages/traceloop-sdk/tests/test_http_client_post.py
  • packages/traceloop-sdk/tests/test_user_feedback.py
  • packages/traceloop-sdk/traceloop/sdk/annotation/base_annotation.py
  • packages/traceloop-sdk/traceloop/sdk/annotation/user_feedback.py
  • packages/traceloop-sdk/traceloop/sdk/client/http.py

Comment thread packages/traceloop-sdk/traceloop/sdk/client/http.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants