Skip to content

fix(google-generativeai): emit choice events from the last chunk - #4418

Open
chiruu12 wants to merge 1 commit into
traceloop:mainfrom
chiruu12:fix/genai-streaming-events
Open

fix(google-generativeai): emit choice events from the last chunk#4418
chiruu12 wants to merge 1 commit into
traceloop:mainfrom
chiruu12:fix/genai-streaming-events

Conversation

@chiruu12

@chiruu12 chiruu12 commented Aug 13, 2026

Copy link
Copy Markdown

Both streaming builders passed the generator to emit_choice_events after the for item in response loop had already drained it. emit_choice_events reads response.candidates, which a generator does not have, so with events enabled every streamed Gemini call raised AttributeError out of the generator into the caller, and the span was never ended.

The non-events branch three lines below already used last_chunk. This makes the events branch do the same, and skips emitting when the stream produced no chunks at all.

Neither builder is decorated @dont_throw and neither ends the span in a finally, which is why this surfaced in user code rather than a log line. Worth fixing, but out of scope here.

Tests in tests/test_streaming_events.py cover the sync builder, the async builder and the empty stream. All three fail on main.

Fixes #4417

Summary by CodeRabbit

  • Bug Fixes

    • Improved synchronous and asynchronous streaming event handling by using the final response chunk.
    • Prevented choice events from being emitted when streaming responses contain no chunks.
  • Tests

    • Added coverage for final-chunk event handling in synchronous and asynchronous streams.
    • Added validation for empty streaming responses.

Copilot AI lite review requested due to automatic review settings August 13, 2026 03:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Synchronous and asynchronous Google Generative AI streaming builders now emit choice events from the final response chunk and skip emission when a stream is empty.

Changes

Google Generative AI streaming events

Layer / File(s) Summary
Use the final streamed chunk
packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.py
Synchronous and asynchronous streaming paths now use last_chunk for choice events and avoid emission when no chunk exists.
Validate streaming event behavior
packages/opentelemetry-instrumentation-google-generativeai/tests/test_streaming_events.py
Tests cover final-chunk event emission for synchronous and asynchronous streams and verify that empty streams emit no events.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to fb0f8

The change is localized to streamed event emission and avoids the reported failure on empty streams; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

  • traceloop/openllmetry#4372: Both changes derive streaming events or attributes from final streamed data, but they affect different instrumentation modules.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary fix: emitting Google Generative AI choice events from the final stream chunk.
Linked Issues check ✅ Passed The changes fix the drained-generator error, use the final chunk for sync and async events, guard empty streams, and add matching tests [#4417].
Out of Scope Changes check ✅ Passed All changes are limited to Google Generative AI streaming event handling and related tests required by the linked issue.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/opentelemetry-instrumentation-google-generativeai/tests/test_streaming_events.py (1)

93-101: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add an empty asynchronous stream test.

The current test covers only the synchronous empty-stream branch. Add the equivalent asynchronous case to protect _abuild_from_streaming_response.

Proposed async empty-stream regression test
+@pytest.mark.asyncio
+async def test_async_streaming_with_no_chunks_does_not_emit(monkeypatch):
+    seen = _streaming_setup(monkeypatch)
+
+    async def agen():
+        for chunk in []:
+            yield chunk
+
+    generator = _abuild_from_streaming_response(
+        MagicMock(), agen(), "gemini-2.0-flash", MagicMock(), None
+    )
+    async for _ in generator:
+        pass
+
+    assert "response" not in seen
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/opentelemetry-instrumentation-google-generativeai/tests/test_streaming_events.py`
around lines 93 - 101, Add an asynchronous counterpart to
test_sync_streaming_with_no_chunks_does_not_emit that exercises
_abuild_from_streaming_response with an empty async iterator, consumes the
result, and asserts "response" is absent from seen while reusing the existing
_streaming_setup setup.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@packages/opentelemetry-instrumentation-google-generativeai/tests/test_streaming_events.py`:
- Around line 93-101: Add an asynchronous counterpart to
test_sync_streaming_with_no_chunks_does_not_emit that exercises
_abuild_from_streaming_response with an empty async iterator, consumes the
result, and asserts "response" is absent from seen while reusing the existing
_streaming_setup setup.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3776eafc-1614-4d1c-a2bb-390b95142320

📥 Commits

Reviewing files that changed from the base of the PR and between 62e24c2 and fb0f81b.

📒 Files selected for processing (2)
  • packages/opentelemetry-instrumentation-google-generativeai/opentelemetry/instrumentation/google_generativeai/__init__.py
  • packages/opentelemetry-instrumentation-google-generativeai/tests/test_streaming_events.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: streaming choice events use the drained generator instead of the last chunk

2 participants