fix: reattach audio input after stream end - #6742
Conversation
- Recreate RTC audio streams when tracks remain subscribed after EOS - Prevent reattachment after unsubscribe or input close
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| while True: | ||
| await super()._forward_task(None, stream, publication, participant) | ||
|
|
||
| # push a silent frame to flush the stt final result if any | ||
| await self._data_ch.send( | ||
| rtc.AudioFrame( | ||
| b"\x00\x00" * silent_samples, | ||
| sample_rate=self._sample_rate, | ||
| num_channels=self._num_channels, | ||
| samples_per_channel=silent_samples, | ||
| ) | ||
| ) | ||
| ) | ||
|
|
||
| if self._stream is not stream or self._publication is not publication: | ||
| return | ||
|
|
||
| # An RTC stream may reach EOS while its track remains subscribed. | ||
| track = publication.track | ||
| self._close_stream() | ||
| if track is None or not publication.subscribed: | ||
| return | ||
|
|
||
| stream = self._create_stream(track, participant) | ||
| self._stream = stream | ||
| self._publication = publication |
There was a problem hiding this comment.
🔴 Audio input can loop forever recreating a dead microphone feed and flooding the pipeline with silence
The microphone feed is rebuilt in an unbounded loop with no delay or retry limit (while True at livekit-agents/livekit/agents/voice/room_io/_input.py:336-360) whenever the feed ends while the track is still marked subscribed, so a permanently dead feed makes the agent spin and continuously inject half-second silence chunks.
Impact: If a participant's microphone feed dies but stays subscribed, the agent burns CPU in a rapid retry loop and keeps pushing silence into speech recognition, which can degrade or stall the session.
Mechanism: EOS-triggered recreate has no guard against immediately-ending streams
After super()._forward_task(...) returns (end of stream), the code sends a 0.5s silent frame (_input.py:340-347), then—if the track object still exists and publication.subscribed is true—calls _create_stream(track, participant) and loops back to read the new stream (_input.py:352-360).
If the underlying track is dead (e.g. the media ended but the publication has not yet been unsubscribed/unpublished), rtc.AudioStream.from_track will yield EOS again immediately, and the loop repeats without any backoff, iteration cap, or check that any frame was actually received. Each iteration:
- schedules another
aclose()task via_close_stream()(_input.py:181-188), - for selector-based noise cancellation, closes and re-creates a processor,
- pushes another 0.5s silent frame into
self._data_ch, which is an unboundedaio.Chan(livekit-agents/livekit/agents/utils/aio/channel.py:49-86), so the send never blocks and provides no natural throttle.
A bounded retry (e.g. only reattach if the previous stream delivered frames, plus a small sleep/backoff) would prevent the spin.
Prompt for agents
In _ParticipantAudioInputStream._forward_task (livekit-agents/livekit/agents/voice/room_io/_input.py), the new reattach loop recreates the rtc AudioStream every time the previous one reaches end-of-stream while publication.track is set and publication.subscribed is True. There is no backoff, retry cap, or check that the previous stream actually produced any frames, so a track whose media is permanently dead (but whose publication is still subscribed) causes a fast infinite loop that repeatedly closes/creates streams, re-runs the noise-cancellation selector, and pushes an unbounded number of 0.5s silent frames into the unbounded self._data_ch. Consider adding a short async sleep between reattach attempts and/or only reattaching when the previous stream delivered at least one frame, and bounding the number of consecutive zero-frame reattaches before giving up (logging a warning).
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Superseded by #6744, which uses the required AGT-linked branch name. |
1 participant
Summary
Testing