Skip to content

fix: keep session audio flowing after STT/VAD close and handoff - #7005

Open
samvallad33 wants to merge 3 commits into
livekit:mainfrom
samvallad33:fix/agents-6919-audio-routing-freeze
Open

fix: keep session audio flowing after STT/VAD close and handoff#7005
samvallad33 wants to merge 3 commits into
livekit:mainfrom
samvallad33:fix/agents-6919-audio-routing-freeze

Conversation

@samvallad33

@samvallad33 samvallad33 commented Aug 27, 2026

Copy link
Copy Markdown

Mid-call, user audio can stop reaching STT/VAD while RoomIO is still subscribed. user_state sticks on listening. _push_audio fans out with send_nowait; ChanClosed on one STT or VAD channel killed _forward_audio_task and stopped pulling RoomIO. A reused _STTPipeline whose pump had already exited stayed attached after activity handoff, so later turns never reached STT.

Isolate ChanClosed per fan-out target so one dead consumer cannot kill the session pump. Keep _forward_audio_task running if push_audio raises. Rebind the STT node at activity detach; replace a reused pipeline when its pump is already done; recreate the STT stream if it ends while audio_ch is still open.

RoomIO publication-SID identity is #6744 (approved). This PR does not change _input.py.

Related to #6919. Not marking Fixes: the live outbound-SIP freeze was not reproduced end-to-end.

uv run pytest tests/test_audio_recognition_push_audio.py tests/test_audio_recognition_handoff.py tests/test_agent_session.py::test_stt_pipeline_recreates_after_clean_stream_end tests/test_agent_session.py::test_stt_pipeline_recreates_stream_after_unrecoverable_error tests/test_agent_session.py::test_stt_pipeline_does_not_recreate_on_non_connection_error tests/test_agent_session.py::test_stt_pipeline_does_not_recreate_stream_while_closing tests/test_agent_session.py::test_stt_pipeline_recreation_uses_rebound_node --unit

Video forwarding is left alone: _forward_video_task has the same shape, but this PR is scoped to the audio path the issue describes.

RoomIO keyed the mic stream on publication SID, so a replacement track under the same publication left the session attached to a dead AudioStream. A closed STT/VAD channel also raised out of _push_audio and killed the session audio pump, and a reused STT pipeline with a finished pump stayed attached after activity handoff.
@samvallad33
samvallad33 force-pushed the fix/agents-6919-audio-routing-freeze branch from 523bf4c to 67d051f Compare August 27, 2026 06:38
Track identity on publication SID is already handled there. Keep the STT/VAD pump and handoff recovery for the session freeze.
@samvallad33
samvallad33 force-pushed the fix/agents-6919-audio-routing-freeze branch from 67d051f to 2334781 Compare August 27, 2026 06:38
@samvallad33 samvallad33 changed the title fix: keep user audio flowing after mic track swap and STT handoff fix: keep session audio flowing after STT/VAD close and handoff Aug 27, 2026
Frames arrive every 10ms, so a persistent push failure logged a traceback
per frame. Log once per run of the same error type instead.
@samvallad33
samvallad33 marked this pull request as ready for review August 28, 2026 01:24
@samvallad33
samvallad33 requested a review from a team as a code owner August 28, 2026 01:24

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant