feat: add livekit transport by monster-anshu · Pull Request #86 · pipecat-ai/pipecat-client-web-transports

monster-anshu · 2026-01-20T17:23:24Z

Adds a new @pipecat-ai/livekit-transport package to the monorepo — a WebRTC transport for @pipecat-ai/client-js built on LiveKit's infrastructure. Unlike the peer-to-peer small-webrtc-transport, this routes media through LiveKit's SFU, enabling multi-participant rooms, edge routing, and production-scale deployments.

What's included

LiveKitTransport (src/liveKitTransport.ts) — extends the base Transport class and implements the full Pipecat
transport lifecycle:
- Connection — flexible auth: fetch { url, token } from an authUrl (GET or POST with body), or pass url/token
  directly to connect(). Constructor accepts LiveKit RoomOptions.
- Device management — enumerate/switch mics, cams, and speakers via navigator.mediaDevices + LiveKit's
  switchActiveDevice; hot-swapping via a devicechange listener; selected-device state synced to callbacks.
- Media — mic/cam enable/disable, screen share (with audio), local + remote track publish/subscribe wired to
  onTrackStarted/onTrackStopped.
- Messaging — RTVI messages over LiveKit data channels (reliable), encoded/decoded via TextEncoder/TextDecoder.
- Participants — join/leave callbacks; first remote participant treated as the bot.
- State — mapped to Pipecat states (disconnected → ready/error), with abort-signal handling on connect.
Package scaffolding — package.json, tsconfig.json, LICENSE (BSD-2-Clause), README.md, CHANGELOG.md, src/index.ts.
Dependencies — livekit-client ^2.21.0; peer dep @pipecat-ai/client-js ^1.13.0.
Docs — root README.md updated to list the new transport; package-lock.json regenerated.

AbaoFromCUG · 2026-05-11T14:25:40Z

Thanks for your PR — I tested your transport and have a few simple pieces of feedback.

1. Support `LiveKitTransport.connect({livekitParams:{token, url}})`

Would it be possible to support passing the token and URL dynamically in connect(), instead of configuring them in the client constructor? This would be especially useful for delayed/lazy connection scenarios. That said, it’s totally fine even without this feature — the transport is already quite great.

2. `LiveKitTransport.sendReadyMessage` 's race condition issue

When (server) LiveKitTransport._on_data_received_wrapper receives the client-ready message sent by (client) LiveKitTransport.sendReadyMessage, there is a fairly high chance that data.participant == None. This looks like a race condition issue.

2026-05-11 22:11:37.958 | ERROR    | pipecat.utils.asyncio.task_manager:run_coroutine:168 - LiveKitTransport#0::LiveKitTransportClient::_async_on_data_received unexpected exception (/home/abao/Documents/work/converge/3rd/pipecat/pipecat/src/pipecat/transports/livekit/transport.py:536): 'NoneType' object has no attribute 'sid'

I can work around it by simply adding a setTimeout:

sendReadyMessage() {
  this.state = 'ready'
  setTimeout(() => {
    // trick delay
    this.sendMessage(RTVIMessage.clientReady())
  }, 1000)
}

It seems that sendReadyMessage() is fired too early, so when the server receives data_received, the remote_participants collection has not been fully initialized yet, causing participant to be None.

Of course, it’s also possible that this is related to my local LiveKit server setup:

docker run --rm \
  -p 7880:7880 \
  -p 7881:7881 \
  -p 7882:7882/udp \
  -e LIVEKIT_KEYS="devkey: secret" \
  livekit/livekit-server \
  --dev

export LIVEKIT_URL=ws://127.0.0.1:7880
export LIVEKIT_API_KEY=devkey
export LIVEKIT_API_SECRET=secret

AbaoFromCUG · 2026-05-11T15:56:22Z

I guess I fond the solution for feedback 2

  async sendReadyMessage() {
    this.state = 'ready'
    await this._room.localParticipant.waitUntilActive()
    this.sendMessage(RTVIMessage.clientReady())
  }

Read waitUntilActive's docs for more information

monster-anshu · 2026-05-11T16:18:04Z

Thanks for the feedback. I will incorporate these changes and update the PR soon.

monster-anshu · 2026-05-24T14:31:57Z

Hey @AbaoFromCUG the PR has been updated.

monster-anshu · 2026-07-24T08:30:13Z

Hey @markbackman @AbaoFromCUG , this PR has been open for a while. Could you please take a look when you get a chance? If there's anything needed from my side, let me know. Once everything looks good, I'd appreciate it if you could merge it.

markbackman · 2026-07-24T13:25:19Z

@monster-anshu reviewing this is on our to do list.

mattieruth

This is fantastic work. In some cases pointing out things i need to update for Daily :). Thanks for this contribution and apologies it took so long to get to. I have a bunch of comments, not too many of which are beyond a nit. I still need to test, though. Do you have plans for adding an example? or have one you've been using? If not, I'll just modify one of our existing ones locally.

mattieruth · 2026-07-24T16:42:16Z

+
 ```

+<<<<<<< HEAD


mattieruth · 2026-07-24T16:42:34Z

 This transport enables a purely WebSocket based connection between clients and your Pipecat application. It implements bidirectional audio and video streaming using a WebSocket for real-time communication.

-It is intended for lightweight implementations, particularly for local development and testing. It expects your Pipecat server to include the corresponding server-side [WebSocketTransport](https://docs.pipecat.ai/api-reference/server/services/transport/websocket-server) implementation.
+# It is intended for lightweight implementations, particularly for local development and testing. It expects your Pipecat server to include the corresponding server-side [WebSocketTransport](https://docs.pipecat.ai/api-reference/server/services/transport/websocket-server) implementation.


this should not be a # heading

mattieruth · 2026-07-24T16:42:51Z

+
+```
+
+> > > > > > > e6ded27 (feat: add livekit transport)


mattieruth · 2026-07-24T16:48:37Z

+[![README](https://img.shields.io/badge/README-goldenrod)](/transports/livekit-transport/README.md)
+![NPM Version](https://img.shields.io/npm/v/@pipecat-ai/livekit-transport)
+
+This Transport uses the [LiveKit](https://livekit.io) real-time communication platform to connect to a bot and stream media over a WebRTC connection. LiveKit provides a scalable, distributed WebRTC infrastructure with features like edge routing, session observability, and recording capabilities.


The second sentence here reads too much like marketing. The Daily Transport provides the exact same set of features but does not include that in the description. We try to be as vendor neutral as possible in the Pipecat ecosystem, so this sentence should either be removed or updated to be more informative of what it enables and that same verbage should be used for Daily as well. These two transports will essentially be equivalent as to what they provide. It just comes down to client preference on vendor.

mattieruth · 2026-07-24T16:53:40Z

@@ -0,0 +1,41 @@
+{
+  "name": "@pipecat-ai/livekit-transport",
+  "version": "1.8.1",


let's start with 1.0.0. The transport versions are not synchronized, so it's a bit odd to start at 1.8.1.

mattieruth · 2026-07-24T19:58:34Z

+
+  async sendReadyMessage() {
+    this.state = "ready";
+    await this._room.localParticipant.waitUntilActive();


i could use some LiveKit guidance here. In general, we want to wait to send the ready message until we know that our webRTC track connections are ready to send/receive audio. The point being that we don't want the bot to start talking to us until we know we will hear the audio. If we send clientReady() too early, then the start of the bot's audio will be clipped. Will waitUntilActive() give us this assurance? The docs say that it's ready to receive data messages, but does that include audio?

mattieruth · 2026-07-24T20:01:10Z

+      const decoder = new TextDecoder();
+      const str = decoder.decode(payload);
+      const msg = JSON.parse(str);
+      if (msg && typeof msg === "object" && "type" in msg) {


add a check for label === 'rtvi-ai'

mattieruth · 2026-07-24T20:02:36Z

+        this.toParticipant(participant)
+      );
+    }
+    if (publication.source === Track.Source.Microphone) {


again, why is the microphone handled differently than cam/screen here?

mattieruth · 2026-07-24T20:04:05Z

+    participant: LocalParticipant
+  ) {
+    if (publication.track?.mediaStreamTrack) {
+      this._callbacks.onTrackStopped?.(


here and in handleLocalTrackPublished we should call onScreenTrackStopped/Started if the track is a screenshare track. (I assume these hit this handler as well)

mattieruth · 2026-07-24T20:11:03Z

+
+  private handleMediaDevicesError(e: Error) {
+    this._callbacks.onDeviceError?.(
+      new DeviceError(["cam", "mic"], "unknown", e.message)


it looks like there is support in LiveKit for determining which device failed and how: https://docs.livekit.io/reference/client-sdk-js/types/ParticipantEventCallbacks.html#mediadeviceserror

we should take advantage of this to be consistent with other transports.

monster-anshu · 2026-07-25T12:48:47Z

I guess I fond the solution for feedback 2
  async sendReadyMessage() {
    this.state = 'ready'
    await this._room.localParticipant.waitUntilActive()
    this.sendMessage(RTVIMessage.clientReady())
  }
Read waitUntilActive's docs for more information

waitUntilActive() resolves on ParticipantInfo_State.ACTIVE (server-side participant state). The SDK documents it as "ready to receive data messages." Audio from the bot only starts after it receives clientReady, so there's no clipping risk — bot audio is a consequence of this message, not concurrent with it.

mattieruth

Thanks for the quick turn-around! This is looking great. I think the only major thing is to figure out the initDevices() logic. I also need to get myself a LiveKit account and test it :)

mattieruth · 2026-07-27T19:56:39Z

-    let { url, token } = params;
-
-    this.state = "connecting";
-    if (params.authUrl) {


I'm guessing you removed this version of connect to encourage the API request to live server-side as part of startBot?

I want users to generate the LiveKit token using their own setup, either through the LiveKit SDK or any other token generation API, and pass the generated token in the connect parameters. This keeps token generation outside the transport layer, allowing the transport to remain responsible only for establishing the connection.

great! and i agree. this keeps the API much cleaner and consistent with other transports.

initDevices() previously stopped tracks after enumeration, breaking lobby/pre-join UX. Now uses createLocalAudioTrack/createLocalVideoTrack so tracks persist and are published via publishTrack() at connect time. Also restores abort signal check post-connect to prevent state landing in "connected" after an aborted connection.

Abort check sat before the track-publish awaits, so a disconnect() landing during publishing would resume _connect() past the guard and set state to "connected" after _disconnect() already set it to "disconnected". Move the guard after the publishes, immediately before the state assignment, so it covers every await in the connect path. Also disconnect the room in the abort branch — room.connect() has already succeeded by then, so a bare return leaks a live room.

Part of the transport contract is to fire onBotConnected/onBotDisconnected whenever the bot join/leaves the connection. As we don't currently support multiple remote bots out of the box, the current standard approach is to simply treat the first remote participant as the bot. Track the bot's participant identity (_botId) the same way Daily tracs its _botId, reuse it in tracks() instead of blindly indexing remoteParticipants[0], and clear it on room disconnect/leave so a stale id doesn't linger across reconnects.

monster-anshu · 2026-07-29T22:08:34Z

Hey @mattieruth , the test pipeline is consistently failing for this PR. Based on the error logs, this doesn't appear to be an issue with the test cases themselves. It looks like a GitHub Actions/environment issue where the "@rolldown/binding-linux-x64-gnu" native dependency isn't being installed, causing Vitest to fail before the tests even start.

mattieruth · 2026-07-30T14:17:52Z

Hey @mattieruth , the test pipeline is consistently failing for this PR. Based on the error logs, this doesn't appear to be an issue with the test cases themselves. It looks like a GitHub Actions/environment issue where the "@rolldown/binding-linux-x64-gnu" native dependency isn't being installed, causing Vitest to fail before the tests even start.

i'll look into. i'm also working on a batch of changes to help get it to the finish line and align with the other transports.

…it-client's native APIs Replaces raw getUserMedia device handling with LiveKit's own track lifecycle (mute/unmute, restartTrack, setMicrophoneEnabled/ setCameraEnabled) for smoother device switching, sticky "default"-device reselection across system default changes, and no track leaks across connect/disconnect. Also fixes bot-connected/disconnected parity, device- error reporting, screen-share vs. generic track callback routing, speaker selection on init, and gates readiness on the bot's media actually being subscribed rather than just being connected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…acks are fired

Fixes duplicate behavior and ensures cleanup happens even on unexpected disconnects

mattieruth · 2026-08-11T19:15:01Z

I have created this PR for your branch that gets us 98% of the way there (i just need to test video, but everything else is working). let me know what you think. this brings parity to the livekit transport, connecting/disconnecting correctly and fixing all the various device state to work both before and after connection to a livekit room. This work also revealed gaps in the server-side livekit transport component. I have a corresponding PR to fix these here: pipecat-ai/pipecat#5297

mattieruth · 2026-08-11T19:40:30Z

Here's a list of all corresponding PRs for getting LiveKit completely wired up in the pipecat ecosystem:

New Livekit client-side Transport PRs:

This PR +
86 livekit fixes monster-anshu/pipecat-client-web-transports#7

Corresponding Server fixes for LiveKit Transport:

Add LiveKit to runner pipecat#5297

Voice-Ui Kit support:

Add livekit transport support voice-ui-kit#159

Pipecat Prebuilt:

Add livekit transport support pipecat-prebuilt#52

monster-anshu · 2026-08-12T05:05:05Z

I have created this PR for your branch that gets us 98% of the way there (i just need to test video, but everything else is working). let me know what you think. this brings parity to the livekit transport, connecting/disconnecting correctly and fixing all the various device state to work both before and after connection to a livekit room. This work also revealed gaps in the server-side livekit transport component. I have a corresponding PR to fix these here: pipecat-ai/pipecat#5297

Sure seem right to me I will also test the video channel as well and will attach the results.

mattieruth · 2026-08-12T15:07:35Z

I have created this PR for your branch that gets us 98% of the way there (i just need to test video, but everything else is working). let me know what you think. this brings parity to the livekit transport, connecting/disconnecting correctly and fixing all the various device state to work both before and after connection to a livekit room. This work also revealed gaps in the server-side livekit transport component. I have a corresponding PR to fix these here: pipecat-ai/pipecat#5297

Sure seem right to me I will also test the video channel as well and will attach the results.

great! would you mind going ahead and merging those changes to your branch so they are pulled in here?

86 livekit fixes

monster-anshu · 2026-08-12T15:25:16Z

! would you mind going ahead and merging those changes to your branch so they are

I have merged the monster-anshu#7 in my branch

monster-anshu force-pushed the add-livekit branch 2 times, most recently from 5eca560 to 38c49ee Compare April 25, 2026 16:36

monster-anshu force-pushed the add-livekit branch from 0ffcf00 to 31de936 Compare May 12, 2026 20:05

monster-anshu force-pushed the add-livekit branch from e276dfb to 32a55f2 Compare May 24, 2026 14:28

jamsea mentioned this pull request Jun 26, 2026

docs: close gaps found in Kapa source analytics audit pipecat-ai/docs#951

Merged

monster-anshu force-pushed the add-livekit branch 2 times, most recently from 1acfc17 to d562ed5 Compare July 24, 2026 08:12

feat: add livekit transport

9cd9cdb

monster-anshu force-pushed the add-livekit branch from b759856 to 9cd9cdb Compare July 24, 2026 08:34

mattieruth reviewed Jul 24, 2026

View reviewed changes

review changes

a544159

mattieruth reviewed Jul 27, 2026

View reviewed changes

monster-anshu and others added 5 commits July 28, 2026 20:58

fix(livekit-transport): only dispatch rtvi-ai data messages

859e205

update test cases

bdc2558

mattieruth and others added 3 commits August 11, 2026 10:30

Fix mic/cam enabled behavior so that state matches reality when callb…

3131091

…acks are fired

Fix disconnection logic so that cleanup happens in handler

20b66e5

Fixes duplicate behavior and ensures cleanup happens even on unexpected disconnects

mattieruth mentioned this pull request Aug 11, 2026

Add LiveKit to runner pipecat-ai/pipecat#5297

Open

This was referenced Aug 11, 2026

Add livekit transport support pipecat-ai/pipecat-prebuilt#52

Draft

Add livekit transport support pipecat-ai/voice-ui-kit#159

Open

Merge pull request #7 from mattieruth/86-livekit-fixes

4c9c4f0

86 livekit fixes

Uh oh!

Conversation

monster-anshu commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AbaoFromCUG commented May 11, 2026

1. Support LiveKitTransport.connect({livekitParams:{token, url}})

2. LiveKitTransport.sendReadyMessage 's race condition issue

Uh oh!

AbaoFromCUG commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

monster-anshu commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

monster-anshu commented May 24, 2026

Uh oh!

monster-anshu commented Jul 24, 2026

Uh oh!

markbackman commented Jul 24, 2026

Uh oh!

mattieruth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

monster-anshu commented Jul 25, 2026

Uh oh!

mattieruth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

monster-anshu commented Jul 29, 2026

Uh oh!

mattieruth commented Jul 30, 2026

Uh oh!

mattieruth commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattieruth commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

monster-anshu commented Aug 12, 2026

Uh oh!

mattieruth commented Aug 12, 2026

Uh oh!

monster-anshu commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

monster-anshu commented Jan 20, 2026 •

edited

Loading

1. Support `LiveKitTransport.connect({livekitParams:{token, url}})`

2. `LiveKitTransport.sendReadyMessage` 's race condition issue

AbaoFromCUG commented May 11, 2026 •

edited

Loading

monster-anshu commented May 11, 2026 •

edited

Loading

mattieruth commented Aug 11, 2026 •

edited

Loading

mattieruth commented Aug 11, 2026 •

edited

Loading