Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 38 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[![Docs](https://img.shields.io/badge/Documentation-blue)](https://docs.pipecat.ai/client/js/transports/transport)
[![Discord](https://img.shields.io/discord/1239284677165056021)](https://discord.gg/pipecat)

A mono-repo to house the various supported Transport options to be used with the pipecat-client-web library. Currently, there are four transports: `small-webrtc-transport`, `daily-transport`, `gemini-live-websocket-transport`, and `openai-realtime-webrtc-transport`.
A mono-repo to house the various supported Transport options to be used with the pipecat-client-web library. Currently, there are six transports: `small-webrtc-transport`, `daily-transport`, `websocket-transport`, `gemini-live-websocket-transport`, `openai-realtime-webrtc-transport`, and `moq-transport`.

## Documentation

Expand All @@ -21,8 +21,8 @@ Pipecat Transports are intended to be used in conjunction with a Pipecat web cli
This Transport creates a peer-to-peer WebRTC connection between the client and the bot process. This Transport is the client-side counterpart to the Pipecat [SmallWebRTCTransport component](https://docs.pipecat.ai/server/services/transport/small-webrtc).

This is the simplest low-latency audio/video transport for Pipecat. This transport is recommended for local development and demos. Things to be aware of:
- This transport is a direct connection between the client and the bot process. If you need multiple clients to connect to the same bot, you will need to use a different transport.
- For production usage at scale, a distributed WebRTC network that can do edge/mesh routing, has session-level observability and metrics, and can offload recording and other auxiliary services is often useful.
- This transport is a direct connection between the client and the bot process. If you need multiple clients to connect to the same bot, you will need to use a different transport.
- For production usage at scale, a distributed WebRTC network that can do edge/mesh routing, has session-level observability and metrics, and can offload recording and other auxiliary services is often useful.

Typical media flow using a SmallWebRTCTransport:
```
Expand Down Expand Up @@ -138,6 +138,41 @@ Media flow using a OpenAIRealTimeWebRTCTransport:
└─────────────────────────────────────┘
```

### [MoqTransport](/transports/moq-transport/README.md)

[![Docs](https://img.shields.io/badge/documentation-blue)](https://docs.pipecat.ai/client/js/transports)
[![README](https://img.shields.io/badge/README-goldenrod)](/transports/moq-transport/README.md)
[![Demo](https://img.shields.io/badge/Demo-forestgreen)](https://github.com/pipecat-ai/voice-ui-kit/tree/main/examples/01-console)
[![NPM Version](https://img.shields.io/npm/v/@pipecat-ai/moq-transport)](https://www.npmjs.com/package/@pipecat-ai/moq-transport)

This Transport uses [Media over QUIC (MoQ)](https://quic.video/) to connect to a bot, either through a MoQ relay or directly to a bot running in serve mode. This Transport is the client-side counterpart to the Pipecat MoQ transport component (`pipecat.transports.moq.transport`).

Typical media flow using a MoqTransport:
```

┌────────────────────────────────────────────┐
│ │
┌───────────────────┐ │ Server ┌─────────┐ │
│ │ │ │Pipecat │ │
│ Client │ RTVI Messages │ │Pipeline │ │
│ │ & │ │ │ │
│ ┌──────────────┐ │ WebTransport │ ┌──────────────┐ media │ ┌─────┐ │ │
│ │ MoqTransport │◄─┼────────────────┼─►│ MoqTransport ┼────────────┼─► STT │ │ │
│ └──────────────┘ │ │ └───────▲──────┘ in │ └──┬──┘ │ │
│ │ │ │ │ │ │ │
└───────────────────┘ │ │ │ ┌──▼──┐ │ │
│ │ │ │ LLM │ │ │
│ │ │ └──┬──┘ │ │
│ │ │ │ │ │
│ │ │ ┌──▼──┐ │ │
│ │ media │ │ TTS │ │ │
│ └───────────────────┼─┴─────┘ │ │
│ out └─────────┘ │
│ │
└────────────────────────────────────────────┘

```

## Local Development

### Build the transport libraries
Expand Down
18 changes: 18 additions & 0 deletions transports/moq-transport/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

[![Docs](https://img.shields.io/badge/documentation-blue)](https://docs.pipecat.ai/client/js/transports)
![NPM Version](https://img.shields.io/npm/v/@pipecat-ai/moq-transport)
[![Demo](https://img.shields.io/badge/Demo-forestgreen)](https://github.com/pipecat-ai/voice-ui-kit/tree/main/examples/01-console)

Media-over-QUIC transport package for use with `@pipecat-ai/client-js`.

Expand Down Expand Up @@ -78,6 +79,23 @@ interface MoqTransportOptions {

Broadcast paths are derived as `<namespace>/<clientId>` (publish) and `<namespace>/<botId>` (subscribe). Audio track names inside each broadcast are discovered from the bot's catalog, so they aren't configured directly.

### Connecting via a bot `/start` endpoint

If your `/start` endpoint returns the bot's MoQ config nested under a `moq` key (the shape the bot's `pipecat.transports.moq.transport` returns), pass that response straight to `connect()` / `PipecatClient.startBotAndConnect()` — `MoqTransport` unwraps it (including base64-decoding `certHash` into `serverCertificateHashes`) in `_validateConnectionParams`, no app-side transform needed:

```json
{
"moq": {
"relayUrl": "https://relay.example.com:4080/moq",
"certHash": "base64-encoded-sha-256-or-null",
"namespace": "pipecat",
"clientId": "client0",
"botId": "bot0",
"transcriptTrack": "transcript.json.z"
}
}
```

### Handling Events

The transport implements the various [Pipecat event handlers](https://docs.pipecat.ai/client/js/api-reference/callbacks). Check out the docs or samples for more info.
Expand Down
Loading