-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Blaze Livekit plugin integration #5050
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
HoangPN711
wants to merge
85
commits into
livekit:main
Choose a base branch
from
Actable-AI:feat/blaze-voicebot-plugin
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
ab5737f
add blaze voicebot plugin
fpt-hoangpn2 bd4c78e
fix comment review
fpt-hoangpn2 9523e04
blaze: align python LLM route and query params with voicebot-call API
fpt-hoangpn2 aa686f3
blaze: register plugin and fix system role check in LLM conversion
fpt-hoangpn2 07a15db
blaze(py): fix python compatibility, timeout precedence, and tool-mod…
fpt-hoangpn2 bbb6853
fix(blaze-python): align retry/error handling with framework and reso…
fpt-hoangpn2 4bd24f9
fix(blaze-python): handle developer role in LLM context and correct T…
fpt-hoangpn2 f159f0b
fix(blaze-python): ignore null SSE content chunks in llm stream extra…
fpt-hoangpn2 6cccc9d
fix(blaze-python): flush output emitter after streaming tts chunks
fpt-hoangpn2 712a29d
fix(blaze-python): use BLAZE_API_TOKEN via api_token config field
fpt-hoangpn2 bd5bfaf
fix(blaze-python): add postponed annotations in _utils for Python 3.9…
fpt-hoangpn2 21eccba
blaze fix: initialize emitter before empty-text early return in tts s…
fpt-hoangpn2 cf10a96
blaze-fix: wrap unexpected httpx failures as APIConnectionError for r…
fpt-hoangpn2 c0f5d22
blaze voicebot add frame accumulation for short/hesitant speech segme…
fpt-hoangpn2 d373c49
blaze: fix _build_tools_param dead code and json body inconsistency
fpt-hoangpn2 f15f539
blaze: fix CI for TTS websocket
fpt-hoangpn2 b8569ee
blaze: fix tool role dropped in LLM and missing _mark_started in TTS
fpt-hoangpn2 950a001
blaze: use APIError subclasses in TTS for framework retry support
fpt-hoangpn2 cc879f0
fix: replace asyncio.timeout with Python 3.10-compatible timeout wrapper
fpt-hoangpn2 b7b2c7c
blaze fix: properly cancel and await reader_task to prevent task leak
fpt-hoangpn2 a9d4ed4
style(blaze-tts): format tts.py to satisfy Ruff CI
fpt-hoangpn2 c6b5818
fix(blaze-tts): avoid non-progress loop when hard-limit split hits ti…
fpt-hoangpn2 32eba39
blaze: bump version to 1.4.4, register in livekit-agents pyproject, a…
fpt-hoangpn2 6ac320f
blaze: align version to 1.5.8 and fix post-merge issues
fpt-hoangpn2 21bfe8e
blaze: align version to 1.5.8 and fix post-merge issues
fpt-hoangpn2 d3cd729
Update version bump to 1.5.9
HoangPN711 d16f649
blaze: bump version to 1.5.10 to align with main
fpt-hoangpn2 2f9d9a0
blaze: bump version to 1.5.11
fpt-hoangpn2 713bde6
bump version 1.5.18
HoangPN711 d26be02
bump version 1.5.19.rc1
fpt-hoangpn2 32867e1
bump version 1.6.0
fpt-hoangpn2 e666987
blaze: fix stt timeout
fpt-hoangpn2 09a6b7d
blaze: fix CI and reset between segment
fpt-hoangpn2 95b9251
blaze: fix timout for llm and tts
fpt-hoangpn2 37274cd
bump version 1.6.3
HoangPN711 6912898
fix(blaze-tts): idle timeout, WS reconnect, and stream guard for sing…
HoangPN711 4a71b59
fix(blaze-tts): resolve ruff and mypy recv return type for CI
HoangPN711 32d4920
fix(blaze): cap TTS WS fast reconnects and expose LLM bot_id as model
HoangPN711 2ce1997
test(blaze): add comprehensive unit tests for STT, TTS, LLM, config, …
HoangPN711 90b14d0
bump version 1.6.4
HoangPN711 e0174f2
fix(blaze): sync TTS segment count and honor plugin timeouts
HoangPN711 b16862f
fix: resolve rebase conflicts, ruff I001, and TTS flush feedback
HoangPN711 49e50e7
feat(blaze): default to latest TTS/STT models
HoangPN711 5f390a6
style(blaze): ruff format stt.py
HoangPN711 8d48dcb
fix(blaze): initialize TTS AudioEmitter before empty-text return
HoangPN711 21a4a7b
fix(blaze): use v2.0 for batch STT /v1/stt/transcribe
HoangPN711 6d81c84
fix(blaze): emit START_OF_SPEECH in native STT stream
HoangPN711 65c157c
fix(blaze): tolerate odd-length PCM WS frames in TTS fade
HoangPN711 5d2ee19
fix(blaze): coerce null STT confidence to 1.0
HoangPN711 0c875fc
fix(blaze): use WebSocket for one-shot TTS synthesize()
HoangPN711 e8d3ff3
fix(blaze): preserve 0.0 STT confidence; only default null
HoangPN711 1425374
fix(blaze): honor explicit connect timeout equal to framework default
HoangPN711 34cbe97
fix(blaze): report STT ready-wait timeout as APITimeoutError
HoangPN711 32b41d2
fix(blaze): address Tina review (URL, authors, TTS tokenizer, tests)
trunghlt 0b58fe8
fix(blaze): use api_token in BlazeConfig docstring example
trunghlt fc628e4
docs(blaze): fix README LLM endpoint path
trunghlt 65a0085
fix(blaze): POST empty messages for agent-first LLM turns
trunghlt 3a4d14c
fix(blaze): send LLM chat body as JSON array of messages
trunghlt 5c4708c
fix(blaze): recover STT/TTS after WS drops; force remote wss
trunghlt c664c4a
merge: resolve conflicts with origin/main (1.6.8)
trunghlt b344111
fix(blaze): isolate STT pending PCM; safer TTS WS reconnect
trunghlt 7d06165
style(blaze): drop unused contextvars import in test
trunghlt 64ab579
Merge origin/main into feat/blaze-voicebot-plugin
trunghlt bbca42f
fix(blaze): make TTS private reconnect resend reachable
trunghlt 2fd340f
style(blaze): ruff format TTS reconnect gate
trunghlt fd5540c
fix(blaze): TTS retry on pre-audio drop; STT pending per instance
trunghlt d59ab2b
fix(blaze): fail TTS mid-turn on reader errors; wss for .local
trunghlt 473f494
fix(blaze): satisfy mypy on TTS asyncio.wait race
trunghlt 2d4cfc2
fix(blaze): weak STT pending, safe timeout env, redact user logs
trunghlt 58f1d02
fix(blaze): ignore non-object SSE payloads in LLM stream
trunghlt 951c4bc
fix(blaze): honor plugin timeouts for framework option copies
trunghlt acc3cf6
fix(blaze): drop STT pending PCM on sample-rate/channel mismatch
trunghlt a75002b
style(blaze): ruff-format STT pending format-mismatch branch
trunghlt 32a1f89
fix(blaze): close STT realtime WS after end_input
trunghlt aaf5c79
fix(blaze): return empty SpeechData for empty STT audio buffer
trunghlt 0bffc1a
fix(blaze): drop blind TTS speech-start ack recv
trunghlt 6ae1f7d
fix(blaze): close open STT turns; strip trailing API URL slash
trunghlt eb3f6b1
merge: resolve conflicts with origin/main (1.6.10)
trunghlt be86781
fix(blaze): consume TTS audio reader exceptions after cancel
trunghlt cea7f42
fix(blaze): treat blank API URL as unset; non-retryable missing token
trunghlt a284fc0
fix(blaze): refresh uv.lock; harden API URL and STT error redaction
trunghlt c69144a
fix(blaze): ruff-format _utils.py; ensure uv.lock lists blaze
trunghlt 0255a7e
fix(blaze): fail fast when streaming TTS lacks auth token
trunghlt 1d941c1
fix(blaze): redact secrets in HTTP error bodies for LLM/STT
trunghlt d7f8622
fix(blaze): surface early TTS speech-end; redact WS error frames
trunghlt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # livekit-plugins-blaze | ||
|
|
||
| Agent Framework plugin for [Blaze AI](https://blaze.vn) services: | ||
|
|
||
| - **STT** (Speech-to-Text) via `POST /v1/stt/transcribe` (batch, default model `v2.0`) and WS `/v1/stt/realtime` (streaming, default model `stt-stream-1.5`) | ||
| - **TTS** (Text-to-Speech) via `WS /v1/tts/realtime` (default model `2.0-realtime`; one-shot `synthesize()` and streaming `stream()` both use WebSocket — there is no HTTP POST for realtime TTS) | ||
| - **LLM** (Conversational AI) via `POST /v1/voicebot-call/{bot_id}/chat-conversion-stream` (SSE) | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| pip install livekit-plugins-blaze | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ```python | ||
| from livekit.plugins import blaze | ||
|
|
||
| stt = blaze.STT(language="vi") | ||
| tts = blaze.TTS(speaker_id="speaker-1") | ||
| llm = blaze.LLM(bot_id="my-chatbot") | ||
| ``` | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| | Variable | Description | Default | | ||
| |---|---|---| | ||
| | `BLAZE_API_URL` | Base URL for Blaze API gateway | `https://api.blaze.vn` | | ||
| | `BLAZE_API_TOKEN` | Bearer token for authentication | | | ||
| | `BLAZE_TTS_TIMEOUT` | TTS idle timeout per WebSocket recv (seconds) | `60` | | ||
| | `BLAZE_TTS_STREAM_TIMEOUT` | Max duration for one streaming TTS turn (seconds) | `300` | |
68 changes: 68 additions & 0 deletions
68
livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| """ | ||
| livekit-plugins-blaze | ||
|
|
||
| LiveKit Agent Framework plugin for Blaze AI services (STT, TTS, LLM). | ||
|
|
||
| This package provides LiveKit agent plugins that interface with Blaze's | ||
| speech-to-text, text-to-speech, and conversational AI services. | ||
|
|
||
| Example: | ||
| >>> from livekit.plugins import blaze | ||
| >>> | ||
| >>> # Create plugins with environment variables (BLAZE_*) | ||
| >>> stt = blaze.STT(language="vi") | ||
| >>> tts = blaze.TTS(speaker_id="speaker-1") | ||
| >>> llm = blaze.LLM(bot_id="my-chatbot") | ||
| >>> | ||
| >>> # Or use shared configuration | ||
| >>> config = blaze.BlazeConfig( | ||
| ... api_url="https://api.example.com", | ||
| ... api_token="my-token", | ||
| ... ) | ||
| >>> stt = blaze.STT(config=config) | ||
| >>> tts = blaze.TTS(config=config, speaker_id="custom-voice") | ||
| >>> llm = blaze.LLM(config=config, bot_id="enterprise-bot") | ||
|
|
||
| Environment Variables: | ||
| BLAZE_API_URL: Base URL for Blaze API gateway | ||
| BLAZE_API_TOKEN: Bearer token for API authentication | ||
| """ | ||
|
|
||
| from livekit.agents import Plugin | ||
|
|
||
| from ._config import BlazeConfig | ||
| from .llm import LLM, LLMStream | ||
| from .log import logger | ||
| from .stt import STT | ||
| from .tts import TTS, ChunkedStream | ||
| from .version import __version__ | ||
|
|
||
| __all__ = [ | ||
| # Version | ||
| "__version__", | ||
| # Configuration | ||
| "BlazeConfig", | ||
| # Plugins | ||
| "STT", | ||
| "TTS", | ||
| "ChunkedStream", | ||
| "LLM", | ||
| "LLMStream", | ||
| ] | ||
|
devin-ai-integration[bot] marked this conversation as resolved.
|
||
|
|
||
|
|
||
| class BlazePlugin(Plugin): | ||
| def __init__(self) -> None: | ||
| super().__init__(__name__, __version__, __package__, logger) | ||
|
|
||
|
|
||
| Plugin.register_plugin(BlazePlugin()) | ||
|
|
||
| # Cleanup docs of unexported modules | ||
| _module = dir() | ||
| NOT_IN_ALL = [m for m in _module if m not in __all__] | ||
|
|
||
| __pdoc__ = {} | ||
|
|
||
| for n in NOT_IN_ALL: | ||
| __pdoc__[n] = False | ||
113 changes: 113 additions & 0 deletions
113
livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/_config.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| """ | ||
| Blaze Configuration Module | ||
|
|
||
| Provides centralized configuration for Blaze services. | ||
| Configuration can be provided via environment variables with BLAZE_ prefix | ||
| or passed explicitly to the constructor. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import os | ||
|
|
||
| from ._utils import validate_api_base_url | ||
| from .log import logger | ||
|
|
||
| _DEFAULT_API_URL = "https://api.blaze.vn" | ||
| _DEFAULT_STT_TIMEOUT = 30.0 | ||
| _DEFAULT_TTS_TIMEOUT = 60.0 | ||
| _DEFAULT_TTS_STREAM_TIMEOUT = 300.0 | ||
| _DEFAULT_LLM_TIMEOUT = 60.0 | ||
|
|
||
|
|
||
| def _env_float(name: str, default: float) -> float: | ||
| """Parse a float env var; empty/invalid values fall back to ``default``.""" | ||
| raw = os.environ.get(name) | ||
| if raw is None: | ||
| return default | ||
| stripped = raw.strip() | ||
| if not stripped: | ||
| return default | ||
| try: | ||
| return float(stripped) | ||
| except ValueError: | ||
| logger.warning( | ||
| "Invalid %s=%r; using default %.1f", | ||
| name, | ||
| raw, | ||
| default, | ||
| ) | ||
| return default | ||
|
|
||
|
|
||
| class BlazeConfig: | ||
| """ | ||
| Configuration for Blaze AI services. | ||
|
|
||
| All services (STT, TTS, LLM) route through a single gateway URL. | ||
| Service-specific configuration (language, speaker, etc.) comes from the | ||
| voicebot ID and is passed as constructor arguments to each plugin. | ||
|
|
||
| Environment Variables: | ||
| BLAZE_API_URL: Base URL for Blaze API gateway | ||
| BLAZE_API_TOKEN: Bearer token for API authentication | ||
| BLAZE_STT_TIMEOUT: STT request timeout in seconds | ||
| BLAZE_TTS_TIMEOUT: TTS per-request timeout in seconds | ||
| BLAZE_TTS_STREAM_TIMEOUT: TTS streaming session timeout in seconds | ||
| BLAZE_LLM_TIMEOUT: LLM request timeout in seconds | ||
|
|
||
| Example: | ||
| >>> from livekit.plugins.blaze import BlazeConfig | ||
| >>> | ||
| >>> # Load from environment variables | ||
| >>> config = BlazeConfig() | ||
| >>> | ||
| >>> # Or provide explicit values | ||
| >>> config = BlazeConfig( | ||
| ... api_url="https://api.blaze.vn", | ||
| ... api_token="my-token", | ||
| ... ) | ||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| *, | ||
| api_url: str | None = None, | ||
| api_token: str | None = None, | ||
| stt_timeout: float | None = None, | ||
| tts_timeout: float | None = None, | ||
| tts_stream_timeout: float | None = None, | ||
| llm_timeout: float | None = None, | ||
| ) -> None: | ||
| # Strip trailing slashes so HTTP paths like f"{api_url}/v1/..." never double-slash. | ||
| # Blank/whitespace env (or constructor) values fall back to the default, same as _env_float. | ||
| # Require absolute http(s) + hostname so a bad BLAZE_API_URL fails fast | ||
| # (WS rewrite must not invent a target that still carries the bearer token). | ||
| self.api_url: str = validate_api_base_url( | ||
| (api_url or os.environ.get("BLAZE_API_URL", "")).strip() or _DEFAULT_API_URL, | ||
| http_only=True, | ||
| ) | ||
| self.api_token: str = api_token or os.environ.get("BLAZE_API_TOKEN", "") | ||
| self.stt_timeout: float = ( | ||
| stt_timeout | ||
| if stt_timeout is not None | ||
| else _env_float("BLAZE_STT_TIMEOUT", _DEFAULT_STT_TIMEOUT) | ||
| ) | ||
| self.tts_timeout: float = ( | ||
| tts_timeout | ||
| if tts_timeout is not None | ||
| else _env_float("BLAZE_TTS_TIMEOUT", _DEFAULT_TTS_TIMEOUT) | ||
| ) | ||
| # Separate timeout for the full TTS streaming session (WebSocket + all batches). | ||
| # Streaming turns can span many text batches and require a longer timeout than | ||
| # individual per-request timeouts. | ||
| self.tts_stream_timeout: float = ( | ||
| tts_stream_timeout | ||
| if tts_stream_timeout is not None | ||
| else _env_float("BLAZE_TTS_STREAM_TIMEOUT", _DEFAULT_TTS_STREAM_TIMEOUT) | ||
| ) | ||
| self.llm_timeout: float = ( | ||
| llm_timeout | ||
| if llm_timeout is not None | ||
| else _env_float("BLAZE_LLM_TIMEOUT", _DEFAULT_LLM_TIMEOUT) | ||
| ) | ||
|
devin-ai-integration[bot] marked this conversation as resolved.
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.