OmniVoice 🌍

     

OmniVoice is a state-of-the-art massive multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it generates high-quality speech with superior inference speed, supporting voice cloning and voice design.

Contents: Key Features | Installation | Quick Start | Python API | Command-Line Tools | Training & Evaluation | Discussion | Citation

Key Features

• 600+ Languages Supported: The broadest language coverage among zero-shot TTS models (full list).
• Voice Cloning: State-of-the-art voice cloning quality.
• Voice Design: Control voices via assigned speaker attributes (gender, age, pitch, dialect/accent, whisper, etc.).
• Fine-grained Control: Non-verbal symbols (e.g., [laughter]) and pronunciation correction via pinyin or phonemes.
• Fast Inference: RTF as low as 0.025 (40x faster than real-time).
• Diffusion Language Model-Style Architecture: A clean, streamlined, and scalable design that delivers both quality and speed.

---

分支说明

完整功能版 OmniVoice 服务器，保留官方所有 Web UI 功能，同时添加 OpenAI 兼容 API。

特点

• 完整 Web UI：声音克隆、声音设计、自动声音三个模式
• OpenAI API：兼容 AnythingLLM 等工具,可以选择克隆音色作为API的默认音色
• 模型共享：Web UI 和 API 共用模型，节省内存

---

uv安装

克隆仓库并创建环境

git clone https://github.com/alex13by/OmniVoice-API.git
cd OmniVoice-API
uv sync

---

Quick Start

运行start_full.bat会自动下载模型到当前位置modems目录下，直到看到： INFO: Uvicorn running on http://0.0.0.0:5050 (Press CTRL+C to quit)

• Running on local URL: http://0.0.0.0:8001

---

AnythingLLM 配置

1. 设置 → Voice & Speech → TTS Provider: Generic OpenAI
2. 配置：
   • API Endpoint: http://localhost:5050/v1
   • API Key: omnivoice
   • Model: tts-1
   • Voice: alloy 或描述如 female, low pitch

---

uv run python omnivoice_full_server.py --web_port 8080 --api_port 5051

Citation

@article{zhu2026omnivoice,
      title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
      author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
      journal={arXiv preprint arXiv:2604.00688},
      year={2026}
}