[Hardware][Ascend][Model] Optimize Qwen3-TTS on 310P

[zyz111222]

Purpose

This PR adds Ascend 310P-specific optimizations for Qwen3-TTS.

The changes are scoped to the 310P platform patch path and avoid changing the
generic Qwen3-TTS model implementation. The patch optimizes both stage-0
Talker/CodePredictor and stage-1 Code2Wav execution on 310P.

Main changes:

• Keep the Mimi ref-audio encoder on CPU for 310P to avoid the second-request
  AICore fault seen with different reference audio lengths.
• Register the Qwen3-TTS Code2Wav 310P patch path.
• Pre-pack Code2Wav Linear weights to FRACTAL_NZ and groups==1 Conv1d /
  ConvTranspose1d weights to FRACTAL_Z after weight loading.
• Replace Code2Wav RMSNorm and RoPE with Ascend fused NPU operators.
• Replace CodePredictor RMSNorm, residual-add RMSNorm, RoPE, attention mask,
  and attention path with 310P-friendly fused/layout-aware operators.
• Pre-pack CodePredictor Linear weights and pre-project codec embedding weights
  to remove repeated per-step projection matmuls.
• Use vLLM-Ascend sampling helpers for the CodePredictor sampling path.
• Align NPU runner calls with the vLLM/vLLM-Omni v0.24 runner interface.

Test Plan

Hardware:

• Ascend 310P

Environment:

• CANN: 9.1.0 beta
• vLLM: v0.24.0
• vLLM-Ascend: vllm-project/vllm-ascend@05e095a
• Model: Qwen/Qwen3-TTS-12Hz-1.7B-Base

Deploy config:

# qwen3_tts_310p.yaml
async_chunk: true
dtype: float16

connectors:
  connector_of_shared_memory:
    name: SharedMemoryConnector
    extra:
      shm_threshold_bytes: 65536
      codec_streaming: true
      connector_get_sleep_s: 0.01
      connector_get_max_wait_first_chunk: 3000
      connector_get_max_wait: 300
      codec_chunk_frames: 25
      codec_left_context_frames: 72
      initial_codec_chunk_frames: 1
      decode_cudagraph_capture_sizes: [25, 73, 97, 169, 325]

stages:
  - stage_id: 0
    max_num_seqs: 10
    gpu_memory_utilization: 0.3
    trust_remote_code: true
    enable_prefix_caching: true
    async_scheduling: true
    max_num_batched_tokens: 512
    max_model_len: 16384
    devices: "0"
    output_connectors:
      to_stage_1: connector_of_shared_memory
    default_sampling_params:
      temperature: 0.9
      top_k: 50
      max_tokens: 4096
      seed: 42
      repetition_penalty: 1.05
    subtalker_sampling_params:
      do_sample: true
      temperature: 0.9
      top_k: 50
      top_p: 1.0

  - stage_id: 1
    max_num_seqs: 10
    gpu_memory_utilization: 0.3
    enforce_eager: false
    trust_remote_code: true
    enable_prefix_caching: false
    async_scheduling: true
    max_num_batched_tokens: 65536
    max_model_len: 65536
    devices: "0"
    input_connectors:
      from_stage_0: connector_of_shared_memory
    default_sampling_params:
      temperature: 0.0
      top_p: 1.0
      top_k: -1
      max_tokens: 65536
      seed: 42
      repetition_penalty: 1.0

platforms:
  npu:
    stages:
      - stage_id: 0
        enforce_eager: false

Compared with the default qwen3_tts.yaml, the 310P benchmark config only
changes the following fields:

  dtype: float16
  stages:
    - stage_id: 0
      max_model_len: 16384

Service command:

vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-Base
--omni
--port 8091
--task-type Base
--deploy-config qwen3_tts_310p.yaml

vLLM Version

v0.24.0

vLLM-Omni Commit

9ceca4e

Test Result

performance：
Use the seed-tts-eavl zh dataset, specifically the 10002287-00000099.wav file, to test the corresponding reference audio and output text. After preheating twice, use curl to send 20 requests and take the average.

precision：
seed-tts-eval zh dataset accuracy results:

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

can you check the recent vllm blog for how to optimize tts serving?

[hsliuustc0106]

the large patch seems hard to maintain in the long run

[zyz111222]

If we want to reduce the number of patches, some modifications need to be incorporated into the mainline. Currently, the accuracy has only been verified on version 310.

[gcanlin]

😂I'm not familiar with 310p. But the current change for 310p is hard to abstract.

[hsliuustc0106]

compare before/after this commit interms off ttfp/rtf/througput

[zyz111222]

compare before/after this commit interms off ttfp/rtf/througput
Before optimization, the RTF was around 1.3, and now it is around 0.85, an improvement of 35%.

[zyz111222]

can you check the recent vllm blog for how to optimize tts serving?

Here, we have mainly optimized the writing and computation process and integrated fusion operators, which are based on the design of vllm ascend.

[FayeSpica]

Solid performance gains on NPU A3.

Bench Script

python benchmarks/tts/bench_tts.py \
    --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --task voice_clone \
    --locale en \
    --concurrency 1 4 8 \
    --num-prompts 32 \
    --dataset-path /root/.cache/seedtts_testset \
    --host localhost --port 8091

Performace

Task: voice_clone (Qwen3-TTS on A3).

Concurrency	RTF (before → after)	Δ RTF	TTFP ms (before → after)	Δ TTFP	Throughput (before → after)	Δ Throughput
1	0.594 → 0.423	−28.8%	194 → 197	+1.5%	1.690 → 2.377	+40.7%
4	0.837 → 0.588	−29.7%	310 → 234	−24.5%	4.575 → 4.000	−12.6%
8	1.015 → 0.762	−24.9%	401 → 377	−6.0%	6.997 → 9.262	+32.4%

[FayeSpica]

Retested on A3, LGTM.

[amy-why-3459]

LGTM

[LKHZ]

wo我测试时出现这个问题，该怎么解决？

[zyz111222]

wo我测试时出现这个问题，该怎么解决？

Currently, 310 only supports the qwen3-tts model. Referenced versions:
vllm: e5588e49bc2642670116664a7fc4096e27adb179
vllm_ascend: 5083d8844310831258f085ea6dfcac4a2f76ef58
vllm_omni: 2d58070

[LKHZ]

我想问一下，在310p上有没有针对qwen3-tts的量化方法。目前RTF1.4-1.7左右，需要量化加快推理速度