use cute-dsl to speedup mxfp QDQ

[xin3he]

VLLM_QDQ_CUTE=1 Qwen3.6-35B-A3B MXFP4-mixed
Avg generation throughput: 2672.3 tokens/s -> 4137.6 tokens/s

Type of Change

feature

Description

This pull request adds a high-performance fused CuTe backend for MXFP4 and MXFP8 QDQ in the vllm-qdq-plugin, with full runtime selection, validation, and CUDA Graph support. The implementation includes new CuTe DSL kernels, integration into the QDQ API, and extensive documentation and benchmarking tools for users and developers.

CuTe QDQ Backend Integration:

• Added a new CuTe backend for MXFP4 and MXFP8 QDQ, using custom CUDA kernels written in the NVIDIA CUTLASS DSL. The backend is automatically selected at runtime when supported (SM80+ GPUs, contiguous input, group size 32, and K divisible by 32), otherwise it falls back to the reference implementation. (src/vllm_qdq_plugin/qdq/cute_kernels.py [1] src/vllm_qdq_plugin/qdq/cute.py [2]
• Integrated backend selection into the main QDQ API, so mxfp4_qdq and mxfp8_qdq transparently use the CuTe backend if enabled via the VLLM_QDQ_CUTE environment variable. (src/vllm_qdq_plugin/qdq/mxfp4.py [1] [2] src/vllm_qdq_plugin/qdq/mxfp8.py [3] [4]
• Registered the new VLLM_QDQ_CUTE environment variable for configuration and validation. (src/vllm_qdq_plugin/envs.py benchmark/vllm-qdq-plugin/src/vllm_qdq_plugin/envs.pyR75)

Developer and User Tooling:

• Added scripts to verify the CuTe DSL installation (scripts/verify_cute_dsl.py) and to benchmark/validate the CuTe QDQ kernels against the reference implementation, reporting accuracy and speedup (scripts/bench_qdq_cute.py). [1] [2]

Documentation Updates:

• Updated the README to document CuTe backend usage, requirements, configuration, and validation/benchmarking steps. Added a section on CUDA Graph support and clarified development installation instructions. (README.md [1] [2] [3] [4]

Internal Refactoring:

• Refactored QDQ reference implementations to allow clean fallback and separation between reference and CuTe logic. (src/vllm_qdq_plugin/qdq/mxfp4.py [1] [2] src/vllm_qdq_plugin/qdq/mxfp8.py [3] [4] [5] [6]

This update provides a significant speedup for QDQ on supported GPUs and enables advanced CUDA features, while maintaining correctness and fallback safety.

Expected Behavior & Potential Risk

the expected behavior that triggered by this PR

How has this PR been tested?

how to reproduce the test (including hardware information)

Dependency Change?

any library dependency introduced or removed

[yiliu30]

Overall, LGTM! Could you also attach the GSM8K accuracy comparison between the CuTe and reference kernels for completeness? Thanks!