From bfe0aa9219518b2a72a265fdaf4e1fe85ada9ae7 Mon Sep 17 00:00:00 2001 From: yiminghub2024 <482890@qq.com> Date: Sat, 15 Aug 2026 21:37:59 +0800 Subject: [PATCH 1/3] Update models.md --- docs/recipes/models.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/recipes/models.md b/docs/recipes/models.md index 167de2f396..ef2ed61e8a 100644 --- a/docs/recipes/models.md +++ b/docs/recipes/models.md @@ -430,7 +430,7 @@ Notes: - The DeepEP layout pulls ahead from mid batch up, where its expert kernels and the second attention replica both pay off. -### Qwen3.8-27B +### Qwen3.8-27B(Blackwell) A dense 27B-class Qwen3.8 FP8 checkpoint on a single GPU, with self-speculative MTP (the draft model path points at the same checkpoint): @@ -455,6 +455,34 @@ tokenspeed serve Qwen/Qwen3.8-27B-FP8 \ --host 0.0.0.0 --port 8000 ``` +### Qwen3.8-27B(Hopper) + +A dense 27B-class Qwen3.8 FP8 checkpoint on a single GPU, with self-speculative +MTP (the draft model path points at the same checkpoint): + +```bash +tokenspeed serve Qwen/Qwen3.8-27B-FP8 \ + --served-model-name Qwen/Qwen3.8-27B-FP8 \ + --world-size 1 \ + --gpu-memory-utilization 0.9 \ + --moe-backend fa3 \ + --drafter-attention-backend fa3 \ + --chunked-prefill-size 8192 \ + --max-model-len 256000 \ + --max-num-seqs 64 \ + --kv-cache-dtype fp8_e4m3 \ + --speculative-algorithm MTP \ + --speculative-draft-model-path Qwen/Qwen3.8-27B-FP8 \ + --speculative-num-steps 3 \ + --speculative-eagle-topk 1 \ + --speculative-num-draft-tokens 4 \ + --reasoning-parser qwen3 \ + --tool-call-parser qwen \ + --disable-kvstore \ + --host 0.0.0.0 --port 8000 +``` + + ## GPT-OSS 20B / 120B Small GPT-OSS launches can start simple. Large GPT-OSS launches usually tune From e115788b59c7df0466fadc3eb2e1924bc4492ce1 Mon Sep 17 00:00:00 2001 From: yiminghub2024 <482890@qq.com> Date: Sat, 15 Aug 2026 21:47:03 +0800 Subject: [PATCH 2/3] Update getting-started.md with kernel installation support hopper Update getting-started.md with kernel installation support hopper it tested on h20 --- docs/guides/getting-started.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/guides/getting-started.md b/docs/guides/getting-started.md index 7729ffb4aa..2d981843f0 100644 --- a/docs/guides/getting-started.md +++ b/docs/guides/getting-started.md @@ -45,12 +45,20 @@ pip install -e "./python" --no-build-isolation ``` Install the kernel package. Its Python package metadata installs the selected -backend dependencies automatically. +backend dependencies automatically(blackwell). ```bash pip install -e tokenspeed-kernel/python/ --no-build-isolation ``` +Install the kernel package. Its Python package metadata installs the selected +backend dependencies automatically(hopper). + +```bash +TOKENSPEED_CUDA_ARCH=90a MAX_JOBS=32 \ + pip install -e tokenspeed-kernel/python/ --no-build-isolation +``` + Install the scheduler package: ```bash From 776037c4336aabeaad45c3ece2f1cb3bf3c639ff Mon Sep 17 00:00:00 2001 From: yiminghub2024 <482890@qq.com> Date: Sun, 16 Aug 2026 07:24:17 +0800 Subject: [PATCH 3/3] Remove moe-backend option from tokenspeed serve command Removed the '--moe-backend fa3' option from the command. --- docs/recipes/models.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/recipes/models.md b/docs/recipes/models.md index ef2ed61e8a..ecaae18bb7 100644 --- a/docs/recipes/models.md +++ b/docs/recipes/models.md @@ -465,7 +465,6 @@ tokenspeed serve Qwen/Qwen3.8-27B-FP8 \ --served-model-name Qwen/Qwen3.8-27B-FP8 \ --world-size 1 \ --gpu-memory-utilization 0.9 \ - --moe-backend fa3 \ --drafter-attention-backend fa3 \ --chunked-prefill-size 8192 \ --max-model-len 256000 \