[Doc][Misc] Update GLM-5 and GLM-5.2 deployment guides with HCCL timeouts and per-node decode configs - #14713
[Doc][Misc] Update GLM-5 and GLM-5.2 deployment guides with HCCL timeouts and per-node decode configs#14713Wyz-134 wants to merge 9 commits into
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request provides comprehensive updates to the deployment documentation for GLM-5 and GLM-5.2 models on Ascend hardware. The changes focus on standardizing configuration scripts, ensuring robust communication settings through new HCCL timeout exports, and providing more granular, node-specific instructions for Prefill-Decode (PD) disaggregation scenarios. These updates improve the reliability and maintainability of the deployment guides. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Doc][Misc] Update GLM-5 and GLM-5.2 documentation for Ascend deploymentSuggested PR Summary:
### What this PR does / why we need it?
This pull request updates the deployment documentation for GLM-5 and GLM-5.2 models on Ascend. Key changes include:
- Updating model paths from placeholders to concrete ModelScope paths.
- Splitting and detailing the configuration templates for Decode Node 0 and Decode Node 1.
- Adding HCCL timeout environment variables (`HCCL_TRANSFER_TIMEOUT`, `HCCL_EXEC_TIMEOUT`, `HCCL_CONNECT_TIMEOUT`) to multiple script templates.
- Removing outdated deployment instructions for quantized models from `GLM5.md`.
Feedback on these changes:
- In `GLM5.2.md`, the description for `GLM-5.2-w4a8c8` was cut off with a semicolon, and its download link was accidentally removed. This should be restored.
### Does this PR introduce _any_ user-facing change?
No, this is a documentation-only update.
### How was this patch tested?
No testing is required as these are documentation-only updates.| - `GLM-5.2-w8a8c8`(Quantized version for Atlas 800 A3): requires 2 Atlas 800 A3 (64GB × 16) node.[Download model weight](https://modelers.cn/models/Eco-Tech/GLM-5.2-w8a8c8). | ||
| - `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios; use `GLM-5.2-w8a8c8` for PD deployment instead. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8). | ||
| - `GLM-5.2-w8a8c8`: requires 2 Atlas 800 A3 (64GB × 16) node.[Download model weight](https://modelers.cn/models/Eco-Tech/GLM-5.2-w8a8c8). | ||
| - `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios; |
There was a problem hiding this comment.
The sentence for GLM-5.2-w4a8c8 is cut off, ending with a semicolon, and the download link has been accidentally removed. Please restore the download link and end the sentence with a period.
| - `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios; | |
| - `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8). |
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
f472a04 to
7db783a
Compare
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
…entries The w8a8c8 entry in chapter 3 (3.1 Model Weight) carried a unique '(Quantized version for Atlas 800 A3)' descriptor; remove it so the entry follows the same format as the other weight entries. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
…s in GLM-5 - Add HCCL_TRANSFER_TIMEOUT/HCCL_EXEC_TIMEOUT/HCCL_CONNECT_TIMEOUT to the A3 multi-node co-located and A3 PD disaggregation scripts. - Split the A3 PD decode nodes into separate 'Decode node 0 (ranks 0-3)' and 'Decode node 1 (ranks 4-7)' configurations, each with its own run_dp_template.sh, keeping the per-node launch commands in the start list. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
…s in GLM-5.2 - Replace the 'vllm serve <MODEL_PATH>' placeholders with the concrete path /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.2-W8A8C8-A3-0808. - Split the PD decode nodes (198K and 1M) into separate 'Decode node 0' (ranks 0-3) and 'Decode node 1' (ranks 4-7) configurations, each with its own run_dp_template.sh; the per-node launch_online_dp.py commands (--dp-rank-start 0/4) stay in the 'start the server on each node' list. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
All Atlas 800 A3 references in chapter 3 (Model Weight) now use the 64GB × 16 node specification (previously 128GB × 16 / 128GB × 8). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
…GLM-5.2 scripts Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
… in GLM-5/5.2 - Remove the GLM-5.2 (BF16 version) entry from the model weight list. - Update all hardware spec parentheses: Atlas 800 A3 -> 128GB x 8, Atlas 800 A2 -> 64GB x 8. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
…LM-5.2 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: w30084620 <w30084620@huawei.com>
774aaf0 to
2f2f2a8
Compare
What this PR does / why we need it?
vllm serve <MODEL_PATH>placeholders with theconcrete weight path
/root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.2-W8A8C8-A3-0808;split the PD decode nodes (198K and 1M) into separate
Decode node 0(ranks 0–3) and
Decode node 1(ranks 4–7) configurations, each with its ownrun_dp_template.sh, keeping the per-nodelaunch_online_dp.pycommands (--dp-rank-start 0/4) in the start list; align thew8a8c8weight entry format in chapter 3 with the other weight entries. - GLM-5: addHCCL_TRANSFER_TIMEOUT/HCCL_EXEC_TIMEOUT/HCCL_CONNECT_TIMEOUTto the A3 multi-node co-located and A3 PDdisaggregation scripts; split the A3 PD decode nodes into separate
Decode node 0(ranks 0–3) andDecode node 1(ranks 4–7) configurations; remove the staleglm-5-w8a8A3 single-node deployment script.Does this PR introduce any user-facing change?
No, documentation-only update.
How was this patch tested?
Built the docs locally with Sphinx (
build succeeded) and previewed therendered GLM-5/GLM-5.2 pages.
No code changes; no tests required.
vLLM version: v0.23.0
vLLM main: vllm-project/vllm@ee0da84