Skip to content

[Doc][Misc] Update GLM-5 and GLM-5.2 deployment guides with HCCL timeouts and per-node decode configs - #14713

Open
Wyz-134 wants to merge 9 commits into
vllm-project:releases/v0.23.0from
Wyz-134:releases/v0.23.0
Open

[Doc][Misc] Update GLM-5 and GLM-5.2 deployment guides with HCCL timeouts and per-node decode configs#14713
Wyz-134 wants to merge 9 commits into
vllm-project:releases/v0.23.0from
Wyz-134:releases/v0.23.0

Conversation

@Wyz-134

@Wyz-134 Wyz-134 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it?

  • GLM-5.2: replace the vllm serve <MODEL_PATH> placeholders with the
    concrete weight path /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.2-W8A8C8-A3-0808;
    split the PD decode nodes (198K and 1M) into separate Decode node 0
    (ranks 0–3) and Decode node 1 (ranks 4–7) configurations, each with its own run_dp_template.sh, keeping the per-node launch_online_dp.py commands (--dp-rank-start 0/4) in the start list; align the w8a8c8 weight entry format in chapter 3 with the other weight entries. - GLM-5: add HCCL_TRANSFER_TIMEOUT/HCCL_EXEC_TIMEOUT/ HCCL_CONNECT_TIMEOUT to the A3 multi-node co-located and A3 PD
    disaggregation scripts; split the A3 PD decode nodes into separate
    Decode node 0 (ranks 0–3) and Decode node 1 (ranks 4–7) configurations; remove the stale glm-5-w8a8 A3 single-node deployment script.

Does this PR introduce any user-facing change?

No, documentation-only update.

How was this patch tested?

  • Built the docs locally with Sphinx (build succeeded) and previewed the
    rendered GLM-5/GLM-5.2 pages.

  • No code changes; no tests required.

  • vLLM version: v0.23.0

  • vLLM main: vllm-project/vllm@ee0da84

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request provides comprehensive updates to the deployment documentation for GLM-5 and GLM-5.2 models on Ascend hardware. The changes focus on standardizing configuration scripts, ensuring robust communication settings through new HCCL timeout exports, and providing more granular, node-specific instructions for Prefill-Decode (PD) disaggregation scenarios. These updates improve the reliability and maintainability of the deployment guides.

Highlights

  • Documentation Updates: Updated GLM-5 and GLM-5.2 deployment guides to include specific model weight paths and refined configuration templates.
  • Configuration Enhancements: Added critical HCCL timeout environment variables (TRANSFER, EXEC, CONNECT) to multi-node and PD disaggregation scripts to improve stability.
  • Deployment Refinement: Split PD decode node configurations into distinct 'Decode node 0' and 'Decode node 1' sections with dedicated templates and parameters for better clarity.
  • Cleanup: Removed stale single-node deployment scripts for GLM-5 to prevent user confusion.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 21, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Suggested PR Title:

[Doc][Misc] Update GLM-5 and GLM-5.2 documentation for Ascend deployment

Suggested PR Summary:

### What this PR does / why we need it?

This pull request updates the deployment documentation for GLM-5 and GLM-5.2 models on Ascend. Key changes include:
- Updating model paths from placeholders to concrete ModelScope paths.
- Splitting and detailing the configuration templates for Decode Node 0 and Decode Node 1.
- Adding HCCL timeout environment variables (`HCCL_TRANSFER_TIMEOUT`, `HCCL_EXEC_TIMEOUT`, `HCCL_CONNECT_TIMEOUT`) to multiple script templates.
- Removing outdated deployment instructions for quantized models from `GLM5.md`.

Feedback on these changes:
- In `GLM5.2.md`, the description for `GLM-5.2-w4a8c8` was cut off with a semicolon, and its download link was accidentally removed. This should be restored.

### Does this PR introduce _any_ user-facing change?

No, this is a documentation-only update.

### How was this patch tested?

No testing is required as these are documentation-only updates.

Comment thread docs/source/tutorials/models/GLM5.2.md Outdated
- `GLM-5.2-w8a8c8`(Quantized version for Atlas 800 A3): requires 2 Atlas 800 A3 (64GB × 16) node.[Download model weight](https://modelers.cn/models/Eco-Tech/GLM-5.2-w8a8c8).
- `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios; use `GLM-5.2-w8a8c8` for PD deployment instead. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8).
- `GLM-5.2-w8a8c8`: requires 2 Atlas 800 A3 (64GB × 16) node.[Download model weight](https://modelers.cn/models/Eco-Tech/GLM-5.2-w8a8c8).
- `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The sentence for GLM-5.2-w4a8c8 is cut off, ending with a semicolon, and the download link has been accidentally removed. Please restore the download link and end the sentence with a period.

Suggested change
- `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios;
- `GLM-5.2-w4a8c8` (experimental): requires 1 Atlas 800 A3 (128GB × 8) node or 2 Atlas 800 A2 (64GB × 16) node. This experimental feature has known accuracy issues in Prefill-Decode (PD) disaggregation scenarios. [Download model weight](https://www.modelscope.cn/models/Eco-Tech/GLM-5.2-w4a8c8).

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

w30084620 and others added 9 commits August 21, 2026 17:41
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
…entries

The w8a8c8 entry in chapter 3 (3.1 Model Weight) carried a unique
'(Quantized version for Atlas 800 A3)' descriptor; remove it so the
entry follows the same format as the other weight entries.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
…s in GLM-5

- Add HCCL_TRANSFER_TIMEOUT/HCCL_EXEC_TIMEOUT/HCCL_CONNECT_TIMEOUT
  to the A3 multi-node co-located and A3 PD disaggregation scripts.
- Split the A3 PD decode nodes into separate 'Decode node 0 (ranks 0-3)'
  and 'Decode node 1 (ranks 4-7)' configurations, each with its own
  run_dp_template.sh, keeping the per-node launch commands in the start list.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
…s in GLM-5.2

- Replace the 'vllm serve <MODEL_PATH>' placeholders with the concrete
  path /root/.cache/modelscope/hub/models/vllm-ascend/GLM-5.2-W8A8C8-A3-0808.
- Split the PD decode nodes (198K and 1M) into separate 'Decode node 0'
  (ranks 0-3) and 'Decode node 1' (ranks 4-7) configurations, each with its
  own run_dp_template.sh; the per-node launch_online_dp.py commands
  (--dp-rank-start 0/4) stay in the 'start the server on each node' list.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
All Atlas 800 A3 references in chapter 3 (Model Weight) now use the
64GB × 16 node specification (previously 128GB × 16 / 128GB × 8).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
…GLM-5.2 scripts

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
… in GLM-5/5.2

- Remove the GLM-5.2 (BF16 version) entry from the model weight list.
- Update all hardware spec parentheses: Atlas 800 A3 -> 128GB x 8,
  Atlas 800 A2 -> 64GB x 8.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
…LM-5.2

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: w30084620 <w30084620@huawei.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant