Skip to content

[BugFix][MRV2] Fix DSpark spec decoding on model runner v2 - #14696

Merged
realliujiaxu merged 1 commit into
vllm-project:mainfrom
wxsIcey:wxs/mrv2/fix-dspark
Aug 21, 2026
Merged

[BugFix][MRV2] Fix DSpark spec decoding on model runner v2#14696
realliujiaxu merged 1 commit into
vllm-project:mainfrom
wxsIcey:wxs/mrv2/fix-dspark

Conversation

@wxsIcey

@wxsIcey wxsIcey commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it?

  • Allocate the MTP pre-hc_head residual buffer for every speculative method that consumes target hidden states (eagle/eagle3/mtp/dflash/dspark/draft_model) instead of only mtp, and only on the last PP rank.
    Previously DSpark crashed during the memory-profiling dummy run with "TypeError: 'NoneType' object is not subscriptable".

  • Use storage_block_size instead of the non-existent block_size attribute
    when building DSpark SWA indices in AscendDSAMetadataBuilder, fixing "AttributeError: 'AscendDSAMetadataBuilder' object has no attribute 'block_size'" in the draft propose path.

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

CI passed with new added/existing test.

Signed-off-by: root <root@vllm-ascend.local>
@wxsIcey
wxsIcey requested a review from weijinqian0 as a code owner August 21, 2026 02:12
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses critical bugs encountered during speculative decoding on Model Runner v2. It ensures proper memory allocation for speculative hidden states across various methods and corrects an attribute reference error in the metadata builder, improving stability during memory profiling and draft proposal phases.

Highlights

  • Buffer Allocation Fix: Updated the MTP pre-hc_head residual buffer allocation to support all speculative methods (eagle, mtp, dflash, dspark, etc.) on the last pipeline parallel rank, preventing runtime crashes.
  • Attribute Correction: Fixed an AttributeError in AscendDSAMetadataBuilder by replacing the non-existent block_size attribute with storage_block_size when building SWA indices.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@wxsIcey wxsIcey added the ready-precise run selected e2e test for pr label Aug 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.


Tip

💡 Consider Linking a Related Issue or RFC

Your PR title contains the [BugFix] tag, indicating a bug fix or new feature.

Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:

  • Fixes #<issue_number>
  • Closes #<issue_number>
  • Resolves #<issue_number>
  • Refs #<rfc_or_issue_number> (for RFCs)

🙏 Thanks for helping us keep the project well-organized!

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Suggested PR Title:

[Attention][BugFix] Fix block size usage and MTP hidden buffer allocation

Suggested PR Summary:

### What this PR does / why we need it?
This PR updates the attention metadata building to use `storage_block_size` instead of `block_size` in `dsa_v1.py`. It also refactors the allocation of `_mtp_hidden_buffer` in DeepSeek-V4 model to only allocate on the last pipeline parallel rank when MTP hidden states are needed (either via Eagle or a draft model).

However, there is an issue in the implementation: `uses_draft_model` is a property on `SpeculativeConfig`, not a method, so calling it as `spec_config.uses_draft_model()` will raise a `TypeError`. This needs to be corrected to `spec_config.uses_draft_model`.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
CI/CD tests.

Comment on lines +817 to +819
needs_mtp_hidden_states = spec_config is not None and (
spec_config.use_eagle() or spec_config.uses_draft_model()
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

In vLLM, uses_draft_model is a property on SpeculativeConfig, not a method. Calling spec_config.uses_draft_model() will raise a TypeError: 'bool' object is not callable. Please access it as a property without parentheses: spec_config.uses_draft_model.

Additionally, verify if use_eagle is also a property or a method on your SpeculativeConfig class to avoid similar errors.

Suggested change
needs_mtp_hidden_states = spec_config is not None and (
spec_config.use_eagle() or spec_config.uses_draft_model()
)
needs_mtp_hidden_states = spec_config is not None and (
spec_config.use_eagle() or spec_config.uses_draft_model
)

MrZ20 added a commit to MrZ20/vllm-ascend that referenced this pull request Aug 21, 2026
Temporarily apply the fix from vllm-project#14696 so the dedicated A3 CI run validates it together with the runner-routing changes.

Signed-off-by: MrZ20 <2609716663@qq.com>
MrZ20 added a commit to MrZ20/vllm-ascend that referenced this pull request Aug 21, 2026
Temporarily apply the fix from vllm-project#14696 so the dedicated A3 CI run validates it together with the runner-routing changes.

Signed-off-by: MrZ20 <2609716663@qq.com>
MrZ20 added a commit to MrZ20/vllm-ascend that referenced this pull request Aug 21, 2026
Temporarily apply the fix from vllm-project#14696 so the dedicated A3 CI run validates it together with the runner-routing changes.

Signed-off-by: MrZ20 <2609716663@qq.com>
@realliujiaxu
realliujiaxu merged commit 4ce367a into vllm-project:main Aug 21, 2026
51 of 55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-precise run selected e2e test for pr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants