[XPU] Fix causal_conv1d_update_xpu signature for GDN speculative decode - #397
Open
dayanandav wants to merge 1 commit into
Open
[XPU] Fix causal_conv1d_update_xpu signature for GDN speculative decode#397dayanandav wants to merge 1 commit into
dayanandav wants to merge 1 commit into
Conversation
The GDN target-verify path in sglang calls `causal_conv1d_update` with
speculative-decoding tree arguments (`intermediate_conv_window`,
`intermediate_state_indices`, `retrieve_next_token`,
`retrieve_next_sibling`, `retrieve_parent_token`). The XPU wrapper only
accepted arguments up to `pad_slot_id`, so serving a hybrid GDN model
(e.g. Qwen3.5) with `--device xpu --speculative-algorithm NEXTN` crashed
during warmup with:
TypeError: causal_conv1d_update_xpu() got an unexpected keyword
argument 'intermediate_conv_window'
Accept and ignore unknown keyword arguments, matching what
`causal_conv1d_fn_xpu` already does, so the XPU wrapper stays tolerant of
upstream signature changes. Add a regression test covering the kwargs
from the failing call site.
Fixes sgl-project/sglang#34720
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dayanandav
marked this pull request as ready for review
August 21, 2026 11:56
Author
|
I recommend this solution on kernel side, @jiayisunx @Xia-Weiwen requested for your review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Serving a hybrid GDN model (e.g. Qwen3.5) on XPU with speculative decoding crashes during warmup:
The GDN target-verify path in sglang (
python/sglang/srt/layers/attention/linear/gdn_backend.py) callscausal_conv1d_updatewith five speculative-decoding tree arguments:intermediate_conv_windowintermediate_state_indicesretrieve_next_tokenretrieve_next_siblingretrieve_parent_tokencausal_conv1d_update_xpuonly accepted arguments up topad_slot_id, so the call raisesTypeErrorbefore reaching the kernel. Only the
TARGET_VERIFYpath is affected; non-speculative decode is fine.Fixes sgl-project/sglang#34720
Modifications
python/sgl_kernel/mamba.py:causal_conv1d_update_xpunow accepts and ignores unknown keywordarguments via
**kwargs, matching whatcausal_conv1d_fn_xpuin the same file already does. Thiskeeps the XPU wrapper tolerant of upstream signature changes instead of hard-failing at warmup.
tests/test_causal_conv1d.py: newtest_causal_conv1d_update_ignores_spec_decode_kwargsregressiontest that reproduces the failing call site (all five kwargs) and asserts both the output and the
in-place
conv_stateupdate still match the reference.Note for reviewers
The tree-topology arguments are accepted but not honored by the SYCL kernel — no intermediate conv
window is checkpointed per draft token. This unblocks the crash and is correct for linear-chain draft
topologies, but tree-based drafting (
topk > 1) would need the checkpointing implemented insrc/sycl/causal_conv1d.cppto be numerically correct. Happy to follow up with either the kernel-sidesupport or an explicit guard/warning for the tree case if maintainers prefer failing loudly over
silently ignoring.
Accuracy Test
tests/test_causal_conv1d.pyon Intel XPU — full file, including the new test:Benchmark & Profiling
Not applicable — signature-only change, no kernel or performance impact.
Checklist
🤖 Generated with Claude Code