Skip to content

[XPU] Fix causal_conv1d_update_xpu signature for GDN speculative decode - #397

Open
dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:fix/causal-conv1d-update-spec-decode-kwargs
Open

[XPU] Fix causal_conv1d_update_xpu signature for GDN speculative decode#397
dayanandav wants to merge 1 commit into
sgl-project:mainfrom
dayanandav:fix/causal-conv1d-update-spec-decode-kwargs

Conversation

@dayanandav

Copy link
Copy Markdown

Motivation

Serving a hybrid GDN model (e.g. Qwen3.5) on XPU with speculative decoding crashes during warmup:

TypeError: causal_conv1d_update_xpu() got an unexpected keyword argument 'intermediate_conv_window'

The GDN target-verify path in sglang (python/sglang/srt/layers/attention/linear/gdn_backend.py) calls
causal_conv1d_update with five speculative-decoding tree arguments:

  • intermediate_conv_window
  • intermediate_state_indices
  • retrieve_next_token
  • retrieve_next_sibling
  • retrieve_parent_token

causal_conv1d_update_xpu only accepted arguments up to pad_slot_id, so the call raises TypeError
before reaching the kernel. Only the TARGET_VERIFY path is affected; non-speculative decode is fine.

Fixes sgl-project/sglang#34720

Modifications

  • python/sgl_kernel/mamba.py: causal_conv1d_update_xpu now accepts and ignores unknown keyword
    arguments via **kwargs, matching what causal_conv1d_fn_xpu in the same file already does. This
    keeps the XPU wrapper tolerant of upstream signature changes instead of hard-failing at warmup.
  • tests/test_causal_conv1d.py: new test_causal_conv1d_update_ignores_spec_decode_kwargs regression
    test that reproduces the failing call site (all five kwargs) and asserts both the output and the
    in-place conv_state update still match the reference.

Note for reviewers

The tree-topology arguments are accepted but not honored by the SYCL kernel — no intermediate conv
window is checkpointed per draft token. This unblocks the crash and is correct for linear-chain draft
topologies, but tree-based drafting (topk > 1) would need the checkpointing implemented in
src/sycl/causal_conv1d.cpp to be numerically correct. Happy to follow up with either the kernel-side
support or an explicit guard/warning for the tree case if maintainers prefer failing loudly over
silently ignoring.

Accuracy Test

tests/test_causal_conv1d.py on Intel XPU — full file, including the new test:

761 passed in 16.33s

Benchmark & Profiling

Not applicable — signature-only change, no kernel or performance impact.

Checklist

🤖 Generated with Claude Code

The GDN target-verify path in sglang calls `causal_conv1d_update` with
speculative-decoding tree arguments (`intermediate_conv_window`,
`intermediate_state_indices`, `retrieve_next_token`,
`retrieve_next_sibling`, `retrieve_parent_token`). The XPU wrapper only
accepted arguments up to `pad_slot_id`, so serving a hybrid GDN model
(e.g. Qwen3.5) with `--device xpu --speculative-algorithm NEXTN` crashed
during warmup with:

    TypeError: causal_conv1d_update_xpu() got an unexpected keyword
    argument 'intermediate_conv_window'

Accept and ignore unknown keyword arguments, matching what
`causal_conv1d_fn_xpu` already does, so the XPU wrapper stays tolerant of
upstream signature changes. Add a regression test covering the kwargs
from the failing call site.

Fixes sgl-project/sglang#34720

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dayanandav

Copy link
Copy Markdown
Author

I recommend this solution on kernel side, @jiayisunx @Xia-Weiwen requested for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [XPU] Qwen3.5 GDN + speculative decode: causal_conv1d_update_xpu() got an unexpected keyword argument 'intermediate_conv_window'

1 participant