[NPUW] Support GQA sliding-window and smooth-softmax in DecomposeGQA

[ankitm3k]

Details

Extends NPUW's DecomposeGQA (decompose_gqa.cpp) to honor two com.microsoft.GroupQueryAttention
features that were previously ignored on the NPU LLM path, matching the common
GroupQueryAttentionDecomposition behavior:

• local_window_size (sliding-window attention) — previously a windowed model was lowered as plain
  causal attention (silently wrong output).
• smooth_softmax / head_sink — previously lowered with a plain softmax (silently wrong denominator).

Two feature commits + one test commit. Touches only decompose_gqa.cpp and adds a unit test.

What changed

Sliding window (local_window_size)

Applied as an additive mask band over NPUW's static, host-managed KV cache. The band is branch-specific
because NPUW's mask coordinates are not the true absolute positions:

• Prefill: past and current share one right-aligned frame, so vert - hori is the true query-key
  distance; mask keys with vert - hori >= local_window_size.
• Generate: past keys are left-aligned in [0, past_seqlen) and the current block sits at the fixed
  capacity slots [C, C+curr), so the mask coordinate is not the query/key position. The window is computed
  in true absolute positions per query row — q_abs(i) = (total - curr) + i, k_abs(h) = h for a past slot
  (h < C) or (total - curr) + (h - C) for a current-block slot — masking when
  q_abs - k_abs >= local_window_size. The past/current boundary is the capacity C, not seqlens_k
  (seqlens_k is wrong once the past overflows the buffer, seqlens_k >= C). The generate base KV-cache mask
  was also generalized from single-token-only to gate on past_seqlen and the whole current block, so it is
  correct for multi-token (speculative) decode.

sliding_window_cache (the physical rolling KV buffer) is intentionally not honored: NPUW manages the KV
cache statically on the host, so applying the window as a mask over the full cache yields the same attention
result and fits NPUW's static-shape design. The frontend guarantees local_window_size >= 1 whenever
sliding_window_cache = 1, so the masking path fully covers the windowed-cache semantics.

smooth_softmax / head_sink

Wired onto SDPA's sink input, mirroring the common decomposition: a [1, num_heads, 1, 1] sink tensor (the
head_sink input reshaped per-head, or a zero broadcast for plain smooth_softmax) passed as the 6th input
of ScaledDotProductAttention with an explicit scale. The extra logit participates in the softmax denominator
and is sliced out. When neither is set, the original 4-/5-input SDPA path is unchanged.

batch > 1 is rejected at the internal-op level, so NPUW inherits that guard; quantized KV (i8/i4/f8) is out of scope here — proper VPUX-native support is tracked separately.

Testing

Added npuw_decompose_gqa_test.cpp (ov_npu_unit_tests, 6 cases): the internal op fully decomposes to a
single SDPA in both prefill and generate branches, the local_window_size band decomposes on both, and
smooth_softmax / head_sink select the 6-input SDPA (sink) form. All pass.

Numerical parity of the windowing coordinates was verified off-tree against a NumPy port of the ORT windowed
attention on the decomposed graph (run on CPU), across generate and prefill, curr = 1 / 2 / 3,
local_window_size = -1 / 1 / 3 / large, and past-overflow (seqlens_k >= capacity): the window masks
exactly the correct keys (plain-causal at a large window, self-only at window 1), stale buffer slots are fully
masked, and single-token decode is unchanged; smooth_softmax (zero sink) and per-head head_sink match to
~1e-7. A committable numerical oracle test (replacing the off-tree check) is a tracked follow-up — the
committed structural tests guard against the pass silently not firing or dropping the window/sink wiring.

Notes / scope

• SWC is not modeled by design (see above), not an omission.
• The mask math is verified at graph level against synthetic buffers; end-to-end interaction with NPUW's live
  host-side KV management (num_stored_tokens / make_tensor_slice) during a real window-crossing
  speculative step is not covered by an automated test.

Tickets:

• ticket-id

AI Assistance

Yes — used for mapping the ONNX Runtime windowed-attention semantics onto NPUW's capacity-relative mask
coordinate system, adversarial correctness review of the generate-branch coordinates, and generating the
NumPy reference tensors.