[QNN:Bugfix] Preserve calibrated MUL activation ranges for QNN export

[yydhYYDH]

Description

This PR fixes activation quantization propagation for LLM NPU export paths.

Binary MUL outputs, such as the gate * up product in SwiGLU, cannot safely derive their quantization range from either input. The product range can be orders of magnitude larger than the operand ranges. Reusing an operand scale causes INT16/UINT16 saturation on QNN and corrupts downstream hidden states, which can result in garbled generation.

This PR partially addresses #4716 by fixing the LLM export portion of the issue, specifically the OmniQuant/SmoothQuant activation-quantization propagation used for QNN/NPU models. It does not claim to resolve Qwen3-VL cases covered by #4716.

The changes:

• Preserve calibrated activation ranges for Binary MUL outputs instead of inferring them from an operand.
• Keep integer index/shape tensors and Gather indices outside activation quantization.
• Apply the corrected propagation rules consistently to OmniQuant and SmoothQuant.

The fix is split into two commits:

11a6c407 [LLM:Bugfix] Protect index tensors during NPU quantization
8682b2bc [LLM:Bugfix] Preserve calibrated MUL activation ranges

For Qwen3-0.6B, the old layer 2 SwiGLU product incorrectly inherited an input scale near 0.0018367. After the fix, the calibrated MUL output and down-projection input both use:

scale = 0.1174984994301709
zero  = -29684

Validation used a full Qwen3-0.6B OmniQuant export with 10 calibration samples, 5 epochs, 4-bit weights, and asymmetric 16-bit activations. All 28 MLP MUL tensors matched their corresponding down-projection input scale and zero point.

QNN offline compilation completed for Snapdragon 8 Elite with soc_id=69, dsp_arch=v79, vtcm_mb=8, and max_history_token=0, producing 30 context binary files. Host CPU, OnePlus 13 CPU, and OnePlus 13 QNN runs all exited with code 0 and generated coherent text.

Reproduction commands

OUT=/tmp/qwen3-mnn

python transformers/llm/export/llmexport.py \
  --path /path/to/Qwen3-0.6B \
  --export mnn \
  --dst_path "$OUT" \
  --mnnconvert /path/to/MNN/build/MNNConvert \
  --quant_bit 4 \
  --quant_block 64 \
  --lm_quant_bit 4 \
  --lm_quant_block 64 \
  --omni \
  --omni_epochs 5 \
  --calib_data /tmp/qwen3_wikitext_10.txt \
  --generate_for_npu \
  --seperate_embed \
  --sym \
  --act_bit 16

python transformers/llm/export/npu/generate_llm_qnn.py \
  --model "$OUT" \
  --soc_id 69 \
  --dsp_arch v79 \
  --vtcm_mb 8 \
  --mnn_path /path/to/MNN/build \
  --chunk_size 128 \
  --max_history_token 0

Test prompt

The same prompt was used for host CPU, OnePlus 13 CPU, and OnePlus 13 QNN:

Explain what a transformer model is in one short sentence.

Test results

Quantization metadata validation:

PASS 28 MLP MUL tensors
layer 2 MUL:       scale=0.1174984994301709, zero=-29684
layer 2 down_proj: scale=0.1174984994301709, zero=-29684

Host MNN CPU generated coherent text:

Okay, the user wants a short explanation of what a transformer model is in one sentence. Let me start by recalling what I know. Transformers are a...

OnePlus 13 CPU result:

output: Okay, the user wants a short explanation of what a transformer model is in one sentence. Transformer models are a type of neural network used to process sequential...

OnePlus 13 QNN result:

output: Okay, the user is asking for a short explanation of what a transformer model is in one sentence. Let me start by recalling what I know. A...

Both mobile runs generated coherent text without the previous garbled-token failure. Performance numbers are included only to identify the validated runs; this PR targets correctness rather than performance.

Module

LLM

Type

• Feature
• Bugfix
• Perf
• Refact
• Style
• Doc
• Test
• Chore

Checklist

• Commit message follows [Module:Type] Description format
• Code compiles without errors
• Tested on relevant platform(s)
• No unrelated format or style changes included

[yydhYYDH]

Additional Qwen3-VL validation has been completed on Snapdragon 8 Elite.

This PR now also fixes the Qwen3-VL QNN offline export and runtime path:

• Read the visual model type from llm_config.json instead of inferring it from the model directory name.
• Declare both image_embeds and deepstack_feature as Qwen3-VL visual graph outputs.
• Generate the correct deepstack_embeds shapes for prefill and decode.
• Handle is_mrope and has_deepstack independently to avoid adding DeepStack inputs to models such as Qwen2.5-VL.
• Slice multimodal RoPE positions and DeepStack features using the current chunk offset during chunked prefill.
• Validate the Qwen3-VL visual module outputs before inference.

Qwen3-VL-2B was exported with 4-bit symmetric weights, asymmetric 16-bit activations, OmniQuant using 10 calibration samples for 5 epochs, and a fixed visual input size of 272×608. The model was compiled for Snapdragon 8 Elite with soc_id=69, dsp_arch=v79, vtcm_mb=8, chunk_size=128, and max_history_token=0, producing 30 LLM binaries and one visual binary.

Reproduction commands:

MODEL_DIR=/path/to/Qwen3-VL-2B-Instruct
OUT_DIR=/path/to/qwen3-vl-2b-mnn
BUILD_DIR=/path/to/MNN/build_qnn
CALIB_DATA=/path/to/wikitext-10.txt

python transformers/llm/export/llmexport.py \
  --path "$MODEL_DIR" \
  --export mnn \
  --dst_path "$OUT_DIR" \
  --mnnconvert "$BUILD_DIR/tools/converter/MNNConvert" \
  --quant_bit 4 \
  --quant_block 64 \
  --lm_quant_bit 4 \
  --lm_quant_block 64 \
  --omni \
  --omni_epochs 5 \
  --calib_data "$CALIB_DATA" \
  --generate_for_npu \
  --seperate_embed \
  --sym \
  --act_bit 16

python transformers/llm/export/npu/generate_llm_qnn.py \
  --model "$OUT_DIR" \
  --soc_id 69 \
  --dsp_arch v79 \
  --vtcm_mb 8 \
  --mnn_path "$BUILD_DIR" \
  --chunk_size 128 \
  --max_history_token 0 \
  --image_sizes 272x608

The test prompt was:

<img><hw>608, 272</hw>/path/on/device/cat.jpg</img>Describe this image in one short sentence.

The four device configurations were run with the same image and prompt:

export LD_LIBRARY_PATH=/path/on/device/runtime/lib

/path/on/device/runtime/bin/llm_demo /path/on/device/model/config_qnn.json /path/on/device/cat_prompt.txt
/path/on/device/runtime/bin/llm_demo /path/on/device/model/config_cpu_llm_qnn_visual.json /path/on/device/cat_prompt.txt
/path/on/device/runtime/bin/llm_demo /path/on/device/model/config_qnn_cpu_visual.json /path/on/device/cat_prompt.txt
/path/on/device/runtime/bin/llm_demo /path/on/device/model/config.json /path/on/device/cat_prompt.txt

The same cat image and prompt were tested with four backend combinations on a OnePlus 13:

Visual backend	LLM backend	Generated text
QNN	QNN	A small, fluffy, light-brown baby goat stands on a pebble in front of a lush green background.
QNN	CPU	A small, orange and white kitten stands on a rock with its ears perked up, looking directly at the camera.
CPU	QNN	A cute, fluffy, young, gingerbread-colored, spotted...
CPU	CPU	A young orange and white kitten stands on a stone with its ears perked and wide eyes looking at the camera, surrounded by lush green grass.

All four combinations completed prefill and decode without graph-shape, graph-execution, or runtime errors. The QNN-visual/CPU-LLM result confirms that the QNN visual graph is working correctly. The remaining visual-language quality loss is localized to the quantized QNN LLM path; the current text-only calibration dataset does not cover visual embedding and DeepStack activation distributions.