Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/DeepSeekOCR2.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,6 @@ Run the following script to execute online inference.
#!/bin/sh

export VLLM_USE_V1=1
export VLLM_ASCEND_ENABLE_NZ=0
export TOKENIZERS_PARALLELISM=false
export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True"
export TASK_QUEUE_ENABLE=1
Expand All @@ -140,6 +139,7 @@ vllm serve /root/.cache/DeepSeek-OCR-2 \
--gpu-memory-utilization 0.8 \
--allowed-local-media-path / \
--additional-config '{
"weight_nz_mode": 0,
"enable_cpu_binding": true,
"multistream_overlap_shared_expert": true,
"ascend_compilation_config": {"fuse_qknorm_rope": false}
Expand Down
14 changes: 5 additions & 9 deletions docs/source/tutorials/models/GLM5.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -1293,7 +1293,6 @@ The 1M context scenarios are validated on Atlas 800 A3 only; the A2 series is no
Recommended command:

```shell
export VLLM_ASCEND_ENABLE_NZ=1
export HCCL_OP_EXPANSION_MODE="AIV"
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=20
Expand Down Expand Up @@ -1322,7 +1321,7 @@ vllm serve <MODEL_PATH> \
--decode-context-parallel-size 16 \
--cp-kv-cache-interleave-size 128 \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes": [4, 16, 128]}' \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true, "enable_sparse_li_c8": true, "enable_cpu_binding": true}' \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true, "enable_sparse_li_c8": true, "enable_cpu_binding": true,"weight_nz_mode":1}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
--quantization ascend \
--enable-expert-parallel \
Expand All @@ -1333,7 +1332,7 @@ Key Parameter Descriptions (in addition to [Single-Node Deployment](#5111-single

**1M-specific environment variables:**

- `VLLM_ASCEND_ENABLE_NZ=1`: Enables NZ format memory layout for the C8 quantized tensors, required for the 1M context deployment.
- `additional_config.weight_nz_mode=1`: Enables NZ format memory layout for the C8 quantized tensors, required for the 1M context deployment.
- `VLLM_WORKER_MULTIPROC_METHOD=spawn`: Uses the spawn start method for multi-process workers (required in this scenario).

**1M-specific vllm serve parameters:**
Expand All @@ -1359,7 +1358,6 @@ export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export VLLM_ASCEND_ENABLE_NZ=1
export HCCL_OP_EXPANSION_MODE="AIV"
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=20
Expand Down Expand Up @@ -1392,7 +1390,7 @@ vllm serve <MODEL_PATH> \
--decode-context-parallel-size 8 \
--cp-kv-cache-interleave-size 128 \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true}' \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true,"weight_nz_mode":1}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
--quantization ascend \
--enable-expert-parallel \
Expand Down Expand Up @@ -1425,7 +1423,6 @@ export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name

export VLLM_ASCEND_ENABLE_NZ=1
export HCCL_OP_EXPANSION_MODE="AIV"
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=20
Expand Down Expand Up @@ -1459,7 +1456,7 @@ vllm serve <MODEL_PATH> \
--decode-context-parallel-size 8 \
--cp-kv-cache-interleave-size 128 \
--enforce-eager \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": true}' \
--additional-config '{"enable_flashcomm1": true, "enable_dsa_cp": true, "ascend_compilation_config": {"enable_npugraph_ex": true}, "multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": true,"weight_nz_mode":1}' \
--speculative-config '{"num_speculative_tokens": 1, "method": "deepseek_mtp", "enforce_eager": true}' \
--quantization ascend \
--enable-expert-parallel \
Expand Down Expand Up @@ -1498,7 +1495,6 @@ export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export VLLM_ASCEND_ENABLE_NZ=1
export HCCL_OP_EXPANSION_MODE="AIV"
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=20
Expand Down Expand Up @@ -1532,7 +1528,7 @@ vllm serve <MODEL_PATH> \
--decode-context-parallel-size 8 \
--cp-kv-cache-interleave-size 128 \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
--additional-config '{"ascend_compilation_config": {"enable_npugraph_ex": true},"multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": true}' \
--additional-config '{"ascend_compilation_config": {"enable_npugraph_ex": true},"multistream_overlap_shared_expert": true,"enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": true,"weight_nz_mode":1}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
--quantization ascend \
--enable-expert-parallel \
Expand Down
6 changes: 3 additions & 3 deletions docs/source/user_guide/configuration/additional_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ The following table lists additional configuration options available in vLLM Asc
| `enable_flashcomm1` | bool | `False` | Whether to enable FlashComm1 optimization. Can also be configured via the `VLLM_ASCEND_ENABLE_FLASHCOMM1` environment variable during the migration period. |
| `msmonitor_use_daemon` | bool | `False` | Whether to use daemon mode for msmonitor. Can also be configured via the `MSMONITOR_USE_DAEMON` environment variable during the migration period. |
| `enable_mlapo` | bool | `True` | Whether to enable MLAPO (Model Layer-wise Adaptive Parallel Optimization). Can also be configured via the `VLLM_ASCEND_ENABLE_MLAPO` environment variable during the migration period. |
| `weight_nz_mode` | int | `1` | Weight NZ mode. Can also be configured via the `VLLM_ASCEND_ENABLE_NZ` environment variable during the migration period. |
| `weight_nz_mode` | int | `1` | Weight NZ mode. `0` disables NZ, `1` enables NZ only for quantized weights, and `2` also enables NZ for BF16/FP16 weights when supported. The legacy `VLLM_ASCEND_ENABLE_NZ` environment variable is no longer supported. |
| `enable_fused_mc2` | int | `0` | Fused MC2 configuration. Can also be configured via the `VLLM_ASCEND_ENABLE_FUSED_MC2` environment variable during the migration period. |
| `enable_transpose_kv_cache_by_block`| bool | `True` | Whether to enable transpose KV cache by block. Can also be configured via the `VLLM_ASCEND_FUSION_OP_TRANSPOSE_KV_CACHE_BY_BLOCK` environment variable during the migration period. |
| `enable_dsa_cp` | bool | `False` | Whether to enable dsa_cp for DeepSeek V3.2, DeepSeek V4, and other models with the same architecture. This feature depends on FlashComm1. Please ensure that FlashComm1 is enabled before enabling this feature.|
Expand Down Expand Up @@ -244,9 +244,9 @@ ShortRequestFirst is a waiting-queue policy for FCFS synchronous or asynchronous

**rl_config**

`rl_config` is a one-click RL mode switch. When `enabled` is `true`, it refreshes the global Ascend configuration on every initialization, forces `AscendConfig.weight_nz_mode=0`, synchronizes `VLLM_ASCEND_ENABLE_NZ=0`, sets `VLLM_SERVER_DEV_MODE=1`, and removes the `expandable_segments` entry from `PYTORCH_NPU_ALLOC_CONF` with an informational log. These fixed RL behaviors are not configurable as `rl_config` sub-fields. When `enabled` is `false`, all other sub-fields are ignored.
`rl_config` is a one-click RL mode switch. When `enabled` is `true`, it refreshes the global Ascend configuration on every initialization, forces `AscendConfig.weight_nz_mode=0`, sets `VLLM_SERVER_DEV_MODE=1`, and removes the `expandable_segments` entry from `PYTORCH_NPU_ALLOC_CONF` with an informational log. These fixed RL behaviors are not configurable as `rl_config` sub-fields. When `enabled` is `false`, all other sub-fields are ignored.

When RL mode is enabled, its fixed NZ and developer-endpoint settings take precedence over top-level configuration and environment variables. `VLLM_BATCH_INVARIANT=1` remains enabled when `rl_config.enable_batch_invariant` is false.
When RL mode is enabled, its fixed NZ setting takes precedence over the top-level `weight_nz_mode` configuration. `VLLM_BATCH_INVARIANT=1` remains enabled when `rl_config.enable_batch_invariant` is false.

| Name | Type | Default | Description |
| ---- | ---- | ------- | ----------- |
Expand Down
12 changes: 8 additions & 4 deletions docs/source/user_guide/feature_guide/sleep_mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@ The following is a simple example of how to use sleep mode.

os.environ["VLLM_USE_MODELSCOPE"] = "True"
os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
os.environ["VLLM_ASCEND_ENABLE_NZ"] = "0"

if __name__ == "__main__":
prompt = "How are you?"
Expand All @@ -109,7 +108,11 @@ The following is a simple example of how to use sleep mode.
print(f"Free memory before sleep: {free / 1024 ** 3:.2f} GiB")
# record npu memory use baseline in case other process is running
used_bytes_baseline = total - free
llm = LLM("Qwen/Qwen2.5-0.5B-Instruct", enable_sleep_mode=True)
llm = LLM(
"Qwen/Qwen2.5-0.5B-Instruct",
enable_sleep_mode=True,
additional_config={"weight_nz_mode": 0},
)
sampling_params = SamplingParams(temperature=0, max_tokens=10)
output = llm.generate(prompt, sampling_params)

Expand Down Expand Up @@ -137,9 +140,10 @@ The following is a simple example of how to use sleep mode.
export VLLM_SERVER_DEV_MODE="1"
export VLLM_WORKER_MULTIPROC_METHOD="spawn"
export VLLM_USE_MODELSCOPE="True"
export VLLM_ASCEND_ENABLE_NZ="0"

vllm serve Qwen/Qwen2.5-0.5B-Instruct --enable-sleep-mode
vllm serve Qwen/Qwen2.5-0.5B-Instruct \
--enable-sleep-mode \
--additional-config='{"weight_nz_mode": 0}'

# after serving is up, post to these endpoints.
# /sleep reads level from the query string (JSON body is ignored).
Expand Down
4 changes: 1 addition & 3 deletions examples/rl/rlhf_async_new_apis.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,9 +190,6 @@ def generate(self, token_ids: list[int], max_new_tokens: int) -> list[int]:
"VLLM_BATCH_INVARIANT": "1",
"HCCL_DETERMINISTIC": "strict",
"LCCL_DETERMINISTIC": "1",
# Disable FRACTAL_NZ mode (also handled by batch invariance override_envs)
"VLLM_ASCEND_ENABLE_NZ": "0",
"VLLM_ASCEND_ENABLE_MATMUL_ALLREDUCE": "0",
# Enable expandable segments for PyTorch NPU allocator
"PYTORCH_NPU_ALLOC_CONF": "expandable_segments:True",
}
Expand All @@ -210,6 +207,7 @@ def generate(self, token_ids: list[int], max_new_tokens: int) -> list[int]:
max_model_len=8192,
distributed_executor_backend="ray",
gpu_memory_utilization=0.75,
additional_config={"weight_nz_mode": 0},
weight_transfer_config=WeightTransferConfig(backend="hccl"),
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ test_cases:
- name: "DeepSeek-V3.2-W8A8-DCP-replicated-indexer"
model: "vllm-ascend/DeepSeek-V3.2-W8A8"
envs:
VLLM_ASCEND_ENABLE_NZ: "1"
HCCL_OP_EXPANSION_MODE: "AIV"
OMP_PROC_BIND: "false"
OMP_NUM_THREADS: "20"
Expand Down Expand Up @@ -63,7 +62,7 @@ test_cases:
- "--compilation-config"
- '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes":[4, 16, 64, 128]}'
- "--additional-config"
- '{"enable_dsa_cp": true, "ascend_compilation_config":{"enable_npugraph_ex": true, "enable_static_kernel": false}, "multistream_overlap_shared_expert": true, "enable_mc2_hierarchy_comm": false, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": false}'
- '{"enable_dsa_cp": true, "ascend_compilation_config":{"enable_npugraph_ex": true, "enable_static_kernel": false}, "multistream_overlap_shared_expert": true, "enable_mc2_hierarchy_comm": false, "enable_sparse_sfa_c8": true, "enable_sparse_li_c8": true, "enable_cpu_binding": true, "recompute_scheduler_enable": false,"weight_nz_mode":1}'
- "--speculative-config"
- '{"num_speculative_tokens": 3, "method": "deepseek_mtp"}'
test_content: []
Expand Down
3 changes: 1 addition & 2 deletions tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ _envs: &envs
TASK_QUEUE_ENABLE: "1"
VLLM_ASCEND_ENABLE_MLAPO: "1"
VLLM_ASCEND_ENABLE_FLASHCOMM1: "1"
VLLM_ASCEND_ENABLE_NZ: "1"

_server_cmd: &server_cmd
- "--enable-expert-parallel"
Expand Down Expand Up @@ -44,7 +43,7 @@ _server_cmd: &server_cmd
- "--speculative-config"
- '{"method":"eagle3", "model":"lightseekorg/kimi-k2.5-eagle3", "num_speculative_tokens":3}'
- "--additional-config"
- '{"enable_shared_expert_dp":true}'
- '{"enable_shared_expert_dp":true,"weight_nz_mode":1}'
- "--mm-processor-cache-gb"
- "0"
- "--mm-encoder-tp-mode"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ test_cases:
OMP_NUM_THREADS: "1"
TASK_QUEUE_ENABLE: "1"
VLLM_ASCEND_ENABLE_FUSED_MC2: "1"
VLLM_ASCEND_ENABLE_NZ: "1"
VLLM_TORCH_PROFILER_WITH_STACK: "0"
VLLM_TORCH_PROFILER_DIR: "./profile"
VLLM_USE_MODELSCOPE: "true"
Expand Down Expand Up @@ -52,7 +51,7 @@ test_cases:
- "--compilation-config"
- '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
- "--additional-config"
- '{"enable_cpu_binding": true, "enable_npugraph_ex": true, "enable_static_kernel": true}'
- '{"enable_cpu_binding": true, "enable_npugraph_ex": true, "enable_static_kernel": true,"weight_nz_mode":1}'
benchmarks:
acc:
case_type: accuracy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ _envs: &envs
PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True"
VLLM_ASCEND_ENABLE_FLASHCOMM1: "1"
VLLM_ASCEND_ENABLE_FUSED_MC2: "1"
VLLM_ASCEND_ENABLE_NZ: "2"
VLLM_ASCEND_BALANCE_SCHEDULING: "1"
SERVER_PORT: "DEFAULT_PORT"

Expand Down Expand Up @@ -66,5 +65,7 @@ test_cases:
server_cmd_extra:
- "--compilation_config"
- '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes": [1,2,4,8,16,24,32]}'
- --additional-config
- '{"weight_nz_mode":2}'
benchmarks:
<<: *benchmarks
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ test_cases:
OMP_NUM_THREADS: "1"
TASK_QUEUE_ENABLE: "1"
VLLM_ASCEND_ENABLE_DENSE_OPTIMIZE: "1"
VLLM_ASCEND_ENABLE_NZ: "1"
VLLM_ASCEND_ENABLE_FUSED_MC2: "1"
SERVER_PORT: "DEFAULT_PORT"
server_cmd:
Expand All @@ -33,7 +32,7 @@ test_cases:
- "--gpu-memory-utilization"
- "0.95"
- "--additional-config"
- '{"enable_cpu_binding":true, "enable_weight_nz_layout":true}'
- '{"enable_cpu_binding":true, "enable_weight_nz_layout":true,"weight_nz_mode":1}'
- "--speculative_config"
- '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3,"enforce_eager": true}'
- "--compilation-config"
Expand Down
3 changes: 2 additions & 1 deletion tests/e2e/pull_request/one_card/rlhf/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,6 @@ def server(
env = {
**os.environ,
"VLLM_SERVER_DEV_MODE": "1",
"VLLM_ASCEND_ENABLE_NZ": "0",
"HF_HUB_OFFLINE": "1",
}
base = _DUMMY_ARGS if dummy_weights else _BASE_ARGS
Expand All @@ -107,6 +106,8 @@ def server(
str(port),
"--served-model-name",
"m",
"--additional-config",
'{"weight_nz_mode": 0}',
*(base + (extra_args or [])),
]
# Establish the test process's NPU device context while the card is still
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
--enable-sleep-mode KV cache allocated via CuMemAllocator; without this
flag sleep/wake are no-ops and the bug cannot trigger.
VLLM_SERVER_DEV_MODE=1
VLLM_ASCEND_ENABLE_NZ=0
--additional-config '{"weight_nz_mode": 0}'
"""

import requests
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ def test_npu_ipc_weight_transfer_updates_server_weights():
"--port",
str(port),
"--trust-remote-code",
"--additional-config",
'{"weight_nz_mode": 0}',
]
# VLLM_SERVER_DEV_MODE registers the dev endpoints; insecure serialization
# lets the server unpickle the IPC handles sent over HTTP. Pin the server to
Expand All @@ -105,7 +107,6 @@ def test_npu_ipc_weight_transfer_updates_server_weights():
"VLLM_SERVER_DEV_MODE": "1",
"VLLM_ALLOW_INSECURE_SERIALIZATION": "1",
"ASCEND_RT_VISIBLE_DEVICES": str(INFERENCE_DEVICE_INDEX),
"VLLM_ASCEND_ENABLE_NZ": "0",
}

with RemoteOpenAIServer(
Expand Down
14 changes: 8 additions & 6 deletions tests/e2e/pull_request/one_card/test_xlite.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,11 @@

# ruff: noqa: E501

import os

import pytest

from tests.e2e.conftest import wait_until_npu_memory_free
from tests.e2e.pull_request.utils import PROMPTS_SHORT, compare_logprobs

os.environ["VLLM_ASCEND_ENABLE_NZ"] = "2"

MODELS: list[str] = ["Qwen/Qwen3-0.6B"]


Expand All @@ -52,7 +48,10 @@ def test_models_with_xlite_decode_only(model: str):
"model_name": model,
"max_model_len": 1024,
"block_size": 128,
"additional_config": {"xlite_graph_config": {"enabled": True, "full_mode": False}},
"additional_config": {
"weight_nz_mode": 2,
"xlite_graph_config": {"enabled": True, "full_mode": False},
},
}
compare_logprobs(runner_kwargs=runner_kwargs, prompts=PROMPTS_SHORT)

Expand All @@ -74,6 +73,9 @@ def test_models_with_xlite_full_mode(model: str):
"model_name": model,
"max_model_len": 1024,
"block_size": 128,
"additional_config": {"xlite_graph_config": {"enabled": True, "full_mode": True}},
"additional_config": {
"weight_nz_mode": 2,
"xlite_graph_config": {"enabled": True, "full_mode": True},
},
}
compare_logprobs(runner_kwargs=runner_kwargs, prompts=PROMPTS_SHORT)
3 changes: 2 additions & 1 deletion tests/e2e/pull_request/two_card/test_hccl_weight_transfer.py
Original file line number Diff line number Diff line change
Expand Up @@ -188,14 +188,15 @@ def test_hccl_weight_transfer_updates_server_weights():
"--port",
str(port),
"--trust-remote-code",
"--additional-config",
'{"weight_nz_mode": 0}',
]
# The dev-mode endpoints (/init_weight_transfer_engine, /update_weights,
# /pause, /resume, ...) are only registered when VLLM_SERVER_DEV_MODE=1.
# Pin the server to NPU 0 so the trainer can own NPU 1 exclusively.
env_dict = {
"VLLM_SERVER_DEV_MODE": "1",
"ASCEND_RT_VISIBLE_DEVICES": "0",
"VLLM_ASCEND_ENABLE_NZ": "0",
}

_log(f"starting server on port {port} (device 0, dummy weights) ...")
Expand Down
12 changes: 8 additions & 4 deletions tests/e2e/pull_request/two_card/test_xlite.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@
from tests.e2e.conftest import DPVllmRunner, VllmRunner, wait_until_npu_memory_free
from tests.e2e.pull_request.utils import PROMPTS_SHORT

os.environ["VLLM_ASCEND_ENABLE_NZ"] = "2"

MODELS: list[str] = ["Qwen/Qwen3-30B-A3B"]
TPDP_SIZES: list[tuple[int, int]] = [(2, 1), (1, 2)]

Expand Down Expand Up @@ -58,7 +56,10 @@ def test_models_with_xlite_decode_only(model: str, tpdp: tuple[int, int]):
enable_expert_parallel=True,
block_size=128,
max_model_len=2048,
additional_config={"xlite_graph_config": {"enabled": True, "full_mode": False}},
additional_config={
"weight_nz_mode": 2,
"xlite_graph_config": {"enabled": True, "full_mode": False},
},
) as vllm_model:
outputs = vllm_model.generate_greedy(PROMPTS_SHORT, 3)

Expand Down Expand Up @@ -89,7 +90,10 @@ def test_models_with_xlite_full_mode(model: str, tpdp: tuple[int, int]):
enable_expert_parallel=True,
block_size=128,
max_model_len=2048,
additional_config={"xlite_graph_config": {"enabled": True, "full_mode": True}},
additional_config={
"weight_nz_mode": 2,
"xlite_graph_config": {"enabled": True, "full_mode": True},
},
) as vllm_model:
outputs = vllm_model.generate_greedy(PROMPTS_SHORT, 3)

Expand Down
1 change: 0 additions & 1 deletion tests/e2e/weekly/single_node/models/test_qwen3_30b_acc.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ async def test_models(model: str, tp_size: int) -> None:
"HCCL_BUFFSIZE": "1024",
"OMP_NUM_THREADS": "1",
"PYTORCH_NPU_ALLOC_CONF": "expandable_segments:True",
"VLLM_ASCEND_ENABLE_NZ": "2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The environment variable VLLM_ASCEND_ENABLE_NZ was set to "2" in this test, but it has been removed without migrating the configuration to --additional-config '{"weight_nz_mode": 2}' in server_args. This will cause the test to run with the default weight_nz_mode of 1, which might lead to accuracy degradation or test failures for the Qwen3-30B-A3B-W8A8 model. Please ensure "weight_nz_mode": 2 is added to the --additional-config argument in server_args.

"MOONCAKE_CONFIG_PATH": "mooncake.json",
}
if tp_size != 1:
Expand Down
Loading
Loading