syangx39 · Oct 26, 2026 · Oct 26, 2026
diff --git a/README.md b/README.md
@@ -183,6 +183,8 @@ Models             | GPU Machine Type
 | **Reflection Beam 501B** | [A4X MAX (NVIDIA GB300)](https://cloud.google.com/compute/docs/accelerator-optimized-machines#a4x-vms) | Dynamo + SGLang | Aggregated Inference | GKE | [Link](./inference/a4xmax/aggregated-serving/dynamo/README.md) |
 | **Reflection Beam 501B** | [A4X MAX (NVIDIA GB300)](https://cloud.google.com/compute/docs/accelerator-optimized-machines#a4x-vms) | Dynamo + SGLang | Disaggregated Inference | GKE | [Link](./inference/a4xmax/disaggregated-serving/dynamo/README.md) |
 | **DeepSeek R1 671B** | [A4X MAX (NVIDIA GB300)](https://cloud.google.com/compute/docs/accelerator-optimized-machines#a4x-vms) | Dynamo + SGLang | Disaggregated Inference | GKE | [Link](./inference/a4xmax/disaggregated-serving/dynamo/README.md) |
+| **DeepSeek V4 Pro (MXFP4 + MTP)** | [A4X MAX (NVIDIA GB300)](https://cloud.google.com/compute/docs/accelerator-optimized-machines#a4x-vms) | Dynamo + SGLang | Disaggregated Inference (InferenceX Pareto sweep) | GKE | [Link](./inference/a4xmax/disaggregated-serving/dynamo/pareto-sweep/README.md) |
+| **DeepSeek R1 671B (NVFP4 / FP8)** | [A4X MAX (NVIDIA GB300)](https://cloud.google.com/compute/docs/accelerator-optimized-machines#a4x-vms) | Dynamo + SGLang | Disaggregated Inference (InferenceX Pareto sweep) | GKE | [Link](./inference/a4xmax/disaggregated-serving/dynamo/pareto-sweep/README.md) |
 
 
 ### Inference benchmarks G4

diff --git a/inference/a4xmax/disaggregated-serving/dynamo/README.md b/inference/a4xmax/disaggregated-serving/dynamo/README.md
@@ -20,6 +20,7 @@ Dynamo provides a disaggregated architecture that separates prefill and decode o
   * [3.1. Supported Models and Deployment Categories](#supported-models)
   * [3.2. Category 1: High Interactivity / Low Latency Deployment](#sglang-high-interactivity)
   * [3.3. Category 2: High Throughput Multi-Node Deployment](#sglang-high-throughput)
+  * [3.4. Category 3: InferenceX Pareto-Sweep Operating Points (DeepSeek-V4-Pro, DeepSeek-R1)](#sglang-pareto-sweep)
 * [4. Inference Request and Benchmarking](#inference-request)
   * [4.1. Verify Deployment Readiness](#verify-readiness)
   * [4.2. Deploy the Benchmark Client](#deploy-bench-client)
@@ -325,6 +326,17 @@ helm install -f ${VALUES_FILE} \
   $REPO_ROOT/src/helm-charts/a4xmax/inference-templates/dynamo-deployment
 ```
 
+<a name="sglang-pareto-sweep"></a>
+### 3.4. Category 3: InferenceX Pareto-Sweep Operating Points (DeepSeek-V4-Pro, DeepSeek-R1)
+
+The [`pareto-sweep/`](./pareto-sweep/README.md) sub-recipe packages every operating point of the public InferenceX GB300 Pareto curves for **DeepSeek-V4-Pro (MXFP4 + EAGLE MTP)**, **DeepSeek-R1 (NVFP4)** and **DeepSeek-R1 (FP8)** as a values file plus a prefill/decode serving-config pair for this chart. Each point is labelled by its position on the curve (low latency, mid-curve, max throughput) and comes with its topology, GPU count and benchmark concurrencies, so you can deploy a specific point from 2 to 18 nodes and reproduce it with the InferenceX 8k/1k methodology.
+
+| Model | Points | Topologies | Recipe |
+| :--- | :--- | :--- | :--- |
+| **DeepSeek-V4-Pro (MXFP4 + MTP)** | p1 to p8 | 1P1D TP4 (8 GPUs) up to 8P1D DEP4/DEP8 (40 GPUs) | [pareto-sweep/README.md#points-dsv4](./pareto-sweep/README.md#points-dsv4) |
+| **DeepSeek-R1 671B (NVFP4)** | low_latency, mid_curve, max_tpt | 1P4D TP4 (20 GPUs), 6P1D TP4/DEP48 and 10P1D TP4/DEP32 (72 GPUs) | [pareto-sweep/README.md#points-dsr1-fp4](./pareto-sweep/README.md#points-dsr1-fp4) |
+| **DeepSeek-R1 671B (FP8)** | low-latency, mid, max | 1P1D TP4 (8 GPUs), 5P1D DEP8/DEP32 and 6P1D DEP8/DEP24 (72 GPUs) | [pareto-sweep/README.md#points-dsr1-fp8](./pareto-sweep/README.md#points-dsr1-fp8) |
+
 <a name="inference-request"></a>
 ## 4. Inference Request and Benchmarking
 

diff --git a/inference/a4xmax/disaggregated-serving/dynamo/pareto-sweep/README.md b/inference/a4xmax/disaggregated-serving/dynamo/pareto-sweep/README.md
diff --git a/...ence/a4xmax/disaggregated-serving/dynamo/pareto-sweep/deepseek-r1-fp8/README.md b/...ence/a4xmax/disaggregated-serving/dynamo/pareto-sweep/deepseek-r1-fp8/README.md
@@ -0,0 +1,11 @@
+# DeepSeek-R1 (FP8): Values Files
+
+Helm values files for the DeepSeek-R1 (FP8) operating points on A4X-MAX. Each file sets the topology and worker environment of one point on the Pareto curve.
+
+Deployment steps, point details and the Pareto map are in the [pareto-sweep recipe](../README.md#points-dsr1-fp8).
+
+| Point | Pareto position | Values file | Serving configs |
+| :--- | :--- | :--- | :--- |
+| **low-latency** | Low latency | [`dsr1_fp8_low_latency.yaml`](dsr1_fp8_low_latency.yaml) | [prefill](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-low-latency-1p1d-tp4-tp4-prefill.yaml), [decode](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-low-latency-1p1d-tp4-tp4-decode.yaml) |
+| **mid** | Mid-curve | [`dsr1_fp8_mid_curve.yaml`](dsr1_fp8_mid_curve.yaml) | [prefill](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-mid-curve-5p1d-dep8-dep32-prefill.yaml), [decode](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-mid-curve-5p1d-dep8-dep32-decode.yaml) |
+| **max** | Max throughput | [`dsr1_fp8_max_throughput.yaml`](dsr1_fp8_max_throughput.yaml) | [prefill](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-max-throughput-6p1d-dep8-dep24-prefill.yaml), [decode](../../../../../../src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-max-throughput-6p1d-dep8-dep24-decode.yaml) |
diff --git a/...4xmax/disaggregated-serving/dynamo/pareto-sweep/deepseek-r1-fp8/dsr1_fp8_low_latency.yaml b/...4xmax/disaggregated-serving/dynamo/pareto-sweep/deepseek-r1-fp8/dsr1_fp8_low_latency.yaml
@@ -0,0 +1,221 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# DeepSeek-R1-0528 FP8 — operating point `ll` (Low latency)
+# Topology: 1 prefill worker(s) x 1 node(s) + 1 decode worker(s) x 1 node(s)
+#           = 2 nodes / 8 GB300 GPUs inside one NVL72 ComputeDomain
+# Source recipe: NVIDIA srt-slurm `gb300-fp8/8k1k/stp/low-latency`
+# Pair with: src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-low-latency-1p1d-tp4-tp4-prefill.yaml
+#            src/frameworks/a4xmax/dynamo-configs/dsr1-fp8-low-latency-1p1d-tp4-tp4-decode.yaml
+
+dynamo:
+  namespace: dynamo-cloud
+  releaseVersion: "1.0.1"
+  deploymentName: dsr1fp8-ll
+  computeDomain:
+    name: dsr1fp8-ll-domain
+    numNodes: 2
+    resourceClaimTemplateName: dsr1fp8-ll-compute-domain-channel
+  # KV transfer rides the NVL72 fabric (MNNVL); no RDMA NIC resource claim is requested.
+  serviceAccountName:
+  frontend:
+    image: nvcr.io/nvidia/ai-dynamo/dynamo-frontend:1.0.1
+    replicas: 10
+    livenessProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 150
+      failureThreshold: 100
+    readinessProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 300
+      failureThreshold: 100
+  decodeWorker:
+    nodeCount: 1
+    replicas: 1
+    envs:
+    - name: HF_TOKEN
+      valueFrom:
+        secretKeyRef:
+          name: hf-token-secret
+          key: HF_TOKEN
+    - name: HF_HUB_ENABLE_HF_TRANSFER
+      value: "1"
+    - name: LD_LIBRARY_PATH
+      value: "/usr/local/ucx/lib:/usr/local/ucx/lib/ucx:/opt/nvidia/nvda_nixl/lib/aarch64-linux-gnu:/opt/nvidia/nvda_nixl/lib/aarch64-linux-gnu/plugins:/usr/local/nvidia/lib64"
+    - name: PYTHONUNBUFFERED
+      value: "1"
+    - name: GLOO_SOCKET_IFNAME
+      value: "eth0"
+    - name: TP_SOCKET_IFNAME
+      value: "eth0"
+    - name: UCX_TLS
+      value: "cuda_copy,cuda_ipc,tcp"
+    - name: UCX_CUDA_IPC_ENABLE_MNNVL
+      value: "y"
+    - name: UCX_MEMTYPE_CACHE
+      value: "n"
+    - name: UCX_MEMTYPE_REG_WHOLE
+      value: "n"
+    - name: MC_FORCE_MNNVL
+      value: "1"
+    - name: MC_TE_METRIC
+      value: "true"
+    - name: NCCL_MNNVL_ENABLE
+      value: "1"
+    - name: NCCL_CUMEM_ENABLE
+      value: "1"
+    - name: SGLANG_MOONCAKE_CUSTOM_MEM_POOL
+      value: "True"
+    - name: SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE
+      value: "100000"
+    - name: SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT
+      value: "100000"
+    - name: SGLANG_DISAGGREGATION_WAITING_TIMEOUT
+      value: "100000"
+    - name: TORCH_DISTRIBUTED_DEFAULT_TIMEOUT
+      value: "1800"
+    - name: DYN_SKIP_SGLANG_LOG_FORMATTING
+      value: "1"
+    - name: SGLANG_USE_MESSAGE_QUEUE_BROADCASTER
+      value: "0"
+    - name: SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK
+      value: "1"
+    - name: SGLANG_DECODE_BOOTSTRAP_TIMEOUT
+      value: "1000"
+    - name: SGLANG_ENABLE_JIT_DEEPGEMM
+      value: "true"
+    livenessProbe:
+      initialDelaySeconds: 600
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 60
+    readinessProbe:
+      initialDelaySeconds: 60
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 60
+    startupProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 1800
+  prefillWorker:
+    nodeCount: 1
+    replicas: 1
+    envs:
+    - name: HF_TOKEN
+      valueFrom:
+        secretKeyRef:
+          name: hf-token-secret
+          key: HF_TOKEN
+    - name: HF_HUB_ENABLE_HF_TRANSFER
+      value: "1"
+    - name: LD_LIBRARY_PATH
+      value: "/usr/local/ucx/lib:/usr/local/ucx/lib/ucx:/opt/nvidia/nvda_nixl/lib/aarch64-linux-gnu:/opt/nvidia/nvda_nixl/lib/aarch64-linux-gnu/plugins:/usr/local/nvidia/lib64"
+    - name: PYTHONUNBUFFERED
+      value: "1"
+    - name: GLOO_SOCKET_IFNAME
+      value: "eth0"
+    - name: TP_SOCKET_IFNAME
+      value: "eth0"
+    - name: UCX_TLS
+      value: "cuda_copy,cuda_ipc,tcp"
+    - name: UCX_CUDA_IPC_ENABLE_MNNVL
+      value: "y"
+    - name: UCX_MEMTYPE_CACHE
+      value: "n"
+    - name: UCX_MEMTYPE_REG_WHOLE
+      value: "n"
+    - name: MC_FORCE_MNNVL
+      value: "1"
+    - name: MC_TE_METRIC
+      value: "true"
+    - name: NCCL_MNNVL_ENABLE
+      value: "1"
+    - name: NCCL_CUMEM_ENABLE
+      value: "1"
+    - name: SGLANG_MOONCAKE_CUSTOM_MEM_POOL
+      value: "True"
+    - name: SGLANG_DISAGGREGATION_HEARTBEAT_MAX_FAILURE
+      value: "100000"
+    - name: SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT
+      value: "100000"
+    - name: SGLANG_DISAGGREGATION_WAITING_TIMEOUT
+      value: "100000"
+    - name: TORCH_DISTRIBUTED_DEFAULT_TIMEOUT
+      value: "1800"
+    - name: DYN_SKIP_SGLANG_LOG_FORMATTING
+      value: "1"
+    - name: SGLANG_USE_MESSAGE_QUEUE_BROADCASTER
+      value: "0"
+    - name: SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK
+      value: "1"
+    - name: SGLANG_ENABLE_JIT_DEEPGEMM
+      value: "true"
+    livenessProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 60
+    readinessProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 60
+    startupProbe:
+      initialDelaySeconds: 3000
+      periodSeconds: 60
+      timeoutSeconds: 30
+      failureThreshold: 1800
+
+secrets:
+  ngc:
+    secretName: nvcr-secret
+  huggingface:
+    secretName: hf-token-secret
+    secretData:
+      token: "hf_api_token"
+
+volumes:
+  useGcs: true
+  gcsfuse:
+    bucketName:
+  ssdMountPath: "/ssd"
+  gcsMounts:
+    mountPath: "/data/model"
+
+service:
+  type: ClusterIP
+  ports:
+    frontend: 8000
+    worker: 9090
+
+workload:
+  model: deepseek-ai/DeepSeek-R1-0528
+  image:
+  framework: sglang
+  configFile: serving-args.yaml
+  configPath: /workload/configs
+
+network:
+  subnetworks: []
+  gibVersion: us-docker.pkg.dev/gce-ai-infra/gpudirect-gib/nccl-plugin-gib-diagnostic-arm64:v1.0.7
+  ncclSettings:
+    - name: NCCL_DEBUG
+      value: "VERSION"
+
+quantizations:
+  - "fp8"