Skip to content

docs: add DeepSeek V3 verification card - #5583

Merged
yaoyu-33 merged 1 commit into
mainfrom
yuya/deepseek-v3-verification-card
Aug 15, 2026
Merged

docs: add DeepSeek V3 verification card#5583
yaoyu-33 merged 1 commit into
mainfrom
yuya/deepseek-v3-verification-card

Conversation

@yaoyu-33

@yaoyu-33 yaoyu-33 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Adds a DeepSeek V3 model verification card using the existing canonical H100, GB200, and GB300 performance recipes.

Changelog

  • Add the required model verification inventory.
  • Record verified H100 BF16 and GB200/GB300 MXFP8 performance results, including tokens per second per GPU.
  • Keep model recipes and attention-mask behavior unchanged.

Validation

  • Model verification card validator
  • pre-commit run --all-files
  • Complete 50-step nemo-ci artifacts with finite metrics, saved resolved configs, and zero skipped or NaN iterations

Before your PR is "Ready for review"

  • Followed the contributor guidelines
  • Added the model verification card
  • No optional-install component changes

@yaoyu-33
yaoyu-33 requested review from a team, dingqingy-nv and malay-nagda as code owners August 14, 2026 16:38
@copy-pr-bot

copy-pr-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33
yaoyu-33 marked this pull request as draft August 14, 2026 16:45
@yaoyu-33
yaoyu-33 force-pushed the yuya/deepseek-v3-verification-card branch from c723de5 to ec9d540 Compare August 14, 2026 16:51
@yaoyu-33 yaoyu-33 changed the title feat(perf): add DeepSeek V3 GB300 verification docs: add DeepSeek V3 verification card Aug 14, 2026
@yaoyu-33
yaoyu-33 force-pushed the yuya/deepseek-v3-verification-card branch from ec9d540 to 90d7744 Compare August 14, 2026 17:01
@claude

claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Light review — DeepSeek V3 GB300 agentic perf recipes + verification card

Scope: two new GB300 MXFP8 agentic repro recipes (PP2/EP32 and PP4/EP32), a shared _apply_agentic_mcore_gb300_mxfp8_settings helper, threading create_attention_mask through the mock-dataset config, a parametrized unit test, and a new DeepSeek-V3 model verification card.

Findings:

  1. Possible phantom attribute: cfg.model.fp8_dot_product_attention (gb300/deepseek_v3.py:239, 256). Everywhere else in the tree this flag is set only on cfg.mixed_precision; mixed_precision.apply() propagates it to the model config only when the model already declares that field (hasattr guard). If fp8_dot_product_attention is not a real DeepSeek model-provider field, the cfg.model. assignments are silent no-ops and the test assertion cfg.model.fp8_dot_product_attention is expected_fp8_attention merely reads back what the recipe wrote, passing without verifying anything. fp8_output_proj is a genuine model field, so that one is fine. Please confirm; if needed, keep only the cfg.mixed_precision. assignment and assert on that. (inline comment left)

  2. New recipes not registered in scripts/performance/dump_perf_configs.py. The large_scale, hsdp, fsdp DeepSeek variants are listed there for config-dump/regression coverage, but the two new agentic_pp2/agentic_pp4 variants are not. If that list is meant to be exhaustive for regression diffing, consider adding the two combos; otherwise ignore.

  3. create_attention_mask default = True in create_mock_dataset_config. This preserves prior behavior for the two existing callers (they pass through recipe.dataset.create_attention_mask), and run_recipe.py now also forwards it. Looks correct, no action needed; just confirm the default matches MCore GPTDatasetConfig so mock runs are unchanged for recipes that do not opt out.

  4. OCI-AGA in docstrings (lines 34, 231, 248). OCI already appears in existing DeepSeek docstrings/READMEs, so this is likely an acceptable public cloud reference rather than a private cluster name. Flagging only so an owner can confirm AGA is not an internal codename per the no-private-run-info card rule.

No typos or logic errors found in the card; it validates structurally against the create-model-verification-card format (single verified pretrain_performance.GB300 leaf, all other items unverified, scoped summary, index synchronized with items).

Suggested test cases:

  • tests/unit_tests/recipes/test_r050_perf_recipe_ports.py::test_deepseek_v3_gb300_agentic_repro_configs (both parametrizations: deepseek_v3_pretrain_256gpu_gb300_fp8mx_agentic_pp2_config, deepseek_v3_pretrain_256gpu_gb300_fp8mx_agentic_pp4_config)
  • tests/functional_tests/test_groups/recipes/test_perf_config_integration.py::TestPerfConfigIntegration::test_deepseek_v3_flat_perf_config_instantiation
  • tests/functional_tests/test_groups/recipes/test_perf_config_integration.py::TestPerfConfigIntegration::test_list_available_config_variants_keeps_suffixless_first (guards variant discovery; verify the new gb300/fp8_mx agentic_pp2/agentic_pp4 suffixes are discoverable and do not perturb the h100/fp8_sc assertion)
  • Card validation: uv run --no-project --with pyyaml python skills/create-model-verification-card/scripts/validate_card.py examples/model_verification_cards/deepseek-v3/card.yaml

Note: no scripts/performance/configs/ tree exists in this repo and no perf/functional launcher references these recipes by name, so the <model>_<gpu>_<precision>_perf glob-mapping does not apply here.

@yaoyu-33
yaoyu-33 force-pushed the yuya/deepseek-v3-verification-card branch from 90d7744 to 25ad419 Compare August 14, 2026 18:14
@yaoyu-33
yaoyu-33 marked this pull request as ready for review August 14, 2026 18:14
@claude

claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

LGTM — new DeepSeek V3 model verification card. Verified against the actual recipe:

  • Topology matches deepseek_v3_pretrain_256gpu_gb300_fp8mx_config(): TP1/PP2/VP8/CP1/EP32/ETP1, GBS/MBS 4096/1.
  • Benchmark-only caveats accurate: recipe sets moe_router_force_load_balancing = True and reduced-precision optimizer moments (exp_avg_dtype/exp_avg_sq_dtype = bfloat16), matching the card disclaimer.
  • Metrics consistent with stated gates: 10556.110 ms <= 11000 ms and 1614.140 >= 1600 TFLOP/s/GPU; loss 11.895 -> 7.014 (finite, decreasing); 64 nodes x 4 = 256 GPUs.
  • verification_index synchronized with items; summary correctly scopes the verified claim to the concrete pretrain_performance.GB300 leaf.

Note: I could not run the bundled validate_card.py here — please confirm it passes locally before merge.

Suggested test cases (card is a YAML artifact; no perf/recipe configs touched):

  • No perf tests impacted.
  • tests/unit_tests/scripts/test_validate_model_verification_card.py
  • tests/unit_tests/skills/create_model_verification_card/test_validate_card.py

@yaoyu-33
yaoyu-33 force-pushed the yuya/deepseek-v3-verification-card branch 2 times, most recently from 67918e4 to 913112e Compare August 14, 2026 18:34
@yaoyu-33 yaoyu-33 added area:model Model implementations and HF bridge logic docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer labels Aug 14, 2026
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33
yaoyu-33 force-pushed the yuya/deepseek-v3-verification-card branch from 913112e to 2c0da4b Compare August 15, 2026 16:53
@yaoyu-33
yaoyu-33 merged commit 6d9a87d into main Aug 15, 2026
4 checks passed
@yaoyu-33
yaoyu-33 deleted the yuya/deepseek-v3-verification-card branch August 15, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:model Model implementations and HF bridge logic docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant