[ruijieguo]

What this changes

Qwen3.8-Flash-Next needs its GR residual streams, PLE external N-gram table, QSA attention and hybrid GDN layers preserved throughout training, checkpointing and rollout. This PR adds the model adapter and the ROLL/runtime paths for frozen-table backbone training with CPU optimizer offload, LoRA SFT/RL/OPD, save/resume, and teacher/reference handling.

Numerical fixes included: native GR FP32 FMA and BF16 materialization with the padded GEMM geometry; the router keeps the checkpoint's model-dtype projection unless FP32/FP64 is requested explicitly; SFT honors explicit step limits; Flash-Next LoRA workers use a single K partition for the dense Triton LoRA shrink, which removes nondeterministic FP32 atomic accumulation (scoped to this model type; it does not enable global batch invariance).

Rebase onto upstream v0.4.0

The branch now contains alibaba/ROLL v0.4.0 (581046a) as a merge, so it no longer conflicts with main. 16 files conflicted; roll/third_party/megatron/offload_states_patch.py was re-expressed on upstream's new backend-based offload store.

Re-running the real 8-GPU paths on the merged tree found regressions that unit tests did not. All are fixed in this branch:

• Grouped expert LoRA rank. Upstream dropped r // moe_router_topk; restored, so expert adapters are rank 6 for r=64 and topk=10 as this branch's checkpoints and export expect. The 2-rank grouped-LoRA test went from 4 failed to 6 passed.
• moe_permute_fusion default. The new TrainingArguments default False overrode the model config's True and moved the frozen-reference sentinel by up to 0.35 nats across data-parallel ranks. It now defaults to None; an explicit value still wins.
• CPU-master offload. Parked parameters keep full-shaped zero-storage views, Hybrid fp16 shards are rebound, and checkpoint loading hands the offload backend to the leaf optimizers.
• Rollout dumps. NonTensorStack columns made every dump file empty under the now-default TransferQueue backend (reproduces on pristine v0.4.0). Fixed.
• Upstream defect, fixed in 4725516. roll/pipeline/rlvr/rewards/__init__.py eagerly imports Geo3kRewardWorker, which needs mathruler, declared in no requirements file, so RLVR jobs abort in an offline environment. Replaced by a PEP 562 lazy export with tests/pipeline/test_rewards_lazy_exports.py.
• vLLM expert-parallel dev builds are judged by capability instead of the release tuple; SFTWorker honors auto_resume; stale fixtures updated.

Upstream defaults that now apply to Flash-Next runs: use_sequence_packing is True by default and Flash-Next Megatron roles reject packed sequences, so the configs set it to false; transfer_backend defaults to TransferQueue; pure_opd_pipeline_type resolves to "rlvr" in pure-OPD mode. Test failures shared with pristine v0.4.0 (10 of 20 upstream offload-state variants, multimodal beam search) or with the pre-merge branch (hybrid-Adam padding) are not from this merge.

Validation on the merged tree (8 x H800, 67c67e8)

The head 4725516 differs from 67c67e8 only in test files and the lazy-export fix above.

Job	Result	Key evidence
LoRA SFT, 6 updates, EP8, 8K	passed	run.exit=0, all seven checks; train loss 2.40 → 1.78; adapter ranks 64 / 6 as before
SFT cold resume from checkpoint-3	bit-exact	failure_count: 0 over 408,806 tensor leaves and 297,936 optimizer items
GRPO, 20 updates, LoRA student, seed 70	passed	driver and independent verifier exit 0; 20 updates on each of 8 ranks; reference and backbone unchanged across 21 versions; 344 of 344 N-gram table checks
LoRA OPD, 20 updates, rank-64 student	passed	20 updates on each of 8 ranks, 52 effective prompt groups, reference and frozen backbone unchanged across 21 versions; teacher is the rebuilt opd-teacher-v3, not the one of the earlier verified run
Full-backbone RL, 20 updates	passed on a clean host	run.exit=0; independent verifier exit 0: 8 ranks × 20 updates, 46 prompt groups, frozen reference unchanged
Full-backbone OPD, 20 updates	12 of 20, then the job's 6 h timeout (exit 124, not memory)	cleared the same step-1 boundary; teacher probe unchanged over 13 versions, student changed every version, 112 of 112 N-gram checks; steps 1 onward ran about 3.8× slower than the pre-merge pass for a reason not yet determined. Not a 20-update pass
Two-node LoRA SFT, two hosts over InfiniBand	ran	4+4 GPUs: 6 of 6 updates twice, bit-identical between runs; 8+8 GPUs: 6 of 6. The driver exits 1 only because its checkpoint inspection cannot see the other node's RNG files; the merged checkpoint passes the validator's contract
HF adapter export of SFT checkpoint-5 (CPU)	matches the earlier export	148,808 tensors; no reload or inference on this export
Env-gated Megatron CUDA suites	passed	41 + 12 cases, plus the grouped expert LoRA test on each rank
Eight-rank suites	16 passed / 2 failed	the examined failure depends on the convolution-patched Megatron checkout, not on this code

Notes:

• Full-backbone RL and OPD were first killed at the start of step 1 on the original validation host by Ray's memory monitor (0.97 of 2015.5 GiB). The same RL job passed on a second, idle host, and worker-side memory traces of all four runs agree on total consumption to within 1%, so the kills come from host memory headroom (Ray processes left by earlier sessions and other tenants), not from the merge. On a 2 TiB host this job needs at least about 1,810 GiB of MemAvailable at launch.
• GRPO uses seed 70 because with seed 42 one step drew only responses that hit the 128-token cap; the pipeline's existing skip guard skipped that update and the driver's end-of-run check, which expects an update every step, failed. Only the validation driver's seed changed; validator and verifier are untouched.
• The two-node runs are not bit-identical to the 2026-10-01 single-node run (lora_B relative L2 about 0.15), but fresh single-node controls on both hosts differ from it by the same amount, so the two-node split is not the cause. The remaining candidate is the container network and InfiniBand device difference. This is an environment finding, not a code change.
• These runs write no checkpoint and show lifecycle behavior only, not convergence, numerical parity or checkpoint acceptance.

Pre-merge validation (September 27, H800): 58 GR tests including exact native BF16 forward comparisons and FP64 derivative checks; router tests against real Megatron gates; the LoRA SFT save and restore lifecycle (checkpoint-3 → updates 4–5 → checkpoint-5 at e6de032); 2-update full-backbone RL and OPD probes on all 8 ranks; LoRA adapter export with exact native reload and sleep/wake on 8 EP ranks after the shrink fix (before it, identical requests drifted by up to 2.94); a TP2/EP8 capacity run at 72.05 GiB per GPU; and 16 deterministic-shrink CUDA regressions at 37e5756. Lifecycle evidence, not convergence or numerical acceptance.

Remaining acceptance gates

• A full 20-update full-backbone OPD pass on the merged tree.
• Full-model native probability parity still fails at atol=rtol=0.02. Fixing native prefill chunks to 2,048 tokens gives a stable native reference, but actor/native parity still fails; the tolerance has not been relaxed.
• Checkpoint and save/resume numerical parity for full-backbone training, and export acceptance (the historical continuation comparison reported 864 FP32 alignment-gap differences).
• Workload-level throughput of the deterministic LoRA shrink; the kernel microbenchmark is 1.01 to 2.10× the native selection, geometric mean 1.22.
• External N-gram validation does not hash every payload byte, so referenced model assets must stay immutable during a run; N-gram tables remain frozen in this scope.

Per-step timings, memory traces and verifier receipts are kept in a separate validation log.

🤖 Generated with Claude Code