-
Notifications
You must be signed in to change notification settings - Fork 4.4k
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
- Status: Open.#6540 In NVIDIA/Megatron-LM;
Offload MoE expert weights to pinned host memory to enable training at smaller EP
enhancementNew feature or requestNew feature or requestStatus: Open.#6491 In NVIDIA/Megatron-LM;- Status: Open.#6474 In NVIDIA/Megatron-LM;
[MFSDP V2][Design] Reduce transient communication-buffer peak from 4B to 3B
enhancementNew feature or requestNew feature or requestwaiting-on-customerWaiting on the original author to respondWaiting on the original author to respondStatus: Open.#6471 In NVIDIA/Megatron-LM;[ENHANCEMENT] Optional output normalization for latent MoE
enhancementNew feature or requestNew feature or requestwaiting-on-maintainersWaiting on maintainers to respondWaiting on maintainers to respondStatus: Open.#6448 In NVIDIA/Megatron-LM;- Status: Open.#6392 In NVIDIA/Megatron-LM;
- Status: Open.#6368 In NVIDIA/Megatron-LM;
- Status: Open.#6363 In NVIDIA/Megatron-LM;
- Status: Open.#6362 In NVIDIA/Megatron-LM;
- Status: Open.#6361 In NVIDIA/Megatron-LM;
- Status: Open.#6325 In NVIDIA/Megatron-LM;
[QUESTION]Megatron-FSDP ZeRO-2 (optim_grads) has higher peak memory than ZeRO-1/ZeRO-3 due to unsharded temporary main_grad bucket allocation
waiting-on-maintainersWaiting on maintainers to respondWaiting on maintainers to respondStatus: Open.#6314 In NVIDIA/Megatron-LM;