All pull requests

Pull requests

• fixed args deepspeed_config for gpt pretraining

  #485·QuPengfei opened Jan 22, 2026·

  ·

• spelling fix: contiguous checkpointing was spelled contigious

  #484·dalongbao opened Nov 4, 2025·

  ·

• Fix AttributeError in hf2megads weight converter

  #483·KeitaW opened Sep 27, 2025·

  ·

• fix pretrain gpt and llama bug

  #476·9LLPPLL6 opened May 8, 2025·

  ·
  2

• FastPersist rebase

  #467·tjruwase opened Feb 25, 2025·

  ·

• support split qkv linear and sp overlap comm

  #415·inkcherry opened Jul 5, 2024·

  ·
  10

• fix --use-cpu-initialization error when expert is not tensor-parallel

  #413·taozhiwei opened Jul 3, 2024·

  ·
  3

• fix NAN loss of rope long context training

  #399·inkcherry opened Jun 5, 2024·

  ·
  1

• convert mds checkpoint to Hf Llama model

  #394·vksastry opened May 31, 2024·

  ·
  1

• ds-sequence-parallel(ulysses) for rope.

  #392·inkcherry opened May 30, 2024·

  ·
  1

• add HFTokenizer option for preprocess_data

  #388·Jianhong-Zhang opened May 17, 2024·

  ·
  1

• Add layer norm weight plus 1

  #378·Yejing-Lai opened Apr 18, 2024·

  ·
  4

• Fix ConstantGradScaler and loss-scale argument not match

  #376·BeingGod opened Apr 12, 2024·

  ·
  1

• Support Llama2Tokenizer

  #375·jinyouzhi opened Apr 11, 2024·

  ·

• collect grad_norm for non pipeline path

  #370·inkcherry opened Mar 21, 2024·

  ·

• optimize the generation of attention mask

  #331·imh966 opened Jan 13, 2024·

  ·

• Enable torch.compile

  #322·tohtana opened Dec 28, 2023·

  ·

• Simplify SP - Opportunity to improve SP scalability

  #301·RezaYazdaniAminabadi opened Nov 28, 2023·

  ·

• support transfer llama hf weight to megatron weight

  #246·uygnef opened Sep 12, 2023·

  ·
  27

• add vit training with TP/PP

  #146·etoilestar opened Jun 9, 2023·

  ·
  3

• attempt at pipelining

  #78·siddharth9820 opened Aug 18, 2022·

  ·

• Add support for DS comms

  #50·Quentin-Anthony opened Jun 13, 2022·

  ·
  1