Add a Mojo GPU port (CUDA + Metal, Mojo 1.0) to notable forks - #858
Open
ulmentflam wants to merge 1 commit into
Open
Add a Mojo GPU port (CUDA + Metal, Mojo 1.0) to notable forks#858ulmentflam wants to merge 1 commit into
ulmentflam wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds one line under the existing Mojo entry in notable forks, following
the pattern of #332.
llm.🔥 is my port of llm.c that
extends @dorjeduck's already-listed Mojo port (Mojo 25.5, CPU) with hand-written
GPU training kernels for Mojo 1.0.0b3: a CUDA/cuBLAS backend, an Apple Metal
backend, and a portable GPU path. What it adds beyond the existing entry: on a
GB10 (DGX Spark) the bf16 path is at parity with llm.c's CUDA trainer (135.97
vs 135.77 ms/step median, B=4, T=1024, GPT-2 124M), and on an M4 Max the Metal
path trains 1.72x faster than PyTorch MPS bf16 (llm.c has no Metal port, so
MPS is the baseline there). Correctness is gated by a ported
test_gpt2.mojo:16 gradient tensors plus a 10-step loss trajectory checked against PyTorch.
Benchmark methodology and reproduction: https://github.com/ulmentflam/llm.mojo#benchmarks
dorjeduck's entry is kept as is; his port is the lineage this builds on.