Add ModelOpt FP8 Cosmos Framework inference - #301
Conversation
|
Thx for the contribution. And is this fp8 inference functionality ready and verified in cosmos-framework? |
ce0cd0b to
3eb17c1
Compare
|
@KrzysztofTomala can you fill the description of this PR, what models does it enable? Cosmos3 Nano/Super? |
|
@qmiao-hub I have added the description @lfengad The fp8 ModelOpt inference path is ready in the imaginaire4 repo, hopefully will be added to cosmos-framework soon |
672923d to
66dc7ad
Compare
Signed-off-by: Krzysztof Tomala <ktomala@nvidia.com>
66dc7ad to
7cc502e
Compare
| "set -euo pipefail\n", | ||
| "\n", | ||
| "cd \"$COSMOS3_REPO\"\n", | ||
| "FP8_CHECKPOINT=\"$COSMOS3_REPO/checkpoints/Cosmos3-Nano-FP8\"\n", |
There was a problem hiding this comment.
We might need to add some comments annotation for how to chang to other FP8 released checkpoints?
There was a problem hiding this comment.
I have added the comment
| " --local-dir \"$FP8_CHECKPOINT\"\n", | ||
| "\n", | ||
| "COSMOS_TRAINING=false CUDA_VISIBLE_DEVICES=\"$CUDA_VISIBLE_DEVICES\" \\\n", | ||
| "MASTER_ADDR=\"$COSMOS3_MASTER_ADDR\" MASTER_PORT=\"$COSMOS3_NANO_TEXT_MASTER_PORT\" RANK=0 WORLD_SIZE=1 LOCAL_RANK=0 \\\n", |
There was a problem hiding this comment.
For the Super case, we might need also consider, or with some comments metion here?
Also, do we need to show the fsdp/multiple gpu case?
|
overall LGTM! @pengcuo could further see any suggestions given. Thx~ |
pengcuo
left a comment
pengcuo
left a comment
There was a problem hiding this comment.
I tested FP8-Nano by following this recipe, and the results are correct.
|
I have one question: How are FP8 models organized on Hugging Face, and how can they be downloaded? |
I have added more info in the notebook, the checkpoint will be released in same huggingface repo as originals, but as separate revision |
4 participants
The change describes how to run FP8 ModelOpt checkpoints using cosmos-framwork
It works for Cosmos3 Nano, Super, Super T2I, Super I2V, Super T2I 4steps, Super I2V 4steps