cookbook(action): add RoboCasa mobile-manipulation post-training recipe - #318
Open
hyzhou-nv wants to merge 2 commits into
Open
cookbook(action): add RoboCasa mobile-manipulation post-training recipe#318hyzhou-nv wants to merge 2 commits into
hyzhou-nv wants to merge 2 commits into
Conversation
Adds a complete RoboCasa recipe to the action finetune cookbook, following the DROID/LIBERO layout: dataset conversion, training launcher, closed-loop evaluation, and the Nano SFT TOML. RoboCasa publishes LeRobot v2.1 while the pinned lerobot is v3.0-only, so a one-time conversion step is included; it works on copies and is idempotent. Evaluation follows the official protocol -- a policy server per GPU with the simulator client in a separate venv, since robosuite and cosmos-framework cannot share one environment. Requires the matching cosmos-framework change that registers the RoboCasa dataset and embodiment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collaborator
|
Thanks! In general it LGTM, but can we keep this recipe focused on training? So files are organized in the same way of other sft recipes. Maybe we can move the eval/dataset related scripts to cosmos-framework and add the usage guidance in the readme? |
Per review feedback, the cookbook now ships training code only, matching the DROID and LIBERO recipes. The evaluator and its helpers move to cosmos-framework as cosmos_framework/simulation/robocasa/, alongside the LIBERO one; its module docstring carries the full invocation. The eval launcher is dropped, since neither DROID nor LIBERO has one. README keeps a short pointer covering what a user still needs to know from the cookbook side: where the evaluator lives, why two Python environments are required, which contract flags must match this recipe, and the gated guardrail download. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a complete RoboCasa recipe to the action finetune cookbook, following the DROID/LIBERO layout: dataset conversion, training launcher, closed-loop evaluation, and the Nano SFT TOML.
RoboCasa publishes LeRobot v2.1 while the pinned lerobot is v3.0-only, so a one-time conversion step is included; it works on copies and is idempotent.
Evaluation follows the official protocol -- a policy server per GPU with the simulator client in a separate venv, since robosuite and cosmos-framework cannot share one environment.
Requires the matching cosmos-framework change that registers the RoboCasa dataset and embodiment.