Code for the paper "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking" (Submitted CoRL 2026).
NOTE: Code during rebuttal updated with 6DoF docking velocity constraint bug-fix and PID Lagrangian code.
Two components are included:
isaaclab/— Isaac Lab fork with the environments, constrained training wrappers, and training/evaluation scriptsskrl/— skrl (https://skrl.readthedocs.io/en/latest/) fork
- NVIDIA GPU with CUDA support
- Docker with the NVIDIA Container Toolkit
The recommended setup uses Docker. Isaac Sim is large and its Python environment is isolated inside the container; the skrl/ fork is mounted from this repo and installed automatically at container startup.
# 1. Log in to the NVIDIA container registry:
docker login nvcr.io
# 2. Build the Isaac Lab image (run from isaaclab/docker/):
cd isaaclab/docker
python3 container.py start base
# 3. Enter the container:
python3 container.py enter baseThe container installs the skrl/ fork (mounted from ../../skrl) into the Isaac Lab Python environment.
All training commands below are run inside the container, with working directory /workspace/isaaclab.
Training logs and checkpoints are written to isaaclab/logs/ on the host (bind-mounted into the container at /workspace/isaaclab/logs/).
Assets: robot and environment USD files are included in isaaclab/assets/ and are baked into the Docker image at build time. No git-lfs or separate asset download is required.
Two platforms are supported. Replace the environment overrides accordingly:
| Platform | robot_name |
task_name |
decimation |
|---|---|---|---|
| Floating Platform (3-DoF) | FloatingPlatform |
GoToPose |
6 |
# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
--headless --num_envs 4096 \
--task Isaac-RANS-Single-v0 \
--algorithm ppo-discrete \
--seed 42 \
env.robot_name=FloatingPlatform \
env.task_name=GoToPose \
env.decimation=6
# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
--headless --num_envs 4096 \
--task Isaac-RANS-Single-cstr-v0 \
--algorithm ppo-discrete-cat \
--seed 42 \
env.robot_name=FloatingPlatform \
env.task_name=GoToPose \
env.decimation=6 \
env.use_constraints=cat \
env.use_bernoulli_cat_termination=False \
env.use_cat_curriculum=False
# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
--headless --num_envs 4096 \
--task Isaac-RANS-Single-cat-sb-v0 \
--algorithm ppo-discrete-cat-sb \
--seed 42 \
env.robot_name=FloatingPlatform \
env.task_name=GoToPose \
env.decimation=6 \
env.use_constraints=cat \
env.use_bernoulli_cat_termination=False \
env.use_cat_curriculum=False
SCoCaT-PEGrad projects the reward policy gradient onto the orthogonal complement of the success gradient, treating success as the primary objective. Implemented in PPO_CAT_PEGRAD (skrl/skrl/agents/torch/ppo/ppo_cat_pegrad.py).
# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
--headless --num_envs 4096 \
--task Isaac-RANS-Single-cstr-pegrad-v0 \
--algorithm ppo-discrete-cat-pegrad \
--seed 42 \
env.robot_name=FloatingPlatform \
env.task_name=GoToPose \
env.decimation=6 \
env.use_constraints=cat \
env.use_bernoulli_cat_termination=False \
env.use_cat_curriculum=False
Logs and checkpoints are written to logs/skrl/ inside the isaaclab/ directory.
The ablation sweeps across algorithm variants (PPO, CaT no-curriculum, CaT with curriculum, SCoCaT, SCoCaT with only reward bonus) across three reward shapes (dense, terminal-proximity, truly sparse) and two success-signal types (binary, continuous).
# Full run (inside the container, from /workspace/isaaclab)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py
# Dry run (print commands without executing)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py --dry_run
# Resume from a specific index (skip completed runs)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py --skip 60
# Run a single condition
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py \
--conditions scocat_success_critic_dense_continuousCondition names follow the pattern {method}_{reward}_{signal}, e.g. cat_curr_dense_binary. See the script header for the full list and descriptions of each axis.
| Algorithm | Class | File |
|---|---|---|
| CaT | PPO_CAT (bootstrap_success=False) |
skrl/skrl/agents/torch/ppo/ppo_cat.py |
| SCoCaT | PPO_CAT (bootstrap_success=True, separate_success_critic=True) |
skrl/skrl/agents/torch/ppo/ppo_cat.py |
| SCoCaT-PEGrad | PPO_CAT_PEGRAD |
skrl/skrl/agents/torch/ppo/ppo_cat_pegrad.py |
The agent class and hyperparameters are selected via the YAML config in isaaclab/source/isaaclab_tasks/isaaclab_tasks/rans/environments/single/agents/.