Skip to content

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SCoCaT: Success-Conditioned Constrained RL for Spacecraft Docking

Code for the paper "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking" (Submitted CoRL 2026).

NOTE: Code during rebuttal updated with 6DoF docking velocity constraint bug-fix and PID Lagrangian code.

Two components are included:

Requirements

Installation

The recommended setup uses Docker. Isaac Sim is large and its Python environment is isolated inside the container; the skrl/ fork is mounted from this repo and installed automatically at container startup.

# 1. Log in to the NVIDIA container registry:
docker login nvcr.io

# 2. Build the Isaac Lab image (run from isaaclab/docker/):
cd isaaclab/docker
python3 container.py start base

# 3. Enter the container:
python3 container.py enter base

The container installs the skrl/ fork (mounted from ../../skrl) into the Isaac Lab Python environment.

All training commands below are run inside the container, with working directory /workspace/isaaclab.

Training logs and checkpoints are written to isaaclab/logs/ on the host (bind-mounted into the container at /workspace/isaaclab/logs/).

Assets: robot and environment USD files are included in isaaclab/assets/ and are baked into the Docker image at build time. No git-lfs or separate asset download is required.

Training — single policy

Two platforms are supported. Replace the environment overrides accordingly:

Platform robot_name task_name decimation
Floating Platform (3-DoF) FloatingPlatform GoToPose 6

PPO (vanilla, unconstrained)

# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
    --headless --num_envs 4096 \
    --task Isaac-RANS-Single-v0 \
    --algorithm ppo-discrete \
    --seed 42 \
    env.robot_name=FloatingPlatform \
    env.task_name=GoToPose \
    env.decimation=6

CaT (stochastic terminations on constraint violations, no success critic)

# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
    --headless --num_envs 4096 \
    --task Isaac-RANS-Single-cstr-v0 \
    --algorithm ppo-discrete-cat \
    --seed 42 \
    env.robot_name=FloatingPlatform \
    env.task_name=GoToPose \
    env.decimation=6 \
    env.use_constraints=cat \
    env.use_bernoulli_cat_termination=False \
    env.use_cat_curriculum=False

SCoCaT (separate success critic + normalized advantage sum)

# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
    --headless --num_envs 4096 \
    --task Isaac-RANS-Single-cat-sb-v0 \
    --algorithm ppo-discrete-cat-sb \
    --seed 42 \
    env.robot_name=FloatingPlatform \
    env.task_name=GoToPose \
    env.decimation=6 \
    env.use_constraints=cat \
    env.use_bernoulli_cat_termination=False \
    env.use_cat_curriculum=False

SCoCaT-PEGrad (PEGrad-style gradient projection)

SCoCaT-PEGrad projects the reward policy gradient onto the orthogonal complement of the success gradient, treating success as the primary objective. Implemented in PPO_CAT_PEGRAD (skrl/skrl/agents/torch/ppo/ppo_cat_pegrad.py).

# Floating Platform
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py \
    --headless --num_envs 4096 \
    --task Isaac-RANS-Single-cstr-pegrad-v0 \
    --algorithm ppo-discrete-cat-pegrad \
    --seed 42 \
    env.robot_name=FloatingPlatform \
    env.task_name=GoToPose \
    env.decimation=6 \
    env.use_constraints=cat \
    env.use_bernoulli_cat_termination=False \
    env.use_cat_curriculum=False

Logs and checkpoints are written to logs/skrl/ inside the isaaclab/ directory.

Reward x success-signal ablation

The ablation sweeps across algorithm variants (PPO, CaT no-curriculum, CaT with curriculum, SCoCaT, SCoCaT with only reward bonus) across three reward shapes (dense, terminal-proximity, truly sparse) and two success-signal types (binary, continuous).

# Full run (inside the container, from /workspace/isaaclab)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py

# Dry run (print commands without executing)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py --dry_run

# Resume from a specific index (skip completed runs)
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py --skip 60

# Run a single condition
./isaaclab.sh -p scripts/reinforcement_learning/skrl/run_reward_success_ablation_v2.py \
    --conditions scocat_success_critic_dense_continuous

Condition names follow the pattern {method}_{reward}_{signal}, e.g. cat_curr_dense_binary. See the script header for the full list and descriptions of each axis.

Algorithm implementations

Algorithm Class File
CaT PPO_CAT (bootstrap_success=False) skrl/skrl/agents/torch/ppo/ppo_cat.py
SCoCaT PPO_CAT (bootstrap_success=True, separate_success_critic=True) skrl/skrl/agents/torch/ppo/ppo_cat.py
SCoCaT-PEGrad PPO_CAT_PEGRAD skrl/skrl/agents/torch/ppo/ppo_cat_pegrad.py

The agent class and hyperparameters are selected via the YAML config in isaaclab/source/isaaclab_tasks/isaaclab_tasks/rans/environments/single/agents/.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages