RobustRLlib

Quick start

Evaluate your algorithm under a shifted environment.

RobustRLlib is installed from a source checkout and used from the root of the checkout. The tutorial quick start runs these steps with their outputs.

# Python 3.8 or newer; the documentation was run with 3.11
conda create -n robustrllib python=3.11 && conda activate robustrllib
pip install -e ".[mujoco]"
pip install -r requirements.txt
pip install pyyaml torch minari

from robustrllib import make_robust, ShiftSpec

# A deployment condition is an ordered list of shifts; the list order is the stacking order.
env = make_robust("Hopper-v5", shifts=[
    ShiftSpec("dynamics", "scale", {"param": "gravity", "factor": 1.3}),  # Dynamic shift
    ShiftSpec("observation", "gauss", {"sigma": 0.05}),                 # Observation shift
], seed=0)

obs, info = env.reset(seed=0)
obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
print(round(env.unwrapped.model.opt.gravity[2], 3))
-12.753

# One experiment card names an algorithm, a task and an evaluation grid.
python baselines/train.py -c robustrllib/configs/experiment/rorl_hopper.yaml --seed 0

# The frozen checkpoint is scored on every condition of the grid: 20 paired episodes per
# condition, the same episodes for every method, unclipped normalized score.
python baselines/evaluate.py --run runs/rorl_hopper/seed0

How it works

Learn more

Algorithms

Robust RL algorithm library construction.

The sixteen robust methods are grouped by data regime and by where robustness enters: the learner, the environment it collects from, or the data it learns from. Each row names the shift the method claims and the base algorithm it is built on.

Shifted-env rollout: rollouts collected from a deliberately perturbed environment. Adversarial network: a learned adversary optimised against the policy. Learned model: a trained next-state predictor used by the robustness mechanism. The six standard references (IQL, TD3+BC, MOPO, SynthER, PPO, SAC) are on the algorithm pages.

Shift toolbox

Six shift sources, five modes, six task families.

A shift has a source, which says what is perturbed, and a mode, which says how the intervention is built: stochastic, adversarial, parametric, non-stationary through a schedule, or a composition of several shifts. A shifted environment is a base task with one wrapper per shift; the task code is not modified.

from robustrllib import make_robust, ShiftSpec

shift = ShiftSpec("dynamics", "scale",
          {"param": "gravity", "factor": 1.3})

env = make_robust("Hopper-v5", shifts=[
    ShiftSpec("observation", "gauss", {"sigma": 0.10}),
], seed=0)

from robustrllib import make_robust, ShiftSpec

shift = ShiftSpec("action", "shift",
          {"shift": 0.05, "clip_first": True})

# train_shifts.yaml, arm reward_delay_64
ShiftSpec("latency", "delay", {"steps": 64, "release": "interval"})

python baselines/train.py -c robustrllib/configs/experiment/sac_hopper.yaml \
    --seed 0 --train-shift reward_delay_64

from robustrllib import make_robust, ShiftSpec

shift = ShiftSpec("latency", "fixed", {"steps": 2})

from robustrllib import make_robust, ShiftSpec

shift = ShiftSpec("appearance", "hue",
          {"geom": "torso_geom", "degrees": 120})

Mode names are the values a ShiftSpec accepts; schedules and compositions apply to every source. Each source has a tutorial page with its parameters.

Tasks

Six task families across three simulators and the Isaac Lab and VLA stacks. Each card shows a trained policy on the nominal task, the shift factors its evaluation grid perturbs, and the shift sources the toolbox offers on it.

Results

A capability map of robust RL algorithms.

Every number comes from the same protocol: the frozen checkpoint of each method, 20 paired episodes per condition, five training seeds, and the normalized score in which 0 is a random policy and 100 the reference policy, unclipped. Shift magnitudes are grouped into quartiles Q1–Q4 by displacement from the nominal setting.

Leaderboard

The same shift interface on Unitree G1 locomotion and Franka drawer opening in Isaac Lab with PPO, DR-PPO, ATLA-PPO and the robust safe methods RAMU-PPO and SPiDR-PPO, and on a vision–language–action policy.

Findings

• Robust RL helps at every magnitude, but both standard and robust methods degrade as shifts intensify.
• Dynamics-aware mechanisms are stronger under dynamics shifts; online robust methods gain most over their base algorithms.
• Claimed channels do not fully predict empirical strengths: a method's strongest shift source is often not the one it was designed for.
• Under compound shifts the dynamics factor costs the most on both tasks, while the next most damaging source differs by task.
• On the robotic tasks, robustness transfers return but not constraint satisfaction, and the semantic shift affects the VLA policy most.

Related benchmarks

Comparison with related benchmarks.

✓ supported · ◐ partial · ✗ absent. MDP shifts counts observation, action, transition and reward/cost. Robust algorithms counts the robust or robust safe methods a benchmark evaluates, without standard algorithms.

BibTeX

@inproceedings{robustrllib2027,
  title     = {RobustRLlib: A Unified Library and Benchmark for
               Robust Reinforcement Learning Algorithms},
  author    = {Anonymous Authors},
  booktitle = {Submitted to the International Conference on
               Learning Representations (ICLR)},
  year      = {2027},
  note      = {Under review}
}