RSC¶

Robust RL against Spurious Correlation

OnlineLearner-centricBase · SACClaims · semantic shift

Causal counterfactual replay that removes correlations a policy would exploit.

Features¶

Mechanism¶

RSC learns a causal graph and a dynamics model over the state variables from replayed experience, then generates counterfactual transitions by intervening on the variables the graph marks as non-causal for the reward. Mixing these into replay removes the spurious correlations a policy would otherwise exploit.

Run the method¶

python baselines/train.py \
    -c robustrllib/configs/experiment/rsc_hopper.yaml --seed 0
python baselines/evaluate.py --run runs/rsc_hopper/seed0

Training is on the nominal task. The configuration files are explained in Train an algorithm, and the evaluation of the frozen checkpoint in Evaluation Protocol.

Robust performance¶

Normalized score of the frozen checkpoint. Q1 to Q4 are the severity quartiles of each shift factor's scales, ordered by displacement from the nominal setting, and All covers every shifted condition. The last row averages the tasks.

References¶

• Ding et al. Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation. NeurIPS, 2023.