ROMB¶

Policy-driven world-model adaptation

OfflineData-centric · generativeBase · IQLClaims · dynamic shift

A world model adapted against the policy under a constrained maximin objective.

Features¶

Mechanism¶

ROMB adapts a learned world model against the policy under a constrained maximin objective. The model is pushed toward transitions that lower the policy's value while a supervised loss keeps it on the data, and the policy is trained on rollouts from the adapted model.

Run the method¶

python baselines/train.py \
    -c robustrllib/configs/experiment/romb_hopper.yaml --seed 0
python baselines/evaluate.py --run runs/romb_hopper/seed0

Training is on the nominal task. The configuration files are explained in Train an algorithm, and the evaluation of the frozen checkpoint in Evaluation Protocol.

Robust performance¶

Normalized score of the frozen checkpoint. Q1 to Q4 are the severity quartiles of each shift factor's scales, ordered by displacement from the nominal setting, and All covers every shifted condition. The last row averages the tasks.

References¶

• Chen et al. Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learning. arXiv:2505.13709, 2025.