Inverse Reinforcement Learning (IRL) aims to reconstruct the reward function from expert demonstrations to facilitate policy learning, and has demonstrated its remarkable success in imitation learning. To promote expert-like behavior, existing IRL methods mainly focus on learning global reward functions to minimize the trajectory difference between the imitator and the expert. However, these global designs are still limited by the redundant noise and error propagation problems, leading to the unsuitable reward assignment and thus downgrading the agent capability in complex multi-stage tasks. In this paper, we propose a novel Curricular Subgoal-based Inverse Reinforcement Learning (CSIRL) framework, that explicitly disentangles one task with several local subgoals to guide agent imitation. Specifically, CSIRL firstly introduces decision uncertainty of the trained agent over expert trajectories to dynamically select specific states as subgoals, which directly determines the exploration boundary of different task stages. To further acquire local reward functions for each stage, we customize a meta-imitation objective based on these curricular subgoals to train an intrinsic reward generator. Experiments on the D4RL and autonomous driving benchmarks demonstrate that the proposed methods yields results superior to the state-of-the-art counterparts, as well as better interpretability. Our code is publicly available at https://github.com/Plankson/CSIRL.
Curricular Subgoals for Inverse Reinforcement Learning
IEEE Transactions on Intelligent Transportation Systems ; 26 , 3 ; 3016-3027
01.03.2025
1929181 byte
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Robot learning for moving between subgoals
British Library Online Contents | 1997
|A Novel Control Architecture for Autonomous Rover with Learning Subgoals in Attribute Space
British Library Conference Proceedings | 1996
|Automatic Logic Circuit Design with Progressive Evolution that Autonomously Gets Subgoals
British Library Online Contents | 1999
|Autonomous Progressive Evolution Using Subgoals as Common Properties in an Elite Population
British Library Online Contents | 2001
|