Inverse Reinforcement Learning (IRL) aims to reconstruct the reward function from expert demonstrations to facilitate policy learning, and has demonstrated its remarkable success in imitation learning. To promote expert-like behavior, existing IRL methods mainly focus on learning global reward functions to minimize the trajectory difference between the imitator and the expert. However, these global designs are still limited by the redundant noise and error propagation problems, leading to the unsuitable reward assignment and thus downgrading the agent capability in complex multi-stage tasks. In this paper, we propose a novel Curricular Subgoal-based Inverse Reinforcement Learning (CSIRL) framework, that explicitly disentangles one task with several local subgoals to guide agent imitation. Specifically, CSIRL firstly introduces decision uncertainty of the trained agent over expert trajectories to dynamically select specific states as subgoals, which directly determines the exploration boundary of different task stages. To further acquire local reward functions for each stage, we customize a meta-imitation objective based on these curricular subgoals to train an intrinsic reward generator. Experiments on the D4RL and autonomous driving benchmarks demonstrate that the proposed methods yields results superior to the state-of-the-art counterparts, as well as better interpretability. Our code is publicly available at https://github.com/Plankson/CSIRL.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Curricular Subgoals for Inverse Reinforcement Learning


    Beteiligte:
    Liu, Shunyu (Autor:in) / Qing, Yunpeng (Autor:in) / Xu, Shuqi (Autor:in) / Wu, Hongyan (Autor:in) / Zhang, Jiangtao (Autor:in) / Cong, Jingyuan (Autor:in) / Chen, Tianhao (Autor:in) / Liu, Yun-Fu (Autor:in) / Song, Mingli (Autor:in)


    Erscheinungsdatum :

    01.03.2025


    Format / Umfang :

    1929181 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Robot learning for moving between subgoals

    Morizane, K. / Yamada, S. / Toyoda, J. | British Library Online Contents | 1997


    A Novel Control Architecture for Autonomous Rover with Learning Subgoals in Attribute Space

    Yairi, T. / Wajima, H. / Nakasuka, S. et al. | British Library Conference Proceedings | 1996


    Automatic Logic Circuit Design with Progressive Evolution that Autonomously Gets Subgoals

    Matsuzaki, M. / Kawai, T. / Ando, H. et al. | British Library Online Contents | 1999


    Demo: Curricular Reinforcement Learning for Robust Policy in Unmanned CarRacing Game

    Tian, Yunzhe / Li, Yike / Xiang, Yingxiao et al. | TIBKAT | 2021


    Autonomous Progressive Evolution Using Subgoals as Common Properties in an Elite Population

    Matsuzaki, M. / Kawai, T. / Ando, H. et al. | British Library Online Contents | 2001