Hierarchical reinforcement learning (HRL) incorporates temporal abstraction into reinforcement learning (RL) by explicitly taking advantage of hierarchical structures. Modern HRL typically designs a hierarchical agent composed of a high-level policy and low-level policies. The high-level policy selects which low-level policy to activate at a lower frequency and the activated low-level policy selects an action at each time step. Recent HRL algorithms have achieved performance gains over standard RL algorithms in synthetic navigation tasks. However, these HRL algorithms still cannot be applied to real-world navigation tasks. One of the main challenges is that real-world navigation tasks require an agent to perform safe and interactive behaviors in dynamic environments. In this paper, we propose imagination-augmented HRL (IAHRL) that efficiently integrates imagination into HRL to enable an agent to learn safe and interactive behaviors in real-world navigation tasks. Imagination is to predict the consequences of actions without interactions with actual environments. The key idea behind IAHRL is that the low-level policies imagine safe and structured behaviors, and then the high-level policy infers interactions with surrounding objects by interpreting the imagined behaviors. We also introduce a new attention mechanism that allows the high-level policy to be permutation-invariant to the order of surrounding objects and to prioritize our agent over them. To evaluate IAHRL, we introduce five complex urban driving tasks, which are among the most challenging real-world navigation tasks. The experimental results indicate that IAHRL enables an agent to perform safe and interactive behaviors, achieving higher success rates and lower average episode steps than baselines.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Imagination-Augmented Hierarchical Reinforcement Learning for Safe and Interactive Autonomous Driving in Urban Environments


    Contributors:

    Published in:

    Publication date :

    2024-12-01


    Size :

    1917743 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Safe Reinforcement Learning for Urban Driving using Invariably Safe Braking Sets

    Krasowski, Hanna / Zhang, Yinqiang / Althoff, Matthias | IEEE | 2022


    Safe Reinforcement Learning with Policy-Guided Planning for Autonomous Driving

    Rong, Jikun / Luan, Nan | British Library Conference Proceedings | 2020



    DEEP HIERARCHICAL REINFORCEMENT LEARNING FOR AUTONOMOUS DRIVING WITH DISTINCT BEHAVIORS

    Chen, Jianyu / Wang, Zining / Tomizuka, Masayoshi | British Library Conference Proceedings | 2018