This article presents a technique for trajectory planning based on parameterized high-level actions. These high-level actions are subtrajectories that have variable shape and duration. The use of high-level actions can improve the performance of guidance algorithms. Specifically, we show how the use of high-level actions improves the performance of guidance policies that are generated via reinforcement learning (RL). RL has shown great promise for solving complex control, guidance, and coordination problems but can still suffer from long training times and poor performance. This work shows how the use of high-level actions reduces the required number of training steps and increases the path performance of an RL-trained guidance policy. We demonstrate the method on a space-shuttle guidance example. We show the proposed method increases the path performance (latitude range) by 18% compared with a baseline RL implementation. Similarly, we show the proposed method achieves steady state during training with approximately 75% fewer training steps. We also show how the guidance policy enables effective performance in an obstacle field. Finally, this article develops a loss function term for policy-gradient-based deep RL, which is analogous to an antiwindup mechanism in feedback control. We demonstrate that the inclusion of this term in the underlying optimization increases the average policy return in our numerical example.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Trajectory Planning With Deep Reinforcement Learning in High-Level Action Spaces


    Contributors:


    Publication date :

    2023-06-01


    Size :

    2470233 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Crossroad lighthouse trajectory planning method based on deep reinforcement learning

    LI BOQI / ZHANG WEIWEI / CHEN JIANG et al. | European Patent Office | 2024

    Free access

    3-D Autonomous Entry Trajectory Planning via Hybrid Action Reinforcement Learning

    Peng, Gaoxiang / Wang, Bo / Liu, Lei et al. | IEEE | 2025


    Trajectory Planning for Automated Parking Systems Using Deep Reinforcement Learning

    Du, Zhuo / Miao, Qiheng / Zong, Changfu | Springer Verlag | 2020


    Deep Reinforcement Learning-Based 3D Trajectory Planning for Cellular Connected UAV

    Xiang Liu / Weizhi Zhong / Xin Wang et al. | DOAJ | 2024

    Free access

    Trajectory Planning of UAV in Unknown Dynamic Environment with Deep Reinforcement Learning

    Wang, Jia / Wang, Weihong / Wu, Qian | Springer Verlag | 2019