In this paper, we propose a new framework for collision-free intelligent vehicle navigation, aiming to successfully avoid obstacles using deep reinforcement learning. The navigation system separates perception and control and utilizes multimodal perception to achieve reliable online interaction with the surroundings. This allows for direct policy learning to generate flexible actions and avoid collisions. Our navigation system establishes a connection between the virtual environment and the real world, allowing learning policies in the virtual environment to be implemented in real-world environments through transfer learning. Our approach aims to integrate camera, Lidar, and Inertial Measurement Unit (IMU) data to construct a multimodal perception-based environment model, which is a state input for reinforcement learning. In this process, we utilize a series of cross-domain self-attention layers to enhance visual and Lidar perception, promoting significant improvements in perception. We also introduce the recurrent deduction to facilitate global decision-making based on local perception. Additionally, we introduce the Self-Assessment Gradient model into the reinforcement learning process to further optimize the learning policy. The experimental results demonstrate the proposed approach reduces the disparity between the virtual environment and the real world, highlighting its superiority over other state-of-the-art methods. Our project page is publicly available at https://github.com/CV4RA/MMAP-DRL-Nav.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multi-Modal Attention Perception for Intelligent Vehicle Navigation Using Deep Reinforcement Learning


    Contributors:
    Li, Zhenyu (author) / Shang, Tianyi (author) / Xu, Pengjie (author)


    Publication date :

    2025-06-01


    Size :

    4372715 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Evidential deep learning-based multi-modal environment perception for intelligent vehicles

    Geletu, Mihreteab Negash / Giurgi, Danut-Vasile / Josso-Laurain, Thomas et al. | IEEE | 2023


    Autonomous vehicle navigation with deep reinforcement learning

    Cabañeros López, Àlex | BASE | 2019

    Free access

    Automatic navigation using deep reinforcement learning

    BALAKRISHNAN KAUSHIK / NARAYANAN PRAVEEN / LAKEHAL-AYAT MOHSEN | European Patent Office | 2023

    Free access

    Automatic Navigation Using Deep Reinforcement Learning

    BALAKRISHNAN KAUSHIK / NARAYANAN PRAVEEN / LAKEHAL-AYAT MOHSEN | European Patent Office | 2019

    Free access

    Intelligent hybrid electric vehicle hierarchical control method based on visual perception and deep reinforcement learning

    TANG XIAOLIN / CHEN JIAXIN / DENG ZHONGWEI et al. | European Patent Office | 2021

    Free access