The main challenge for future autonomous vehicles is to identify their location and body pose in real time during driving, that is, “where am I? and how will I go?”. We address the problems of pose estimation and scene perception from continuous visual frames in intelligent vehicle. Recent advanced technology in the domain of deep learning proposes to train some learning models for vehicle’s series detection tasks in a supervised or unsupervised manner, which has numerous advances over traditional approaches, mainly reflected in the absence of manual calibration and synchronization of the camera and IMU. In the paper, we propose a novel approach for pose estimation and scene recognition with a deep fusion of multi-modal neural features in the manner of unsupervised. Firstly, low-cost camera and IMU are used to extract original visual and inertial data, then the visual and inertial encoders are utilized to encoder the feature of the two modes. Then, a Long Short-Term Memory (LSTM) takes in the combined feature representation (visual and inertial), and outputs the pose information of intelligent vehicle through the next three fully connectional neural layers. Further, we also propose to train a slight-weight convolutional neural network (CNN) with only five convolutional modules (13 convolutional neural layers) for the representation of the salient features in driving scene, by comparison with the scenes in database, to identify the location of vehicle in a special scene. All of the above processes are carried out in an end-to-end fashion. Lastly, we evaluate the proposed method on some driving datasets, e.g. KITTI and VPRICE, and the results show the proposed approach is able to improve the level of autonomy of intelligent vehicle greatly.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multi-Modal Neural Feature Fusion for Pose Estimation and Scene Perception of Intelligent Vehicle


    Weitere Titelangaben:

    Sae Technical Papers


    Beteiligte:
    Zhou, Aiguo (Autor:in) / Pu, Jiakun (Autor:in) / Yu, Jiangyang (Autor:in) / Wei, Ronghui (Autor:in) / Dong, Zhenbiao (Autor:in) / Li, Zhenyu (Autor:in)

    Kongress:

    SAE WCX Digital Summit ; 2021



    Erscheinungsdatum :

    2021-04-06




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Print


    Sprache :

    Englisch




    Multi-Modal Neural Feature Fusion for Pose Estimation and Scene Perception of Intelligent Vehicle

    Zhou, Aiguo / Li, Zhenyu / Pu, Jiakun et al. | British Library Conference Proceedings | 2021


    Complementary Multi–Modal Sensor Fusion for Resilient Robot Pose Estimation in Subterranean Environments

    Khattak, Shehryar / Nguyen, Huan / Mascarich, Frank et al. | IEEE | 2020



    Pose estimation method for unmanned aerial vehicle based on point line and plane feature fusion

    LU RENQUAN / TAO JIE / PENG HUI et al. | Europäisches Patentamt | 2024

    Freier Zugriff