Robust scene perception is an essential prerequisite to ensure the reliability in ship autonomous driving. However, it is a challenging task in inland river because of the complicated and changeable environment as well as high-density ships in narrow waterway. As one of the primary technologies, obstacle trajectory locating and tracking has been widely explored in recent years. Current approaches strictly rely on lidar as only depth awareness sensor and the limited measurement range severely restricts them for distant object identification. On this account, we creatively propose a point cloud-light field fusion perception framework in this paper for the first time. Specifically, in detection stage, the former undertakes precise close object perception and the latter completes distant object locating through light field stereo matching. In tracking stage, a novel four-phase data association that combines multiple attributes from position, point cloud and image domains is utilized for accurate object matching across frames. To validate the effectiveness of our multimodal perception strategy, we implement an acquisition system consisting of two lidars and four sets of simplified light field cameras on a ship to conduct actual testing. Extensive experimental results show that the proposed framework achieves superior 3D object locating and tracking performance, far surpassing the state-of-the-art methods in terms of accuracy and real-time.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multimodal Perception Integrating Point Cloud and Light Field for Ship Autonomous Driving


    Contributors:
    Cong, Ruixuan (author) / Sheng, Hao (author) / Zhao, Mingyuan (author) / Yang, Dazhi (author) / Wang, Tun (author) / Chen, Rongshan (author) / Shen, Jiahao (author)


    Publication date :

    2024-09-01


    Size :

    1950321 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Self-Supervised Point Cloud Prediction for Autonomous Driving

    Du, Ronghua / Feng, Rongying / Gao, Kai et al. | IEEE | 2024


    Point Cloud Automatic Annotation Framework for Autonomous Driving

    Zhao, Chaoran / Peng, Bo / Azumi, Takuya | IEEE | 2024



    AUTONOMOUS DRIVING SYSTEM FOR SHIP

    KANG DAE WOONG | European Patent Office | 2023

    Free access