Current perception models for different tasks usually exist in modular forms on Unmanned Surface Vehicles (USVs), which infer extremely slowly in parallel on edge devices, causing the asynchrony between perception results and USV position, and leading to error decisions of autonomous navigation. Compared with Unmanned Ground Vehicles (UGVs), the robust perception of USVs develops relatively slowly. Moreover, most current multi-task perception models are huge in parameters, slow in inference and not scalable. Oriented on this, we propose Achelous, a low-cost and fast unified panoptic perception framework for water-surface perception based on the fusion of a monocular camera and 4D mmWave radar. Achelous can simultaneously perform five tasks, detection and segmentation of visual targets, drivable-area segmentation, waterline segmentation and radar point cloud segmentation. Besides, models in Achelous family, with less than around 5 million parameters, achieve about 18 FPS on an NVIDIA Jetson AGX Xavier, 11 FPS faster than HybridNets, and exceed YOLOX-Tiny and Segformer-B0 on our collected dataset about $\mathbf{5}\ \mathbf{mAP}_{\mathbf{50}-\mathbf{95}}$ and 0.7 mIoU, especially under situations of adverse weather, dark environments and camera failure. To our knowledge, Achelous is the first comprehensive panoptic perception framework combining vision-level and point-cloud-level tasks for water-surface perception. To promote the development of the intelligent transportation community, we release our codes in https://github.com/GuanRunwei/Achelous.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Achelous: A Fast Unified Water-Surface Panoptic Perception Framework Based on Fusion of Monocular Camera and 4D mmWave Radar


    Contributors:
    Guan, Runwei (author) / Yao, Shanliang (author) / Zhu, Xiaohui (author) / Man, Ka Lok (author) / Lim, Eng Gee (author) / Smith, Jeremy (author) / Yue, Yong (author) / Yue, Yutao (author)


    Publication date :

    2023-09-24


    Size :

    4985753 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Sparse U-PDP: A Unified Multi-Task Framework for Panoptic Driving Perception

    Wang, Hai / Qiu, Meng / Cai, Yingfeng et al. | IEEE | 2023


    Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles

    P, Aparna M / Thayyil Ravi, Arunkrishna / Jose, Edwin et al. | SAE Technical Papers | 2022


    Traffic Incident Detection Based on mmWave Radar and Improvement Using Fusion with Camera

    Zhimin Tao / Yanbing Li / Pengcheng Wang et al. | DOAJ | 2022

    Free access

    A DNN-LSTM based Target Tracking Approach using mmWave Radar and Camera Sensor Fusion

    Sengupta, Arindam / Jin, Feng / Cao, Siyang | IEEE | 2019


    Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles

    Jose, Edwin / P, Aparna M / Patil, Mrinalini et al. | British Library Conference Proceedings | 2022