Depth completion aims to recover dense depth maps from sparse depth measurements. It is a fundamental challenge in computer vision that is faced in numerous applications, such as robotics, UAVs, and autonomous driving. With the recent advancement in hardware, millimeter-wave (mmWave) radar technology is now commonly employed in high-level perception tasks of autonomous driving. However, compared to LiDAR point clouds, mmWave radar point clouds tend to be sparse and include several ghost targets. To address these challenges, we propose a semantic-guided network architecture to perform depth diffusion by utilizing the consistent relationship between semantics and depth. Our method first utilizes semantic features from images to extend the height measurements, and then transforms the global depth completion task into a series of category-specific depth diffusion tasks to learn the semantic-depth priors and accommodate radar sparsity. In addition, a gradient-guided Smooth-Edge loss is developed to explicitly constrain the semantic regional consistency and border discontinuity. Meanwhile, a 4D dataset that features Camera, 4D Radar, and LiDAR measurement in diverse scenes is collected to conduct experiments. Extensive experiments demonstrate the superior performance of our proposed method over existing monocular and fusion-based approaches.
Semantic-Guided Depth Completion From Monocular Images and 4D Radar Data
IEEE Transactions on Intelligent Vehicles ; 9 , 9 ; 5606-5617
2024-09-01
4474594 byte
Article (Journal)
Electronic Resource
English
Instance-Aware Monocular 3D Semantic Scene Completion
IEEE | 2024
|Deterministic Guided LiDAR Depth Map Completion
IEEE | 2021
|Network architecture for the joint learning of monocular depth prediction and completion
European Patent Office | 2023
|Non-Guided Depth Completion with Adversarial Networks
IEEE | 2018
|