Multi-fisheye System has the advantages of sufficient overlap and the ability to capture a complete 360° scene, which is beneficial for the omnidirectional depth estimation task. However, due to the severe distortion of the fisheye images, it is hard for such systems to extract and match features to predict an accurate depth. In this work, on the basis of a multi-fisheye system, we present a novel end-to-end deep learning architecture for omnidirectional depth estimation: 1) to capture the reliable features of the distorted fisheye image, a multi-scale feature extraction and aggregation module is improved, which can adaptively obtain the global context information to represent the features; 2) to leverage more aligned features, especially those in the overlap between multi-fisheye images, we construct a fusion cost volume to combine similarity and semantic information, which can enhance the feature discriminability; and 3) to refine the omnidirectional depth map efficiently, a cascaded cost regularization architecture is proposed. Instead of several costly 3D convolutions, the 3D BSConv based on intra-kernel correlations is introduced to regularize the cost. The proposed method can incrementally predict the depth map from coarse to fine, and reduce the network computational complexity significantly. The experiments in several public indoor and outdoor synthetic datasets demonstrate that the proposed method outperforms some state-of-the-art methods in terms of a synthesis of accuracy and speed, with fewer model parameters.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Omnidirectional Depth Estimation With Hierarchical Deep Network for Multi-Fisheye Navigation Systems


    Contributors:
    Su, Xiaojie (author) / Liu, Shimin (author) / Li, Rui (author)

    Published in:

    Publication date :

    2023-12-01


    Size :

    4004804 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

    Kumar, Varun Ravi / Milz, Stefan / Witt, Christian et al. | IEEE | 2018



    Monocular Fisheye Depth Estimation for UAV Applications with Segmentation Feature Integration

    Jaisawal, Pravin Kumar / Papakonstantinou, Stephanos / Gollnick, Volker | IEEE | 2024



    Omnidirectional stereo systems for robot navigation

    Adorni, Giovanni / Mordonini, Monica / Cagnoni, Stefano et al. | IEEE | 2003