Over recent years, there has been an increase in research interest regarding depth estimation using multiple-spectrum images from Visible-Light (VIS) and Thermal-Infrared (TIR) cameras. Depth estimation with multi-spectrum images has the potential for more reliable depth estimation under varying illumination conditions. However, most of the existing multi-spectrum depth estimation methods rely on the supervision of ground-truth depth information or other additional complex labels or equipment, resulting in limited convenience for practical application. To address these challenges, we propose a new unsupervised all-day depth estimation framework with multi-spectrum stereo images, specifically, a thermal image (acting as the right view) and a visible-light image (acting as the left view), and they are denoted as a single frame of multi-spectrum stereo sequences. The network uses multiple frames of multi-spectrum stereo images as input during training and a single frame of multi-spectrum stereo images as input for testing. We improve the depth estimation accuracy by combining the intra-spectrum temporal consistency and the cross-spectrum spatial consistency. A unique self-teacher structure is integrated to augment the quality of nighttime depth estimation using daytime image data. Simultaneously, we release a large multi-spectrum stereo dataset, which includes TIR-VIS stereo images collected during the day and night. Experimental results reveal that our method achieves a significant improvement in all-day depth estimation by effectively utilizing multi-spectrum stereo images.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Unsupervised Multi-Spectrum Stereo Depth Estimation for All-Day Vision


    Contributors:
    Guo, Yubin (author) / Kong, Hui (author) / Gu, Shuo (author)

    Published in:

    Publication date :

    2024-01-01


    Size :

    3695516 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Unsupervised Cross-Spectrum Depth Estimation by Visible-Light and Thermal Cameras

    Guo, Yubin / Qi, Xinlei / Xie, Jin et al. | IEEE | 2023


    Multi-label Depth Estimation for Graph Cuts Stereo Problems

    Papadakis, N. | British Library Online Contents | 2010


    Cyclopean stereo vision for depth perception [1915-13]

    Yeh, C.-P. / Wang, S.-P. / SPIE et al. | British Library Conference Proceedings | 1993


    Depth map estimation with stereo images

    European Patent Office | 2017

    Free access