Cross-spectrum depth estimation aims to provide a reliable depth map under variant-illumination conditions with a pair of dual-spectrum images. It is valuable for autonomous driving applications when vehicles are equipped with two cameras of different modalities. However, images captured by different-modality cameras can be photometrically quite different, which makes cross-spectrum depth estimation a very challenging problem. Moreover, the shortage of large-scale open-source datasets also retards further research in this field. In this paper, we propose an unsupervised visible light(VIS)-image-guided cross-spectrum (i.e., thermal and visible-light, TIR-VIS in short) depth-estimation framework. The input of the framework consists of a cross-spectrum stereo pair (one VIS image and one thermal image). First, we train a depth-estimation base network using VIS-image stereo pairs. To adapt the trained depth-estimation network to the cross-spectrum images, we propose a multi-scale feature-transfer network to transfer features from the TIR domain to the VIS domain at the feature level. Furthermore, we introduce a mechanism of cross-spectrum depth cycle-consistency to improve the depth estimation result of dual-spectrum image pairs. Meanwhile, we release to society a large cross-spectrum dataset with visible-light and thermal stereo images captured in different scenes. The experiment result shows that our method achieves better depth-estimation results than the compared existing methods. Our code and dataset are available on https://github.com/whitecrow1027/CrossSP_Depth.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Unsupervised Cross-Spectrum Depth Estimation by Visible-Light and Thermal Cameras


    Contributors:
    Guo, Yubin (author) / Qi, Xinlei (author) / Xie, Jin (author) / Xu, Cheng-Zhong (author) / Kong, Hui (author)

    Published in:

    Publication date :

    2023-10-01


    Size :

    6850553 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Unsupervised Multi-Spectrum Stereo Depth Estimation for All-Day Vision

    Guo, Yubin / Kong, Hui / Gu, Shuo | IEEE | 2024


    Vehicle Distance Measurement based on Visible Light Communication Using Stereo Cameras

    Huang, Ruiyi / Yamazato, Takaya / Kinoshita, Masayuki et al. | IEEE | 2021


    VEHICLE DISTANCE MEASUREMENT BASED ON VISIBLE LIGHT COMMUNICATION USING STEREO CAMERAS

    Huang, Ruiyi / Yamazato, Takaya / Kinoshita, Masayuki et al. | British Library Conference Proceedings | 2021


    Depth range accuracy for plenoptic cameras

    Monteiro, Nuno Barroso / Marto, Simão / Barreto, João Pedro et al. | British Library Online Contents | 2018