Depth map estimation from images is a crucial task in self-driving applications. Existing methods can be categorized into two groups: multi-view stereo and monocular depth estimation. The former requires cameras to have large overlapping areas and a sufficient baseline between them, while the latter that processes each image independently can hardly guarantee the structure consistency between cameras. In this paper, we propose a novel self-supervised multi-camera collaborative depth prediction method with latent diffusion models, which does not require large overlapping areas while maintaining structure consistency between cameras. Specifically, we introduce MCDP, a new generative foundation model for estimating depth attributes for multi-cameras. We formulate the depth estimation as a weighted combination of depth bases, in which the weights are updated iteratively by the recurrent refinement strategy. During the iterative update, the results of depth estimation are compared across cameras, and the information of overlapping areas is propagated to the whole depth maps with the help of basis formulation in diffusion process. We integrate the GRU-based Weight Net into the diffusion process, allowing the refined hidden state to serve as a conditional input to accurately control the next iterative denoising step. Furthermore, by incorporating the proposed depth consistency loss, we ensure structural consistency across cameras, even in regions with minimal overlap. Experimental results on DDAD, NuScenes, Cityscapes, and Waymo Open Datasets demonstrate the superior performance of our method, and show great help for the downstream task.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Self-Supervised Multi-Camera Collaborative Depth Prediction With Latent Diffusion Models


    Contributors:
    Xu, Jialei (author) / Liu, Xianming (author) / Bai, Yuanchao (author) / Jiang, Junjun (author) / Ji, Xiangyang (author)


    Publication date :

    2025-07-01


    Size :

    6825676 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Shared median-scaling metric for multi-camera self-supervised depth evaluation

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | European Patent Office | 2023

    Free access

    Self-occlusion masks to improve self-supervised monocular depth estimation in multi-camera settings

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | European Patent Office | 2024

    Free access

    SELF-OCCLUSION MASKS TO IMPROVE SELF-SUPERVISED MONOCULAR DEPTH ESTIMATION IN MULTI-CAMERA SETTINGS

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | European Patent Office | 2022

    Free access

    SceneDiffusion: Conditioned Latent Diffusion Models for Traffic Scene Prediction

    Balasubramanian, Lakshman / Wurst, Jonas / Egolf, Robin et al. | IEEE | 2023


    Semi-supervised Unified Latent Factor learning with multi-view data

    Jiang, Y. | British Library Online Contents | 2014