Depth map estimation from images is a crucial task in self-driving applications. Existing methods can be categorized into two groups: multi-view stereo and monocular depth estimation. The former requires cameras to have large overlapping areas and a sufficient baseline between them, while the latter that processes each image independently can hardly guarantee the structure consistency between cameras. In this paper, we propose a novel self-supervised multi-camera collaborative depth prediction method with latent diffusion models, which does not require large overlapping areas while maintaining structure consistency between cameras. Specifically, we introduce MCDP, a new generative foundation model for estimating depth attributes for multi-cameras. We formulate the depth estimation as a weighted combination of depth bases, in which the weights are updated iteratively by the recurrent refinement strategy. During the iterative update, the results of depth estimation are compared across cameras, and the information of overlapping areas is propagated to the whole depth maps with the help of basis formulation in diffusion process. We integrate the GRU-based Weight Net into the diffusion process, allowing the refined hidden state to serve as a conditional input to accurately control the next iterative denoising step. Furthermore, by incorporating the proposed depth consistency loss, we ensure structural consistency across cameras, even in regions with minimal overlap. Experimental results on DDAD, NuScenes, Cityscapes, and Waymo Open Datasets demonstrate the superior performance of our method, and show great help for the downstream task.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Self-Supervised Multi-Camera Collaborative Depth Prediction With Latent Diffusion Models


    Beteiligte:
    Xu, Jialei (Autor:in) / Liu, Xianming (Autor:in) / Bai, Yuanchao (Autor:in) / Jiang, Junjun (Autor:in) / Ji, Xiangyang (Autor:in)


    Erscheinungsdatum :

    01.07.2025


    Format / Umfang :

    6825676 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Shared median-scaling metric for multi-camera self-supervised depth evaluation

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Self-occlusion masks to improve self-supervised monocular depth estimation in multi-camera settings

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    SELF-OCCLUSION MASKS TO IMPROVE SELF-SUPERVISED MONOCULAR DEPTH ESTIMATION IN MULTI-CAMERA SETTINGS

    GUIZILINI VITOR / AMBRUS RARES ANDREI / GAIDON ADRIEN DAVID et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    SceneDiffusion: Conditioned Latent Diffusion Models for Traffic Scene Prediction

    Balasubramanian, Lakshman / Wurst, Jonas / Egolf, Robin et al. | IEEE | 2023


    Semi-supervised Unified Latent Factor learning with multi-view data

    Jiang, Y. | British Library Online Contents | 2014