In a large-scale epidemic, reducing direct contact among medical personnel, attendants and patients has become a necessary means of epidemic prevention and control. Intelligent vehicles and mobile robots in the hospital environment, such as disinfection vehicles, logistics vehicles, nursing robots, and guiding robots, play an important role in improving the operational efficiency of the medical system and promoting epidemic prevention and governance. Powerful capabilities of environmental spatial perception and reconstruction are the keys to accurate localization, navigation, and obstacle avoidance for intelligent vehicles and autonomous robots in such operations. Omnidirectional perception is becoming increasingly important and proliferative in autonomous vehicles and robots since its wide field of view significantly enhances the perception ability. However, the lack of dense and accurate 360° depth datasets has brought the challenge to the omnidirectional perception. In this paper, we propose a depth-sensing and reconstruction system to address this challenge in the large-scale indoor environment. First, we design an omnidirectional depth completion convolutional neural network model, in which a spherical normalized convolutional and the unit sphere area-based loss are introduced to extract features from cross-modal omnidirectional input with unequal sparsity and deal with the imbalanced data distribution and distortion in the panoramic input. In addition, we present a 3D reconstruction system by integrating our depth completion into omnidirectional localization and dense mapping. We evaluate our method on 360D large-scale indoor datasets and real-world sequences of a challenging hospital scene. Extensive experiments show that the proposed method outperforms the other state-of-the-art (SoTA) approaches in terms of depth completion and 3D reconstruction.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Cross-Modal 360° Depth Completion and Reconstruction for Large-Scale Indoor Environment


    Contributors:
    Liu, Ruyu (author) / Zhang, Guodao (author) / Wang, Jiangming (author) / Zhao, Shuwen (author)

    Published in:

    Publication date :

    2022-12-01


    Size :

    4453799 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    APS: A Large-Scale Multi-modal Indoor Camera Positioning System

    Ghofrani, Ali / Toroghi, Rahil Mahdian / Tabatabaie, Seyed Mojtaba | British Library Conference Proceedings | 2021


    CASwin Transformer: A Hierarchical Cross Attention Transformer for Depth Completion

    Feng, Chunyu / Wang, Xiaonian / Zhang, Yangyang et al. | IEEE | 2022


    A Gauge Field Model of Modal Completion

    Citti, G. | British Library Online Contents | 2015


    Sparse Auxiliary Network for Depth Completion

    GUIZILINI VITOR / AMBRUS RARES A / GAIDON ADRIEN DAVID | European Patent Office | 2022

    Free access

    Deterministic Guided LiDAR Depth Map Completion

    Krauss, Bryan / Schroeder, Gregory / Gustke, Marko et al. | IEEE | 2021