An artificial intelligence framework is described that incorporates a number of neural networks and a number of transformers for converting a two-dimensional image into three-dimensional semantic information. Neural networks convert one or more images into a set of image feature maps, depth information associated with the one or more images, and query proposals based on the depth information. A first transformer implements a cross-attention mechanism to process the set of image feature maps in accordance with the query proposals. The output of the first transformer is combined with a mask token to generate initial voxel features of the scene. A second transformer implements a self-attention mechanism to convert the initial voxel features into refined voxel features, which are up-sampled and processed by a lightweight neural network to generate the three-dimensional semantic information, which may be used by, e.g., an autonomous vehicle for various advanced driver assistance system (ADAS) functions.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    SPARSE VOXEL TRANSFORMER FOR CAMERA-BASED 3D SEMANTIC SCENE COMPLETION


    Beteiligte:
    LI YIMING (Autor:in) / YU ZHIDING (Autor:in) / CHOY CHRISTOPHER B (Autor:in) / XIAO CHAOWEI (Autor:in) / ALVAREZ LOPEZ JOSE MANUEL (Autor:in) / FIDLER SANJA (Autor:in) / ANANDKUMAR ANIMASHREE (Autor:in)

    Erscheinungsdatum :

    2024-03-14


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    IPC:    G06T Bilddatenverarbeitung oder Bilddatenerzeugung allgemein , IMAGE DATA PROCESSING OR GENERATION, IN GENERAL / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06V



    Instance-Aware Monocular 3D Semantic Scene Completion

    Xiao, Haihong / Xu, Hongbin / Kang, Wenxiong et al. | IEEE | 2024


    Photorealistic Scene Reconstruction by Voxel Coloring

    Seitz, S. M. / Dyer, C. R. | British Library Online Contents | 1999


    Monocular localization within sparse voxel maps

    Wong, David / Kawanishi, Yasutomo / Deguchi, Daisuke et al. | IEEE | 2017


    Dense Top-View Semantic Completion With Sparse Guidance and Online Distillation

    Gu, Shuo / Lu, Jiacheng / Yang, Jian et al. | IEEE | 2024