The invention relates to an automatic driving man-machine interaction takeover training method and system based on digital twinning, and belongs to the field of three-dimensional modeling and computer vision. Firstly, a plurality of static 2D images are obtained through multi-angle collection of a modeling object by a camera, and camera internal and external parameters corresponding to each image are estimated through tools such as COLMAP and the like; providing a segmentation prompt for the target object on each image, segmenting the image by using an SAM large model, and removing background noise to obtain a segmented image of the target object; and inputting the segmented image and camera parameters into a NeRF neural radiation field algorithm, performing three-dimensional reconstruction and rendering, and converting an implicit three-dimensional model into a display three-dimensional model by using a point cloud reconstruction technology so as to obtain a high-precision digital twin geometric model. According to the method, the powerful semantic comprehension capability of the SAM large model and the efficient three-dimensional reconstruction capability of the NeRF neural radiation field algorithm can be effectively utilized, and automation and precision of digital twin modeling are realized.

    本发明涉及一种基于数字孪生的自动驾驶人机交互接管训练方法和系统,属于三维建模与计算机视觉领域。首先通过相机对建模物体的多角度采集得到多张静态2D图像,并通过COLMAP等工具估计每张图像对应的相机内外参;然后在每张图像上为目标物体提供分割提示并利用SAM大模型对图像进行分割,去除背景噪声,得到目标物体的分割图像;接着将分割图像及相机参数输入NeRF神经辐射场算法,进行三维重建与渲染,利用点云重建技术将隐式三维模型转换为显示三维模型进而得到高精度的数字孪生几何模型。该方法能够有效地利用SAM大模型的强大语义理解能力和NeRF神经辐射场算法的高效三维重建能力,实现数字孪生建模的自动化和精确化。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Automatic driving man-machine interaction takeover training method and system based on digital twinning


    Weitere Titelangaben:

    一种基于数字孪生的自动驾驶人机交互接管训练方法和系统


    Beteiligte:
    YUAN DONGFENG (Autor:in) / MA MINGDI (Autor:in) / ZHANG RUHUA (Autor:in)

    Erscheinungsdatum :

    2024-03-08


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06T Bilddatenverarbeitung oder Bilddatenerzeugung allgemein , IMAGE DATA PROCESSING OR GENERATION, IN GENERAL



    Automatic driving takeover request reminding device

    QIU ZIHAO / HU XINKUI / LIU HONGYAN et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Driver takeover capability prediction method under automatic driving system

    ZHANG XIQIAO / SHI JINYU / DU JIAMING et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    Automatic driving hierarchical takeover interaction method and device and readable storage medium

    GAO XIUJING / LIU XIN / HUANG HONGWU et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    DRIVING TAKEOVER APPARATUS

    MIKI YUKI | Europäisches Patentamt | 2021

    Freier Zugriff

    L2-level automatic driving vehicle takeover alarm system and method

    REN CHUANBING / MA FENGLIN / YANG DAQIAN et al. | Europäisches Patentamt | 2021

    Freier Zugriff