The invention relates to an automatic driving man-machine interaction takeover training method and system based on digital twinning, and belongs to the field of three-dimensional modeling and computer vision. Firstly, a plurality of static 2D images are obtained through multi-angle collection of a modeling object by a camera, and camera internal and external parameters corresponding to each image are estimated through tools such as COLMAP and the like; providing a segmentation prompt for the target object on each image, segmenting the image by using an SAM large model, and removing background noise to obtain a segmented image of the target object; and inputting the segmented image and camera parameters into a NeRF neural radiation field algorithm, performing three-dimensional reconstruction and rendering, and converting an implicit three-dimensional model into a display three-dimensional model by using a point cloud reconstruction technology so as to obtain a high-precision digital twin geometric model. According to the method, the powerful semantic comprehension capability of the SAM large model and the efficient three-dimensional reconstruction capability of the NeRF neural radiation field algorithm can be effectively utilized, and automation and precision of digital twin modeling are realized.

    本发明涉及一种基于数字孪生的自动驾驶人机交互接管训练方法和系统,属于三维建模与计算机视觉领域。首先通过相机对建模物体的多角度采集得到多张静态2D图像,并通过COLMAP等工具估计每张图像对应的相机内外参;然后在每张图像上为目标物体提供分割提示并利用SAM大模型对图像进行分割,去除背景噪声,得到目标物体的分割图像;接着将分割图像及相机参数输入NeRF神经辐射场算法,进行三维重建与渲染,利用点云重建技术将隐式三维模型转换为显示三维模型进而得到高精度的数字孪生几何模型。该方法能够有效地利用SAM大模型的强大语义理解能力和NeRF神经辐射场算法的高效三维重建能力,实现数字孪生建模的自动化和精确化。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving man-machine interaction takeover training method and system based on digital twinning


    Additional title:

    一种基于数字孪生的自动驾驶人机交互接管训练方法和系统


    Contributors:
    YUAN DONGFENG (author) / MA MINGDI (author) / ZHANG RUHUA (author)

    Publication date :

    2024-03-08


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06T Bilddatenverarbeitung oder Bilddatenerzeugung allgemein , IMAGE DATA PROCESSING OR GENERATION, IN GENERAL



    Automatic driving takeover system based on driving style

    ZHAO YUCHENG / YANG ZHIJIE / HE TAO et al. | European Patent Office | 2025

    Free access

    Vehicle automatic driving takeover method and system

    LI YUN / YIN LEILEI / CHENG KANG et al. | European Patent Office | 2025

    Free access

    Automatic driving takeover request reminding device

    QIU ZIHAO / HU XINKUI / LIU HONGYAN et al. | European Patent Office | 2023

    Free access

    Automatic driving behavior decision optimization system based on digital twinning

    LIU YI / ZHAO FENG / ZHU YING | European Patent Office | 2024

    Free access

    Driver takeover capability prediction method under automatic driving system

    ZHANG XIQIAO / SHI JINYU / DU JIAMING et al. | European Patent Office | 2021

    Free access