In the era of big data, the massive multi-modal data accumulated in traditional manufacturing enterprises are chaotic and redundant, the huge hidden value in those data cannot be mined through complementary learning among multi-modal data so as to realize the transformation of enterprises from digitalization to intelligence. In order to solve the problem of multi-source heterogeneous data perception fusion in intelligent manufacturing, this paper proposes a multi-modal data cross-media perception fusion algorithm based on digital twin. Firstly, aiming at the synchronous fusion of physical space and information space in the manufacturing process, the digital twin modeling method and multi-perception fusion method are adopted to restore complete product processing processes in real-time and dynamically. Secondly, the multi-modal data feature extraction models are established respectively to solve the cross-media perception fusion problem of multi-modal data. Then, the method of multi-feature fusion and multi-model coupling is adopted to disperse the uncertainty of these models. Finally, an overall solution of omnidirectional perception and cross-media virtual reality fusion for multi-modal data processing is realized. The research results of this paper also have a certain practical value to intelligent processing workshop and their assembly processes.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Research on Multi-modal Data Cross-Media Perception Fusion Algorithm Based on Digital Twin


    Additional title:

    Lect. Notes Electrical Eng.


    Contributors:
    Wang, Yi (editor) / Martinsen, Kristian (editor) / Yu, Tao (editor) / Wang, Kesheng (editor) / Wan, Xiang (author) / Liu, Lilan (author) / Feng, Bowen (author) / Chen, Chen (author) / Wang, Yi (author)

    Conference:

    International Workshop of Advanced Manufacturing and Automation ; 2020 ; Zhanjiang, China October 12, 2020 - October 13, 2020



    Publication date :

    2021-01-23


    Size :

    9 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English





    Cross-media multi-modal aircraft

    LIU JIAPENG | European Patent Office | 2024

    Free access

    ESCAPE Data Collection for Multi-Modal Data Fusion Research

    Zulch, Peter / Distasio, Marcello / Cushman, Todd et al. | IEEE | 2019



    Aerodynamic Data Fusion Toward the Digital Twin Paradigm

    Renganathan, S. Ashwin / Harada, Kohei / Mavris, Dimitri N. | AIAA | 2020