The invention provides a vehicle-mounted gesture interaction method and system based on a monocular camera, and belongs to the technical field of vehicle-mounted man-machine interaction and gesture recognition. The vehicle-mounted gesture interaction method comprises the steps that an image is collected by a monocular camera, a gesture three-dimensional reconstruction model is obtained through hand recognition, segmentation, feature extraction, 2D detection, 3D detection and model fitting in sequence, and then the gesture three-dimensional reconstruction model is rendered into a gesture depth map and a gesture two-dimensional map; a three-dimensional space motion track of a hand is detected by utilizing an inter-frame difference method, gestures are matched by utilizing a pre-stored gesture data set, and finally a control instruction is generated and sent to a vehicle-mounted multimedia system and a man-machine interaction system. According to the method, the monocular camera is used for carrying out gesture three-dimensional estimation, various hand images are fitted, and compared with a traditional gesture interaction method, the method has the advantages of being lower in equipment cost, better in recognition precision and robustness and larger in upgrading cost.

    本发明提供了一种基于单目相机的车载手势交互方法及系统,属于车载人机交互和手势识别技术领域。所述车载手势交互方法包括:单目相机采集图像,依次经过手部识别、分割、特征提取、2D检测、3D检测和模型拟合得到手势三维重建模型,然后将手势三维重建模型渲染为手势深度图和手势二维图,利用帧间差分法检测手部的三维空间运动轨迹、利用预存的手势数据集匹配手势,最后生成控制指令给车载多媒体系统和人机交互系统。该方法利用单目相机进行手势三维估计,并拟合出多种手部的图像,相较于传统的手势交互方法拥有更低的设备成本、更优的识别精度和鲁棒性、更大的升级成本优势。


    Access

    Download


    Export, share and cite



    Title :

    Vehicle-mounted gesture interaction method and system based on monocular camera


    Additional title:

    一种基于单目相机的车载手势交互方法及系统


    Contributors:

    Publication date :

    2023-08-29


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60R Fahrzeuge, Fahrzeugausstattung oder Fahrzeugteile, soweit nicht anderweitig vorgesehen , VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR / G06V



    Gesture cues for conversational interaction in monocular video

    Quek, F. / McNeill, D. / Ansari, R. et al. | IEEE | 1999


    Vehicle-mounted virtual screen information interaction system based on gesture recognition

    LI JIEHUI / WANG JIAN / ZHOU DEFENG et al. | European Patent Office | 2021

    Free access

    Vehicle-mounted camera, vehicle-mounted system and method of using vehicle-mounted camera

    KAZAMA TAKESHI / KIMURA TAKAYUKI / SHIMIZU NOBUHISA | European Patent Office | 2022

    Free access

    VEHICLE-MOUNTED CAMERA, VEHICLE-MOUNTED SYSTEM AND METHOD OF USING VEHICLE-MOUNTED CAMERA

    KAZAMA TAKESHI / KIMURA TAKAYUKI / SHIMIZU NOBUHISA | European Patent Office | 2020

    Free access

    VEHICLE-MOUNTED CAMERA AND VEHICLE-MOUNTED CAMERA SYSTEM

    KOUROGI NAOKI / MONJI TATSUHIKO | European Patent Office | 2016

    Free access