A multi-modal deep learning-based driver drowsiness detection system using an image and PPG data of the present invention comprises: a pre-processing unit which extracts information on the position of the eyes and the mouth from a facial image of a driver and, at the same time, collecting photoplethysmogram (PPG) data of the driver; a multi-modal network which integrally fuses the information on the position of the eyes and the mouth with the PPG data; and a classification network which uses the fusion data produced as the information on the position of the eyes and the mouth is fused with the PPG data in the multi-modal network to recognize the drowsiness of the driver. Therefore, by considering all of the image data of the driver, such as the eyes and the mouth, and the photoplethysmogram (PPG) data, the drowsiness of the driver can be recognized at high precision.

    본 발명에 따른 이미지와 PPG 데이터를 이용한 멀티모달 딥러닝 기반의 운전자 졸음감지 시스템은 운전자의 얼굴 이미지로부터 눈과 입의 위치정보를 추출하고, 동시에 상기 운전자의 PPG(Photoplethysmogram:생리학 데이터)를 수집하는 전처리부; 상기 눈과 입의 위치정보를 상기 PPG와 하나로 융합하는 멀티모달 네트워크; 및 상기 멀티모달 네트워크에서 융합된 눈과 입의 위치정보를 상기 PPG와 융합한 융합 데이터를 가지고 운전자의 졸음상태를 파악하는 분류 네트워크;를 포함하여 운전자의 이미지 데이터(눈, 입)와 생리학 데이터(PPG) 모두 고려함으로써 정확도 높은 운전자의 졸음상태를 파악할 수 있는 효과가 있다.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    DRIVER DROWSINESS DETECTION SYSTEM USING IMAGE AND PPG DATA BASED ON MULTIMODAL DEEP LEARNING


    Weitere Titelangaben:

    이미지와 PPG 데이터를 이용한 멀티모달 딥러닝 기반의 운전자 졸음감지 시스템


    Beteiligte:
    LEE KYU CHUL (Autor:in) / BACK MOON KI (Autor:in)

    Erscheinungsdatum :

    02.04.2020


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Koreanisch


    Klassifikation :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / B60K Anordnung oder Einbau von Antriebseinheiten oder von Kraft- bzw. Drehmomentübertragungen in Fahrzeugen , ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES / B60R Fahrzeuge, Fahrzeugausstattung oder Fahrzeugteile, soweit nicht anderweitig vorgesehen , VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR



    Driver Drowsiness Detection Using Deep Learning

    Pawar, Rupali / Wamburkar, Saloni / Deshmukh, Rutuja et al. | IEEE | 2021


    Driver Drowsiness Detection Using Deep Learning

    Jain, Anuj Kumar / Sharma, Vikrant / Goel, Sandeep et al. | IEEE | 2023


    Driver Drowsiness Detection using Deep Learning

    Nandhini, P.S. / Kuppuswami, S. / Malliga, S. et al. | IEEE | 2022


    Enhanced Driver Drowsiness Detection using Deep Learning

    Singh Dipender / Singh Avtar | DOAJ | 2023

    Freier Zugriff

    Driver drowsiness detection using multimodal sensor fusion

    Andreeva, Elena O. / Aarabi, Parham / Philiastides, Marios G. et al. | SPIE | 2004