According to various embodiments, illustrated are a system, a method, and a medium for operating an autonomous driving vehicle (ADV). In embodiments, a plurality of machine learning modules extracts characteristics respectively from audio data and visual data captured by a sensor mounted on the autonomous driving vehicle, and then the extracted characteristics are fused to implement connection characteristic vectors. The connection characteristic vectors are provided as an input for multi-layer perceptron (MLP) to generate detection results related to an existence of an emergency vehicle in a surrounding environment. The autonomous driving vehicle takes appropriate actions by using the detection result to follow local traffic rules.

    다양한 실시형태에 따라, 자율 주행 차량(ADV)을 작동하기 위한 시스템, 방법 및 매체를 설명한다. 실시형태에서 복수의 기계 학습 모델을 사용하여 자율 주행 차량에 장착되는 센서에 의해 캡처된 오디오 데이터 및 시각적 데이터로부터 특징을 각각 추출하고, 다음 추출된 이러한 특징을 융합하여 연결 특징 벡터를 구축한다. 연결 특징 벡터는 다층 퍼셉트론(MLP)에 입력으로서 제공되어, 주위 환경 중 긴급 차량의 존재 여부와 관련되는 감지 결과를 생성한다. 자율 주행 차량은 감지 결과를 사용하여 적절한 조치를 취함으로써 로컬의 교통 규칙을 준수할 수 있다.


    Access

    Download


    Export, share and cite



    Title :

    MACHINE LEARNING MODEL TO FUSE EMERGENCY VEHICLE AUDIO AND VISUAL DETECTION


    Additional title:

    긴급 차량 오디오 및 시각적 감지를 융합한 기계 학습 모델


    Contributors:
    XU KECHENG (author) / SUN HONGYI (author) / LUO QI (author) / WANG WEI (author) / LIN ZEJUN (author) / REYNOLDS WESLEY (author) / LIU FENG (author) / HU JIANGTAO (author) / MIAO JINGHAO (author)

    Publication date :

    2022-05-09


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Korean


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06T Bilddatenverarbeitung oder Bilddatenerzeugung allgemein , IMAGE DATA PROCESSING OR GENERATION, IN GENERAL / H04R LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS , Lautsprecher, Mikrofone, Schallplatten-Tonabnehmer oder ähnliche akustische, elektromechanische Wandler



    MACHINE LEARNING MODEL TO FUSE EMERGENCY VEHICLE AUDIO AND VISUAL DETECTION

    XU KECHENG / SUN HONGYI / LUO QI et al. | European Patent Office | 2022

    Free access

    Machine learning model to fuse emergency vehicle audio and visual detection

    XU KECHENG / SUN HONGYI / LUO QI et al. | European Patent Office | 2023

    Free access

    Machine learning model fusing emergency vehicle audio and visual detection

    XU KECHENG / SUN HONGYI / LUO QI et al. | European Patent Office | 2021

    Free access

    EMERGENCY VEHICLE AUDIO AND VISUAL DETECTION POST FUSION

    XU KECHENG / SUN HONGYI / LUO QI et al. | European Patent Office | 2022

    Free access

    EMERGENCY VEHICLE DETECTION FUSING AUDIO AND VISUAL DATA

    XU KECHENG / SUN HONGYI / LUO QI et al. | European Patent Office | 2022

    Free access