The invention discloses a vehicle-mounted sign language interaction method and device and a computer readable storage medium. The method comprises the steps of obtaining an input text; performing emotion recognition on the input text by using a large language model to obtain emotion information of the user, and extracting entity object information in the input text; analyzing a current driving state, and determining a current interaction situation in combination with the input text; determining a user intention in combination with the interaction situation, the emotion information and the entity object information; if it is determined that the user intends to operate the vehicle, the input text is converted into an operation instruction and sent to the vehicle-mounted system, and if it is determined that the user intends to conduct conventional communication, the information output action is executed. According to the method, the user intention can be accurately determined through the natural language model in combination with emotion recognition, interaction situation recognition and content entity object recognition, mutual translation between sign language information and language information and vehicle operation are effectively achieved, and the vehicle-mounted system interaction experience of hearing disorder and aphasia people is greatly improved.

    本发明公开了一种车载手语交互方法、装置及计算机可读存储介质,方法包括:获取输入文本;利用大语言模型对输入文本进行情感识别以获得用户的情感信息,并提取输入文本中的实体对象信息;分析当前行车状态,结合输入文本确定当前的交互情境;结合交互情境、情感信息和实体对象信息确定用户意图;如果确定用户意图为操作车辆,则将输入文本转化成操作指令并发送到车载系统,如果确定用户意图为常规交流,则执行信息输出动作。本发明通过自然语言模型,结合情感识别、交互情境识别以及内容实体对象识别可以准确地对用户意图进行确定,有效实现将手语信息和语言信息之间进行相互翻译,和车辆操作,大大提高了听力障碍和失语人群的车载系统交互体验。


    Access

    Download


    Export, share and cite



    Title :

    Vehicle-mounted sign language interaction method and device and computer readable storage medium


    Additional title:

    一种车载手语交互方法、装置及计算机可读存储介质


    Contributors:
    JOHN MELBA (author)

    Publication date :

    2024-04-05


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60R Fahrzeuge, Fahrzeugausstattung oder Fahrzeugteile, soweit nicht anderweitig vorgesehen , VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06V



    HUMAN-COMPUTER INTERACTION METHOD, VEHICLE-MOUNTED DEVICE AND READABLE STORAGE MEDIUM

    CHUNG TING-HAO / WANG YU-CHING / HUANG TZU-KUEI et al. | European Patent Office | 2021

    Free access


    Positioning method, vehicle-mounted device and computer readable storage medium

    YANG XIAOLONG / WU YONG | European Patent Office | 2022

    Free access

    Vehicle monitoring method, vehicle-mounted terminal and computer readable storage medium

    ZHUANG JINING / ZHANG LIANG / HUANG QIUGUANG et al. | European Patent Office | 2021

    Free access