The invention discloses an intelligent screen control system based on a large scenarized model, which belongs to the technical field of intelligent vehicle-mounted devices, and comprises a semantic understanding agent used for identifying user requirements according to user text contents and generating operation requirements; the scenarized interface analysis agent is used for performing analysis according to the screenshot image, identifying an interface menu and interface elements and generating a scenarized operation list; the matching agent is used for matching the operation demand with the scenarized operation list, generating an operation instruction and performing interaction determination on the operation instruction and a user; the simulation interaction agent is used for simulating the operation of a user on a vehicle machine screen according to the operation instruction and converting the simulation operation into an operation sequence which can be executed by a vehicle machine operation system; the multi-mode execution agent is used for sending the operation sequence to the vehicle machine; according to the system, through cooperative work of the multi-agent module, accurate recognition and response to user requirements are achieved, and corresponding vehicle-machine interaction operation is generated.

    本发明公开了一种基于场景化大模型的智能屏幕操控系统,属于智能车载技术领域,包括:语义理解智能体,用于根据用户文字内容识别用户需求,生成操作需求;场景化界面分析智能体,用于根据截屏图像进行分析,识别界面菜单和界面元素,生成场景化操作列表;匹配智能体,用于对操作需求和场景化操作列表进行匹配,生成操作指令,并将操作指令与用户交互确定;模拟交互智能体,用于根据操作指令模拟用户在车机屏幕上的操作,并将模拟操作转化为车机操作系统可执行的操作序列;多模执行智能体,用于将操作序列发送车机;该系统,通过多智能体模块的协同工作,实现对用户需求的准确识别和响应,并生成相应的车机交互操作。


    Access

    Download


    Export, share and cite



    Title :

    Intelligent screen control system based on large scenarized model


    Additional title:

    一种基于场景化大模型的智能屏幕操控系统


    Contributors:
    LI JINGDA (author)

    Publication date :

    2024-12-31


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G10L Analyse oder Synthese von Sprache , SPEECH ANALYSIS OR SYNTHESIS / B60K Anordnung oder Einbau von Antriebseinheiten oder von Kraft- bzw. Drehmomentübertragungen in Fahrzeugen , ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES / B60R Fahrzeuge, Fahrzeugausstattung oder Fahrzeugteile, soweit nicht anderweitig vorgesehen , VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06V




    Scenarized FAO fault handling simulation system

    XING YIHUA / SUN JIYING / YANG YANG et al. | European Patent Office | 2024

    Free access


    Vehicle scenarized power distribution method and device, medium and equipment

    ZHOU HAOFU / FANG WEIJIA / SI HUACHAO et al. | European Patent Office | 2024

    Free access

    Scenarized interaction method and system based on 3D engine, electronic equipment, storage medium and vehicle

    YU HAO / ZHENG HONGLI / ZHAO MOHAN et al. | European Patent Office | 2023

    Free access