The invention belongs to the technical field of fatigue driving detection, and particularly relates to a large model cue word multi-scale spatio-temporal feature fusion fatigue driving detection method, which comprises the following steps of: generating a corresponding cue word by utilizing a pre-trained large model, extracting a semantic embedding vector of the cue word, performing multi-scale sampling on a video frame input into a mixed expert model, and extracting a semantic embedding vector of the cue word; the method comprises the following steps of: cascading cue word features with video sequence channels of various scales, fusing multi-scale features to generate a micro-expression space-time motion feature map, inputting the micro-expression space-time motion feature map into a lightweight Transform network, combining a dynamically updated comprehensive learning vector, outputting a fatigue probability through a classifier, and judging whether fatigue driving exists or not based on threshold early warning. And finally, training the hybrid expert model by using a cross entropy loss function in combination with an optimizer. According to the method, subtle changes of facial expressions of the driver can be comprehensively captured, space-time motion features are efficiently modeled through guidance of large model cues, and an efficient and accurate solution is provided for fatigue driving detection of the vehicle driver.
本发明属于疲劳驾驶检测技术领域,具体涉及一种大模型提示词多尺度时空特征融合疲劳驾驶检测方法,该方法利用预训练大模型生成对应提示词并提取提示词的语义嵌入向量,然后对输入混合专家模型的视频帧多尺度采样,将提示词特征与各尺度视频序列通道级联,融合多尺度特征生成微表情时空运动特征图,并输入轻量化Transformer网络,结合动态更新的综合学习向量,通过分类器输出疲劳概率,基于阈值预警判断是否疲劳驾驶,最后使用交叉熵损失函数结合优化器对混合专家模型进行训练。该方法能够全面捕捉驾驶员面部表情细微变化,并通过大模型提示词引导,高效建模时空运动特征,为车辆驾驶员疲劳驾驶检测提供了一种高效、精准的解决方案。
Large model cue word multi-scale spatial-temporal feature fusion fatigue driving detection method
大模型提示词多尺度时空特征融合疲劳驾驶检测方法
2025-04-15
Patent
Electronic Resource
Chinese
IPC: | G06V / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion |
Fatigue Driving Detection Based on Multi Feature Fusion
IEEE | 2019
|Fatigue driving detection method and system based on multi-modal feature fusion
European Patent Office | 2025
|Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning
IET | 2019
|Fatigue driving detection model based on multi‐feature fusion and semi‐supervised active learning
Wiley | 2019
|