With high-definition sensors and sophisticated machine vision algorithms, the visual perception capability of autonomous vehicle (AV) has largely advanced. However, the visual perception performance of AVs may still be unstable in complex traffic environment. To improve the robustness and capability of risk detection of AV visual perception system, this work proposes a framework to fuse human gaze and the object detection results from vehicle vision based on the Laplacian Pyramid algorithm. We evaluate the proposed method on a level-2 AV to perceive the interactive vehicles at unsignalized intersections. Using Extended Kalman Filter, the trajectory of the human drivers’ gaze and the anchor boxes from AV object detection are fused. Results reveal that with human-vehicle visual fusion, the actual trajectory of interactive vehicles can be predicted more accurately than separately using human gaze or object detection algorithm. The findings show that human-vehicle visual fusion improves the perception accuracy and robustness of interactive objects in complex traffic environment. The method has the potential to enhance the attention mechanism of AV vision.
Improving Autonomous Vehicle Visual Perception by Fusing Human Gaze and Machine Vision
IEEE Transactions on Intelligent Transportation Systems ; 24 , 11 ; 12716-12725
2023-11-01
3122332 byte
Article (Journal)
Electronic Resource
English
EMS-Vision: Gaze Control in Autonomous Vehicles
British Library Conference Proceedings | 2000
|EMS-vision: gaze control in autonomous vehicles
IEEE | 2000
|Vehicle detection fusing 2D visual features
IEEE | 2004
|MPP1.01 Vehicle Detection Fusing 2D Visual Features
British Library Conference Proceedings | 2004
|On-road vehicle detection fusing radar and vision
IEEE | 2011
|