With high-definition sensors and sophisticated machine vision algorithms, the visual perception capability of autonomous vehicle (AV) has largely advanced. However, the visual perception performance of AVs may still be unstable in complex traffic environment. To improve the robustness and capability of risk detection of AV visual perception system, this work proposes a framework to fuse human gaze and the object detection results from vehicle vision based on the Laplacian Pyramid algorithm. We evaluate the proposed method on a level-2 AV to perceive the interactive vehicles at unsignalized intersections. Using Extended Kalman Filter, the trajectory of the human drivers’ gaze and the anchor boxes from AV object detection are fused. Results reveal that with human-vehicle visual fusion, the actual trajectory of interactive vehicles can be predicted more accurately than separately using human gaze or object detection algorithm. The findings show that human-vehicle visual fusion improves the perception accuracy and robustness of interactive objects in complex traffic environment. The method has the potential to enhance the attention mechanism of AV vision.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Improving Autonomous Vehicle Visual Perception by Fusing Human Gaze and Machine Vision


    Contributors:
    Zhao, Yiyue (author) / Lei, Cailin (author) / Shen, Yu (author) / Du, Yuchuan (author) / Chen, Qijun (author)

    Published in:

    Publication date :

    2023-11-01


    Size :

    3122332 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    EMS-Vision: Gaze Control in Autonomous Vehicles

    Pellkofer, M. / Dickmanns, E. D. / IEEE | British Library Conference Proceedings | 2000


    EMS-vision: gaze control in autonomous vehicles

    Pellkofer, M. / Dickmanns, E.D. | IEEE | 2000


    Vehicle detection fusing 2D visual features

    Hoffman, C. / Dang, T. / Stiller, C. | IEEE | 2004


    MPP1.01 Vehicle Detection Fusing 2D Visual Features

    Hoffmann, C. / Dang, T. / Stiller, C. et al. | British Library Conference Proceedings | 2004


    On-road vehicle detection fusing radar and vision

    Xin Liu, / Zhenping Sun, / Hangen He, | IEEE | 2011