As a common multi-functional engineering equipment, excavators are widely used in civil construction, coal mining, power engineering, etc. The excellent performance of the excavator not only greatly improves the work efficiency during the construction process, but also effectively saves labor costs. However, due to the complexity of the working environment of the excavator and the blind area of the excavator itself, the driver cannot make timely judgments on the surrounding environment, which may cause potential threats to pedestrians. In response to such problems, this paper proposes a multi-sensor fusion detection method applied to excavators to provide vision assistance for excavator drivers, thereby reducing the risk of pedestrian casualties. Based on the results of the joint calibration, the transformation relationship between the camera and lidar coordinate systems is determined. Combining the detection results of the pedestrian detection algorithm YOLO-v5 and the segmented image information, the position of the pedestrian in the image can be inversely mapped to the 3D point clouds via the matrix transformation, which can accurately display the position of the pedestrian in the point clouds, consequently making up for the lack of depth information in the image. The experimental results show that our method can effectively extract the location information of pedestrians from the complex background environment and realize timely pedestrian alarm.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Active Pedestrian Detection for Excavator Robots based on Multi-Sensor Fusion


    Contributors:
    Zou, Meiyuan (author) / Yu, Jiajie (author) / Lu, Bo (author) / Chi, Wenzheng (author) / Sun, Lining (author)


    Publication date :

    2022-07-17


    Size :

    1410203 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-ultrasonic sensor fusion for mobile robots

    Zou Yi, / Ho Yeong Khing, / Chua Chin Seng, et al. | IEEE | 2000


    Multi-ultrasonic Sensor Fusion for Mobile Robots

    Yi, Z. / Khing, H. Y. / Seng, C. C. et al. | British Library Conference Proceedings | 2000


    Occlusion aware sensor fusion for early crossing pedestrian detection

    Palffy, Andras / Kooij, Julian F. P. / Gavrila, Dariu M. | IEEE | 2019


    A Scheme on Pedestrian Detection using Multi-Sensor Data Fusion for Smart Roads

    Wang, Hui / Li, Changle / Zhang, Yao et al. | IEEE | 2020


    Sensor fusion with Occupancy Fusion Map for pedestrian detection in outdoor environment

    Lu, Zheyuan / Hu, Zhencheng / Uchimura, Keiichi et al. | IEEE | 2008