A major challenge for autonomous vehicles is when an AV encounters a flagman who regulates traffic near a construction workzone area or at an automotive crash site. In such real-world scenarios, recognizing the flagman's gesture is an essential AV function that has not received much attention. A key piece of earlier work used a chroma-key (green background) screen to generate datasets and evaluate the accuracy of flagman gesture recognition. We find that this background-agnostic approach drops significantly in performance when the chroma-key background is replaced with images from real-world traffic scenarios. In this paper, we extend that baseline approach by adding contextual information and boosting the baseline's accuracy and robustness. First, we replace the chroma-key background to create virtually augmented (VA) images to capture real-world contexts. Next, we propose and evaluate three approaches: (1) Extract CNN features directly from the “raw” VA images, (2) Extracting CNN features after embedding flagman skeleton and prop information in the VA images, and (3) Utilize an attention mechanism via a transformer with padding masks. All three approaches outperform the baseline method. Notably, our Two-Stage Classifier with Transformer approach boosts the baseline's performance from an F-score of 32% to 80% in challenging VA traffic scenarios.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Enhancing Flagman Gesture Recognition with Virtual Augmentation


    Contributors:


    Publication date :

    2022-10-08


    Size :

    2309706 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Electronic Flagman Vehicle Approach Warning System

    Johnson, B. / Olde-Byvank, T. / International Heavy Haul Association | British Library Conference Proceedings | 1991


    Control Virtual Human with Speech Recognition and Gesture Recognition Technology

    Zhao, Wei / Xie, XiaoFang / Yang, XiangHong | Springer Verlag | 2012


    GESTURE RECOGNITION DEVICE AND GESTURE RECOGNITION METHOD

    HIGUCHI DAIKI / OHASHI DAISUKE | European Patent Office | 2022

    Free access

    GESTURE RECOGNITION ASSISTANCE APPARATUS AND GESTURE RECOGNITION METHOD

    KIM JEONG / PARK JANG WOONG | European Patent Office | 2024

    Free access

    GESTURE RECOGNITION SYSTEM AND METHOD FOR GESTURE RECOGNITION

    GERMINIASI DA SILVEIRA DENNIS BROWN | European Patent Office | 2024

    Free access