A major challenge for autonomous vehicles is when an AV encounters a flagman who regulates traffic near a construction workzone area or at an automotive crash site. In such real-world scenarios, recognizing the flagman's gesture is an essential AV function that has not received much attention. A key piece of earlier work used a chroma-key (green background) screen to generate datasets and evaluate the accuracy of flagman gesture recognition. We find that this background-agnostic approach drops significantly in performance when the chroma-key background is replaced with images from real-world traffic scenarios. In this paper, we extend that baseline approach by adding contextual information and boosting the baseline's accuracy and robustness. First, we replace the chroma-key background to create virtually augmented (VA) images to capture real-world contexts. Next, we propose and evaluate three approaches: (1) Extract CNN features directly from the “raw” VA images, (2) Extracting CNN features after embedding flagman skeleton and prop information in the VA images, and (3) Utilize an attention mechanism via a transformer with padding masks. All three approaches outperform the baseline method. Notably, our Two-Stage Classifier with Transformer approach boosts the baseline's performance from an F-score of 32% to 80% in challenging VA traffic scenarios.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Enhancing Flagman Gesture Recognition with Virtual Augmentation


    Beteiligte:
    He, Mengwen (Autor:in) / Shi, Weijing (Autor:in) / Rajkumar, Ragunathan (Autor:in)


    Erscheinungsdatum :

    08.10.2022


    Format / Umfang :

    2309706 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Electronic Flagman Vehicle Approach Warning System

    Johnson, B. / Olde-Byvank, T. / International Heavy Haul Association | British Library Conference Proceedings | 1991


    Control Virtual Human with Speech Recognition and Gesture Recognition Technology

    Zhao, Wei / Xie, XiaoFang / Yang, XiangHong | Springer Verlag | 2012


    GESTURE RECOGNITION DEVICE AND GESTURE RECOGNITION METHOD

    HIGUCHI DAIKI / OHASHI DAISUKE | Europäisches Patentamt | 2022

    Freier Zugriff

    GESTURE RECOGNITION ASSISTANCE APPARATUS AND GESTURE RECOGNITION METHOD

    KIM JEONG / PARK JANG WOONG | Europäisches Patentamt | 2024

    Freier Zugriff

    GESTURE RECOGNITION SYSTEM AND METHOD FOR GESTURE RECOGNITION

    GERMINIASI DA SILVEIRA DENNIS BROWN | Europäisches Patentamt | 2024

    Freier Zugriff