Recognizing human actions accurately from video data is a difficult task because of the variability in human motion, appearance, and context. A new approach to human action recognition is proposed in this study, which uses a combination of Variational Autoencoder (VAE) and Transformer models. The VAE with IWGECO optimization is utilized for acquiring a compact and discriminative representation of the input 2D skeletons from video frames along with its spatial relations. This encoded representation of 2D poses is then fed into a Transformer model for sequence modeling and classification, resulting in effective recognition of human actions. The proposed approach is evaluated on the MPOSE2021 benchmark dataset, outperforming existing methods (90.6%) with an average classification accuracy of 92.0%. The model performs well on real-life data too showcasing its generalizing potential. The study also addresses the domain shift problem occurring in real-world situations by using a pre-trained VAE and subsequently training the Transformer on the MPOSE2021 dataset. The results demonstrate the effectiveness of combining VAE and Transformer models for human action recognition.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Improved ELBO-assisted Transformer for Skeleton-Based Action Recognition


    Contributors:


    Publication date :

    2023-09-24


    Size :

    8172729 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Resetting a fixed broken ELBO

    Cukier, Robert I. | ArXiv | 2023

    Free access

    Multimodal ELBO with Diffusion Decoders

    Wesego, Daniel / Rooshenas, Pedram | ArXiv | 2024

    Free access

    Deep Dive into Semi-Supervised ELBO for Improving Classification Performance

    Niloy, Fahim Faisal / Amin, M. Ashraful / Rahman, AKM Mahbubur et al. | ArXiv | 2021

    Free access

    The ELBO of Variational Autoencoders Converges to a Sum of Three Entropies

    Damm, Simon / Forster, Dennis / Velychko, Dmytro et al. | ArXiv | 2020

    Free access

    Driver-Skeleton: A Dataset for Driver Action Recognition

    Lin, Zeyang / Liu, Yinchuan / Zhang, Xuetao | IEEE | 2021