Incorporating visual cues to anticipate and describe future events holds significant promise for enhancing user-friendly early warning systems in emergency response scenarios. However, a notable limitation in existing video captioning techniques is their predominant focus on describing ongoing events within observed videos. The challenging task of predicting captions for unobserved videos based on observed visual cues remains largely unaddressed. In response to this gap, we introduce a novel neural network architecture termed the Anticipation Video Captioning Transformer, which is built upon the transformer architecture and comprises three essential modules. The first module serves as a video feature extractor, harnessing the capabilities of a video transformer to extract spatiotemporal features from the observed video data. The second module is a multimodal mask language model to learn the intricate correlations between video content and corresponding captions. The third module is a decoder, generating observed and anticipation video captions. In assessing the efficacy of our proposed method and its potential applicability in emergency scenarios, we have developed a specialized dataset dedicated to aerial refueling anticipation video captioning. Our experimental evaluations encompass a diverse range of qualitative and quantitative analyses, all of which consistently demonstrate the effectiveness of our approach in furnishing user-friendly anticipation captions. Overall, our work represents a significant step forward in video captioning, extending its capabilities beyond merely describing the present state of affairs to encompass the anticipation of future events. This innovation can potentially enhance early warning systems and improve emergency response procedures.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Anticipation Video Captioning of Aerial Refueling Based on Combined Attention Masking Mechanism


    Contributors:
    Wu, Shuai (author) / Tong, Wei (author) / Duan, Ya (author) / Yang, Weidong (author) / Zhu, Guangyu (author) / Wu, Edmond Q. (author)

    Published in:

    Publication date :

    2024-03-01


    Size :

    1778356 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Aerial refueling hose reel mechanism

    YANG PENGTAO / ZHANG GAN'EN / CHU BINBIN et al. | European Patent Office | 2021

    Free access

    Traffic Scenario Understanding and Video Captioning via Guidance Attention Captioning Network

    Liu, Chunsheng / Zhang, Xiao / Chang, Faliang et al. | IEEE | 2024


    Aerial refueling aircraft

    KAYAMA TSUNEO | European Patent Office | 2023

    Free access

    Aerial refueling lamp with hidden refueling function and manufacturing method of aerial refueling lamp

    BI JING / LU JIANFEI / LUO FEI et al. | European Patent Office | 2023

    Free access