Incorporating visual cues to anticipate and describe future events holds significant promise for enhancing user-friendly early warning systems in emergency response scenarios. However, a notable limitation in existing video captioning techniques is their predominant focus on describing ongoing events within observed videos. The challenging task of predicting captions for unobserved videos based on observed visual cues remains largely unaddressed. In response to this gap, we introduce a novel neural network architecture termed the Anticipation Video Captioning Transformer, which is built upon the transformer architecture and comprises three essential modules. The first module serves as a video feature extractor, harnessing the capabilities of a video transformer to extract spatiotemporal features from the observed video data. The second module is a multimodal mask language model to learn the intricate correlations between video content and corresponding captions. The third module is a decoder, generating observed and anticipation video captions. In assessing the efficacy of our proposed method and its potential applicability in emergency scenarios, we have developed a specialized dataset dedicated to aerial refueling anticipation video captioning. Our experimental evaluations encompass a diverse range of qualitative and quantitative analyses, all of which consistently demonstrate the effectiveness of our approach in furnishing user-friendly anticipation captions. Overall, our work represents a significant step forward in video captioning, extending its capabilities beyond merely describing the present state of affairs to encompass the anticipation of future events. This innovation can potentially enhance early warning systems and improve emergency response procedures.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Anticipation Video Captioning of Aerial Refueling Based on Combined Attention Masking Mechanism


    Beteiligte:
    Wu, Shuai (Autor:in) / Tong, Wei (Autor:in) / Duan, Ya (Autor:in) / Yang, Weidong (Autor:in) / Zhu, Guangyu (Autor:in) / Wu, Edmond Q. (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.03.2024


    Format / Umfang :

    1778356 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Aerial refueling hose reel mechanism

    YANG PENGTAO / ZHANG GAN'EN / CHU BINBIN et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    Traffic Scenario Understanding and Video Captioning via Guidance Attention Captioning Network

    Liu, Chunsheng / Zhang, Xiao / Chang, Faliang et al. | IEEE | 2024


    Aerial refueling aircraft

    KAYAMA TSUNEO | Europäisches Patentamt | 2023

    Freier Zugriff

    Aerial refueling lamp with hidden refueling function and manufacturing method of aerial refueling lamp

    BI JING / LU JIANFEI / LUO FEI et al. | Europäisches Patentamt | 2023

    Freier Zugriff