Low-altitude unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), which boast high-resolution imaging and agile maneuvering capabilities, are widely utilized in military scenarios and generate a vast amount of image data that can be leveraged for textual intelligence generation to support military decision making. Military image captioning (MilitIC), as a visual-language learning task, provides innovative solutions for military image understanding and intelligence generation. However, the scarcity of military image datasets hinders the advancement of MilitIC methods, especially those based on deep learning. To overcome this limitation, we introduce an open-access benchmark dataset, which was termed the Military Objects in Real Combat (MOCO) dataset. It features real combat images captured from the perspective of low-altitude UAVs or UGVs, along with a comprehensive set of captions. Furthermore, we propose a novel encoder–augmentation–decoder image-captioning architecture with a map augmentation embedding (MAE) mechanism, MAE-MilitIC, which leverages both image and text modalities as a guiding prefix for caption generation and bridges the semantic gap between visual and textual data. The MAE mechanism maps both image and text embeddings onto a semantic subspace constructed by relevant military prompts, and augments the military semantics of the image embeddings with attribute-explicit text embeddings. Finally, we demonstrate through extensive experiments that MAE-MilitIC surpasses existing models in performance on two challenging datasets, which provides strong support for intelligence warfare based on military UAVs and UGVs.


    Access

    Download


    Export, share and cite



    Title :

    Military Image Captioning for Low-Altitude UAV or UGV Perspectives


    Contributors:
    Lizhi Pan (author) / Chengtian Song (author) / Xiaozheng Gan (author) / Keyu Xu (author) / Yue Xie (author)


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Enhanced Dense Image Captioning Based On Transformers

    Goswami, Tilottama / Potu, Sathvika / Reddy, Kuntluri Prasanna et al. | IEEE | 2024


    Remote Sensing Image Captioning Using Transformer

    Wang, Binze / Xi, Jiangbo / Wang, Xingrun et al. | British Library Conference Proceedings | 2022


    Remote Sensing Image Captioning Using Transformer

    Wang, Binze / Xi, Jiangbo / Wang, Xingrun et al. | Springer Verlag | 2022