In urban environments, complex transportation demands, including land, air, and maritime transport, are increasingly growing. While significant advancements have been made in land transportation and autonomous driving, research on urban air mobility (UAM) systems is still in its early stages. This paper presents an innovative urban air unmanned aerial vehicle (UAV) framework: AirVista, which is designed and built based on the Artificial Systems, Computational experiments, and Parallel execution (ACP) approach, integrated with a multimodal large language model (MLLM) agent. Considering that UAM tasks often require UAVs to possess fine-grained spatial perception and reasoning capabilities, and given that existing MLLMs are somewhat lacking in exploring 3D spaces, this paper further proposes an instruction fine-tuning strategy integrated with 3D spatial knowledge, which has been validated experimentally for its effectiveness. Additionally, to enhance the understanding of the efficiency of MLLM for UAV tasks, this paper delves into prompt fine-tuning templates tailored for UAV task decomposition. Through experimental demonstrations, we showcase that the prompt-tuned MLLM exhibits efficient task decomposition and execution capabilities in handling complex UAV tasks.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    AirVista: Empowering UAVs with 3D Spatial Reasoning Abilities Through a Multimodal Large Language Model Agent


    Contributors:
    Lin, Fei (author) / Tian, Yonglin (author) / Wang, Yunzhe (author) / Zhang, Tengchao (author) / Zhang, Xinyuan (author) / Wang, Fei-Yue (author)


    Publication date :

    2024-09-24


    Size :

    2406670 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Empowering Sustainable Tourism Mobility Through Multimodal Transportation in Rural Areas

    Baumann, Mirjam / Schubert, Iljana / Del Duce, Andrea | Springer Verlag | 2025

    Free access

    LLMSTP: Empowering Swarm Task Planning with Large Language Models

    Yu, Hongbo / Wang, Chang / Wu, Lizhen et al. | Springer Verlag | 2025


    Visual-Spatial Abilities of Pilots

    I. E. Dror / S. M. Kosslyn / W. L. Waag | NTIS | 1993



    Large Language Models for UAVs: Current State and Pathways to the Future

    Javaid, Shumaila / Saeed, Nasir / He, Bin | ArXiv | 2024

    Free access