In urban environments, complex transportation demands, including land, air, and maritime transport, are increasingly growing. While significant advancements have been made in land transportation and autonomous driving, research on urban air mobility (UAM) systems is still in its early stages. This paper presents an innovative urban air unmanned aerial vehicle (UAV) framework: AirVista, which is designed and built based on the Artificial Systems, Computational experiments, and Parallel execution (ACP) approach, integrated with a multimodal large language model (MLLM) agent. Considering that UAM tasks often require UAVs to possess fine-grained spatial perception and reasoning capabilities, and given that existing MLLMs are somewhat lacking in exploring 3D spaces, this paper further proposes an instruction fine-tuning strategy integrated with 3D spatial knowledge, which has been validated experimentally for its effectiveness. Additionally, to enhance the understanding of the efficiency of MLLM for UAV tasks, this paper delves into prompt fine-tuning templates tailored for UAV task decomposition. Through experimental demonstrations, we showcase that the prompt-tuned MLLM exhibits efficient task decomposition and execution capabilities in handling complex UAV tasks.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    AirVista: Empowering UAVs with 3D Spatial Reasoning Abilities Through a Multimodal Large Language Model Agent


    Beteiligte:
    Lin, Fei (Autor:in) / Tian, Yonglin (Autor:in) / Wang, Yunzhe (Autor:in) / Zhang, Tengchao (Autor:in) / Zhang, Xinyuan (Autor:in) / Wang, Fei-Yue (Autor:in)


    Erscheinungsdatum :

    24.09.2024


    Format / Umfang :

    2406670 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Empowering Sustainable Tourism Mobility Through Multimodal Transportation in Rural Areas

    Baumann, Mirjam / Schubert, Iljana / Del Duce, Andrea | Springer Verlag | 2025

    Freier Zugriff

    LLMSTP: Empowering Swarm Task Planning with Large Language Models

    Yu, Hongbo / Wang, Chang / Wu, Lizhen et al. | Springer Verlag | 2025


    Visual-Spatial Abilities of Pilots

    I. E. Dror / S. M. Kosslyn / W. L. Waag | NTIS | 1993



    Large Language Models for UAVs: Current State and Pathways to the Future

    Javaid, Shumaila / Saeed, Nasir / He, Bin | ArXiv | 2024

    Freier Zugriff