In urban environments, complex transportation demands, including land, air, and maritime transport, are increasingly growing. While significant advancements have been made in land transportation and autonomous driving, research on urban air mobility (UAM) systems is still in its early stages. This paper presents an innovative urban air unmanned aerial vehicle (UAV) framework: AirVista, which is designed and built based on the Artificial Systems, Computational experiments, and Parallel execution (ACP) approach, integrated with a multimodal large language model (MLLM) agent. Considering that UAM tasks often require UAVs to possess fine-grained spatial perception and reasoning capabilities, and given that existing MLLMs are somewhat lacking in exploring 3D spaces, this paper further proposes an instruction fine-tuning strategy integrated with 3D spatial knowledge, which has been validated experimentally for its effectiveness. Additionally, to enhance the understanding of the efficiency of MLLM for UAV tasks, this paper delves into prompt fine-tuning templates tailored for UAV task decomposition. Through experimental demonstrations, we showcase that the prompt-tuned MLLM exhibits efficient task decomposition and execution capabilities in handling complex UAV tasks.
AirVista: Empowering UAVs with 3D Spatial Reasoning Abilities Through a Multimodal Large Language Model Agent
24.09.2024
2406670 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Empowering Sustainable Tourism Mobility Through Multimodal Transportation in Rural Areas
Springer Verlag | 2025
|LLMSTP: Empowering Swarm Task Planning with Large Language Models
Springer Verlag | 2025
|Visual-Spatial Abilities of Pilots
NTIS | 1993
|Large Language Models for UAVs: Current State and Pathways to the Future
ArXiv | 2024
|