A number of real-time object detection, tracking, and autonomy artificial intelligence (AI) and machine learning (ML) algorithms are being proposed for unmanned aerial vehicles (UAVs). A big challenge is can we stress test these algorithms, identify their strengths and weaknesses, and assess if the UAV is safe and trustworthy? The process of collecting real-world UAV data is costly, time consuming, and riddled by lack of quality geospatial ground truth and metadata. Herein, we outline a fully automated framework and work ow to address the above challenges using free or low-cost assets, the photorealistic Unreal Engine (UE), and AirSim aerial platform simulator. Specifically, we discuss the rapid prototyping of an outdoor environment combined with the robotic operating system (ROS) for abstracting UAV data collection, control, and processing. Real and accurate ground truth is collected and metrics are presented for individual frame and entire flight collection evaluation. Metrics recorded and analyzed include percentage of scene mapped, 3D mapping accuracy, time to complete task, object detection and tracking statistics, battery usage, altitude (from ground), collisions, and other statistics. These metrics are computed in general and with respect to context, e.g., clutter, view angle, etc. Overall, the proposed work is an automated way to explore UAV operation before real-world testing or deployment. Promising preliminary results are discussed for an outdoor environment with vegetation, short and long range objects, buildings, people, vehicles, and other features for a UAV performing loitering and interrogation.
Unreal engine-based photorealistic aerial data generation and unit testing of artificial intelligence algorithms
Geospatial Informatics XII ; 2022 ; Orlando,Florida,United States
Proc. SPIE ; 12099
2022-05-27
Conference paper
Electronic Resource
English
Vehicle Modeling in Unreal Engine 4
TIBKAT | 2022
|Visual Simulator for Quadcopters Based on Unreal Engine
Springer Verlag | 2025
|British Library Conference Proceedings | 2019
|