A number of real-time object detection, tracking, and autonomy artificial intelligence (AI) and machine learning (ML) algorithms are being proposed for unmanned aerial vehicles (UAVs). A big challenge is can we stress test these algorithms, identify their strengths and weaknesses, and assess if the UAV is safe and trustworthy? The process of collecting real-world UAV data is costly, time consuming, and riddled by lack of quality geospatial ground truth and metadata. Herein, we outline a fully automated framework and work ow to address the above challenges using free or low-cost assets, the photorealistic Unreal Engine (UE), and AirSim aerial platform simulator. Specifically, we discuss the rapid prototyping of an outdoor environment combined with the robotic operating system (ROS) for abstracting UAV data collection, control, and processing. Real and accurate ground truth is collected and metrics are presented for individual frame and entire flight collection evaluation. Metrics recorded and analyzed include percentage of scene mapped, 3D mapping accuracy, time to complete task, object detection and tracking statistics, battery usage, altitude (from ground), collisions, and other statistics. These metrics are computed in general and with respect to context, e.g., clutter, view angle, etc. Overall, the proposed work is an automated way to explore UAV operation before real-world testing or deployment. Promising preliminary results are discussed for an outdoor environment with vegetation, short and long range objects, buildings, people, vehicles, and other features for a UAV performing loitering and interrogation.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Unreal engine-based photorealistic aerial data generation and unit testing of artificial intelligence algorithms


    Contributors:

    Conference:

    Geospatial Informatics XII ; 2022 ; Orlando,Florida,United States


    Published in:

    Proc. SPIE ; 12099


    Publication date :

    2022-05-27





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Unreal Engine 4 for Automation

    Pukki, Evgeny | BASE | 2021

    Free access

    Vehicle Modeling in Unreal Engine 4

    Sapienza, Francis / Parker, Michael / Shoop, Sally et al. | TIBKAT | 2022


    Visual Simulator for Quadcopters Based on Unreal Engine

    Chi, Qingkai / Jiang, Pengzhi / Yin, Yongxin | Springer Verlag | 2025



    An Efficient Simulation Platform for Testing and Validating Autonomous Navigation Algorithms for Multi-rotor UAVs Based on Unreal Engine

    Yang, Xinglong / Zou, Danping / Pei, Ling et al. | British Library Conference Proceedings | 2019