In autonomous and mobile robotics, one of the main challenges is the robust on-the-fly perception of the environment, which is often unknown and dynamic, like in autonomous drone racing. In this work, we propose a novel deep neural network-based perception method for racing gate detection – PencilNet 1 – which relies on a lightweight neural network backbone on top of a pencil filter. This approach unifies predictions of the gates' 2D position, distance, and orientation in a single pose tuple. We show that our method is effective for zero-shot sim-to-real transfer learning that does not need any real-world training samples. Moreover, our framework is highly robust to illumination changes commonly seen under rapid flight compared to state-of-art methods. A thorough set of experiments demonstrates the effectiveness of this approach in multiple challenging scenarios, where the drone completes various tracks under different lighting conditions.


    Access

    Download


    Export, share and cite



    Title :

    PencilNet:Zero-Shot Sim-to-Real Transfer Learning for Robust Gate Perception in Autonomous Drone Racing


    Contributors:

    Publication date :

    2022-10-01


    Remarks:

    Pham , X H , Sarabakha , A , Odnoshyvkin , M & Kayacan , E 2022 , ' PencilNet : Zero-Shot Sim-to-Real Transfer Learning for Robust Gate Perception in Autonomous Drone Racing ' , IEEE Robotics and Automation Letters , vol. 7 , no. 4 , pp. 11847 - 11854 . https://doi.org/10.1109/LRA.2022.3207545



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629



    Are We Ready for Autonomous Drone Racing? The UZH-FPV Drone Racing Dataset

    Delmerico, Jeffrey / Cieslewski, Titus / Rebecq, Henri et al. | BASE | 2019

    Free access

    Real Time Embedded System Framework for Autonomous Drone Racing using Deep Learning Techniques

    Jung, Sunggoo / Lee, Hanseob / Hwang, Sunyou et al. | AIAA | 2018


    Overcoming the Blind Spot in CNN-based Gate Detection for Autonomous Drone Racing

    Cocoma-Ortega, J. Arturo / Rojas-Perez, L. Oyuki / Cabrera-Ponce, Aldrich A. et al. | IEEE | 2019


    Drone Racing Frame

    RHEE JUN WHI | European Patent Office | 2020

    Free access

    Real Time Embedded System Framework for Autonomous Drone Racing using Deep Learning Techniques (AIAA 2018-2138)

    Jung, Sunggoo / Lee, Hanseob / Hwang, Sunyou et al. | British Library Conference Proceedings | 2018