In this work we present a new approach on the fusion of multi-modal data from visible-light and thermal camera sensors in order to efficiently detect objects in visually-degraded environments using aerial robots. Through a bottom-up approach inspired by the principles of visual saliency and how it models the attention mechanism of mammals, a novel algorithm that builds individual conspicuity maps across color channels and across visible-light and longwave infrared spectra is proposed. More specifically, two input images - a radiometric thermal image and a visible-light RGB image - are fed respectively into an intensity channel, and a set of intensity and color-opponent channels. For every channel, image pyramids are computed to allow searching in different scales. The eventual feature maps are subsequently combined into one saliency map. The approach is computationally efficient and tailored to the limited processing capacities found onboard Micro Aerial Vehicles, while at the same time providing fast real-time object detections as required for the purposes of autonomous navigation and object search. Integrating thermal vision together with visible-light data, the method builds on top of a rich representation that allows resilient operation even in conditions of visual degradation and specifically in cases of low-light or obscurant-filled (e.g., dust, fog) settings which can be penetrated using thermal vision. The presented algorithm and overall system-level solution is built to facilitate efficient, fast and reliable object detection during autonomous aerial robotic exploration of subterranean environments, contributing novel capacity beyond the context of aerial robotic navigation. Our approach is extensively evaluated in the framework of real-life field deployments in underground metal mines in Northern Nevada.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multi-modal Visual-Thermal Saliency-based Object Detection in Visually-degraded Environments


    Beteiligte:


    Erscheinungsdatum :

    01.03.2020


    Format / Umfang :

    9254549 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Autonomous exploration of visually-degraded environments using aerial robots

    Papachristos, Christes / Khattak, Shehryar / Alexis, Kostas | IEEE | 2017


    Robust autonomous flight in constrained and visually degraded shipboard environments

    Fang, Zheng / Yang, Shichao / Jain, Sezal et al. | British Library Online Contents | 2017


    Autonomous UAV Navigation for Target Detection in Visually Degraded and GPS Denied Environments

    Boiteau, Sebastien / Vanegas, Fernando / Sandino, Juan et al. | IEEE | 2023


    Visual-Thermal Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments

    Khattak, Shehryar / Papachristos, Christos / Alexis, Kostas | IEEE | 2019


    Image quality metrics for degraded visual environments

    Blanton, W. Brendan / Allen, Robert C. / Schramm, Erich et al. | SPIE | 2017