In machine learning, the ability to reliably determine potential pitfalls in the decision making process of an algorithm is essential. In previous research, the fusion of passive radio frequency (P-RF) histograms generated from in-phase quadrature component (I/Q) data and enhanced electro-optical (EO) data are fused together in order to implement classification and tracking of different vehicle targets using the AFRL's ESCAPE dataset. In previous research, the impact of the P-RF data was confirmed to essential for achieving a higher performance when fused with EO data. This research, however, did not provide an intuitive means by which inferences and explanations for a human expert could interpret. In this paper, saliency maps are implemented in order to visualize the impact of P-RF data in the fusion model and thereby confirm the role the different modalities play in the heterogeneous sensor fusion in an intuitive manner. These visualizations provide context for which pixels activate neurons in the final layer of the model. Overlaying the two modality inputs with respect to time, the method presented in this paper is able to provide explainability for the fusion model while also achieving an F1 score of 0.9. The research in this paper uses the distribution and frequency of the appearance of different types of visualizations. Combined this with context from the scenarios with respect to the timeline of events, it becomes possible to draw inferences for how the two modalities are utilized by the fusion model.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Visualizations of Fusion of Electro Optical (EO) and Passive Radio-Frequency (PRF) Data


    Beteiligte:
    Vakil, Asad (Autor:in) / Blasch, Erik (Autor:in) / Ewing, Robert (Autor:in) / Li, Jia (Autor:in)


    Erscheinungsdatum :

    16.08.2021


    Format / Umfang :

    1272562 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Passive Optical Links Supporting Spaceborne Low Radio Frequency Interferometric Telescope

    Bhide, Subrahmanya V. / Bianconi, Simone / Yi, Lin | IEEE | 2024


    Solar System Visualizations

    Brown, Alison M. | NTRS | 2005


    Research and development of an integrated electro optical and radio frequency aperturel

    DesAutels, G.L. / Welsh, B.M. / Beyerle, A.P. | IEEE | 2003



    3.0202 Research and Development of an Integrated Electro-Optical and Radio Frequency Aperture

    Institute of Electrical and Electronics Engineers | British Library Conference Proceedings | 2003