Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving. Due to the importance of these components, there has been a significant amount of interest and research in perception and trajectory forecasting, resulting in a wide variety of approaches. Common to most works, however, is the use of the same few accuracy-based evaluation metrics, e.g., intersection-over-union, displacement error, log-likelihood, etc. While these metrics are informative, they are task-agnostic and outputs that are evaluated as equal can lead to vastly different outcomes in downstream planning and decision making. In this work, we take a step back and critically assess current evaluation metrics, proposing task-aware metrics as a better measure of performance in systems where they are deployed. Experiments on an illustrative simulation as well as real-world autonomous driving data validate that our proposed task-aware metrics are able to account for outcome asymmetry and provide a better estimate of a model’s closed-loop performance.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Injecting Planning-Awareness into Prediction and Detection Evaluation


    Beteiligte:
    Ivanovic, Boris (Autor:in) / Pavone, Marco (Autor:in)


    Erscheinungsdatum :

    2022-06-05


    Format / Umfang :

    3354496 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch






    Advanced long products - injecting efficiency into modern engines

    Renz, Gerhard / Henze, Alexandra / Janßen, Peter | Tema Archiv | 2007


    DEVICE FOR INJECTING IONS INTO A STREAM OF AIR

    YAM GAN SEM / MENG TEO CHOON | Europäisches Patentamt | 2016

    Freier Zugriff