On intelligent vehicles, Deep Neural Networks (DNNs) may run on devices whose computational load varies over time. Within the context of variable network architectures, that can be used to constrain the inference latency for real-time deployment with varying system resources, we introduce LeAF (Legacy Augmentation for Flexible inference), a novel paradigm to augment a pre-trained DNN with trainable, shallow execution paths that can run in place of the legacy ones. While preserving the legacy DNN weights, LeAF allows changing the DNN architecture with minimal overhead to effectively adapt to different system performance targets. LeAF-ResNet-50 has less than 14% storage overhead over the legacy DNN; its accuracy varies from the legacy 76.1% to 70.15% (up to 5% better than Slimmable [1] with a latency that is 37% better than OFA [2] on an A100 GPU with batch size 256). Our analysis shows the importance of considering not only the target device, but also the batch size and the temporal dynamic of the DNN configuration to optimize the performances of variable architecture DNNs, LeAF in particular.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Augmenting Legacy Networks for Flexible Inference


    Beteiligte:
    Clemons, Jason (Autor:in) / Frosio, Iuri (Autor:in) / Shen, Maying (Autor:in) / Alvarez, Jose M. (Autor:in) / Keckler, Stephen W. (Autor:in)


    Erscheinungsdatum :

    2023-06-04


    Format / Umfang :

    1325805 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Augmenting Adaptive Approach to Control of Flexible Systems

    Anthony Calise / Bong-Jun Yang / James Craig | AIAA | 2004


    Shape memory alloys for augmenting damping of flexible structures

    Thomson, P. / Balas, G. / Nalbantoglu, V. | AIAA | 1996


    An Augmenting Adaptive Approach to Control of Flexible Systems

    Calise, Anthony / Yang, Bong-Jun / Craig, James | AIAA | 2002