We develop a deep generative model built on a fully differentiable simulator for multi-agent trajectory prediction. Agents are modeled with conditional recurrent variational neural networks (CVRNNs), which take as input an ego-centric birdview image representing the current state of the world and output an action, consisting of steering and acceleration, which is used to derive the subsequent agent state using a kinematic bicycle model. The full simulation state is then differentiably rendered for each agent, initiating the next time step. We achieve state-of-the-art results on the INTERACTION dataset, using standard neural architectures and a standard variational training objective, producing realistic multi-modal predictions without any ad-hoc diversity-inducing losses. We conduct ablation studies to examine individual components of the simulator, finding that both the kinematic bicycle model and the continuous feedback from the birdview image are crucial for achieving this level of performance. We name our model ITRA, for “Imagining the Road Ahead”.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation


    Contributors:


    Publication date :

    2021-09-19


    Size :

    3600019 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-agent trajectory prediction

    NARAYANAN SRIRAM NOCHUR / LIU BUYU / MOSLEMI RAMIN et al. | European Patent Office | 2023

    Free access

    MULTI-AGENT TRAJECTORY PREDICTION

    NARAYANAN SRIRAM NOCHUR / LIU BUYU / MOSLEMI RAMIN et al. | European Patent Office | 2021

    Free access

    Multi-modal multi-agent trajectory prediction

    SUN PEI / ZHAO HANG / MCCAULEY ALEXANDER et al. | European Patent Office | 2024

    Free access

    MULTI-MODAL MULTI-AGENT TRAJECTORY PREDICTION

    SUN PEI / ZHAO HANG / MCCAULEY ALEXANDER et al. | European Patent Office | 2022

    Free access

    Formation Planning with Multi-Agent Trajectory Prediction

    Wang, Yijie / Wang, Gang / Zhou, Ziyu et al. | IEEE | 2024