Neural network simulation is an important tool for generating and evaluating hypotheses on the structure, dynamics and function of neural circuits. For scientific questions addressing organisms operating autonomously in their environments, in particular where learning is involved, it is crucial to be able to operate such simulations in a closed-loop fashion. In such a set-up, the neural agent continuously receives sensory stimuli from the environment and provides motor signals that manipulate the environment or move the agent within it. So far, most studies requiring such functionality have been conducted with custom simulation scripts and manually implemented tasks. This makes it difficult for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. The resulting toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments with various levels of complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym. We compare its performance to a previously suggested neural network model of reinforcement learning in the basal ganglia and a generic Q-learning algorithm.


    Access

    Download


    Export, share and cite



    Title :

    A closed-loop toolchain for neural network simulations of learning autonomous agents


    Contributors:

    Publication date :

    2019-01-01


    Remarks:

    Frontiers in computational neuroscience 13, 46 (2019). doi:10.3389/fncom.2019.00046


    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629 / 006



    BATTERY MANAGEMENT MODEL IN THE LOOP TOOLCHAIN AND SOC ALGORITHM ASSESSMENT

    Joest, D. / Krause, F. / Sauer, D. U. | TIBKAT | 2020


    BATTERY MANAGEMENT MODEL IN THE LOOP TOOLCHAIN AND SOC ALGORITHM ASSESSMENT

    Joest, D. / Krause, F. / Sauer, D. U. | TIBKAT | 2020


    BATTERY MANAGEMENT MODEL IN THE LOOP TOOLCHAIN AND SOC ALGORITHM ASSESSMENT

    Joest, D. / Krause, F. / Sauer, D. U. | TIBKAT | 2020


    Learning Realistic Traffic Agents in Closed-loop

    Zhang, Chris / Tu, James / Zhang, Lunjun et al. | ArXiv | 2023

    Free access

    BATTERY MANAGEMENT MODEL IN THE LOOP TOOLCHAIN AND SOC ALGORITHM ASSESSMENT

    Joest, D. / Krause, F. / Sauer, D. U. | TIBKAT | 2020