In many real-world settings, a team of agents must coordinate its behaviour while acting in a decentralised fashion. At the same time, it is often possible to train the agents in a centralised fashion where global state information is available and communication constraints are lifted. Learning joint action-values conditioned on extra state information is an attractive way to exploit centralised learning, but the best strategy for then extracting decentralised policies is unclear. Our solution is QMIX, a novel value-based method that can train decentralised policies in a centralised end-to-end fashion. QMIX employs a mixing network that estimates joint action-values as a monotonic combination of per-agent values. We structurally enforce that the joint-action value is monotonic in the per-agent values, through the use of non-negative weights in the mixing network, which guarantees consistency between the centralised and decentralised policies. To evaluate the performance of QMIX, we propose the StarCraft Multi-Agent Challenge (SMAC) as a new benchmark for deep multi-agent reinforcement learning. We evaluate QMIX on a challenging set of SMAC scenarios and show that it significantly outperforms existing multi-agent reinforcement learning methods.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Monotonic value function factorisation for deep multi-agent reinforcement learning


    Beteiligte:
    Rashid, T (Autor:in) / Samvelyan, M (Autor:in) / Schröder de Witt, C (Autor:in) / Farquhar, G (Autor:in) / Foerster, JN (Autor:in) / Whiteson, S (Autor:in)

    Erscheinungsdatum :

    2020-09-11


    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    006 / 629




    Multi-Agent Deep Reinforcement Learning in Vehicular OCC

    Islam, Amirul / Musavian, Leila / Thomos, Nikolaos | IEEE | 2022


    DEEP REINFORCEMENT LEARNING FOR MULTI-AGENT AUTONOMOUS SATELLITE INSPECTION

    Lei, Henry H. / Shubert, Matt / Damron, Nathan et al. | Springer Verlag | 2024


    Autonomous Separation Assurance with Deep Multi-Agent Reinforcement Learning

    Brittain, Marc W. / Yang, Xuxi / Wei, Peng | AIAA | 2021


    Coordination and communication in deep multi-agent reinforcement learning

    Schroeder de Witt, CA | BASE | 2022

    Freier Zugriff