Abstract In this paper, we consider a distributed reinforcement learning setting where agents are communicating with a central entity in a shared environment to maximize a global reward. A main challenge in this setting is that the randomness of the wireless channel perturbs each agent’s model update while multiple agents’ updates may cause interference when communicating under limited bandwidth. To address this issue, we propose a novel distributed reinforcement learning algorithm based on the alternating direction method of multipliers (ADMM) and “over air aggregation” using analog transmission scheme, referred to as A-RLADMM. Our algorithm incorporates the wireless channel into the formulation of the ADMM method, which enables agents to transmit each element of their updated models over the same channel using analog communication. Numerical experiments on a multi-agent collaborative navigation task show that our proposed algorithm significantly outperforms the digital communication baseline of A-RLADMM (DRLADMM), the lazily aggregated policy gradient (RL-LAPG), as well as the analog and the digital communication versions of the vanilla FL, (A-FRL) and (D-FRL) respectively.


    Access

    Download


    Export, share and cite



    Title :

    Communication-efficient and federated multi-agent reinforcement learning



    Publication date :

    2022-01-01


    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629





    A Selective Federated Reinforcement Learning Strategy for Autonomous Driving

    Fu, Yuchuan / Li, Changle / Yu, F. Richard et al. | IEEE | 2023


    Experience generalization for multi-agent reinforcement learning

    Pegoraro, Renê / Costa, AHR / Ribeiro, CHC | BASE | 2001

    Free access

    Vehicular cooperative perception through action branching and federated reinforcement learning

    Abdel-Aziz, M. K. (Mohamed K.) / Perfecto, C. (Cristina) / Samarakoon, S. (Sumudu) et al. | BASE | 2022

    Free access