This paper applies federated reinforcement learning (FRL) in cellular vehicle-to-everything (C-V2X) communication to enable vehicles to learn communication parameters in collaboration with a parameter server that is embedded in an unmanned aerial vehicle (UAV). Different sensors in vehicles capture different types of data, contributing to data heterogeneity. C-V2X communication networks impose additional communication overhead in order to converge to a global model when the sensor data are not independent-and-identically-distributed (non-i.i.d.). Consequently, the training time for local model updates also varies considerably. Using FRL, we accelerated this convergence by minimizing communication rounds, and we delayed it by exploring the correlation between the data captured by various vehicles in subsequent time steps. Additionally, as UAVs have limited battery power, processing of the collected information locally at the vehicles and then transmitting the model hyper-parameters to the UAVs can optimize the available power consumption pattern. The proposed FRL algorithm updates the global model through adaptive weighing of Q-values at each training round. By measuring the local gradients at the vehicle and the global gradient at the UAV, the contribution of the local models is determined. We quantify these Q-values using nonlinear mappings to reinforce positive rewards such that the contribution of local models is dynamically measured. Moreover, minimizing the number of communication rounds between the UAVs and vehicles is investigated as a viable approach for minimizing delay. A performance evaluation revealed that the FRL approach can yield up to a 40% reduction in the number of communication rounds between vehicles and UAVs when compared to gross data offloading.


    Access

    Download


    Export, share and cite



    Title :

    Federated Reinforcement Learning for Collaborative Intelligence in UAV-Assisted C-V2X Communications


    Contributors:


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Communication-efficient and federated multi-agent reinforcement learning

    Krouka, M. (Mounssif) / Elgabli, A. (Anis) / Issaid, C. B. (Chaouki Ben) et al. | BASE | 2022

    Free access


    A Selective Federated Reinforcement Learning Strategy for Autonomous Driving

    Fu, Yuchuan / Li, Changle / Yu, F. Richard et al. | IEEE | 2023


    RCFL: Redundancy-Aware Collaborative Federated Learning in Vehicular Networks

    Hui, Yilong / Hu, Jie / Cheng, Nan et al. | IEEE | 2024


    Vehicular cooperative perception through action branching and federated reinforcement learning

    Abdel-Aziz, M. K. (Mohamed K.) / Perfecto, C. (Cristina) / Samarakoon, S. (Sumudu) et al. | BASE | 2022

    Free access