Stable and reliable wireless communications are absolutely essential in achieving safe, efficient, and comfortable autonomous driving systems, and require more channel resources. Cognitive radio (CR) alleviates the spectrum shortage by exploiting spectrum resources that are not used by primary users (PUs). Conventional CR-based vehicle-to-everything (V2X) communications still suffer from spectrum scarcity when the demands for wireless resources are excessive. Most previous analyses of this problem have adopted too simple models for V2X that assume constant vehicle velocity and that coarsely discretize vehicle positions. More realistic models for V2X are too topologically complex and too dynamically variant to find stable solutions of optimization problems by using conventional off-policy-based reinforcement learning (RL) algorithms such as Q-learning and deep Q-network (DQN) based on reply memories. Considering the above situation, this paper firstly builds a precise and realistic autonomous driving testbed by 3D engine. In the testbed, all vehicles move in an autopilot mode with random velocity and direction, which can provide more realistic traffic flow, positions, and motion for all vehicles. Due to the constrain of Kullback-Leibler divergence between updated and previous policies, on-policy RL algorithms such as proximal policy optimization (PPO) can find stable solutions to complex optimization problems. Inspired by this property, we expand the single-agent RL algorithm PPO into a multi-agent algorithm MA-PPO (multi-action proximal policy optimization). Computer simulations using the new testbed show the MA-PPO algorithm can gradually converge to more stable solutions and can achieve more steady and efficient data transmission than DQN.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Reinforcement Learning-Based Cognitive Radio Transmission Scheduling in Vehicular Systems


    Contributors:
    Li, Yun (author) / Chang, Yuyuan (author) / Fukawa, Kazuhiko (author) / Kodama, Naoki (author)


    Publication date :

    2023-06-01


    Size :

    2626266 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Deep Reinforcement Learning based Adaptive Transmission Control in Vehicular Networks

    Liu, Mingyuan / Quan, Wei / Yu, Chengxiao et al. | IEEE | 2021


    AoI-Aware Dynamic User Scheduling in Vehicular Networks Based on Soft Reinforcement Learning

    Huang, Zhisen / Guo, Chongtao / Chen, Jiayi et al. | IEEE | 2023


    Robust beamforming for cognitive radio based vehicular communication

    Alam, Md Monzurul / Bhattarai, Sudeep / Hong, Liang et al. | IEEE | 2013


    Multi-Objective Reinforcement Learning for Cognitive Radio-Based Satellite Communications

    Ferreira, Paulo Victor R. / Paffenroth, Randy / Wyglinski, Alexander M. et al. | NTRS | 2016


    VEHICULAR RADIO WAVE TRANSMISSION COMPONENT

    KITAYAMA KEIICHI / MORI SATOSHI / NARITA YASUHIDE et al. | European Patent Office | 2019

    Free access