The invention provides an unmanned aerial vehicle group communication and sensing integrated energy consumption optimization method based on reinforcement learning, and the method comprises the following steps: 1, assigning an initial value to a Q value function network which reflects a state and action relation through the positioning and sensing performance of an unmanned aerial vehicle group; 2, judging a current state; selecting a certain working point as an initial state; 3, selecting a current action based on an epsilon-greedy strategy according to the current state; 4, introducing the perception performance of the unmanned aerial vehicle and the energy consumption of the communication task into the design of a Reward reward function, and obtaining the actual environment reward value of the last-step action and the next state; 5, updating the Q value function network by using the actual environment reward value of the action in the previous step; and 6, setting the new state as the current state, and repeating the steps 3-6 until the value in the Q value function network is converged. According to the method, the problem of relatively high energy consumption in the ISAC network based on the unmanned aerial vehicle group in the prior art can be solved, meanwhile, the real-time service performance of communication and positioning of terminal ground users is ensured, and the service life of the network is effectively prolonged.
本发明提出一种基于强化学习的无人机群通感一体能耗优化方法,包括如下步骤:一,利用无人机集群的定位感知性能,为反映状态与动作关系的Q值函数网络赋初值;二,判断当前状态;选定某一工作点作为初始状态;三,根据当前状态,基于ε‑greedy策略来选择当前动作;四,将无人机感知性能和通信任务能量消耗引入Reward奖励函数的设计中,并获取上一步动作的实际环境奖励值,以及下一个状态;五,利用上一步动作的实际环境奖励值,更新Q值函数网络;六,将新状态设置为当前状态,重复三~六,直到Q值函数网络中的值达到收敛。本发明能解决现有技术在基于无人机群的ISAC网络中能耗较大的问题,同时保障终端地面用户的通信和定位的实时服务性能,有效改善网络服务寿命。
Unmanned aerial vehicle group communication and sensing integrated energy consumption optimization method based on reinforcement learning
基于强化学习的无人机群通感一体能耗优化方法
2023-10-17
Patent
Electronic Resource
Chinese
Reinforcement Learning Based Communication Security for Unmanned Aerial Vehicles
Springer Verlag | 2021
|Unmanned aerial vehicle and unmanned aerial vehicle group
European Patent Office | 2021
|European Patent Office | 2024
|Hydrogen fuel unmanned aerial vehicle energy management method based on deep reinforcement learning
European Patent Office | 2021
|Multi-unmanned aerial vehicle communication rate optimization method
European Patent Office | 2024
|