Dynamic Spectrum Aggregation and Access Scheme Based on Multi-Agent Actor-Critic Reinforcement Learning
Wenjiao Ding, Wensheng Zhang, Deqiang Wang, Jian Feng Sun, Cheng‐Xiang Wang · 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP) · 2021
In order to relieve the pressure on limited spectrum resources, we investigate a dynamic spectrum aggregation and access problem based on multi-agent reinforcement learning, in which multiple secondary users (SUs) can access multiple fragmented idle channels. At each time slot, multiple SUs with different bandwidth demands, spectrum sensing capabilities, and aggregation capabilities sense channels that are not occupied by primary users (PUs), and aggregate these discrete idle channels to form spectrum segments. SUs select appropriate spectrum segments to access according to their demands. After transmission, SUs receive channel feedback with channel access information to determine whether their transmissions are successful. We propose a maximum entropy based multi-agent actor-critic (ME-MAAC) algorithm to achieve the above work. The simulation results show that the proposed algorithm is effective and can achieve better performance than Deep Q-Network (DQN) algorithm when SUs have different capabilities.