A Novel Adaptive Call Admission Control Scheme for Distributed Reinforcement Learning Based Dynamic Spectrum Access in Cellular Networks
Nils Morozs, Tim Clarke, David Grace · 2013
Abstract—This paper introduces a novel Q-value based adap-tive call admission control scheme (Q-CAC) for distributed reinforcement learning (RL) based dynamic spectrum access (DSA) in mobile cellular networks, which provides a good quality of service (QoS) without the need for spectrum sensing. A DSA algorithm has been developed in this paper using the stateless Q-learning algorithm with Win-or-Learn-Fast (WoLF) learning rates. Its performance was analysed using the spatial distribution of the probabilities of call blocking (BP) and dropping (DP) across the network and compared to that of a 100 % accurate spectrum sensing based DSA scheme. The Q-CAC scheme demonstrated good controllability of the blocking probability using a Q-value based call admission threshold parameter. It significantly reduced spatial fluctuations in BP and DP, thus providing more cells with acceptable quality of service (QoS).