Deep Q-Learning with Multiband Sensing for Dynamic Spectrum Access
Ha Q. Nguyen, Binh Thanh Nguyen, Trung Q. Dong, Dat T. Ngo, Anh-Tuan Nguyen · 2018
We study a dynamic spectrum access situation where, in each time slot, a single cognitive agent decides to either stay idle or access one of the N frequency channels based on its sensing of the whole spectrum. The channels are occupied or vacant according to N independent nonidentical 2-state Markov chains. We prove that the optimal access policy can easily be found if the state transition probabilities of all channels are known. When the agent has no knowledge about the channel model, we propose to use the deep Q-learning method to learn a state-action value function that determines an access policy from the observed states of all channels. In this method, the optimal Q-function is approximated with a neural network of all dense layers that is trained via experience replay. We demonstrate through experiments that the learning-based policies consistently achieve performances that are close to the optimal ones.