Sequential channel selection for decentralized cognitive radio sensor network based on modified Q-Learning algorithm
Fanzi Zeng, Hanshan Liu, Jisheng Xu · 2016
Cognitive radio sensor network (CRSN) is a novel wireless sensor network, which can effectively alleviate the scarcity of spectrum resource by equipping the cognitive technology. Usually, CRSN deploys numerous sensor nodes around the monitoring area. But the energy and storage of those nodes are both limited. Compared with the centralized CRSN, we focus more on a decentralized situation without the control center or cluster in this paper, especially on a circumstance with few information exchanges among sensor nodes. In the case of that, our goal is to help each sensor node to find an optimal strategy of channel selection in the decentralized CRSN quickly, and then, we apply a Multi-Agent Best-Response Q-Learning algorithm based on Timesharing-Tracking Framework to sequential channel selection for the first time. The algorithm not only has a low storage cost, but also can overcome the poor convergence of strategy caused by Multi-Agent simultaneous learning. For comparison, the optimal channel assignment in centralized network, the Multi-Agent Independent Q-Learning and the Win-or-Learn-Fast Policy Hill Climbing these three algorithms are taken into account, and experimental data validates that our proposed algorithm is feasible in decentralized network.