Single‐state Q‐learning for self‐organised radio resource management in dual‐hop 5G high capacity density networks
Tao Jiang, Qiyang Zhao, David Grace, Alister Graham Burr, Tim Clarke · Transactions on Emerging Telecommunications Technologies · 2016
ABSTRACT In this paper, a dual‐hop wireless backhaul and small cell access network has been exploited with effective spectrum sharing to provide 1 Gb/s/km2 ultra‐high capacity density for 5G ultra‐dense network deployments. We develop a single‐state Q‐learning‐based radio resource management algorithm for dynamic spectrum access creating a self‐organised network. It intelligently utilises the instantaneous spectrum observation information from spectrum sensing or a database to learn long‐term optimised decisions based on historical information of the system. The conventional Q‐learning algorithm with state‐action pairs has been simplified to a stateless format and applied in a fully distributed manner on individual data file transmissions, which reduces the complexity of the learning model and improves the applicability of Q‐learning algorithms to self‐organised wireless networks. The results show that not only does the proposed algorithm completely remove the requirement for frequency planning but it also improves the convergence, Quality of Service and system capacity substantially by achieving higher link capacity on both access and backhaul networks. Copyright © 2016 John Wiley & Sons, Ltd.