Satellite Network Random Access Based on Multi-Agent Q-Learning
Guoyi Zhang, Hongyan Xu, Changqing Lai, Kai Wang, Xingxing Wang, Yun Xiu Bai, Chong Wang, Hao Qi, Yinlong Liu · 2024
Satellite networks enhance terrestrial systems by providing extensive coverage, high flexibility, and robust anti-interference capabilities. With ongoing advancements, these net-works face challenges in user access scheduling, especially with Low Earth Orbit (LEO) satellites, which experience significant Doppler shifts and dynamic connectivity due to high velocities. Traditional access methods often prioritize signal quality, over-looking resource contention, which can negatively impact service quality for un-accessed users. To address these challenges, we propose a dynamic random access model tailored for satellite networks, optimizing resource distribution based on user de-mands. This model formulates an optimization problem aimed at maximizing system service utility and develops an optimal access and resource allocation scheme. Our two-stage algorithm, Q-Iearning-based Satellite Access Resource and Power Scheduling (Q-SARPS), employs multi-agent Q-Iearning to efficiently manage resource allocation while simplifying the complexity of maintaining a comprehensive Q- value table. Experimental results indicate that Q-SARPS consistently achieves higher system utility with lower computational costs, thereby enhancing efficiency in resource management. By refining this algorithm and integrating it into complex networks, we aim to enhance user experience and reliability in the evolving landscape of satellite communications.