Real-Time Routing Design for LEO Satellite Networks: An Enhanced Multi-Agent DRL Approach
Hailong Liao, Xian Zhang, Jiaen Zhou, Xuehua Li · 2024
Low Earth Orbit (LEO) satellite networks have been heralded as a promising solution for provisioning space-based Internet services. However, the inherent dynamic topology, characterized by frequent variations in inter-satellite link states, causes pronounced fluctuations in latency and exacerbated packet loss rates. To address the challenge, in this paper, we formulate a routing design and optimization problem to maximize the average throughput with considering the end-to-end latency constraint and the real-time routing demand in LEO satellite networks. To solve this problem, we design an enhanced Multi-Agent Deep Reinforcement Learning (MADRL) algorithm, which encapsulates the intricate interplay among LEO satellite agents and their environs within a Real-Time Markov Decision Process (RTMDP) framework, and introduces the self-attention mech-anism to endow agents with a better grasp of environmental state dynamics. Simulation results demonstrate that our proposed routing design outperforms the benchmark schemes in terms of both end-to-end latency and network average throughput.