Q-Learning-Based Adaptive Backoff Window MAC Protocol Leveraging Historical Success Rates and Real-Time Feedback
Jiyuan Wang, Tingting Lyu, Jiaqi Cui, Yan Zhang, Yuhan Yao · 2025
An adaptive contention-window media access control protocol (CW-Q-MAC) based on Q-learning is proposed to address the frequent collisions and low throughput caused by long propagation delays and limited bandwidth in underwater acoustic communication networks. This protocol employs a reinforcement learning mechanism that enables nodes to autonomously learn backoff strategies according to their local environmental state, available actions, and a designed reward function, thereby achieving intelligent scheduling and dynamic optimization of channel resources. To accommodate the characteristics of underwater acoustic communication, we define a state representation that combines historical success rates with the outcome of the most recent transmission and construct a reward function specifically tailored for underwater backoffwindow adjustment. Additionally, a cosine - annealing learning rate schedule and a momentum mechanism are introduced to accelerate convergence and suppress drastic performance fluctuations and high - frequency oscillations in the network. CW-Q-MAC is fully distributed: each node adaptively adjusts its contention window based solely on its own communication status, without relying on global information. This significantly mitigates network contention and improves channel - utilization efficiency. Simulation results demonstrate that, compared to the traditional ALOHA protocol and fixed - contention - window MAC (CWMAC) schemes, CW-Q-MAC achieves substantial improvements in both throughput and average end - to - end delay, exhibiting strong adaptability and robustness.