Research on TCP Congestion Control Strategy Based on Proximal Policy Optimization
Yuyu Yuan · 2023
The existing TCP congestion control mechanis ms are gradually becoming inadequate for the complex and dynamic network environments. Currently, adjusting conge stion window policies according to predefined rules results i n significant window oscillations and low stability. Building upon the existing congestion window algorithms, a congestio n window adjustment strategy based on near-end policy opti mization has been proposed. This strategy abstracts the con gestion window adjustment process into a Markov decision process, creating an interaction between an agent and the ne twork environment. Based on observed state characteristics, it selects the optimal strategy for dynamically adjusting the congestion window, The agent, through the introduction of v alue function advantage estimation, iteratively updates strat egies to maximize cumulative rewards, assessing the advanta ge level of each action. By incorporating advantage estimatio n into the loss function of policy optimization, it can better m easure the effectiveness of policy updates, resulting in more accurate and stable strategy updates. The agent learns and s elf-optimizes from past experiences, without relying on man ually defined rules or prior knowledge of network scenarios. Experimental results indicate that, compared to mainstrea m congestion control algorithms such as TCP New Reno and TCP Cubic, this approach effectively regulates the sending rate and improves throughput.