LARE: A Linear Approximate Reinforcement Learning Based Adaptive Routing for Network-on-Chips
Shaocong Wang, Xiaoyun Zhang, Dezun Dong, Cunlu Li, Zicong Wang, Zongmao Zhang · 2023
The routing algorithm is crucial for network performance in network-on-chips (NoCs). With emerging applications bring new features to NoCs with more complex and time-varying traffic, which turns the routing computation process into multi-objective optimization. However, We found that the existing routing algorithms cannot effectively achieve load-balanced between different traffic due to the static method of routing design. The routing algorithm design space will increase if all factors affecting routing are considered. Reinforcement learning (RL) methods have demonstrated promising opportunities applied to architecture design exploration. In this paper, we proposed a novel RL framework for adaptive routing design in NoCs. This method uses network information to select the best path to achieve load balance and lower communication latency at the same time. Unfortunately, with this method, the implementation overhead of RL increases rapidly as the network scale increases. Therefore, we introduce a linear function of approximate RL-based adaptive routing (LARE) to reduce implementation over-head. We conduct extensive experiments against state-of-the-art routing algorithms to evaluate our design. Simulation results demonstrate the benefits of our design under synthetic traffic workloads and real applications. In addition, LARE can achieve similar network performance with traditional RL implementation with a much lower hardware overhead.