A Table-Free Approximate Q-Learning-Based Thermal-Aware Adaptive Routing for Optical NoCs
Wenfei Zhang, Yaoyao Ye · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · 2020
Optical networks-on-chips (NoCs) based on silicon photonics have been proposed as an emerging communication architecture for many-core chip multiprocessors. However, the thermal sensitivity of silicon photonics is one of the major challenges. Q-learning-based adaptive routing has been proposed in related work to mitigate the thermal issue. However, table overhead of the traditional table-based Q-routing would scale up quickly with the increase of network size. In this article, we propose a table-free approximate Q-learning-based thermal-aware adaptive routing to find optimal low-loss paths in the presence of on-chip temperature variations. The simulation results show that the proposed table-free approximate Q-learning-based adaptive routing can converge faster and it can achieve similar optimization effect as compared to the best optimization effect of the traditional table-based Q-routing. The performance gap between the proposed approximation method and the traditional table-based Q-routing expands when the network size increases.