LCT-TL : Learning Classifier Table (LCT) with Transfer Learning for runtime SoC performance-power optimization
Anmol Prakash Surhonne, Florian Maurer, Thomas J. Wild, Andreas Herkersdorf · 2023
Learning classifier tables (LCTs) are lightweight and classifier-based reinforcement learning building blocks in hardware that inherit the concepts of learning classifier systems. LCTs are deployed to learn and optimize potentially conflicting objectives and constraints e.g., achieving a desired performance target (IPS) under a constraint (power budget). These are reflected by the reward function which provides rewards for the LCT. A supervisor provides LCTs with the objectives/targets and constraints by translating application requirements and system policies for the LCT. The supervisor combined with the LCTs enable self-adaptive and self-optimization properties in a system-on-chip. The objectives and constraints for the LCTs are changed in runtime by the supervisor resulting in different reward functions. The LCT has to learn these different reward functions by trial and error. The rate at which the LCT learns about these changes can be accelerated by applying transfer learning. LCS systems are represented by a population of classifiers. Each classifier is made up of condition, action and bookkeeping parameters like prediction, error, accuracy, experience and numerosity which directly influence the performance of an LCT. In this work, we propose LCT-TL: LCT with transfer learning, where we propose two strategies that selectively transfer the bookkeeping parameters for a change in target/constraint. We evaluate the strategies by deploying LCTs as DVFS controllers for performance-power optimization. Experimental results show that the LCT-TL performs significantly better than the state-of-the-art LCT and a tabular Q-learning agent. LCT-TL doesn’t require any additional hardware resources and negligible soft-ware overhead.