LCT-DER: Learning Classifier Table with Dynamic-Sized Experience Replay for Run-time SoC Performance-Power Optimization
Anmol Prakash Surhonne, Florian Maurer, Thomas J. Wild, Andreas Herkersdorf · 2023
Learning classifier tables (LCTs) are lightweight, classifier based, hardware implemented reinforcement learning (RL) building blocks which enable self-adaptivity and self-optimization properties in multicore systems. LCTs are deployed per-core to learn and optimize potentially conflicting objectives and constraints. Experience replay (ER) is a replay memory technique in RL, where agents experiences are stored in a buffer and are used to improve the learning process. Implementing an ER buffer in hardware requires memory and is expensive. We introduce LCT-DER: LCT with dynamic-sized experience replay, where the classifier population and experiences share the same memory by exploiting the concept of macro-classifiers. LCT-DER performing DVFS achieves 44.5% and 4.5% lower number of power budget overshoots and IPS difference compared to a standard LCT without requiring additional memory.