Adaptive embedded control of cyber‐physical systems using reinforcement learning
Hamid Mirzaei Buini, Steffen Peter, Tony D. Givargis · IET Cyber-Physical Systems Theory & Applications · 2017
Embedded control parameters of cyber‐physical systems (CPS), such as sampling rate, are typically invariant and designed with a worst case scenario in mind. In an over‐engineered system, control parameters are assigned values that satisfy system‐wide performance requirements at the expense of excessive energy and resource overheads. Dynamic and adaptive control parameters can reduce the overhead but are complex and require in‐depth knowledge of the CPS and its operating environment – which typically is unavailable during design time. The authors investigate the application of reinforcement learning (RL) to dynamically adapt high‐level system parameters, at run time, as a function of the system state. RL is an alternative approach to the classical control theory for CPSs that can learn and adapt control properties without the need of an in‐depth controller model. Specifically, we show that RL can modulate sampling times to save processing power without compromising control quality. We apply a novel statistical cloud‐based evaluation framework to study the validity of our approach for the cart‐pole balancing control problem as well as the well‐known mountain car problem. The results show an improved real‐world power efficiency of up to 20% compared with an optimal system with fixed controller settings.