Continuous Safety Assessment of Updated Supervised Learning Models in Shadow Mode
Houssem Guissouma, Moritz Zink, Eric Sax · 2023
Over-The-Air (OTA) updates play an essential role in the lifecycle management of modern Cyber Physical Systems (CPSs). They are deployed in short time periods to fix bugs and introduce new features. However, an important part of these updates affects safety-critical functions, and thus, requires thorough verification and validation. Particular care must be taken when using machine learning algorithms, for which it is more difficult to test all conceivable corner cases during the development process. To prevent potential unforeseen misbehavior after deployment, we introduce a method for runtime evaluation of updates in shadow mode using contract specifications. The method focuses on supervised learning models and is embedded in a workflow for iterative training. This enables carrying out reliable field testing and obtaining a realistic evaluation of the planned updates before release. Finally, we evaluate our approach on a prototype Electronic Control Unit (ECU) implementing an automotive Lane Keep Assist (LKA) system.