Automated Test Process for Highly Configurable Software-Defined Mobility Systems in CI/CD
Lennard Hettich, Tobias Pett, Ann-Therese Tabea Nägele, Marc Schindewolf, Halit Eriş, Stefan Wagner, Eric Sax, Ina Schaefer, Michael Weyrich · SAE technical papers on CD-ROM/SAE technical paper series · 2025
In the pursuit of customizability and evolvability of vehicle functions, manufacturers shift towards software-defined vehicles to enable flexible customization and over-the-air updates. This results in multiple variants and versions of a vehicle model. While shifting to software-defined vehicles (SDVs) adds value and flexibility for customers, manufacturers struggle with homologating new and updated functionality because existing testing processes do not scale for high-frequency release cycles that limit available testing resources. Overcoming this challenge by using a coherent test process designed for testing continuously evolving variant-rich systems will be one of the key enablers. This paper presents an innovative end-to-end pipeline for efficient and comprehensive testing of variant-rich vehicle functionality tailored to an application in continuous development. Our transferable test pipeline employs sample-based variant selection, a software-in-the-loop environment for executing selected variants, and scenario-based testing using mutation-based scenario fuzzing. A central test controller is responsible for managing the process. We evaluate our test pipeline with regard to its fault detection capability, using the YOLO object detection algorithm as a variant-rich test object in the CARLA simulator. Our results show that the testing process outperforms random variant sampling and scenario mutation in detecting faults.