When to Apply Scenario-Based Testing to a New System Domain?
Nicola Kolb, Tabea Schmidt, Joachim Hans Fröhlich, Alexander Pretschner · 2023
Autonomously acting cyber-physical systems are emerging across a wide range of applications. An approach that has been pioneered in the automotive industry to assess the safe operation of these systems is scenario-based testing (SBT). While there is growing interest in employing SBT in other domains, it is often unclear whether it is economically feasible or applicable in a new domain and context, particularly for those practitioners without prior experience in SBT. A potentially unjustified application of SBT can undermine the approach's credibility. While previous papers have applied SBT to various systems, such as autonomous cars, ships, and aerial vehicles, they have not yet specified general criteria that qualify these systems for an application of SBT while disqualifying others. In this work, we present a methodological guideline that offers constructive assistance to prevent unsuitable use of SBT. First, we outline general, system-specific, and environment-specific criteria for determining the suitability of SBT. Then, we propose guiding questions for practitioners to aid them in deciding if SBT is suitable for their domain and guide them at the start of the realization phase.