Characterisation of Challenges for Testing of Adaptive Systems
Bento R. Siqueira, Fabiano Cutigi Ferrari, Marcel Akira Serikawa, Ricardo Menotti, Valter Vieira de Camargo · 2016
Context: Exercising the implementation of an adaptive system (AS) effectively, in order to detect faults, is not a trivial task. This is due to characteristics of this type of systems, such as the high number of configurations and runtime adaptations. In this context, the characterisation of challenges for testing ASs may support the definition of suitable testing strategies. However, such characterisation is currently spread over several studies in the literature. Objective: analysing testing-related challenges for ASs which are described in the literature, and establishing a generic classification. Method: we extended a Systematic Literature Review (SLR) of studies that address challenges for AS testing. By analysing the combined results (original and extended), we mapped descriptions found in individual studies, based on their similarities, in order to create a list of generic challenges. Results: a list of 12 generic challenges based on 34 specific ones described by several authors. Additionally, we analysed the relationship between the challenges and inherent properties of ASs. Conclusion: different terminology, varied levels of details and particular contexts (e.g. specific AS implementations) are recurring problems related to the characterisation of challenges for testing ASs. Generalisations such as the one proposed in this work may help researchers and practitioners do devise customised testing strategies that focus in recurring, inherent properties of ASs.