Systematic Generation of Test and Fault Cases for Space Application Validation
Ana Maria Ambrósio, Eliane Martins, Nandamudi Lankalapalli Vijaykumar, Solon Carvalho · Biblioteca Digital da Memória Científica do INPE (National Institute for Space Research) · 2005
The most critical activity of a V&V process is the test design, as it should be systematic and less dependent of the expert inspiration, mainly for companies developing software whose failures place millions of dollars at risk. Additionally, the trend towards service standardization for the most common space applications motivated us to define a conformance testing methodology added with fault injection concepts. The methodology, named CoFI, defines steps to generate tests that cover the conformance of an implementation with respect to a standard specification. It allows to generate repeatable and controllable test cases and is conceived to be automated and easy to learn. The idea is to translate the service behavior, written in natural language, into a FSM-based notation, then automatically generate test cases. Its main characteristic is to separate in distinct diagrams the normal, the exceptional behavior, those explicitly-specified and those based on mapping an external fault model. The paper describes the CoFI testing methodology and its use in two real case studies. In the first the TC Verification service specified in ECSS-E_70-41A was used for showing the feasibility of generating test cases from a publicly recognized standard specification. In the second, an OBDH-Scientific_Experiment protocol, developed at INPE, was used for evaluating the set of test and fault cases created by the CoFI methodology. Preliminary results pointed out that: (i) the methodology is simple and effective specially for creating fault cases based on a fault model; (ii) current standard service specification requires extra information to achieve an applicable set of test cases. Effectiveness metrics of the tests were obtained with specification-based mutant analysis. 1.