Model-based dependability evaluation of complex critical control systems
Francesco Flammini · Università degli Studi di Napoli Federico II · 2007
testing can be synthetically defined as a configuration independent and autoinstantiating approach to system testing of large computer based control installations. In other words, it consists in having an abstract test specification, written without referring to any specific system installation, and a mechanism to automatically detect the specific configuration of the control system and instantiate accordingly the abstract test suite into testMODEL-BASED TECHNIQUES FOR THE FUNCTIONAL ANALYSIS OF COMPLEX CRITICAL SYSTEMS 44 cases to be physically executed on the system under verification. The configuration data depends on the type and number of devices to be used, which in turn is usually installation specific, while the control logic or algorithms is configuration independent in most cases. This means that control actions performed by the actuators depend on device classes and subsets related to the specific installation and on their interrelationships; however, such dependency does not impact on the generality of system functional requirements and of the corresponding test specification. The approach presented in this paper provides a general methodology and algorithm for the efficient customization of system test-suite to a specific configuration, by using an automatic generation algorithm. With respect to traditional approaches, in which such an objective is achieved manually, this allows for a great saving in time and safer results, thus reducing the time to market for any new developed system, as demonstrated by our multi-year testing experience. The approach is based on the assumptions on control system architecture reported in Section 2.1 (Figure 9) which are quite general: its applicability is then a consequence of how well the system under verification can be abstracted into such a hypothesized structure. There are reasons to think that most real world control systems fit reasonably well such general model. 5.2. Abstract testing methodology Before presenting the abstract testing methodology, two introductory statements are necessary: 1) It should be clear that abstract testing is not a functional test specification methodology, but a useful complement to it. We will start from system requirements with the only aim to show more clearly how it should be employed in a real testing process (how to specify functional tests is object of Section 4); 2) Abstract testing is not meant to discover most configuration errors, as it is based on configuration itself; it allows, instead, to cover system configuration, besides control code, and thus provides a form of strong integration testing between control software and underlying configuration. The verification that the system is configured properly for the specific installation should be performed separately (e.g. by a diversity based approach) and such an aspect is not in scope of this approach. The abstract test specification is performed starting from system functional requirements and using a precise formalism, in order not to generate ambiguities when the abstract test algorithm will interpret it. System specification is usually written in natural language, using a form for requirements which can be easily conveyed into the following general one: “When system is in state SI and receives an input I from sensors SEN, then it shall actuate output O using actuators ACT and transit in state SO”, where S, I, O, SEN, and ACT are respectively lists, vectors or equivalence classes (determined by particular properties) of states, inputs, outputs, sensors and actuators. Herein after, with no loss of generality, we will usually assume to deal with generic “properties”, used to select objects of any class (i.e. S, I, O, SEN, ACT) within requirements, coherently with an abstract specification which should identify entities only according to their properties of interest (i.e. attributes’ range of values and relationships with other entities). Usually, informal specification only indicates changes in outputs or output states, assuming the rest remains the same; obviously, this does not influence the generality of the proposed form. Therefore, a general format for abstract test description (or Test Case, TC), formalizing the functional requirement, could be the following: STATEI – INPUT → OUTPUT – STATEO Equation 2 MODEL-BASED TECHNIQUES FOR THE FUNCTIONAL ANALYSIS OF COMPLEX CRITICAL SYSTEMS 45 for each significant system state and input stated by the specification (system level state based testing has been introduced in Section 2.3). STATEI and STATEO represent respectively input and output states. INPUT includes both input values and the involved sensors; similarly, OUTPUT also refers to both actuators and output values. Therefore INPUT = {I, SEN} and OUTPUT = {O, ACT}. Each macro variable in the left part of Equation 2 is a combination of elementary variables (namely “influence variables”, see Section 4.2; e.g. STATEI = (StateI1, StateI2, ...)), which satisfy a given condition (e.g. SI) and have to be instantiated according to such condition in order to generate an executable test-case (e.g. StateI1 = sI1, StateI2 = sI2, ...). Dealing with critical systems, we assume to consider any combination of inputs respecting SI and I, despite of possible redundancies which could be eliminated by defining proper (i.e. safe) reduction criteria to be applied on the test set (as explained in Section 4). The macro variables on the right of expression (1), instead, must be checked after test execution in order to verify that their instances satisfy the O and SO conditions. Note that in general STATEI ≠ STATEO (intended as sets), that is the state variables to be checked do not have to be the same defined in input state. This leaves test engineers free to define different subsets of interest on input and output states, thus implicitly defining equivalence classes. Finally, note that according to the general structure presented in previous section, system state is exhaustively given by defining the value of all attributes of all its Processes. Of course, such a generalization comprises the simplest cases, in which e.g. the requirement (and thus the test) specifies single input and output instances.