A Model For Assessment Module Selection InProduct Certification
Fang Qiu, R. Fole, Richard J. Cole · WIT transactions on information and communication technologies · 1970
In this paper a model is presented for the selection of software product assessment modules, called 'Assessment bricks'. This is part of a development of an overall architecture for a tool (A Software Assessment Advisor) to act as an automated assistant to aid the assessment procedure. The model proposed consists of two sub-models: a suitability model and an applicability model. The suitability model is used to select an initial set of bricks which are suitable for the proposed assessment. The criteria used in this selection include: software characteristics to be assessed; product category; degree of relevance of bricks to identified characteristics and usability of bricks in previous assessments. The selection process involves a fuzzy algorithm and a set of rules. The applicability model provides further refinement of the selection depending upon the availability of product parts (such as documentation) , data and tools. This brick selection model is being incorporated in the Software Assessment Advisor for testing and validation. INTRODUCTION As indicated in an earlier paper[l], a Software Assessment Advisor(SAA) is a necessary tool to assist with the task of selecting a set of relevant software attributes and a suitable subset of assessment modules at the first stage of software assessment and certification. The framework of this SAA has been developed by using Expert Systems techniques. The SAA system contains a hybrid knowledge representation, a reasoning strategy and a knowledge learning Transactions on Information and Communications Technologies vol 9, © 1994 WIT Press, www.witpress.com, ISSN 1743-3517 4 Software Quality Management algorithm. The knowledge representation takes a rule-based form for the procedural knowledge which performs the attribute and assessment module (brick) selection and a frame-based form for the declarative knowledge which includes the knowledge of product aspects and the prior assessment experiences from the case studies. The reasoning strategy concerns a rule-based deduction for the basic reasoning in the selection and a frame-based closest example identification to supplement the basic reasoning. The knowledge learning algorithm involves case-based updating on the importance weights used in the rule-based selection. Further to the development of the SAA framework in [1], the first phase of implementation includes the program modules of system scheduler, input/output scheduler, reasoning scheduler, rule-based attribute selection, frame-based case identifying and a user friendly interface in Windows for man-machine communication. The implemented part is written in C and Sun View currently running under UNIX on SUN workstations. key next step in building the SAA is the development of an appropriate model for assessment module selection given that a set of relevant software attributes has been suggested for assessment and certification. About 100 assessment modules, called 'Assessment bricks', have been produced in the ESPRIT project SCOPE[2] as a modular approach for software product assessment. The approach aims to combine the use of standard assessment packages to suit the needs of any specific software product evaluation. 'Brick' encapsulates a set of measurement and assessment techniques. The basic idea in this modular approach is to consider a brick as a black-box for its selection and the evaluation of its results. This allows an effective selection and application of measurement and assessment techniques from the large set of those available. The feasibility of applying bricks has been demonstrated by a substantial number of case studies selected from specific examples of industrial software development [3] [4]. One of the main problems which has arisen concerns the selection of an appropriate set of bricks for a specific product assessment. It has been indicated by a majority of the case studies that a 'brick selection assistant' tool is necessary for an appropriate and efficient selection. model is proposed in this paper to meet the brick selection requirements; this will complete the SAA system framework. The model is constructed by using knowledge elicited from case study experiences. It consists of two submodels: a suitability model and an applicability model. The suitability model is used to select an initial set of bricks which are suitable for the proposed assessment. This selection is based on several criteria, a fuzzy algorithm and a set of selection rules. The applicability model provides further refinement of the selection depending upon the availability of product parts and tools. The following sections introduce the basic concept of assessment bricks, discuss the factors involved in the brick selection, describe the brick selection model and give an illustrative example of the selection process. Transactions on Information and Communications Technologies vol 9, © 1994 WIT Press, www.witpress.com, ISSN 1743-3517 Building Quality into Software 5 ASSESSMENT BRICKS IN THE PRODUCT CERTIFICATION PROCESS The need for a modular framework for quality assessment has been investigated in the SCOPE project. It was indicated[5]: A large majority of software assessment approaches are either oriented towards very specific products or based on the evaluation of the development process. An assessment approach aimed at all types of software product is missing, but the reason is not the lack of applicable techniques. Many quantitative techniques applicable to the various elements of a software product have been extensively experimented. However, their application is not standardised and no global framework exists. The concept of 'Assessment brick' proposed by SCOPE[6] is aimed at the development of such a modular framework for quality assessment that could combine the use of standard assessment packages to suit the needs of any specific software evaluation. The concept of 'Assessment bricks' The concept of a 'brick' is of an assessment module that encapsulates a set of measurement and assessment techniques. The module can be standardised and used to build up a software assessment plan. Essentially a 'brick' is a document that describes an assessment technique, a standard way of applying it and the result it can provide when used during the assessment task. There are two categories of 'brick': basic brick and composite brick. basic brick combines elementary metrics that can be obtained using the same technique or related techniques. composite brick combines the results of lower level 'bricks' in order to produce a more abstract result. The composite bricks are organised as a hierarchy. Product components & process evidence Estimated application cost BRICK INTERFACE DESCRIPTION BRICK APPLICATION GUIDELINES • Scope • Required input • Data to be collected • Output Estimated cost Analysis guidelines Assessment method Applicable sub-bricks • Decision guidelines • Applicable tools I Report Decision / Measure Fig. 1 The brick document framework Transactions on Information and Communications Technologies vol 9, © 1994 WIT Press, www.witpress.com, ISSN 1743-3517