Evaluating integrated assessment tools for policy support
Jacques-Éric Bergez, Marijke Kuiper, Olivier Thérond, Marie Taverne, Hatem Belhouchette, Wery, J., Brouwer, F., Van Ittersum, M. · HAL (Le Centre pour la Communication Scientifique Directe) · 2010
Integrated Assessment Modelling tools are complex tools requiring specific evaluation methodologies. Based on the example of the SEAMLESS-Integrated framework, we show how the conceptual, technical and system evaluation steps of the different components(procedures, quantitative models, graphic user interfaces) were performed by a multidisciplinary team. To make the not-yet-available tool real, mock-up and test cases were mobilized throughout the development process in order to integrate final end-users in the evaluation process. The main lessons from the project are that the evaluation required: i) the use of prototypes to advance properly in the design and testing (spiral methodology); ii) the use of case studies to stick to the end-users requirements; iii) a proper timing of development and delivery in order to keep on schedule and leave time to the evaluation process; iv) a multidisciplinary team of evaluators as tools are of diverse types and v) that it is difficult to keep independence between testers, end-users and modellers in order to guaranty transparency in the development and evaluation process.