Evaluation Report
Lina Orrell Francine Amon · Zenodo (CERN European Organization for Nuclear Research) · 2022
This report documents the evaluation of usability, usefulness, and the costs and benefits of using the ASSISTANCE SA platform. The evaluations of usability and usefulness were conducted via observations, questionnaires, and focus groups during the three demonstration pilots. The cost-benefit analysis also used questionnaires and the focus groups held during the pilot, as well as interviews. Usability was measured using a Systematic Usability Scale (SUS) to assess the ability of first responders to use the situation awareness system (SA) tablets in the field and in tabletop exercises during the demonstration pilots. Usefulness, which is defined for this work as the overall SA platform’s ability to enhance first responder capabilities, efficiency, and safety, was measured using an SA score based on the principles established by Endsley in 1995. The cost-benefit analysis is based on the intersection of the first responder organisation’s willingness to pay for the ASSISTANCE SA platform (representing costs) and the benefits to European society from reduced lives lost in earthquakes, industrial accidents and terrorist attacks.