Evaluating National Cyber Exercise Programs
Tímea Páhi, Andras Balint Toth, Simon Tjoa · 2025
Our wealth, economy, and society increasingly rely on the effective operation of ICT core services. This is emphasized by EU legislation, which mandates EU Member States to identify and safeguard their critical information infrastructures. In order to achieve this objective, it is essential for critical infrastructure providers to implement sufficient cybersecurity measures, which generally include a range of technical and organizational measures. Establishing a robust cyber defense is essential for safeguarding individual organizations. Management structures and response strategies are typically coordinated for this purpose, requiring participation in different cyber exercises. On a national level, it is very challenging to correctly evaluate the progress of a complete national Cyber Exercise Program (later CXP) including numerous different cyber exercises. This article focuses on various metrics that can be employed to evaluate the progress of national cyber exercises as a program, intending to provide a significant contribution. This article aims to address this challenge by introducing a 5-step CXP Evaluation Framework and a Goal-Question-Metric (GQM) methodology specifically designed for assessing national Cyber Exercise Programs. Alongside the established GQM approach, we present a use case to assess and enhance the efficacy of cyber exercises, ensuring that they fulfill their intended goals and strengthen the overall resilience of critical information infrastructures. Furthermore, we are providing an extensive analysis of the Data Collection and Analysis process, incorporating specific examples derived from the cyber-exercise life-cycle.