Novel framework for automated testing of ill-defined human–robot interaction environments
Aitor Aguirre-Ortuzar, Íñigo Elguea-Aguinaco, Nestor Arana-Arexolaleiba, Leire Etxeberria, Joseba Andoni Agirre · Journal of Systems and Software · 2025
As automated systems advance in complexity, comprehensive testing becomes crucial, particularly for human–robot interaction (HRI) environments where human unpredictability creates ill-defined testing domains that challenge conventional software testing approaches. In interactive robotics, evaluation criteria extend beyond performance to include critical safety considerations. This paper introduces a novel automated testing framework combining runtime monitoring with constraint-based techniques for HRI environments. The framework employs a three-level cognitive oracle architecture – observation, interpretation, and diagnosis – that automatically evaluates the correctness of human and robot actions without requiring expert human intervention. The approach uses constraint-based modeling to handle the non-deterministic nature of HRI scenarios while ensuring safety compliance. Validation through five test cases in a refrigerator disassembly simulation demonstrates the framework’s effectiveness in detecting safety violations and procedural errors under environmental uncertainties.