The Causal Testing Framework
Michael S. Foster, Andrew G. Clark, Christopher Wild, Farhad Allian, Robert D. Turner, Richard J. Somers, Nicholas R. Latimer, Neil Walkinshaw, Robert M. Hierons · The Journal of Open Source Software · 2025
Scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal Inference (CI) (Hernán & Robins, 2020; Pearl, 2009) has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse observational data instead of costly randomised trials. CI works by using domain knowledge to identify and mitigate for biases in the data, enabling them to answer causal questions that concern the effect of changing some feature on the observed outcome. The Causal Testing Framework (CTF) is a software testing framework that uses CI techniques to establish causal effects between software variables from pre-existing runtime data rather than having to collect bespoke, highly curated datasets especially for testing.