Adaptive Learning Systems and Interference in Causal Inference
Alexander Olof Savi, Nick ten Broeke, Abe Dirk Hofman · 2021
Adaptive learning systems can be susceptible to between-subject cross-condition interference by design. This interference has important implications for the implementation and evaluation of A/B tests in such systems, as it obstructs causal inference and hurts external validity. We illustrate the problem in an Elo based adaptive learning system, discuss sources and degrees of interference, and provide solutions, using an example in the study of dropout.