Using Marginal Models to Adjust for Statistical Bias in the Analysis of State Transitions
Jeffrey Matayoshi, Shamya Karumbaiah · 2021
Many areas of educational research require the analysis of data that have an inherent sequential or temporal ordering. In certain cases, researchers are specifically interested in the transitions between different states—or events—in these sequences, with the goal being to understand the significance of these transitions; one notable example is the study of affect dynamics, which aims to identify important transitions between affective states. Unfortunately, a recent study has revealed a statistical bias with several metrics used to measure and compare these transitions, possibly causing these metrics to return unexpected and inflated values. This issue then causes extra difficulties when interpreting the results of these transition metrics. Building on this previous work, in this study we look in more detail at the specific mechanisms that are responsible for the bias with these metrics. After giving a theoretical explanation for the issue, we present an alternative procedure that attempts to address the problem with the use of marginal models. We then analyze the effectiveness of this procedure, both by running simulations and by applying it to actual student data. The results indicate that the marginal model procedure seemingly compensates for the bias observed in other transition metrics, thus resulting in more accurate estimates of the significance of transitions between states.