An Eye-Tracking Study of Statistical Reasoning With Tree Diagrams and 2 × 2 Tables
Georg Bruckmaier, Karin Binder, Stefan Krauß, Han-Min Kufner · Frontiers in Psychology · 2019
In order to support people’s statistical and especially Bayesian reasoning, in cognitive psychology the change of information format from probabilities (e.g., 80%) into natural frequencies (e.g., “80 out of 100”) has strongly dominated discussions. Furthermore, in the framework of teaching statistics and probability at school and university, visualizations such as tree diagrams and 2×2 tables are prominent tools for fostering understanding. Previous work of our research group (Binder et al., 2015) demonstrated that–despite their widespread use in statistical textbooks—both visualizations are only of restricted help when filled with probabilities, but that they can foster insight when equipped with frequencies. In the present article, we examine by the method of eye-tracking why probabilistic 2×2 tables and tree diagrams do not work regarding Bayesian inferences (i.e., which errors occur and whether they can be explained by scan paths), and why the same visualizations are of great help when filled with frequencies (i.e. how scan paths change, when the information format changes in both visualizations). For checking the appropriateness of heat maps (that summarize scan paths) for explaining underlying reasoning processes, we first asked N=24 participants for marginal, conjoint, (non-inverted) conditional probabilities (or frequencies, respectively). All inferences in the present study were based solely on tree diagrams or 2×2 tables, displaying either the famous “mammography context” or an “economics context”. The information format in these visualizations varied systematically, while additional textual wording was not presented. After analyses of eye movements proved to be an adequate, reliable, and valid measure of people’s strategies in these “basic” tasks, the same participants were presented respective Bayesian tasks yielding noticeable differences—with respect to both qualitative and quantitative measures—between the correct and incorrect solutions. Altogether, 20 inferences of each participant (twelve non-Bayesian and eight Bayesian) resulted in a total of 20×24 = 480 inferences that were condensed in heat maps for analyses aiming to explaining errors in the line of theoretical classifications (e.g., Gigerenzer and Hoffrage, 1995). In deepening our understanding of the cognitive problems underlying Bayesian reasoning errors we continue recent research by our working group (Weber et al., 2018).