Confirmation Bias and Other Flaws in Citing Pass Rate Studies
Simon, Andrew Luxton-Reilly, Oluwatoyin Adelakun-Adeyemo · 2021
There have been four principal studies on pass rates in introductory programming courses, in 2007, 2014, and two in 2019. These studies, the first two in particular, are extremely widely cited, but it appears that some of the citing papers misrepresent what was written in the original papers. We present an analysis of nearly 600 papers that cite one or more of these four papers, indicating what sorts of message they impute to the papers. We find that well over half of the citing papers fail to accurately represent the papers' findings, instead citing the papers as evidence either of a commonly accepted belief or of their own -- sometimes quite bizarre -- assertions.