Reply: A plea for confidence intervals and consideration of generalizability in diagnostic studies
Chris D Frost, Constantinos Kallis · Brain · 2008
Sir, We read with interest the recent article by Klöppel and colleagues (2008) on automatic classification of MR scans in Alzheimer's disease and the coverage on the BBC's website. In particular, we were concerned that the claim on the website that ‘computers “spot Alzheimer's fast” ’ cannot be justified on the basis of this research. Although the authors do not make this claim in their paper, in our opinion they do not pay enough attention to two limitations of their work, which may have encouraged the media over-interpretation of their findings. The first limitation of their work is the relatively small sample size. It has been recognized for many years that confidence intervals should be reported in medical research (Altman et al., 1983; Gardner and Altman, 1986). Indeed, some journals recommend that these be included in publications in preference to the rather over-used P-values. We were disappointed that they were not reported here as their inclusion would have been revealing. For example in Group 1 (confirmed Alzheimer's disease cases from the US versus controls) although the estimated sensitivity and specificity are both 95%, an exact 95% confidence interval on each of these extends from 75.1% to 99.9%. We believe that a balanced interpretation of the authors' findings would include reference to the fact that their results are consistent with somewhat lower (and higher) sensitivities and specificities.