Significance Levels for the Assessment of Anomalous Phenomena
Robert Matthews · 1999
A b stract — S c i e n t i f ic evidence for ano m a lous phenomena is frequently sup-p o rt ed by conventional measures of st a t ist ical significance such as p- v a l u es. H ow e ve r, th ese measures have been shown to be unreliable in d ic a t o rs of th e e x istence of genuine effects, and ro u t inely exaggerate the true significance of e x perimental data. They are, more ove r, es pecially unsuitable for the asses s-ment of ano m a lous phenomena. More appropriate st a t ist ical techniques are a v a i lable, but pose their own problems when applied to ano m a lous pheno m e-na. I outline an approach to hypoth es is test ing which allows conve n t i o n a l m e a s u res of significance to be re t a in ed, while off e r ing substantially low e r r isk of seeing significance in chance eff e c t s. K e y w o rds: st a t ist ical significance — p- v a l u es — Bayesian in f e re n c e