Significance is not the whole story – decision making in hypothesis testing has two sorts of possible errors
James Nicholson · 2019
Hypothesis testing has come under scrutiny, and attack, because of the way it is being misused. Much of the misuse seems to stem from a fundamental lack of understanding of some key principles within the methodology. In particular, losing sight of the fact that there are two possible wrong decisions. Where p-values are used, and computed by software, it is very difficult to maintain the perspective of the test as trying to identify shifts in parameters – because ‘significance’ has been viewed as the holy grail. The foundations for understanding hypothesis testing are undermined in some curricula where key aspects, such as the existence of two potential errors in the decision, are omitted. This paper will develop a pedagogical basis for teaching the logical foundations of hypothesis testing, and will provide links to electronic resources to support this approach.