The decision to reject or retain H0 depends on the announced level of significance, α. α is a statement of risk that the researcher is willing to assume in making a decision about H0.

When H0 is true (µ0 = µtrue), 5% of all possible sample means nevertheless will lead to the conclusion that H0 is false because 5% of the sample means fall in the “rejection"" region of the sampling distribution, even though these extreme means will occur (though rarely) when H0 is true. Thus, there’s a probability of .05 that H0 will be rejected when it is actually true. Rejecting a true H0 is a decision error.

Level of significance, α, gives the probability of rejecting H0 when it is actually true. Rejecting H0 when it is true is known as a Type I error.

A Type I error is getting statistically significant results “when you shouldn’t."" To reduce the risk of making such an error, the researcher can set α at a lower level.