What’s wrong with post hoc power analyses?

When a test returns a result that is statistically nonsignificant, the question arises, “does this result mean there is no effect or did my study lack statistical power to detect?” It’s a fair question, but one which power analysis cannot answer.

Recall that statistical power is the probability that a test will correctly reject a false null hypothesis. Statistical power only has relevance when the null is false. The problem is that a nonsignificant result does not tell us whether the null is true or false. To calculate power after the fact is to make an assumption (that the null is false) that is not supported by the data.

Source: The Essential Guide to Effect Sizes

Comments are closed.