What’s wrong with post hoc power analyses?

When a test returns a result that is statistically nonsignificant, the question arises, “does this result mean there is no effect or did my study lack statistical power to detect?” It’s a fair question, but one which power analysis cannot answer.

Recall that statistical power is the probability that a test will correctly reject a false null hypothesis. Statistical power only has relevance when the null is false. The problem is that a nonsignificant result does not tell us whether the null is true or false. To calculate power after the fact is to make an assumption (that the null is false) that is not supported by the data.

Source: The Essential Guide to Effect Sizes

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.