When a test returns a result that is statistically nonsignificant, the question arises, “does this result mean there is no effect or did my study lack statistical power to detect?” It’s a fair question, but one which power analysis cannot answer.

Recall that statistical power is the probability that a test will correctly reject a false null hypothesis. Statistical power only has relevance when the null is false. The problem is that a nonsignificant result does not tell us whether the null is true or false. To calculate power after the fact is to make an assumption (that the null is false) that is not supported by the data.

This entry was posted on Monday, May 31st, 2010 at 12:43 am and is filed under power analysis, statistical power. You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

“The primary product of a research inquiry is one or more measures of effect size, not p values.”
~ Jacob Cohen

How to manuals

“Statistical significance is the least interesting thing about the results. You should describe the results in terms of measures of magnitude – not just, does a treatment affect people, but how much does it affect them.”
~ Gene Glass