There is nothing cast in stone regarding the appropriate level of statistical power, but Cohen (1988) reasoned that studies should be designed in such a way that they have an 80% probability of detecting an effect when there is an effect there to be detected. To put it another way, studies should have no more than a 20% probability of making a Type II error (recall that power = 1 – β).

How did Cohen come up with 80%? In his mind this figure represented a reasonable balance between alpha and beta risk. Cohen reasoned that most researchers would view Type I errors as being four times more serious than Type II errors and therefore deserving of more stringent safeguards. Thus, if alpha significance levels are set at .05, then beta levels should be set at .20 and power (which = 1 – β) should be .80.

Cohen’s four-to-one weighting of beta-to-alpha risk serves as a good default that will be reasonable in many settings. But the ideal level of power in any given test situation will depend on the circumstances.

For instance, if past research tells you that there is virtually no chance of committing a Type I error (because there really is an effect there to be detected), then it may be irrational to adopt a stringent level of alpha at the expense of beta. A more rational approach would be to balance the error rates or even swing them in favor of protecting us against making the only type of error that can be made.

This entry was posted on Monday, May 31st, 2010 at 1:15 am and is filed under statistical power, Type I error, Type II error. You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

“The primary product of a research inquiry is one or more measures of effect size, not p values.”
~ Jacob Cohen

How to manuals

“Statistical significance is the least interesting thing about the results. You should describe the results in terms of measures of magnitude – not just, does a treatment affect people, but how much does it affect them.”
~ Gene Glass