How do I calculate statistical power?

Paul Ellis

The power of any test of statistical significance will be affected by four main parameters:

  1. the effect size
  2. the sample size (N)
  3. the alpha significance criterion (α)
  4. statistical power, or the chosen or implied beta (β)

All four parameters are mathematically related. If you know any three of them you can figure out the fourth.

Why is this good to know?

If you knew prior to conducting a study that you had, at best, only a 30% chance of getting a statistically significant result, would you proceed with the study? Or would you like to know in advance the minimum sample size required to have a decent chance of detecting the effect you are studying? These are the sorts of questions that power analysis can answer.

Let’s take the first example where we want to know the prospective power of our study and, by association, the implied probability of making a Type II error. In this type of analysis we would make statistical power the outcome contingent on the other three parameters. This basically means that the probability of getting a statistically significant result will be high when the effect size is large, the N is large, and the chosen level of alpha is relatively high (or relaxed).

For example, if I had a sample of N = 100 and I expected to find an effect size equivalent to r = .30, a quick calculation would reveal that I have an 57% chance of obtaining a statistically significant result using a two-tailed test with alpha set at the conventional level of .05. If I had a sample twice as large, the probability that my results will turn out to be statistically significant would be 86%.

Or let’s say we want to know the minimum sample size required to give us a reasonable chance (.80) of detecting an effect of certain size given a conventional level of alpha (.05). We can look up a power table or plug the numbers into a power calculator to find out.

For example, if I desired an 80% probability of detecting an effect that I expect will be equivalent to r = .30 using a two-tailed test with conventional levels of alpha, a quick calculation reveals that I will need an N of at least 84. If I decide a one-tailed test is sufficient, reducing my need for power, my minimum sample size falls to 67.

For more, see my book Statistical Power Trip:

Uncategorized