I’m a scientist. I deal with abstract concepts. Why should I try and interpret the substantive significance of my results?

Short answer #1: we have a moral obligation to ourselves and society to do so.
Short answer #2: why do research that has no substantive value?

Consider two researchers conducting the exact same project investigating the relationship between, say, worker training and productivity. The first researcher draws his conclusions solely by looking at the p values generated by tests of statistical significance. The second researcher draws her conclusions by examining her estimates of the effect size. Judge for yourself who tells the more compelling story:

Researcher 1: “The results of my study reveal that worker training has a statistically significant and positive effect on productivity.”

Researcher 2: “The results of my study reveal that an annual $4000 investment in worker training boosts productivity by 15% leading to a net gain for a typical middle-sized firm of $1m in revenue.”

Did you spot the difference?

Researcher 1’s conclusion is yawn-inducing. He tells us nothing we couldn’t have guessed before the study was done.

Researcher 2’s conclusion, on the other, will make the front page of Managers’ Monthly. By translating her observed effect into a meaningful metric, she has made her result relevant to stakeholders beyond the academic community.

Imagine that. Imagine if we all did research that actually mattered? Then perhaps regular folks would stop making jokes about people who work in ivory towers.

For more on dealing with the challenge of interpretation, see The Essential Guide to Effect Sizes.

Comments are closed.