What are two approaches to reviewing past research?

May 30, 2010
  1. the qualitative approach, also known as the narrative review
  2. the quantitative approach, also known as meta-analysis

The narrative review is useful for documenting the unfolding story of a particular research theme. The aim is to summarize and synthesize the conclusions of others into a compelling narrative about the effect of interest. Of course, this can be tricky to do when previous researchers have come to different conclusions or have drawn conclusions about substantive effects by looking at p values.

In contrast, meta-analysis completely ignores the conclusions that others have drawn and looks instead at the evidence that has been collected. Evidence, in this case, refers to study-specific estimates of a common population effect size.

By combining the independent estimates into an average effect size, a meta-analysis is able to draw an overall conclusion regarding the direction and magnitude of the effect of interest.

Source: The Essential Guide to Effect Sizes

What’s wrong with the traditional narrative review of the literature?

May 30, 2010

What’s right with it?!

Most of us take a qualitative approach to reviewing the literature for no reason other than that’s what we were taught to do or what we’ve read. But there are at least four problems with narrative reviews:

  1. they are rarely comprehensive
  2. they are highly susceptible to reviewer bias
  3. they seldom take into account differences in the quality of studies
  4. they often come to the wrong conclusion or no conclusion at all, hence the oft-heard call for further research

On each count meta-analysis offers a superior alternative.

For more, see The Essential Guide to Effect Sizes.