What I Believe

Belief Requires No Evidence

Belief and knowledge become easily confused in matters of nutrition, obesity, and health. Mind you, there’s nothing wrong with strong beliefs. But believing something to be true is very different from knowing it, and that’s where the problem lies.

We have a lot of strong beliefs about what should work to prevent or treat obesity, and all too often we stop short of actually proving it. Such is the case in a recent study published in the International Journal of Behavioral Nutrition and Physical Activity. Rachel Burke and colleagues concluded that their HealthMPowers program is effective “in producing positive change in school policies and practices, student knowledge and behaviors, and student fitness and BMI, supporting the use of holistic interventions to address childhood obesity.”

The problem is that their data proved no such thing. In a commentary just published by the same journal, Asheley Cockerel Skinner and a distinguished group of scientists from four different centers of obesity research pointed out that a well-known statistical phenomenon (regression to the mean) explains the patterns observed in this study — not the effectiveness of the HealthMPowers program.

This is because regression to the mean has been proven to produce an improvement in BMI z-scores without any intervention — and the improvement seen in this study was less that would be expected from this effect. In other words, the effect observed almost certainly had nothing to do with the program being studied.

Skinner et al emphasize the importance of having good evidence in this realm, saying:

We recognize the difficulties in performing school-based research, but we must consider the conclusions that can be drawn from uncontrolled studies. Research on childhood obesity interventions and policies is desperately needed, but it must also provide real evidence of which policies are effective and for which groups of children.

Unfortunately, some people are impatient with the tedious work of collecting real evidence for what works and what doesn’t. We’ve been told that:

  • “Policy research is uniquely challenging and difficult. It requires a different standard of evidence.”
  • “We’re dealing with a flood of obesity and waiting for the perfect sandbag doesn’t make for good flood management.”

The challenge of collecting real evidence is undeniable. But two decades of ineffective efforts to slow the rise in obesity should make it plain that strong beliefs alone will not be enough to reverse this epidemic.

“In God we trust; all others must bring data.” — Edward Deming

Click here to read the commentary by Skinner et al and here to read the original study.

What I Believe, image © Botgirl Questi / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


4 Responses to “Belief Requires No Evidence”

  1. May 11, 2015 at 12:28 pm, Mary-Jo said:

    It’s unfortunate that all the fantastic work of this HealthMPowers organization will be now marred by their eagerness to publish its merits and in the process found it necessary to manipulate data. One would think that with such impressive institutions involved (Rollins at Emory U, CDC), better rigor and discretion would have been exercised in reviewing statistical analyses done vs. conclusions made. However, this would not be the first time, nor will it be the last where a prestigious institution would be linked to falsification/manipulation of data and conclusions. The obesity epidemic, especially pediatric obesity, indeed brings out the desperate in many of us — even our best and brightest. Ironically, I can remember, not too long ago, when it was malnutrition of hospitalized patients, many of whom were in university hospitals, that resulted in a few of the best nutrition experts in the field, from top medical schools, falsifying data and exaggerating conclusions. I still think a well-designed study of school-based interventions to help improve childhood fitness and decrease obesity prevalence would be great.

    • May 11, 2015 at 5:26 pm, Ted said:

      You’re right, Mary-Jo.

  2. May 11, 2015 at 12:59 pm, Joe Gitchell said:

    Hard work is hard to do….

    • May 11, 2015 at 5:26 pm, Ted said:

      …but easy to describe. Thanks, Joe!