Obesity Prevention: Where No Effect Is Evidence of Effectiveness

Standards of evidence can seem a little fuzzy in this age of debates with alternative facts. But serious scientists have pretty clear standards. In obesity prevention, though, we wonder about some of the studies that sneak into journals. Take for example this study in Australia. As we wrote months ago, the authors found no overall effect for their prevention program. And yet, in their abstract, they claimed to have found “evidence of effectiveness.”

In our simple way of thinking, you can’t have it both ways.

Fearless Resistance to Facts

Even now, when challenged on this obvious discrepancy, the authors of this study are resolute.

Bryan McComb and colleagues wrote to the editors of the Australian and New Zealand Journal of Public Health to explain why the effectiveness claims are false. The journal published their letter this week. At the heart of the issue is an invalid analysis, they said:

This kind of analysis, known as differences in nominal significance (DINS) analysis, is “invalid, producing conclusions which are, potentially, highly misleading.” As George et al. explain, “a researcher should never use the nominal significance of a pre-post difference within a group to make inferences about differences between groups.”

The original authors, Mary Malakellis et al, defended their claims. “Our conclusion used qualifying statements that there was ‘some evidence,'” they said.

Certainly, absence of proof is not proof of absence. But neither is it the basis for making any claims about efficacy. No effect detected is no effect. Period. Try again, if you like. But don’t mislead people.

Why Worry About Evidence?

In the realm of obesity prevention, dismissing evidence is hardly an isolated problem. As we wrote yesterday, some public health advocates praise the idea of taking action without worrying about sufficient evidence.

A recent report on the Sydney Playground Project presented a negative result as if it were positive. It’s a remarkable echo of representations made by Malakellis et al. Likewise, the LA Sprouts program recently republished claims that they previously withdrew from Obesity because those claims were invalid. The authors made no mention of the prior publication.

We could go on.

Obesity prevention is important. It’s so important that we have to get to the truth of what works and what doesn’t. Pretending to know something works isn’t good enough. It’s OK to try and fail. But hiding the truth of those failures is not OK. It will ensure continued failures.

Click here for the original publication by Malakellis et al, here for the letter by McComb et al, and here for the response Malakellis. For perspective on evidence-based public health policy, click here.

Addition-Subtraction, photograph © Rodger Evans / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


December 29, 2017

3 Responses to “Obesity Prevention: Where No Effect Is Evidence of Effectiveness”

  1. December 29, 2017 at 9:14 am, Al Lewis said:

    This happens even in Health Affairs. An author gave people various incentives and penalties to get them to lose weight. None did. His conclusion: they just hadn’t found the right combination of incentives and penalties yet.

    In other words, there has to be a pony in there somewhere…

    • December 29, 2017 at 9:41 am, Ted said:

      Thanks, Al! Gotta love that metaphor.

      I’m fine with people wanting to keep digging. But honesty is essential for progress.

  2. December 29, 2017 at 2:18 pm, David Brown said:

    Ted says, “Honesty is essential for progress.” Exactly so. However, perception can blind honest scientists to the truth causing them to embrace fallacy. There’s a herd mentality in academia that hampers progress. The problem? Nutritionism. “Nutritionism is the dominant paradigm within nutrition science itself, and frames much professional- and government-endorsed dietary advice. But over the past couple of decades nutritionism has been co-opted by the food industry and has become a powerful means of marketing their products. At the same time, nutritionism has moved from the margins to the center of the public’s understanding of food and health, and has thereby increased the susceptibility of nutrition-conscious individuals to the food industry’s marketing strategies.”