La-Mar Reducing Soap

Before, After, Pre, Post, and Bogus Claims

Seeing is believing. So showing before and after results is a tried and true way to persuade people that something works. The effect must be real. You can see for yourself. Unfortunately, for weight management it’s the basis for bogus claims of effectiveness.

In a letter to the editor, Bridget Hannon and colleagues point out an example of such flawed claims – published in the British Journal of Nutrition.

The PEACH Intervention

The flawed claims come from a childhood obesity intervention with a clever acronym. It’s the PEACH lifestyle program for families. What’s not to love? Fruits and vegetables, healthier families and children. It’s all there in a tidy package, with a memorable name.

But there’s just one problem. The claim that it works is based on a pre/post evaluation. The researchers don’t have a control group in their study design. All they have is a sample of children with excess weight and obesity. They all participated in the program.

Then after the program, children’s BMI z-scores were lower. Thus, researchers claim PEACH is an effective program. That’s the conclusion Carly Moores et al published in May.

Regression to the Mean

The core problem is regression to the mean (RTM). Anytime you select a study group defined by high or low values at the start of a study, it’s a problem. At the end of the study, you can almost guarantee that those values will come closer to normal. That’s RTM at work. You don’t have to do anything. Outliers naturally drift back to the middle.

Therefore, Hannon et al explain that the claim of effectiveness for the PEACH program doesn’t hold up to scrutiny:

In both PEACH articles mentioned, the changes in child BMI z-score can be explained by RTM, and therefore, the results cannot substantiate conclusions of intervention effectiveness or efficacy. Because the conclusions of this study could influence future decision-making as to the best practices for childhood obesity treatment, it is essential that these results be interpreted correctly and causal inference not be exaggerated.

A peachy acronym is no substitute for a valid study. Beware of RTM. It means that before and after advertising for weight loss is inherently misleading. And a pre/post analysis will almost always be inadequate for proving efficacy.

Show us your control group, please.

Click here for the original study and here for the letter by Hannon et al. As of this writing, BJN hasn’t published a response from Moores et al. For more on RTM in obesity studies, click here and here.

La-Mar Reducing Soap, advertising by La-Mar Laboratories c. 1920s

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


 

August 31, 2018