Bushel of Apples

A Bumper Crop of False Promises about Weight and Health

We shouldn’t be surprised. In this season, both hucksters and health reporters are burying us in false promises about weight and health. The advertisements for dietary supplements are annoying enough. We keep our finger on the mute button. And we can filter tabloid headlines about Kerry Katona’s protein injections out of our news feed.

But the bumper crop of false promises is coming even from sources that should know better: otherwise legitimate reporters and scientists.

WSJ: Eat Whatever You Want, as Much as You Want

From no less than the Wall Street Journal, we have a piece of puffery on time-restricted feeding. The article promises:

A Diet Strategy That Counts Time, Not Calories; You can eat whatever you want with time-restricted feeding, just not whenever you want. The weight-loss regime limits eating to a 12-hour window each day and is good for diabetes prevention, longevity and blood pressure

For proof, the reporter points to a two-year old article in Cell Metabolism. It’s an uncontrolled study of a smartphone app in only eight subjects with excess weight. The researchers simply did a pre/post comparison. Based on that, they claimed that using their smartphone app led to reduced body weight, more energy, and better sleep.

If a sample size of only eight isn’t enough, the pre/post design should tell you that these claims are unreliable. That’s because of something called regression to the mean. Whenever you study a group of people who score high on any measure – in this case weight – the next time you take that same measure, it will go down. For studies of excess weight and obesity, this means that you almost always need a control group if you want valid results.

Pre/post comparisons just don’t cut it.

Confusing a “Link” with a Cause and Effect

A favorite tactic we see is to make claims based on a observational studies. “What do chubby kids eat?” asks a headline in the Atlantic. What follows is a long list of foods associated with obesity. “The only foods associated with staying at a healthy weight? Whole grains and high-fiber breakfast cereal,” says the reporter.

This report is based on an observational study published in Health Affairs. Such observational studies are everywhere we turn in nutrition and obesity research. And even though they can’t prove anything about cause and effect, reporters and the public seem to think they do. In a new report published on BioRxiv, Leslie Myint and colleagues suggest a reason for this:

Nearly every scientific study, regardless of the study design, includes explanation for observed effects. Our results suggest that these explanations may be misleading to the audience of these [observational] data analyses.

In other words, publishing observational data and then explaining why you might be seeing a link in that data directly leads people to believe that a cause and effect is at work. Disclaimers are beside the point.

Clearly, we need higher standards for reporting about obesity, nutrition, and health. Until those standards rise, critical thinking is your only defense.

For more on regression to the mean, click here. For a short course on causality, click here.

Bushel of Apples, photograph © Rob MacEwan / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


January 11, 2018