Imagination Running Wild in Research

Sometimes human creativity simply makes our heads spin. Take this example of imagination running wild in obesity research. Last year, Frontiers in Psychiatry published a study of the possible placebo effect of an imaginary low-calorie diet. The diet was a fiction – no reduction in calories. What’s not to like about an imaginary placebo effect?

Making this quest seem worthwhile, the researcher found the effect he was seeking. The study’s author, Valentin Stefanov Panayotov, concluded:

Despite some methodological biases of the study construct, in our opinion, a placebo effect could partially explain the changes in the experimental group.

But in the end, it looks like all this imaginative thinking doesn’t quite hold up to scrutiny.

A Flawed Analysis for an Imaginary Treatment

Coincidentally, we wrote last year about ten ways to reach false conclusions in obesity studies – an inventory of common errors. One of those common errors is a DINS error – a difference in nominal significance. In plain language, it’s a pair of bogus baseline comparisons. It happens when researchers see one treatment group improve from baseline, while a control group does not. Such an outcome often tempts researchers to say they’ve proved the treatment works. Even though they haven’t.

But that’s not a valid conclusion. It’s a DINS error. To prove effectiveness, one must compare between groups, not within groups. It’s a basic principle of statistical analysis. And so, that error renders the imaginary quest for a placebo effect unresolved. Panayotov compared the results within groups, not between groups.

Sharing Is Good

Fortunately, as any good scientist should, Panayotov shared his data for validation with Stephanie Dickinson, Greyson Foote, and David Allison. When these three scientists re-analyzed the data, they found mixed results, published this week. The imaginary diet did not show a significant placebo effect on fat mass. However, the effect on body mass and body mass index remained significant.

Does this mean that the placebo effect of an imaginary diet is fiction? Is it merely the product of a wild imagination? Not necessarily. Instead, we have an open question to explore more fully and with a bit more rigor.

Click here for the original paper and here for the re-analysis.

Spaniel Chasing Imaginary Sticks, photograph © Alan Piper / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

May 1, 2020