Eliminating Scientific Error by Banning the Word

Correcting errors is such a nuisance. People feel bad about it. Some people simply can’t admit errors, so it can get messy. But errors are inevitable. Thus it’s really important to correct errors when they are discovered. Especially in scientific research. Nonetheless, one journal seems to have a different solution. Nutrients – an open-access nutrition journal – tells us that the journal has a policy banning the word “error” in the title of a letter to the editor correcting an error appearing in the journal.

Is this a blindingly obvious way to eliminate errors? Or is it blindness to errors?

The Tale of an Invalidating Error

This journey of discovery began with an interesting, small study of isomaltulose. We wrote about it back in January. On the surface, it looked like this sweetener might be a natural alternative to sucrose that leads people to eat less and lose a little bit of fat.

However, it turned out that the first impression in this case was deceiving. The study suffers from something called a Differences in Nominal Significance (DINS) error. That is an error that occurs when you have two experimental groups and one has a significant change from baseline, but the other does not. A naive – and false – interpretation is that the two groups are different. But what matters is a direct comparison of one group to the other.

In this study, a direct statistical comparison of the two groups showed no difference. Thus the claim that subjects in this study ate less and lost fat when they ate isomaltulose was false. The study did not prove this to be true.

Reluctance to Concede an Error

The scientists who discovered this error, Colby Vorland and Andrew Brown, invited us to join them in writing the journal to explain this error and ask them to correct it:

Unfortunately, there are errors in the statistical conduct and interpretation that do not support these conclusions.

The authors of the study disagreed. While they admit that they did not find a statistically significant difference between sucrose and isomaltulose, they insist that they found evidence that:

Might suggest that participants on a LGI [low glycemic index] diet with ISO achieve higher weight loss as well as fat mass loss compared to their counterparts in the SUC group.

This seems like a bit of a mea non culpa to us.

Banning Error

Most researchers are eager to identify mistakes and move on. This group was not. But even more interesting was the response of the journal. Our letter to the editor was all about a DINS error in this paper. So the title for our submission named it – a Differences in Nominal Significance Error.

Nope, said the journal: “We are not allowed to have the word like ‘error’ in the title.”

Unfortunately, banning “error” from your vocabulary does not eliminate errors. It merely makes them harder to identify and discuss.

Nutrients is an odd journal. In 2018, all of its senior editors resigned, alleging that the publisher pressured them to accept papers of mediocre quality and importance. Our experience with trying to correct the error in this paper makes us think the problem identified in 2018 might be persistent.

Click here for the original study, here for our letter to the editor, here for the authors’ response, and here for our further commentary on PubPeer.

Error on Green, painting by Paul Klee / WikiArt

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

August 8, 2020