Data Thugs, Pajamas, and Ultra-Processed Food

Data Thugs, Pajamas, and Ultra-Processed Food

Nevermind. As far as we can tell, that’s the bottom line on a convoluted story about data thugs, pajamas, and a provocative study of ultra-processed food. It starts with the study by Kevin Hall et al that found ultra-processed food can cause people to eat more and gain weight. All by itself, that finding was enough to spark intense debates. Then, on the eve of the inauguration holiday, self-described data thugs stepped up to raise questions. They thought they had found issues with the data.

What ensued were three blog posts – including one from Retraction Watch – as well as many updates to those posts. Along the way, intense and confusing communication went back and forth. Questions went to Hall on January 19. On the 21st, before Hall could answer all of those questions, those posts went public.

It took a few more days, but in the end, the net result was a measure of satisfaction. One of the bloggers, Nick Brown, summed it up saying:

“I believe that these responses adequately address all of the points that I made in the original post.”

Simply stated, this study’s findings withstood intense scrutiny.

The Weight of Pajamas?

These folks really dug into the details of the data coming out of Hall’s provocative study. One bone of contention was the weight of pajamas. This was a tightly controlled study. Subjects lived for a month in a dorm setting where researchers controlled what they ate and drank. They weighed the subjects every morning at six am – in pajamas from the clinic. So they weighed the pajamas, too. The needed to deduct pajama weight from total weight to get a precise measure of body weight.

However, the data thugs found issues with the details of how all this weighing was done. They really dug into it because they thought they saw non-random pattern in the significant digits. They wondered if this meant the data was not totally reliable.

When you have a big, complicated study like this one, many such questions can arise – and they did.

Transparency and Rigor

All of this creates a very uncomfortable, but important situation. Kevin Hall, who found himself on the hot seat here, explains the importance:

“While I knew we had not engaged in scientific misconduct, even the best scientists make errors and it’s very important to correct the scientific record. We do this regularly, but it’s uncomfortable when others claim to have found errors, or worse, evidence of misconduct.”

So the rigor of the inquiries and the transparency of the researchers serve an important purpose. They serve to assure us these data on ultra-processed food are reliable.

Humility and Grace

This episode also points to the importance of humility and grace. As Hall notes, even the best scientists make mistakes. Data thugs do, too. Humility is essential so that we can recognize mistakes. Grace is necessary to make it possible for people to acknowledge them.

Everyone seems to agree this was a useful exercise that led to a better understanding of a very important study. But still, we can debate how graceful the data thugs were in this situation. From our perspective, it seems they imposed time pressures that only added confusion, not clarity.

Click here, here, and here to read the three blogs about this important study. For more about this experience from Hall’s perspective, you’ll want to read this thread on Twitter.

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

January 30, 2021