Spinning Out of Control
Is scientific spin spinning out of control? Shall we blame reporters, journal editors, university press officers, researchers, or an old favorite, industry?
The most recent case in point emanates from the New England Journal of Medicine. That venerable publication assiduously cultivates an image of stodgy, conservative, academic credibility. But they also have a YouTube channel to prove that they can keep up with the times and pimp their academic wisdom on social media. So for a recent publication of an important study about a new obesity treatment, they prepared one of their NEJM QuickTake videos to post alongside the study and on YouTube (embedded here).
The trouble is that in their animated illustration of the results, they visually exaggerated the clinical effect of the treatment studied. They showed waist circumference shrinking by 11% while the study only found a reduction of 7%. This is the kind of mistake that big pharma or big food would be loathe to make because FDA or FTC would be all over them.
Three sharp-eyed experts (David Allison, Diana Thomas, and Steven Heymsfield) noticed the discrepancy, measured the visuals, and pointed out the error in PubMed Commons, a forum for sharing perspective on the medical publications. Asked to comment on the situation, Allison focused on the importance of accuracy, saying:
As health scientists, we should take quantification seriously and try to present clear and accurate information to each other and the public at every turn.
There’s quite a literature building on the misrepresentation of medical research findings. Some of the concerns involve errors in statistical methods that get buried or overlooked. Misrepresentation of observational studies is a common and seemingly growing concern. Wang et al assessed this problem in a recent research letter published by JAMA Internal Medicine and called for greater attention to the limitations of observational research.
More troubling, though, is the concern that results are sometimes overstated in peer-reviewed literature and exaggerated in academic press releases and health news reporting. Worth noting is an analysis by Yavchitz in PLOS Medicine that found “spin” in roughly half of the press releases and media coverage of randomized controlled trials. They conclude that journal reviewers and editors bear responsibility for eliminating this problem, because the seeds for it are planted in peer reviewed abstracts.
The lazy assumption that bias comes only from industry funding needs to die. Bias can come from anyone who believes in their own work and their institution. Scientific rigor is the only answer.
Click here for the critique of the NEJM video and click here for an analysis of the rising rate of retractions for flawed publications.
Spin (spiral stairs of the Vatican Museums), photograph © Maciek Lulko / flickr
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
August 6, 2015
