Peer Review: Stressed or Broken?
Lapses in peer review, scientific integrity, and reporting on scientific studies have been getting extraordinary attention lately. Is this a sign of rigorous attention to an important matter or a sign of a system that isn’t working quite right?
Two incidents have garnered particular attention. First was the recent retraction of research on changing attitudes about gay marriage from the journal Science. This resulted when other researchers, digging into the details of the study, found evidence that methods and data may have been misrepresented. The incident attracted wide attention because of the journal’s prominence and the wide reporting that the study had received from news media.
The other incident is a deliberate ruse by John Bohannon, a science journalist. Bohannon, working with one doctor and a group of friends in science media and PR, organized a deliberately flawed dieting study that purported to prove dark chocolate could accelerate weight loss. They purposefully violated basic principles of statistical analysis to get a result they could represent as significant. Then they sent it out to scientific journals that were hungry for buzz-worthy papers authors would pay to publish. In no time, their bogus findings were published in the International Archives of Medicine and were generating headlines around the world. “Peer review” by the journal resulted in not one word of the paper being changed before publication.
What does all this mean?
The New York Times offers up two factors to consider. Noteworthy retractions of peer-reviewed papers seem to be growing. The Times published an impressive list of noteworthy examples. They also reported this week on the growing inclination of some scientific journals to publish sensational studies to attract attention in news media.
We don’t have empiric evidence of a cause and effect relationship. We do have a strong suspicion that the hunger for a big media splash might be facilitating publications that, upon reflection, don’t pass muster.
It’s good that these lapses are getting attention. Maybe it’s a sign that peer review is stressed, but working well. But we wonder about undetected errors. Critical thinking is essential, but it’s not always popular.
Click here to read more from the New York Times on seeking a media splash for research publications. Click here for more on the chocolate weight loss ruse.
Markup, photograph © Wonderlane / flickr
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
