Is Reviewer 2 the Source of All Our Woes?
Welcome to the world of peer review. This is the vital tool for bringing you science that you can trust. Legions of earnest experts look over research papers for strengths and weaknesses and overall merit before they appears in press. But urban legend holds that there’s a villain lurking in the process – Reviewer 2. There’s even a Facebook group with more than 35,000 members vowing that “Reviewer 2 must be Stopped!”
So naturally, inquiring minds want to know: Is Reviewer 2 really all that bad? Thus we have a peer-reviewed study to dispel the myth of Reveiwer 2’s malevolence. Writing in Social Science Quarterly, David Peterson tells us:
Reviewer #2 is not the problem. Reviewer #3 is. In fact, he is such a bad actor that he even gets the unwitting Reviewer #2 blamed for his bad behavior.
Data from 1,323 Manuscripts
Peterson analyzed categorical data from four years of reviews of manuscripts for Political Behavior. He compared the ordinal score that each reviewer gave a manuscript and looked for evidence that a reviewer gave scores more than one category below the scores that other reviewers gave. The central question was simple. Was Reviewer 2 more likely to give a much lower score than other reviewers?
The short answer is no.
It turns out that Reviewer 3 is the negative outlier – the most likely source of unfairly negative reviews. At least for this journal during four years of reviews. Peterson goes one step further and calls Reviewer 3 a “crafty cretin” who sinks the work of peers while having Reviewer 2 take the blame.
A Necessary Thorn
Peer review is important. In the midst of a pandemic, we wait for peer review of scientific papers so that we can trust the insights they bring. It helps to weed out unreliable research in obesity, nutrition, and health. Even after publication, peer review continues and sometimes results in calls for a retraction. This is evidence that peer review is still working.
It’s vitally important. Even if Reviewer 3 is a thorn in our side.
Click here for Peterson’s paper.
Pier Review, photograph © Kevin Dooley / flickr
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
July 7, 2020
