Head Spinning Bias About Funding Bias

Head Spinning Bias About Funding Bias

“Don’t trust research funded by industry.” Suspicion runs deep about commercial funding for research, especially in nutrition. So, we imagine many heads were nodding recently when Justin Rankin and colleagues reported a signal of funding bias in obesity studies.

Consistent with prevailing beliefs, their report suggested that industry-funded studies were the most likely to selectively report results. But it turns out that precisely the opposite was true.

Selective Outcome Reporting in Obesity Research

Rankin et al took a close look at an important aspect of scientific integrity – specifying a primary outcome up front and sticking with it. They examined 142 obesity studies and found discrepancies in reported outcomes for 15% of them. Sometimes, the reports demoted a primary outcome. They said that it was secondary or even less important. In other cases, a primary outcome appeared after the fact. Nearly a third of the studies they examined had no registration records.

In other words, they found a real problem with selective outcome reporting.

Funding Bias

But they went further and broke down reporting problems according to the source of funding: public agencies, private agencies, or industry. According to popular beliefs, selective reporting might be more common in industry-funded studies. And that’s what Rankin et al reported in their paper. They said:

Industry-funded trials were most likely to contain at least one major discrepancy.

But in a letter published this week, David Allison and Diana Thomas pointed out that the opposite was true:

Our reading of Rankin et al.’s Table 3 indicates that industry-funded trials were least likely to contain at least one major discrepancy. That is, of all funding sources considered, industry-funded trials had the greatest fidelity in reporting outcomes commensurate with those in the study registration. In fact, the rate of reporting discrepancies in industry-funded trials was roughly half that of publically-funded trials, and roughly one-fourth that of privately-funded trials.

Everyone Brings a Bias

It’s a reflex. If industry funds a study, then some folks will dismiss the results regardless of the merits of research.

But this example suggest that we may have a bias about bias. The fact is that bias comes from many sources. Objective research and scientific integrity serve an important purpose – discovering truth. Careful attention to data and methods is critically important for filtering out bias.

Perhaps it’s more important than making assumptions about funding bias.

Click here for the study by Rankin et al, here for the letter from Allison and Thomas, and here for the response from the authors. For a thoughtful essay on funding bias by Andrew Brown, click here.

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

August 8, 2017