Are Randomized Controlled Trials Overrated?

Randomized controlled trials are either a nuisance or a godsend – just depending on the question at hand and the questioner. A recent kerfuffle about flossing has people (like Jamie Holmes at New America) suggesting that they might be overrated. Writing in the New York Times Holmes says:

Experiments, of course, are invaluable and have, in the past, shown the consensus opinion of experts to be wrong. But those who fetishize this methodology, as the flossing example shows, can also impair progress toward the truth. A strong demand for evidence is a good thing. But nurturing a more nuanced view of expertise should be part of that demand.

The Associated Press stirred up quite a controversy with a report in August that asserted “there’s little proof that flossing works.” Controversy sells newspapers and racks up lots of clicks, so this story was quite a success. But did it inform readers or leave them confused?

In retrospect, this controversy gave us the worst of both worlds. We had a news cycle that left the public with unwarranted doubts about flossing. Nobody seriously suggests that risks might outweigh the benefits of flossing. Making matters worse, we didn’t get any enlightenment to offset the confusion. Randomized controlled studies can’t answer some questions. But a helpful discussion of that problem was mostly lacking in the flossing story.

Dealing with the limits of experimental evidence has special importance in questions related to nutrition and obesity. Dietary recommendations are pieced together from a patchwork of experimental evidence and epidemiology. The recent reversal on decades of advice about cutting fat consumption shows that big mistakes are possible when experimental evidence is lacking.

Obesity, poor nutrition, and sedentary lives are driving a global pandemic of chronic diseases. Solving that problem will require critical thinking, high standards of scientific rigor, and research with the power to resolve critical questions about what works and what doesn’t.

Expert opinion and epidemiology are necessary but insufficient inputs for sound decision making. Randomized controlled trials and other forms of probative research are essential.

For more on probative research in obesity, click here and here.

Experiment, photograph © in transition / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


November 27, 2016

One Response to “Are Randomized Controlled Trials Overrated?”

  1. November 27, 2016 at 6:55 am, Al Lewis said:

    The wellness industry to the rescue! On many occasions, these Einsteins prove exactly the opposite of what they intend to prove without realizing it. In this case, reading the link (and the comments!) will show that while attempting to prove that people with a gene for obesity who are coached can lose so much weight in 12 months that they massively reduce healthcare spending, they ended up showing the reverse.

    By doing so, they showed something much more fundamentally valuable: that using the same sample, an RCT yields a vastly different and far more credible result than a participants-to-non-participants comparison. http://thehealthcareblog.com/blog/2015/12/16/genetic-testing-the-new-frontier-of-wellness-madness/