Are Randomized Controlled Trials Overrated?

Randomized controlled trials are either a nuisance or a godsend – just depending on the question at hand and the questioner. A recent kerfuffle about flossing has people (like Jamie Holmes at New America) suggesting that they might be overrated. Writing in the New York Times Holmes says:

Experiments, of course, are invaluable and have, in the past, shown the consensus opinion of experts to be wrong. But those who fetishize this methodology, as the flossing example shows, can also impair progress toward the truth. A strong demand for evidence is a good thing. But nurturing a more nuanced view of expertise should be part of that demand.

The Associated Press stirred up quite a controversy with a report in August that asserted “there’s little proof that flossing works.” Controversy sells newspapers and racks up lots of clicks, so this story was quite a success. But did it inform readers or leave them confused?

In retrospect, this controversy gave us the worst of both worlds. We had a news cycle that left the public with unwarranted doubts about flossing. Nobody seriously suggests that risks might outweigh the benefits of flossing. Making matters worse, we didn’t get any enlightenment to offset the confusion. Randomized controlled studies can’t answer some questions. But a helpful discussion of that problem was mostly lacking in the flossing story.

Dealing with the limits of experimental evidence has special importance in questions related to nutrition and obesity. Dietary recommendations are pieced together from a patchwork of experimental evidence and epidemiology. The recent reversal on decades of advice about cutting fat consumption shows that big mistakes are possible when experimental evidence is lacking.

Obesity, poor nutrition, and sedentary lives are driving a global pandemic of chronic diseases. Solving that problem will require critical thinking, high standards of scientific rigor, and research with the power to resolve critical questions about what works and what doesn’t.

Expert opinion and epidemiology are necessary but insufficient inputs for sound decision making. Randomized controlled trials and other forms of probative research are essential.

For more on probative research in obesity, click here and here.

Experiment, photograph © in transition / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

November 27, 2016