Examining a Model

Model Assumptions for Driving Health Policy

“I was never involved in a model. At least this kind of a model,” said our President at a recent press briefing. But now it seems that all of us are getting a crash course in modeling and model assumptions for making health policy. Perhaps we will learn to think more critically about the output of such models as a result.

Assumptions Drive the Model

The important thing to know about models is that they are only as good as the assumptions you feed them. Every model has assumptions. And every good modeler makes their assumptions very explicit. That’s essential because the assumptions play a big part in the results a model produces. If you want to know how reasonable the results of a model are, then you have to look hard at the assumptions that produce the answer.

Even the best model, with flawed assumptions, can produce forecasts that don’t line up with reality.

We’re getting that lesson with models of how many people will die from the coronavirus. Early on, researchers at Imperial College London estimated that as many as 2.2 million Americans could die from the coronavirus pandemic. But the key assumption was no social distancing. Shortly after that, the benchmark was 100,000 to 200,000 deaths when the President came out with recommendations for social distancing.

Right now, it looks like it looks like we will reach the awful milestone of 50,000 deaths by the end of April. On top of that, the head of the CDC, Robert Redfield, says the second wave of this pandemic, next fall, could be even worse. Thus, numbers change when new data inform our assumptions.

Assumptions in Obesity-Related Policy

Lives are at stake now with the coronavirus, so people are paying really close attention to these models. That’s a good thing. Because often we see models to inform health policy about obesity with some really iffy assumptions.

Take, for example, the assumption that soda taxes will reduce the prevalence of obesity. It’s never been observed, but it’s widely assumed. In one model, that assumption was all it took for the modelers to conclude that soda taxes are more effective against obesity than bariatric surgery. From that assumption, we see headlines that claim “Soda taxes will save half a million kids from obesity.”

All Forecasts Are Wrong

The only thing certain about a forecast is that it will turn out to be wrong. What’s up for debate is how far wrong they will be. Models are quite helpful for producing forecasts, but they are only as good as their assumptions. By looking closely at those assumptions, we can think critically about how far wrong a forecast might be.

For good reasons, people are now paying close attention to the models and forecasts about the pandemic. If, in the process, we learn to think more critically about models and their assumptions, that will be a good thing.

Click here for a thoughtful examination of models for the pandemic and how they’re being reported. For a review of modeling studies of the obesity pandemic, click here.

Examining a Model, photograph © Fyn Kynd / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


April 22, 2020

2 Responses to “Model Assumptions for Driving Health Policy”

  1. April 23, 2020 at 3:18 am, Mary-jo said:

    I did a course on modeling for part of my epidemiology studies. One can be duped into thinking there is some spectacular AI occurring when you see the colorful, intricate, seemingly incredible curves, graphs, and statements generated. But, acknowledging all is based on assumptions entered is an understatement. There is a tremendous amount of subjectivity involved. One researcher can enter one set of data that they believe is robust vs. another researcher entering a whole different set of data that THEY believe is robust for the same question and very different results ensue. Models are amazing TOOLS, but as foci of strategy or policy decisions, not so much!

  2. April 23, 2020 at 5:31 pm, Richard Atkinson said:

    Models are abused by our political leaders for their own purposes and as noted above, the results depend heavily on assumptions, which very often are adjusted for the answer wanted. Of note in the above column is the assumption that distancing works and is critical. Very few countries tested that so data are limited. Sweden did and today’s deaths per million population is lower in Sweden than most other Northern European countries. Perhaps the utility of lockdowns can be tested by the current proposals to do a rolling relaxation of lockdowns. This is really important to study because with the next pandemic the precedent has been set that the economy has to go in the toilet – not a good assumption if not true!