The Problem with Obesity Scorecards

For many years, the Trust for America’s Health has been cranking out obesity scorecards. In concert with the Robert Wood Johnson Foundation, they’ve just rolled out the 2016 edition of The State of Obesity. More a media tool than an objective analysis, this report generates an impressive volume of headlines:

Do You Live in One of America’s Fattest States?

WVa Near Top of Obesity List

Obesity Rates Decline in Four States

Louisiana Has Higher Rate of Adult Obesity than Any State

Modest Progress or a New Normal in War on Obesity?

The trouble is that the data used to generate this report comes from notoriously unreliable self-reports. The numbers are a compilation what people claim their height and weight to be when called on the telephone. And those reports are always skewed by what people wish were the truth. People generally report being taller and lighter than their actual measurements.

So, the unfortunate truth is that we don’t have a reliable ranking of obesity rates. In two recent publications, a group of investigators have looked at problems with estimates of both childhood and adult obesity based on these self reports.

In the case of childhood obesity, they found that the resulting errors range “from a 278% overestimate among males aged 2–5 years to a 44% underestimate among females aged 14–17 years.” In adults, they found a 23% underestimate of severe obesity.

But more important is the unreliability of the trends and rankings that are at the heart of these PR reports. The misreporting is not stable over time or across regions. In different times and different places, people tend to shade the truth a bit more or less.

In a 2014 study, Ahn Le and colleagues found that measured data for height and weight shows a very different regional pattern in obesity rates than self-reported data do. When people actually step on the scales, it turns out that the highest obesity rates are in the North Central regions of the U.S., not in the South Central regions.

The conclusion from self reports was the opposite.

Likewise, campaigns to urge people to move more, eat less, and weigh less can lead people to skew their self-reported weights more and more over time. But self-reports that show progress are convenient. They’re especially convenient if the goal is to report progress resulting from all of the good work you are doing.

Nevermind that the rate of obesity in women just hit a record high of 40% in 2014. That fact cannot be found in the State of Obesity. It doesn’t fit the narrative that all is proceeding according to plan.

Click here for the State of Obesity Report and here for more on the subject from Medscape. Click here and here for the studies of errors in self-reported height and weight. Click here for the study by Le et al on regional differences in misreporting.

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.

September 6, 2016

2 Responses to “The Problem with Obesity Scorecards”

September 06, 2016 at 2:40 pm, Amy Endrizal said:

I was floored to learn that my local health department’s obesity report, broken down by neighborhood and zip code, are based on self-reported weight and height data provided to the DMV! As you point out, people tend to dial down their weight and ramp up their height (maybe even more so for driver’s licenses that people carry everywhere and show to others). Worse, the data may be almost 10 years old! Budgets, deadlines, and convenience are legitimate considerations, but not at the expense of rotten data. GIGO.
- September 06, 2016 at 4:03 pm, Ted said:
  
  GIGO indeed. Just another sign that people are not taking this seriously.