Seeking Gold in the Myths of Self-Reported Weight
Across four millenia, alchemists sought to purify common materials and transform them into gold. In a possibly similar quest, Zachary Ward and Steven Gortmaker wish to assure us they have a reliable method for correcting the errors in data on self-reported height and weight. In other words, it’s no big deal if we don’t have actual measurements of height and weight in studying obesity. They can transform self-reports into something that’s just as good.
Katherine Flegal, Barry Grubard, and John Ioannidis beg to differ. Last year, they wrote:
“The suggested method has little effect on misclassification, can introduce new errors, and could magnify errors associated with factors, such as age, race, educational level, or other characteristics. State-level estimates and projections of obesity prevalence from values adjusted by this method may be incorrect.”
Can We Decode the Fibs People Tell About Height and Weight?
It’s a fact. People tell fibs about their height and weight. Extra height is desirable characteristic. So self-reports of height come out higher than reality. Even a president might fudge on self-reported height.
But extra weight is not so desirable in our fat-phobic culture. So those self-reports run low.
Back in 2019, Ward and colleagues published a paper in the New England Journal of Medicine. In that paper, they claimed they had methods to fix all of this. Their methods “correct for self-reporting bias,” they wrote.
Flegal and colleagues had doubts, though. So they published an analysis of a large data set with both self-reported and measured height and weight. They tested the ability of Ward’s method and found that it fixed some errors, but added new ones. This reminds us of the old maxim of garbage in, garbage out.
From there we have an exchange of responses from Ward and Flegal. Ward says he’s offering a method that “demonstrably” improves BMI estimated from self-reports. Flegal replies that Ward has “no evidence” of “any net improvement.”
The Alchemy of Self-Reports
The core problem here is that different people fib about height and weight in different ways, for different reasons. The misreporting and the reasons for it are invisible unless we have actual measurements.
This is a problem in obesity and nutrition research. It’s tough to get actual measures of height, weight, diet, and exercise. But self-reports make lousy substitutes.
Alchemists never did get gold from base metals. When people claim to have “methods” to make unreliable self-reports into good measures of obesity, it sounds a lot like alchemy.
Click here for Ward’s original publication in NEJM, here for Flegal’s analysis of the method in it, here for Ward’s dispute of that analysis, and here for Flegal’s reply.
The Alchemist in Search of the Philosophers Stone, painting by Joseph Wright of Derby / WikiArt
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
January 31, 2022
February 02, 2022 at 6:51 pm, Katherine Flegal said:
For anyone interested in further consideration of self-reported vs. measured weight and height, we also published a paper several years ago showing how self-reported BMI can lead to bias. In fact the bias can be such as to actually reverse the direction of a hazard ratio: Flegal KM, Kit BK, Graubard BI. 2018. Bias in Hazard Ratios Arising From Misclassification According to Self-Reported Weight and Height in Observational Studies of Body Mass Index and Mortality. Am J Epidemiol 187: 125-34
A common (very common!) error is to assert that because the correlation between measured vs reported BMI is characteristically high, around 0.97, that means that there is good agreement. Bland and Altman in 1986 already discussed why correlation is not a measure of agreement, but this gets widely ignored. However, the article by Rimm et al showing a .97 correlation between measured weight and reported weight also shows a misclassification into quintiles of almost 50%, nicely illustrating the point that a high correlation does not indicate good agreement.
February 04, 2022 at 2:14 pm, Katherine Flegal said:
The underlying problem is that the errors in self-reported weight and height are systematic and vary with the true values. Someone who is 6’5″ is less likely to overreport height than someone who is 5’11”. Someone who weighs 250 lbs is likely to underreport weight more than someone who weighs 130 lbs, and someone whose weight is very low is not unlikely to overreport their weight. As a result, it’s fairly easy to estimate self-reported weight and height from measured weight and height, but very difficult to estimate measured weight and height from self-reported.