Three Conclusions Disconnected from Data

It’s pretty common to read headlines about obesity and nutrition studies that have nothing to do with what the studies actually showed. Sadly, we’re used to seeing this a lot. But more commonly than you would think, the conclusion in a study itself is disconnected from the study’s actual findings. Reader beware.

Here are three recent unfortunate examples:

  1. Water ≠ Energy-Dense Food. Investigators in a new study published in Obesity measured fullness in children and adolescents after drinking water. They found that the youth with obesity drank 20% more water before they felt full. They concluded that “A reduction in energy-dense foods in the diet of obese children and adolescents appears to be a necessary strategy for managing body weight.” The trouble is that they did not study the energy density of foods.
  2. Menu Labeling ≠ Healthier Choices. MMWR recently published an analysis of data from the Behavioral Risk Factor Surveillance system on consumer use of menu labeling for calorie information in restaurants. They found that 66% of adults who have seen the information either never (43%) or only sometimes (23%) use it. They concluded that “for persons who want to reduce their caloric intake at restaurants, ML can help them select items with lower calorie content.” The trouble is that they did not study the impact of labeling on food choices.
  3. Making Kids Concerned about Their Weight ≠ Preventing Childhood Obesity. In Preventing Chronic Disease, researchers recently published a study that found  roughly a quarter of children and adolescents underestimated their weight status. Likewise, parents underestimated their children’s weight status. They observed a lower likelihood of trying to lose weight when any of these groups underestimated weight status. They concluded that “efforts to prevent childhood obesity should incorporate education for both children and parents regarding the proper identification and interpretation of actual weight status.” The trouble is that they collected no data on the prevention of childhood obesity. They did not even collect data on the feasibility of persuading these people that they have obesity.

Obesity and nutrition elicit strong feelings and opinions. Such strong feelings may contribute to the risk of confirmation bias. People look for information that supports their beliefs. Sometimes they discount information that doesn’t.

Such bias is a source of human error that scientists and peer reviewers must guard against through disciplined thinking.

“Enough research will tend to support your conclusions.” — Arthur Bloch

Click here to read the study of fullness, here to read the study of menu labeling, and here to read the study of weight status misperceptions. Click here to read more about confirmation bias.

Disconnected, photograph © larsomat / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


2 Responses to “Three Conclusions Disconnected from Data”

  1. August 05, 2014 at 6:18 am, Mary-Jo said:

    It is so important to choose the right variables and tests for investigations so as to get unbiased information that either supports or negates your hypothesis(es) and then to present appropriate conclusions. We can infer or suggest or further speculate, based on the data, but those types of ‘observations’ should also be stated clearly, as such. Everyone is so desperately trying figure out how to contain this obesity epidemic, the topic seems to be fair game for all kinds of headline manipulation and sensational claims — even in the better publications. We really have to be careful about all of this — it helps nothing and no one. I think it was Benjamin Disraeli who said, ‘There are three types of lies — lies, damn lies, and statistics.’

    • August 05, 2014 at 6:25 am, Ted said:

      All true. Thanks, Mary-Jo!