Three Holes in the Obesity Evidence Base

Three holes in the obesity evidence base are hidden from view for many people and yet people who know the obesity literature know that they lie in wait to trap the unsuspecting. The effect is to fool people into thinking we know more than we do about obesity. People routinely accept hypotheses and preliminary findings as definitive, rather than forge ahead with research that will prove or refute the point. These three holes have critical importance:

  1. Food Intake. Given the critical importance of dietary intake data, it’s stunning to think that so many analyses rely upon self-reports. Keeping a food diary is a key tool for behavioral weight management. But for scientific analysis, it’s inadequate. Dale Schoeller and a dazzling array of leading obesity researchers pointed out the need for an alternative in the American Journal of Clinical Nutrition last summer. The title — “Self-report–based estimates of energy intake offer an inadequate basis for scientific conclusions” — sums up their position concisely.  Validation data for yet another tool to collect objective energy intake data was recently published in IEEE Biomedical Engineering. Such tools are sorely needed.
  2. BMI. In obesity studies you will routinely see a disclaimer about self-reported height and weight as a limitation. It’s so routine that people ignore it like the fine print in a regulatory disclosure. But as Scott Keith and colleagues pointed out in 2006, it’s worth remembering that this limitation “may account for important discrepancies in published findings.” Eight years later, scientific publications continue to minimize or ignore this caveat.
  3. Randomization. Over and over again we generate hypotheses about obesity through epidemiologic analyses that prove nothing about causality. As those analyses are repeated on different datasets, people gradually become more and more comfortable accepting the hypotheses as confirmed. This is how the idea that breastfeeding provides important protection for infants from obesity became entrenched. Even after definitive evidence reviews concluded that no benefit has been proven, these correlations continue to be accepted as cause and effect relationships. We need more definitive experiments that can prove something, and less speculative research to support popular theories.

“Most of our assumptions have outlived their uselessness.” — Marshall McLuhan

Click here to read about a novel device for monitoring food intake, here for more on the limitations of self-reported food intake, here for more on the limitations of BMI self reports, and here for more on the need for more genuine experimentation in obesity and nutrition research.

Leaf Detail at Phipps Conservatory, photograph © elston / flickr

Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.


6 Responses to “Three Holes in the Obesity Evidence Base”

  1. April 05, 2014 at 5:17 pm, Rose Aspalter said:

    Thanks für the stimulating articles!
    In this case I disagree. At least partially.
    More precise devices to monitor food intake and physical activity are actually of help. But
    1) the “novel” device to which you link here, isn’ recording what you are eating – and that is by f ar the most important information
    2) they do not even report the precise amount!
    However, in addition to a dietary protocol it could be used as a control system, reflecting propability of accuracy of the dietary records.

    As for self reported data: Let me ask the other way round: Why should they be less acurate than the investigators data? Or in other words: You trust the investigetors data. What is the rational to look at users data as less valid than the investigors data – who has an interest in “nice data”, too, 😉

    I developed an online program for longterm monitoring of calorie intake an consuption. User are keeping records on average for more than 100 days! It just doesn’t make sense, to put in false data for such a long time! you do it for the befenit of overview and control. So, in IMHO, accuracy is not guaranteed by the person, who measures and reports data, but by the interest of this person.

    Kind regards from Vienna!

    • April 05, 2014 at 7:06 pm, Ted said:

      Thank you for your thoughtful comments. And thanks for giving thought to the best way to address these questions.

  2. April 11, 2014 at 2:08 pm, Sam Yaffe said:

    I have patients who use food diaries to try to lose weight, and many of them admit to fudging their data, either because they don’t like the data, they find it discouraging, or its just too much trouble for them to record one little snack after another. It is often a challenge to help them find motivation adequate to keep them keeping accurate data.

    • April 11, 2014 at 2:35 pm, Ted said:

      Good perspective. John Foreyt recently spoke at a meeting, offering the perspective that regardless of their poor accuracy for research purposes, food diaries are useful clinical tools.

  3. April 12, 2014 at 2:59 pm, Rose Aspalter said:

    Thanks Ted! in my opinion, even a “semiprecise” diary is better than no one.

    However, the problem reported by Sam, is a serious one.
    And ideed, that was exactly my experience.
    And thats why I started to devolope a program, initially just for my own private convenience, but than more and more for general use. It is now online under the Trademark KiloCoach (I am not sure, if it is wellcomed to name it here, but you can leave the name, if you prefer.)

    @ fudging their data:
    In my opinion, you overcome the problem if data entry is directly and immediatly percieved as useful. Unfortunately, only few programs manage to achieve this level of usability. (“Usability” in a double sense: 1) Users convenience, 2) Usefulness 😉 )
    If we hadn’t managed that, poeple wouldnt use it on average for more than 100 days!

    Nevertheless, users of the KiloCoach-Progrm sometimes report the same feelings. E.g. if they have a realy “bad day” in terms of the calory balance. We are helping pealple by the concept of a “compensation day” and by offering program-features which call them “back to line”: graphs who cannot be completed, if diary is incomplete, a weight ticker, the calorie-clock and other features …

    If “bad days” happen, the user must have other reasons to continue recording.

    • April 12, 2014 at 4:00 pm, Ted said:

      Good perspective. Thanks!