Obesity Prevention: Proving a Point or Seeking the Facts?
It sounds wonderful. This program is ideal for integrating health promotion more intensively into the everyday life of children. Best of all, it only takes is a very low dose of encouragement to Join the Healthy Boat. It’s easy and effective, say the folks who designed and tested this incredible program. But we can’t help but wonder. Were these researchers seeking the facts of how well their wonderful program works? Or were they just proving a point?
A Cluster Randomized Study
The study of this program used a design called cluster randomization. That means that assignment to the experimental and control groups was not by individuals. Rather, it was by groups. All the children in 30 kindergartens received the Healthy Boat program. In the control group, children in 27 other kindergartens received nothing except a baseline and a follow-up assessment. Because of this design, the study’s analysis plan must account for the clusters.
Investigators reported that they found a significant effect on BMI percentiles, physical activity, and endurance.
Questions
However, in a letter to the journal, Colby Vorland and colleagues raised a series of four questions about the analysis. First, it did not appear to account adequately for the clustering or a 43 percent dropout rate. Second, the authors switched their primary and secondary outcomes specified in the analysis plan. Third, it was not clear that the effect on BMI percentiles was significant when controlling for migration background in their full model. Fourth, parental questionnaires for collecting health behavior data introduced the possibility of social desirability bias.
Vorland et al suggested that a re-analysis would be helpful.
“No Need to Reanalyze”
In their response, Susanne Kobel et al did not agree. But we have a hard time following their logic. Their statistical tests were “explorative,” they wrote. Not confirmatory. Nevertheless, they went on to confirm that they “were able to demonstrate a significant impact on outcomes.”
Reading all of this, we honestly don’t know what to think. Except, perhaps, these researchers really believe in their program. Thus, we’re left with a simple question. Were they seeking facts? Or instead, simply seeking to prove a point?
Click here for the study, here for questions from Vorland et al, and here for the response. For more on ways to avoid some of these problems, click here.
Hide & Seek, photograph by Spyros Papaspyropoulos, licensed under CC BY-NC-ND 2.0
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
December 11, 2019
