A Cluster of Unreliable Prevention Studies
The search for effective prevention strategies in obesity is daunting. For decades now, researchers have been casting about for effective ways to educate, nudge, or cajole groups of people into moving more and eating less or better. Trying to influence a group of people means that controlled studies of interventions can wind up being cluster-randomized studies. But these cluster randomization trials can yield unreliable conclusions about obesity prevention if the analysis does not account for the clusters.
You might think this seems like a trip into statistical weeds, but it might be surprising to realize how often these cluster analysis problems crop up.
A Montessori School Nutrition Program
In the Journal of School Health, Nicole Vitale and Catherine Coccia reported on a program aimed at correcting poor dietary habits that can lead to chronic disease later in life. Their study population was children four to six years old in a Montessori school. Their nutrition education program focused on fruits and vegetables. They had 22 children in their intervention group and 29 in a control group.
Based on increased vegetable intake, nutrition knowledge, and liking of tomatoes, Vitale and Coccia concluded that their program was effective.
But there’s just one little problem. They did not account for the clustering in their study design when they analyzed the results. Thus in a letter to the editor, Lilian Golzarri-Arroyo and colleagues from the Indiana University School of Public Health tell us that these conclusions are unreliable:
“Vitale and Coccia used repeated measures ANCOVA and accounted for correlation due to repeated measures. However, neither clustering at the classroom nor at the block level was taken into account.
“As a result of the invalid statistical analysis choices, the reported results and conclusions of the study are unsubstantiated. For correcting the scientific record, reanalysis of the data using a valid alternative approach and republication of the study are warranted.”
Flustered by Clusters
Daphne Sze Ki Cheung and colleagues recently reported on the effectiveness of a music with movement program for preventing anxiety, depression, and stress in persons with dementia and their caregivers. But they didn’t account for clustering in their analysis. Marleigh Hefner and colleagues explained this issue with a letter to the editor. In their response, Cheung et al brushed off the concern:
“Various teams of authors adopted a similar analytical approach too.”
They’re exactly right on this point. Lots of people make such mistakes in analyzing clustered randomization trial designs. We’ve reported on it before. So maybe it’s time for editors and reviewers to be alert to problems with cluster analyses in prevention studies – especially when the clusters are neglected.
Click here for the study by Vitale and Coccia and here for the LTE about it. On the study by Cheung et al, click here for the original paper, here for the letter about the problems with it, and here for the response. For more on the analysis of cluster randomization designs, click here.
Bee and a Grape Cluster, photograph by Marya, licensed under CC BY 2.0
Subscribe by email to follow the accumulating evidence and observations that shape our view of health, obesity, and policy.
March 18, 2023
