Registration status and methodological reporting of randomized controlled trials in obesity research: A review

Evidence of misuse of nonparametric tests in the presence of heteroscedasticity within obesity research

BACKGROUND

Classic nonparametric tests (cNPTs), like Kruskal-Wallis or Mann-Whitney U, are sometimes used to detect differences in central tendency ( i.e., means or medians). However, when the tests' assumptions are violated, such as in the presence of unequal variance and other forms of heteroscedasticity, they are no longer valid for testing differences in central tendency. Yet, sometimes researchers erroneously use cNPTs to account for heteroscedasticity.

OBJECTIVE

To document the appropriateness of cNPT use in obesity literature, characterize studies that use cNPTs, and evaluate the citation and public sharing patterns of these articles.

METHODS

We reviewed obesity studies published in 2017 to determine whether the authors used cNPTs: (1) to correct for heteroscedasticity (invalid); (2) when heteroscedasticity was clearly not present (correct); or (3) when it was unclear whether heteroscedasticity was present (unclear). Open science R packages were used to transparently search literature and extract data on how often papers with errors have been cited in academic literature, read in Mendeley, and disseminated in the media.

RESULTS

We identified nine studies that used a cNPT in the presence of heteroscedasticity (some because of the mistaken rationale that the test corrected for heteroscedasticity), 25 articles that did not explicitly state whether heteroscedasticity was present when a cNPT was used, and only four articles that appropriately reported that heteroscedasticity was not present when a cNPT was used. Errors were found in observational and interventional studies, in human and rodent studies, and only when studies were unregistered. Studies with errors have been cited 113 times, read in Mendeley 123 times, and disseminated in the media 41 times, by the public, scientists, science communicators, and doctors.

CONCLUSIONS

Examples of inappropriate use of cNPTs exist in the obesity literature, and those articles perpetuate the errors via various audiences and dissemination platforms.

Persistent confusion in nutrition and obesity research about the validity of classic nonparametric tests in the presence of heteroscedasticity: evidence of the problem and valid alternatives

The use of classic nonparametric tests (cNPTs), such as the Kruskal-Wallis and Mann-Whitney U tests, in the presence of unequal variance for between-group comparisons of means and medians may lead to marked increases in the rate of falsely rejecting null hypotheses and decreases in statistical power. Yet, this practice remains prevalent in the scientific literature, including nutrition and obesity literature. Some nutrition and obesity studies use a cNPT in the presence of unequal variance (i.e., heteroscedasticity), sometimes because of the mistaken rationale that the test corrects for heteroscedasticity. Herein, we discuss misconceptions of using cNPTs in the presence of heteroscedasticity. We then discuss assumptions, purposes, and limitations of 3 common tests used to test for mean differences between multiple groups, including 2 parametric tests: Fisher's ANOVA and Welch's ANOVA; and 1 cNPT: the Kruskal-Wallis test. To document the impact of heteroscedasticity on the validity of these tests under conditions similar to those used in nutrition and obesity research, we conducted simple simulations and assessed type I error rates (i.e., false positives, defined as incorrectly rejecting the null hypothesis). We demonstrate that type I error rates for Fisher's ANOVA, which does not account for heteroscedasticity, and Kruskal-Wallis, which tests for differences in distributions rather than means, deviated from the expected significance level. Greater deviation from the expected type I error rate was observed as the heterogeneity increased, especially in the presence of an imbalanced sample size. We provide brief tutorial guidance for authors, editors, and reviewers to identify appropriate statistical tests when test assumptions are violated, with a particular focus on cNPTs.

A proposed standardized approach to studying attrition in pediatric weight management

Pediatric obesity is a major public health issue. Lifestyle and behavioral interventions are the foundation of pediatric weight management; however, intervention effectiveness is compromised when families (children, youth, and/or parent[s]) discontinue care prematurely. Intervention attrition minimizes the potential health benefits derived from interventions, results in inefficient use of health services resources, and can magnify health disparities. Most attrition research in pediatric weight management has been descriptive, highlighting the need to advance the field, both academically and clinically. Herein, we propose a standard approach to studying attrition in pediatric weight management interventions to enhance our understanding, elevate the quality of research, enable study-to-study comparisons, and inform strategies designed to mitigate its impact. We focus on three issues. First, "Conceptualization and operationalization," whereby the processes underlying attrition from interventions should be decomposed into clinically important phases that are defined based on intervention characteristics. Relatedly, theoretically relevant variables should be identified with different mechanisms driving attrition in each phase. We propose a matrix of attrition, a tool designed to delineate the relevant stages of attrition and associated variables of analytical value. Second, "Pre-study" underscores the value of developing a plan to study attrition a priori rather than post hoc, including variable and sample size considerations, which broadens the range and quality of analysis. Finally, "Post-study" emphasizes comprehensive reporting of attrition, outlines typical comparisons of analytical interest, and statistical techniques used to handle missing data. Implications for clinical practice in pediatric weight management are discussed. Clinical Trial Registration: Not applicable.

Registration of published randomized trials: a systematic review and meta-analysis

BACKGROUND

Prospective trial registration is a powerful tool to prevent reporting bias. We aimed to determine the extent to which published randomized controlled trials (RCTs) were registered and registered prospectively.

METHODS

We searched MEDLINE and EMBASE from January 2005 to October 2017; we also screened all articles cited by or citing included and excluded studies, and the reference lists of related reviews. We included studies that examined published RCTs and evaluated their registration status, regardless of medical specialty or language. We excluded studies that assessed RCT registration status only through mention of registration in the published RCT, without searching registries or contacting the trial investigators. Two independent reviewers blinded to the other's work performed the selection. Following PRISMA guidelines, two investigators independently extracted data, with discrepancies resolved by consensus. We calculated pooled proportions and 95% confidence intervals using random-effects models.

RESULTS

We analyzed 40 studies examining 8773 RCTs across a wide range of clinical specialties. The pooled proportion of registered RCTs was 53% (95% confidence interval 44% to 58%), with considerable between-study heterogeneity. A subset of 24 studies reported data on prospective registration across 5529 RCTs. The pooled proportion of prospectively registered RCTs was 20% (95% confidence interval 15% to 25%). Subgroup analyses showed that registration was higher for industry-supported and larger RCTs. A meta-regression analysis across 19 studies (5144 RCTs) showed that the proportion of registered trials significantly increased over time, with a mean proportion increase of 27%, from 25 to 52%, between 2005 and 2015.

CONCLUSIONS

The prevalence of trial registration has increased over time, but only one in five published RCTs is prospectively registered, undermining the validity and integrity of biomedical research.

Methodological and reporting quality in laboratory studies of human eating behavior

The methodological quality and reporting practices of laboratory studies of human eating behavior determine the validity and replicability of nutrition science. The aim of this research was to examine basic methodology and reporting practices in recent representative laboratory studies of human eating behavior. We examined laboratory studies of human eating behavior (N = 140 studies) published during 2016. Basic methodology (e.g., sample size, use of participant blinding) and reporting practices (e.g., information on participant characteristics) were assessed for each study. Some information relating to participant characteristics (e.g., age, gender) and study methodology (e.g., length of washout periods in within-subjects studies) were reported in the majority of studies. However, other aspects of study reporting, including participant eligibility criteria and how sample size was determined were frequently not reported. Studies often did not appear to standardize pre-test meal appetite or attempt to blind participants to study aims. The average sample size of studies was small (between-subjects design studies in particular) and the primary statistical analyses in a number of studies (24%) were reliant on very small sample sizes that would be likely to produce unreliable results. There are basic methodology and reporting practices in the laboratory study of human eating behavior that are sub-optimal and this is likely to be affecting the validity and replicability of research. Recommendations to address these issues are discussed.