Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis

OBJECTIVES

To assess the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.

DESIGN

A cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, we used random effects models to determine the impact of unregistered studies on meta-analytic results.

MAIN OUTCOMES AND MEASURES

We report the proportion of registered but unpublished trials and the proportion of published but unregistered trials. We also assessed whether primary outcomes were consistent between registration and publication. For meta-analyses, we used standardised mean differences (SMDs).

RESULTS

Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 50% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: -0.53, 95% CI -0.87 to -0.20) than registered RCTs (SMD: -0.14, 95% CI -0.35 to 0.07).

CONCLUSIONS

Registration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.

Homeopathic Preparations for Preventing and Treating Acute Upper Respiratory Tract Infections in Children: A Systematic Review and Meta-Analysis

BACKGROUND

Acute upper respiratory tract infections (ARTIs) are common and mostly self-limiting. A range of treatments are used with the aim to cure or treat symptoms, including widespread use of homeopathic treatments.

OBJECTIVE

To undertake a systematic review and meta-analysis of trials with the highest level of evidence, to establish the benefits and risks for oral homeopathic remedies used to treat and prevent ARTIs in children.

DATA SOURCES

MEDLINE, Embase, CINAHL, AMED, CAMbase, British Homeopathic Library, CENTRAL, WHO ICTRP and ClinicalTrials.gov registers to March 2018.

STUDY ELIGIBILITY, PARTICIPANTS, AND INTERVENTIONS

Double-blinded randomized trials in children, treated with oral homeopathic remedies versus placebo or conventional treatments for ARTI.

APPRAISAL AND SYNTHESIS METHODS

Studies were reviewed in duplicate for inclusion, data extraction, and risk of bias. Meta-analysis was performed on only 4 outcomes. Other outcomes were reported narratively.

RESULTS

Eight studies (1562 children) were included. Four studies examined treatment and 4 prevention of ARTIs. Four studies involved homeopaths individualizing treatment versus four with non-individualized treatments. Three studies had high risk of bias in at least 1 domain. All studies with low risk of bias showed no benefit from homeopathy; trials at uncertain and high risk of bias reported beneficial effects. Two individualized treatment studies (N = 155) did not show benefit on short-term or long-term cure. Prevention trials showed no significant outcomes: recurrence of ARTIs. No serious events were reported.

LIMITATIONS

Methodological inconsistencies and heterogeneity.

CONCLUSIONS

The effectiveness for homeopathic remedies for childhood ARTIs is not supported in higher quality trials.

Homeopathic Remedies in Psychiatric Disorders: A Meta-analysis of Randomized Controlled Trials

PURPOSE/BACKGROUND

Homeopathy is a complementary and alternative medicine. Conclusive evidence on the plausibility, efficacy, and safety of these treatments is not currently available. Nonetheless, homeopathic remedies (HRs) are widespread throughout the world and especially in mental disorders. The aim is to assess the efficacy of HRs in the treatment of mental disorders.

METHODS/PROCEDURES

We performed a Medline/Embase search for studies written in English and published from any date to October 23, 2018. All randomized controlled trials enrolling patients with any psychiatric disorder and comparing HR with placebo, no treatment, or other psychotropic drugs were included.

FINDINGS/RESULTS

A total of 212 studies were screened, 9 met all selection criteria and reported data on major depressive disorder (MDD) (n = 4), generalized anxiety disorder (n = 1), attention-deficit/hyperactivity disorder (n = 2), and premenstrual syndrome/dysphoric disorder (n = 2). Eight of 9 randomized controlled trials showed high risk of bias. Homeopathy showed greater efficacy in MDD compared with fluoxetine, and in premenstrual syndrome/dysphoric disorder compared with placebo, whereas no difference emerged between homeopathy and placebo in MDD and attention deficit/hyperactivity disorder.

IMPLICATIONS/CONCLUSIONS

Available data on homeopathy in psychiatric disorders are insufficient to support their use in clinical practice.