From registration to publication: A study on Dutch academic randomized controlled trials

Trials that turn from retrospectively registered to prospectively registered: a cohort study of "retroactively prospective" clinical trial registration using history data

BACKGROUND

Prospective registration of clinical trials is mandated by various regulations. However, clinical trial registries like ClinicalTrials.gov allow registry entries to be updated at any time, and key study elements, including the start date, may change before the first patient is enrolled. If a trial changes its start date after recruiting began, however, it may indicate a reason for concern. This study aimed to measure the rate of "retroactively prospective" trials. This refers to trials that are originally registered retrospectively, with the start date before the registration date, but that retroactively change their start date to be after the registration date, making them appear as if they were prospectively registered.

METHODS

We retrieved clinical trial history data for all clinical trials registered on ClinicalTrials.gov with a first registration date in the year 2015 (N = 11,908). Using automated analyses, we determined the timepoints of registration in relation to the start date of the trial over time. For retroactively prospective trials and a set of control trials, we manually checked the accompanying publications to determine which start date they report and whether they report changes to the start date.

RESULTS

We found 235 clinical trials to be retroactively prospective, comprising 2.0% of all clinical trials in our sample of 11,908 trials. Among the 113 retroactively prospective clinical trials with an accompanying publication, 12 (10.6%) explicitly stated in the publication that they had been prospectively registered.

CONCLUSIONS

Retroactively prospective trial registration happens in one in 50 trials. While these changes to the start date could be mistakes or legitimate edits based on the most up-to-date information, they could also indicate a retrospectively registered trial that has been made to appear as a prospectively registered trial, which would lead to biases unapparent to reviewers. Our results point to the need for more transparent reporting of changes to a trial's details and have implications for the review and conduct of clinical trials, with our fully automated and freely available tools allowing reviewers or editors to detect these changes.

TRIAL REGISTRATION

The preregistered protocol of our study is available via https://osf.io/rvq53 . The most recent version of the protocol lists all deviations from the original study plan, including the rationale behind the changes, and additional analyses that were conducted.

Is our public research money well spent? Publication of research outputs from Health Research Council of New Zealand-funded studies: a cross-sectional study

OBJECTIVE

To evaluate the reporting of results from the projects and programmes funded by the Health Research Council (HRC) New Zealand.

DESIGN

A cross-sectional analysis.

SETTING

Research projects and programmes funded by the HRC New Zealand from 2006 to 2014.

PARTICIPANTS

Publicly available data provided by the HRC.

MAIN OUTCOME MEASURES

The number and proportion with evidence of publication and dissemination of a research output from HRC grants and the time taken to disseminate the results.

RESULTS

Of the 374 HRC grants from 2006 to 2014, there was no evidence of publication or reporting of any research output for 48 studies (13%). Of the 326 (87%) grants with research outputs, there was a mean dissemination time of 4.73 years (SD 2.37). The total funding provided by the HRC was NZ$471 663 336, while the 48 grants with no evidence of dissemination represented NZ$47 095 727 (10%).

CONCLUSIONS

Thirteen per cent of the HRC projects and programmes from 2006 to 2014 have not contributed to the healthcare evidence as their results remain unknown.

Predicting Publication of Clinical Trials Using Structured and Unstructured Data: Model Development and Validation Study

BACKGROUND

Publication of registered clinical trials is a critical step in the timely dissemination of trial findings. However, a significant proportion of completed clinical trials are never published, motivating the need to analyze the factors behind success or failure to publish. This could inform study design, help regulatory decision-making, and improve resource allocation. It could also enhance our understanding of bias in the publication of trials and publication trends based on the research direction or strength of the findings. Although the publication of clinical trials has been addressed in several descriptive studies at an aggregate level, there is a lack of research on the predictive analysis of a trial's publishability given an individual (planned) clinical trial description.

OBJECTIVE

We aimed to conduct a study that combined structured and unstructured features relevant to publication status in a single predictive approach. Established natural language processing techniques as well as recent pretrained language models enabled us to incorporate information from the textual descriptions of clinical trials into a machine learning approach. We were particularly interested in whether and which textual features could improve the classification accuracy for publication outcomes.

METHODS

In this study, we used metadata from ClinicalTrials.gov (a registry of clinical trials) and MEDLINE (a database of academic journal articles) to build a data set of clinical trials (N=76,950) that contained the description of a registered trial and its publication outcome (27,702/76,950, 36% published and 49,248/76,950, 64% unpublished). This is the largest data set of its kind, which we released as part of this work. The publication outcome in the data set was identified from MEDLINE based on clinical trial identifiers. We carried out a descriptive analysis and predicted the publication outcome using 2 approaches: a neural network with a large domain-specific language model and a random forest classifier using a weighted bag-of-words representation of text.

RESULTS

First, our analysis of the newly created data set corroborates several findings from the existing literature regarding attributes associated with a higher publication rate. Second, a crucial observation from our predictive modeling was that the addition of textual features (eg, eligibility criteria) offers consistent improvements over using only structured data (F₁-score=0.62-0.64 vs F₁-score=0.61 without textual features). Both pretrained language models and more basic word-based representations provide high-utility text representations, with no significant empirical difference between the two.

CONCLUSIONS

Different factors affect the publication of a registered clinical trial. Our approach to predictive modeling combines heterogeneous features, both structured and unstructured. We show that methods from natural language processing can provide effective textual features to enable more accurate prediction of publication success, which has not been explored for this task previously.

Investigator initiated trials versus industry sponsored trials - translation of randomized controlled trials into clinical practice (IMPACT)

BACKGROUND

Healthcare decisions are ideally based on clinical trial results, published in study registries, as journal articles or summarized in secondary research articles. In this research project, we investigated the impact of academically and commercially sponsored clinical trials on medical practice by measuring the proportion of trials published and cited by systematic reviews and clinical guidelines.

METHODS

We examined 691 multicenter, randomized controlled trials that started in 2005 or later and were completed by the end of 2016. To determine whether sponsorship/funding and place of conduct influence a trial's impact, we created four sub-cohorts of investigator initiated trials (IITs) and industry sponsored trials (ISTs): 120 IITs and 171 ISTs with German contribution compared to 200 IITs and 200 ISTs without German contribution. We balanced the groups for study phase and place of conduct. German IITs were funded by the German Research Foundation (DFG), the Federal Ministry of Education and Research (BMBF), or by another non-commercial research organization. All other trials were drawn from the German Clinical Trials Register or ClinicalTrials.gov. We investigated, to what extent study characteristics were associated with publication and impact using multivariable logistic regressions.

RESULTS

For 80% of the 691 trials, results were published as result articles in a medical journal and/or study registry, 52% were cited by a systematic review, and 26% reached impact in a clinical guideline. Drug trials and larger trials were associated with a higher probability to be published and to have an impact than non-drug trials and smaller trials. Results of IITs were more often published as a journal article while results of ISTs were more often published in study registries. International ISTs less often gained impact by inclusion in systematic reviews or guidelines than IITs.

CONCLUSION

An encouraging high proportion of the clinical trials were published, and a considerable proportion gained impact on clinical practice. However, there is still room for improvement. For publishing study results, study registries have become an alternative or complement to journal articles, especially for ISTs. IITs funded by governmental bodies in Germany reached an impact that is comparable to international IITs and ISTs.

From registration to publication: A study on Dutch academic randomized controlled trials

INTRODUCTION

Registration of clinical trials has been initiated in order to assess adherence of the reported results to the original trial protocol. This study aimed to investigate the publication rates, timely dissemination of results, and the prevalence of consistency in hypothesis, sample size, and primary endpoint of Dutch investigator-initiated randomized controlled clinical trials (RCTs).

METHODS

All Dutch investigator-initiated RCTs with a completion date between December 31, 2010, and January 1, 2012, and registered in the Trial Register of The Netherlands database were included. PubMed was searched for the publication of these RCT results until September 2016, and the time to the publication date was calculated. Consistency in hypothesis, sample size, and primary endpoint compared with the registry data were assessed.

RESULTS

The search resulted in a total of 168 Dutch investigator-initiated RCTs. In September 2016, the results of 129 (77%) trials had been published, of which 50 (39%) within 2 years after completion of accrual. Consistency in hypothesis with the original protocol was observed in 108 (84%) RCTs; in 71 trials (55%), the planned sample size was reached; and 103 trials (80%) presented the original primary endpoint. Consistency in all three parameters was observed in 50 studies (39%).

CONCLUSION

This study shows that approximately one out of four Dutch investigator-initiated RCTs remains unpublished 5 years after initiation. The observed low overall consistency with the initial study outline is a matter of concern and warrants improvements in trial design and assessment of trial feasibility.