Sharing Individual Participant Data (IPD) within the Context of the Trial Reporting System (TRS)

A comparison of machine learning methods to find clinical trials for inclusion in new systematic reviews from their PROSPERO registrations prior to searching and screening

Searching for trials is a key task in systematic reviews and a focus of automation. Previous approaches required knowing examples of relevant trials in advance, and most methods are focused on published trial articles. To complement existing tools, we compared methods for finding relevant trial registrations given a International Prospective Register of Systematic Reviews (PROSPERO) entry and where no relevant trials have been screened for inclusion in advance. We compared SciBERT-based (extension of Bidirectional Encoder Representations from Transformers) PICO extraction, MetaMap, and term-based representations using an imperfect dataset mined from 3632 PROSPERO entries connected to a subset of 65,662 trial registrations and 65,834 trial articles known to be included in systematic reviews. Performance was measured by the median rank and recall by rank of trials that were eventually included in the published systematic reviews. When ranking trial registrations relative to PROSPERO entries, 296 trial registrations needed to be screened to identify half of the relevant trials, and the best performing approach used a basic term-based representation. When ranking trial articles relative to PROSPERO entries, 162 trial articles needed to be screened to identify half of the relevant trials, and the best-performing approach used a term-based representation. The results show that MetaMap and term-based representations outperformed approaches that included PICO extraction for this use case. The results suggest that when starting with a PROSPERO entry and where no trials have been screened for inclusion, automated methods can reduce workload, but additional processes are still needed to efficiently identify trial registrations or trial articles that meet the inclusion criteria of a systematic review.

Addressing Systematic Missing Data in the Context of Causally Interpretable Meta-analysis

Evidence synthesis involves drawing conclusions from trial samples that may differ from the target population of interest, and there is often heterogeneity among trials in sample characteristics, treatment implementation, study design, and assessment of covariates. Stitching together this patchwork of evidence requires subject-matter knowledge, a clearly defined target population, and guidance on how to weigh evidence from different trials. Transportability analysis has provided formal identifiability conditions required to make unbiased causal inference in the target population. In this manuscript, we review these conditions along with an additional assumption required to address systematic missing data. The identifiability conditions highlight the importance of accounting for differences in treatment effect modifiers between the populations underlying the trials and the target population. We perform simulations to evaluate the bias of conventional random effect models and multiply imputed estimates using the pooled trials sample and describe causal estimators that explicitly address trial-to-target differences in key covariates in the context of systematic missing data. Results indicate that the causal transportability estimators are unbiased when treatment effect modifiers are accounted for in the analyses. Results also highlight the importance of carefully evaluating identifiability conditions for each trial to reduce bias due to differences in participant characteristics between trials and the target population. Bias can be limited by adjusting for covariates that are strongly correlated with missing treatment effect modifiers, including data from trials that do not differ from the target on treatment modifiers, and removing trials that do differ from the target and did not assess a modifier.

Depression screening tool accuracy individual participant data meta-analyses: data contribution was associated with multiple factors

OBJECTIVES

To examine the proportion of eligible primary studies that contributed data, study characteristics associated with data contribution, and reasons for noncontribution using diagnostic test accuracy Individual Participant Data Meta-Analysis (IPDMA) data sets from the DEPRESsion Screening Data project.

STUDY DESIGN AND SETTING

We reviewed data set contributions from four IPDMAs. A multivariable logistic regression model was fitted to evaluate study factors associated with data contribution.

RESULTS

Of 456 eligible studies from four included IPDMAs, 295 (65%) contributed data. More recent year of publication and higher journal impact factor were associated with greater odds of data contribution. Studies conducted in Europe (excluding the United Kingdom), Oceania, Canada, the Middle East, Africa, and Central or South America (reference = the United States), that have recruitment from inpatient care or nonmedical settings (reference = outpatient), that reported screening accuracy results, or that drew negative conclusions (reference = positive conclusions) were more likely to contribute data. Studies of the Geriatric Depression Scale (reference = the Patient Health Questionnaire) or lacking funding information were negatively associated with data contribution. Over 80% of noncontributions were due to authors being unreachable or data being unavailable.

CONCLUSION

The study identified factors associated with data contribution that may support future research to promote data contribution to IPDMAs.

Ten (not so) simple rules for clinical trial data-sharing

Clinical trial data-sharing is seen as an imperative for research integrity and is becoming increasingly encouraged or even required by funders, journals, and other stakeholders. However, early experiences with data-sharing have been disappointing because they are not always conducted properly. Health data is indeed sensitive and not always easy to share in a responsible way. We propose 10 rules for researchers wishing to share their data. These rules cover the majority of elements to be considered in order to start the commendable process of clinical trial data-sharing: Rule 1: Abide by local legal and regulatory data protection requirementsRule 2: Anticipate the possibility of clinical trial data-sharing before obtaining fundingRule 3: Declare your intent to share data in the registration stepRule 4: Involve research participantsRule 5: Determine the method of data accessRule 6: Remember there are several other elements to shareRule 7: Do not proceed aloneRule 8: Deploy optimal data management to ensure that the data shared is usefulRule 9: Minimize risksRule 10: Strive for excellence.

Data-sharing and re-analysis for main studies assessed by the European Medicines Agency-a cross-sectional study on European Public Assessment Reports

BACKGROUND

Transparency and reproducibility are expected to be normative practices in clinical trials used for decision-making on marketing authorisations for new medicines. This registered report introduces a cross-sectional study aiming to assess inferential reproducibility for main trials assessed by the European Medicines Agency.

METHODS

Two researchers independently identified all studies on new medicines, biosimilars and orphan medicines given approval by the European Commission between January 2017 and December 2019, categorised as 'main studies' in the European Public Assessment Reports (EPARs). Sixty-two of these studies were randomly sampled. One researcher retrieved the individual patient data (IPD) for these studies and prepared a dossier for each study, containing the IPD, the protocol and information on the conduct of the study. A second researcher who had no access to study reports used the dossier to run an independent re-analysis of each trial. All results of these re-analyses were reported in terms of each study's conclusions, p-values, effect sizes and changes from the initial protocol. A team of two researchers not involved in the re-analysis compared results of the re-analyses with published results of the trial.

RESULTS

Two hundred ninety-two main studies in 173 EPARs were identified. Among the 62 studies randomly sampled, we received IPD for 10 trials. The median number of days between data request and data receipt was 253 [interquartile range 182-469]. For these ten trials, we identified 23 distinct primary outcomes for which the conclusions were reproduced in all re-analyses. Therefore, 10/62 trials (16% [95% confidence interval 8% to 28%]) were reproduced, as the 52 studies without available data were considered non-reproducible. There was no change from the original study protocol regarding the primary outcome in any of these ten studies. Spin was observed in the report of one study.

CONCLUSIONS

Despite their results supporting decisions that affect millions of people's health across the European Union, most main studies used in EPARs lack transparency and their results are not reproducible for external researchers. Re-analyses of the few trials with available data showed very good inferential reproducibility.

TRIAL REGISTRATION

https://osf.io/mcw3t/.

Identifying unreported links between ClinicalTrials.gov trial registrations and their published results

A substantial proportion of trial registrations are not linked to corresponding published articles, limiting analyses and new tools. Our aim was to develop a method for finding articles reporting the results of trials that are registered on ClinicalTrials.gov when they do not include metadata links. We used a set of 27,280 trial registration and article pairs to train and evaluate methods for identifying missing links in both directions-from articles to registrations and from registrations to articles. We trained a classifier with six distance metrics as feature representations to rank the correct article or registration, using recall@K to evaluate performance and compare to baseline methods. When identifying links from registrations to published articles, the classifier ranked the correct article first (recall@1) among 378,048 articles in 80.8% of evaluation cases and 34.9% in the baseline method. Recall@10 was 85.1% compared to 60.7% in the baseline. When predicting links from articles to registrations, recall@1 was 83.4% for the classifier and 39.8% in the baseline. Recall@10 was 89.5% compared to 65.8% in the baseline. The proposed method improves on our baseline document similarity method to be feasible for identifying missing links in practice. Given a ClinicalTrials.gov registration, a user checking 10 ranked articles can expect to identify the matching article in at least 85% of cases, if the trial has been published. The proposed method can be used to improve the coupling of ClinicalTrials.gov and PubMed, with applications related to automating systematic review and evidence synthesis processes.

Data sharing-trialists' plans at registration, attitudes, barriers and facilitators: A cohort study and cross-sectional survey

Data unavailability impedes research transparency and is a major problem for individual participant data (IPD) meta-analyses as it reduces statistical power, increases risk of bias, and may even preclude completion. The primary objectives of this study were to determine IPD sharing plans reported in recently registered clinical trial registration records, how data sharing commitment relates to clinical trial characteristics, and principal investigators' attitudes, motivations and barriers to data sharing. The secondary objective was to derive recommendations to overcome identified barriers to data sharing. This was a retrospective cohort study of all interventional trials registered on the Australian New Zealand Clinical Trials Registry (ANZCTR) from 1 December 2018 to 30 November 2019, and an online cross-sectional survey of their principal investigators. In the cohort study of all clinical trials registered on the ANZCTR in the study period (n = 1517), commitment to share data was low (22%, 329/1517). In the cross-sectional survey (n = 281, 23% response rate), principal investigators showed strong support for the concept of data sharing (77%, 216/281) but a substantially lower intention to actually share data from their clinical trials (40%, 111/281). Major barriers to data sharing included lacking informed consent to share data, protecting participant confidentiality and preventing misinterpretation of data or misleading secondary analyses. There is a gap between high in-principle support for data sharing, and low in-practice intention from investigators to share data from their own clinical trials. Multiple pathways exist to bridge this gap by addressing the identified barriers to data sharing.

When Evidence Goes "Missing in Action": Implications for Patient Management in Cardiac Surgery

Best-practice clinical decision-making for patient blood management (PBM) and transfusion in cardiac surgery requires high-quality, timely information. However, evidence will be misleading if published information lags too far behind evolving practice, or if trial results are biased, incomplete, or unreported. The result is that providers are deprived of accurate data, and patients will not receive best possible care. Publicly accessible trial registries provide information for structured audits of reporting compliance, and appraisal of evidence attrition and distortion. Trials related to blood management and transfusion in cardiac surgery and those registered in ClinicalTrials.gov were evaluated for relevance, reliability, transparency, timeliness, and prevalence of unreported trial results. Evidence was considered to have "disappeared" if no results were posted to the registry and no related PUBMED publications were available by July 2019. Data were summarized by descriptive statistics. A total of 181 registered trials were surveyed; 52% were prospectively registered. Most commonly reported primary outcomes were laboratory surrogate measures (34%). Patient- and practice-relevant outcomes-mortality/major morbidity (7%), transfusion (27%), and major bleeding (28%)-were less common. Only seven studies posted results to the registry within the mandated 12 months from study completion; median time to posting was 17 (interquartile range [IQR] 13, 37) months. Trial results for 58% were unreported 3-9 years after trial completion. A staggering amount of clinical trial evidence for PBM in cardiac surgery is missing from publicly accessible records and the literature. Investigators must be incentivized to promptly and completely report all results. Penalties for noncompliance are already in place and should be enforced. Simplified information linkage, centralized and routine audit cycles, and prioritization of robust "living" reviews may be more positive motivators. Implementation will require a sea change in the prevailing culture of research reporting, plus coordinated efforts of clinicians, applied statisticians, information technology specialists, and research librarians.

Is it time for computable evidence synthesis?

Efforts aimed at increasing the pace of evidence synthesis have been primarily focused on the use of published articles, but these are a relatively delayed, incomplete, and at times biased source of study results data. Compared to those in bibliographic databases, structured results data available in trial registries may be more timely, complete, and accessible, but these data remain underutilized. Key advantages of using structured results data include the potential to automatically monitor the accumulation of relevant evidence and use it to signal when a systematic review requires updating, as well as to prospectively assign trials to already published reviews. Shifting focus to emerging sources of structured trial data may provide the impetus to build a more proactive and efficient system of continuous evidence surveillance.

Individual participant data meta-analyses (IPDMA): data contribution was associated with trial corresponding author country, publication year, and journal impact factor

OBJECTIVES

The objectives were to determine the proportion of eligible randomized controlled trials (RCTs) that contributed data to individual participant data meta-analyses (IPDMAs) and explore associated factors.

STUDY DESIGN AND SETTING

IPDMAs with ≥10 eligible RCTs were identified by searching MEDLINE, EMBASE, CINAHL, and Cochrane May 1, 2015 to February 13, 2017. Mixed-effect logistic regression was used to identify factors associated with data contribution.

RESULTS

Of 774 eligible RCTs from 35 included IPDMAs, 517 (67%, 95% confidence interval [CI]: 63%-70%) contributed data. Compared to RCTs from journals with low-impact factors (0-2.4), RCTs from journals with higher impact factors were more likely to contribute data: impact factor 5.0-9.9, odds ratio [OR] 2.6, 95% CI: 1.37-4.86; impact factor: 10.0-19.9, OR: 5.7, 95% CI: 3.0-10.8; impact factor >20.0, OR: 4.6, 95% CI: 1.9-11.4. RCTs from the United Kingdom were more likely to contribute data than those from the United States (reference; OR: 2.4, 95% CI, 1.3-4.6). There was an increase in OR per publication year (OR: 1.05, 95% CI: 1.02-1.09).

CONCLUSION

The country where RCTs are conducted, impact factor of the journal where RCTs are published, and RCT publication year were associated with data contribution in IPDMAs with ≥10 eligible RCTs.

Mapping the preclinical to clinical evidence and development trajectory of the oncolytic virus talimogene laherparepvec (T-VEC): a systematic review

OBJECTIVE

This study aimed to conduct a systematic review of preclinical and clinical evidence to chart the successful trajectory of talimogene laherparepvec (T-VEC) from the bench to the clinic.

DESIGN

This study was a systematic review. The primary outcome of interest was the efficacy of treatment, determined by complete response. Abstract and full-text selection as well as data extraction were done by two independent reviewers. The Cochrane risk of bias tool was used to assess the risk of bias in studies.

SETTING

Embase, Embase Classic and OvidMedline were searched from inception until May 2016 to assess its development trajectory to approval in 2015.

PARTICIPANTS

Preclinical and clinical controlled comparison studies, as well as observational studies.

INTERVENTIONS

T-VEC for the treatment of any malignancy.

RESULTS

8852 records were screened and five preclinical (n=150 animals) and seven clinical studies (n=589 patients) were included. We saw large decreases in T-VEC's efficacy as studies moved from the laboratory to patients, and as studies became more methodologically rigorous. Preclinical studies reported complete regression rates up to 100% for injected tumours and 80% for contralateral tumours, while the highest degree of efficacy seen in the clinical setting was a 24% complete response rate, with one study experiencing a complete response rate of 0%. We were unable to reliably assess safety due to the lack of reporting, as well as the heterogeneity seen in adverse event definitions. All preclinical studies had high or unclear risk of bias, and all clinical studies were at a high risk of bias in at least one domain.

CONCLUSIONS

Our findings illustrate that even successful biotherapeutics may not demonstrate a clear translational road map. This emphasises the need to consider increasing rigour and transparency along the translational pathway.

PROSPERO REGISTRATION NUMBER

CRD42016043541.

How Clinical Trial Data Sharing Platforms Can Advance the Study of Biomarkers

Although data sharing platforms host diverse data types the features of these platforms are well-suited to facilitating biomarker research. Given the current state of biomarker discovery, an innovative paradigm to accelerate biomarker discovery is to utilize platforms such as Vivli to leverage researchers' abilities to integrate certain classes of biomarkers.

The risk of conclusion change in systematic review updates can be estimated by learning from a database of published examples

OBJECTIVES

To determine which systematic review characteristics are needed to estimate the risk of conclusion change in systematic review updates.

STUDY DESIGN AND SETTING

We applied classification trees (a machine learning method) to model the risk of conclusion change in systematic review updates, using pairs of systematic reviews and their updates as samples. The classifiers were constructed using a set of features extracted from systematic reviews and the relevant trials added in published updates. Model performance was measured by recall, precision, and area under the receiver operating characteristic curve (AUC).

RESULTS

We identified 63 pairs of systematic reviews and updates, of which 20 (32%) exhibited a change in conclusion in their updates. A classifier using information about new trials exhibited the highest performance (AUC: 0.71; recall: 0.75; precision: 0.43) compared to a classifier that used fewer features (AUC: 0.65; recall: 0.75; precision: 0.39).

CONCLUSION

When estimating the risk of conclusion change in systematic review updates, information about the sizes of trials that will be added in an update are most useful. Future tools aimed at signaling conclusion change risks would benefit from complementary tools that automate screening of relevant trials.

Software engineering principles address current problems in the systematic review ecosystem

Systematic reviewers are simultaneously unable to produce systematic reviews fast enough to keep up with the availability of new trial evidence while overproducing systematic reviews that are unlikely to change practice because they are redundant or biased. Although the transparency and completeness of trial reporting has improved with changes in policy and new technologies, systematic reviews have not yet benefited from the same level of effort. We found that new methods and tools used to automate aspects of systematic review processes have focused on improving the efficiency of individual systematic reviews rather than the efficiency of the entire ecosystem of systematic review production. We use software engineering principles to review challenges and opportunities for improving the interoperability, integrity, efficiency, and maintainability. We conclude by recommending ways to improve access to structured systematic review results. Major opportunities for improving systematic reviews will come from new tools and changes in policy focused on doing the right systematic reviews rather than just doing more of them faster.

How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider

ClinicalTrials.gov, a repository of information about clinical studies and their results, together with specialised search tools, provides a unique window into the clinical research enterprise, which includes all initiated, ongoing, and completed or terminated clinical studies. Researchers are increasingly using information from the database to assess research reporting practices, or to characterise the clinical research enterprise. Conducting valid analyses requires an understanding of both the capabilities and limitations of the database (that is, intrinsic factors) as well as reporting policies and other factors external to the database that influence the types of studies in ClinicalTrials.gov in a specified time. This article discusses 10 key issues that researchers need to consider when using the database to conduct research.

Evidence appraisal: a scoping review, conceptual framework, and research agenda

Objective

Critical appraisal of clinical evidence promises to help prevent, detect, and address flaws related to study importance, ethics, validity, applicability, and reporting. These research issues are of growing concern. The purpose of this scoping review is to survey the current literature on evidence appraisal to develop a conceptual framework and an informatics research agenda.

Methods

We conducted an iterative literature search of Medline for discussion or research on the critical appraisal of clinical evidence. After title and abstract review, 121 articles were included in the analysis. We performed qualitative thematic analysis to describe the evidence appraisal architecture and its issues and opportunities. From this analysis, we derived a conceptual framework and an informatics research agenda.

Results

We identified 68 themes in 10 categories. This analysis revealed that the practice of evidence appraisal is quite common but is rarely subjected to documentation, organization, validation, integration, or uptake. This is related to underdeveloped tools, scant incentives, and insufficient acquisition of appraisal data and transformation of the data into usable knowledge.

Discussion

The gaps in acquiring appraisal data, transforming the data into actionable information and knowledge, and ensuring its dissemination and adoption can be addressed with proven informatics approaches.

Conclusions

Evidence appraisal faces several challenges, but implementing an informatics research agenda would likely help realize the potential of evidence appraisal for improving the rigor and value of clinical evidence.

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine

OBJECTIVES

To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes.

DESIGN

Survey of published RCTs.

SETTING

PubMed/Medline.

ELIGIBILITY CRITERIA

RCTs that had been submitted and published by The BMJ and PLOS Medicine subsequent to the adoption of data sharing policies by these journals.

MAIN OUTCOME MEASURE

The primary outcome was data availability, defined as the eventual receipt of complete data with clear labelling. Primary outcomes were reanalyzed to assess to what extent studies were reproduced. Difficulties encountered were described.

RESULTS

37 RCTs (21 from The BMJ and 16 from PLOS Medicine) published between 2013 and 2016 met the eligibility criteria. 17/37 (46%, 95% confidence interval 30% to 62%) satisfied the definition of data availability and 14 of the 17 (82%, 59% to 94%) were fully reproduced on all their primary outcomes. Of the remaining RCTs, errors were identified in two but reached similar conclusions and one paper did not provide enough information in the Methods section to reproduce the analyses. Difficulties identified included problems in contacting corresponding authors and lack of resources on their behalf in preparing the datasets. In addition, there was a range of different data sharing practices across study groups.

CONCLUSIONS

Data availability was not optimal in two journals with a strong policy for data sharing. When investigators shared data, most reanalyses largely reproduced the original results. Data sharing practices need to become more widespread and streamlined to allow meaningful reanalyses and reuse of data.

TRIAL REGISTRATION

Open Science Framework osf.io/c4zke.

Trialists' Intent to Share Individual Participant Data as Disclosed at ClinicalTrials.gov

This study summarizes responses to trial registration fields on the ClinicalTrials.gov website asking users if they plan to share individual participant data and what information they would be willing to share.

Unreported links between trial registrations and published articles were identified using document similarity measures in a cross-sectional analysis of ClinicalTrials.gov

OBJECTIVES

Trial registries can be used to measure reporting biases and support systematic reviews, but 45% of registrations do not provide a link to the article reporting on the trial. We evaluated the use of document similarity methods to identify unreported links between ClinicalTrials.gov and PubMed.

STUDY DESIGN AND SETTING

We extracted terms and concepts from a data set of 72,469 ClinicalTrials.gov registrations and 276,307 PubMed articles and tested methods for ranking articles across 16,005 reported links and 90 manually identified unreported links. Performance was measured by the median rank of matching articles and the proportion of unreported links that could be found by screening ranked candidate articles in order.

RESULTS

The best-performing concept-based representation produced a median rank of 3 (interquartile range [IQR] 1-21) for reported links and 3 (IQR 1-19) for the manually identified unreported links, and term-based representations produced a median rank of 2 (1-20) for reported links and 2 (IQR 1-12) in unreported links. The matching article was ranked first for 40% of registrations, and screening 50 candidate articles per registration identified 86% of the unreported links.

CONCLUSION

Leveraging the growth in the corpus of reported links between ClinicalTrials.gov and PubMed, we found that document similarity methods can assist in the identification of unreported links between trial registrations and corresponding articles.

Linking quality indicators to clinical trials: an automated approach

OBJECTIVE

Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations.

DESIGN

A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov.

INTERVENTION

Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity.

MAIN OUTCOME MEASURE

Number of care indicators that mapped to outcome measures in clinical trials.

RESULTS

While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method.

CONCLUSIONS

We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

A systematic review of the processes used to link clinical trial registrations to their published results

BACKGROUND

Studies measuring the completeness and consistency of trial registration and reporting rely on linking registries with bibliographic databases. In this systematic review, we quantified the processes used to identify these links.

METHODS

PubMed and Embase databases were searched from inception to May 2016 for studies linking trial registries with bibliographic databases. The processes used to establish these links were categorised as automatic when the registration identifier was available in the bibliographic database or publication, or manual when linkage required inference or contacting of trial investigators. The number of links identified by each process was extracted where available. Linear regression was used to determine whether the proportions of links available via automatic processes had increased over time.

RESULTS

In 43 studies that examined cohorts of registry entries, 24 used automatic and manual processes to find articles; 3 only automatic; and 11 only manual (5 did not specify). Twelve studies reported results for both manual and automatic processes and showed that a median of 23% (range from 13 to 42%) included automatic links to articles, while 17% (range from 5 to 42%) of registry entries required manual processes to find articles. There was no evidence that the proportion of registry entries with automatic links had increased (R ² = 0.02, p = 0.36). In 39 studies that examined cohorts of articles, 21 used automatic and manual processes; 9 only automatic; and 2 only manual (7 did not specify). Sixteen studies reported numbers for automatic and manual processes and indicated that a median of 49% (range from 8 to 97%) of articles had automatic links to registry entries, and 10% (range from 0 to 28%) required manual processes to find registry entries. There was no evidence that the proportion of articles with automatic links to registry entries had increased (R ² = 0.01, p = 0.73).

CONCLUSIONS

The linkage of trial registries to their corresponding publications continues to require extensive manual processes. We did not find that the use of automatic linkage has increased over time. Further investigation is needed to inform approaches that will ensure publications are properly linked to trial registrations, thus enabling efficient monitoring of trial reporting.

Use of the National Heart, Lung, and Blood Institute Data Repository

BACKGROUND

Research on data sharing from clinical trials has focused on elucidating perceptions, barriers, and attitudes among trialists and study participants with respect to sharing data. However, little information exists regarding utilization or associated publication of articles once clinical trial data have been widely shared.

METHODS

We analyzed administrative records of investigator requests for data access, linked publications, and bibliometrics to describe the use of the National Heart, Lung, and Blood Institute data repository.

RESULTS

From January 2000 through May 2016, a total of 370 investigators requested data from 1 or more clinical trials. Requests for trial data have been increasing, with 195 investigators (53%) initiating requests during the last 4.4 years of the study period. The predominant reason for requesting data was post hoc secondary analysis of new questions (72%), followed by analytic or statistical approaches to clinical trials (9%) and meta-analyses or pooled study research (7%). Of 172 requests with online project descriptions, only 2 requests were initiated for reanalysis of primary-outcome findings. Data from 88 of 100 available clinical trials were requested at least once, and the median time from repository availability to first request was 235 days. A total of 277 articles were published on the basis of data from 47 trials. Citation metrics from 224 articles indicated that half of the publications have cumulative citations that rank in the top 34% normalized for subject category and year of publication.

CONCLUSIONS

Demand for trial data for secondary analysis has been increasing. Requesting data for the a priori purpose of reanalysis or verification of original findings was rare.

Clinical trial registries: more international, converging efforts are needed

Clinical trial registries are being increasingly acknowledged worldwide. We searched for possibly trustworthy online registries that are not already included in the International Clinical Trials Registry Platform to evaluate whether other useful trial data sources exist and whether they could potentially be consulted, since the strategy search within this platform has recently been questioned. Fifty-nine registries were initially identified, and 11 of them fit the criteria applied and were analyzed for quality and usability. Four additional, potentially reliable registries were identified that researchers could exploit in order to obtain a more global view of the issue being investigated.

Update on Trial Registration 11 Years after the ICMJE Policy Was Established

In the decade following the journal editors’ trial registration policy, a global trial reporting system (TRS) has arisen to supplement journal publication by increasing the transparency and accountability of the clinical research enterprise (CRE), which ultimately advances evidence-based medicine. Trial registration a foundation component of the TRS. In this article, we assess impact of the trial registration on the CRE with respect to two key goals: (1) establishing a publicly accessible and structured public record of all trials and (2) ensuring access to date-stamped protocol details that change during a study. After characterizing international trial registry landscape, we summarize the published evidence of the impact of the registration laws and policies on the CRE to date. We present three analyses using ClinicalTrials.gov registration data to illustrate approaches for assessing and monitoring the TRS: (1) timing of registration (i.e., prior to trial initiation [prospective] or after trial initiation [retrospective or “late”]; (2) degree of specificity and consistency of registered primary outcome measures compared to descriptions in study protocols and published articles; and (3) a survey of the published literature to characterize how ClinicalTrials.gov data has been used in research on the CRE. These findings suggest that, while the TRS is largely moving towards goals, key stakeholders need to do more in the next decade.

Sharing all types of clinical data and harmonizing journal standards

Despite recent efforts to enforce policies requiring the sharing of data underlying clinical findings, current policies of biomedical journals remain largely heterogeneous. As this heterogeneity does not optimally serve the cause of data sharing, a first step towards better harmonization would be the requirement of a data sharing statement for all clinical studies and not simply for randomized studies. Although the publication of a data sharing statement does not imply that all data is made readily available, such a policy would swiftly implement a cultural change in the definition of scientific outputs. Currently, a scientific output only corresponds to a study report published in a medical journal, while in the near future it might consist of all materials described in the manuscript, including all relevant raw data. When such a cultural shift has been achieved, the logical conclusion would be for biomedical journals to require authors to make all data fully available without restriction as a condition for publication.

Can Data Sharing Become the Path of Least Resistance?

In this month’s editorial, the PLOS Medicine editors note recent progress towards data sharing as the community norm in medical research and the barriers that remain.