A web-based tool for automatically linking clinical trials to their publications

Planned but ever published? A retrospective analysis of clinical prediction model studies registered on clinicaltrials.gov since 2000

OBJECTIVES

To describe the characteristics and publication outcomes of clinical prediction model studies registered on clinicaltrials.gov since 2000.

STUDY DESIGN AND SETTING

Observational studies registered on clinicaltrials.gov between January 1, 2000, and March 2, 2022, describing the development of a new clinical prediction model or the validation of an existing model for predicting individual-level prognostic or diagnostic risk were analyzed. Eligible clinicaltrials.gov records were classified by modeling study type (development, validation) and the model outcome being predicted (prognostic, diagnostic). Recorded characteristics included study status, sample size information, Medical Subject Headings, and plans to share individual participant data. Publication outcomes were analyzed by linking National Clinical Trial numbers for eligible records with PubMed abstracts.

RESULTS

Nine hundred twenty-eight records were analyzed from a possible 89,896 observational study records. Publications searches found 170 matching peer-reviewed publications for 137 clinicaltrials.gov records. The estimated proportion of records with 1 or more matching publications after accounting for time since study start was 2.8% at 2 years (95% CI: 1.7%, 3.9%), 12.3% at 5 years (9.8% to 14.9%) and 27% at 10 years (23% to 33%). Stratifying records by study start year indicated that publication proportions improved over time. Records tended to prioritize the development of new prediction models over the validation of existing models (76%; 704/928 vs. 24%; 182/928). At the time of download, 27% of records were marked as complete, 35% were still recruiting, and 14.7% had unknown status. Only 7.4% of records stated plans to share individual participant data.

CONCLUSION

Published clinical prediction model studies are only a fraction of overall research efforts, with many studies planned but not completed or published. Improving the uptake of study preregistration and follow-up will increase the visibility of planned research. Introducing additional registry features and guidance may improve the identification of clinical prediction model studies posted to clinical registries.

Race and ethnicity in multiple sclerosis phase 3 clinical trials: A systematic review

BACKGROUND

Distinctive differences in multiple sclerosis (MS) have been observed by race and ethnicity. We aim to (1) assess how often race and ethnicity were reported in clinical trials registered on ClinicalTrials.gov, (2) evaluate whether the population was diverse enough, and (3) compare with publications.

METHODS

We included phase 3 clinical trials registered with results on ClinicalTrials.gov between 2007 and 2023. When race and/or ethnicity were reported, we searched for the corresponding publications.

RESULTS

Out of the 99 included studies, 56% reported race and/or ethnicity, of which only 26% of those primarily completed before 2017. Studies reporting race or ethnicity contributed to a total of 33,891 participants, mainly enrolled in Eastern Europe. Most were White (93%), and the median percentage of White participants was 93% (interquartile range (IQR) = 86%-98%), compared to 3% for Black (IQR = 1%-12%) and 0.2% for Asian (IQR = 0%-1%). Four trials omitted race and ethnicity in publications and even when information was reported, some discrepancies in terminology were identified and categories with fewer participants were often collapsed.

CONCLUSION

More efforts should be done to improve transparency, accuracy, and representativeness, in publications and at a design phase, by addressing social determinants of health that historically limit the enrollment of underrepresented population.

Linking Cancer Clinical Trials to their Result Publications

The results of clinical trials are a valuable source of evidence for researchers, policy makers, and healthcare professionals. However, online trial registries do not always contain links to the publications that report on their results, instead requiring a time-consuming manual search. Here, we explored the application of pre-trained transformer-based language models to automatically identify result-reporting publications of cancer clinical trials by computing dense vectors and performing semantic search. Models were fine-tuned on text data from trial registry fields and article metadata using a contrastive learning approach. The best performing model was PubMedBERT, which achieved a mean average precision of 0.592 and ranked 70.3% of a trial's publications in the top 5 results when tested on the holdout test trials. Our results suggest that semantic search using embeddings from transformer models may be an effective approach to the task of linking trials to their publications.

Automated data analysis of unstructured grey literature in health research: A mapping review

The amount of grey literature and 'softer' intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews. We searched six databases to cover both health- and computer-science literature. After deduplication, 10% of the search results were screened by two reviewers, the remainder was single-screened up to an estimated 95% sensitivity; screening was stopped early after screening an additional 1000 results with no new includes. All full texts were retrieved, screened, and extracted by a single reviewer and 10% were checked in duplicate. We included 84 papers covering automation for health-related social media, internet fora, news, patents, government agencies and charities, or trial registers. From each paper, we extracted data about important functionalities for users of the tool or method; information about the level of support and reliability; and about practical challenges and research gaps. Poor availability of code, data, and usable tools leads to low transparency regarding performance and duplication of work. Financial implications, scalability, integration into downstream workflows, and meaningful evaluations should be carefully planned before starting to develop a tool, given the vast amounts of data and opportunities those tools offer to expedite research.

Publication of Results of Registered Trials With Published Study Protocols, 2011-2022

Importance

Publishing study protocols might reduce research waste because of unclear methods or incomplete reporting; on the other hand, there might be few additional benefits of publishing protocols for registered trials that are never completed or published. No study has investigated the proportion of published protocols associated with published results.

Objective

To estimate the proportion of published trial protocols for which there are not associated published results.

Design, Setting, and Participants

This cross-sectional study used stratified random sampling to identify registered clinical trials with protocols published between January 2011 and August 2022 and indexed in PubMed Central. Ongoing studies and those within 1 year of the primary completion date on ClinicalTrials.gov were excluded. Published results were sought from August 2022 to March 2023 by searching ClinicalTrials.gov, emailing authors, and using an automated tool, as well as through incidental discovery.

Main Outcomes and Measures

The primary outcome was a weighted estimate of the proportion of registered trials with published protocols that also had published main results. The proportion of trials with unpublished results was estimated using a weighted mean.

Results

From 1500 citations that were screened, 308 clinical trial protocols were included, and it was found that 87 trials had not published their main results. Most included trials were investigator-initiated evaluations of nonregulated products. When published, results appeared a mean (SD) of 3.4 (2.0) years after protocol publications. With the use of a weighted mean, an estimated 4754 (95% CI, 4296-5226) eligible clinical trial protocols were published and indexed in PubMed Central between 2011 and 2022. In the weighted analysis, 1708 of those protocols (36%; 95% CI, 31%-41%) were not associated with publication of main results. In a sensitivity analysis excluding protocols published after 2019, an estimated 25% (95% CI, 20%-30%) of 3670 (95% CI, 3310-4032) protocol publications were not associated with publication of main results.

Conclusions and Relevance

This cross-sectional study of clinical trial protocols published on PubMed Central between 2011 and 2022 suggests that many protocols were not associated with subsequent publication of results. The overall benefits of publishing study protocols might outweigh the research waste caused by unnecessary protocol publications.

A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses

Living reviews are an increasingly popular research paradigm. The purpose of a 'living' approach is to allow rapid collation, appraisal and synthesis of evolving evidence on an important research topic, enabling timely influence on patient care and public health policy. However, living reviews are time- and resource-intensive. The accumulation of new evidence and the possibility of developments within the review's research topic can introduce unique challenges into the living review workflow. To investigate the potential of software tools to support living systematic or rapid reviews, we present a narrative review informed by an examination of tools contained on the Systematic Review Toolbox website. We identified 11 tools with relevant functionalities and discuss the important features of these tools with respect to different steps of the living review workflow. Four tools (NestedKnowledge, SWIFT-ActiveScreener, DistillerSR, EPPI-Reviewer) covered multiple, successive steps of the review process, and the remaining tools addressed specific components of the workflow, including scoping and protocol formulation, reference retrieval, automated data extraction, write-up and dissemination of data. We identify several ways in which living reviews can be made more efficient and practical. Most of these focus on general workflow management, or automation through artificial intelligence and machine-learning, in the screening process. More sophisticated uses of automation mostly target living rapid reviews to increase the speed of production or evidence maps to broaden the scope of the map. We use a case study to highlight some of the barriers and challenges to incorporating tools into the living review workflow and processes. These include increased workload, the need for organisation, ensuring timely dissemination and challenges related to the development of bespoke automation tools to facilitate the review process. We describe how current end-user tools address these challenges, and which knowledge gaps remain that could be addressed by future tool development. Dedicated web presences for automatic dissemination of in-progress evidence updates, rather than solely relying on peer-reviewed journal publications, help to make the effort of a living evidence synthesis worthwhile. Despite offering basic living review functionalities, existing end-user tools could be further developed to be interoperable with other tools to support multiple workflow steps seamlessly, to address broader automatic evidence retrieval from a larger variety of sources, and to improve dissemination of evidence between review updates.

Time to Publication for Randomized Clinical Trials Presented as Abstracts at Three Gastroenterology and Hepatology Conferences in 2017

Background & Aims

Results of randomized clinical trials are often first presented as conference abstracts, but these abstracts may be difficult to find, and trial results included in the abstract may not be followed by subsequent journal publications. In a review of abstracts submitted to eight major medical and surgical conferences in 2017, we identified 237 abstracts reporting primary results of randomized clinical trials accepted for presentation at three major gastroenterology and hepatology conferences. The aims of this new analysis were to determine the publication rate for these abstracts and the proportion of publications that included trial registration numbers in the publication abstract.

Methods

Clinical trial registries, PubMed, Europe PMC, and Google Scholar were searched through November 1, 2021, for publications reporting trial results for the selected abstracts. Publications were reviewed to determine if they included a trial registration number and if the registration number was in the abstract.

Results

Publications were found for 157 abstracts (66%) within four years of the conference. Publications were found more frequently for the 194 abstracts reporting results of registered trials (144, 74%) than for the 43 abstracts reporting unregistered trials (13, 30%), but only 67% of these 144 publications included the registration number in the publication abstract. Ten unpublished trials had summary results posted on ClinicalTrials.gov.

Conclusions

Clinical trial results could be more accessible if all trials were registered, authors included registration numbers in both conference and journal abstracts, and journal editors required the inclusion of registration numbers in publication abstracts for registered clinical trials.