Clinical Trial Registry Use in Orthopaedic Surgery Systematic Reviews

Do prospective randomized controlled trials comply with filed protocols? Spin study of 206 trials from 2010 to 2023

INTRODUCTION

Prospective randomized controlled trials (RCT) have a robust methodology, but some distortions may occur during the course of study. Some protocols may not be available at the time an article is reading. Some authors may change the methodology between the time the protocol was submitted and when the trial results are actually published. Others may distort results to favor more attractive findings and draw conclusions that support prior hypotheses. This has rarely been investigated and none explored the RCTs published in the Journal of Bone and Joint Surgery (JBJS). Therefore, we did a retrospective investigation aiming to determine: (1) the proportion of trials with a protocol deposited and accessible to the reader, (2) whether the trials scrupulously followed the filed protocols.

HYPOTHESIS

Protocols were available in over 80% of cases, and these protocols were followed in over 80% of trials for the primary endpoint.

PATIENTS AND METHODS

This was a retrospective study of articles published in the JBJS between January 2010 and November 2023. Differences in primary and secondary endpoints between protocols and articles were sought.

RESULTS

Of the 206 RCT articles studied, 113 (54.9%) described clear and identifiable endpoints, and 93 (45.1%) were not identifiable and were inferred in the results; 184 (89.3%) articles identified a trial protocol. For the 184 articles (89.3%) declaring a trial protocol in the text, 23 (11.1%) protocols were not accessible. In all, 45 articles (21.8%) thus had no protocol available on the Internet (i.e., not available to the reader either because it was not cited in the text or because it was not accessible) so we analyzed 161 articles. The primary endpoint remained unchanged in 97 articles (60.2%) out of the 161 studied, was changed in 64 articles (39.8%), and was lacking (protocol not accessible) in 45 articles (21.8% of all articles). The secondary endpoints of the articles were unchanged in 61 articles (37.9%) out of the 161 studied.

DISCUSSION

Like other leading journals, JBJS publishes RCT articles containing a significant proportion of inconsistencies between preoperative trial protocols and the methods actually used in the research.

LEVEL OF EVIDENCE

III; retrospective comparative study non randomized.

Availability of results of clinical trials registered on EU Clinical Trials Register: cross sectional audit study

Objective

To identify the availability of results for trials registered on the European Union Clinical Trials Register (EUCTR) compared with other dissemination routes to understand its value as a results repository.

Design

Cross sectional audit study.

Setting

EUCTR protocols and results sections, data extracted 1-3 December 2020.

Population

Random sample of 500 trials registered on EUCTR with a completion date of more than two years from the beginning of searches (ie, 1 December 2018).

Main outcome measures

Proportion of trials with results across the examined dissemination routes (EUCTR, ClinicalTrials.gov, ISRCTN registry, and journal publications), and for each dissemination route individually. Prespecified secondary outcomes were number and proportion of unique results, and the timing of results, for each dissemination route.

Results

In the sample of 500 trials, availability of results on EUCTR (53.2%, 95% confidence interval 48.8% to 57.6%) was similar to the peer reviewed literature (58.6%, 54.3% to 62.9%) and exceeded the proportion of results available on other registries with matched records. Among the 383 trials with any results, 55 (14.4%, 10.9% to 17.9%) were only available on EUCTR. Also, after the launch of the EUCTR results database, median time to results was fastest on EUCTR (1142 days, 95% confidence interval 812 to 1492), comparable with journal publications (1226 days, 1074 to 1551), and exceeding ClinicalTrials.gov (3321 days, 1653 to undefined). For 117 trials (23.4%, 19.7% to 27.1%), however, results were published elsewhere but not submitted to the EUCTR registry, and no results were located in any dissemination route for 117 trials (23.4%, 19.7% to 27.1).

Conclusions

EUCTR should be considered in results searches for systematic reviews and can help researchers and the public to access the results of clinical trials, unavailable elsewhere, in a timely way. Reporting requirements, such as the EU's, can help in avoiding research waste by ensuring results are reported. The registry's true value, however, is unrealised because of inadequate compliance with EU guidelines, and problems with data quality that complicate the routine use of the registry. As the EU transitions to a new registry, continuing to emphasise the importance of EUCTR and the provision of timely and complete data is critical. For the future, EUCTR will still hold important information from the past two decades of clinical research in Europe. With increased efforts from sponsors and regulators, the registry can continue to grow as a source of results of clinical trials, many of which might be unavailable from other dissemination routes.

Trial Registry Searches In Plastic Surgery Systematic Reviews: A Meta-epidemiological Study

INTRODUCTION

Clinical trial registry searches for unpublished clinical trial data are a means of mitigating publication bias within systematic reviews (SRs). The purpose of our study is to look at the rate of clinical trial registry searches conducted by SRs in the top five Plastic and Reconstructive Surgery journals.

METHODS

We identified the top five plastic and reconstructive surgery journals using the Google h-5 index. We then searched Pubmed for SRs published in these journals and compared them to plastic surgery SRs published in the Cochrane Collaboration for SRs over the last 5 y. We included all SRs that were published within these top five journals and Cochrane between December 6, 2016 and December 6, 2021. We then conducted a secondary analysis on clinicaltrials.gov looking for unpublished clinical trials for 100 randomized SRs that did not conduct a clinical trial registry search.

RESULTS

In SRs, 3.3% (17/512) from plastic surgery journals conducted trial registry searches. In comparison, 95.0% (38/40) of Cochrane Collaboration SRs conducted trial registry searches. Our secondary analysis found that 50% (50/100) of SRs could have included at least one unpublished clinical trial data set.

CONCLUSIONS

We found that plastic surgery SRs rarely include searches for unpublished clinical trial data in clinical trial registries. To improve the data completeness of SRs in plastic surgery journals, we recommend journals alter their author guidelines to require a clinical trial registry search for unpublished literature.

Cemented vs Uncemented hemiarthroplasties for femoral neck fractures: An overlapping systematic review and evidence appraisal

BACKGROUND

The purpose of our study is to assess the methodology of overlapping systematic reviews related to cemented vs uncemented hip hemiarthroplasties for the treatment of femoral neck fractures to find the study with the best evidence. Also, we assess the gaps in methodology and information to help with direction of future studies.

METHODS

A systematic search was conducted in September 2022 using Pubmed, Embase, and Cochrane Library. Clinical outcome data and characteristics of each study were extracted to see which treatment had better favorability. The outcomes and characteristics extracted from each study includes, first author, search date, publication journal and date, number of studies included, databases, level of evidence, software used, subgroup analyses that were conducted, and heterogeneity with the use of I2 statistics Methodological quality information was extracted from each study using four different methodologic scores (Oxford Levels of Evidence; Assessment of Multiple Systematic Reviews (AMSTAR); Quality of reporting of meta-analyses (QUROM); Oxman and Guyatt. After that, the Jadad decision algorithm was used to identify which studies in our sample contained the best available evidence. Finally, overlap of each systematic review was assessed using Corrected Covered Area (CCA) to look at redundancy and research waste among the systematic reviews published on the topic.

RESULTS

After screening, 12 studies were included in our sample. For the Oxford Levels of Evidence, we found that all the studies were Level I evidence. For the QUORUM assessment, we had 1 study with the highest score of 18. Additionally, we did the Oxman and Guyatt assessment, where we found 4 studies with a maximum score of 6. Finally, we did an AMSTAR assessment and found 2 studies with a score of 9. After conducting the methodological scores; the authors determined that Li. L et al 2021 had the highest quality. In addition, it was found that the CCA found among the primary studies in each systematic review calculated to .22. Any CCA above .15 is considered "very high overlap".

CONCLUSIONS

The best available evidence suggests that Cemented HAs are better at preventing Prosthesis-related complications. Conversely, the best evidence also suggests that Cemented HA also results in longer operative time and increased intraoperative blood loss. When conducting future systematic reviews related to the topic, we ask that authors restrict conducting another systematic review until new evidence emerges so as not to confuse the clinical decision-making of physicians.

Impact of searching clinical trials registers in systematic reviews of pharmaceutical and non-pharmaceutical interventions: Reanalysis of meta-analyses

Systematic reviewers are advised to search trials registers to minimise risk of reporting biases. However, there has been little research on the impact of searching trials registers on the results of meta-analyses. We aimed to evaluate the impact of searching clinical trials registers for systematic reviews of pharmaceutical or non-pharmaceutical interventions. We searched PubMed, Scopus, Science Citation Index and Social Sciences Citation Index, and Education Collection for systematic reviews with meta-analyses indexed from 2 November to 2 December 2020. A random sample of systematic reviews was initially drawn, and for reviews which considered randomised trials eligible for inclusion, which had not searched a trials register, we searched ClinicalTrials.gov, EudraCT, ANZCTR, and the WHO ICTRP search portal for eligible trials. We compared meta-analytic effect estimates before and after including results from additional trials identified. We found additional trials for 63% (63/101) of eligible reviews; however, trials with results that could contribute to a meta-analysis were identified for only 20% (20/101) of the reviews. On average, there was no difference in the meta-analytic effect estimates before versus after adding the new trials. In summary, searching clinical trial registers led to identification of additional trials for many reviews; however, very few trials had results available for inclusion in meta-analyses. Including results from the new trials led to no change in the meta-analytic estimates, on average. Trials registers would be even more valuable to systematic reviewers if more trialists made use of them (i.e., registered their trials and posted results in a timely manner).

Network meta-analysis: What is its purpose in Orthopaedic literature?

Systematic reviews, of level-I primary literature, are the gold standard for the formation of Clinical Practice Guidelines in Orthopaedic Surgery. When systematic reviews have multiple groups of data, meta-analyses can be conducted to analyse the direct comparison of the data points (pairwise meta-analysis). Over recent years, statisticians have created a new statistical model called network meta-analyses that can be applied to systematic reviews. network meta-analyses allow for comparison of different treatment outcomes that may or may not have been directly assessed through level-I primary studies. network meta-analyses are appearing more and more in Orthopaedic Surgery literature; therefore, in this article, we discuss what a Network Meta-analysis is and its application in Orthopaedics.

Is Our Science Representative? A Systematic Review of Racial and Ethnic Diversity in Orthopaedic Clinical Trials from 2000 to 2020

BACKGROUND

A lack of racial and ethnic representation in clinical trials may limit the generalizability of the orthopaedic evidence base as it applies to patients in underrepresented minority populations and perpetuate existing disparities in use, complications, or functional outcomes. Although some commentators have implied the need for mandatory race or ethnicity reporting across all orthopaedic trials, the usefulness of race or ethnic reporting likely depends on the specific topic, prior evidence of disparities, and individualized study hypotheses.

QUESTIONS/PURPOSES

In a systematic review, we asked: (1) What proportion of orthopaedic clinical trials report race or ethnicity data, and of studies that do, how many report data regarding social covariates or genomic testing? (2) What trends and associations exist for racial and ethnic reporting among these trials between 2000 and 2020? (3) What is the racial or ethnic representation of United States trial participants compared with that reported in the United States Census?

METHODS

We performed a systematic review of randomized controlled trials with human participants published in three leading general-interest orthopaedic journals that focus on clinical research: The Journal of Bone and Joint Surgery, American Volume; Clinical Orthopaedics and Related Research; and Osteoarthritis and Cartilage. We searched the PubMed and Embase databases using the following inclusion criteria: English-language studies, human studies, randomized controlled trials, publication date from 2000 to 2020, and published in Clinical Orthopaedics and Related Research; The Journal of Bone and Joint Surgery, American Volume; or Osteoarthritis and Cartilage. Primary outcome measures included whether studies reported participant race or ethnicity, other social covariates (insurance status, housing or homelessness, education and literacy, transportation, income and employment, and food security and nutrition), and genomic testing. The secondary outcome measure was the racial and ethnic categorical distribution of the trial participants included in the studies reporting race or ethnicity. From our search, 1043 randomized controlled trials with 184,643 enrolled patients met the inclusion criteria. Among these studies, 21% (223 of 1043) had a small (< 50) sample size, 56% (581 of 1043) had a medium (50 to 200) sample size, and 23% (239 of 1043) had a large (> 200) sample size. Fourteen percent (141 of 1043) were based in the Northeast United States, 9.2% (96 of 1043) were in the Midwest, 4.7% (49 of 1043) were in the West, 7.2% (75 of 1043) were in the South, and 65% (682 of 1043) were outside the United States. We calculated the overall proportion of studies meeting the inclusion criteria that reported race or ethnicity. Then among the subset of studies reporting race or ethnicity, we determined the overall rate and distribution of social covariates and genomic testing reporting. We calculated the proportion of studies reporting race or ethnicity that also reported a difference in outcome by race or ethnicity. We calculated the proportion of studies reporting race or ethnicity by each year in the study period. We also calculated the proportions and 95% CIs of individual patients in each racial or ethnic category of the studies meeting the inclusion criteria.

RESULTS

During the study period (2000 to 2020), 8.5% (89 of 1043) of studies reported race or ethnicity. Of the trials reporting this factor, 4.5% (four of 89) reported insurance status, 15% (13 of 89) reported income, 4.5% (four of 89) reported housing or homelessness, 18% (16 of 89) reported education and literacy, 0% (0 of 89) reported transportation, and 2.2% (two of 89) reported food security or nutrition of trial participants. Seventy-eight percent (69 of 89) of trials reported no social covariates, while 22% (20 of 89) reported at least one. However, 0% (0 of 89) of trials reported genomic testing. Additionally, 5.6% (five of 89) of these trials reported a difference in outcomes by race or ethnicity. The proportion of studies reporting race or ethnicity increased, on average, by 0.6% annually (95% CI 0.2% to 1.0%; p = 0.02). After controlling for potentially confounding variables such as funding source, we found that studies with an increased sample size were more likely to report data by race or ethnicity; location in North America overall, Europe, Asia, and Australia or New Zealand (compared with the Northeast United States) were less likely to; and specialty-topic studies (compared with general orthopaedics research) were less likely to. Our sample of United States trials contained 18.9% more white participants than that reported in the United States Census (95% CI 18.4% to 19.4%; p < 0.001), 5.0% fewer Black participants (95% CI 4.6% to 5.3%; p < 0.001), 17.0% fewer Hispanic participants (95% CI 16.8% to 17.1%; p < 0.001), 5.3% fewer Asian participants (95% CI 5.2% to 5.4%; p < 0.001), and 7.5% more participants from other groups (95% CI 7.2% to 7.9%; p < 0.001).

CONCLUSION

Reporting of race or ethnicity data in orthopaedic clinical trials is low compared with other medical fields, although the proportion of diseases warranting this reporting might be lower in orthopaedics.

CLINICAL RELEVANCE

Investigators should initiate discussions about race and ethnicity reporting in the early stages of clinical trial development by surveying available published evidence for relevant health disparities, social determinants, and, when warranted, genomic risk factors. The decision to include or exclude race and ethnicity data in study protocols should be based on specific hypotheses, necessary statistical power, and an appreciation for unmeasured confounding. Future studies should evaluate cost-efficient mechanisms for obtaining baseline social covariate data and investigate researcher perspectives on current administrative workflows and decision-making algorithms for race and ethnicity reporting.