The darker side of randomized trials: recruitment challenges

The death of a neurotrauma trial lessons learned from the prematurely halted randomized evaluation of surgery in elderly with traumatic acute subdural hematoma (RESET-ASDH) trial

Introduction

Acute subdural hematoma (ASDH) due to traumatic brain injury (TBI) constitutes an increasing global health problem, especially in the elderly population. Treatment decisions on surgical versus conservative management pose a neurosurgical dilemma. Large practice variation exists between countries, hospitals, and individual neurosurgeons, illustrating the presence of 'clinical equipoise'. The RESET-ASDH trial aimed to address this dilemma but was terminated prematurely due to insufficient patient recruitment.

Research question

What factors may have contributed to the premature discontinuation of the RESET-ASDH trial?

Materials and methods

The RESET-ASDH was a multicenter randomized controlled trial (RCT) comparing functional outcome at 1 year after early surgery or an initial conservative treatment in elderly patients (≥65 years) with a traumatic ASDH. Logs of registry data, medical-ethical approval timelines and COVID-19 related research documents were analyzed. Furthermore, non-structured interviews with involved clinical research personnel were conducted.

Results

The concept of clinical equipoise was broadly misinterpreted by neurosurgeons as individual uncertainty, hampering patient recruitment. Also, the elderly target population complicated the inclusion process as elderly and their informal caregivers were hesitant to participate in our acute surgical trial. Moreover, the COVID-19 pandemic added additional hurdles like delayed medical-ethical approval, a decline in eligible patients and repeated trial halts during the peaks of the pandemic.

Discussion and conclusion

The premature termination of the RESET-ASDH study may have been related to the trial's methodology and target population with an additional impact of COVID-19. Future acute neurosurgical trials in elderly may consider these challenges to prevent premature trial termination.

Natural Experiments as a Study Method in Spinal Trauma Surgery: A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVES

To determine if the natural experiment design is a useful research methodology concept in spinal trauma care, and to determine if this methodology can be a viable alternative when randomized controlled trials are either infeasible or unethical.

METHODS

A Medline, Embase and Cochrane database search was performed between 2004 and 2023 for studies comparing different treatment modalities of spinal trauma. All observational studies with a natural experiment design comparing different treatment modalities of spinal trauma were included. Data extraction and quality assessment with the MINORS criteria was performed.

RESULTS

Four studies with a natural experiment design regarding patients with traumatic spinal fractures were included. All studies were retrospective, one study collected follow-up data prospectively. Three studies compared different operative treatment modalities, whereas one study compared different antibiotic treatment strategies. Two studies compared preferred treatment modalities between expertise centers, one study between departments (neuro- and orthopedic surgery) and one amongst surgeons. For the included retrospective studies, MINORS scores (maximum score 18) were high ranging from 12-17 and with a mean (SD) of 14.6 (1.63).

CONCLUSIONS

Since 2004 only four studies using a natural experiment design have been conducted in spinal trauma. In the included studies, comparability of patient groups was high emphasizing the potential of natural experiments in spinal trauma research. Natural experiments design should be considered more frequently in future research in spinal trauma as they may help to address difficult clinical problems when RCT's are infeasible or unethical.

Evidence-Based Surgery: What Can Intra-Operative Images Contribute?

Evidence-based medicine integrates results from randomized controlled trials (RCTs) and meta-analyses, combining the best external evidence with individual clinical expertise and patients' preferences. However, RCTs of surgery differ from those of medicine in that surgical performance is often assumed to be consistent. Yet, evaluating whether each surgery is performed to the same standard is quite challenging. As a primary issue, the novelty of this review is to emphasize-with a focus on orthopedic trauma-the advantage of having complete intra-operative image documentation, allowing the direct evaluation of the quality of the intra-operative technical performance. The absence of complete intra-operative image documentation leads to the inhomogeneity of case series, yielding inconsistent results due to the impossibility of a secondary analysis. Thus, comparisons and the reproduction of studies are difficult. Access to complete intra-operative image data in surgical RCTs allows not only secondary analysis but also comparisons with similar cases. Such complete data can be included in electronic papers. Offering these data to peers-in an accessible link-when presenting papers facilitates the selection process and improves publications for readers. Additionally, having access to the full set of image data for all presented cases serves as a rich resource for learning. It enables the reader to sift through the information and pinpoint the details that are most relevant to their individual needs, allowing them to potentially incorporate this knowledge into daily practice. A broad use of the concept of complete intra-operative image documentation is pivotal for bridging the gap between clinical research findings and real-world applications. Enhancing the quality of surgical RCTs would facilitate the equalization of evidence acquisition in both internal medicine and surgery. Joint effort by surgeons, scientific societies, publishers, and healthcare authorities is needed to support the ideas, implement economic requirements, and overcome the mental obstacles to its realization.

Methodological aspects of a randomized within-patient concurrent controlled design for clinical trials in spine surgery

INTRODUCTION

Randomized controlled trials are considered the highest level of evidence, but their feasibility in the surgical field is severely hampered by methodological and practical issues. Concurrent comparison between the experimental and control conditions within the same patient can be an effective strategy to mitigate some of these challenges and improve generalizability, mainly by the elimination of between-patient variability and reduction of the required sample size. This article aims (1) to describe the methodological aspects of a randomized within-patient controlled trial and (2) to quantify the added value of this design, based on a recently completed randomized within-patient controlled trial on bone grafts in instrumented lumbar posterolateral spinal fusion.

METHODS

Boundary conditions for the application of the randomized within-patient controlled trial design were identified. Between-patient variability was quantified by the intraclass correlation coefficient and concordance in the primary fusion outcome. Sample size, study duration and costs were compared with a classic randomized controlled trial design.

RESULTS

Boundary conditions include the concurrent application of the experimental and control conditions to identical but physically separated sites. Moreover, the outcome of interest should be local, uncorrelated and independently assessable. The spinal fusion outcomes within a patient were found to be more similar than between different patients (intraclass correlation coefficient 32% and concordance 64%), demonstrating a clear effect of patient-related factors. The randomized within-patient controlled trial design allowed a reduction of the sample size to one-third of a parallel-group randomized controlled trial, thereby halving the trial duration and costs.

CONCLUSION

When suitable, the randomized within-patient controlled trial is an efficient design that provides a solution to some of the considerable challenges of a classic randomized controlled trial in (spine) surgery. This design holds specific promise for efficacy studies of non-active bone grafts in instrumented posterolateral fusion surgery.

Trauma Risk Score Matching for Observational Studies in Orthopedic Trauma

OBJECTIVES

To determine if matching by trauma risk score is non-inferior to matching by chronic comorbidities and/or a combination of demographic and patient characteristics in observational studies of acute trauma in a hip fracture model.

DESIGN

Retrospective cohort study SETTING: Level-1 Trauma Center PATIENTS: 1,590 hip fracture [AO/OTA 31A and 31B] patients age 55 and over treated between October 2014 and February 2020 at 4 hospitals within a single academic medical center.

INTERVENTION

Repeatedly matching randomized subsets of patients by (1) Score for Trauma Triage in Geriatric and Middle-Aged (STTGMA), (2) Charlson Comorbidity Index (CCI), or (3) a combination of sex, age, CCI and body mass index (BMI).

MAIN OUTCOME MEASUREMENTS

"Matching failures" where rate of significant differences in variables of matched cohorts exceeds the 5% expected by chance.

RESULTS

STTGMA and combination matching resulted in no "matching failures". Matching by CCI alone resulted in "matching failures" of BMI, ASA class, STTGMA, major complications, sepsis, pneumonia, acute respiratory failure, and 90-day readmission.

CONCLUSIONS

STTGMA matching in observational cohort studies is less likely to yield significant differences of demographics and outcomes than CCI matching. STTGMA matching is noninferior to matching a combination of demographic variables optimized for each treatment cohort. STTGMA matching is apt to reflect equipoise of health at admission and outcome likelihood in observational cohort studies of orthopedic trauma, while maintaining consistent weighting of demographic and injury characteristic variables that may expand the generalizability of these studies.

LEVEL OF EVIDENCE

Level III.

The challenge of equipoise in trials with a surgical and non-surgical comparison: a qualitative synthesis using meta-ethnography

BACKGROUND

Randomised controlled trials in surgery can be a challenge to design and conduct, especially when including a non-surgical comparison. As few as half of initiated surgical trials reach their recruitment target, and failure to recruit is cited as the most frequent reason for premature closure of surgical RCTs. The aim of this qualitative evidence synthesis was to identify and synthesise findings from qualitative studies exploring the challenges in the design and conduct of trials directly comparing surgical and non-surgical interventions.

METHODS

A qualitative evidence synthesis using meta-ethnography was conducted. Six electronic bibliographic databases (Medline, Central, Cinahl, Embase and PsycInfo) were searched up to the end of February 2018. Studies that explored patients' and health care professionals' experiences regarding participating in RCTs with a surgical and non-surgical comparison were included. The GRADE-CERQual framework was used to assess confidence in review findings.

RESULTS

In total, 3697 abstracts and 49 full texts were screened and 26 published studies reporting experiences of patients and healthcare professionals were included. The focus of the studies (24/26) was primarily related to the challenge of recruitment. Two studies explored reasons for non-compliance to treatment allocation following randomisation. Five themes related to the challenges to these types of trials were identified: (1) radical choice between treatments; (2) patients' discomfort with randomisation: I want the best treatment for me as an individual; (3) challenge of equipoise: patients' a priori preferences for treatment; (4) challenge of equipoise: clinicians' a priori preferences for treatment and (5) imbalanced presentation of interventions.

CONCLUSION

The marked dichotomy between the surgical and non-surgical interventions was highlighted in this review as making recruitment to these types of trials particularly challenging. This review identified factors that increase our understanding of why patients and clinicians may find equipoise more challenging in these types of trials compared to other trial comparisons. Trialists may wish to consider exploring the balance of potential factors influencing patient and clinician preferences towards treatments before they start recruitment, to enable issues specific to a particular trial to be identified and addressed. This may enable trial teams to make more efficient considered design choices and benefit the delivery of such trials.

Barriers to recruitment to an orthopaedic randomized controlled trial comparing two surgical procedures for ankle arthritis : a qualitative study

Aims

A multicentre, randomized, clinician-led, pragmatic, parallel-group orthopaedic trial of two surgical procedures was set up to obtain high-quality evidence of effectiveness. However, the trial faced recruitment challenges and struggled to maintain recruitment rates over 30%, although this is not unusual for surgical trials. We conducted a qualitative study with the aim of gathering information about recruitment practices to identify barriers to patient consent and participation to an orthopaedic trial.

Methods

We collected 11 audio recordings of recruitment appointments and interviews of research team members (principal investigators and research nurses) from five hospitals involved in recruitment to an orthopaedic trial. We analyzed the qualitative data sets thematically with the aim of identifying aspects of informed consent and information provision that was either unclear, disrupted, or hindered trial recruitment.

Results

Recruiters faced four common obstacles when recruiting to a surgical orthopaedic trial: patient preferences for an intervention; a complex recruitment pathway; various logistical issues; and conflicting views on equipoise. Clinicians expressed concerns that the trial may not show significant differences in the treatments, validating their equipoise. However, they experienced role conflicts due to their own preference and perceived patient preference for an intervention arm.

Conclusion

This study provided initial information about barriers to recruitment to an orthopaedic randomized controlled trial. We shared these findings in an all-site investigators’ meeting and encouraged researchers to find solutions to identified barriers; this led to the successful completion of recruitment. Complex trials may benefit for using of a mixed-methods approach to mitigate against recruitment failure, and to improve patient participation and informed consent.

Cite this article: Bone Jt Open 2021;2(8):631–637.

Barriers to recruitment, retention and intervention delivery in a randomized trial among patients with firearm injuries

BACKGROUND

We discuss barriers to recruitment, retention, and intervention delivery in a randomized controlled trial (RCT) of patients presenting with firearm injuries to a Level 1 trauma center. The intervention was adapted from the Critical Time Intervention and included a six-month period of support in the community after hospital discharge to address recovery goals. This study was one of the first RCTs of a hospital- and community-based intervention provided solely among patients with firearm injuries.

MAIN TEXT

Barriers to recruitment included limited staffing, coupled with wide variability in length of stay and admission times, which made it difficult to predict the best time to recruit. At the same time, more acutely affected patients needed more time to stabilize in order to determine whether eligibility criteria were met. Barriers to retention included insufficient patient resources for stable housing, communication and transportation, as well as limited time for patients to meet with study staff to respond to follow-up surveys. These barriers similarly affected intervention delivery as patients who were recruited, but had fewer resources to help with recovery, had lower intervention engagement. These barriers fall within the broader context of system avoidance (e.g., avoiding institutions that keep formal records). Since the patient sample was racially diverse with the majority of patients having prior criminal justice system involvement, this may have precluded active participation from some patients, especially those from communities that have been subject to long and sustained history of trauma and racism. We discuss approaches to overcoming these barriers and the importance of such efforts to further implement and evaluate hospital-based violence intervention programs in the future.

CONCLUSION

Developing strategies to overcome barriers to data collection and ongoing participant contact are essential to gathering robust information to understand how well violence prevention programs work and providing the best care possible for people recovering from injuries.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02630225 . Registered 12/15/2015.

Recruitment patterns in a large international randomized controlled trial of perioperative care in cancer patients

INTRODUCTION

The Prophylactic Antibiotic Regimens in Tumor Surgery (PARITY) randomized controlled trial (RCT) was the first study to prospectively enroll and randomize orthopedic oncology patients in multiple centers internationally. The objective of this study was to describe recruitment patterns, to examine the differences in enrollment across different PARITY sites, and to identify variables associated with differing levels of recruitment.

METHODS

Data from this study was obtained from the PARITY trial Methods Center and records of correspondence between the Methods Center and recruiting sites. We performed descriptive statistics to report the recruitment patterns over time. We compared recruitment, time to set up, and time to enroll the first patient between North American and international sites, private and public healthcare models, and the presence or absence of research personnel. Two-tailed non-paired t tests were performed to test average monthly recruitment rates between groups.

RESULTS

A total of 602 patients from 36 North American and 12 international sites were recruited from 2013 to 2019. North American sites were able to become fully enrollment-ready at an average of 19.5 months and international sites at an average of 27 months. Once enrolling, international sites were able to enroll 0.59 patients per/month whereas North American sites averaged a monthly recruitment rate of 0.2 patients/month once enrolling. Sites with research personnel reached enrollment-ready status at an average of 19.3 months and sites without research support at an average of 30.3 months. Once enrolling, the recruitment rate was 0.28 patients/month and 0.2 patients per month for sites with and without research support, respectively. Publicly funded sites had a monthly enrollment of 0.4 patients/month whereas privately funded sites had a monthly enrollment rate of 0.17 patients/month.

CONCLUSIONS

As a collaborative group, the PARITY investigators increased the pace of recruitment throughout the trial, likely by increasing the number of active sites. The longer time to start-up at international sites may be due to the complex governing regulations of pharmaceutical trials. Nevertheless, international sites should be considered essential as they recruited significantly more patients per month once active. The absence of research support personnel may lead to delays in the time to start-up. The results of the current study will provide guidance for choosing which sites to recruit for participation in future collaborative clinical trials in orthopedic oncology and other surgical specialties.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01479283 . Prospectively registered on November 24, 2011.

Recruitment and retention of participants in UK surgical trials: survey of key issues reported by trial staff

BACKGROUND

Recruitment and retention of participants in surgical trials is challenging. Knowledge of the most common and problematic issues will aid future trial design. This study aimed to identify trial staff perspectives on the main issues affecting participant recruitment and retention in UK surgical trials.

METHODS

An online survey of UK surgical trial staff was performed. Respondents were asked whether or not they had experienced a range of recruitment and retention issues, and, if yes, how relatively problematic these were (no, mild, moderate or serious problem).

RESULTS

The survey was completed by 155 respondents including 60 trial managers, 53 research nurses, 20 trial methodologists and 19 chief investigators. The three most common recruitment issues were: patients preferring one treatment over another (81·5 per cent of respondents); clinicians' time constraints (78·1 per cent); and clinicians preferring one treatment over another (76·8 per cent). Seven recruitment issues were rated moderate or serious problems by a majority of respondents, the most problematic being a lack of eligible patients (60·3 per cent). The three most common retention issues were: participants forgetting to return questionnaires (81·4 per cent); participants found to be ineligible for the trial (74·3 per cent); and long follow-up period (70·7 per cent). The most problematic retention issues, rated moderate or serious by the majority of respondents, were participants forgetting to return questionnaires (56·4 per cent) and insufficient research nurse time/funding (53·6 per cent).

CONCLUSION

The survey identified a variety of common recruitment and retention issues, several of which were rated moderate or serious problems by the majority of participating UK surgical trial staff. Mitigation of these problems may help boost recruitment and retention in surgical trials.

Study methodology in trauma care: towards question-based study designs

The randomized controlled trial (RCT) in surgery may not always be ethical, feasible, or necessary to address a particular research question about the effect of a surgical intervention. If so, properly designed and conducted observational (non-randomized) studies may be valuable alternatives for an RCT and produce credible results. In this paper, we discus differences between RCTs and observational studies and differentiate between three types of comparisons of surgical interventions. We assert that results of different designs should be regarded as complementary to each other when evaluating surgical interventions. Criteria for credible observational research are presented to provide guidance for future observational research of surgical interventions. We argue that the research question that is being asked should guide the discussion about the value of a particular study design.

Which orthopaedic trauma patients are likely to refuse to participate in a clinical trial? A latent class analysis

OBJECTIVE

The study aimed to assess systematic differences in the characteristics of patients that consented for the trial compared with the broader pool of eligible patients in a large, pragmatic orthopaedic trauma trial.

DESIGN

A retrospective observational study performed from April 2017 to March 2018.

SETTING

Academic trauma centre in Baltimore, USA.

PARTICIPANTS

There were 642 eligible adult trial participants with an operative fracture to the appendicular skeleton and were indicated for blood clot prophylaxis. The median age of the sample was 50 years (IQR: 31-63), and 60% were male.

PRIMARY OUTCOME MEASURE

The primary outcome was the refusal to enrol in the trial. Demographic and injury covariates were included in iterations of latent class models. The final model was selected based on a minimum Bayesian information criterion.

RESULTS

The final model identified three clusters with five covariates predictive of cluster membership (age, neighbourhood-based socioeconomic status, alcohol use, multiple fractures, multiple surgeries). The three clusters were associated with 22% (Cluster 1), 38% (Cluster 2) and 62% (Cluster 3) refusal rates, respectively. Members of Cluster 3 (n=84) were most commonly between 66 and 80 years of age (49% vs 6% (Cluster 1) and 21% (Cluster 2)), of high neighbourhood-based socioeconomic status (85% vs 63% (Cluster 1) and 8% (Cluster 2)), with isolated fractures (100% vs 80% (Cluster 1) and 92% (Cluster 2)), and were less likely to have multiple surgeries compared with the other clusters (28% vs 47% (Cluster 1) and 35% (Cluster 2)).

CONCLUSION

In this study, the likelihood of refusing to participate in the trial ranged from 22% to 62% in the three identified clusters. Elderly age, high socioeconomic status, and less severe injuries defined the cluster that was most likely to refuse trial participation.

TRIAL REGISTRATION NUMBER

NCT02984384.

The impact of surgeon and patient treatment preferences in an orthopaedic trauma surgery trial

Background

Surgeon and patient treatment preferences are important threats to the internal and external validity of surgical trials such as PROFHER, which compared surgical versus non-surgical treatment for displaced fractures of the proximal humerus in adults. We explored the treatment preferences expressed by surgeons and patients in the trial and how these impacted on patient selection, trial conduct and patient outcome.

Methods

A series of exploratory secondary analyses of the PROFHER trial data were undertaken. We reviewed the extent of surgeon and patient treatment preferences (surgery or not surgery) at screening (n = 1250) as well as prior preference (including no preference) of randomised patients (n = 250), and assessed their impact on recruitment and adherence to follow-up and rehabilitation. Changes in treatment after 2 years’ follow-up were explored. Patient preference and characteristics associated with trial inclusion or treatment preference (t test, chi-squared test, Wilcoxon rank-sum test) were included as treatment interaction terms in the primary trial analysis of shoulder functioning (Oxford Shoulder Score, OSS).

Results

Surgeons excluded 17% of otherwise eligible patients based on lack of equipoise; these patients had less complex fractures (p < 0.001) and tended to be older (p = 0.062). Surgeons were more likely to recommend surgery for patients under 65 years of age (p = 0.059) and who had injured their right shoulder (p = 0.052). Over half of eligible patients (56%) did not consent to take part in the trial; these patients tended to be older (p = 0.022), with a preference for not surgery (74%; which was associated with older age, p = 0.039). There were no differential treatment effects (p value of interaction) for shoulder functioning (OSS) based on subgroups of patient preference (p = 0.751), age group (p = 0.264), fracture type (p = 0.954) and shoulder dominance (p = 0.850). Patients who were randomised to their preferred treatment had better follow-up rates (94 vs 84% at 2 years) and treatment adherence (90 vs 83% reported completing home exercises). Patients who were not randomised to their preferred treatment were more likely to change their treatment preference at 24 months (60 vs 26%).

Conclusions

The robustness of the PROFHER trial findings was confirmed against possible bias introduced by surgeon and patient preferences. The importance of collecting preference data is highlighted.

Trial registration

ISRCTN50850043. Registered on 25 March 2008.

Enrollment Obstacles in a Randomized Controlled Trial: A Performance Survey of Enrollment in BEST-CLI Sites

BACKGROUND

Although randomized controlled trials (RCTs) provide the most reliable form of scientific evidence, they are challenging to complete because of a variety of enrollment obstacles. We evaluated obstacles in a large RCT by comparing survey results at high-performing sites (HPS) and low-performing sites (LPS).

METHODS

The Best Endovascular versus Best Surgical Therapy in Patients with Critical Limb Ischemia (BEST-CLI) trial is a prospective, pragmatic, multicenter, and multispecialty RCT that will compare clinical outcomes, quality of life, and cost in patients with CLI randomized to surgical bypass or endovascular therapy. BEST-CLI aims to enroll 2100 patients at 160 sites in North America, Europe, and New Zealand. We surveyed the 30 HPS and 30 LPS to assess perceptions of enrollment obstacles. HPS were defined by enrollment of 0.5 subjects or more per month or more than 8 total subjects enrolled. LPS were defined by enrollment of 0.1 subjects per month or only 1 subject total. Responses were compared by site performance status.

RESULTS

There were 22 of 30 (73%) HPS and 14 of 30 (47%) LPS that answered the survey (P = 0.06), including 17 investigators and 31 coordinators. The mean total enrollment and rate of enrollment at HPS and LPS were 12.5 subjects at 1.5 subjects/month and 1.0 subject at 0.1 subjects/month, respectively. The most common barrier to enrollment at HPS was difficulty convincing patients and their families to participate (36%), whereas at LPS both difficulty convincing patients and difficulty motivating investigators to enroll (29% each) were most frequently cited. At HPS, the most common obstacle to consenting patients for the trial was patient/family having strong preference toward revascularization strategy (32%) and at LPS it was patient/family not wanting to have treatment chosen at random (36%). At 55% of HPS and 43% of LPS, the trial team was reported as extremely collaborative (P = 0.73), whereas 68% of HPS and 64% of LPS reported having identified a trial champion on their team (P = 1). The most restrictive perceived enrollment criterion at HPS was prior index limb stenting with significant restenosis (32%), whereas at LPS it was excessive risk for surgical bypass (43%). Materials to aid enrollment were used equally at HPS and LPS: patient brochures at 59% HPS and 64% LPS (P = 1); investigator talking points at 45% of HPS and 36% of LPS (P = 0.73).

CONCLUSIONS

Patient perceptions and investigator biases are significant challenges to enrollment in large RCTs. In the BEST-CLI trial, difficulty convincing patients and families to allow treatment randomization and difficulty in motivating investigators were major enrollment obstacles.

An approach for transgender population information extraction and summarization from clinical trial text

Background

Gender information frequently exists in the eligibility criteria of clinical trial text as essential information for participant population recruitment. Particularly, current eligibility criteria text contains the incompleteness and ambiguity issues in expressing transgender population, leading to difficulties or even failure of transgender population recruitment in clinical trial studies.

Methods

A new gender model is proposed for providing comprehensive transgender requirement specification. In addition, an automated approach is developed to extract and summarize gender requirements from unstructured text in accordance with the gender model. This approach consists of: 1) the feature extraction module, and 2) the feature summarization module. The first module identifies and extracts gender features using heuristic rules and automatically-generated patterns. The second module summarizes gender requirements by relation inference.

Results

Based on 100,134 clinical trials from ClinicalTrials.gov, our approach was compared with 20 commonly applied machine learning methods. It achieved a macro-averaged precision of 0.885, a macro-averaged recall of 0.871 and a macro-averaged F₁-measure of 0.878. The results illustrated that our approach outperformed all baseline methods in terms of both commonly used metrics and macro-averaged metrics.

Conclusions

This study presented a new gender model aiming for specifying the transgender requirement more precisely. We also proposed an approach for gender information extraction and summarization from unstructured clinical text to enhance transgender-related clinical trial population recruitment. The experiment results demonstrated that the approach was effective in transgender criteria extraction and summarization.

Connecting the public with clinical trial options: The ResearchMatch Trials Today tool

Potential participants seek information about clinical trials for many reasons, but the process can be challenging. We analyzed 101,249 searches in ResearchMatch Trials Today, a free interface to recruiting trials from ClinicalTrials.gov. Searches from March 2015 to November 2016 included a broad range of conditions and healthy volunteer concepts, including 12,649 unique topics. Trials Today data indicate that it is being used to identify trials on a variety of topics.

Factors influencing the enrollment in randomized controlled trials in orthopedics

Background

Low enrollment rates are a threat to the external validity of clinical trials. The purpose of this study was to identify factors associated with lower enrollment rates in randomized controlled trials (RCTs) involving orthopedic procedures.

Methods

We performed a search in PubMed/MEDLINE for RCTs that involved any orthopedic surgical procedure, compared different intraoperative interventions, were published in English in a peer-reviewed journal between 2003 and 2014, and reported the numbers of both enrolled and eligible subjects. The primary outcome was the enrollment rate, defined as the number of enrolled subjects divided by the number of eligible subjects. We used a meta-regression to identify factors associated with lower enrollment rates.

Results

The combined estimate of enrollment rate across all 393 studies meeting inclusion criteria was 90% (95% CI: 89–92%). Trials in North America had significantly lower enrollment rates compared to trials in the rest of the world (80% vs. 92%, p < 0.0001). Trials comparing operative and non-operative treatments had significantly lower enrollment rates than trials comparing two different operative interventions (80% vs. 91%, p < 0.0001). Among trials comparing operative and non-operative interventions, there was a marked difference in enrollment rate by region: 49% in North America and 86% elsewhere (p < 0.0001).

Conclusions

RCTs investigating orthopedic procedures have variable enrollment rates depending on their location and the difference between the interventions being studied. North American trials that compare operative and non-operative interventions have the lowest enrollment rates. Investigators planning RCTs would be well advised to consider these data in planning recruitment efforts.

The (in)stability of 21st century orthopedic patient contact information and its implications on clinical research: A cross-sectional study

BACKGROUND

In clinical research, minimizing patients lost to follow-up is essential for data validity. Researchers can employ better methodology to prevent patient loss. We examined how orthopedic surgery patients' contact information changes over time to optimize data collection for long-term outcomes research.

METHODS

Patients presenting to orthopedic outpatient clinics completed questionnaires regarding methods of contact: home phone, cell phone, mailing address, and e-mail address. They reported currently available methods of contact, if they changed in the past 5 and 10 years, and when they changed. Differences in the rates of change among methods were assessed via Fisher's exact tests. Whether participants changed any of their contact information in the past 5 and 10 years was determined via multivariate modeling, controlling for demographic variables.

RESULTS

Among 152 patients, 51% changed at least one form of contact information within 5 years, and 66% changed at least one form within 10 years. The rate of change for each contact method was similar over 5 (15%-28%) and 10 years (26%-41%). One patient changed all four methods of contact within the past 5 years and seven within the past 10 years. Females and younger patients were more likely to change some type of contact information.

CONCLUSION

The type of contact information least likely to change over 5-10 years is influenced by demographic factors such as sex and age, with females and younger participants more likely to change some aspect of their contact information. Collecting all contact methods appears necessary to minimize patients lost to follow-up, especially as technological norms evolve.

Has the Level of Evidence of Podium Presentations at the Musculoskeletal Tumor Society Annual Meeting Changed Over Time?

BACKGROUND

Level of evidence (LOE) framework is a tool with which to categorize clinical studies based on their likelihood to be influenced by bias. Improvements in LOE have been demonstrated throughout orthopaedics, prompting our evaluation of orthopaedic oncology research LOE to determine if it has changed in kind.

QUESTIONS/PURPOSES

(1) Has the LOE presented at the Musculoskeletal Tumor Society (MSTS) annual meeting improved over time? (2) Over the past decade, how do the MSTS and Orthopaedic Trauma Association (OTA) annual meetings compare regarding LOE overall and for the subset of therapeutic studies?

METHODS

We reviewed abstracts from MSTS and OTA annual meeting podium presentations from 2005 to 2014. Three independent reviewers evaluated a total of 1222 abstracts for study type and LOE; there were 577 abstracts from MSTS and 645 from OTA. Changes in the distributions of study type and LOE over time were evaluated by Pearson chi-square test.

RESULTS

There was no change over time in MSTS LOE for all study types (p = 0.13) and therapeutic (p = 0.36) study types during the reviewed decade. In contrast, OTA LOE increased over this time for all study types (p < 0.01). The proportion of Level I therapeutic studies was higher at the OTA than the MSTS (3% [14 of 413] versus 0.5% [two of 387], respectively), whereas the proportion of Level IV studies was lower at the OTA than the MSTS (32% [134 of 413] versus 75% [292 of 387], respectively) during the reviewed decade. The proportion of controlled therapeutic studies (LOE I through III) versus uncontrolled studies (LOE IV) increased over time at OTA (p < 0.021), but not at MSTS (p = 0.10).

CONCLUSIONS

Uncontrolled case series continue to dominate the MSTS scientific program, limiting progress in evidence-based clinical care. Techniques used by the OTA to improve LOE may be emulated by the MSTS. These techniques focus on broad participation in multicenter collaborations that are designed in a comprehensive manner and answer a pragmatic clinical question.

Concept and implementation of a study dashboard module for a continuous monitoring of trial recruitment and documentation

BACKGROUND

The difficulty of managing patient recruitment and documentation for clinical trials prompts a demand for instruments for closely monitoring these critical but unpredictable processes. Increasingly adopted Electronic Data Capture (EDC) applications provide novel opportunities to reutilize stored information for an efficient management of traceable trial workflows. In related clinical and administrative settings, so-called digital dashboards that continuously visualize time-dependent parameters have recently met a growing acceptance. To investigate the technical feasibility of a study dashboard for monitoring the progress of patient recruitment and trial documentation, we set out to develop a propositional prototype in the form of a separate software module.

METHODS

After narrowing down functional requirements in semi-structured interviews with study coordinators, we analyzed available interfaces of a locally deployed EDC application, and designed the prototypical study dashboard based on previous findings. The module thereby leveraged a standardized export format in order to extract and import relevant trial data into a clinical data warehouse. Web-based reporting tools then facilitated the definition of diverse views, including diagrams of the progress of patient accrual and form completion at different granularity levels. To estimate the utility of the dashboard and its compatibility with current workflows, we interviewed study coordinators after a demonstration of sample outputs from ongoing trials.

RESULTS

The employed tools promoted a rapid development. Displays of the implemented dashboard are organized around an entry page that integrates key metrics for available studies, and which links to more detailed information such as study-specific enrollment per center. The interviewed experts commented that the included graphical summaries appeared suitable for detecting that something was generally amiss, although practical remedies would mostly depend on additional information such as access to the original patient-specific data. The dependency on a separate application was seen as a downside. Interestingly, the prospective users warned that in some situations knowledge of specific accrual statistics might undermine blinding in a subtle yet intricate fashion, so ignorance of certain patient features was seen as sometimes preferable for reproducibility.

DISCUSSION

Our proposed study dashboard graphically recaps key progress indicators of patient accrual and trial documentation. The modular implementation illustrates the technical feasibility of the approach. The use of a study dashboard might introduce certain technical requirements as well as subtle interpretative complexities, which may have to be weighed against potential efficiency gains.

Will they participate? Predicting patients' response to clinical trial invitations in a pediatric emergency department

Objective (1) To develop an automated algorithm to predict a patient’s response (ie, if the patient agrees or declines) before he/she is approached for a clinical trial invitation; (2) to assess the algorithm performance and the predictors on real-world patient recruitment data for a diverse set of clinical trials in a pediatric emergency department; and (3) to identify directions for future studies in predicting patients’ participation response.

Materials and Methods We collected 3345 patients’ response to trial invitations on 18 clinical trials at one center that were actively enrolling patients between January 1, 2010 and December 31, 2012. In parallel, we retrospectively extracted demographic, socioeconomic, and clinical predictors from multiple sources to represent the patients’ profiles. Leveraging machine learning methodology, the automated algorithms predicted participation response for individual patients and identified influential features associated with their decision-making. The performance was validated on the collection of actual patient response, where precision, recall, F-measure, and area under the ROC curve were assessed.

Results Compared to the random response predictor that simulated the current practice, the machine learning algorithms achieved significantly better performance (Precision/Recall/F-measure/area under the ROC curve: 70.82%/92.02%/80.04%/72.78% on 10-fold cross validation and 71.52%/92.68%/80.74%/75.74% on the test set). By analyzing the significant features output by the algorithms, the study confirmed several literature findings and identified challenges that could be mitigated to optimize recruitment.

Conclusion By exploiting predictive variables from multiple sources, we demonstrated that machine learning algorithms have great potential in improving the effectiveness of the recruitment process by automatically predicting patients’ participation response to trial invitations.

Research gaps in pancreatic cancer research and comparative effectiveness research methodologies

Despite advances in cancer care, pancreatic adenocarcinoma remains one of the most lethal tumors. Most patients with pancreatic cancer are diagnosed with late stage disease, and approximately 6 % of patients are alive 5 years after diagnosis. Of the 10-20 % of patients who are candidates for resection and multi-modality therapy, most will succumb to the disease with 5-year survival rates only reaching approximately 25 % (Lim et al. in Annals of surgery 237(1):74-85, 2003 [1]; Trede et al. in Annals of surgery 211(4):447-458, 1990 [2]; Crist et al. in Annals of surgery 206(3):358-365, 1987 [3]). Clearly, there is a need to improve the management of this disease. To identify gaps in research and formulate strategies to address these issues, we designed a framework to encompass the scope of research for pancreatic cancer. In this chapter, we will examine each topic heading within this framework for gaps in knowledge and present research strategies focusing on diverse comparative effectiveness research (CER) methodologies to address the identified gaps.

Prostate capsule sparing versus nerve sparing radical cystectomy for bladder cancer: results of a randomized, controlled trial

PURPOSE

Prostate capsule sparing and nerve sparing cystectomies are alternative procedures for bladder cancer that may decrease morbidity while achieving cancer control. However, to our knowledge the comparative effectiveness of these approaches has not been established. We evaluated functional and oncologic outcomes in patients undergoing these procedures.

MATERIALS AND METHODS

We performed a single institution trial in patients with bladder cancer in whom transurethral prostatic urethral biopsy and transrectal prostate biopsy were negative. Men were randomized to prostate capsule sparing or nerve sparing cystectomy with neobladder creation and stratified by Sexual Health Inventory for Men score (greater than 21 vs 21 or less). Our primary end point was 12-month overall urinary function as measured by Bladder Cancer Index. Secondary end points included sexual function, cancer control and complications.

RESULTS

A total of 40 patients were enrolled in the study with 20 patients in each arm. Urinary function at 12 months decreased by 13 and 28 points in the prostate capsule and nerve sparing groups, respectively (p = 0.10). Sexual function followed a similar pattern (p = 0.06). There was no difference in recurrence-free, metastasis-free or overall survival (each p >0.05). The rate of incidentally detected prostate cancer was similar (p = 0.15).

CONCLUSIONS

Our study provides a randomized comparison of prostate capsule sparing and nerve sparing cystectomy techniques. We found no difference in functional or oncologic outcomes between the 2 approaches, although our study was underpowered due to a lack of patient accrual.

ClinicalTrials.gov as a data source for semi-automated point-of-care trial eligibility screening

Background

Implementing semi-automated processes to efficiently match patients to clinical trials at the point of care requires both detailed patient data and authoritative information about open studies.

Objective

To evaluate the utility of the ClinicalTrials.gov registry as a data source for semi-automated trial eligibility screening.

Methods

Eligibility criteria and metadata for 437 trials open for recruitment in four different clinical domains were identified in ClinicalTrials.gov. Trials were evaluated for up to date recruitment status and eligibility criteria were evaluated for obstacles to automated interpretation. Finally, phone or email outreach to coordinators at a subset of the trials was made to assess the accuracy of contact details and recruitment status.

Results

24% (104 of 437) of trials declaring on open recruitment status list a study completion date in the past, indicating out of date records. Substantial barriers to automated eligibility interpretation in free form text are present in 81% to up to 94% of all trials. We were unable to contact coordinators at 31% (45 of 146) of the trials in the subset, either by phone or by email. Only 53% (74 of 146) would confirm that they were still recruiting patients.

Conclusion

Because ClinicalTrials.gov has entries on most US and many international trials, the registry could be repurposed as a comprehensive trial matching data source. Semi-automated point of care recruitment would be facilitated by matching the registry's eligibility criteria against clinical data from electronic health records. But the current entries fall short. Ultimately, improved techniques in natural language processing will facilitate semi-automated complex matching. As immediate next steps, we recommend augmenting ClinicalTrials.gov data entry forms to capture key eligibility criteria in a simple, structured format.

Do surgeon credentials affect the rate of incidental durotomy during spine surgery

PURPOSE

Incidental durotomy is a potential complication of spinal surgery which can cause a number of intra-operative and post-operative complications. The purpose of this study was to determine if the primary operator's credentials impacted on the incidence of durotomy intra-operatively.

METHODS

Prospectively collected data of operator credentials in relation to the incidence of durotomy were acquired from the International Eurospine Tango database. The significance of variability and risk factors between operators was measured using the Chi-squared test.

RESULTS

Data from a total of 3,764 patients were captured from the Tango registry. Of these 162 (4.3%) had a durotomy. Of the total number of patients, the primary operator was neurosurgical in 1,369 (36.4%) cases; orthopaedic in 180 (4.8%) cases; other (pre-certification) in 236 (6.3%) cases; specialised spine surgeon in 1,741 (46.3%) cases; 6 cases had missing operator data. cerebrospinal fluid (CSF) leak occurred in 57 (4.16%) of neurosurgeon-operated cases; 5 (2.78%) orthopaedic-operated cases; 19 (4.06%) of other surgeon-operated cases; and 81 (4.65%) in specialised spine surgeon-operated cases. Using Chi-squared test, the significance of the variation in incidence of CSF leak between primary operator groups was not statistically significant (P = 0.1405).

CONCLUSION

From the data captured and analysed, the rate of durotomy ranged from 2.78 to 4.65% between operator groups with a mean rate of 4.3%. The primary operator credentials do not appear to significantly impact the rate of durotomy in spine surgery.

Strategies to improve clinical research in surgery through international collaboration

More than 235 million patients undergo surgery every year worldwide, but less than 1% are enrolled in surgical clinical trials--few of which are international collaborations. Several levels of action are needed to improve this situation. International research collaborations in surgery between developed and developing countries could encourage capacity building and quality improvement, and mutually enhance care for patients with surgical disorders. Low-income and middle-income countries increasingly report much the same range of surgical diseases as do high-income countries (eg, cancer, cardiovascular disease, and the surgical sequelae of metabolic syndrome); collaboration is therefore of mutual interest. Large multinational trials that cross cultures and levels of socioeconomic development might have faster results and wider applicability than do single-country trials. Surgeons educated in research methods, and aided by research networks and trial centres, are needed to foster these international collaborations. Barriers to collaboration could be overcome by adoption of global strategies for regulation, health insurance, ethical approval, and indemnity coverage for doctors.