Using Bayesian hierarchical modeling for performance evaluation of clinical trial accrual for a cancer center

Introduction

Slow patient accrual in cancer clinical trials is always a concern. In 2021, the University of Kansas Comprehensive Cancer Center (KUCC), an NCI-designated comprehensive cancer center, implemented the Curated Cancer Clinical Outcomes Database (C3OD) to perform trial feasibility analyses using real-time electronic medical record data. In this study, we proposed a Bayesian hierarchical model to evaluate annual cancer clinical trial accrual performance.

Methods

The Bayesian hierarchical model uses Poisson models to describe the accrual performance of individual cancer clinical trials and a hierarchical component to describe the variation in performance across studies. Additionally, this model evaluates the impacts of the C3OD and the COVID-19 pandemic using posterior probabilities across evaluation years. The performance metric is the ratio of the observed accrual rate to the target accrual rate.

Results

Posterior medians of the annual accrual performance at the KUCC from 2018 to 2023 are 0.233, 0.246, 0.197, 0.150, 0.254, and 0.340. The COVID-19 pandemic partly explains the drop in performance in 2020 and 2021. The posterior probability that annual accrual performance is better with C3OD in 2023 than pre-pandemic (2019) is 0.935.

Conclusions

This study comprehensively evaluates the annual performance of clinical trial accrual at the KUCC, revealing a negative impact of COVID-19 and an ongoing positive impact of C3OD implementation. Two sensitivity analyses further validate the robustness of our model. Evaluating annual accrual performance across clinical trials is essential for a cancer center. The performance evaluation tools described in this paper are highly recommended for monitoring clinical trial accrual.

Protocol of the Comparison of Intravesical Therapy and Surgery as Treatment Options (CISTO) study: a pragmatic, prospective multicenter observational cohort study of recurrent high-grade non-muscle invasive bladder cancer

BACKGROUND

Bladder cancer poses a significant public health burden, with high recurrence and progression rates in patients with non-muscle-invasive bladder cancer (NMIBC). Current treatment options include bladder-sparing therapies (BST) and radical cystectomy, both with associated risks and benefits. However, evidence supporting optimal management decisions for patients with recurrent high-grade NMIBC remains limited, leading to uncertainty for patients and clinicians. The CISTO (Comparison of Intravesical Therapy and Surgery as Treatment Options) Study aims to address this critical knowledge gap by comparing outcomes between patients undergoing BST and radical cystectomy.

METHODS

The CISTO Study is a pragmatic, prospective observational cohort trial across 36 academic and community urology practices in the US. The study will enroll 572 patients with a diagnosis of recurrent high-grade NMIBC who select management with either BST or radical cystectomy. The primary outcome is health-related quality of life (QOL) at 12 months as measured with the EORTC-QLQ-C30. Secondary outcomes include bladder cancer-specific QOL, progression-free survival, cancer-specific survival, and financial toxicity. The study will also assess patient preferences for treatment outcomes. Statistical analyses will employ targeted maximum likelihood estimation (TMLE) to address treatment selection bias and confounding by indication.

DISCUSSION

The CISTO Study is powered to detect clinically important differences in QOL and cancer-specific survival between the two treatment approaches. By including a diverse patient population, the study also aims to assess outcomes across the following patient characteristics: age, gender, race, burden of comorbid health conditions, cancer severity, caregiver status, social determinants of health, and rurality. Treatment outcomes may also vary by patient preferences, health literacy, and baseline QOL. The CISTO Study will fill a crucial evidence gap in the management of recurrent high-grade NMIBC, providing evidence-based guidance for patients and clinicians in choosing between BST and radical cystectomy. The CISTO study will provide an evidence-based approach to identifying the right treatment for the right patient at the right time in the challenging clinical setting of recurrent high-grade NMIBC.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03933826. Registered on May 1, 2019.

Exploring implementation outcomes in the clinical trial context: a qualitative study of physician trial stakeholders

INTRODUCTION

Cancer clinical trials can be considered evidence-based interventions with substantial benefits, but suffer from poor implementation leading to low enrollment and frequent failure. Applying implementation science approaches such as outcomes frameworks to the trial context could aid in contextualizing and evaluating trial improvement strategies. However, the acceptability and appropriateness of these adapted outcomes to trial stakeholders are unclear. For these reasons, we interviewed cancer clinical trial physician stakeholders to explore how they perceive and address clinical trial implementation outcomes.

METHODS

We purposively selected 15 cancer clinical trial physician stakeholders from our institution representing different specialties, trial roles, and trial sponsor types. We performed semi-structured interviews to explore a previous adaptation of Proctor's Implementation Outcomes Framework to the clinical trial context. Emergent themes from each outcome were developed.

RESULTS

The implementation outcomes were well understood and applicable (i.e., appropriate and acceptable) to clinical trial stakeholders. We describe cancer clinical trial physician stakeholder understanding of these outcomes and current application of these concepts. Trial feasibility and implementation cost were felt to be most critical to trial design and implementation. Trial penetration was most difficult to measure, primarily due to eligible patient identification. In general, we found that formal methods for trial improvement and trial implementation evaluation were poorly developed. Cancer clinical trial physician stakeholders referred to some design and implementation techniques used to improve trials, but these were infrequently formally evaluated or theory-based.

CONCLUSION

Implementation outcomes adapted to the trial context were acceptable and appropriate to cancer clinical trial physician stakeholders. Use of these outcomes could facilitate the evaluation and design of clinical trial improvement interventions. Additionally, these outcomes highlight potential areas for the development of new tools, for example informatics solutions, to improve the evaluation and implementation of clinical trials.

Can we predict trial failure among older adult-specific clinical trials using trial-level factors?

INTRODUCTION

Conducting older adult-specific clinical trials can help overcome the lack of clinical evidence for older adults due to their underrepresentation in clinical trials. Understanding factors contributing to the successful completion of such trials can help trial sponsors and researchers prioritize studies and optimize study design. We aimed to develop a model that predicts trial failure among older adult-specific cancer clinical trials using trial-level factors.

MATERIALS AND METHODS

We identified phase 2-4 interventional cancer clinical trials that ended between 2008 and 2019 and had the minimum age limit of 60 years old or older using Aggregate Analysis of ClinicalTrials.gov data. We defined trial failure as closed early for reasons other than interim results or toxicity or completed with a sample of <85% of the targeted size. Candidate trial-level predictors were identified from a literature review. We evaluated eight types of machine learning algorithms to find the best model. Model fitting and testing were performed using 5-fold nested cross-validation. We evaluated the model performance using the area under receiver operating characteristic curve (AUROC).

RESULTS

Of 209 older adult-specific clinical trials, 87 were failed trials per the definition of trial failure. The model with the highest AUROC in the validation set was the least absolute shrinkage and selection operator (AUROC in the test set = 0.70; 95% confidence interval [CI]: 0.53, 0.86). Trial-level factors included in the best model were the study sponsor, the number of participating centers, the number of modalities, the level of restriction on performance score, study location, the number of arms, life expectancy restriction, and the number of target size. Among these factors, the number of centers (odds ratio [OR] = 0.83, 95% CI: 0.71, 0.94), study being in non-US only vs. US only (OR = 0.32, 95% CI: 0.12, 0.82), and life expectancy restriction (OR = 2.17, 95% CI: 1.04, 4.73) were significantly associated with the trial failure.

DISCUSSION

We identified trial-level factors predictive of trial failure among older adult-specific clinical trials and developed a prediction model that can help estimate the risk of failure before a study is conducted. The study findings could aid in the design and prioritization of future older adult-specific clinical trials.

Assessing clinical trial failure risk factors and reasons in gastric cancer

BACKGROUND

Gastric cancer is one of the leading cancer-related death causes. Enormous efforts have been focused on this field in these years. However, clinical trial failure is becoming a massive obstacle for researchers to apply their research results for clinical use. This study aimed to analyze the reasons behind clinical failures and identify potential risk factors of clinical trial failures.

METHODS

On December, 1, 2021, we queried ClinicalTrials.gov for gastric cancer listed in phase II/III. We included trials specifying their interests in "stomach cancer", "Stomach Neoplasms", "Gastric Cancer", "Gastric Neoplasms", "Gastric Carcinoma", "Stomach Carcinoma", "Gastroesophageal Junction Cancer". Exclude criteria are: (1) Trials that start prior to 01/01/2007 and start after 12/01/2020; (2) Trials with "not yet recruiting", "suspended", "withdrawn", or "unknown" status; (3) Trials do not provide an anticipated accrual number or a start date.

RESULTS

A total of 567 trials are included. 10.2% of these trials are failed. 16 (2.82%) were terminated for good reasons, and 42 (7.41%) were terminated for bad reasons. Multi-centre (P-value = 0.088) and anticipated accrual (P-value = 0.099) are potential risk factors for clinical failures in the simple logistic regression model. After considering the interaction between multi-centre and anticipated accrual, the odds ratio of anticipated accrual is 0.60 (P-value = 0.009) in single centre trials. In multi-centre trials, the odds ratio of anticipated accrual is 0.72 (P-value = 0.025). The primary reason for gastric cancer trial terminations is recruitment failure.

CONCLUSION

The rate that trials terminated in gastric cancer has decreased compared to previous studies. Comparing to other types of oncology trials, poor accrual continues to be the predominant reason, followed by business or sponsor reasons. Single-center trials with smaller anticipated accrual number are more likely to be terminated which may resulted by limited resources invested to the trial. Single-center design exacerbated the difficulty of participant recruitment. Future studies need to continue tracking the rate of trial termination across oncology and whether the reasons behind them have changed.

Applying implementation frameworks to the clinical trial context

BACKGROUND

Clinical trials advance science, benefit society, and provide optimal care to individuals with some conditions, such as cancer. However, clinical trials often fail to reach their endpoints, and low participant enrollment remains a critical problem with trial conduct. In these ways, clinical trials can be considered beneficial evidence-based practices suffering from poor implementation. Prior approaches to improving trials have had difficulties with reproducibility and limited impact, perhaps due to the lack of an underlying trial improvement framework. For these reasons, we propose adapting implementation science frameworks to the clinical trial context to improve the implementation of clinical trials.

MAIN TEXT

We adapted an outcomes framework (Proctor's Implementation Outcomes Framework) and a determinants framework (the Consolidated Framework for Implementation Research) to the trial context. We linked these frameworks to ERIC-based improvement strategies and present an inferential process model for identifying and selecting trial improvement strategies based on the Implementation Research Logic Model. We describe example applications of the framework components to the trial context and present a worked example of our model applied to a trial with poor enrollment. We then consider the implications of this approach on improving existing trials, the design of future trials, and assessing trial improvement interventions. Additionally, we consider the use of implementation science in the clinical trial context, and how clinical trials can be "test cases" for implementation research.

CONCLUSIONS

Clinical trials can be considered beneficial evidence-based interventions suffering from poor implementation. Adapting implementation science approaches to the clinical trial context can provide frameworks for contextual assessment, outcome measurement, targeted interventions, and a shared vocabulary for clinical trial improvement. Additionally, exploring implementation frameworks in the trial context can advance the science of implementation through both "test cases" and providing fertile ground for implementation intervention design and testing.

Envisioning clinical trials as complex interventions

Clinical trials are critical components of modern health care and infrastructure. Trials benefit society through scientific advancement and individual patients through trial participation. In fact, billions of dollars are spent annually in support of these benefits. Despite the massive investments, clinical trials often fail to accomplish their primary aims and trial enrollment rates remain low. Prior efforts to improve trial conduct and enrollment have had limited success, perhaps due to oversimplification of the complex, multilevel nature of trials. For these reasons, the authors propose applying implementation science to the clinical trials context. In this commentary, the authors posit clinical trials as complex, multilevel evidence-based interventions with significant societal and individual benefits yet with persistent gaps in implementation. An application of implementation science concepts to the clinical trials context as means to build common vocabulary and establish a platform for applying implementation science and practice to improve clinical trial conduct is introduced. Applying implementation science to the clinical trials context can augment improvement efforts and build capacity for better and more efficient evidence-based care for all patients and trial stakeholders throughout the clinical trials enterprise.

Evolving a national clinical trials learning health system

Clinical trials generate key evidence to inform decision making, and also benefit participants directly. However, clinical trials frequently fail, often struggle to enroll participants, and are expensive. Part of the problem with trial conduct may be the disconnected nature of clinical trials, preventing rapid data sharing, generation of insights and targeted improvement interventions, and identification of knowledge gaps. In other areas of healthcare, a learning health system (LHS) has been proposed as a model to facilitate continuous learning and improvement. We propose that an LHS approach could greatly benefit clinical trials, allowing for continuous improvements to trial conduct and efficiency. A robust trial data sharing system, continuous analysis of trial enrollment and other success metrics, and development of targeted trial improvement interventions are potentially key components of a Trials LHS reflecting the learning cycle and allowing for continuous trial improvement. Through the development and use of a Trials LHS, clinical trials could be treated as a system, producing benefits to patients, advancing care, and decreasing costs for stakeholders.

Prostate cancer clinical trial completion: The role of geography

BACKGROUND/AIMS

One in five cancer clinical trials fails with another third failing to meet enrollment goals. Prior efforts to improve enrollment focus on patient facing interventions, but geographic factors such as regional cancer incidence may doom trials before they even begin. For these reasons, we examined associations of regional prostate cancer incidence with trial termination, and identified scientifically-underserved areas where future trials might thrive.

METHODS

We merged US phase 2-3 prostate cancer clinical trial data from ClinicalTrials.gov with prostate cancer incidence data from statecancerprofiles.cancer.gov. We matched trial information from 293 closed and 560 active trials with incidence data for 2947 counties. Using multivariable logistic regression, we identified associations with trial termination. We identified 'scientifically-underserved' counties with the highest cancer incidence quintile (>61 annual cases) but lowest active trials quintile (0 or 1 trial).

RESULTS

Of 293 closed trials, one in three was terminated (n = 96, 32.8%). On multivariable analysis, only lower regional prostate cancer incidence was associated with higher likelihood of premature trial termination (OR 0.98, 95% CI [0.96-0.99] for every 100 cases, p = 0.03). We identified 188 counties with >61 annual prostate cancer cases but 0 or 1 active trials, indicating potential scientifically-underserved areas.

CONCLUSIONS

In this novel study, we found prostate cancer trials in areas with low prostate cancer incidence were more likely to fail. We also identified scientifically-underserved areas where trials might thrive. Our findings provide a more nuanced understanding of clinical trial feasibility and upstream opportunities for improvement.

A review of pragmatic trials found a high degree of diversity in design and scope, deficiencies in reporting and trial registry data, and poor indexing

OBJECTIVE

We established a large database of trials to serve as a resource for future methodological and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic trials including research areas, identification as pragmatic, quality of trial registry data and enrolment.

STUDY DESIGN AND SETTING

Trials were identified by a validated search filter and included if a primary report of a health-related randomized trial published January 2014-April 2019. Data were collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text.

RESULTS

4337 eligible trials were identified from 13,065 records, of which 1988 were registered in ClinicalTrials.gov. Research areas were diverse, with the most common being general and internal medicine; public, environmental and occupational health; and health care sciences and services. The term "pragmatic" was seldom used in titles or abstracts. Several domains in ClinicalTrials.gov had questionable data quality. We estimated that one-fifth of trials under-accrued by at least 15%.

CONCLUSION

There is a need to improve reporting of pragmatic trials and quality of trial registry data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined recruitment approaches. The diversity of pragmatic trials should be reflected in future ethical analyses.

Estimating the rate and reasons of clinical trial failure in urologic oncology

OBJECTIVES

Clinical trials are pillars of modern clinical evidence generation. However, the clinical trial enterprise can be inefficient, and trials often fail before their planned endpoint is reached. We sought to estimate how often urologic oncology trials fail, why trials fail, and associations with trial failure.

METHODS

We queried phase 2/3 urologic clinical trial data from ClinicalTrials.gov registered between 2007 and 2019, with status marked as active, completed, or terminated. We extracted relevant trial data, including anticipated and actual accrual, from trial records and ClinicalTrials.gov archives. We manually coded reasons given in the "why stopped" free text field for trial failure into categories (poor accrual, interim results, toxicity/adverse events, study agent unavailable, canceled by the sponsor, inadequate budget, logistics, trial no longer needed, principal investigator left, no reason given, or other). We considered trials terminated for safety or efficacy to be completed trials. Trials marked as terminated for other reasons were considered failed trials. We then estimated the rate of trial failure using competing risks methods. Finally, we assessed associations with trial failure using a Cox proportional hazards model.

RESULTS

A total of 1,869 urologic oncology trials were included. Of these, 225 (12.0%) failed, and 51 (2.7%) were terminated for "good" reasons (e.g., toxicity, efficacy). Of the 225 failed trials, 122 (54%) failed due to poor accrual. Failed trials had a lower anticipated accrual than successfully completed trials (55 vs. 63 patients, P<0.001). A total of 6,832 patients were actually accrued to failed trials. The 10-year estimated risk of trial failure was 17% (95% CI 15%-22%). Single center trials, phase 3 trials, drug trials, and trials with exclusively USA sites were more likely to fail.

CONCLUSION

We estimate that 17%, or roughly 1 in 6, of urologic oncology trials fail, most frequently for poor accrual. Further investigations are needed into systemic, trial, and site-specific factors that may impact accrual and successful trial completion.