Population characteristics in cervical cancer trials: search for external validity

A novel approach to assessing disparity in representativeness of clinical trial participants within a large midwestern healthcare system

Background

Representativeness in clinical trials (CT) serves as a metric of access to healthcare and reflects differences that may determine differential efficacy of medical interventions; thus, quantifying representativeness in CT participation is critical.

Methods

This retrospective, descriptive study utilized patient demographic data extracted from the largest Midwestern non-profit healthcare system. Using data between January 1, 2019 and December 31, 2021, a CT Participant Sample of 4,537 system patients who were active CT participants was compared to a CT Patient Population of 195,726 system patients receiving care by the PI of active CTs, which represented the target population. Chi-square goodness-of-fit tests were used to test differences in distributions of demographic variables between groups, indicating disparity in CT participation. Two metrics adapted from literature - participation incidence disparity (PID) and participation incidence ratio (PIR) - were calculated to quantify absolute and relative disparity in representativeness proportions, respectively. Descriptive approaches to assessing representativeness are also provided.

Results

Results showed significant differences by race/ethnicity (χ2 = 50.64; p < 0.0001), age categories (χ2 = 56.64; p < 0.0001), and insurance (χ2 = 41.29; p < 0.0001). PID and PIR metrics revealed reduced CT participation among non-White racial/ethnic groups and increased CT participation among White Non-Hispanic patients. Further, CT participants ≥80 or Worker's Compensation were underrepresented while those with Self-Pay insurance were overrepresented as CT participants.

Conclusions

Despite progress, continued efforts to not only enroll participants into CTs that are representative of the healthcare system and region, but also to better assess representativeness quantitatively are still needed.

Excluded versus included patients in a randomized controlled trial of infections caused by carbapenem-resistant Gram-negative bacteria: relevance to external validity

Background

Population external validity is the extent to which an experimental study results can be generalized from a specific sample to a defined population. In order to apply the results of a study, we should be able to assess its population external validity. We performed an investigator-initiated randomized controlled trial (RCT) (AIDA study), which compared colistin-meropenem combination therapy to colistin monotherapy in the treatment of patients infected with carbapenem-resistant Gram-negative bacteria. In order to examine the study’s population external validity and to substantiate the use of AIDA study results in clinical practice, we performed a concomitant observational trial.

Methods

The study was conducted between October 1st, 2013 and January 31st, 2017 (during the RCTs recruitment period) in Greece, Israel and Italy. Patients included in the observational arm of the study have fulfilled clinical and microbiological inclusion criteria but were excluded from the RCT due to receipt of colistin for > 96 h, refusal to participate, or prior inclusion in the RCT. Non-randomized cases were compared to randomized patients. The primary outcome was clinical failure at 14 days of infection onset.

Results

Analysis included 701 patients. Patients were infected mainly with Acinetobacter baumannii [78.2% (548/701)]. The most common reason for exclusion was refusal to participate [62% (183/295)]. Non-randomized and randomized patients were similar in most of the demographic and background parameters, though randomized patients showed minor differences towards a more severe infection. Combination therapy was less common in non-randomized patients [31.9% (53/166) vs. 51.2% (208/406), p = 0.000]. Randomized patients received longer treatment of colistin [13 days (IQR 10–16) vs. 8.5 days (IQR 0–15), p = 0.000]. Univariate analysis showed that non-randomized patients were more inclined to clinical failure on day 14 from infection onset [82% (242/295) vs. 75.5% (307/406), p = 0.042]. After adjusting for other variables, non-inclusion was not an independent risk factor for clinical failure at day 14.

Conclusion

The similarity between the observational arm and RCT patients has strengthened our confidence in the population external validity of the AIDA trial. Adding an observational arm to intervention studies can help increase the population external validity and improve implementation of study results in clinical practice.

Trial registration

The trial was registered with ClinicalTrials.gov, number NCT01732250 on November 22, 2012.

Assessing the population representativeness of colorectal cancer treatment clinical trials

The generalizability (external validity) of clinical trials has long been a concern for both clinical research community as well as the general public. Results of trials that do not represent the target population may not be applicable to the broader patient population. In this study, we used a previously published metric Generalizability Index for Study Traits (GIST) to assess the population representativeness of colorectal cancer (CRC) treatment trials. Our analysis showed that the quantitative eligibility criteria of CRC trials are in general not restrictive. However, the qualitative eligibility criteria in these trials are with moderate or strict restrictions, which may impact their population representativeness of the real-world patient population.

Generalizing results of randomized trials to clinical practice: reliability and cautions

BackgroundWell designed randomized controlled trials provide reliable evidence of treatment effects, but there is no consensus on how best to apply these results to clinical practice. The main concerns are that populations enrolled in trials are more selected than those treated in a clinical setting, and whether the treatment effects observed in trials will also be observed in clinical practice.

MethodsAn informal literature review was undertaken to find studies analysing the issue of generalizing trial results (external validity) to clinical practice.

ResultsMost of the studies focused on differences in patients characteristics (age, gender, severity of disease, concomitant treatments and so on) between the clinical trial population and a ‘real world’ clinical population. None provided good evidence of a reduction in the treatment effect in the trial compared to what might happen in clinical practice for simple pharmacological treatments. However complex treatments like surgery or percutaneous interventional procedures, had a greater potential for variation. Extrapolating treatments to different health care settings from the trial can result in important variations in treatment effects.

ConclusionsComplex therapies need careful consideration before they can be applied routinely from trials into practice, and applying results from one health care environment to a different one should be carried out with caution. Generalizing results from well conducted trials to clinical practice can mostly be carried out with confidence, especially for simple therapies with good evidence of benefit.

A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results

Randomized controlled trials (RCTs) are conducted under idealized and rigorously controlled conditions that may compromise their external validity. A literature review was conducted of published English language articles that reported the findings of studies assessing external validity by a comparison of the patient sample included in RCTs reporting on pharmaceutical interventions with patients from everyday clinical practice. The review focused on publications in the fields of cardiology, mental health, and oncology. A range of databases were interrogated (MEDLINE; EMBASE; Science Citation Index; Cochrane Methodology Register). Double-abstract review and data extraction were performed as per protocol specifications. Out of 5,456 de-duplicated abstracts, 52 studies met the inclusion criteria (cardiology, n = 20; mental health, n = 17; oncology, n = 15). Studies either performed an analysis of the baseline characteristics (demographic, socioeconomic, and clinical parameters) of RCT-enrolled patients compared with a real-world population, or assessed the proportion of real-world patients who would have been eligible for RCT inclusion following the application of RCT inclusion/exclusion criteria. Many of the included studies concluded that RCT samples are highly selected and have a lower risk profile than real-world populations, with the frequent exclusion of elderly patients and patients with co-morbidities. Calculation of ineligibility rates in individual studies showed that a high proportion of the general disease population was often excluded from trials. The majority of studies (n = 37 [71.2 %]) explicitly concluded that RCT samples were not broadly representative of real-world patients and that this may limit the external validity of the RCT. Authors made a number of recommendations to improve external validity. Findings from this review indicate that there is a need to improve the external validity of RCTs such that physicians treating patients in real-world settings have the appropriate evidence on which to base their clinical decisions. This goal could be achieved by trial design modification to include a more representative patient sample and by supplementing RCT evidence with data generated from observational studies. In general, a thoughtful approach to clinical evidence generation is required in which the trade-offs between internal and external validity are considered in a holistic and balanced manner.

Survival of patients with chronic myelocytic leukemia: comparisons of estimates from clinical trial settings and population-based cancer registries

INTRODUCTION

The survival of patients with chronic myelocytic leukemia (CML) has improved during the past decades. However, there have been discrepancies between results reported from clinical trials and population-based studies. We aimed to elucidate the extent of these discrepancies.

METHODS

We examined the 5-year survival rate of patients in clinical trials of CML treatment and compared these results with the survival of patients in the general population using the Surveillance, Epidemiology, and End Results (SEER) database, correcting for differences in the age structure of the patient populations.

RESULTS

Twenty-nine trials were identified for data extraction. The survival rate calculated from SEER data was lower than the survival rate in clinical trials in the corresponding period, with differences of 2.1%-50.7%. Age-adapted survival was similar for four trials, but differences up to 35.8% were seen in most. Limitations of the study include the lack of information on chemotherapy in the SEER database and possible heterogeneity of cases.

DISCUSSION

The survival rate in clinical trials of CML treatment is higher than the survival rate of all patients with CML. We speculate that the difference may be a result of access to better medications, selection of healthier patients for trials, and the time necessary for adoption of new treatments. This finding underscores the need for population-based studies to give a more realistic idea of survival for patients with a given malignancy in the general population.

An evidence-based estimation of local control and survival benefit of radiotherapy for breast cancer

BACKGROUND AND PURPOSE

Survival benefits from radiotherapy for breast cancer described in randomised trials represent only those patients eligible for trials. We estimated the benefit of radiotherapy as an adjuvant treatment for the entire population of breast cancer patients if evidence-based treatment guidelines were followed.

MATERIALS AND METHODS

Evidences on 10-year local control and overall survival gain (radiotherapy vs no radiotherapy) were identified from review of literature. The data were incorporated into the optimal radiotherapy utilization tree that we previously reported for all categories of breast cancer patients and overall local control and survival benefits were estimated.

RESULTS

The gains in 10-year local control and overall survival from optimal treatment of all breast cancer patients were 11.1% (95% CI 10.8-11.2%) and 3.1% (95% CI 3.0-3.4%), respectively. The stage-based estimates in local control and survival benefit were: 8% and 0% for Ductal Carcinoma in situ (DCIS), 12% and 2% for stage I-II cancers and 13% and 20% for stage III cancers.

CONCLUSIONS

Our model was able to estimate the contribution of radiotherapy in breast cancer treatment if all patients were treated according to the recommended guidelines. These estimates could be used to benchmark population-based survival reports and to assess the cost-effectiveness of radiotherapy for breast cancer treatment.