Practical considerations of utilizing propensity score methods in clinical development using real-world and historical data

Association between chiropractic spinal manipulation and cauda equina syndrome in adults with low back pain: Retrospective cohort study of US academic health centers

BACKGROUND

Cauda equina syndrome (CES) is a lumbosacral surgical emergency that has been associated with chiropractic spinal manipulation (CSM) in case reports. However, identifying if there is a potential causal effect is complicated by the heightened incidence of CES among those with low back pain (LBP). The study hypothesis was that there would be no increase in the risk of CES in adults with LBP following CSM compared to a propensity-matched cohort following physical therapy (PT) evaluation without spinal manipulation over a three-month follow-up period.

METHODS

A query of a United States network (TriNetX, Inc.) was conducted, searching health records of more than 107 million patients attending academic health centers, yielding data ranging from 20 years prior to the search date (July 30, 2023). Patients aged 18 or older with LBP were included, excluding those with pre-existing CES, incontinence, or serious pathology that may cause CES. Patients were divided into two cohorts: (1) LBP patients receiving CSM or (2) LBP patients receiving PT evaluation without spinal manipulation. Propensity score matching controlled for confounding variables associated with CES.

RESULTS

67,220 patients per cohort (mean age 51 years) remained after propensity matching. CES incidence was 0.07% (95% confidence intervals [CI]: 0.05-0.09%) in the CSM cohort compared to 0.11% (95% CI: 0.09-0.14%) in the PT evaluation cohort, yielding a risk ratio and 95% CI of 0.60 (0.42-0.86; p = .0052). Both cohorts showed a higher rate of CES during the first two weeks of follow-up.

CONCLUSIONS

These findings suggest that CSM is not a risk factor for CES. Considering prior epidemiologic evidence, patients with LBP may have an elevated risk of CES independent of treatment. These findings warrant further corroboration. In the meantime, clinicians should be vigilant to identify LBP patients with CES and promptly refer them for surgical evaluation.

Depressive symptoms among people under COVID-19 quarantine or self-isolation in Korea: a propensity score matching analysis

Introduction

This study aims to determine the effect of COVID-19-related hospital isolation or self-isolation on depression using the propensity score matching method.

Methods

Data on 217,734 participants were divided into groups based on whether or not they underwent quarantine for their COVID-19 diagnosis. COVID-19-related anxiety, depressive symptoms, subjective health status, and perceived stress were evaluated.

Results

Based on the calculated propensity score, we matched the quarantined group and non-quarantined group using 1:2 matching with nearest neighbor matching and a caliper width of 0.1. Within the quarantined group, 16.4% of participants experienced significant depressive symptoms, which was significantly higher than that of the non-quarantined group. However, there was no significant difference between the two groups in COVID-19-related anxiety, self-rated health status, and perceived stress. In our multiple logistic regression analysis with related variables corrected, the quarantined group was 1.298 times more likely to have depressive symptoms than the non-quarantined group (95% CI = 1.030-1.634).

Conclusion

Our study confirmed that COVID-19 quarantine is associated with depressive symptoms. These results indicate that healthcare policymakers and healthcare professionals must consider the negative mental and physical effects of quarantine when determining quarantine measures during an infectious disease disaster such as the COVID-19 pandemic.

BASIC: A Bayesian adaptive synthetic-control design for phase II clinical trials

BACKGROUND

Randomized controlled trials are considered the gold standard for evaluating experimental treatments but often require large sample sizes. Single-arm trials require smaller sample sizes but are subject to bias when using historical control data for comparative inferences. This article presents a Bayesian adaptive synthetic-control design that exploits historical control data to create a hybrid of a single-arm trial and a randomized controlled trial.

METHODS

The Bayesian adaptive synthetic control design has two stages. In stage 1, a prespecified number of patients are enrolled in a single arm given the experimental treatment. Based on the stage 1 data, applying propensity score matching and Bayesian posterior prediction methods, the usefulness of the historical control data for identifying a pseudo sample of matched synthetic-control patients for making comparative inferences is evaluated. If a sufficient number of synthetic controls can be identified, the single-arm trial is continued. If not, the trial is switched to a randomized controlled trial. The performance of The Bayesian adaptive synthetic control design is evaluated by computer simulation.

RESULTS

The Bayesian adaptive synthetic control design achieves power and unbiasedness similar to a randomized controlled trial but on average requires a much smaller sample size, provided that the historical control data patients are sufficiently comparable to the trial patients so that a good number of matched controls can be identified in the historical control data. Compared to a single-arm trial, The Bayesian adaptive synthetic control design yields much higher power and much smaller bias.

CONCLUSION

The Bayesian adaptive synthetic-control design provides a useful tool for exploiting historical control data to improve the efficiency of single-arm phase II clinical trials, while addressing the problem of bias when comparing trial results to historical control data. The proposed design achieves power similar to a randomized controlled trial but may require a substantially smaller sample size.

Incidence and risk outcomes of second primary malignancy of patients with post-operative colorectal cancer

BACKGROUND AND AIMS

This study aimed to investigate the incidence and the risk factors of incidence for second primary malignancies (SPMs) onset among survivors diagnosed with colorectal cancer (CRC).

METHODS

A large population-based cohort study was performed. Data of patients diagnosed with CRC was identified and extracted from 8 cancer registries of Surveillance, Epidemiology, and End Results database from January 1990 to December 2017. The outcome of interest was percentage and common sites of SPM onset after primary CRC diagnosis. The cumulative incidence and standardize incidence rates (SIRs) were also reported. Afterwards, we estimated sub-distribution hazards ratios (SHRs) and relative risks (RRs) for SPM occurrence using multivariable competing-risk and Poisson regression models, respectively.

RESULTS

A total of 152,402 patients with CRC were included to analyze. Overall, 23,816 patients of all CRC survivors (15.6%) were reported SPM occurrence. The highest proportion of SPMs development after primary CRC diagnosis was second CRC, followed by lung and bronchus cancer among all survivors. Also, CRC survivors were more susceptible to develop second gastrointestinal cancers (GICs). Besides, pelvic cancers were analyzed with a relative high proportion among patients who received RT in comparison to those without RT. The cumulative incidence of all SPMs onset was 22.16% (95% CI: 21.82-22.49%) after near 30-year follow-up. Several factors including older age, male, married status, and localized stage of CRC were related to the high risk of SPMs onset. In treatment-specific analyses, RT was related to a higher cumulative incidence of SPMs occurrence (all SPMs: 14.08% vs. 8.72%; GICs: 2.67% vs. 2.04%; CRC: 1.01% vs. 1.57%; all p < 0.01). Furthermore, the increased risk of SPMs onset was found among patients who received RT than patients within the NRT group (SHR: 1.50, 95% CI: 1.32-1.71), p < 0.01; RR: 1.61, 95% CI: 1.45-1.79, p < 0.01).

CONCLUSION

The present study described the incidence pattern of SPM among CRC survivors and identified the risk factors of the SPM onset. RT treatment for patients diagnosed with CRC may increase the risk of SPMs occurrence. The findings suggest the need for long-term follow-up surveillance for these patients.

Association between chiropractic spinal manipulative therapy and benzodiazepine prescription in patients with radicular low back pain: a retrospective cohort study using real-world data from the USA

OBJECTIVES

Although chiropractic spinal manipulative therapy (CSMT) and prescription benzodiazepines are common treatments for radicular low back pain (rLBP), no research has examined the relationship between these interventions. We hypothesise that utilisation of CSMT for newly diagnosed rLBP is associated with reduced odds of benzodiazepine prescription through 12 months' follow-up.

DESIGN

Retrospective cohort study.

SETTING

National, multicentre 73-million-patient electronic health records-based network (TriNetX) in the USA, queried on 30 July 2021, yielding data from 2003 to the date of query.

PARTICIPANTS

Adults aged 18-49 with an index diagnosis of rLBP were included. Serious aetiologies of low back pain, structural deformities, alternative neurological lesions and absolute benzodiazepine contraindications were excluded. Patients were assigned to cohorts according to CSMT receipt or absence. Propensity score matching was used to control for covariates that could influence the likelihood of benzodiazepine utilisation.

OUTCOME MEASURES

The number, percentage and OR of patients receiving a benzodiazepine prescription over 3, 6 and 12 months' follow-up prematching and postmatching.

RESULTS

After matching, there were 9206 patients (mean (SD) age, 37.6 (8.3) years, 54% male) per cohort. Odds of receiving a benzodiazepine prescription were significantly lower in the CSMT cohort over all follow-up windows prematching and postmatching (p<0.0001). After matching, the OR (95% CI) of benzodiazepine prescription at 3 months was 0.56 (0.50 to 0.64), at 6 months 0.61 (0.55 to 0.68) and 12 months 0.67 (0.62 to 0.74). Sensitivity analysis suggested a patient preference to avoid prescription medications did not explain the study findings.

CONCLUSIONS

These findings suggest that receiving CSMT for newly diagnosed rLBP is associated with reduced odds of receiving a benzodiazepine prescription during follow-up. These results provide real-world evidence of practice guideline-concordance among patients entering this care pathway. Benzodiazepine prescription for rLBP should be further examined in a randomised trial including patients receiving chiropractic or usual medical care, to reduce residual confounding.

Statistical methods of indirect comparison with real-world data for survival endpoint under non-proportional hazards

In clinical studies that utilize real-world data, time-to-event outcomes are often germane to scientific questions of interest. Two main obstacles are the presence of non-proportional hazards and confounding bias. Existing methods that could adjust for NPH or confounding bias, but no previous work delineated the complexity of simultaneous adjustments for both. In this paper, a propensity score stratified MaxCombo and weighted Cox model is proposed. This model can adjust for confounding bias and NPH and can be pre-specified when NPH pattern is unknown in advance. The method has robust performance as demonstrated in simulation studies and in a case study.

Optimal sample size determination for single-arm trials in pediatric and rare populations with Bayesian borrowing

In many therapeutic areas with unmet medical needs, such as pediatric oncology and rare diseases, one of the deterrent factors for clinical trial interpretability is the limited sample size with less-than-ideal operating characteristics. Single arm is usually the only viable design due to feasibility and ethical concerns. For the trial results to be more interpretable and conclusive, the evaluation of operating characteristics, such as type I error rate and power, and the appropriate utilization of prior information for study design, shall be prespecified and fully investigated during the trial planning phase. So far, very few existing literature addressed optimal sample size determination issues for the planning of pediatric and rare population trials, with majority of research focusing on analysis perspective with focus on Bayesian borrowing. In practice, when a single-arm trial is designed for rare population, it is not uncommon that the only information available is from an earlier trial and/or a few clinical publications based on observational studies, often constituting mixed or uncertain conclusions. In light of this, an optimal Bayesian sample size determination method for single-arm trial with binary or continuous endpoint is proposed, where conflicting prior beliefs can be readily incorporated. Prior effective sample size can be calculated to assess the robustness as well as the prior information borrowed. Moreover, due to the lack of closed-form posterior distributions in general, an alternative approach for calculating Bayesian power is described. Simulation studies are provided to demonstrate the utility of the proposed methods. In addition, a case study in pediatric patients with leukemia is included to illustrate the proposed method with the existing approaches.

Optimized Weighted Nearest Neighbours Matching Algorithm for Control Group Selection

An essential criterion for the proper implementation of case-control studies is selecting appropriate case and control groups. In this article, a new simulated annealing-based control group selection method is proposed, which solves the problem of selecting individuals in the control group as a distance optimization task. The proposed algorithm pairs the individuals in the n-dimensional feature space by minimizing the weighted distances between them. The weights of the dimensions are based on the odds ratios calculated from the logistic regression model fitted on the variables describing the probability of membership of the treated group. For finding the optimal pairing of the individuals, simulated annealing is utilized. The effectiveness of the newly proposed Weighted Nearest Neighbours Control Group Selection with Simulated Annealing (WNNSA) algorithm is presented by two Monte Carlo studies. Results show that the WNNSA method can outperform the widely applied greedy propensity score matching method in feature spaces where only a few covariates characterize individuals and the covariates can only take a few values.

Use of Propensity Scoring and Its Application to Real-World Data: Advantages, Disadvantages, and Methodological Objectives Explained to Researchers Without Using Mathematical Equations

Real-time data collection of patient health status and medications is sped up with modern electronic devices and technologies. As real-world data provide enormous research opportunities, propensity score (PS) methods have been getting attention due to their theoretical grounds in a nonrandomized study setting. In contrast to randomized clinical trials, observational clinical data obtained from a real-world database may not have balanced distributions of patient characteristics between treatment and control groups at the beginning of the respective study. These imbalanced distributions may cause a bias in an estimated treatment effect, which needs to be eliminated. Propensity scoring is one class of statistical methods to address the imbalance issue of real-world data sets. This article provides basic concepts and assesses advantages, disadvantages, and methodological objectives of propensity scoring. Targeting clinical pharmacology researchers with limited statistical background, 5 representative methods are reviewed and visualized: matching, stratification, covariate modeling, inverse probability of treatment weighting, and doubly robust methods. Examples of applications of PS methods were selected from the literature of outcomes research and drug development, nephrology, and pediatrics. Opportunities of applications related to these examples are described. Furthermore, potential future applications of PS methods in clinical pharmacology are discussed. The 21st Century Cures Act signed in 2016 encourages scientists to find opportunities to apply propensity scoring to real-world data. This article underscores that scientists need to justify their choice of statistical methods, whether a PS method or an alternative method, based on their clinical study design, statistical assumptions, and research objectives.

Propensity-score-based meta-analytic predictive prior for incorporating real-world and historical data

As the availability of real-world data sources (eg, EHRs, claims data, registries) and historical data has rapidly surged in recent years, there is an increasing interest and need from investigators and health authorities to leverage all available information to reduce patient burden and accelerate both drug development and regulatory decision making. Bayesian meta-analytic approaches are a popular historical borrowing method that has been developed to leverage such data using robust hierarchical models. The model structure accounts for various degrees of between-trial heterogeneity, resulting in adaptively discounting the external information in the case of data conflict. In this article, we propose to integrate the propensity score method and Bayesian meta-analytic-predictive (MAP) prior to leverage external real-world and historical data. The propensity score methodology is applied to select a subset of patients from external data that are similar to those in the current study with regards to key baseline covariates and to stratify the selected patients together with those in the current study into more homogeneous strata. The MAP prior approach is used to obtain stratum-specific MAP prior and derive the overall propensity score integrated meta-analytic predictive (PS-MAP) prior. Additionally, we allow for tuning the prior effective sample size for the proposed PS-MAP prior, which quantifies the amount of information borrowed from external data. We evaluate the performance of the proposed PS-MAP prior by comparing it to the existing propensity score-integrated power prior approach in a simulation study and illustrate its implementation with an example of a single-arm phase II trial.

Optimizing clinical research procedures in public health emergencies

Public Health Emergencies of International Concern, such as the coronavirus disease 2019 pandemic, have a devastating impact on an individual and societal level, and there is an urgent need to learn, understand and bridge the therapeutic gap at a time of extreme stress on the patient, health care systems and staff. Well-designed, controlled clinical trials play a crucial role in the discovery of novel diagnostic and management strategies; however, these catastrophic circumstances pose unique challenges in initiating research studies at institutional, national, and international levels, highlighting the importance of a coordinated, collaborative approach. This review discusses key elements necessary to consider for developing clinical trials within a Public Health Emergency setting.