A comparison of confounding adjustment methods for assessment of asthma controller medication effectiveness

A methodological review of the high-dimensional propensity score in comparative-effectiveness and safety-of-interventions research finds incomplete reporting relative to algorithm development and robustness

OBJECTIVES

The use of secondary databases has become popular for evaluating the effectiveness and safety of interventions in real-life settings. However, the absence of important confounders in these databases is challenging. To address this issue, the high-dimensional propensity score (hdPS) algorithm was developed in 2009. This algorithm uses proxy variables for mitigating confounding by combining information available across several healthcare dimensions. This study assessed the methodology and reporting of the hdPS in comparative effectiveness and safety research.

STUDY DESIGN AND SETTING

In this methodological review, we searched PubMed and Google Scholar from July 2009 to May 2022 for studies that used the hdPS for evaluating the effectiveness or safety of healthcare interventions. Two reviewers independently extracted study characteristics and assessed how the hdPS was applied and reported. Risk of bias was evaluated with the Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I) tool.

RESULTS

In total, 136 studies met the inclusion criteria; the median publication year was 2018 (Q1-Q3 2016-2020). The studies included 192 datasets, mostly North American databases (n = 132, 69%). The hdPS was used in primary analysis in 120 studies (88%). Dimensions were defined in 101 studies (74%), with a median of 5 (Q1-Q3 4-6) dimensions included. A median of 500 (Q1-Q3 200-500) empirically identified covariates were selected. Regarding hdPS reporting, only 11 studies (8%) reported all recommended items. Most studies (n = 81, 60%) had a moderate overall risk of bias.

CONCLUSION

There is room for improvement in the reporting of hdPS studies, especially regarding the transparency of methodological choices that underpin the construction of the hdPS.

Evaluating the Performance of High-Dimensional Propensity Scores Compared with Standard Propensity Scores for Comparing Antihypertensive Therapies in the CPRD GOLD Database

INTRODUCTION

Propensity score (PS) matching is widely used in medical record studies to create balanced treatment groups, but relies on prior knowledge of confounding factors. High-dimensional PS (hdPS) is a semi-automated algorithm that selects variables with the highest potential for confounding from medical databases. The objective of this study was to evaluate performance of hdPS and PS when used to compare antihypertensive therapies in the UK clinical practice research datalink (CPRD) GOLD database.

METHODS

Patients initiating antihypertensive treatment with either monotherapy or bitherapy were extracted from the CPRD GOLD database. Simulated datasets were generated using plasmode simulations with a marginal hazard ratio (HRm) of 1.29 for bitherapy versus monotherapy for reaching blood pressure control at 3 months. Either 16 or 36 known covariates were forced into the PS and hdPS models, and 200 additional variables were automatically selected for hdPS. Sensitivity analyses were conducted to assess the impact of removing known confounders from the database on hdPS performance.

RESULTS

With 36 known covariates, the estimated HRm (RMSE) was 1.31 (0.05) for hdPS and 1.30 (0.04) for PS matching; the crude HR was 0.68 (0.61). Using 16 known covariates, the estimated HRm (RMSE) was 1.23 (0.10) and 1.09 (0.20) for hdPS and PS, respectively. Performance of hdPS was not compromised when known confounders were removed from the database.

RESULTS ON REAL DATA

With 49 investigator-selected covariates, the HR was 1.18 (95% CI 1.10; 1.26) for PS and 1.33 (95% CI 1.22; 1.46) for hdPS. Both methods yielded the same conclusion, suggesting superiority of bitherapy over monotherapy for time to blood pressure control.

CONCLUSION

HdPS can identify proxies for missing confounders, thereby having an advantage over PS in case of unobserved covariates. Both PS and hdPS showed superiority of bitherapy over monotherapy for reaching blood pressure control.

High-dimensional propensity scores improved the control of indication bias in surgical comparative effectiveness studies

OBJECTIVES

The objective of the study is to evaluate the performance of high-dimensional propensity scores (hdPSs) for controlling indication bias as compared with propensity scores (PSs) in surgical comparative effectiveness studies.

STUDY DESIGN AND SETTING

Patients who underwent interventional transcatheter aortic valve implantation (TAVI) or surgical aortic valve replacement (SAVR) between 2013 and 2017 were included from the French nationwide hospitals. At each hospital level, matched pairs of patients who underwent TAVI and SAVR were formed using PSs, considering 20 patient baseline characteristics, and hdPSs, considering the same patient characteristics and 300 additional variables from procedure and diagnosis codes the year before surgery. We compared death, reoperation, and stroke up to 3 years between TAVI and SAVR using Cox or Fine and Gray models.

RESULTS

Before matching, 12 of 20 patient characteristics were imbalanced between the included patients who underwent TAVI and SAVR. No significant imbalance persisted after matching with both methods. Hazard ratio of 1-year death, reoperation, and stroke was 1.3 [1.1; 1.4], 1.6 [1.1; 2.4], and 1.4 [1.2; 1.7] for TAVI relative to SAVR with PSs (n = 9,498 pairs) and 1.1 [1.0; 1.3], 1.3 [0.8; 2.0], and 1.3 [1.0; 1.6] with hdPSs (n = 7,157).

CONCLUSION

HdPS estimations were more consistent with results seen in randomized controlled trials. The HdPS is a promising alternative for the PS to control indication bias in comparative studies of surgical procedures.

Oral Corticosteroids and Risk of Preterm Birth in the California Medicaid Program

BACKGROUND

There is limited information regarding the impact of dose and gestational timing of oral corticosteroid (OCS) use on preterm birth (PTB), especially among women with asthma.

OBJECTIVES

To evaluate OCS dose and timing on PTB for asthma and, as a comparison, systemic lupus erythematosus (SLE).

METHODS

We used health care data from California Medicaid enrollees linked to birth certificates (2007-2013), identifying women with asthma (n = 22,084) and SLE (n = 1174). We estimated risk ratios (RR) for OCS cumulative dose trajectories and other disease-related medications before gestational day 140 and hazard ratios (HR) for time-varying exposures after day 139.

RESULTS

For asthma, PTB risk was 14.0% for no OCS exposure and 14.3%, 16.8%, 20.5%, and 32.7% in low, medium, medium-high, and high cumulative dose trajectory groups, respectively, during the first 139 days. The high-dose group remained associated with PTB after adjustment (adjusted RR [aRR]: 1.46; 95% confidence interval [CI]: 1.00, 2.15). OCS dose after day 139 was not clearly associated with PTB, nor were controller medications. For SLE, PTB risk for no OCS exposure was 24.9%, and it was 39.1% in low- and 61.2% in high-dose trajectory groups. aRR were 1.80 (95% CI: 1.34, 2.40) for high and 1.24 (95% CI: 0.97, 1.58) for low groups. Only prednisone equivalent dose >20 mg/day after day 139 was associated with increased PTB (adjusted HR: 2.54; 95% CI: 1.60, 4.03).

CONCLUSIONS

For asthma, higher OCS doses early in pregnancy, but not later, were associated with increased PTB. For SLE, higher doses early and later in pregnancy were associated with PTB.

Comparing the high-dimensional propensity score for use with administrative data with propensity scores derived from high-quality clinical data

Administrative healthcare databases are increasingly being used for research purposes. When used to estimate the effects of treatments and interventions, an important limitation of these databases is the lack of information on important confounding variables. The high-dimensional propensity score (hdPS) is an algorithm that generates a large number of empirically-derived covariates using administrative healthcare databases. The hdPS has been described as enabling adjustment by proxy, in which a large number of empirically-derived covariates may serve as proxies for unmeasured confounding variables. We examined the validity of this assumption using samples of patients hospitalized with acute myocardial infarction (AMI) and congestive heart failure (CHF), for whom both administrative data and detailed clinical data were available. We considered three treatments in AMI patients: angiotensin-converting enzyme inhibitors, beta-blockers, and statins, while the first two treatments were also considered in CHF patients. We considered three propensity scores: (a) one derived using detailed clinical data; (b) the hdPS derived from administrative data; and (c) one derived from administrative data using expert opinion. Using each propensity score, we estimated inverse probability of treatment (IPT) weights. For each sample and treatment combination, and for each of the two propensity scores derived using administrative data, there were clinical variables not measured in administrative data that remained imbalanced after incorporating the IPT weights. However, the propensity score derived using clinical data always resulted in all clinical variables being balanced. When estimating hazard ratios, for some samples and treatment combinations, the hazard ratios estimated using the hdPS were more similar to those obtained using the clinical propensity score than were those obtained using the expert-derived propensity score. However, for other combinations, the effects estimated using the expert-derived propensity score were more similar to those obtained using the clinical propensity score than were those derived using the hdPS.

Defining asthma and assessing asthma outcomes using electronic health record data: a systematic scoping review

There is currently no consensus on approaches to defining asthma or assessing asthma outcomes using electronic health record-derived data. We explored these approaches in the recent literature and examined the clarity of reporting.

We systematically searched for asthma-related articles published between January 1, 2014 and December 31, 2015, extracted the algorithms used to identify asthma patients and assess severity, control and exacerbations, and examined how the validity of these outcomes was justified.

From 113 eligible articles, we found significant heterogeneity in the algorithms used to define asthma (n=66 different algorithms), severity (n=18), control (n=9) and exacerbations (n=24). For the majority of algorithms (n=106), validity was not justified. In the remaining cases, approaches ranged from using algorithms validated in the same databases to using nonvalidated algorithms that were based on clinical judgement or clinical guidelines. The implementation of these algorithms was suboptimally described overall.

Although electronic health record-derived data are now widely used to study asthma, the approaches being used are significantly varied and are often underdescribed, rendering it difficult to assess the validity of studies and compare their findings. Given the substantial growth in this body of literature, it is crucial that scientific consensus is reached on the underlying definitions and algorithms.

Association between rehabilitation attendance and physical function following discharge after total knee arthroplasty: prospective cohort study

BACKGROUND

Rehabilitation is widely advocated and provided as a standard of care for patients with total knee arthroplasty (TKA) but its effects on intermediate- to longer-term physical function is unclear. Also unknown is the relationship between the number of rehabilitation sessions attended and functional outcomes.

METHODS

We conducted a prospective cohort study of 1540 patients who had undergone TKA and were referred for rehabilitation. Physical function was indexed by the Short-Form 36 (SF-36) physical function score at 6 months post-TKA. We used multivariable linear regression to assess the association between rehabilitation attendance and Month-6 physical function. Among patients who attended rehabilitation, multivariable linear regression was used to examine the dose-response association between the number of sessions attended and Month-6 physical function.

RESULTS

Of the 1540 patients, 68 patients did not attend rehabilitation, 86 patients attended one session, and 1386 patients attended two or more sessions. Adjusted for the propensity to attend rehabilitation, rehabilitation attendance was independently associated with better Month-6 SF-36 physical function (point estimate, 5.0 points; 95% CI, 0.5-9.5; P = 0.028 compared with patients with no rehabilitation). Among patients who attended rehabilitation, attending five sessions was associated with a 3.6-point increase in SF-36 scores (95% CI, 0.8-6.5; P = 0.01) relative to patients who attended one session.

CONCLUSIONS

Rehabilitation attendance post-TKA is associated with an increase in self-report physical function. Among patients who attended rehabilitation, a modest dose-response relationship was observed between the number of sessions and functional outcomes.

The Potential of High-Dimensional Propensity Scores in Health Services Research: An Exemplary Study on the Quality of Care for Elective Percutaneous Coronary Interventions

OBJECTIVE

Evaluating the potential of the high-dimensional propensity score (HDPS) to control for residual confounding in studies analyzing quality of care based on administrative health insurance data.

DATA SOURCE

Secondary data from 2004 to 2009 from three German statutory health insurance providers.

STUDY DESIGN

We conducted a retrospective cohort study in patients with elective percutaneous coronary interventions (PCIs) and compared the mortality risk between the in- and outpatient setting using Cox regression. Adjustment for predefined confounders was performed using conventional propensity score (PS) techniques. Further, an HDPS was calculated based on predefined and empirically selected confounders from the database.

PRINCIPAL FINDINGS

Conventional PS methods showed a decreased mortality risk for outpatient compared to inpatient PCIs, while trimming of patients with nonoverlap in the HDPS distribution and weighting resulted in a comparable risk. Most comorbidities were less prevalent in the HDPS-trimmed population compared to the original one.

CONCLUSION

The HDPS methodology may reduce residual confounding by rendering the studied cohort more comparable through restriction. However, results cannot be generalized for the entire study population. To provide unbiased results, full assessment of all unmeasured confounders from proxy information in the database would be necessary.

Impact of Copayment Changes on Children's Albuterol Inhaler Use and Costs after the Clean Air Act Chlorofluorocarbon Ban

OBJECTIVE

To examine changes in children's albuterol use and out-of-pocket (OOP) costs in response to increased copayments after the Food and Drug Administration banned inhalers with chlorofluorocarbon (CFC) propellants.

SETTING

Four health maintenance organizations (HMOs), two that increased copayments for albuterol inhalers that went from generic CFC-containing to branded CFC-free versions, and two that retained generic copayments for CFC-free inhalers (controls). We included children with asthma aged 4-17 years with commercial coverage from 2007 to 2010.

DESIGN

Interrupted time series with comparison series.

DATA

We obtained enrollee and plan characteristics from enrollment files, and utilization data from pharmacy and medical claims; OOP expenditures were extracted from pharmacy claims for two HMOs with cost data available.

FINDINGS

There were no significant differences in albuterol use between the group with increased cost-sharing and controls with respect to changes after the policy change. There was a postpolicy increase of $6.11 OOP per month per child using albuterol among those with increased cost-sharing versus $0.36 in controls; the difference between groups was significant (p < .01).

CONCLUSIONS

Increased copayments for brand-name CFC-free albuterol after the CFC ban did not lead to a decrease in children's albuterol use, but it led to a modest increase in OOP costs.

Novel Longitudinal and Propensity Score Matched Analysis of Hands-On Cooking and Nutrition Education versus Traditional Clinical Education among 627 Medical Students

Background. Physicians are inadequately equipped to respond to the global obesity and nutrition-associated chronic disease epidemics. We investigated superiority of simulation-based medical education with deliberate practice (SBME-DP) hands-on cooking and nutrition elective in a medical school-based teaching kitchen versus traditional clinical education for medical students. Materials and Methods. A 59-question panel survey was distributed to an entire medical school twice annually from September 2012 to May 2014. Student diet and attitudes and competencies (DACs) counseling patients on nutrition were compared using conditional multivariate logistic regression, propensity score-weighted, and longitudinal panel analyses. Inverse-variance weighted meta-analysis (IVWM) was used for planned subgroup analysis by year and treatment estimates across the three methods. Results. Of the available 954 students, 65.72% (n = 627) unique students were followed to produce 963 responses. 11.32% (n = 109) of responses were from 84 subjects who participated in the elective. SBME-DP versus traditional education significantly improved fruit and vegetable diet (OR = 1.38, 95% CI: 1.07-1.79, p = 0.013) and attitudes (OR = 1.81, 95% CI: 1.40-2.35, p < 0.001) and competencies (OR = 1.72, 95% CI: 1.54-1.92, p < 0.001). Conclusions. This study reports for the first time superiority longitudinally for SBME-DP style nutrition education for medical students which has since expanded to 13 schools.

Asthma Treatments and Mental Health Visits After a Food and Drug Administration Label Change for Leukotriene Inhibitors

PURPOSE

In 2009, the US Food and Drug Administration (FDA) mandated a label change for leukotriene inhibitors (LTIs) to include neuropsychiatric adverse events (eg, depression and suicidality) as a precaution. This study investigated how this label change affected the use of LTIs and other asthma controller medications, mental health visits, and suicide attempts.

METHODS

We analyzed data (2005-2010) from 5 large health plans in the US Population-Based Effectiveness in Asthma and Lung Diseases (PEAL) Network. The study cohort included children and adolescents (n = 30,000), young adults (n = 20,000), and adults (n = 90,000) with asthma. We used interrupted time series to examine changes in rates of LTI dispensings, non-LTI dispensings, mental health visits, and suicide attempts (using a validated algorithm based on a combination of diagnoses of injury or poisoning and psychiatric conditions).

FINDINGS

The label change was associated with abrupt reductions in LTI use among all age groups (relative reductions of 8.3%, 15.1%, and 6.0% among adolescents, young adults, and adults, respectively, compared with expected rates at 1 year after the warnings). Although we detected immediate offset increases in non-LTI asthma medication use, these increases were not sustained among adolescents and young adults. There were small increases in mental health visits among LTI users.

IMPLICATIONS

The FDA label change for LTIs communicated possible risk of neuropsychiatric events. Communication and enhanced awareness may have increased reporting of mental health symptoms among young adults and adults. It is important to assess intended and unintended consequences of FDA warnings and label changes.