Propensity score prediction for electronic healthcare databases using Super Learner and High-dimensional Propensity Score Methods

Association of pulmonary artery catheter with in-hospital outcomes after cardiac surgery in the United States: National Inpatient Sample 1999-2019

To examine associations of pulmonary artery catheter (PAC) use with in-hospital death and hospital length of stay (days) overall and within subgroups of hospitalized cardiac surgery patients. Secondary analyses of 1999-2019 National Inpatient Sample data were performed using 969,034 records (68% male, mean age: 65 years) representing adult cardiac surgery patients in the United States. A subgroup of 323,929 records corresponded to patients with congestive heart failure, pulmonary hypertension, mitral/tricuspid valve disease and/or combined surgeries. We evaluated PAC in relation to clinical outcomes using regression and targeted maximum likelihood estimation (TMLE). Hospitalized cardiac surgery patients experienced more in-hospital deaths and longer stays if they had ≥ 1 subgroup characteristics. For risk-adjusted models, in-hospital deaths were similar among recipients and non-recipients of PAC (odds ratio [OR] 1.04, 95% confidence interval [CI] 0.96, 1.12), although PAC was associated with more in-hospital deaths among the subgroup with congestive heart failure (OR 1.14, 95% CI 1.03, 1.26). PAC recipients experienced shorter stays than non-recipients (β =  - 0.40, 95% CI - 0.64, - 0.15), with variations by subgroup. We obtained comparable results using TMLE. In this retrospective cohort study, PAC was associated with shorter stays and similar in-hospital death rates among cardiac surgery patients. Worse clinical outcomes associated with PAC were observed only among patients with congestive heart failure. Prospective cohort studies and randomized controlled trials are needed to confirm and extend these preliminary findings.

Propensity score analysis with missing data using a multi-task neural network

BACKGROUND

Propensity score analysis is increasingly used to control for confounding factors in observational studies. Unfortunately, unavoidable missing values make estimating propensity scores extremely challenging. We propose a new method for estimating propensity scores in data with missing values.

MATERIALS AND METHODS

Both simulated and real-world datasets are used in our experiments. The simulated datasets were constructed under 2 scenarios, the presence (T = 1) and the absence (T = 0) of the true effect. The real-world dataset comes from LaLonde's employment training program. We construct missing data with varying degrees of missing rates under three missing mechanisms: MAR, MCAR, and MNAR. Then we compare MTNN with 2 other traditional methods in different scenarios. The experiments in each scenario were repeated 20,000 times. Our code is publicly available at https://github.com/ljwa2323/MTNN .

RESULTS

Under the three missing mechanisms of MAR, MCAR and MNAR, the RMSE between the effect and the true effect estimated by our proposed method is the smallest in simulations and in real-world data. Furthermore, the standard deviation of the effect estimated by our method is the smallest. In situations where the missing rate is low, the estimation of our method is more accurate.

CONCLUSIONS

MTNN can perform propensity score estimation and missing value filling at the same time through shared hidden layers and joint learning, which solves the dilemma of traditional methods and is very suitable for estimating true effects in samples with missing values. The method is expected to be broadly generalized and applied to real-world observational studies.

A two-stage super learner for healthcare expenditures

Objective

To improve the estimation of healthcare expenditures by introducing a novel method that is well-suited to situations where data exhibit strong skewness and zero-inflation.

Data Sources

Simulations, and two real-world datasets: the 2016-2017 Medical Expenditure Panel Survey (MEPS); the Back Pain Outcomes using Longitudinal Data (BOLD).

Study Design

Super learner is an ensemble machine learning approach that can combine several algorithms to improve estimation. We propose a two-stage super learner that is well suited for healthcare expenditure data by separately estimating the probability of any healthcare expenditure and the mean amount of healthcare expenditure conditional on having healthcare expenditures. These estimates can then be combined to yield a single estimate of expenditures for each observation. The analytical strategy can flexibly incorporate a range of individual estimation approaches for each stage of estimation, including both regression-based approaches and machine learning algorithms such as random forests. We compare the performance of the two-stage super learner with a one-stage super learner, and with multiple individual algorithms for estimation of healthcare cost under a broad range of data settings in simulated and real data. The predictive performance was compared using Mean Squared Error and R².

Conclusions

Our results indicate that the two-stage super learner has better performance compared with a one-stage super learner and individual algorithms, for healthcare cost estimation under a wide variety of settings in simulations and in empirical analyses. The improvement of the two-stage super learner over the one-stage super learner was particularly evident in settings when zero-inflation is high.

Self-Expandable Metal Stent as a Bridge to Surgery for Left-Sided Acute Malignant Colorectal Obstruction: Optimal Timing for Elective Surgery

OBJECTIVES

This randomized, single-center, retrospective, comparative cohort study is aimed at investigating the optimal time interval from self-expandable metal stent (SEMS) placement to surgery and potential risk factors for complications in patients with acute malignant colorectal obstruction.

METHODS

A total of 64 patients with left-sided acute malignant colorectal obstruction treated with SEMS placement and subsequent surgery between January 2013 and September 2020 were enrolled and allocated to a case group (SEMS placing time ≤ 14 days; n = 19 patients) and a control group (SEMS placing time > 14 days; n = 45 patients). The primary outcome was the difference in baseline information, patients' conditions during surgery, and postoperative conditions between the two groups. The secondary outcome included potential risk factors of postoperative complications. The propensity score matching (PSM) and super learner (SL) methods were used to eliminate multiple confounding factors of baseline data. A cohort of 21 samples was used for external validation, comprising 6 cases and 15 controls.

RESULTS

A significant difference was observed between the two groups in intraoperative blood loss (P = 0.009), postoperative hospital stay (P = 0.002), postoperative complications (Clavien-Dindo grading ≥ II) (P < 0.001), stoma creation (P < 0.001), and primary anastomosis (P < 0.001). After a 1 : 3 PSM analysis, no statistically significant differences between eight confounding variables of the two groups were observed (P > 0.05). Caliper set as 0.2 multiple logistic regression analysis showed that the potential risk factor for postoperative complications was SEMS placing time (RR = 0.109, 95% confidence interval (CI) = 0.028-0.433; P = 0.002), indicating that SEMS placing time > 14 days was an independent risk factor for postoperative complications in bridge-to-surgery (BTS) setting. The area under the AUC curve was 76.7% and validated using the validation cohort.

CONCLUSIONS

Long duration of SEMS placement (>14 days) may not influence surgical difficulty but could increase the risk of postoperative complications.

Synthetic Negative Controls: Using Simulation to Screen Large-scale Propensity Score Analyses

The propensity score has become a standard tool to control for large numbers of variables in healthcare database studies. However, little has been written on the challenge of comparing large-scale propensity score analyses that use different methods for confounder selection and adjustment. In these settings, balance diagnostics are useful but do not inform researchers on which variables balance should be assessed or quantify the impact of residual covariate imbalance on bias. Here, we propose a framework to supplement balance diagnostics when comparing large-scale propensity score analyses. Instead of focusing on results from any single analysis, we suggest conducting and reporting results for many analytic choices and using both balance diagnostics and synthetically generated control studies to screen analyses that show signals of bias caused by measured confounding. To generate synthetic datasets, the framework does not require simulating the outcome-generating process. In healthcare database studies, outcome events are often rare, making it difficult to identify and model all predictors of the outcome to simulate a confounding structure closely resembling the given study. Therefore, the framework uses a model for treatment assignment to divide the comparator population into pseudo-treatment groups where covariate differences resemble those in the study cohort. The partially simulated datasets have a confounding structure approximating the study population under the null (synthetic negative control studies). The framework is used to screen analyses that likely violate partial exchangeability due to lack of control for measured confounding. We illustrate the framework using simulations and an empirical example.

Deep Ensemble Machine Learning Framework for the Estimation of PM2.5 Concentrations

BACKGROUND

Accurate estimation of historical PM2.5 (particle matter with an aerodynamic diameter of less than 2.5μm) is critical and essential for environmental health risk assessment.

OBJECTIVES

The aim of this study was to develop a multiple-level stacked ensemble machine learning framework for improving the estimation of the daily ground-level PM2.5 concentrations.

METHODS

An innovative deep ensemble machine learning framework (DEML) was developed to estimate the daily PM2.5 concentrations. The framework has a three-stage structure: At the first stage, four base models [gradient boosting machine (GBM), support vector machine (SVM), random forest (RF), and eXtreme gradient boosting (XGBoost)] were used to generate a new data set of PM2.5 concentrations for training the next-stage learners. At the second stage, three meta-models [RF, XGBoost, and Generalized Linear Model (GLM)] were used to estimate PM2.5 concentrations using a combination of the original data set and the predictions from the first-stage models. At the third stage, a nonnegative least squares (NNLS) algorithm was employed to obtain the optimal weights for PM2.5 estimation. We took the data from 133 monitoring stations in Italy as an example to implement the DEML to predict daily PM2.5 at each 1km×1km grid cell from 2015 to 2019 across Italy. We evaluated the model performance by performing 10-fold cross-validation (CV) and compared it with five benchmark algorithms [GBM, SVM, RF, XGBoost, and Super Learner (SL)].

RESULTS

The results revealed that the PM2.5 prediction performance of DEML [coefficients of determination (R2)=0.87 and root mean square error (RMSE)=5.38μg/m3] was superior to any benchmark models (with R2 of 0.51, 0.76, 0.83, 0.70, and 0.83 for GBM, SVM, RF, XGBoost, and SL approach, respectively). DEML displayed reliable performance in capturing the spatiotemporal variations of PM2.5 in Italy.

DISCUSSION

The proposed DEML framework achieved an outstanding performance in PM2.5 estimation, which could be used as a tool for more accurate environmental exposure assessment. https://doi.org/10.1289/EHP9752.

Applying Machine Learning in Distributed Data Networks for Pharmacoepidemiologic and Pharmacovigilance Studies: Opportunities, Challenges, and Considerations

Increasing availability of electronic health databases capturing real-world experiences with medical products has garnered much interest in their use for pharmacoepidemiologic and pharmacovigilance studies. The traditional practice of having numerous groups use single databases to accomplish similar tasks and address common questions about medical products can be made more efficient through well-coordinated multi-database studies, greatly facilitated through distributed data network (DDN) architectures. Access to larger amounts of electronic health data within DDNs has created a growing interest in using data-adaptive machine learning (ML) techniques that can automatically model complex associations in high-dimensional data with minimal human guidance. However, the siloed storage and diverse nature of the databases in DDNs create unique challenges for using ML. In this paper, we discuss opportunities, challenges, and considerations for applying ML in DDNs for pharmacoepidemiologic and pharmacovigilance studies. We first discuss major types of activities performed by DDNs and how ML may be used. Next, we discuss practical data-related factors influencing how DDNs work in practice. We then combine these discussions and jointly consider how opportunities for ML are affected by practical data-related factors for DDNs, leading to several challenges. We present different approaches for addressing these challenges and highlight efforts that real-world DDNs have taken or are currently taking to help mitigate them. Despite these challenges, the time is ripe for the emerging interest to use ML in DDNs, and the utility of these data-adaptive modeling techniques in pharmacoepidemiologic and pharmacovigilance studies will likely continue to increase in the coming years.

A Comparison of Confounder Selection and Adjustment Methods for Estimating Causal Effects Using Large Healthcare Databases

PURPOSE

Confounding adjustment is required to estimate the effect of an exposure on an outcome in observational studies. However, variable selection and unmeasured confounding are particularly challenging when analyzing large healthcare data. Machine learning methods may help address these challenges. The objective was to evaluate the capacity of such methods to select confounders and reduce unmeasured confounding bias.

METHODS

A simulation study with known true effects was conducted. Completely synthetic and partially synthetic data incorporating real large healthcare data were generated. We compared Bayesian Adjustment for Confounding, Generalized Bayesian Causal Effect Estimation, Group Lasso and Doubly Robust Estimation, high-dimensional propensity score, and scalable collaborative targeted maximum likelihood algorithms. For the high-dimensional propensity score, two adjustment approaches targeting the effect in the whole population were considered: full matching and inverse probability weighting.

RESULTS

In scenarios without hidden confounders, most methods were essentially unbiased. The bias and variance of the high-dimensional propensity score varied considerably according to the number of variables selected by the algorithm. In scenarios with hidden confounders, substantial bias reduction was achieved by using machine learning methods to identify proxies as compared to adjusting only by observed confounders. High-dimensional propensity score and Group Lasso performed poorly in the partially synthetic simulation. Bayesian Adjustment for Confounding, Generalized Bayesian Causal Effect Estimation, and scalable collaborative targeted maximum likelihood algorithms performed particularly well.

CONCLUSIONS

Machine learning can help to identify measured confounders in large healthcare databases. They can also capitalize on proxies of unmeasured confounders to substantially reduce residual confounding bias. This article is protected by copyright. All rights reserved.

Hospitalization outcomes among brain metastasis patients receiving radiation therapy with or without stereotactic radiosurgery from the 2005-2014 Nationwide Inpatient Sample

The purpose of this study was to compare hospitalization outcomes among US inpatients with brain metastases who received stereotactic radiosurgery (SRS) and/or non-SRS radiation therapies without neurosurgical intervention. A cross-sectional study was conducted whereby existing data on 35,199 hospitalization records (non-SRS alone: 32,981; SRS alone: 1035; SRS + non-SRS: 1183) from 2005 to 2014 Nationwide Inpatient Sample were analyzed. Targeted maximum likelihood estimation and Super Learner algorithms were applied to estimate average treatment effects (ATE), marginal odds ratios (MOR) and causal risk ratio (CRR) for three distinct types of radiation therapy in relation to hospitalization outcomes, including length of stay (' ≥ 7 days' vs. ' < 7 days') and discharge destination ('non-routine' vs. 'routine'), controlling for patient and hospital characteristics. Recipients of SRS alone (ATE = - 0.071, CRR = 0.88, MOR = 0.75) or SRS + non-SRS (ATE = - 0.17, CRR = 0.70, MOR = 0.50) had shorter hospitalizations as compared to recipients of non-SRS alone. Recipients of SRS alone (ATE = - 0.13, CRR = 0.78, MOR = 0.59) or SRS + non-SRS (ATE = - 0.17, CRR = 0.72, MOR = 0.51) had reduced risks of non-routine discharge as compared to recipients of non-SRS alone. Similar analyses suggested recipients of SRS alone had shorter hospitalizations and similar risk of non-routine discharge when compared to recipients of SRS + non-SRS radiation therapies. SRS alone or in combination with non-SRS therapies may reduce the risks of prolonged hospitalization and non-routine discharge among hospitalized US patients with brain metastases who underwent radiation therapy without neurosurgical intervention.

Identifying Drugs Inducing Prematurity by Mining Claims Data with High-Dimensional Confounder Score Strategies

BACKGROUND

Pregnant women are largely exposed to medications. However, knowledge is lacking about their effects on pregnancy and the fetus.

OBJECTIVE

This study sought to evaluate the potential of high-dimensional propensity scores and high-dimensional disease risk scores for automated signal detection in pregnant women from medico-administrative databases in the context of drug-induced prematurity.

METHODS

We used healthcare claims and hospitalization discharges of a 1/97th representative sample of the French population. We tested the association between prematurity and drug exposure during the trimester before delivery, for all drugs prescribed to at least five pregnancies. We compared different strategies (1) for building the two scores, including two machine-learning methods and (2) to account for these scores in the final logistic regression models: adjustment, weighting, and matching. We also proposed a new signal detection criterion derived from these scores: the p value relative decrease. Evaluation was performed by assessing the relevance of the signals using a literature review and clinical expertise.

RESULTS

Screening 400 drugs from a cohort of 57,407 pregnancies, we observed that choosing between the two machine-learning methods had little impact on the generated signals. Score adjustment performed better than weighting and matching. Using the p value relative decrease efficiently filtered out spurious signals while maintaining a number of relevant signals similar to score adjustment. Most of the relevant signals belonged to the psychotropic class with benzodiazepines, antidepressants, and antipsychotics.

CONCLUSIONS

Mining complex healthcare databases with statistical methods from the high-dimensional inference field may improve signal detection in pregnant women.

A General Propensity Score for Signal Identification Using Tree-Based Scan Statistics

The tree-based scan statistic (TreeScan; Martin Kulldorff, Harvard Medical School, Boston, Massachusetts) is a data-mining method that adjusts for multiple testing of correlated hypotheses when screening thousands of potential adverse events for signal identification. Simulation has demonstrated the promise of TreeScan with a propensity score (PS)-matched cohort design. However, it is unclear which variables to include in a PS for applied signal identification studies to simultaneously adjust for confounding across potential outcomes. We selected 4 pairs of medications with well-understood safety profiles. For each pair, we evaluated 5 candidate PSs with different combinations of 1) predefined general covariates (comorbidity, frailty, utilization), 2) empirically selected (data-driven) covariates, and 3) covariates tailored to the drug pair. For each pair, statistical alerting patterns were similar with alternative PSs (≤11 alerts in 7,996 outcomes scanned). Inclusion of covariates tailored to exposure did not appreciably affect screening results. Inclusion of empirically selected covariates can provide better proxy coverage for confounders but can also decrease statistical power. Unlike tailored covariates, empirical and predefined general covariates can be applied "out of the box" for signal identification. The choice of PS depends on the level of concern about residual confounding versus loss of power. Potential signals should be followed by pharmacoepidemiologic assessment where confounding control is tailored to the specific outcome(s) under investigation.

Prediction of Dead Oil Viscosity: Machine Learning vs. Classical Correlations

Dead oil viscosity is a critical parameter to solve numerous reservoir engineering problems and one of the most unreliable properties to predict with classical black oil correlations. Determination of dead oil viscosity by experiments is expensive and time-consuming, which means developing an accurate and quick prediction model is required. This paper implements six machine learning models: random forest (RF), lightgbm, XGBoost, multilayer perceptron (MLP) neural network, stochastic real-valued (SRV) and SuperLearner to predict dead oil viscosity. More than 2000 pressure–volume–temperature (PVT) data were used for developing and testing these models. A huge range of viscosity data were used, from light intermediate to heavy oil. In this study, we give insight into the performance of different functional forms that have been used in the literature to formulate dead oil viscosity. The results show that the functional form f(γAPI,T), has the best performance, and additional correlating parameters might be unnecessary. Furthermore, SuperLearner outperformed other machine learning (ML) algorithms as well as common correlations that are based on the metric analysis. The SuperLearner model can potentially replace the empirical models for viscosity predictions on a wide range of viscosities (any oil type). Ultimately, the proposed model is capable of simulating the true physical trend of the dead oil viscosity with variations of oil API gravity, temperature and shear rate.

A machine learning compatible method for ordinal propensity score stratification and matching

Although machine learning techniques that estimate propensity scores for observational studies with multivalued treatments have advanced rapidly in recent years, the development of propensity score adjustment techniques has not kept pace. While machine learning propensity models provide numerous benefits, they do not produce a single variable balancing score that can be used for propensity score stratification and matching. This issue motivates the development of a flexible ordinal propensity scoring methodology that does not require parametric assumptions for the propensity model. The proposed method fits a one-parameter power function to the cumulative distribution function (CDF) of the generalized propensity score (GPS) vector resulting from any machine learning propensity model, and is henceforth called the GPS-CDF method. The estimated parameter from the GPS-CDF method, ã , is a scalar balancing score that can be used to group similar subjects in outcome analyses. Specifically, subjects who received different levels of the treatment are stratified or matched based on their ã value to produce unbiased estimates of the average treatment effect (ATE). Simulation studies presented show remediation of covariate balance, minimal bias in ATEs, and maintain coverage probability. The proposed method is applied to the Mexican-American Tobacco use in Children (MATCh) study to determine whether an ordinal treatment of exposure to smoking imagery in movies causes cigarette experimentation in Mexican-American adolescents.

A novel approach for propensity score matching and stratification for multiple treatments: Application to an electronic health record-derived study

Currently, methods for conducting multiple treatment propensity scoring in the presence of high-dimensional covariate spaces that result from "big data" are lacking-the most prominent method relies on inverse probability treatment weighting (IPTW). However, IPTW only utilizes one element of the generalized propensity score (GPS) vector, which can lead to a loss of information and inadequate covariate balance in the presence of multiple treatments. This limitation motivates the development of a novel propensity score method that uses the entire GPS vector to establish a scalar balancing score that, when adjusted for, achieves covariate balance in the presence of potentially high-dimensional covariates. Specifically, the generalized propensity score cumulative distribution function (GPS-CDF) method is introduced. A one-parameter power function fits the CDF of the GPS vector and a resulting scalar balancing score is used for matching and/or stratification. Simulation results show superior performance of the new method compared to IPTW both in achieving covariate balance and estimating average treatment effects in the presence of multiple treatments. The proposed approach is applied to a study derived from electronic medical records to determine the causal relationship between three different vasopressors and mortality in patients with non-traumatic aneurysmal subarachnoid hemorrhage. Results suggest that the GPS-CDF method performs well when applied to large observational studies with multiple treatments that have large covariate spaces.