Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression

Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force

Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.

Dynamic Ensemble Selection for Imbalanced Data Streams With Concept Drift

Ensemble learning, as a popular method to tackle concept drift in data stream, forms a combination of base classifiers according to their global performances. However, concept drift generally occurs in local data space, causing significantly different performances of a base classifier at different locations. Thus, employing global performance as a criterion to select base classifier is inappropriate. Moreover, data stream is often accompanied by class imbalance problem, which affects the classification accuracy of ensemble learning on minority instances. To drawback these problems, a dynamic ensemble selection for imbalanced data streams with concept drift (DES-ICD) is proposed. For data arrived in chunk-by-chunk, a novel synthetic minority oversampling technique with adaptive nearest neighbors (AnnSMOTE) is developed to generate new minority instances that conform to the new concept. Following that, DES-ICD creates a base classifier on newly arrived data chunk balanced by AnnSMOTE and merges it with historical base classifiers to form a candidate classifier pool. For each query instance, the optimal combination is constructed in terms of the performance of candidate classifiers in its neighborhood. Experimental results for nine synthetic and five real-world datasets show that the proposed method outperforms seven comparative methods on classification accuracy and tracks new concepts in an imbalanced data stream more preciously.

Machine learning for improving high-dimensional proxy confounder adjustment in healthcare database studies: an overview of the current literature

Controlling for large numbers of variables that collectively serve as 'proxies' for unmeasured factors can often improve confounding control in pharmacoepidemiologic studies utilizing administrative healthcare databases. There is a growing body of evidence showing that data-driven machine learning algorithms for high-dimensional proxy confounder adjustment can supplement investigator-specified variables to improve confounding control compared to adjustment based on investigator-specified variables alone. Consequently, there has been a recent focus on the development of data-driven methods for high-dimensional proxy confounder adjustment. In this paper, we discuss the considerations underpinning three areas for data-driven high-dimensional proxy confounder adjustment: 1) feature generation-transforming raw data into covariates (or features) to be used for proxy adjustment; 2) covariate prioritization, selection and adjustment; and 3) diagnostic assessment. We survey current approaches and recent advancements within each area, including the most widely used approach to proxy confounder adjustment in healthcare database studies (the high-dimensional propensity score or hdPS). We also discuss limitations of the hdPS and outline recent advancements that incorporate the principles of proxy adjustment with machine learning extensions to improve performance. We further discuss challenges and avenues of future development within each area. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE). This article is protected by copyright. All rights reserved.

Methods for Evaluating Environmental Health Impacts at Different Stages of the Policy Process in Cities

PURPOSE OF REVIEW

Evaluating the environmental health impacts of urban policies is critical for developing and implementing policies that lead to more healthy and equitable cities. This article aims to (1) identify research questions commonly used when evaluating the health impacts of urban policies at different stages of the policy process, (2) describe commonly used methods, and (3) discuss challenges, opportunities, and future directions.

RECENT FINDINGS

In the diagnosis and design stages of the policy process, research questions aim to characterize environmental problems affecting human health and to estimate the potential impacts of new policies. Simulation methods using existing exposure-response information to estimate health impacts predominate at these stages of the policy process. In subsequent stages, e.g., during implementation, research questions aim to understand the actual policy impacts. Simulation methods or observational methods, which rely on experimental data gathered in the study area to assess the effectiveness of the policy, can be applied at these stages. Increasingly, novel techniques fuse both simulation and observational methods to enhance the robustness of impact evaluations assessing implemented policies. The policy process consists of interdependent stages, from inception to end, but most reviewed studies focus on single stages, neglecting the continuity of the policy life cycle. Studies assessing the health impacts of policies using a multi-stage approach are lacking. Most studies investigate intended impacts of policies; focusing also on unintended impacts may provide a more comprehensive evaluation of policies.

Development and validation of a meta-learner for combining statistical and machine learning prediction models in individuals with depression

BACKGROUND

The debate of whether machine learning models offer advantages over standard statistical methods when making predictions is ongoing. We discuss the use of a meta-learner model combining both approaches as an alternative.

METHODS

To illustrate the development of a meta-learner, we used a dataset of 187,757 people with depression. Using 31 variables, we aimed to predict two outcomes measured 60 days after initiation of antidepressant treatment: severity of depressive symptoms (continuous) and all-cause dropouts (binary). We fitted a ridge regression and a multi-layer perceptron (MLP) deep neural network as two separate prediction models ("base-learners"). We then developed two "meta-learners", combining predictions from the two base-learners. To compare the performance across the different methods, we calculated mean absolute error (MAE, for continuous outcome) and the area under the receiver operating characteristic curve (AUC, for binary outcome) using bootstrapping.

RESULTS

Compared to the best performing base-learner (MLP base-learner, MAE at 4.63, AUC at 0.59), the best performing meta-learner showed a 2.49% decrease in MAE at 4.52 for the continuous outcome and a 6.47% increase in AUC at 0.60 for the binary outcome.

CONCLUSIONS

A meta-learner approach may effectively combine multiple prediction models. Choosing between statistical and machine learning models may not be necessary in practice.

Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging

Brain imaging research enjoys increasing adoption of supervised machine learning for single-participant disease classification. Yet, the success of these algorithms likely depends on population diversity, including demographic differences and other factors that may be outside of primary scientific interest. Here, we capitalize on propensity scores as a composite confound index to quantify diversity due to major sources of population variation. We delineate the impact of population heterogeneity on the predictive accuracy and pattern stability in 2 separate clinical cohorts: the Autism Brain Imaging Data Exchange (ABIDE, n = 297) and the Healthy Brain Network (HBN, n = 551). Across various analysis scenarios, our results uncover the extent to which cross-validated prediction performances are interlocked with diversity. The instability of extracted brain patterns attributable to diversity is located preferentially in regions part of the default mode network. Collectively, our findings highlight the limitations of prevailing deconfounding practices in mitigating the full consequences of population diversity.

Human-induced arsenic pollution modeling in surface waters - An integrated approach using machine learning algorithms and environmental factors

In recent years, assessment of sediment contamination by heavy metals, i.e., arsenic, has attracted the interest of scientists worldwide. The present study provides a new methodology to better understand the factors influencing surface water vulnerability to arsenic pollution by two advanced machine learning algorithms including boosted regression trees (BRT) and random forest (RF). Based on the sediment quality guidelines (Effects range low) polluted and non-polluted arsenic sediment samples were defined with concentrations >8 ppm and <8 ppm, respectively. Different conditioning factors such as topographical, lithology, erosion, hydrological, and anthropogenic factors were acquired to model surface waters' vulnerability to arsenic. We trained and validated the models using 70 and 30% of both polluted and non-polluted samples, respectively, and generated surface vulnerability maps. To verify the maps to arsenic pollution, the receiver operating characteristics (ROC) curve was implemented. The results approved the acceptable performance of the RF and BRT algorithms with an area under ROC values of 85% and 75.6%, respectively. Further, the findings showed higher importance of precipitation, slope aspect, distance from residential areas, and slope length in arsenic pollution in the modeling process. Erosion, lithology, and land use maps were introduced as the least important factors. The introduced methodology can be used to define the most vulnerable areas to arsenic pollution in advance and implement proper remediation actions to reduce the damages.

Robust Machine Learning for Treatment Effects in Multilevel Observational Studies Under Cluster-level Unmeasured Confounding

Recently, machine learning (ML) methods have been used in causal inference to estimate treatment effects in order to reduce concerns for model mis-specification. However, many ML methods require that all confounders are measured to consistently estimate treatment effects. In this paper, we propose a family of ML methods that estimate treatment effects in the presence of cluster-level unmeasured confounders, a type of unmeasured confounders that are shared within each cluster and are common in multilevel observational studies. We show through simulation studies that our proposed methods are robust from biases from unmeasured cluster-level confounders in a variety of multilevel observational studies. We also examine the effect of taking an algebra course on math achievement scores from the Early Childhood Longitudinal Study, a multilevel observational educational study, using our methods. The proposed methods are available in the CURobustML R package.

Predicting program outcomes for vocational rehabilitation customers: A machine learning approach

BACKGROUND: The Vocational Rehabilitation (VR) program provides support and services to people with disabilities who want to work. OBJECTIVE: Approximately one-third of eligible VR customers are employed when they exit the program. The remainder either exit without ever receiving services or without employment after receiving services. In this study, we explore how customer characteristics and VR services predict these outcomes. METHODS: We examined VR case level data from the RSA-911 files. Machine learning techniques allowed us to explore a large number of potential predictors of VR outcomes while requiring fewer assumptions than traditional regression methods. RESULTS: Consistent with existing literature, customers who are employed at application are more likely to exit with employment, and those with mental health conditions or low socioeconomic status are less likely to exit with employment. Some customers with low or no earnings at application who are not identified in prior studies are more likely than others to have poor program outcomes, including those with developmental disability who are under 18, customers without developmental or learning disabilities, and customers who do not receive employment or restoration services. CONCLUSIONS: VR counselors and administrators should consider implementing early, targeted interventions for newly identified at-risk groups of VR customers.

Data-Driven Machine Learning in Environmental Pollution: Gains and Problems

The complexity and dynamics of the environment make it extremely difficult to directly predict and trace the temporal and spatial changes in pollution. In the past decade, the unprecedented accumulation of data, the development of high-performance computing power, and the rise of diverse machine learning (ML) methods provide new opportunities for environmental pollution research. The ML methodology has been used in satellite data processing to obtain ground-level concentrations of atmospheric pollutants, pollution source apportionment, and spatial distribution modeling of water pollutants. However, unlike the active practices of ML in chemical toxicity prediction, advanced algorithms such as deep neural networks in environmental process studies of pollutants are still deficient. In addition, over 40% of the environmental applications of ML go to air pollution, and its application range and acceptance in other aspects of environmental science remain to be increased. The use of ML methods to revolutionize environmental science and its problem-solving scenarios has its own challenges. Several issues should be taken into consideration, such as the tradeoff between model performance and interpretability, prerequisites of the machine learning model, model selection, and data sharing.

Applying Machine Learning in Distributed Data Networks for Pharmacoepidemiologic and Pharmacovigilance Studies: Opportunities, Challenges, and Considerations

Increasing availability of electronic health databases capturing real-world experiences with medical products has garnered much interest in their use for pharmacoepidemiologic and pharmacovigilance studies. The traditional practice of having numerous groups use single databases to accomplish similar tasks and address common questions about medical products can be made more efficient through well-coordinated multi-database studies, greatly facilitated through distributed data network (DDN) architectures. Access to larger amounts of electronic health data within DDNs has created a growing interest in using data-adaptive machine learning (ML) techniques that can automatically model complex associations in high-dimensional data with minimal human guidance. However, the siloed storage and diverse nature of the databases in DDNs create unique challenges for using ML. In this paper, we discuss opportunities, challenges, and considerations for applying ML in DDNs for pharmacoepidemiologic and pharmacovigilance studies. We first discuss major types of activities performed by DDNs and how ML may be used. Next, we discuss practical data-related factors influencing how DDNs work in practice. We then combine these discussions and jointly consider how opportunities for ML are affected by practical data-related factors for DDNs, leading to several challenges. We present different approaches for addressing these challenges and highlight efforts that real-world DDNs have taken or are currently taking to help mitigate them. Despite these challenges, the time is ripe for the emerging interest to use ML in DDNs, and the utility of these data-adaptive modeling techniques in pharmacoepidemiologic and pharmacovigilance studies will likely continue to increase in the coming years.

Pneumococcal vaccination prevented severe LRTIs in adults: a causal inference framework applied in registry data

OBJECTIVES

We estimated the effect of pneumococcal vaccination (PV) on acute lower respiratory tract infections (LRTIs) in various age and risk groups using different methods within a causal inference methodological framework.

STUDY DESIGN AND SETTING

We used data from a general practitioners' morbidity registry for the year 2019. Both traditional statistical methods (regression-based and propensity score methods) and machine learning techniques were deployed. Multiple imputation was used to account for missing data. Relative risks (RRs) with 95% confidence intervals were estimated. Sensitivity analyses were performed to account for the severity of LRTIs and differences in vaccination registration.

RESULTS

All methods showed a standardized mean difference below 0.1 for each covariate. No method was found to be superior to another. PV (combination of conjugate and polysaccharide vaccine) had an overall protective effect for severe LRTIs. PV was protective in different age and risk groups, especially in people aged 50-84 years with an intermediate risk group.

CONCLUSION

Using several techniques, PV was found to prevent severe LRTIs and confirmed the recommendations of the Belgian Superior Health Council.

AIPW: An R Package for Augmented Inverse Probability-Weighted Estimation of Average Causal Effects

An increasing number of recent studies have suggested that doubly robust estimators with cross-fitting should be used when estimating causal effects with machine learning methods. However, not all existing programs that implement doubly robust estimators support machine learning methods and cross-fitting, or provide estimates on multiplicative scales. To address these needs, we developed AIPW, a software package implementing augmented inverse probability weighting (AIPW) estimation of average causal effects in R (R Foundation for Statistical Computing, Vienna, Austria). Key features of the AIPW package include cross-fitting and flexible covariate adjustment for observational studies and randomized controlled trials (RCTs). In this paper, we use a simulated RCT to illustrate implementation of the AIPW estimator. We also perform a simulation study to evaluate the performance of the AIPW package compared with other doubly robust implementations, including CausalGAM, npcausal, tmle, and tmle3. Our simulation showed that the AIPW package yields performance comparable to that of other programs. Furthermore, we also found that cross-fitting substantively decreases the bias and improves the confidence interval coverage for doubly robust estimators fitted with machine learning algorithms. Our findings suggest that the AIPW package can be a useful tool for estimating average causal effects with machine learning methods in RCTs and observational studies.

Stratified Restricted Mean Survival Time Model for Marginal Causal Effect in Observational Survival Data

Time to event outcomes is commonly encountered in epidemiologic research. Multiple papers have discussed the inadequacy of using the hazard ratio as a causal effect measure due to its noncollapsibility and the time-varying nature. In this paper, we further clarified that the hazard ratio might be used as a conditional causal effect measure, but it is generally not a valid marginal effect measure, even under randomized design. We proposed to use the restricted mean survival time (RMST) difference as a causal effect measure, since it essentially measures the mean difference over a specified time horizon and has a simple interpretation as the area under survival curves. For observational studies, propensity score adjustment can be implemented with RMST estimation to remove observed confounding bias. We proposed a propensity score stratified RMST estimation strategy, which performs well in our simulation evaluation and is relatively easy to implement for epidemiologists in practice. Our stratified RMST estimation includes two different versions of implementation, depending on whether researchers want to involve regression modeling adjustment, which provides a powerful tool to examine the marginal causal effect with observational survival data.

Assessing Fairness in the Presence of Missing Data

Missing data are prevalent and present daunting challenges in real data analysis. While there is a growing body of literature on fairness in analysis of fully observed data, there has been little theoretical work on investigating fairness in analysis of incomplete data. In practice, a popular analytical approach for dealing with missing data is to use only the set of complete cases, i.e., observations with all features fully observed to train a prediction algorithm. However, depending on the missing data mechanism, the distribution of complete cases and the distribution of the complete data may be substantially different. When the goal is to develop a fair algorithm in the complete data domain where there are no missing values, an algorithm that is fair in the complete case domain may show disproportionate bias towards some marginalized groups in the complete data domain. To fill this significant gap, we study the problem of estimating fairness in the complete data domain for an arbitrary model evaluated merely using complete cases. We provide upper and lower bounds on the fairness estimation error and conduct numerical experiments to assess our theoretical results. Our work provides the first known theoretical results on fairness guarantee in analysis of incomplete data.

Reasons for Utilizing Telemedicine during and after the COVID-19 Pandemic: An Internet-Based International Study

The COVID-19 pandemic challenges healthcare services. Concomitantly, this pandemic had a stimulating effect on technological expansions related to telehealth and telemedicine. We sought to elucidate the principal patients' reasons for using telemedicine during the COVID-19 pandemic and the propensity to use it thereafter. Our primary objective was to identify the reasons of the survey participants' disparate attitudes toward the use of telemedicine. We performed an online, multilingual 30-question survey for 14 days during March-April 2021, focusing on the perception and usage of telemedicine and their intent to use it after the pandemic. We analyzed the data to identify the attributes influencing the intent to use telemedicine and built decision trees to highlight the most important related variables. We examined 473 answers: 272 from Israel, 87 from Uruguay, and 114 worldwide. Most participants were women (64.6%), married (63.8%) with 1-2 children (52.9%), and living in urban areas (84.6%). Only a third of the participants intended to continue using telemedicine after the COVID-19 pandemic. Our main findings are that an expected substitution effect, technical proficiency, reduced queueing times, and peer experience are the four major factors in the overall adoption of telemedicine. Specifically, (1) for most participants, the major factor influencing their telemedicine usage is the implicit expectation that such a visit will be a full substitute for an in-person appointment; (2) another factor affecting telemedicine usage by patients is their overall technical proficiency and comfort level in the use of common web-based tools, such as social media, while seeking relevant medical information; (3) time saving as telemedicine can allow for asynchronous communications, thereby reducing physical travel and queuing times at the clinic; and finally (4) some participants have also indicated that telemedicine seems more attractive to them after watching family and friends (peer experience) use it successfully.

Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings

Background

Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.

Methods

We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.

Results

Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.

Conclusions

The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01454-z.

Pharmacoepidemiology, Machine Learning, and COVID-19: An Intent-to-Treat Analysis of Hydroxychloroquine, With or Without Azithromycin, and COVID-19 Outcomes Among Hospitalized US Veterans

Hydroxychloroquine (HCQ) was proposed as an early therapy for coronavirus disease 2019 (COVID-19) after in vitro studies indicated possible benefit. Previous in vivo observational studies have presented conflicting results, though recent randomized clinical trials have reported no benefit from HCQ amongst hospitalized COVID-19 patients. We examined the effects of HCQ alone, and in combination with azithromycin, in a hospitalized COVID-19 positive, United States (US) Veteran population using a propensity score adjusted survival analysis with imputation of missing data. From March 1, 2020 through April 30, 2020, 64,055 US Veterans were tested for COVID-19 based on Veteran Affairs Healthcare Administration electronic health record data. Of the 7,193 positive cases, 2,809 were hospitalized, and 657 individuals were prescribed HCQ within the first 48-hours of hospitalization for the treatment of COVID-19. There was no apparent benefit associated with HCQ receipt, alone or in combination with azithromycin, and an increased risk of intubation when used in combination with azithromycin [Hazard Ratio (95% Confidence Interval): 1.55 (1.07, 2.24)]. In conclusion, we assessed the effectiveness of HCQ with or without azithromycin in treating patients hospitalized with COVID-19 using a national sample of the US Veteran population. Using rigorous study design and analytic methods to reduce confounding and bias, we found no evidence of a survival benefit from the administration of HCQ.

Random Forests Approach for Causal Inference with Clustered Observational Data

There is a growing interest in using machine learning (ML) methods for causal inference due to their (nearly) automatic and flexible ability to model key quantities such as the propensity score or the outcome model. Unfortunately, most ML methods for causal inference have been studied under single-level settings where all individuals are independent of each other and there is little work in using these methods with clustered or nested data, a common setting in education studies. This paper investigates using one particular ML method based on random forests known as Causal Forests to estimate treatment effects in multilevel observational data. We conduct simulation studies under different types of multilevel data, including two-level, three-level, and cross-classified data. Our simulation study shows that when the ML method is supplemented with estimated propensity scores from multilevel models that account for clustered/hierarchical structure, the modified ML method outperforms preexisting methods in a wide variety of settings. We conclude by estimating the effect of private math lessons in the Trends in International Mathematics and Science Study data, a large-scale educational assessment where students are nested within schools.

Use of Propensity Scoring and Its Application to Real-World Data: Advantages, Disadvantages, and Methodological Objectives Explained to Researchers Without Using Mathematical Equations

Real-time data collection of patient health status and medications is sped up with modern electronic devices and technologies. As real-world data provide enormous research opportunities, propensity score (PS) methods have been getting attention due to their theoretical grounds in a nonrandomized study setting. In contrast to randomized clinical trials, observational clinical data obtained from a real-world database may not have balanced distributions of patient characteristics between treatment and control groups at the beginning of the respective study. These imbalanced distributions may cause a bias in an estimated treatment effect, which needs to be eliminated. Propensity scoring is one class of statistical methods to address the imbalance issue of real-world data sets. This article provides basic concepts and assesses advantages, disadvantages, and methodological objectives of propensity scoring. Targeting clinical pharmacology researchers with limited statistical background, 5 representative methods are reviewed and visualized: matching, stratification, covariate modeling, inverse probability of treatment weighting, and doubly robust methods. Examples of applications of PS methods were selected from the literature of outcomes research and drug development, nephrology, and pediatrics. Opportunities of applications related to these examples are described. Furthermore, potential future applications of PS methods in clinical pharmacology are discussed. The 21st Century Cures Act signed in 2016 encourages scientists to find opportunities to apply propensity scoring to real-world data. This article underscores that scientists need to justify their choice of statistical methods, whether a PS method or an alternative method, based on their clinical study design, statistical assumptions, and research objectives.

CXCL10 levels at hospital admission predict COVID-19 outcome: hierarchical assessment of 53 putative inflammatory biomarkers in an observational study

Background

Host inflammation contributes to determine whether SARS-CoV-2 infection causes mild or life-threatening disease. Tools are needed for early risk assessment.

Methods

We studied in 111 COVID-19 patients prospectively followed at a single reference Hospital fifty-three potential biomarkers including alarmins, cytokines, adipocytokines and growth factors, humoral innate immune and neuroendocrine molecules and regulators of iron metabolism. Biomarkers at hospital admission together with age, degree of hypoxia, neutrophil to lymphocyte ratio (NLR), lactate dehydrogenase (LDH), C-reactive protein (CRP) and creatinine were analysed within a data-driven approach to classify patients with respect to survival and ICU outcomes. Classification and regression tree (CART) models were used to identify prognostic biomarkers.

Results

Among the fifty-three potential biomarkers, the classification tree analysis selected CXCL10 at hospital admission, in combination with NLR and time from onset, as the best predictor of ICU transfer (AUC [95% CI] = 0.8374 [0.6233–0.8435]), while it was selected alone to predict death (AUC [95% CI] = 0.7334 [0.7547–0.9201]). CXCL10 concentration abated in COVID-19 survivors after healing and discharge from the hospital.

Conclusions

CXCL10 results from a data-driven analysis, that accounts for presence of confounding factors, as the most robust predictive biomarker of patient outcome in COVID-19.

Graphic abstract

Supplementary Information

The online version contains supplementary material available at 10.1186/s10020-021-00390-4.

The Role of Sample Size to Attain Statistically Comparable Groups - A Required Data Preprocessing Step to Estimate Causal Effects With Observational Data

Background: Propensity score methods provide data preprocessing tools to remove selection bias and attain statistically comparable groups - the first requirement when attempting to estimate causal effects with observational data. Although guidelines exist on how to remove selection bias when groups in comparison are large, not much is known on how to proceed when one of the groups in comparison, for example, a treated group, is particularly small, or when the study also includes lots of observed covariates (relative to the treated group's sample size). Objectives: This article investigates whether propensity score methods can help us to remove selection bias in studies with small treated groups and large amount of observed covariates. Measures: We perform a series of simulation studies to study factors such as sample size ratio of control to treated units, number of observed covariates and initial imbalances in observed covariates between the groups of units in comparison, that is, selection bias. Results: The results demonstrate that selection bias can be removed with small treated samples, but under different conditions than in studies with large treated samples. For example, a study design with 10 observed covariates and eight treated units will require the control group to be at least 10 times larger than the treated group, whereas a study with 500 treated units will require at least, only, two times bigger control group. Conclusions: To confirm the usefulness of simulation study results for practice, we carry out an empirical evaluation with real data. The study provides insights for practice and directions for future research.

Comparison of machine learning methods for estimating case fatality ratios: An Ebola outbreak simulation study

BACKGROUND

Machine learning (ML) algorithms are now increasingly used in infectious disease epidemiology. Epidemiologists should understand how ML algorithms behave within the context of outbreak data where missingness of data is almost ubiquitous.

METHODS

Using simulated data, we use a ML algorithmic framework to evaluate data imputation performance and the resulting case fatality ratio (CFR) estimates, focusing on the scale and type of data missingness (i.e., missing completely at random-MCAR, missing at random-MAR, or missing not at random-MNAR).

RESULTS

Across ML methods, dataset sizes and proportions of training data used, the area under the receiver operating characteristic curve decreased by 7% (median, range: 1%-16%) when missingness was increased from 10% to 40%. Overall reduction in CFR bias for MAR across methods, proportion of missingness, outbreak size and proportion of training data was 0.5% (median, range: 0%-11%).

CONCLUSION

ML methods could reduce bias and increase the precision in CFR estimates at low levels of missingness. However, no method is robust to high percentages of missingness. Thus, a datacentric approach is recommended in outbreak settings-patient survival outcome data should be prioritised for collection and random-sample follow-ups should be implemented to ascertain missing outcomes.

Heterogeneity by global and textural feature analysis in F-18 FP-CIT brain PET images for diagnosis of Parkinson's disease

BACKGROUND

The quantification of heterogeneity for the striatum and whole brain with F-18 FP-CIT PET images will be useful for diagnosis. The index obtained from texture analysis on PET images is related to pathological change that the neuronal loss of the nigrostriatal tract is heterogeneous according to the disease state. The aim of this study is to evaluate various heterogeneity indices of F-18 FP-CIT PET images in the diagnosis of Parkinson's disease (PD) patients and to access the diagnostic accuracy of the indices using machine learning (ML).

METHODS

This retrospective study included F-18 FP-CIT PET images of 31 PD and 31 age-matched health controls (HC). The volume of interest was delineated according to iso-contour lines around standardized uptake value (SUV) 3.0 g/ml for each region of the striatum by PMod 3.603. One hundred eight heterogeneity indices were calculated using CGITA to find indices from which the PD and HC were classified using statistical significance. PD group was classified by constructing a 2-dimensional or 3-dimensional phase space quantifier using these heterogeneity indices. We used 71 heterogeneity indices to classify PD from HC using ML for dimensional reduction.

RESULTS

The heterogeneity indices for classifying PD from HC were size-zone variability, contrast, inverse difference-moment, and homogeneity in the order of low P value. Three-dimensional quantifiers composed of normalized-contrast, code-similarity, and contrast were more clearly classified than 2-dimensional ones. After 71-dimensional reduction using PCA, classification was possible by logistic regression with 91.3% accuracy. The 2 groups were classified with an accuracy of 85.5% using the support vector machine and 88.4% using the random forest. The classification accuracy using the eXtreme Gradient Boosting was 95.7%, and feature importance was highest in order of SUV bias-corrected kurtosis, size-zone-variability, intensity-variability, and high-intensity-zone-variability.

CONCLUSION

It was confirmed that PD patients is more clearly classified than the conventional 2-dimensional quantifier by introducing a 3-dimensional phase space quantifier. We observed that ML can be used to classify the 2 groups in an easy and explanatory manner. For the discrimination of the disease, 24 heterogeneity indices were found to be statistically useful, and the major cut-off values of 3 heterogeneity indices were size-zone variability (1906.44), intensity variability (129.21), and high intensity zone emphasis (800.29).

An introduction to inverse probability of treatment weighting in observational research

In this article we introduce the concept of inverse probability of treatment weighting (IPTW) and describe how this method can be applied to adjust for measured confounding in observational research, illustrated by a clinical example from nephrology. IPTW involves two main steps. First, the probability—or propensity—of being exposed to the risk factor or intervention of interest is calculated, given an individual’s characteristics (i.e. propensity score). Second, weights are calculated as the inverse of the propensity score. The application of these weights to the study population creates a pseudopopulation in which confounders are equally distributed across exposed and unexposed groups. We also elaborate on how weighting can be applied in longitudinal studies to deal with informative censoring and time-dependent confounding in the setting of treatment-confounder feedback.

Uncovering Sociological Effect Heterogeneity Using Tree-Based Machine Learning

Individuals do not respond uniformly to treatments, such as events or interventions. Sociologists routinely partition samples into subgroups to explore how the effects of treatments vary by selected covariates, such as race and gender, on the basis of theoretical priors. Data-driven discoveries are also routine, yet the analyses by which sociologists typically go about them are often problematic and seldom move us beyond our biases to explore new meaningful subgroups. Emerging machine learning methods based on decision trees allow researchers to explore sources of variation that they may not have previously considered or envisaged. In this article, the authors use tree-based machine learning, that is, causal trees, to recursively partition the sample to uncover sources of effect heterogeneity. Assessing a central topic in social inequality, college effects on wages, the authors compare what is learned from covariate and propensity score-based partitioning approaches with recursive partitioning based on causal trees. Decision trees, although superseded by forests for estimation, can be used to uncover subpopulations responsive to treatments. Using observational data, the authors expand on the existing causal tree literature by applying leaf-specific effect estimation strategies to adjust for observed confounding, including inverse propensity weighting, nearest neighbor matching, and doubly robust causal forests. We also assess localized balance metrics and sensitivity analyses to address the possibility of differential imbalance and unobserved confounding. The authors encourage researchers to follow similar data exploration practices in their work on variation in sociological effects and offer a straightforward framework by which to do so.

Thirteen Questions About Using Machine Learning in Causal Research (You Won't Believe the Answer to Number 10!)

Machine learning is gaining prominence in the health sciences, where much of its use has focused on data-driven prediction. However, machine learning can also be embedded within causal analyses, potentially reducing biases arising from model misspecification. Using a question-and-answer format, we provide an introduction and orientation for epidemiologists interested in using machine learning but concerned about potential bias or loss of rigor due to use of "black box" models. We conclude with sample software code that may lower the barrier to entry to using these techniques.

Cervical cancer risk and access: Utilizing three statistical tools to assess Haitian women in South Florida

Although decreasing rates of cervical cancer in the U.S. are attributable to health policy, immigrant women, particularly Haitians, experience disproportionate disease burden related to delayed detection and treatment. However, risk prediction and dynamics of access remain largely underexplored and unresolved in this population. This study seeks to assess cervical cancer risk and access of unscreened Haitian women. Extracted and merged from two studies, this sample includes n = 346 at-risk Haitian women in South Florida, the largest U.S. enclave of Haitians (ages 30–65 and unscreened in the previous three years). Three approaches (logistic regression [LR]; classification and regression trees [CART]; and random forest [RF]) were employed to assess the association between screening history and sociodemographic variables. LR results indicated women who reported US citizenship (OR = 3.22, 95% CI = 1.52–6.84), access to routine care (OR = 2.11, 95%CI = 1.04–4.30), and spent more years in the US (OR = 1.01, 95%CI = 1.00–1.03) were significantly more likely to report previous screening. CART results returned an accuracy of 0.75 with a tree initially splitting on women who were not citizens, then on 43 or fewer years in the U.S., and without access to routine care. RF model identified U.S. years, citizenship, and access to routine care as variables of highest importance indicated by greatest mean decreases in Gini index. The model was .79 accurate (95% CI = 0.74–0.84). This multi-pronged analysis identifies previously undocumented barriers to health screening for Haitian women. Recent US immigrants without citizenship or perceived access to routine care may be at higher risk for disease due to barriers in accessing U.S. health-systems.

Propensity Score Analysis with Partially Observed Baseline Covariates: A Practical Comparison of Methods for Handling Missing Data

(1) Background: Propensity score methods gained popularity in non-interventional clinical studies. As it may often occur in observational datasets, some values in baseline covariates are missing for some patients. The present study aims to compare the performances of popular statistical methods to deal with missing data in propensity score analysis. (2) Methods: Methods that account for missing data during the estimation process and methods based on the imputation of missing values, such as multiple imputations, were considered. The methods were applied on the dataset of an ongoing prospective registry for the treatment of unprotected left main coronary artery disease. The performances were assessed in terms of the overall balance of baseline covariates. (3) Results: Methods that explicitly deal with missing data were superior to classical complete case analysis. The best balance was observed when propensity scores were estimated with a method that accounts for missing data using a stochastic approximation of the expectation-maximization algorithm. (4) Conclusions: If missing at random mechanism is plausible, methods that use missing data to estimate propensity score or impute them should be preferred. Sensitivity analyses are encouraged to evaluate the implications methods used to handle missing data and estimate propensity score.

Using Propensity Scores for Causal Inference: Pitfalls and Tips

Methods based on propensity score (PS) have become increasingly popular as a tool for causal inference. A better understanding of the relative advantages and disadvantages of the alternative analytic approaches can contribute to the optimal choice and use of a specific PS method over other methods. In this article, we provide an accessible overview of causal inference from observational data and two major PS-based methods (matching and inverse probability weighting), focusing on the underlying assumptions and decision-making processes. We then discuss common pitfalls and tips for applying the PS methods to empirical research and compare the conventional multivariable outcome regression and the two alternative PS-based methods (ie, matching and inverse probability weighting) and discuss their similarities and differences. Although we note subtle differences in causal identification assumptions, we highlight that the methods are distinct primarily in terms of the statistical modeling assumptions involved and the target population for which exposure effects are being estimated.

Reflection on modern methods: when worlds collide-prediction, machine learning and causal inference

Causal inference requires theory and prior knowledge to structure analyses, and is not usually thought of as an arena for the application of prediction modelling. However, contemporary causal inference methods, premised on counterfactual or potential outcomes approaches, often include processing steps before the final estimation step. The purposes of this paper are: (i) to overview the recent emergence of prediction underpinning steps in contemporary causal inference methods as a useful perspective on contemporary causal inference methods, and (ii) explore the role of machine learning (as one approach to 'best prediction') in causal inference. Causal inference methods covered include propensity scores, inverse probability of treatment weights (IPTWs), G computation and targeted maximum likelihood estimation (TMLE). Machine learning has been used more for propensity scores and TMLE, and there is potential for increased use in G computation and estimation of IPTWs.

Perspective: Big Data and Machine Learning Could Help Advance Nutritional Epidemiology

The field of nutritional epidemiology faces challenges posed by measurement error, diet as a complex exposure, and residual confounding. The objective of this perspective article is to highlight how developments in big data and machine learning can help address these challenges. New methods of collecting 24-h dietary recalls and recording diet could enable larger samples and more repeated measures to increase statistical power and measurement precision. In addition, use of machine learning to automatically classify pictures of food could become a useful complimentary method to help improve precision and validity of dietary measurements. Diet is complex due to thousands of different foods that are consumed in varying proportions, fluctuating quantities over time, and differing combinations. Current dietary pattern methods may not integrate sufficient dietary variation, and most traditional modeling approaches have limited incorporation of interactions and nonlinearity. Machine learning could help better model diet as a complex exposure with nonadditive and nonlinear associations. Last, novel big data sources could help avoid unmeasured confounding by offering more covariates, including both omics and features derived from unstructured data with machine learning methods. These opportunities notwithstanding, application of big data and machine learning must be approached cautiously to ensure quality of dietary measurements, avoid overfitting, and confirm accurate interpretations. Greater use of machine learning and big data would also require substantial investments in training, collaborations, and computing infrastructure. Overall, we propose that judicious application of big data and machine learning in nutrition science could offer new means of dietary measurement, more tools to model the complexity of diet and its relations with diseases, and additional potential ways of addressing confounding.

Assessing Heterogeneity of Treatment Effects in Observational Studies

Here we describe methods for assessing heterogeneity of treatment effects over prespecified subgroups in observational studies, using outcome-model-based (g-formula), inverse probability weighting, doubly robust, and matching estimators of subgroup-specific potential outcome means, conditional average treatment effects, and measures of heterogeneity of treatment effects. We compare the finite-sample performance of different estimators in simulation studies where we vary the total sample size, the relative frequency of each subgroup, the magnitude of treatment effect in each subgroup, and the distribution of baseline covariates, for both continuous and binary outcomes. We find that the estimators' bias and variance vary substantially in finite samples, even when there is no unobserved confounding and no model misspecification. As an illustration, we apply the methods to data from the Coronary Artery Surgery Study (August 1975-December 1996) to compare the effect of surgery plus medical therapy with that of medical therapy alone for chronic coronary artery disease in subgroups defined by previous myocardial infarction or left ventricular ejection fraction.

Propensity score matching with R: conventional methods and new features

It is increasingly important to accurately and comprehensively estimate the effects of particular clinical treatments. Although randomization is the current gold standard, randomized controlled trials (RCTs) are often limited in practice due to ethical and cost issues. Observational studies have also attracted a great deal of attention as, quite often, large historical datasets are available for these kinds of studies. However, observational studies also have their drawbacks, mainly including the systematic differences in baseline covariates, which relate to outcomes between treatment and control groups that can potentially bias results. Propensity score methods, which are a series of balancing methods in these studies, have become increasingly popular by virtue of the two major advantages of dimension reduction and design separation. Within this approach, propensity score matching (PSM) has been empirically proven, with outstanding performances across observational datasets. While PSM tutorials are available in the literature, there is still room for improvement. Some PSM tutorials provide step-by-step guidance, but only one or two packages have been covered, thereby limiting their scope and practicality. Several articles and books have expounded upon propensity scores in detail, exploring statistical principles and theories; however, the lack of explanations on function usage in programming language has made it difficult for researchers to understand and follow these materials. To this end, this tutorial was developed with a six-step PSM framework, in which we summarize the recent updates and provide step-by-step guidance to the R programming language. This tutorial offers researchers with a broad survey of PSM, ranging from data preprocessing to estimations of propensity scores, and from matching to analyses. We also explain generalized propensity scoring for multiple or continuous treatments, as well as time-dependent PSM. Lastly, we discuss the advantages and disadvantages of propensity score methods.

Machine Learning for Causal Inference: On the Use of Cross-fit Estimators

BACKGROUND

Modern causal inference methods allow machine learning to be used to weaken parametric modeling assumptions. However, the use of machine learning may result in complications for inference. Doubly robust cross-fit estimators have been proposed to yield better statistical properties.

METHODS

We conducted a simulation study to assess the performance of several different estimators for the average causal effect. The data generating mechanisms for the simulated treatment and outcome included log-transforms, polynomial terms, and discontinuities. We compared singly robust estimators (g-computation, inverse probability weighting) and doubly robust estimators (augmented inverse probability weighting, targeted maximum likelihood estimation). We estimated nuisance functions with parametric models and ensemble machine learning separately. We further assessed doubly robust cross-fit estimators.

RESULTS

With correctly specified parametric models, all of the estimators were unbiased and confidence intervals achieved nominal coverage. When used with machine learning, the doubly robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.

CONCLUSIONS

Due to the difficulty of properly specifying parametric models in high-dimensional data, doubly robust estimators with ensemble learning and cross-fitting may be the preferred approach for estimation of the average causal effect in most epidemiologic studies. However, these approaches may require larger sample sizes to avoid finite-sample issues.

Randomized clinical trials of cardiovascular disease in obstructive sleep apnea: understanding and overcoming bias

Three recent randomized control trials (RCTs) found that treatment of obstructive sleep apnea (OSA) with continuous positive airway pressure (CPAP) did not reduce rates of future cardiovascular events. This article discusses the biases in these RCTs that may explain their negative results, and how to overcome these biases in future studies. First, sample selection bias affected each RCT. The subjects recruited were not patients typically presenting for treatment of OSA. In particular, subjects with excessive sleepiness were excluded due to ethical concerns. As recent data indicate that the excessively sleepy OSA subtype has increased cardiovascular risk, subjects most likely to benefit from treatment were excluded. Second, RCTs had low adherence to therapy. Reported adherence is lower than found clinically, suggesting it is in part related to selection bias. Each RCT showed a CPAP benefit consistent with epidemiological studies when restricting to adherent patients, but was underpowered. Future studies need to include sleepy individuals and maximize adherence. Since it is unethical and impractical to randomize very sleepy subjects to no therapy, alternative designs are required. Observational designs using propensity scores, which are accepted by FDA for studies of medical devices, provide an opportunity. The design needs to ensure covariate balance, including measures assessing healthy user and healthy adherer biases, between regular users of CPAP and non-users. Sensitivity analyses can evaluate the robustness of results to unmeasured confounding, thereby improving confidence in conclusions. Thus, these designs can robustly assess the cardiovascular benefit of CPAP in real-world patients, overcoming biases in RCTs.

A comparison of reweighting estimators of average treatment effects in real world populations

Regulatory agencies typically evaluate the efficacy and safety of new interventions and grant commercial approval based on randomized controlled trials (RCTs). Other major healthcare stakeholders, such as insurance companies and health technology assessment agencies, while basing initial access and reimbursement decisions on RCT results, are also keenly interested in whether results observed in idealized trial settings will translate into comparable outcomes in real world settings-that is, into so-called "real world" effectiveness. Unfortunately, evidence of real world effectiveness for new interventions is not available at the time of initial approval. To bridge this gap, statistical methods are available to extend the estimated treatment effect observed in a RCT to a target population. The generalization is done by weighting the subjects who participated in a RCT so that the weighted trial population resembles a target population. We evaluate a variety of alternative estimation and weight construction procedures using both simulations and a real world data example using two clinical trials of an investigational intervention for Alzheimer's disease. Our results suggest an optimal approach to estimation depends on the characteristics of source and target populations, including degree of selection bias and treatment effect heterogeneity.

Neural Networks to Estimate Generalized Propensity Scores for Continuous Treatment Doses

BACKGROUND

The generalized propensity score (GPS) addresses selection bias due to observed confounding variables and provides a means to demonstrate causality of continuous treatment doses with propensity score analyses. Estimating the GPS with parametric models obliges researchers to meet improbable conditions such as correct model specification, normal distribution of variables, and large sample sizes.

OBJECTIVES

The purpose of this Monte Carlo simulation study is to examine the performance of neural networks as compared to full factorial regression models to estimate GPS in the presence of Gaussian and skewed treatment doses and small to moderate sample sizes.

RESEARCH DESIGN

A detailed conceptual introduction of neural networks is provided, as well as an illustration of selection of hyperparameters to estimate GPS. An example from public health and nutrition literature uses residential distance as a treatment variable to illustrate how neural networks can be used in a propensity score analysis to estimate a dose-response function of grocery spending behaviors.

RESULTS

We found substantially higher correlations and lower mean squared error values after comparing true GPS with the scores estimated by neural networks. The implication is that more selection bias was removed using GPS estimated with neural networks than using GPS estimated with classical regression.

CONCLUSIONS

This study proposes a new methodological procedure, neural networks, to estimate GPS. Neural networks are not sensitive to the assumptions of linear regression and other parametric models and have been shown to be a contender against parametric approaches to estimate propensity scores for continuous treatments.

Free Access to a Broad Contraceptive Method Mix and Women's Contraceptive Choice: Evidence from Sub-Saharan Africa

Financial barriers may restrict women's ability to use their preferred contraceptive methods, especially long-acting reversible contraceptives (LARC). Providing free access to a broad contraceptive method mix, including both LARC and short-acting reversible contraceptives (SARC), may increase contraceptive use, meet women's various fertility needs, and increase their agency in contraceptive decisions. Linking facility and individual data from eight countries in sub-Saharan Africa, we use a propensity score approach combined with machine learning techniques to examine how free access to a broad contraceptive method mix affects women's contraceptive choice. Free access to both LARC and SARC was associated with an increase of 3.2 percentage points (95 percent confidence interval: 0.006, 0.058) in the likelihood of contraceptive use, driven by greater use of SARC. Among contraceptive users, free access did not prompt women to switch to LARC and had no effect on contraceptive decision-making. The price effects were larger among older and more educated women, but free access was associated with lower contraceptive use among adolescents. While free access to contraceptives is associated with a modest increase in contraceptive use for some women, removing user fees alone does not address all barriers women face, especially for the most vulnerable groups of women.

Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation

Synthetic data provides a privacy protecting mechanism for the broad usage and sharing of healthcare data for secondary purposes. It is considered a safe approach for the sharing of sensitive data as it generates an artificial dataset that contains no identifiable information. Synthetic data is increasing in popularity with multiple synthetic data generators developed in the past decade, yet its utility is still a subject of research. This paper is concerned with evaluating the effect of various synthetic data generation and usage settings on the utility of the generated synthetic data and its derived models. Specifically, we investigate (i) the effect of data pre-processing on the utility of the synthetic data generated, (ii) whether tuning should be applied to the synthetic datasets when generating supervised machine learning models, and (iii) whether sharing preliminary machine learning results can improve the synthetic data models. Lastly, (iv) we investigate whether one utility measure (Propensity score) can predict the accuracy of the machine learning models generated from the synthetic data when employed in real life. We use two popular measures of synthetic data utility, propensity score and classification accuracy, to compare the different settings. We adopt a recent mechanism for the calculation of propensity, which looks carefully into the choice of model for the propensity score calculation. Accordingly, this paper takes a new direction with investigating the effect of various data generation and usage settings on the quality of the generated data and its ensuing models. The goal is to inform on the best strategies to follow when generating and using synthetic data.

Clinical Evidence for Locoregional Surgery of the Primary Tumor in Patients with De Novo Stage IV Breast Cancer

BACKGROUND

Whether primary tumor surgery is better than no surgery in patients with de novo stage IV breast cancer remains controversial.

METHODS

This study combined prospective clinical trials and a multicenter cohort to evaluate the impact of locoregional surgery in de novo stage IV breast cancer. The GRADE approach was used to assess the quality of evidence in meta-analysis, and propensity score matching analysis was used in the cohort study. This study was registered with PROSPERO CRD42016043766 and ClinicalTrials.gov NCT04456855.

RESULTS

A total of 1110 patients from six trials and 353 patients from the cohort study were included. The meta-analysis showed that compared with no surgery, locoregional surgery did not prolong overall survival (hazard ratio [HR] = 0.90, P = 0.40; moderate-quality) but had a significantly longer locoregional progression-free survival (HR = 0.23, P < 0.001; moderate-quality). The subgroup analysis of solitary bone-only metastasis (HR = 0.47, P = 0.04; high-quality) resulted in prolonged overall survival. In the cohort study, locoregional surgery showed a survival benefit (HR = 0.63, P = 0.041) before matching, but not (HR = 0.84, P = 0.579) after matching. Patients with bone-only metastasis showed a survival advantage in surgery compared with no surgery before matching (HR = 0.36, P = 0.034) as well as after matching (HR = 0.18, P = 0.017).

CONCLUSIONS

This study indicated that locoregional surgery had a significantly longer locoregional progression-free survival than no surgery in de novo stage IV breast cancer, and patients with bone-only metastasis tended to show an overall survival benefit from surgery.

Comorbidity Pattern Analysis for Predicting Amyotrophic Lateral Sclerosis

Electronic Medical Records (EMRs) can be used to create alerts for clinicians to identify patients at risk and to provide useful information for clinical decision-making support. In this study, we proposed a novel approach for predicting Amyotrophic Lateral Sclerosis (ALS) based on comorbidities and associated indicators using EMRs. The medical histories of ALS patients were analyzed and compared with those of subjects without ALS, and the associated comorbidities were selected as features for constructing the machine learning and prediction model. We proposed a novel weighted Jaccard index (WJI) that incorporates four different machine learning techniques to construct prediction systems. Alternative prediction models were constructed based on two different levels of comorbidity: single disease codes and clustered disease codes. With an accuracy of 83.7%, sensitivity of 78.8%, specificity of 85.7%, and area under the receiver operating characteristic curve (AUC) value of 0.907 for the single disease code level, the proposed WJI outperformed the traditional Jaccard index (JI) and scoring methods. Incorporating the proposed WJI into EMRs enabled the construction of a prediction system for analyzing the risk of suffering a specific disease based on comorbidity combinatorial patterns, which could provide a fast, low-cost, and noninvasive evaluation approach for early diagnosis of a specific disease.