Decision curve analysis to evaluate the clinical benefit of prediction models

There is increased interest in the use of prediction models to guide clinical decision-making in orthopedics. Prediction models are typically evaluated in terms of their accuracy: discrimination (area-under-the-curve [AUC] or concordance index) and calibration (a plot of predicted vs. observed risk). But it can be hard to know how high an AUC has to be in order to be "high enough" to warrant use of a prediction model, or how much miscalibration would be disqualifying. Decision curve analysis was developed as a method to determine whether use of a prediction model in the clinic to inform decision-making would do more good than harm. Here we give a brief introduction to decision curve analysis, explaining the critical concepts of net benefit and threshold probability. We briefly review some prediction models reported in the orthopedic literature, demonstrating how use of decision curves has allowed conclusions as to the clinical value of a prediction model. Conversely, papers without decision curves were unable to address questions of clinical value. We recommend increased use of decision curve analysis to evaluate prediction models in the orthopedics literature.

Predicting Progression Patterns of Type 2 Diabetes using Multi-sensor Measurements

Type 2 diabetes - a prevalent chronic disease worldwide - increases risk for serious health consequences including heart and kidney disease. Forecasting diabetes progression can inform disease management strategies, thereby potentially reducing the likelihood or severity of its consequences. We use continuous glucose monitoring and actigraphy data from 54 individuals with Type 2 diabetes to predict their future hemoglobin A1c, HDL cholesterol, LDL cholesterol, and triglyceride levels one year later. We use a combination of convolutional and recurrent neural networks to develop a deep neural network architecture that can learn the dynamic patterns in different sensors' data and combine those patterns with additional demographic and lab data. To further demonstrate the generalizability of our models, we also evaluate their performance using an independent public dataset of individuals with Type 1 diabetes. In addition to diabetes, our approach could be useful for other serious and chronic physical illness, where dynamic (e.g., from multiple sensors) and static (e.g., demographic) data are used for creating predictive models.

Predictive models for response to non-invasive brain stimulation in stroke: A critical review of opportunities and pitfalls

BACKGROUND

Noninvasive brain stimulation has been successfully applied to improve stroke-related impairments in different behavioral domains. Yet, clinical translation is limited by heterogenous outcomes within and across studies. It has been proposed to develop and apply noninvasive brain stimulation in a patient-tailored, precision medicine-guided fashion to maximize response rates and effect magnitude. An important prerequisite for this task is the ability to accurately predict the expected response of the individual patient.

OBJECTIVE

This review aims to discuss current approaches studying noninvasive brain stimulation in stroke and challenges associated with the development of predictive models of responsiveness to noninvasive brain stimulation.

METHODS

Narrative review.

RESULTS

Currently, the field largely relies on in-sample associational studies to assess the impact of different influencing factors. However, the associational approach is not valid for making claims of prediction, which generalize out-of-sample. We will discuss crucial requirements for valid predictive modeling in particular the presence of sufficiently large sample sizes.

CONCLUSION

Modern predictive models are powerful tools that must be wielded with great care. Open science, including data sharing across research units to obtain sufficiently large and unbiased samples, could provide a solid framework for addressing the task of building robust predictive models for noninvasive brain stimulation responsiveness.

Predicting length of stay and mortality among hospitalized patients with type 2 diabetes mellitus and hypertension

BACKGROUND

Type 2 diabetes mellitus (T2DM) and hypertension (HTN), both non-communicable diseases, are leading causes of death globally, with more imbalances in lower middle-income countries. Furthermore, poor treatment and management are known to lead to intensified healthcare utilization and increased medical care costs and impose a significant societal burden, in these countries, including Indonesia. Predicting future clinical outcomes can determine the line of treatment and value of healthcare costs, while ensuring effective patient care. In this paper, we present the prediction of length of stay (LoS) and mortality among hospitalized patients at a tertiary referral hospital in Tasikmalaya, Indonesia, between 2016 and 2019. We also aimed to determine how socio-demographic characteristics, and T2DM- or HTN-related comorbidities affect inpatient LoS and mortality.

METHODS

We analyzed insurance claims data of 4376 patients with T2DM or HTN hospitalized in the referral hospital. We used four prediction models based on machine-learning algorithms for LoS prediction, in relation to disease severity, physician-in-charge, room type, co-morbidities, and types of procedures performed. We used five classifiers based on multilayer perceptron (MLP) to predict inpatient mortality and compared them according to training time, testing time, and Area under Receiver Operative Curve (AUROC). Classifier accuracy measures, which included positive predictive value (PPV), negative predictive value (NPV), F-Measure, and recall, were used as performance evaluation methods.

RESULTS

A Random forest best predicted inpatient LoS (R2, 0.70; root mean square error [RMSE], 1.96; mean absolute error [MAE], 0.935), and the gradient boosting regression model also performed similarly (R2, 0.69; RMSE, 1.96; MAE, 0.935). For inpatient mortality, best results were observed using MLP with back propagation (AUROC 0.899; 69.33 and 98.61 for PPV and NPV, respectively). The other classifiers, stochastic gradient descent with regression loss function, Huber, and random forest models all showed an average performance.

CONCLUSIONS

Linear regression model best predicted LoS and mortality was best predicted using MLP. Patients with primary diseases such as T2DM or HTN may have comorbidities that can prolong inpatient LoS. Physicians play an important role in disseminating health related information. These predictions could assist in the development of health policies and strategies that reduce disease burden in resource-limited settings.

Decision tree analysis for pathogen identification based on circumstantial factors in outbreaks of bovine respiratory disease in calves

Respiratory tract infections continue to be a leading cause of economic loss, hampered animal welfare and intensive antimicrobial use in cattle operations, worldwide. To better target antimicrobial therapy, control and prevention towards the involved pathogens, there is a growing interest in microbiological tests on respiratory samples. However, these tests are time consuming, cost money and sampling might compromise animal welfare. Therefore, the objective of the present study was to develop immediately applicable decision trees for pathogen identification in outbreaks of bovine respiratory disease based on circumstantial factors. Data from a cross sectional study, involving 201 outbreaks of bovine respiratory disease in dairy and beef farms between 2016 and 2019 was used. Pathogens were identified by a semi-quantitative PCR (polymerase chain reaction) on a pooled non-endoscopic broncho-alveolar lavage sample from clinically affected animals. Potential risk factors of involved animals, environment, management and housing were obtained by enquiry. Classification and regression tree analysis was used for decision tree development with cross-validation. Different trees were constructed, involving a general 3-group classification tree (viruses, Mycoplasma bovis or Pasteurellaceae family) and a tree for each single pathogen. The general 3- group classification tree was 52.7 % accurate and had a sensitivity of 81.5 % and a specificity 52.2 % for viruses, respectively 51.7 % and 84.4 % for M. bovis and 28.9 % and 93.6 % for Pasteurellaceae. The single-pathogen trees were more specific than sensitive: Histophilus somni (Se = 25.8 %; Sp = 94.5 %), Mannheimia haemolytica (Se = 69.2 %; Sp = 70.6 %), bovine coronavirus (Se = 42.2 %; Sp = 89.6 %) and bovine respiratory syncytial virus (Se = 34.0 %; Sp = 96.6 %). For Pasteurella multocida, M. bovis and parainfluenzavirus type 3 no meaningful tree was obtained. The concept and trees are promising, but currently lack sensitivity and specificity in order to be a reliable tool for practice. For now, the obtained trees can already be informative for decision making to some extend depending on the end node in which an outbreak falls.

Utility of machine learning algorithms in degenerative cervical and lumbar spine disease: a systematic review

Machine learning is a rapidly evolving field that offers physicians an innovative and comprehensive mechanism to examine various aspects of patient data. Cervical and lumbar degenerative spine disorders are commonly age-related disease processes that can utilize machine learning to improve patient outcomes with careful patient selection and intervention. The aim of this study is to examine the current applications of machine learning in cervical and lumbar degenerative spine disease. A systematic review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A search of PubMed, Embase, Medline, and Cochrane was conducted through May 31st, 2020, using the following terms: "artificial intelligence" OR "machine learning" AND "neurosurgery" AND "spine." Studies were included if original research on machine learning was utilized in patient care for degenerative spine disease, including radiographic machine learning applications. Studies focusing on robotic applications in neurosurgery, navigation, or stereotactic radiosurgery were excluded. The literature search identified 296 papers, with 35 articles meeting inclusion criteria. There were nine studies involving cervical degenerative spine disease and 26 studies on lumbar degenerative spine disease. The majority of studies for both cervical and lumbar spines utilized machine learning for the prediction of postoperative outcomes, with 5 (55.6%) and 15 (61.5%) studies, respectively. Machine learning applications focusing on degenerative lumbar spine greatly outnumber the current volume of cervical spine studies. The current research in lumbar spine also demonstrates more advanced clinical applications of radiographic, diagnostic, and predictive machine learning models.

Household-level risk factors for Aedes aegypti pupal density in Guayaquil, Ecuador

Background

Vector-borne diseases are a major cause of disease burden in Guayaquil, Ecuador, especially arboviruses spread by Aedes aegypti mosquitoes. Understanding which household characteristics and risk factors lead to higher Ae. aegypti densities and consequent disease risk can help inform and optimize vector control programs.

Methods

Cross-sectional entomological surveys were conducted in Guayaquil between 2013 and 2016, covering household demographics, municipal services, potential breeding containers, presence of Ae. aegypti larvae and pupae, and history of using mosquito control methods. A zero-truncated negative binomial regression model was fitted to data for estimating the household pupal index. An additional model assessed the factors of the most productive breeding sites across all of the households.

Results

Of surveyed households, 610 satisfied inclusion criteria. The final household-level model found that collection of large solid items (e.g., furniture and tires) and rainfall the week of and 2 weeks before collection were negatively correlated with average pupae per container, while bed canopy use, unemployment, container water volume, and the interaction between large solid collection and rainfall 2 weeks before the sampling event were positively correlated. Selection of these variables across other top candidate models with ∆AICc < 1 was robust, with the strongest effects from large solid collection and bed canopy use. The final container-level model explaining the characteristics of breeding sites found that contaminated water is positively correlated with Ae. aegypti pupae counts while breeding sites composed of car parts, furniture, sewerage parts, vases, were all negatively correlated.

Conclusions

Having access to municipal services like bulky item pickup was effective at reducing mosquito proliferation in households. Association of bed canopy use with higher mosquito densities is unexpected, and may be a consequence of large local mosquito populations or due to limited use or effectiveness of other vector control methods. The impact of rainfall on mosquito density is multifaceted, as it may both create new habitat and “wash out” existing habitat. Providing services and social/technical interventions focused on monitoring and eliminating productive breeding sites is important for reducing aquatic-stage mosquito densities in households at risk for Ae. aegypti-transmitted diseases.

Graphical Abstract

Supplementary Information

The online version contains supplementary material available at 10.1186/s13071-021-04913-0.

Predictors of serious, preventable, and costly medical complications in a population of adult spinal deformity patients

BACKGROUND CONTEXT

In 2008, the Centers for Medicare and Medicaid Services (CMS) established a list of hospital-acquired conditions (HACs) with significant deleterious effects on both patients and providers. Adult spinal deformity (ASD) surgery is complex and highly invasive, and as such may result in significant morbidity including these HACs.

PURPOSE

Identify predictors for developing the most common HACs among adult spinal deformity (ASD) patients undergoing corrective surgery.

STUDY DESIGN/SETTING

Retrospective analysis.

PATIENT SAMPLE

One thousand one hundred and seventy-one ASD patients.

OUTCOME MEASURES

HACs, Health-Related Quality of Life scores(HRQLs), Reoperation, Integrated Health State (IHS) METHODS: ASD pts undergoing surgery (>18 years, scoliosis ≥20°, SVA ≥5 cm, PT ≥25° and/or TK >60°) with complete data at BL and up to 2 years post-op were included. Patients were stratified by presence of >1 HAC, defined as at least one superficial/deep SSI, UTI, DVT, or PE within a 30-day post-op window. Random forest analysis generated 5,000 Conditional Inference Trees to compute a variable importance table for top predictors of HACs. An area-under-the-curve (AUC) methodology compared normalized HRQL scores between groups to determine an IHS with 2-year follow-up.

RESULTS

Total of 1,171 pts (59.8 years, 76.2%F, 28.1kg/m²) underwent corrective ASD surgery, with 1,053 pts in the non-HAC group and 118 in the HAC group. Of these pts, 25.4% had UTI, 15.4% DVT, 19.2% superficial SSI, 20.8% deep SSI, and 19.2% PE. HAC pts were on average older (63.5 vs 59.3, p=.004) and more often frail (51.3 vs 39.7%, p=.021) than non-HAC pts. Postop LOS and reoperation were most associated with HAC groups: [1] LOS >7 days [2] reoperation. Patient-related predictors of HACs were [3] age >50 yerr, [4] frailty, and [13] BMI >31. Procedure-related predictors of HACs were [5] operative-time >405 minutes, [6] levels fused >9, EBL >1450 mL, and [11] decompression. BL radiographic predictors were [7] PT >20°, [9] PI-LL>6°, [10] TL Cobb angle >15°, [12] SVA C7-S1 >29 mm. No differences were observed between groups with regards to IHS ODI (0.73 vs 0.74, p=.863), SRS (1.3 vs1.3, p=.374), NRS Back (0.6 vs 0.6, p=.158). HAC had higher rates of reoperation than non-HAC (0.08 vs 0.01, p=.066), and any HAC within 30-days of index was a significant predictor of reoperation (OR: 2.448 [1.94-3.09], p<.001).

CONCLUSIONS

In a population of ASD patients, HACs were associated with length of stay, reoperation, age, and frailty. Radiographic parameters such as pelvic tilt >20°, PI-LL >6°, & SVA >29 mm also increased odds of HACs, and should raise postoperative awareness for HAC development.

Prognostic model of long-term advanced stage (IIIB-IV) EGFR mutated non-small cell lung cancer (NSCLC) survivors using real-life data

BACKGROUND

There is a lack of useful diagnostic tools to identify EGFR mutated NSCLC patients with long-term survival. This study develops a prognostic model using real world data to assist clinicians to predict survival beyond 24 months.

METHODS

EGFR mutated stage IIIB and IV NSCLC patients diagnosed between January 2009 and December 2017 included in the Spanish Lung Cancer Group (SLCG) thoracic tumor registry. Long-term survival was defined as being alive 24 months after diagnosis. A multivariable prognostic model was carried out using binary logistic regression and internal validation through bootstrapping. A nomogram was developed to facilitate the interpretation and applicability of the model.

RESULTS

505 of the 961 EGFR mutated patients identified in the registry were included, with a median survival of 27.73 months. Factors associated with overall survival longer than 24 months were: being a woman (OR 1.78); absence of the exon 20 insertion mutation (OR 2.77); functional status (ECOG 0-1) (OR 4.92); absence of central nervous system metastases (OR 2.22), absence of liver metastases (OR 1.90) or adrenal involvement (OR 2.35) and low number of metastatic sites (OR 1.22). The model had a good internal validation with a calibration slope equal to 0.781 and discrimination (optimism corrected C-index 0.680).

CONCLUSIONS

Survival greater than 24 months can be predicted from six pre-treatment clinicopathological variables. The model has a good discrimination ability. We hypothesized that this model could help the selection of the best treatment sequence in EGFR mutation NSCLC patients.

Predicting postoperative surgical site infection with administrative data: a random forests algorithm

Background

Since primary data collection can be time-consuming and expensive, surgical site infections (SSIs) could ideally be monitored using routinely collected administrative data. We derived and internally validated efficient algorithms to identify SSIs within 30 days after surgery with health administrative data, using Machine Learning algorithms.

Methods

All patients enrolled in the National Surgical Quality Improvement Program from the Ottawa Hospital were linked to administrative datasets in Ontario, Canada. Machine Learning approaches, including a Random Forests algorithm and the high-performance logistic regression, were used to derive parsimonious models to predict SSI status. Finally, a risk score methodology was used to transform the final models into the risk score system. The SSI risk models were validated in the validation datasets.

Results

Of 14,351 patients, 795 (5.5%) had an SSI. First, separate predictive models were built for three distinct administrative datasets. The final model, including hospitalization diagnostic, physician diagnostic and procedure codes, demonstrated excellent discrimination (C statistics, 0.91, 95% CI, 0.90–0.92) and calibration (Hosmer-Lemeshow χ² statistics, 4.531, p = 0.402).

Conclusion

We demonstrated that health administrative data can be effectively used to identify SSIs. Machine learning algorithms have shown a high degree of accuracy in predicting postoperative SSIs and can integrate and utilize a large amount of administrative data. External validation of this model is required before it can be routinely used to identify SSIs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01369-9.

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level cross-validation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions ~30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps < .001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

Quantitative Modeling of External Ventricular Drain Output to Predict Shunt Dependency in Aneurysmal Subarachnoid Hemorrhage: Cohort Study

BACKGROUND

Acute hydrocephalus is a common complication of aneurysmal subarachnoid hemorrhage (aSAH); however, attempts to predict shunt-dependent chronic hydrocephalus using clinical parameters have been equivocal.

METHODS

Cohort study of aSAH is treated with external ventricular drainage (EVD) placement at our institution, 2001-2016, via logistic regression. EVD-related parameters included mean/total EVD output (days 0-2), EVD days, EVD days ≤ 5 mmHg, and wean/clamp fails. aSAH outcomes assessed included ventriculoperitoneal shunt (VPS) placement, delayed cerebral ischemia (DCI), radiographic infarction (RI), symptomatic vasospasm (SV), age, and aSAH grades.

RESULTS

Two hundred and ten aSAH patients underwent EVD treatment for a median 12 days (range 1-54); 85 required VPS (40%). On univariate analysis, EVD output, total EVD days, EVD days ≤ 5 mmHg, and wean/clamp trial failures were significantly associated with VPS placement (p < 0.01 for all parameters). No EVD output parameter demonstrated a significant association with DCI, RI, or SV. On multivariate analysis, EVD output was a significant predictor of VPS placement, after adjusting for age and clinical and radiological grades; the optimal threshold for predicting VPS placement was mean daily output > 204 ml on days 0-2 (OR 2.59, 95% CI 1.31-5.07). Multiple wean failures were associated with unfavorable functional outcome, after adjusting for age, grade, and VPS placement (OR 1.65, 95% CI 1.10-2.47). We developed a score incorporating age, grade and EVD parameters (MAGE) for predicting VPS placement after aSAH.

CONCLUSIONS

EVD output parameters and wean/clamp trial failures predicted shunt dependence in an age- and grade-adjusted multivariable model. Early VPS placement may be warranted in patients with MAGE score ≥ 4, particularly following 2 failed wean trials.

Evaluation of the added value of viral genomic information for predicting severity of influenza infection

BACKGROUND

The severity of an influenza infection is influenced by both host and viral characteristics. This study aims to assess the relevance of viral genomic data for the prediction of severe influenza A(H3N2) infections among patients hospitalized for severe acute respiratory infection (SARI), in view of risk assessment and patient management.

METHODS

160 A(H3N2) influenza positive samples from the 2016-2017 season originating from the Belgian SARI surveillance were selected for whole genome sequencing. Predictor variables for severity were selected using a penalized elastic net logistic regression model from a combined host and genomic dataset, including patient information and nucleotide mutations identified in the viral genome. The goodness-of-fit of the model combining host and genomic data was compared using a likelihood-ratio test with the model including host data only. Internal validation of model discrimination was conducted by calculating the optimism-adjusted area under the Receiver Operating Characteristic curve (AUC) for both models.

RESULTS

The model including viral mutations in addition to the host characteristics had an improved fit ([Formula: see text]=12.03, df = 3, p = 0.007). The optimism-adjusted AUC increased from 0.671 to 0.732.

CONCLUSIONS

Adding genomic data (selected season-specific mutations in the viral genome) to the model containing host characteristics improved the prediction of severe influenza infection among hospitalized SARI patients, thereby offering the potential for translation into a prospective strategy to perform early season risk assessment or to guide individual patient management.

Improvement of APACHE II score system for disease severity based on XGBoost algorithm

BACKGROUND

Prognostication is an essential tool for risk adjustment and decision making in the intensive care units (ICUs). In order to improve patient outcomes, we have been trying to develop a more effective model than Acute Physiology and Chronic Health Evaluation (APACHE) II to measure the severity of the patients in ICUs. The aim of the present study was to provide a mortality prediction model for ICUs patients, and to assess its performance relative to prediction based on the APACHE II scoring system.

METHODS

We used the Medical Information Mart for Intensive Care version III (MIMIC-III) database to build our model. After comparing the APACHE II with 6 typical machine learning (ML) methods, the best performing model was screened for external validation on anther independent dataset. Performance measures were calculated using cross-validation to avoid making biased assessments. The primary outcome was hospital mortality. Finally, we used TreeSHAP algorithm to explain the variable relationships in the extreme gradient boosting algorithm (XGBoost) model.

RESULTS

We picked out 14 variables with 24,777 cases to form our basic data set. When the variables were the same as those contained in the APACHE II, the accuracy of XGBoost (accuracy: 0.858) was higher than that of APACHE II (accuracy: 0.742) and other algorithms. In addition, it exhibited better calibration properties than other methods, the result in the area under the ROC curve (AUC: 0.76). we then expand the variable set by adding five new variables to improve the performance of our model. The accuracy, precision, recall, F1, and AUC of the XGBoost model increased, and were still higher than other models (0.866, 0.853, 0.870, 0.845, and 0.81, respectively). On the external validation dataset, the AUC was 0.79 and calibration properties were good.

CONCLUSIONS

As compared to conventional severity scores APACHE II, our XGBoost proposal offers improved performance for predicting hospital mortality in ICUs patients. Furthermore, the TreeSHAP can help to enhance the understanding of our model by providing detailed insights into the impact of different features on the disease risk. In sum, our model could help clinicians determine prognosis and improve patient outcomes.

Predictive modeling of haloacetonitriles under uniform formation conditions

The objective of this study was to develop models to predict the formation of HANs under uniform formation conditions (UFC) in chlorinated, choraminated, and perchlorinated/chloraminated waters of different origins. Model equations were developed using multiple linear regression analysis to predict the formation of dichloroacetonitrile (DCAN), HAN4 (trichloroacetonitrile [TCAN], DCAN, bromochloroacetonitrile [BCAN], and dibromoacetonitrile [DBAN]) and HAN6 (HAN4 plus monochloroacetonitrile, monobromoacetonitrile). The independent variables covered a wide range of values, and included ultraviolet absorbance,(UV₂₅₄) dissolved organic carbon (DOC), dissolved organic nitrogen (DON), specific UV absorbance at 254 (SUVA₂₅₄), bromide (Br^-), pH, oxidant dose, contact time, and temperature. The regression coefficients (r²) of HAN4 and HAN6 models for natural organic matter (NOM), algal organic matter (AOM), and effluent organic matter (EfOM)  impacted waters were within the range of 60-88%, while the r² values of HAN4 and DCAN models for both groundwater and distribution systems were lower, in the range of 41-66%. The r² values for the DCAN model were mostly higher in the individual types as compared to the cumulative analysis of all source water data together. This was attributed to differences in HAN precursor characteristics. For chlorination, among all variables, pH was found to be the most significant descriptor in the model equations describing the formation of DCAN, HAN4, and HAN6, and it was negatively correlated with HAN formation in the distribution system, groundwater, AOM, and NOM samples, while it showed an inverse relationship with HAN6 formation in EfOM impacted waters. During chloramination, pH was the most influential model descriptor for DCAN formation in the NOM. Prechlorination dose was the most predominant parameter for prechlorination/chloramination, and it was positively correlated with HAN4 formation in AOM impacted waters.

A Novel, Potentially Universal Machine Learning Algorithm to Predict Complications in Total Knee Arthroplasty

BACKGROUND

There remains a lack of accurate and validated outcome-prediction models in total knee arthroplasty (TKA). While machine learning (ML) is a powerful predictive tool, determining the proper algorithm to apply across diverse data sets is challenging. AutoPrognosis (AP) is a novel method that uses automated ML framework to incorporate the best performing stages of prognostic modeling into a single well-calibrated algorithm. We aimed to compare various ML methods to AP in predictive performance of complications after TKA.

METHODS

Thirty-eight preoperative patient demographics and clinical features from all primary TKAs performed at California-licensed hospitals between 2015 and 2017 were evaluated as predictors of major complications after TKA. Traditional logistic regression (LR), various other ML methods (XGBoost, Gradient Boosting, AdaBoost, and Random Forest), and AP were used for model building to determine discriminative power (area under receiver operating curve), calibration (Brier score), and feature importance.

RESULTS

Between 2015 and 2017, there were a total of 156,750 TKAs with 1109 (0.7%) total major complications. AP had the highest discriminative performance with area under receiver operating curve 0.679 compared with LR, XGBoost, Gradient Boosting, AdaBoost, and Random Forest (0.617, 0.601, 0.662, 0.657, and 0.545, respectively). AP (Brier score 0.007) had similar calibration as the other ML methods (0.006, 0.006, 0.022, 0.007, and 0.008, respectively). The variables that are most important for AP differ from those that are most important for LR.

CONCLUSION

Compared to conventional ML algorithms, AP has superior discriminative ability with similar calibration and suggests nonlinear relationships between variables in outcomes of TKA.

A logistic analysis prediction model of TMJ condylar erosion in patients with TMJ arthralgia

BACKGROUND

In terms of diagnostic and therapeutic management, clinicians should adequately address the frequent aspects of temporomandibular joint (TMJ) osteoarthritis (OA) associated with disk displacement. Condylar erosion (CE) is considered an inflammatory subset of OA and is regarded as a sign of progressive OA changes potentially contributing to changes in dentofacial morphology or limited mandibular growth. The purpose of this study was to establish a risk prediction model of CE by a multivariate logistic regression analysis to predict the individual risk of CE in TMJ arthralgia. It was hypothesized that there was a closer association between CE and magnetic resonance imaging (MRI) indicators.

METHODS

This retrospective paired-design study enrolled 124 consecutive TMJ pain patients and analyzed the clinical and TMJ-related MRI data in predicting CE. TMJ pain patients were categorized according to the research diagnostic criteria for temporomandibular disorders (RDC/TMD) Axis I protocol. Each patient underwent MRI examination of both TMJs, 1-7 days following clinical examination.

RESULTS

In the univariate analysis analyses, 9 influencing factors were related to CE, of which the following 4 as predictors determined the binary multivariate logistic regression model: missing posterior teeth (odds ratio [OR] = 1.42; P = 0.018), RDC/TMD of arthralgia coexistant with disk displacement without reduction with limited opening (DDwoR/wLO) (OR = 3.30, P = 0.007), MRI finding of disk displacement without reduction (OR = 10.96, P < 0.001), and MRI finding of bone marrow edema (OR = 11.97, P < 0.001). The model had statistical significance (chi-square = 148.239, Nagelkerke R square = 0.612, P < 0.001). Out of the TMJs, 83.9% were correctly predicted to be CE cases or Non-CE cases with a sensitivity of 81.4% and a specificity of 85.2%. The area under the receiver operating characteristic curve was 0.916.

CONCLUSION

The established prediction model using the risk factors of TMJ arthralgia may be useful for predicting the risk of CE. The data suggest MRI indicators as dominant factors in the definition of CE. Further research is needed to improve the model, and confirm the validity and reliability of the model.

Predicting the Estrogen Receptor Activity of Environmental Chemicals by Single-Cell Image Analysis and Data-driven Modeling

A comprehensive evaluation of toxic chemicals and understanding their potential harm to human physiology is vital in mitigating their adverse effects following exposure from environmental emergencies. In this work, we develop data-driven classification models to facilitate rapid decision making in such catastrophic events and predict the estrogenic activity of environmental toxicants as estrogen receptor-α (ERα) agonists or antagonists. By combining high-content analysis, big-data analytics, and machine learning algorithms, we demonstrate that highly accurate classifiers can be constructed for evaluating the estrogenic potential of many chemicals. We follow a rigorous, high throughput microscopy-based high-content analysis pipeline to measure the single cell-level response of benchmark compounds with known in vivo effects on the ERα pathway. The resulting high-dimensional dataset is then pre-processed by fitting a non-central gamma probability distribution function to each feature, compound, and concentration. The characteristic parameters of the distribution, which represent the mean and the shape of the distribution, are used as features for the classification analysis via Random Forest (RF) and Support Vector Machine (SVM) algorithms. The results show that the SVM classifier can predict the estrogenic potential of benchmark chemicals with higher accuracy than the RF algorithm, which misclassifies two antagonist compounds.

Quantitative analysis of EEG reactivity for neurological prognostication after cardiac arrest

OBJECTIVE

To test whether 1) quantitative analysis of EEG reactivity (EEG-R) using machine learning (ML) is superior to visual analysis, and 2) combining quantitative analyses of EEG-R and EEG background pattern increases prognostic value for prediction of poor outcome after cardiac arrest (CA).

METHODS

Several types of ML models were trained with twelve quantitative features derived from EEG-R and EEG background data of 134 adult CA patients. Poor outcome was a Cerebral Performance Category score of 3-5 within 6 months.

RESULTS

The Random Forest (RF) trained on EEG-R showed the highest AUC of 83% (95-CI 80-86) of tested ML classifiers, predicting poor outcome with 46% sensitivity (95%-CI 40-51) and 89% specificity (95%-CI 86-92). Visual analysis of EEG-R had 80% sensitivity and 65% specificity. The RF was also the best classifier for EEG background (AUC 85%, 95%-CI 83-88) at 24 h after CA, with 62% sensitivity (95%-CI 57-67) and 84% specificity (95%-CI 79-88). Combining EEG-R and EEG background RF classifiers reduced the number of false positives.

CONCLUSIONS

Quantitative EEG-R using ML predicts poor outcome with higher specificity, but lower sensitivity compared to visual analysis of EEG-R, and is of some additional value to ML on EEG background data.

SIGNIFICANCE

Quantitative EEG-R using ML is a promising alternative to visual analysis and of some added value to ML on EEG background data.

Predictive modeling in neurocritical care using causal artificial intelligence

Artificial intelligence (AI) and digital twin models of various systems have long been used in industry to test products quickly and efficiently. Use of digital twins in clinical medicine caught attention with the development of Archimedes, an AI model of diabetes, in 2003. More recently, AI models have been applied to the fields of cardiology, endocrinology, and undergraduate medical education. The use of digital twins and AI thus far has focused mainly on chronic disease management, their application in the field of critical care medicine remains much less explored. In neurocritical care, current AI technology focuses on interpreting electroencephalography, monitoring intracranial pressure, and prognosticating outcomes. AI models have been developed to interpret electroencephalograms by helping to annotate the tracings, detecting seizures, and identifying brain activation in unresponsive patients. In this mini-review we describe the challenges and opportunities in building an actionable AI model pertinent to neurocritical care that can be used to educate the newer generation of clinicians and augment clinical decision making.

Inclusion of Frailty Improves Predictive Modeling for Postoperative Outcomes in Surgical Management of Primary and Secondary Lumbar Spine Tumors

BACKGROUND

Malignant spinal tumors are common, continually increasing in incidence as a function of improved survival times for patients with cancer. Using predictive analytics and propensity score matching, we evaluated the influence of frailty on postoperative complications compared with age in patients with malignant neoplasms of the lumbar spine.

METHODS

We used the Nationwide Readmissions Database from 2016 and 2017 to identify patients with malignant neoplasms of the lumbar spine who received a fusion procedure. Patient frailty was queried using the Johns Hopkins Adjusted Clinical Groups. Propensity score matching for age, sex, Charlson Comorbidity Index, surgical approach, and number of levels fused was implemented between frail and nonfrail patients, identifying 533 frail patients and 538 nonfrail patients. The area under the curve (AUC) of each ROC served as a proxy for model performance.

RESULTS

Frail patients reported significantly higher inpatient lengths of stay, costs, infection, posthemorrhagic anemia, and urinary tract infections (P < 0.05). In addition, frail patients were more often discharged to skilled nursing facilities and short-term hospitals compared with nonfrail patients (P < 0.0001). Regression models for mortality (AUC = 0.644), nonroutine discharge (AUC = 0.600), and acute infection (AUC = 0.666) were improved when using frailty as the primary predictor. These models were also improved using frailty when predicting 30-day readmission and 90-day hardware failure.

CONCLUSIONS

Frailty demonstrated a significant relationship with increased postoperative patient complications, length of stay, costs, and acute complications in patients receiving fusion following resection of a malignant neoplasm of the lumbar spine region. Frailty demonstrated better predictive validity of outcomes compared with patient age.

Predicting Mental Health Treatment Access Among Adolescents With Elevated Depressive Symptoms: Machine Learning Approaches

A large proportion of adolescents experiencing depression never access treatment. To increase access to effective mental health care, it is critical to understand factors associated with increased versus decreased odds of adolescent treatment access. This study used individual depression symptoms and sociodemographic variables to predict whether and where adolescents with depression accessed mental health treatments. We performed a pre-registered, secondary analysis of data from the 2017 National Survey of Drug Use and Health (NSDUH), a nationally representative sample of non-institutionalized civilians in the United States. Using four cross-validated random forest models, we predicted whether adolescents with elevated past-year depressive symptoms (N = 1,671; ages 12-17 years) accessed specific mental health treatments in the previous 12 months ("yes/no" for inpatient, outpatient, school, any). 53.38% of adolescents with elevated depressive symptoms accessed treatment of any kind. Even with depressive symptoms and sociodemographic factors included as predictors, pre-registered random forests explained < 0.00% of pseudo out-of-sample deviance in adolescent access to inpatient, outpatient, school, or overall treatments. Exploratory elastic net models explained 0.80-2.50% of pseudo out-of-sample deviance in adolescent treatment access across all four treatment types. Neither individual depressive symptoms nor any socioeconomic variables meaningfully predicted specific or overall mental health treatment access in adolescents with elevated past-year symptoms. This study highlights substantial limitations in our capacity to predict whether and where adolescents access mental health treatment and underscores the broader need for more accessible, scalable adolescent depression treatments.

Predicting youth at high risk of aging out of foster care using machine learning methods

BACKGROUND

Youth who exit the nation's foster care system without permanency are at high risk of experiencing difficulties during the transition to adulthood.

OBJECTIVE

To present an illustrative test of whether an algorithmic decision aid could be used to identify youth at risk of existing foster care without permanency.

METHODS

For youth placed in foster care between ages 12 and 14, we assessed the risk of exiting care without permanency by age 18 based on their child welfare service involvement history. To develop predictive risk models, 28 years (1991-2018) of child welfare service records from California were used. Performances were evaluated using F1, AUC, and precision and recall scores at k %. Algorithmic racial bias and fairness was also examined.

RESULTS

The gradient boosting decision tree and random forest showed the best performance (F1 score = .54-.55, precision score = .62, recall score = .49). Among the top 30 % of youth the model identified as high risk, half of all youth who exited care without permanency were accurately identified four to six years prior to their exit, with a 39 % error rate. Although racial disparities between Black and White youth were observed in imbalanced error rates, calibration and predictive parity were satisfied.

CONCLUSIONS

Our study illustrates the manner in which potential applications of predictive analytics, including those designed to achieve universal goals of permanency through more targeted allocations of resources, can be tested. It also assesses the model using metrics of fairness.

An intelligent payment card fraud detection system

Payment cards offer a simple and convenient method for making purchases. Owing to the increase in the usage of payment cards, especially in online purchases, fraud cases are on the rise. The rise creates financial risk and uncertainty, as in the commercial sector, it incurs billions of losses each year. However, real transaction records that can facilitate the development of effective predictive models for fraud detection are difficult to obtain, mainly because of issues related to confidentially of customer information. In this paper, we apply a total of 13 statistical and machine learning models for payment card fraud detection using both publicly available and real transaction records. The results from both original features and aggregated features are analyzed and compared. A statistical hypothesis test is conducted to evaluate whether the aggregated features identified by a genetic algorithm can offer a better discriminative power, as compared with the original features, in fraud detection. The outcomes positively ascertain the effectiveness of using aggregated features for undertaking real-world payment card fraud detection problems.