The derivation of an International Classification of Diseases, Tenth Revision-based trauma-related mortality model using machine learning

Prediction of mortality among severely injured trauma patients A comparison between TRISS and machine learning-based predictive models

BACKGROUND

Given the huge impact of trauma on hospital systems around the world, several attempts have been made to develop predictive models for the outcomes of trauma victims. The most used, and in many studies most accurate predictive model, is the "Trauma Score and Injury Severity Score" (TRISS). Although it has proven to be fairly accurate and is widely used, it has faced criticism for its inability to classify more complex cases. In this study, we aimed to develop machine learning models that better than TRISS could predict mortality among severely injured trauma patients, something that has not been studied using data from a nationwide register before.

METHODS

Patient data was collected from the national trauma register in Sweden, SweTrau. The studied period was from the 1st of January 2015 to 31st of December 2019. After feature selection and multiple imputation of missing data three machine learning (ML) methods (Random Forest, eXtreme Gradient Boosting, and a Generalized Linear Model) were used to create predictive models. The ML models and TRISS were then tested on predictive ability for 30-day mortality.

RESULTS

The ML models were well-calibrated and outperformed TRISS in all the tested measurements. Among the ML models, the eXtreme Gradient Boosting model performed best with an AUC of 0.91 (0.88-0.93).

CONCLUSION

This study showed that all the developed ML-based prediction models were superior to TRISS for the prediction of trauma mortality.

Contemporary National Incidence and Outcomes of Acute Limb Ischemia

INTRODUCTION

Acute Limb Ischemia (ALI) is a morbid and deadly diagnosis. However, existing epidemiologic studies describing ALI predate the introduction of the Affordable Care Act in 2010 and direct oral anticoagulants in 2011. Thus, we synergized the National Inpatient Sample (NIS) and United States (U.S.) Census to define contemporary trends in the incidence, treatment, and outcomes of ALI in the US.

METHODS

We included emergent admissions of adults with primary diagnosis of lower extremity ALI in survey-weighted NIS data (2005-2020). Mann-Kendal trend test evaluated ALI incidence (primary outcome), anticoagulation usage, insurance coverage, revascularization type, and in-hospital amputation/death. Multivariable logistic regression quantified covariate associations with in-hospital amputation/death.

RESULTS

Of 582,322,862 estimated hospitalizations in the NIS, 227,440 met inclusion criteria (mean age 68.80 years, 49.94% women, 76.66% White). ALI incidence peaked in 2006 (7.16/100,000 person-years) but has declined since 2015 to 4.16/100,000 person-years in 2020 (ptrend=.008). Endovascular revascularization, anticoagulation, and Medicaid coverage increased, while self-pay insurance decreased (ptrend<.05). Amputation rates significantly decreased from 8.04% to 6.54% (ptrend=.01) while death rate remained at 5.59% (ptrend=.16) over the study period. Pre-hospitalization anticoagulation was associated with decreased amputation (aOR=0.74 [95%CI 0.65-0.84]) and death (aOR=0.50 [95%CI 0.43-0.57]). When controlling for covariates, women had a higher risk of death (aOR=1.17 [95%CI 1.07-1.27], p<.0001), while Black patients had a higher risk of amputation (aOR=1.24 [95%CI 1.10-1.41], p<.0001).

CONCLUSIONS

Our U.S. population based epidemiological study demonstrates that ALI incidence and in-hospital amputation rates are decreasing, while mortality remains unchanged. We further highlight the ongoing need for ALI investigation specifically as it relates to access to care, antithrombotic therapy use, treatment strategy, and strategies to combat gender and racial disparities.

Percutaneous thrombectomy for acute limb ischemia is associated with equivalent limb and mortality outcomes compared with open thrombectomy

BACKGROUND

Acute limb ischemia (ALI) carries a 15% to 20% risk of combined death or amputation at 30 days and 50% to 60% at 1 year. Percutaneous mechanical thrombectomy (PT) is an emerging minimally invasive alternative to open thrombectomy (OT). However, ALI thrombectomy cases are omitted from most quality databases, limiting comparisons of limb and survival outcomes between PT and OT. Therefore, our aim was to compare in-hospital outcomes between PT and OT using the National Inpatient Sample.

METHODS

We analyzed survey-weighted National Inpatient Sample data (2015-2020) to include emergent admissions of aged adults (50+ years) with a primary diagnosis of lower extremity ALI undergoing index procedures within 2 days of hospitalization. We excluded hospitalizations with concurrent trauma or dissection diagnoses and index procedures using catheter-directed thrombolysis. Our primary outcome was composite in-hospital major amputation or death. Secondary outcomes included in-hospital major amputation, death, in-hospital reintervention (including angioplasty/stent, thrombolysis, PT, OT, or bypass), and extended length of stay (eLOS; defined as LOS >75th percentile). Adjusted odds ratios (aORs) with 95% confidence intervals (95% CIs) were generated by multivariable logistic regression, adjusting for demographics, frailty (Risk Analysis Index), secondary diagnoses including atrial fibrillation and peripheral artery disease, hospital characteristics, and index procedure data including the anatomic thrombectomy level and fasciotomy. A priori subgroup analyses were performed using interaction terms.

RESULTS

We included 23,795 survey-weighted ALI hospitalizations (mean age: 72.2 years, 50.4% female, 79.2% White, and 22.3% frail), with 7335 (30.8%) undergoing PT. Hospitalization characteristics for PT vs OT differed by atrial fibrillation (28.7% vs 36.5%, P < .0001), frequency of intervention at the femoropopliteal level (86.2% vs 88.8%, P = .009), and fasciotomy (4.8% vs 6.9%, P = .006). In total, 2530 (10.6%) underwent major amputation or died. Unadjusted (10.1% vs 10.9%, P = .43) and adjusted (aOR = 0.96 [95% CI, 0.77-1.20], P = .74) risk did not differ between the groups. PT was associated with increased odds of reintervention (aOR = 2.10 [95% CI, 1.72-2.56], P < .0001) when compared with OT, but this was not seen in the tibial subgroup (aOR = 1.31 [95% CI, 0.86-2.01], P = .21, Pinteraction < .0001). Further, 79.1% of PT hospitalizations undergoing reintervention were salvaged with endovascular therapy. Lastly, PT was associated with significantly decreased odds of eLOS (aOR = 0.80 [95% CI, 0.69-0.94], P = .005).

CONCLUSIONS

PT was associated with comparable in-hospital limb salvage and mortality rates compared with OT. Despite an increased risk of reintervention, most PT reinterventions avoided open surgery, and PT was associated with a decreased risk of eLOS. Thus, PT may be an appropriate alternative to OT in appropriately selected patients.

Identifying Age-Specific Risk Factors for Poor Outcomes After Trauma With Machine Learning

INTRODUCTION

Risk stratification for poor outcomes is not currently age-specific. Risk stratification of older patients based on observational cohorts primarily composed of young patients may result in suboptimal clinical care and inaccurate quality benchmarking. We assessed two hypotheses. First, we hypothesized that risk factors for poor outcomes after trauma are age-dependent and, second, that the relative importance of various risk factors are also age-dependent.

METHODS

A cohort study of severely injured adult trauma patients admitted to the intensive care unit 2014-2018 was performed using trauma registry data. Random forest algorithms predicting poor outcomes (death or complication) were built and validated using three cohorts: (1) patients of all ages, (2) younger patients, and (3) older patients. Older patients were defined as aged 55 y or more to maintain consistency with prior trauma literature. Complications assessed included acute renal failure, acute respiratory distress syndrome, cardiac arrest, unplanned intubation, unplanned intensive care unit admission, and unplanned return to the operating room, as defined by the trauma quality improvement program. Mean decrease in model accuracy (MDA), if each variable was removed and scaled to a Z-score, was calculated. MDA change ≥4 standard deviations between age cohorts was considered significant.

RESULTS

Of 5489 patients, 25% were older. Poor outcomes occurred in 12% of younger and 33% of older patients. Head injury was the most important predictor of poor outcome in all cohorts. In the full cohort, age was the most important predictor of poor outcomes after head injury. Within age cohorts, the most important predictors of poor outcomes, after head injury, were surgery requirement in younger patients and arrival Glasgow Coma Scale in older patients. Compared to younger patients, head injury and arrival Glasgow Coma Scale had the greatest increase in importance for older patients, while systolic blood pressure had the greatest decrease in importance.

CONCLUSIONS

Supervised machine learning identified differences in risk factors and their relative associations with poor outcomes based on age. Age-specific models may improve hospital benchmarking and identify quality improvement targets for older trauma patients.

Internal and external validation of an updated ICD-10-CA to AIS-2005 update 2008 algorithm

BACKGROUND

Administrative data are a powerful tool for population-level trauma research but lack the trauma-specific diagnostic and injury severity codes needed for risk-adjusted comparative analyses. The objective of this study was to validate an algorithm to derive Abbreviated Injury Scale (AIS-2005 update 2008) severity scores from Canadian International Classification of Diseases (ICD-10-CA) diagnostic codes in administrative data.

METHODS

This was a retrospective cohort study using data from the 2009 to 2017 Ontario Trauma Registry for the internal validation of the algorithm. This registry includes all patients treated at a trauma center who sustained a moderate or severe injury or were assessed by a trauma team. It contains both ICD-10-CA codes and injury scores assigned by expert abstractors. We used Cohen's kappa (𝜅) coefficient to compare AIS-2005 Update 2008 scores assigned by expert abstractors to those derived using the algorithm and the intraclass correlation coefficient to compare assigned and derived Injury Severity Scores. Sensitivity and specificity for detection of a severe injury (AIS score, ≥ 3) were then calculated. For the external validation of the algorithm, we used administration data to identify adults who either died in an emergency department or were admitted to hospital in Ontario secondary to a traumatic injury (2009-2017). Logistic regression was used to evaluate the discriminative ability and calibration of the algorithm.

RESULTS

Of 41,869 patients in the Ontario Trauma Registry, 41,793 (99.8%) had at least one diagnosis matched to the algorithm. Evaluation of AIS scores assigned by expert abstractors and those derived using the algorithm demonstrated a high degree of agreement in identification of patients with at least one severe injury (𝜅 = 0.75; 95% confidence interval [CI], 0.74-0.76). Likewise, algorithm-derived scores had a strong ability to rule in or out injury with AIS ≥ 3 (specificity, 78.5%; 95% CI, 77.7-79.4; sensitivity, 95.1; 95% CI, 94.8-95.3). There was strong correlation between expert abstractor-assigned and crosswalk-derived Injury Severity Score (intraclass correlation coefficient, 0.80; 95% CI, 0.80-0.81). Among the 130,542 patients identified using administrative data, the algorithm retained its discriminative properties.

CONCLUSION

Our ICD-10-CA to AIS-2005 update 2008 algorithm produces reliable estimates of injury severity and retains its discriminative properties with administrative data. Our findings suggest that this algorithm can be used for risk adjustment of injury outcomes when using population-based administrative data.

LEVEL OF EVIDENCE

Diagnostic Tests/Criteria; Level II.

Development and Validation of a Model to Quantify Injury Severity in Real Time

Importance

Quantifying injury severity is integral to trauma care benchmarking, decision-making, and research, yet the most prevalent metric to quantify injury severity-Injury Severity Score (ISS)- is impractical to use in real time.

Objective

To develop and validate a practical model that uses a limited number of injury patterns to quantify injury severity in real time through 3 intuitive outcomes.

Design, Setting, and Participants

In this cohort study for prediction model development and validation, training, development, and internal validation cohorts comprised 223 545, 74 514, and 74 514 admission encounters, respectively, of adults (age ≥18 years) with a primary diagnosis of traumatic injury hospitalized more than 2 days (2017-2018 National Inpatient Sample). The external validation cohort comprised 3855 adults admitted to a level I trauma center who met criteria for the 2 highest of the institution's 3 trauma activation levels.

Main Outcomes and Measures

Three outcomes were hospital length of stay, probability of discharge disposition to a facility, and probability of inpatient mortality. The prediction performance metric for length of stay was mean absolute error. Prediction performance metrics for discharge disposition and inpatient mortality were average precision, precision, recall, specificity, F1 score, and area under the receiver operating characteristic curve (AUROC). Calibration was evaluated using calibration plots. Shapley addictive explanations analysis and bee swarm plots facilitated model explainability analysis.

Results

The Length of Stay, Disposition, Mortality (LDM) Injury Index (the model) comprised a multitask deep learning model trained, developed, and internally validated on a data set of 372 573 traumatic injury encounters (mean [SD] age = 68.7 [19.3] years, 56.6% female). The model used 176 potential injuries to output 3 interpretable outcomes: the predicted hospital length of stay, probability of discharge to a facility, and probability of inpatient mortality. For the external validation set, the ISS predicted length of stay with mean absolute error was 4.16 (95% CI, 4.13-4.20) days. Compared with the ISS, the model had comparable external validation set discrimination performance (facility discharge AUROC: 0.67 [95% CI, 0.67-0.68] vs 0.65 [95% CI, 0.65-0.66]; recall: 0.59 [95% CI, 0.58-0.61] vs 0.59 [95% CI, 0.58-0.60]; specificity: 0.66 [95% CI, 0.66-0.66] vs 0.62 [95%CI, 0.60-0.63]; mortality AUROC: 0.83 [95% CI, 0.81-0.84] vs 0.82 [95% CI, 0.82-0.82]; recall: 0.74 [95% CI, 0.72-0.77] vs 0.75 [95% CI, 0.75-0.76]; specificity: 0.81 [95% CI, 0.81-0.81] vs 0.76 [95% CI, 0.75-0.77]). The model had excellent calibration for predicting facility discharge disposition, but overestimated inpatient mortality. Explainability analysis found the inputs influencing model predictions matched intuition.

Conclusions and Relevance

In this cohort study using a limited number of injury patterns, the model quantified injury severity using 3 intuitive outcomes. Further study is required to evaluate the model at scale.

Artificial intelligence and machine learning for hemorrhagic trauma care

Artificial intelligence (AI), a branch of machine learning (ML) has been increasingly employed in the research of trauma in various aspects. Hemorrhage is the most common cause of trauma-related death. To better elucidate the current role of AI and contribute to future development of ML in trauma care, we conducted a review focused on the use of ML in the diagnosis or treatment strategy of traumatic hemorrhage. A literature search was carried out on PubMed and Google scholar. Titles and abstracts were screened and, if deemed appropriate, the full articles were reviewed. We included 89 studies in the review. These studies could be grouped into five areas: (1) prediction of outcomes; (2) risk assessment and injury severity for triage; (3) prediction of transfusions; (4) detection of hemorrhage; and (5) prediction of coagulopathy. Performance analysis of ML in comparison with current standards for trauma care showed that most studies demonstrated the benefits of ML models. However, most studies were retrospective, focused on prediction of mortality, and development of patient outcome scoring systems. Few studies performed model assessment via test datasets obtained from different sources. Prediction models for transfusions and coagulopathy have been developed, but none is in widespread use. AI-enabled ML-driven technology is becoming integral part of the whole course of trauma care. Comparison and application of ML algorithms using different datasets from initial training, testing and validation in prospective and randomized controlled trials are warranted for provision of decision support for individualized patient care as far forward as possible.

Model for Predicting In-Hospital Mortality of Physical Trauma Patients Using Artificial Intelligence Techniques: Nationwide Population-Based Study in Korea

BACKGROUND

Physical trauma-related mortality places a heavy burden on society. Estimating the mortality risk in physical trauma patients is crucial to enhance treatment efficiency and reduce this burden. The most popular and accurate model is the Injury Severity Score (ISS), which is based on the Abbreviated Injury Scale (AIS), an anatomical injury severity scoring system. However, the AIS requires specialists to code the injury scale by reviewing a patient's medical record; therefore, applying the model to every hospital is impossible.

OBJECTIVE

We aimed to develop an artificial intelligence (AI) model to predict in-hospital mortality in physical trauma patients using the International Classification of Disease 10th Revision (ICD-10), triage scale, procedure codes, and other clinical features.

METHODS

We used the Korean National Emergency Department Information System (NEDIS) data set (N=778,111) compiled from over 400 hospitals between 2016 and 2019. To predict in-hospital mortality, we used the following as input features: ICD-10, patient age, gender, intentionality, injury mechanism, and emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and procedure codes. We proposed the ensemble of deep neural networks (EDNN) via 5-fold cross-validation and compared them with other state-of-the-art machine learning models, including traditional prediction models. We further investigated the effect of the features.

RESULTS

Our proposed EDNN with all features provided the highest area under the receiver operating characteristic (AUROC) curve of 0.9507, outperforming other state-of-the-art models, including the following traditional prediction models: Adaptive Boosting (AdaBoost; AUROC of 0.9433), Extreme Gradient Boosting (XGBoost; AUROC of 0.9331), ICD-based ISS (AUROC of 0.8699 for an inclusive model and AUROC of 0.8224 for an exclusive model), and KTAS (AUROC of 0.1841). In addition, using all features yielded a higher AUROC than any other partial features, namely, EDNN with the features of ICD-10 only (AUROC of 0.8964) and EDNN with the features excluding ICD-10 (AUROC of 0.9383).

CONCLUSIONS

Our proposed EDNN with all features outperforms other state-of-the-art models, including the traditional diagnostic code-based prediction model and triage scale.

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

Background

Precise models are necessary to estimate mortality risk following traumatic injury to inform clinical decision making or quantify hospital performance. The Trauma and Injury Severity Score (TRISS) has been the historical gold standard in survival prediction but its limitations are well-characterized. The present study used International Classification of Diseases 10^thRevision (ICD-10) injury codes with machine learning approaches to develop models whose performance was compared to that of TRISS.

Methods

The 2015–2017 National Trauma Data Bank was used to identify patients following trauma-related admission. Injury codes from ICD-10 were grouped by clinical relevance into 1,495 variables. The TRISS score, which comprises the Injury Severity Score, age, mechanism (blunt vs penetrating) as well as highest 24-hour values for systolic blood pressure (SBP), respiratory rate (RR) and Glasgow Coma Scale (GCS) was calculated for each patient. A base eXtreme gradient boosting model (XGBoost), a machine learning technique, was developed using injury variables as well as age, SBP, RR, mechanism and GCS. Prediction of in-hospital survival and other in-hospital complications were compared between both models using receiver operating characteristic (ROC) and reliability plots. A complete XGBoost model, containing injury variables, vitals, demographic information and comorbidities, was additionally developed.

Results

Of 1,380,740 patients, 1,338,417 (96.9%) survived to discharge. Compared to survivors, those who died were older and had a greater prevalence of penetrating injuries (18.0% vs 9.44%). The base XGBoost model demonstrated a greater receiver-operating characteristic (ROC) than TRISS (0.950 vs 0.907) which persisted across sub-populations and secondary endpoints. Furthermore, it exhibited high calibration across all risk levels (R² = 0.998 vs 0.816). The complete XGBoost model had an exceptional ROC of 0.960.

Conclusions

We report improved performance of machine learning models over TRISS. Our model may improve stratification of injury severity in clinical and quality improvement settings.

Machine learning-based modeling of acute respiratory failure following emergency general surgery operations

Background

Emergency general surgery (EGS) operations are associated with substantial risk of morbidity including postoperative respiratory failure (PRF). While existing risk models are not widely utilized and rely on traditional statistical methods, application of machine learning (ML) in prediction of PRF following EGS remains unexplored.

Objective

The present study aimed to develop ML-based prediction models for respiratory failure following EGS and compare their performance to traditional regression models using a nationally-representative cohort.

Methods

Non-elective hospitalizations for EGS (appendectomy, cholecystectomy, repair of perforated ulcer, large or small bowel resection, lysis of adhesions) were identified in the 2016–18 Nationwide Readmissions Database. Factors associated with PRF were identified using ML techniques and logistic regression. The performance of XGBoost and logistic regression was evaluated using the receiver operating characteristic curve and coefficient of determination (R²). The impact of PRF on mortality, length of stay (LOS) and hospitalization costs was secondarily assessed using generalized linear models.

Results

Of 1,003,703 hospitalizations, 8.8% developed PRF. The XGBoost model exhibited slightly superior discrimination compared to logistic regression (0.900, 95% CI 0.899–0.901 vs 0.894, 95% CI 0.862–0.896). Compared to logistic regression, XGBoost demonstrated excellent calibration across all risk levels (R²: 0.998 vs 0.962). Congestive heart failure, neurologic disorders, and coagulopathy were significantly associated with increased risk of PRF. After risk-adjustment, PRF was associated with 10-fold greater odds (95% confidence interval (CI) 9.8–11.1) of mortality and incremental increases in LOS by 3.1 days (95% CI 3.0–3.2) and $11,900 (95% CI 11,600–12,300) in costs.

Conclusions

Logistic regression and XGBoost perform similarly in overall classification of PRF risk. However, due to superior calibration at extremes of risk, ML-based models may prove more useful in the clinical setting, where probabilities rather than classifications are desired.