Development and validation of a portable platform for deploying decision-support algorithms in prehospital settings

Prediction of Occult Hemorrhage in the Lower Body Negative Pressure Model: Initial Validation of Machine Learning Approaches

INTRODUCTION

Detection of occult hemorrhage (OH) before progression to clinically apparent changes in vital signs remains an important clinical problem in managing trauma patients. The resource-intensiveness associated with continuous clinical patient monitoring and rescue from frank shock makes accurate early detection and prediction with noninvasive measurement technology a desirable innovation. Despite significant efforts directed toward the development of innovative noninvasive diagnostics, the implementation and performance of the newest bedside technologies remain inadequate. This poor performance may reflect the limitations of univariate systems based on one sensor in one anatomic location. It is possible that when signals are measured with multiple modalities in multiple locations, the resulting multivariate anatomic and temporal patterns of measured signals may provide additional discriminative power over single technology univariate measurements. We evaluated the potential superiority of multivariate methods over univariate methods. Additionally, we utilized machine learning-based models to compare the performance of noninvasive-only to noninvasive-plus-invasive measurements in predicting the onset of OH.

MATERIALS AND METHODS

We applied machine learning methods to preexisting datasets derived using the lower body negative pressure human model of simulated hemorrhage. Employing multivariate measured physiological signals, we investigated the extent to which machine learning methods can effectively predict the onset of OH. In particular, we applied 2 ensemble learning methods, namely, random forest and gradient boosting.

RESULTS

Analysis of precision, recall, and area under the receiver operating characteristic curve showed a superior performance of multivariate approach to that of the univariate ones. In addition, when using both invasive and noninvasive features, random forest classifier had a recall 95% confidence interval (CI) of 0.81 to 0.86 with a precision 95% CI of 0.65 to 0.72. Interestingly, when only noninvasive features were employed, the results worsened only slightly to a recall 95% CI of 0.80 to 0.85 and a precision 95% CI of 0.61 to 0.73.

CONCLUSIONS

Multivariate ensemble machine learning-based approaches for the prediction of hemodynamic instability appear to hold promise for the development of effective solutions. In the lower body negative pressure multivariate hemorrhage model, predictions based only on noninvasive measurements performed comparably to those using both invasive and noninvasive measurements.

Identification of major hemorrhage in trauma patients in the prehospital setting: diagnostic accuracy and impact on outcome

Background

Hemorrhage is the most common cause of potentially preventable death after injury. Early identification of patients with major hemorrhage (MH) is important as treatments are time-critical. However, diagnosis can be difficult, even for expert clinicians. This study aimed to determine how accurate clinicians are at identifying patients with MH in the prehospital setting. A second aim was to analyze factors associated with missed and overdiagnosis of MH, and the impact on mortality.

Methods

Retrospective evaluation of consecutive adult (≥16 years) patients injured in 2019-2020, assessed by expert trauma clinicians in a mature prehospital trauma system, and admitted to a major trauma center (MTC). Clinicians decided to activate the major hemorrhage protocol (MHPA) or not. This decision was compared with whether patients had MH in hospital, defined as the critical admission threshold (CAT+): administration of ≥3 U of red blood cells during any 60-minute period within 24 hours of injury. Multivariate logistical regression analyses were used to analyze factors associated with diagnostic accuracy and mortality.

Results

Of the 947 patients included in this study, 138 (14.6%) had MH. MH was correctly diagnosed in 97 of 138 patients (sensitivity 70%) and correctly excluded in 764 of 809 patients (specificity 94%). Factors associated with missed diagnosis were penetrating mechanism (OR 2.4, 95% CI 1.2 to 4.7) and major abdominal injury (OR 4.0; 95% CI 1.7 to 8.7). Factors associated with overdiagnosis were hypotension (OR 0.99; 95% CI 0.98 to 0.99), polytrauma (OR 1.3, 95% CI 1.1 to 1.6), and diagnostic uncertainty (OR 3.7, 95% CI 1.8 to 7.3). When MH was missed in the prehospital setting, the risk of mortality increased threefold, despite being admitted to an MTC.

Conclusion

Clinical assessment has only a moderate ability to identify MH in the prehospital setting. A missed diagnosis of MH increased the odds of mortality threefold. Understanding the limitations of clinical assessment and developing solutions to aid identification of MH are warranted.

Level of evidence

Level III-Retrospective study with up to two negative criteria.

Study type

Original research; diagnostic accuracy study.

Methods used to evaluate usability of mobile clinical decision support systems for healthcare emergencies: a systematic review and qualitative synthesis

Objective

The aim of this study was to determine the methods and metrics used to evaluate the usability of mobile application Clinical Decision Support Systems (CDSSs) used in healthcare emergencies. Secondary aims were to describe the characteristics and usability of evaluated CDSSs.

Materials and Methods

A systematic literature review was conducted using Pubmed/Medline, Embase, Scopus, and IEEE Xplore databases. Quantitative data were descriptively analyzed, and qualitative data were described and synthesized using inductive thematic analysis.

Results

Twenty-three studies were included in the analysis. The usability metrics most frequently evaluated were efficiency and usefulness, followed by user errors, satisfaction, learnability, effectiveness, and memorability. Methods used to assess usability included questionnaires in 20 (87%) studies, user trials in 17 (74%), interviews in 6 (26%), and heuristic evaluations in 3 (13%). Most CDSS inputs consisted of manual input (18, 78%) rather than automatic input (2, 9%). Most CDSS outputs comprised a recommendation (18, 78%), with a minority advising a specific treatment (6, 26%), or a score, risk level or likelihood of diagnosis (6, 26%). Interviews and heuristic evaluations identified more usability-related barriers and facilitators to adoption than did questionnaires and user testing studies.

Discussion

A wide range of metrics and methods are used to evaluate the usability of mobile CDSS in medical emergencies. Input of information into CDSS was predominantly manual, impeding usability. Studies employing both qualitative and quantitative methods to evaluate usability yielded more thorough results.

Conclusion

When planning CDSS projects, developers should consider multiple methods to comprehensively evaluate usability.

APPRAISE-HRI: AN ARTIFICIAL INTELLIGENCE ALGORITHM FOR TRIAGE OF HEMORRHAGE CASUALTIES

ABSTRACT

Background: Hemorrhage remains the leading cause of death on the battlefield. This study aims to assess the ability of an artificial intelligence triage algorithm to automatically analyze vital-sign data and stratify hemorrhage risk in trauma patients. Methods: Here, we developed the APPRAISE-Hemorrhage Risk Index (HRI) algorithm, which uses three routinely measured vital signs (heart rate and diastolic and systolic blood pressures) to identify trauma patients at greatest risk of hemorrhage. The algorithm preprocesses the vital signs to discard unreliable data, analyzes reliable data using an artificial intelligence-based linear regression model, and stratifies hemorrhage risk into low (HRI:I), average (HRI:II), and high (HRI:III). Results: To train and test the algorithm, we used 540 h of continuous vital-sign data collected from 1,659 trauma patients in prehospital and hospital (i.e., emergency department) settings. We defined hemorrhage cases (n = 198) as those patients who received ≥1 unit of packed red blood cells within 24 h of hospital admission and had documented hemorrhagic injuries. The APPRAISE-HRI stratification yielded a hemorrhage likelihood ratio (95% confidence interval) of 0.28 (0.13-0.43) for HRI:I, 1.00 (0.85-1.15) for HRI:II, and 5.75 (3.57-7.93) for HRI:III, suggesting that patients categorized in the low-risk (high-risk) category were at least 3-fold less (more) likely to have hemorrhage than those in the average trauma population. We obtained similar results in a cross-validation analysis. Conclusions: The APPRAISE-HRI algorithm provides a new capability to evaluate routine vital signs and alert medics to specific casualties who have the highest risk of hemorrhage, to optimize decision-making for triage, treatment, and evacuation.

Development and validation of a prehospital-stage prediction tool for traumatic brain injury: a multicentre retrospective cohort study in Korea

OBJECTIVES

Predicting diagnosis and prognosis of traumatic brain injury (TBI) at the prehospital stage is challenging; however, using comprehensive prehospital information and machine learning may improve the performance of the predictive model. We developed and tested predictive models for TBI that use machine learning algorithms using information that can be obtained in the prehospital stage.

DESIGN

This was a multicentre retrospective study.

SETTING AND PARTICIPANTS

This study was conducted at three tertiary academic emergency departments (EDs) located in an urban area of South Korea. The data from adult patients with severe trauma who were assessed by emergency medical service providers and transported to three participating hospitals between 2014 to 2018 were analysed.

RESULTS

We developed and tested five machine learning algorithms-logistic regression analyses, extreme gradient boosting, support vector machine, random forest and elastic net (EN)-to predict TBI, TBI with intracranial haemorrhage or injury (TBI-I), TBI with ED or admission result of admission or transferred (TBI with non-discharge (TBI-ND)) and TBI with ED or admission result of death (TBI-D). A total of 1169 patients were included in the final analysis, and the proportions of TBI, TBI-I, TBI-ND and TBI-D were 24.0%, 21.5%, 21.3% and 3.7%, respectively. The EN model yielded an area under receiver-operator curve of 0.799 for TBI, 0.844 for TBI-I, 0.811 for TBI-ND and 0.871 for TBI-D. The EN model also yielded the highest specificity and significant reclassification improvement. Variables related to loss of consciousness, Glasgow Coma Scale and light reflex were the three most important variables to predict all outcomes.

CONCLUSION

Our results inform the diagnosis and prognosis of TBI. Machine learning models resulted in significant performance improvement over that with logistic regression analyses, and the best performing model was EN.

A Machine Learning-Based Model to Predict Acute Traumatic Coagulopathy in Trauma Patients Upon Emergency Hospitalization

Acute traumatic coagulopathy (ATC) is an extremely common but silent murderer; this condition presents early after trauma and impacts approximately 30% of severely injured patients who are admitted to emergency departments (EDs). Given that conventional coagulation indicators usually require more than 1 hour after admission to yield results—a limitation that frequently prevents the ability for clinicians to make appropriate interventions during the optimal therapeutic window—it is clearly of vital importance to develop prediction models that can rapidly identify ATC; such models would also facilitate ancillary resource management and clinical decision support. Using the critical care Emergency Rescue Database and further collected data in ED, a total of 1385 patients were analyzed and cases with initial international normalized ratio (INR) values >1.5 upon admission to the ED met the defined diagnostic criteria for ATC; nontraumatic conditions with potentially disordered coagulation systems were excluded. A total of 818 individuals were collected from Emergency Rescue Database as derivation cohorts, then were split 7:3 into training and test data sets. A Pearson correlation matrix was used to initially identify likely key clinical features associated with ATC, and analysis of data distributions was undertaken prior to the selection of suitable modeling tools. Both machine learning (random forest) and traditional logistic regression were deployed for prediction modeling of ATC. After the model was built, another 587 patients were further collected in ED as validation cohorts. The ATC prediction models incorporated red blood cell count, Shock Index, base excess, lactate, diastolic blood pressure, and potential of hydrogen. Of 818 trauma patients filtered from the database, 747 (91.3%) patients did not present ATC (INR ≤ 1.5) and 71 (8.7%) patients had ATC (INR > 1.5) upon admission to the ED. Compared to the logistic regression model, the model based on the random forest algorithm showed better accuracy (94.0%, 95% confidence interval [CI]: 0.922-0.954 to 93.5%, 95% CI: 0.916-0.95), precision (93.3%, 95% CI: 0.914-0.948 to 93.1%, 95% CI: 0.912-0.946), F1 score (93.4%, 95% CI: 0.915-0.949 to 92%, 95% CI: 0.9-0.937), and recall score (94.0%, 95% CI: 0.922-0.954 to 93.5%, 95% CI: 0.916-0.95) but yielded lower area under the receiver operating characteristic curve (AU-ROC) (0.810, 95% CI: 0.673-0.918 to 0.849, 95% CI: 0.732-0.944) for predicting ATC in the trauma patients. The result is similar in the validation cohort. The values for classification accuracy, precision, F1 score, and recall score of random forest model were 0.916, 0.907, 0.901, and 0.917, while the AU-ROC was 0.830. The values for classification accuracy, precision, F1 score, and recall score of logistic regression model were 0.905, 0.887, 0.883, and 0.905, while the AU-ROC was 0.858. We developed and validated a prediction model based on objective and rapidly accessible clinical data that very confidently identify trauma patients at risk for ATC upon their arrival to the ED. Beyond highlighting the value of ED initial laboratory tests and vital signs when used in combination with data analysis and modeling, our study illustrates a practical method that should greatly facilitates both warning and guided target intervention for ATC.

Tachycardic and non-tachycardic responses in trauma patients with haemorrhagic injuries

BACKGROUND

Analyses of large databases have demonstrated that the association between heart rate (HR) and blood loss is weaker than what is taught by Advanced Trauma Life Support training. However, those studies had limited ability to generate a more descriptive paradigm, because they only examined a single HR value per patient.

METHODS

In a comparative, retrospective analysis, we studied the temporal characteristics of HR through time in adult trauma patients with haemorrhage, based on documented injuries and transfusion of ≥3 units of red blood cells (RBCs). We analysed archived vital-sign data of up to 60 min during either pre-hospital or emergency department care.

RESULTS

We identified 133 trauma patients who met the inclusion criteria for major haemorrhage and 1640 control patients without haemorrhage. There were 55 haemorrhage patients with a normal median HR and 78 with tachycardia. Median ΔHR was -0.8 and +0.7 bpm per 10 min, respectively. Median time to documented hypotension was 8 and 5 min, respectively. RBCs were not significantly different; median volumes were 6 (IQR: 4-13) and 10 units (IQR: 5-16), respectively. Time-to-hypotension and mortality were not significantly different. Tachycardic patients were significantly younger (P < 0.05). Only 10 patients with normal HR developed transient/temporary tachycardia, and only 11 tachycardic patients developed a transient/temporary normal HR.

CONCLUSIONS

The current analysis suggests that some trauma patients with haemorrhage are continuously tachycardic while others have a normal HR. For both cohorts, hypotension typically develops within 30 min, without any consistent temporal increases or trends in HR.

On-admission blood pressure and pulse rate in trauma patients and their correlation with mortality: Cushing's phenomenon revisited

BACKGROUND

Injury-induced alteration in initial physiological responses such as hypertension and heart rate (HR) has a significant effect on mortality. Research on such associations from our country-India is limited. The present study investigates the injury-induced early blood pressure (BP) and HR changes and their association with mortality.

MATERIALS AND METHODS

The data were selected from Towards Improved Trauma Care Outcomes collected from October 1, 2013, to July 24, 2014. Patients above 18 years of age with documented systolic BP (SBP) and HR were selected. BP was categorized into hypotension (SBP <90 mmHg), hypertension (SBP >140 mmHg), and normal (SBP 90-140 mmHg). HR was categorized into bradycardia (HR <60 beats/min [bpm]), tachycardia (HR >100 bpm), and normal (HR 60-100 bpm). These categories were compared with mortality.

RESULTS

A total of 10,200 patients were considered for the study. Mortality rate was 24%. Mortality among females was more than males. Patients with normal BP and HR had 20% of mortality. Mortality in patients with abnormal BP and HR findings was 36%. Mortality was higher among hypotension-bradycardia patients (80%) followed by hypertension-bradycardia patients (58%) and tachycardia hypotension patients (48%). Elderly patients were at higher risk of deaths with an overall mortality of 35% compared to 23% of adults.

CONCLUSION

The study reports that initial combination of hypotension-bradycardia had higher mortality rate. Specific precautions in prehospital care should be given to trauma patients with these findings. Further prospective study in detail should be considered for exploring this abnormality.

A platform for real-time acquisition and analysis of physiological data in hospital emergency departments

An opportunity exists for automated clinical decision support, in which raw source data from a conventional physiological monitoring system are continuously streamed to an independent analysis platform. Such a system would enable a wider range of functionality than offered by the source monitoring system. Although vendor solutions for this purpose are emerging, we developed our own system in order to control the expense and to permit forensic analysis of the internal core functionality of the system. In this report, we describe a platform that can provide decision support for trauma patients in an Emergency Department (ED). System evaluation spanned 39 days, and included a total of 2200 patient session hrs of real-time monitoring. We highlight the technical issues that we confronted, including protection of the core monitoring network, the real-time communication of electronic medical data, and the reliability of the real-time analysis. Detailing these nuanced technical issues may be valuable to other software developers or for those interested in investing in a vendor solution for similar functionality.