Predicting venous thromboembolism in hospitalized trauma patients: a combination of the Caprini score and data-driven machine learning model

Identification of key risk factors for venous thromboembolism in urological inpatients based on the Caprini scale and interpretable machine learning methods

PURPOSE

To identify the key risk factors for venous thromboembolism (VTE) in urological inpatients based on the Caprini scale using an interpretable machine learning method.

METHODS

VTE risk data of urological inpatients were obtained based on the Caprini scale in the case hospital. Based on the data, the Boruta method was used to further select the key variables from the 37 variables in the Caprini scale. Furthermore, decision rules corresponding to each risk level were generated using the rough set (RS) method. Finally, random forest (RF), support vector machine (SVM), and backpropagation artificial neural network (BPANN) were used to verify the data accuracy and were compared with the RS method.

RESULTS

Following the screening, the key risk factors for VTE in urology were "(C1) Age," "(C2) Minor Surgery planned," "(C3) Obesity (BMI > 25)," "(C8) Varicose veins," "(C9) Sepsis (< 1 month)," (C10) "Serious lung disease incl. pneumonia (< 1month) " (C11) COPD," "(C16) Other risk," "(C18) Major surgery (> 45 min)," "(C19) Laparoscopic surgery (> 45 min)," "(C20) Patient confined to bed (> 72 h)," "(C18) Malignancy (present or previous)," "(C23) Central venous access," "(C31) History of DVT/PE," "(C32) Other congenital or acquired thrombophilia," and "(C34) Stroke (< 1 month." According to the decision rules of different risk levels obtained using the RS method, "(C1) Age," "(C18) Major surgery (> 45 minutes)," and "(C21) Malignancy (present or previous)" were the main factors influencing mid- and high-risk levels, and some suggestions on VTE prevention were indicated based on these three factors. The average accuracies of the RS, RF, SVM, and BPANN models were 79.5%, 87.9%, 92.6%, and 97.2%, respectively. In addition, BPANN had the highest accuracy, recall, F1-score, and precision.

CONCLUSIONS

The RS model achieved poorer accuracy than the other three common machine learning models. However, the RS model provides strong interpretability and allows for the identification of high-risk factors and decision rules influencing high-risk assessments of VTE in urology. This transparency is very important for clinicians in the risk assessment process.

Machine Learning-Based Predictive Models for Patients with Venous Thromboembolism: A Systematic Review

BACKGROUND

 Venous thromboembolism (VTE) is a chronic disorder with a significant health and economic burden. Several VTE-specific clinical prediction models (CPMs) have been used to assist physicians in decision-making but have several limitations. This systematic review explores if machine learning (ML) can enhance CPMs by analyzing extensive patient data derived from electronic health records. We aimed to explore ML-CPMs' applications in VTE for risk stratification, outcome prediction, diagnosis, and treatment.

METHODS

 Three databases were searched: PubMed, Google Scholar, and IEEE electronic library. Inclusion criteria focused on studies using structured data, excluding non-English publications, studies on non-humans, and certain data types such as natural language processing and image processing. Studies involving pregnant women, cancer patients, and children were also excluded. After excluding irrelevant studies, a total of 77 studies were included.

RESULTS

 Most studies report that ML-CPMs outperformed traditional CPMs in terms of receiver operating area under the curve in the four clinical domains that were explored. However, the majority of the studies were retrospective, monocentric, and lacked detailed model architecture description and external validation, which are essential for quality audit. This review identified research gaps and highlighted challenges related to standardized reporting, reproducibility, and model comparison.

CONCLUSION

 ML-CPMs show promise in improving risk assessment and individualized treatment recommendations in VTE. Apparently, there is an urgent need for standardized reporting and methodology for ML models, external validation, prospective and real-world data studies, as well as interventional studies to evaluate the impact of artificial intelligence in VTE.

A Real-Time Automated Machine Learning Algorithm for Predicting Mortality in Trauma Patients: Survey Says it's Ready for Prime-Time

BACKGROUND

Though artificial intelligence ("AI") has been increasingly applied to patient care, many of these predictive models are retrospective and not readily available for real-time decision-making. This survey-based study aims to evaluate implementation of a new, validated mortality risk calculator (Parkland Trauma Index of Mortality, "PTIM") embedded in our electronic healthrecord ("EHR") that calculates hourly predictions of mortality with high sensitivity and specificity.

METHODS

This is a prospective, survey-based study performed at a level 1 trauma center. An anonymous survey was sent to surgical providers and regarding PTIM implementation. The PTIM score evaluates 23 variables including Glasgow Coma Score (GCS), vital signs, and laboratory data.

RESULTS

Of the 40 completed surveys, 35 reported using PTIM in decision-making. Prior to reviewing PTIM, providers identified perceived top 3 predictors of mortality, including GCS (22/38, 58%), age (18/35, 47%), and maximum heart rate (17/35, 45%). Most providers reported the PTIM assisted their treatment decisions (27/35, 77%) and timing of operative intervention (23/35, 66%). Many providers agreed that PTIM integrated into rounds and patient assessment (22/36, 61%) and that it improved efficiency in assessing patients' potential mortality (21/36, 58%).

CONCLUSIONS

Artificial intelligence algorithms are mostly retrospective and lag in real-time prediction of mortality. To our knowledge, this is the first real-time, automated algorithm predicting mortality in trauma patients. In this small survey-based study, we found PTIM assists in decision-making, timing of intervention, and improves accuracy in assessing mortality. Next steps include evaluating the short- and long-term impact on patient outcomes.

Padua prediction score may be inappropriate for VTE risk assessment in hospitalized patients with acute respiratory conditions: A Chinese single-center cohort study

Background

The Padua Prediction Score (PPS) recommended by the guidelines lacks effective external validation in a Chinese cohort. This study sought to assess the accuracy of the PPS to predict venous thromboembolism (VTE) risk in medical inpatients with acute respiratory conditions.

Methods

This consecutive cohort study included 1,574 inpatients from January to August 2019. The occurrence rate of VTE in patients classified at high-risk and low-risk groups according to PPS and Caprini risk assessment model (RAM) was compared. The discriminatory capability of the RAMs was evaluated in all the patients and the subgroup without pharmacological prophylaxis. Reclassification parameters were also used to assess the clinical utility.

Results

170 (10.8%) patients were objectively confirmed as having VTE during hospitalization. The incidence rate of VTE in low-risk patients was 6.3% by PPS, which was significantly higher than that by Caprini RAM (2.6%, p < 0.001). The area under the curve (AUC) for PPS and Caprini RAM was 0.714 (95%CI, 0.672-0.756) and 0.760 (95%CI, 0.724-0.797), respectively (p = 0.003). The AUC of Caprini RAM was larger than PPS even in subgroups without pharmacological prophylaxis (0.774 vs 0.709, p = 0.002). Compared with Caprini RAM, the net reclassification index was estimated at 0.037 (p = 0.436), and integrated discrimination improvement was 0.015 (p = 0.495) by PPS.

Conclusions

According to our cohort study, PPS may not be appropriate to predict VTE risk in hospitalized patients with acute respiratory conditions. An accurate, widely applicable, validated RAM needs to be further constructed in Chinese medical inpatients.

Correlation Between Statin Use and Symptomatic Venous Thromboembolism Incidence in Patients With Ankle Fracture: A Machine Learning Approach

BACKGROUND

Identifying factors that correlate with the incidence of venous thromboembolism (VTE) has the potential to improve VTE prevention and positively influence decision-making regarding prophylaxis. In this study, we aimed to investigate the correlation between statin consumption and the incidence of VTE in patients who sustained an ankle fracture.

METHODS

In this retrospective, case-controlled study, cases were those who developed VTE and controls were those who had no VTE, and the ratio was 1:4. Patients' demographics, history of hyperlipidemia, and reported statins use were obtained. A random forest classifier (RFC) model was used to predict whether statin consumers were at risk of VTE after ankle fracture regardless of VTE prophylaxis administration based on statin consumption, body mass index (BMI), age, and biological sex.

RESULTS

Of the 1175 patients with ankle fractures, 238 had confirmed VTE (case group), and 937 had no symptomatic VTE (control group; ratio 1:4). Fifty (21%) cases and 407 (43%) controls were on a statin. Statin users had a significantly lower incidence of VTE after ankle fracture, odds ratio (OR) = 0.35, 95% CI: 0.25, 0.49, P < .001. Our model showed an area under the receiving operator curve (AUROC) of 78%, a sensitivity of 73%, and a specificity of 83% in predicting the risk of VTE. The importance of the predictors of VTE, other than the use of statins (model importance = 0.1), were age (model importance of 0.72), BMI (model importance of 0.24), and biological sex (model importance of 0.02).

CONCLUSION

Statins were significantly associated with a lower rate of VTE in our population of patients who sustained an ankle fracture.

LEVELS OF EVIDENCE

3.

Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study

Introduction

Venous thromboembolism (VTE) risk assessment at admission is of great importance for early screening and timely prophylaxis and management during hospitalization. The purpose of this study is to develop and validate novel risk assessment models at admission based on machine learning (ML) methods.

Methods

In this retrospective study, a total of 3078 individuals were included with their Caprini variables within 24 hours at admission. Then several ML models were built, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). The prediction performance of ML models and the Caprini risk score (CRS) was then validated and compared through a series of evaluation metrics.

Results

The values of AUROC and AUPRC were 0.798 and 0.303 for LR, 0.804 and 0.360 for RF, and 0.796 and 0.352 for XGB, respectively, which outperformed CRS significantly (0.714 and 0.180, P < 0.001). When prediction scores were stratified into three risk levels for application, RF could obtain more reasonable results than CRS, including smaller false positive alerts and larger lower-risk proportions. The boosting results of stratification were further verified by the net-reclassification-improvement (NRI) analysis.

Discussion

This study indicated that machine learning models could improve VTE risk prediction at admission compared with CRS. Among the ML models, RF was found to have superior performance and great potential in clinical practice.

A nomogram to predict postoperative deep vein thrombosis in patients with femoral fracture: a retrospective study

OBJECTIVE

The implementation of more active anticoagulant prevention and treatment measures has indeed led to a significant reduction in the incidence of perioperative deep vein thrombosis (DVT) among patients with bone trauma. However, it is important to note that despite these efforts, the incidence of DVT still remains relatively high. According to the Caprini score, all patients undergoing major orthopedic surgery were defined as the high-risk group for DVT. Stratifying the risk further within high-risk groups for DVT continues to present challenges. As a result, the commonly used Caprini score during the perioperative period is not applicable to orthopedic patients. We attempt to establish a specialized model to predict postoperative DVT risk in patients with femoral fracture.

METHODS

We collected the clinical data of 513 patients undergoing femoral fracture surgery in our hospital from May 2018 to December 2019. According to the independent risk factors of DVT obtained by univariate and multivariate logistic regression analysis, the corresponding nomogram model was established and verified internally. The discriminative capacity of nomogram was evaluated by receiver operating characteristic (ROC) curve and area under the curve (AUC). The calibration curve used to verify model consistency was the fitted line between predicted and actual incidences. The clinical validity of the nomogram model was assessed using decision curve analysis (DCA) which could quantify the net benefit of different risk threshold probabilities. Bootstrap method was applied to the internal validation of the nomogram model. Furthermore, a comparison was made between the Caprini score and the developed nomogram model.

RESULTS

The Caprini scores of subjects ranged from 5 to 17 points. The incidence of DVT was not positively correlated with the Caprini score. The predictors of the nomogram model included 10 risk factors such as age, hypoalbuminemia, multiple trauma, perioperative red blood cell infusion, etc. Compared with the Caprini scale (AUC = 0.571, 95% CI 0.479-0.623), the calibration accuracy and identification ability of nomogram were higher (AUC = 0.865,95% CI 0.780-0.935). The decision curve analysis (DCA) indicated the clinical effectiveness of nomogram was higher than the Caprini score.

CONCLUSIONS

The nomogram was established to effectively predict postoperative DVT in patients with femoral fracture. To further reduce the incidence, more specialized risk assessment models for DVT should take into account the unique risk factors and characteristics associated with specific patient populations.

The Thresholds of Caprini Score Associated With Increased Risk of Venous Thromboembolism Across Different Specialties: A Systematic Review

OBJECTIVE

Estimation of the specific thresholds of the Caprini risk score (CRS) that are associated with the increased incidence of venous thromboembolism (VTE) across different specialties, including identifying the highest level of risk.

BACKGROUND

Accurate risk assessment remains an important but often challenging aspect of VTE prophylaxis. One well-established risk assessment model is CRS, which has been validated in thousands of patients from many different medical and surgical specialties.

METHODS

A search of MEDLINE and the Cochrane Library was performed in March 2022. Manuscripts that reported on (1) patients admitted to medical or surgical departments and (2) had their VTE risk assessed by CRS and (3) reported on the correlation between the score and VTE incidence, were included in the analysis.

RESULTS

A total of 4562 references were identified, and the full text of 202 papers was assessed for eligibility. The correlation between CRS and VTE incidence was reported in 68 studies that enrolled 4,207,895 patients. In all specialties, a significant increase in VTE incidence was observed in patients with a CRS of ≥5. In most specialties thresholds of ≥7, ≥9, and ≥11 to 12 were associated with dramatically increased incidences of VTE. In COVID-19, cancer, trauma, vascular, general, head and neck, and thoracic surgery patients with ≥9 and ≥11 to 12 scores the VTE incidence was extremely high (ranging from 13% to 47%).

CONCLUSION

The Caprini score is being used increasingly to predict VTE in many medical and surgical specialties. In most cases, the VTE risk for individual patients increases dramatically at a threshold CRS of 7 to 11.

Artificial intelligence and machine learning for hemorrhagic trauma care

Artificial intelligence (AI), a branch of machine learning (ML) has been increasingly employed in the research of trauma in various aspects. Hemorrhage is the most common cause of trauma-related death. To better elucidate the current role of AI and contribute to future development of ML in trauma care, we conducted a review focused on the use of ML in the diagnosis or treatment strategy of traumatic hemorrhage. A literature search was carried out on PubMed and Google scholar. Titles and abstracts were screened and, if deemed appropriate, the full articles were reviewed. We included 89 studies in the review. These studies could be grouped into five areas: (1) prediction of outcomes; (2) risk assessment and injury severity for triage; (3) prediction of transfusions; (4) detection of hemorrhage; and (5) prediction of coagulopathy. Performance analysis of ML in comparison with current standards for trauma care showed that most studies demonstrated the benefits of ML models. However, most studies were retrospective, focused on prediction of mortality, and development of patient outcome scoring systems. Few studies performed model assessment via test datasets obtained from different sources. Prediction models for transfusions and coagulopathy have been developed, but none is in widespread use. AI-enabled ML-driven technology is becoming integral part of the whole course of trauma care. Comparison and application of ML algorithms using different datasets from initial training, testing and validation in prospective and randomized controlled trials are warranted for provision of decision support for individualized patient care as far forward as possible.

Clustering of trauma patients based on longitudinal data and the application of machine learning to predict recovery

Predicting recovery after trauma is important to provide patients a perspective on their estimated future health, to engage in shared decision making and target interventions to relevant patient groups. In the present study, several unsupervised techniques are employed to cluster patients based on longitudinal recovery profiles. Subsequently, these data-driven clusters were assessed on clinical validity by experts and used as targets in supervised machine learning models. We present a formalised analysis of the obtained clusters that incorporates evaluation of (i) statistical and machine learning metrics, (ii) clusters clinical validity with descriptive statistics and medical expertise. Clusters quality assessment revealed that clusters obtained through a Bayesian method (High Dimensional Supervised Classification and Clustering) and a Deep Gaussian Mixture model, in combination with oversampling and a Random Forest for supervised learning of the cluster assignments provided among the most clinically sensible partitioning of patients. Other methods that obtained higher classification accuracy suffered from cluster solutions with large majority classes or clinically less sensible classes. Models that used just physical or a mix of physical and psychological outcomes proved to be among the most sensible, suggesting that clustering on psychological outcomes alone yields recovery profiles that do not conform to known risk factors.

A gradient boosting tree model for multi-department venous thromboembolism risk assessment with imbalanced data

Venous thromboembolism (VTE) is the world's third most common cause of vascular mortality and a serious complication from multiple departments. Risk assessment of VTE guides clinical intervention in time and is of great importance to in-hospital patients. Traditional VTE risk assessment methods based on scaling tools, which always require rules carefully designed by human experts, are difficult to apply to large-population scenarios since the manually designed rules are not guaranteed to be accurate to all populations. In contrast, with the development of the electronic health record (EHR) datasets, data-driven machine-learning-based risk assessment methods have proven superior predictability in many studies in recent years. This paper uses the gradient boosting tree model to study the VTE risk assessment problem with multi-department data. There exist two distinct characteristics of VTE data collected at the level of the entire hospital: its wide distribution and heterogeneity across multiple departments. To this end, we consider the prediction task over multiple departments as a multi-task learning process, and introduce the algorithm of a task-aware tree-based method TSGB to tackle the multi-task prediction problem. Although the introduction of multi-task learning improves overall across-department performance, we reveal the problem of task-wise performance decline while dealing with imbalanced VTE data volume. According to the analysis, we finally propose two variants of TSGB to alleviate the problems and further boost the prediction performance. Compared with state-of-the-art rule-based and multi-task tree-based methods, the experimental results show the proposed methods not only improve the overall across-department AUC performance effectively, but also ensure the improvement of performance over every single department prediction.

An abbreviated Caprini model for VTE risk assessment in trauma

The Caprini risk assessment model (RAM) is widely used to assess risk of venous thromboembolism (VTE). However, it is cumbersome with 31 variables and poses challenges with inter-rater reliability. This study aimed to determine if an abbreviated model could perform similarly in VTE risk assessment. We performed a retrospective review of trauma patients ≥ 18 years old and admitted for over 24 h at a Level I trauma center from January 1, 2018, to December 31, 2018. Demographic and clinical data were analyzed to generate Caprini scores. Using a p-value cutoff of < 0.05, the individual components of the original Caprini RAM most highly associated with VTE were identified and used to calculate an abbreviated Caprini score. Logistic regression assessed odds of inpatient VTE with the original or abbreviated Caprini RAMs. Receiver operating characteristic curves and c-statistics were generated to assess discriminatory ability. The study sample included 1279 patients. Ten risk factors were included in the abbreviated model (recent major surgery, length of surgery > 2 h, transfusion, restricted mobility > 72 h, central venous catheter, current major surgery, age, history of VTE, hip or leg fracture, and serious trauma). Compared to the original, the abbreviated model had a similar odds ratio (1.17 vs 1.07, both p-values < 0.001), c-statistic (0.747 vs 0.753), sensitivity (0.73 vs 0.76) and specificity (0.62 vs 0.61). An abbreviated Caprini RAM performs similarly to the original, may streamline workflow and allow for automation in electronic health records, potentially enhancing its use in resource limited settings.