External validation of a machine learning model to predict hemodynamic instability in intensive care unit

Prediction of teicoplanin plasma concentration in critically ill patients: a combination of machine learning and population pharmacokinetics

BACKGROUND

Teicoplanin has been widely used in patients with infections caused by Staphylococcus aureus, especially for critically ill patients. The pharmacokinetics (PK) of teicoplanin vary between individuals and within the same individual. We aim to establish a prediction model via a combination of machine learning and population PK (PPK) to support personalized medication decisions for critically ill patients.

METHODS

A retrospective study was performed incorporating 33 variables, including PPK parameters (clearance and volume of distribution). Multiple algorithms and Shapley additive explanations were employed for feature selection of variables to determine the strongest driving factors.

RESULTS

The performance of each algorithm with PPK parameters was superior to that without PPK parameters. The composition of support vector regression, categorical boosting and a backpropagation neural network (7:2:1) with the highest R2 (0.809) was determined as the final ensemble model. The model included 15 variables after feature selection, of which the predictive performance was superior to that of models considering all variables or using only PPK. The R2, mean absolute error, mean squared error, absolute accuracy (±5 mg/L) and relative accuracy (±30%) of external validation were 0.649, 3.913, 28.347, 76.12% and 76.12%, respectively.

CONCLUSIONS

Our study offers a non-invasive, fast and cost-effective prediction model of teicoplanin plasma concentration in critically ill patients. The model serves as a fundamental tool for clinicians to determine the effective plasma concentration range of teicoplanin and formulate individualized dosing regimens accordingly.

Perioperative Patients With Hemodynamic Instability: Consensus Recommendations of the Anesthesia Patient Safety Foundation

In November of 2022, the Anesthesia Patient Safety Foundation held a Consensus Conference on Hemodynamic Instability with invited experts. The objective was to review the science and use expert consensus to produce best practice recommendations to address the issue of perioperative hemodynamic instability. After expert presentations, a modified Delphi process using discussions, voting, and feedback resulted in 17 recommendations regarding advancing the perioperative care of the patient at risk of, or with, hemodynamic instability. There were 17 high-level recommendations. These recommendations related to the following 7 domains: Current Knowledge (5 statements); Preventing Hemodynamic Instability-Related Harm During All Phases of Care (4 statements); Data-Driven Quality Improvement (3 statements); Informing Patients (2 statements); The Importance of Technology (1 statement); Launch a National Campaign (1 statement); and Advancing the Science (1 statement). A summary of the recommendations is presented in Table 1 .

Optimizing artificial intelligence in sepsis management: Opportunities in the present and looking closely to the future

Sepsis remains a major challenge internationally for healthcare systems. Its incidence is rising due to poor public awareness and delays in its recognition and subsequent management. In sepsis, mortality increases with every hour left untreated. Artificial intelligence (AI) is transforming worldwide healthcare delivery at present. This review has outlined how AI can augment strategies to address this global disease burden. AI and machine learning (ML) algorithms can analyze vast quantities of increasingly complex clinical datasets from electronic medical records to assist clinicians in diagnosing and treating sepsis earlier than traditional methods. Our review highlights how these models can predict the risk of sepsis and organ failure even before it occurs. This gives providers additional time to plan and execute treatment plans, thereby avoiding increasing complications associated with delayed diagnosis of sepsis. The potential for cost savings with AI implementation is also discussed, including improving workflow efficiencies, reducing administrative costs, and improving healthcare outcomes. Despite these advantages, clinicians have been slow to adopt AI into clinical practice. Some of the limitations posed by AI solutions include the lack of diverse data sets for model building so that they are widely applicable for routine clinical use. Furthermore, the subsequent algorithms are often based on complex mathematics leading to clinician hesitancy to embrace such technologies. Finally, we highlight the need for robust political and regulatory frameworks in this area to achieve the trust and approval of clinicians and patients to implement this transformational technology.

A Delphi Process to Identify Relevant Outcomes That May Be Associated With a Predictive Analytic Tool to Detect Hemodynamic Deterioration in the Intensive Care Unit

Background The critical care literature has seen an increase in the development and validation of tools using artificial intelligence for early detection of patient events or disease onset in the intensive care unit (ICU). The hemodynamic stability index (HSI) was found to have an AUC of 0.82 in predicting the need for hemodynamic intervention in the ICU. Future studies using this tool may benefit from targeting those outcomes that are more relevant to clinicians and most achievable. Methods A three-round Delphi study was conducted with a panel of 10 critical care physicians and three nurses in the United States to identify outcomes that may be most relevant and achievable with the HSI when evaluated for use in the ICU. To achieve criteria for relevance, at least 65% of panelists had to rate an outcome as a 4 or 5 on a 5-point scale. Results Nineteen of 24 outcomes that may be associated with the HSI achieved consensus for relevance. The Kemeny-Young approach was used to develop a matrix depicting the distribution of outcomes considering both relevance and achievability. "Reduces time spent in hemodynamic instability" and "reduces times to recognition of hemodynamic instability" were the highest-ranking outcomes considering both relevance and achievability. Conclusion This Delphi study was a feasible method to identify relevant outcomes that may be associated with an appropriate predictive analytic tool in the ICU. These findings can provide insight to researchers looking to study such tools to impact outcomes relevant to critical care practitioners. Future studies should test these tools in the ICU that target the most clinically relevant and achievable outcomes, such as time spent hemodynamically unstable or time until actionable nursing assessment or treatment.

Internal and External Validation of the Generalizability of Machine Learning Algorithms in Predicting Non-home Discharge Disposition Following Primary Total Knee Joint Arthroplasty

BACKGROUND

Nonhome discharge disposition following primary total knee arthroplasty (TKA) is associated with a higher rate of complications and constitutes a socioeconomic burden on the health care system. While existing algorithms predicting nonhome discharge disposition varied in degrees of mathematical complexity and prediction power, their capacity to generalize predictions beyond the development dataset remains limited. Therefore, this study aimed to establish the machine learning model generalizability by performing internal and external validations using nation-scale and institutional cohorts, respectively.

METHODS

Four machine learning models were trained using the national cohort. Recursive feature elimination and hyper-parameter tuning were applied. Internal validation was achieved through five-fold cross-validation during model training. The trained models' performance was externally validated using the institutional cohort and assessed by discrimination, calibration, and clinical utility.

RESULTS

The national (424,354 patients) and institutional (10,196 patients) cohorts had non-home discharge rates of 19.4 and 36.4%, respectively. The areas under the receiver operating curve of the model predictions were 0.83 to 0.84 during internal validation and increased to 0.88 to 0.89 during external validation. Artificial neural network and histogram-based gradient boosting elicited the best performance with a mean area under the receiver operating curve of 0.89, calibration slope of 1.39, and Brier score of 0.14, which indicated that the two models were robust in distinguishing non-home discharge and well-calibrated with accurate predictions of the probabilities. The low inter-dataset similarity indicated reliable external validation. Length of stay, age, body mass index, and sex were the strongest predictors of discharge destination after primary TKA.

CONCLUSION

The machine learning models demonstrated excellent predictive performance during both internal and external validations, supporting their generalizability across different patient cohorts and potential applicability in the clinical workflow.