Predicting Prolonged Apnea During Nurse-Administered Procedural Sedation: Machine Learning Study

Decoding machine learning in nursing research: A scoping review of effective algorithms

INTRODUCTION

The rapid evolution of artificial intelligence (AI) technology has revolutionized healthcare, particularly through the integration of AI into health information systems. This transformation has significantly impacted the roles of nurses and nurse practitioners, prompting extensive research to assess the effectiveness of AI-integrated systems. This scoping review focuses on machine learning (ML) used in nursing, specifically investigating ML algorithms, model evaluation methods, areas of focus related to nursing, and the most effective ML algorithms.

DESIGN

The scoping review followed the Preferred Reporting Items for Systematic Review and Meta-Analysis Extension for Scoping Reviews (PRISMA-ScR) guidelines.

METHODS

A structured search was performed across seven databases according to PRISMA-ScR: PubMed, EMBASE, CINAHL, Web of Science, OVID, PsycINFO, and ProQuest. The quality of the final reviewed studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI).

RESULTS

Twenty-six articles published between 2019 and 2023 met the inclusion and exclusion criteria, and 46% of studies were conducted in the US. The average MERSQI score was 12.2, indicative of moderate- to high-quality studies. The most used ML algorithm was Random Forest. The four second-most used were logistic regression, least absolute shrinkage and selection operator, decision tree, and support vector machine. Most ML models were evaluated by calculating sensitivity (recall)/specificity, accuracy, receiver operating characteristic (ROC), area under the ROC (AUROC), and positive/negative prediction value (precision). Half of the studies focused on nursing staff or students and hospital readmission or emergency department visits. Only 11 articles reported the most effective ML algorithm(s).

CONCLUSION

The scoping review provides insights into the current status of ML research in nursing and recognition of its significance in nursing research, confirming the benefits of ML in healthcare. Recommendations include incorporating experimental designs in research studies to optimize the use of ML models across various nursing domains.

CLINICAL RELEVANCE

The scoping review demonstrates substantial clinical relevance of ML applications for nurses, nurse practitioners, administrators, and researchers. The integration of ML into healthcare systems and its impact on nursing practices have important implications for patient care, resource management, and the evolution of nursing research.

Integrated pulmonary index during procedural sedation and analgesia: A cluster-randomized trial

AIM

To evaluate the effectiveness of utilizing the integrated pulmonary index for capnography implementation during sedation administered by nurses.

DESIGN

Cluster-randomized trial.

METHODS

Participants were enrolled from the interventional radiology department at an academic hospital in Canada. Nurses were randomized to either enable or disable the Integrated Pulmonary Index feature of the capnography monitor. Procedures were observed by a research assistant to collect information about alarm performance characteristics. The primary outcome was the number of seconds in an alert condition state without an intervention being applied.

RESULTS

The number of seconds in an alarm state without intervention was higher in the group that enabled the integrated pulmonary index compared to the group that disabled this feature, but this difference did not reach statistical significance. Likewise, the difference between groups for the total alarm duration, total number of alarms and the total number of appropriate alarms was not statistically significant. The number of inappropriate alarms was higher in the group that enabled the Integrated Pulmonary Index, but this estimate was highly imprecise. There was no difference in the odds of an adverse event (measured by the Tracking and Reporting Outcomes of Procedural Sedation tool) occurring between groups. Desaturation events were uncommon and brief in both groups but the area under the SpO2 90% desaturation curve scores were lower for the group that enabled the integrated pulmonary index.

CONCLUSION

Enabling the integrated pulmonary index during nurse-administered procedural sedation did not reduce nurses' response times to alarms. Therefore, integrating multiple physiological parameters related to respiratory assessment into a single index did not lower the threshold for intervention by nurses.

IMPLICATIONS FOR THE PROFESSION AND/OR PATIENT CARE

The time it takes to respond to capnography monitor alarms will not be reduced if the integrated pulmonary Iindex feature of capnography monitors is enabled during nurse-administered procedural sedation.

IMPACT

Results do not support the routine enabling of the integrated pulmonary index when nurses use capnography to monitor patients during procedural sedation as a strategy to reduce the time it takes to initiate responses to alarms.

REPORTING METHOD

CONSORT.

PATIENT OR PUBLIC CONTRIBUTION

There was no patient or public contribution.

TRIAL REGISTRATION

This study was prospectively registered at ClinicalTrials.gov (ID: NCT05068700).

Deep learning classification of capnography waveforms: secondary analysis of the PRODIGY study

Capnography monitors trigger high priority 'no breath' alarms when CO2 measurements do not exceed a given threshold over a specified time-period. False alarms occur when the underlying breathing pattern is stable, but the alarm is triggered when the CO2 value reduces even slightly below the threshold. True 'no breath' events can be falsely classified as breathing if waveform artifact causes an aberrant spike in CO2 values above the threshold. The aim of this study was to determine the accuracy of a deep learning approach to classifying segments of capnography waveforms as either 'breath' or 'no breath'. A post hoc secondary analysis of data from 9 North American sites included in the PRediction of Opioid-induced Respiratory Depression In Patients Monitored by capnoGraphY (PRODIGY) study was conducted. We used a convolutional neural network to classify 15 s capnography waveform segments drawn from a random sample of 400 participants. Loss was calculated over batches of 32 using the binary cross-entropy loss function with weights updated using the Adam optimizer. Internal-external validation was performed by iteratively fitting the model using data from all but one hospital and then assessing its performance in the remaining hospital. The labelled dataset consisted of 10,391 capnography waveform segments. The neural network's accuracy was 0.97, precision was 0.97 and recall was 0.96. Performance was consistent across hospitals in internal-external validation. The neural network could reduce false capnography alarms. Further research is needed to compare the frequency of alarms derived from the neural network with the standard approach.

A Machine Learning Approach for Predicting Readmission Charges Billed by Hospitals

Background

The Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.

Objective

We aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).

Methods

We defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.

Results

Our preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.

Conclusions

Models built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.

Predicting Prolonged Apnea During Nurse-Administered Procedural Sedation: Machine Learning Study

BACKGROUND

Capnography is commonly used for nurse-administered procedural sedation. Distinguishing between capnography waveform abnormalities that signal the need for clinical intervention for an event and those that do not indicate the need for intervention is essential for the successful implementation of this technology into practice. It is possible that capnography alarm management may be improved by using machine learning to create a "smart alarm" that can alert clinicians to apneic events that are predicted to be prolonged.

OBJECTIVE

To determine the accuracy of machine learning models for predicting at the 15-second time point if apnea will be prolonged (ie, apnea that persists for >30 seconds).

METHODS

A secondary analysis of an observational study was conducted. We selected several candidate models to evaluate, including a random forest model, generalized linear model (logistic regression), least absolute shrinkage and selection operator regression, ridge regression, and the XGBoost model. Out-of-sample accuracy of the models was calculated using 10-fold cross-validation. The net benefit decision analytic measure was used to assist with deciding whether using the models in practice would lead to better outcomes on average than using the current default capnography alarm management strategies. The default strategies are the aggressive approach, in which an alarm is triggered after brief periods of apnea (typically 15 seconds) and the conservative approach, in which an alarm is triggered for only prolonged periods of apnea (typically >30 seconds).

RESULTS

A total of 384 apneic events longer than 15 seconds were observed in 61 of the 102 patients (59.8%) who participated in the observational study. Nearly half of the apneic events (180/384, 46.9%) were prolonged. The random forest model performed the best in terms of discrimination (area under the receiver operating characteristic curve 0.66) and calibration. The net benefit associated with the random forest model exceeded that associated with the aggressive strategy but was lower than that associated with the conservative strategy.

CONCLUSIONS

Decision curve analysis indicated that using a random forest model would lead to a better outcome for capnography alarm management than using an aggressive strategy in which alarms are triggered after 15 seconds of apnea. The model would not be superior to the conservative strategy in which alarms are only triggered after 30 seconds.