1
|
Hsu CC, Chu CC, Ng CJ, Lin CH, Lo HY, Chen SY. Machine learning models for predicting unscheduled return visits of patients with abdominal pain at emergency department and validation during COVID-19 pandemic: A retrospective cohort study. Medicine (Baltimore) 2024; 103:e37220. [PMID: 38394532 PMCID: PMC11309716 DOI: 10.1097/md.0000000000037220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 01/18/2024] [Indexed: 02/25/2024] Open
Abstract
Machine learning (ML) models for predicting 72-hour unscheduled return visits (URVs) for patients with abdominal pain in the emergency department (ED) were developed in a previous study. This study refined the data to adjust previous prediction models and evaluated the model performance in future data validation during the COVID-19 era. We aimed to evaluate the practicality of the ML models and compare the URVs before and during the COVID-19 pandemic. We used electronic health records from Chang Gung Memorial Hospital from 2018 to 2019 as a training dataset, and various machine learning models, including logistic regression (LR), random forest (RF), extreme gradient boosting (XGB), and voting classifier (VC) were developed and subsequently used to validate against the 2020 to 2021 data. The models highlighted several determinants for 72-hour URVs, including patient age, prior ER visits, specific vital signs, and medical interventions. The LR, XGB, and VC models exhibited the same AUC of 0.71 in the testing set, whereas the VC model displayed a higher F1 score (0.21). The XGB model demonstrated the highest specificity (0.99) and precision (0.64) but the lowest sensitivity (0.01). Among these models, the VC model showed the most favorable, balanced, and comprehensive performance. Despite the promising results, the study illuminated challenges in predictive modeling, such as the unforeseen influences of global events, such as the COVID-19 pandemic. These findings not only highlight the significant potential of machine learning in augmenting emergency care but also underline the importance of iterative refinement in response to changing real-world conditions.
Collapse
Affiliation(s)
- Chun-Chuan Hsu
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, Linkou, Taoyuan City 333, Taiwan
| | - Cheng-C.J. Chu
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital, Linkou, Taoyuan City 333, Taiwan
| | - Chip-Jin Ng
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, Linkou, Taoyuan City 333, Taiwan
| | - Ching-Heng Lin
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital, Linkou, Taoyuan City 333, Taiwan
- Bachelor Program in Artificial Intelligence, Chang Gung University, Taoyuan City 333, Taiwan
| | - Hsiang-Yun Lo
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, Linkou, Taoyuan City 333, Taiwan
| | - Shou-Yen Chen
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, Linkou, Taoyuan City 333, Taiwan
| |
Collapse
|
2
|
Lee YC, Ng CJ, Hsu CC, Cheng CW, Chen SY. Machine learning models for predicting unscheduled return visits to an emergency department: a scoping review. BMC Emerg Med 2024; 24:20. [PMID: 38287243 PMCID: PMC10826225 DOI: 10.1186/s12873-024-00939-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 01/22/2024] [Indexed: 01/31/2024] Open
Abstract
BACKGROUND Unscheduled return visits (URVs) to emergency departments (EDs) are used to assess the quality of care in EDs. Machine learning (ML) models can incorporate a wide range of complex predictors to identify high-risk patients and reduce errors to save time and cost. However, the accuracy and practicality of such models are questionable. This review compares the predictive power of multiple ML models and examines the effects of multiple research factors on these models' performance in predicting URVs to EDs. METHODS We conducted the present scoping review by searching eight databases for data from 2010 to 2023. The criteria focused on eligible articles that used ML to predict ED return visits. The primary outcome was the predictive performances of the ML models, and results were analyzed on the basis of intervals of return visits, patient population, and research scale. RESULTS A total of 582 articles were identified through the database search, with 14 articles selected for detailed analysis. Logistic regression was the most widely used method; however, eXtreme Gradient Boosting generally exhibited superior performance. Variations in visit interval, target group, and research scale did not significantly affect the predictive power of the models. CONCLUSION This is the first study to summarize the use of ML for predicting URVs in ED patients. The development of practical ML prediction models for ED URVs is feasible, but improving the accuracy of predicting ED URVs to beyond 0.75 remains a challenge. Including multiple data sources and dimensions is key for enabling ML models to achieve high accuracy; however, such inclusion could be challenging within a limited timeframe. The application of ML models for predicting ED URVs may improve patient safety and reduce medical costs by decreasing the frequency of URVs. Further research is necessary to explore the real-world efficacy of ML models.
Collapse
Affiliation(s)
- Yi-Chih Lee
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, College of Medicine, Taoyuan City, 333, Taiwan
| | - Chip-Jin Ng
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, College of Medicine, Taoyuan City, 333, Taiwan
| | - Chun-Chuan Hsu
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, College of Medicine, Taoyuan City, 333, Taiwan
| | - Chien-Wei Cheng
- Department of Emergency Medicine, Chang Gung Memorial Hospital, Keelung and Chang Gung University, College of Medicine, No. 5 Fushing St., Gueishan Shiang, Taoyuan City, 333, Taiwan
| | - Shou-Yen Chen
- Department of Emergency Medicine, Chang Gung Memorial Hospital and Chang Gung University, College of Medicine, Taoyuan City, 333, Taiwan.
- Department of Emergency Medicine, Chang Gung Memorial Hospital, Keelung and Chang Gung University, College of Medicine, No. 5 Fushing St., Gueishan Shiang, Taoyuan City, 333, Taiwan.
| |
Collapse
|
3
|
Lee AR, Park H, Yoo A, Kim S, Sunwoo L, Yoo S. Risk Prediction of Emergency Department Visits in Patients With Lung Cancer Using Machine Learning: Retrospective Observational Study. JMIR Med Inform 2023; 11:e53058. [PMID: 38055320 PMCID: PMC10733827 DOI: 10.2196/53058] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 10/31/2023] [Accepted: 11/24/2023] [Indexed: 12/07/2023] Open
Abstract
BACKGROUND Patients with lung cancer are among the most frequent visitors to emergency departments due to cancer-related problems, and the prognosis for those who seek emergency care is dismal. Given that patients with lung cancer frequently visit health care facilities for treatment or follow-up, the ability to predict emergency department visits based on clinical information gleaned from their routine visits would enhance hospital resource utilization and patient outcomes. OBJECTIVE This study proposed a machine learning-based prediction model to identify risk factors for emergency department visits by patients with lung cancer. METHODS This was a retrospective observational study of patients with lung cancer diagnosed at Seoul National University Bundang Hospital, a tertiary general hospital in South Korea, between January 2010 and December 2017. The primary outcome was an emergency department visit within 30 days of an outpatient visit. This study developed a machine learning-based prediction model using a common data model. In addition, the importance of features that influenced the decision-making of the model output was analyzed to identify significant clinical factors. RESULTS The model with the best performance demonstrated an area under the receiver operating characteristic curve of 0.73 in its ability to predict the attendance of patients with lung cancer in emergency departments. The frequency of recent visits to the emergency department and several laboratory test results that are typically collected during cancer treatment follow-up visits were revealed as influencing factors for the model output. CONCLUSIONS This study developed a machine learning-based risk prediction model using a common data model and identified influencing factors for emergency department visits by patients with lung cancer. The predictive model contributes to the efficiency of resource utilization and health care service quality by facilitating the identification and early intervention of high-risk patients. This study demonstrated the possibility of collaborative research among different institutions using the common data model for precision medicine in lung cancer.
Collapse
Affiliation(s)
- Ah Ra Lee
- Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| | - Hojoon Park
- Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| | - Aram Yoo
- Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| | - Seok Kim
- Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| | - Leonard Sunwoo
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| | - Sooyoung Yoo
- Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
| |
Collapse
|
4
|
Dunnion M, Ryan A, Goode D, McIlfatrick S. Supporting older people following out of hours discharge from the Emergency Department: An integrative review of the literature. Int J Older People Nurs 2023; 18:e12529. [PMID: 36866513 DOI: 10.1111/opn.12529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 01/19/2023] [Accepted: 01/29/2023] [Indexed: 03/04/2023]
Abstract
BACKGROUND For many older people the emergency department (ED) is an important but sometimes difficult step in their healthcare journey. They often attend the ED with co and multi morbidities. Discharge home at evenings and weekends when post-discharge support services are limited can result in a delay or failure to follow through on their discharge plan leading to adverse health outcomes and in some cases, readmission to ED. OBJECTIVE The aim of this integrative review was to identify and appraise the support available to older people following discharge from the ED out of hours (OOH). METHODS For this review, out of hours referred to those times after 17.30 until 08.00 a.m. on Mondays to Fridays, all hours on weekends and public holidays. Whittemore and Knafl's (Journal of Advanced Nursing, 2005;52:546), framework was used to guide all stages of the review process. Articles were retrieved following a rigorous search of published works using various databases, the grey literature and hand search of the reference lists of the studies included. RESULTS In total 31 articles were included in the review. These comprised systematic reviews, randomised control studies, cohort studies and surveys. Main themes identified included processes that enable support, support provision by health and social care professionals and telephone follow-up. Results identified a significant dearth of out of hours discharge research and a strong recommendation for more concise and thorough research in this important area of care transition. CONCLUSION Older person discharge home from the ED presents an associated risk as previous research has identified frequent readmission and periods of ill health and dependency. Out of hours discharge can be even more problematic when it may be difficult to arrange support services and ensure continuity of care. Further work in this area is required, taking cognisance of the findings and recommendations identified in this review.
Collapse
Affiliation(s)
- Mary Dunnion
- School of Nursing, Department of Life and Health Science, Ulster University, Coleraine, UK
| | - Assumpta Ryan
- School of Nursing and Institute of Nursing and Health Research, Ulster University, Coleraine, UK
| | - Debbie Goode
- School of Nursing and Institute of Nursing and Health Research, Ulster University, Coleraine, UK
| | - Sonja McIlfatrick
- School of Nursing and Institute of Nursing and Health Research, Ulster University, Coleraine, UK
| |
Collapse
|
5
|
Fan L, Xue H, Wang Q, Yan Y, Du W. Development and Validation of a Nomogram for Predicting Risk of Emergency Department Revisits in Chinese Older Patients. Healthc Policy 2022; 15:2283-2295. [PMID: 36510569 PMCID: PMC9738955 DOI: 10.2147/rmhp.s391731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 11/26/2022] [Indexed: 12/12/2022] Open
Abstract
Objective The identification of older patients at risk of repeated emergency department (ED) visits is crucial for managing preventable adverse outcomes. This study aims to identify risk factors associated with ED revisits and to develop and validate a nomogram for predicting risk of geriatric ED revisits. Methods This was a cohort study comprising 553 older patients, who attended the two tertiary hospitals EDs in China from August 2018 to February 2019 and were prospectively followed for any unplanned revisit within 1 year after discharge. Patients were randomly assigned to a training or validation set at a ratio of 2:1. Stepwise selection procedure was applied to select factors associated with ED revisits for inclusion in a multivariable logistic model from which a nomogram was elaborated. Discrimination, calibration and clinical utility of the nomogram were assessed using C-statistic, calibration plot, Hosmer-Lemeshow test, and decision curve analysis (DCA). Results The final nomogram included four predictors for ED revisits: age, BMI, frailty and polypharmacy. Older patients having revisits were more likely to be frail (OR = 1.17, p = 0.031), have polypharmacy (OR = 1.69, p = 0.049) or BMI <18.5 kg/m2 (OR = 2.45, p = 0.025), and were less likely to be older than 90 years (OR = 0.21, p = 0.002). The nomogram demonstrated acceptable discrimination ability in the training (C-index = 0.661) and validation sets (C-index = 0.651), satisfactory calibration (p > 0.05), and good clinical applicability. Conclusion A nomogram incorporating four obtainable variables was constructed to individualize ED readmission risk in older patients. These patients may benefit from early triage and better-targeted care if considering the nomogram as a clinical decision aid.
Collapse
Affiliation(s)
- Lijun Fan
- School of Public Health, Southeast University, Nanjing, People’s Republic of China,School of Public Health, Sun Yat-sen University, Guangzhou, People’s Republic of China
| | - Hui Xue
- School of Public Health, Southeast University, Nanjing, People’s Republic of China
| | - Qian Wang
- Guangdong Provincial Geriatrics Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, People’s Republic of China
| | - Yuhan Yan
- General Hospital of Eastern Theater Command, Nanjing, People’s Republic of China,Yuhan Yan, General Hospital of Eastern Theater Command, Nanjing, 210009, People’s Republic of China, Email
| | - Wei Du
- School of Public Health, Southeast University, Nanjing, People’s Republic of China,Correspondence: Wei Du, School of Public Health, Southeast University, Nanjing, 210009, People’s Republic of China, Email
| |
Collapse
|