1
|
Hinson JS, Zhao X, Klein E, Badaki‐Makun O, Rothman R, Copenhaver M, Smith A, Fenstermacher K, Toerper M, Pekosz A, Levin S. Multisite development and validation of machine learning models to predict severe outcomes and guide decision-making for emergency department patients with influenza. J Am Coll Emerg Physicians Open 2024; 5:e13117. [PMID: 38500599 PMCID: PMC10945311 DOI: 10.1002/emp2.13117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 01/10/2024] [Accepted: 01/25/2024] [Indexed: 03/20/2024] Open
Abstract
Objective Millions of Americans are infected by influenza annually. A minority seek care in the emergency department (ED) and, of those, only a limited number experience severe disease or death. ED clinicians must distinguish those at risk for deterioration from those who can be safely discharged. Methods We developed random forest machine learning (ML) models to estimate needs for critical care within 24 h and inpatient care within 72 h in ED patients with influenza. Predictor data were limited to those recorded prior to ED disposition decision: demographics, ED complaint, medical problems, vital signs, supplemental oxygen use, and laboratory results. Our study population was comprised of adults diagnosed with influenza at one of five EDs in our university health system between January 1, 2017 and May 18, 2022; visits were divided into two cohorts to facilitate model development and validation. Prediction performance was assessed by the area under the receiver operating characteristic curve (AUC) and the Brier score. Results Among 8032 patients with laboratory-confirmed influenza, incidence of critical care needs was 6.3% and incidence of inpatient care needs was 19.6%. The most common reasons for ED visit were symptoms of respiratory tract infection, fever, and shortness of breath. Model AUCs were 0.89 (95% CI 0.86-0.93) for prediction of critical care and 0.90 (95% CI 0.88-0.93) for inpatient care needs; Brier scores were 0.026 and 0.042, respectively. Importantpredictors included shortness of breath, increasing respiratory rate, and a high number of comorbid diseases. Conclusions ML methods can be used to accurately predict clinical deterioration in ED patients with influenza and have potential to support ED disposition decision-making.
Collapse
Affiliation(s)
- Jeremiah S. Hinson
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- Malone Center for Engineering in HealthcareJohns Hopkins University Whiting School of EngineeringBaltimoreMarylandUSA
| | - Xihan Zhao
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Eili Klein
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- One Health TrustWashingtonDistrict of ColumbiaUSA
| | - Oluwakemi Badaki‐Makun
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- Department of PediatricsJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Richard Rothman
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Martin Copenhaver
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Aria Smith
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- Malone Center for Engineering in HealthcareJohns Hopkins University Whiting School of EngineeringBaltimoreMarylandUSA
| | - Katherine Fenstermacher
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Matthew Toerper
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Andrew Pekosz
- Department of Microbiology and ImmunologyJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Scott Levin
- Department of Emergency MedicineJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- Malone Center for Engineering in HealthcareJohns Hopkins University Whiting School of EngineeringBaltimoreMarylandUSA
| |
Collapse
|
2
|
Wang P, Zhang W, Wang H, Shi C, Li Z, Wang D, Luo L, Du Z, Hao Y. Predicting the incidence of infectious diarrhea with symptom surveillance data using a stacking-based ensembled model. BMC Infect Dis 2024; 24:265. [PMID: 38408967 PMCID: PMC10898154 DOI: 10.1186/s12879-024-09138-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 02/14/2024] [Indexed: 02/28/2024] Open
Abstract
BACKGROUND Infectious diarrhea remains a major public health problem worldwide. This study used stacking ensemble to developed a predictive model for the incidence of infectious diarrhea, aiming to achieve better prediction performance. METHODS Based on the surveillance data of infectious diarrhea cases, relevant symptoms and meteorological factors of Guangzhou from 2016 to 2021, we developed four base prediction models using artificial neural networks (ANN), Long Short-Term Memory networks (LSTM), support vector regression (SVR) and extreme gradient boosting regression trees (XGBoost), which were then ensembled using stacking to obtain the final prediction model. All the models were evaluated with three metrics: mean absolute percentage error (MAPE), root mean square error (RMSE), and mean absolute error (MAE). RESULTS Base models that incorporated symptom surveillance data and weekly number of infectious diarrhea cases were able to achieve lower RMSEs, MAEs, and MAPEs than models that added meteorological data and weekly number of infectious diarrhea cases. The LSTM had the best prediction performance among the four base models, and its RMSE, MAE, and MAPE were: 84.85, 57.50 and 15.92%, respectively. The stacking ensembled model outperformed the four base models, whose RMSE, MAE, and MAPE were 75.82, 55.93, and 15.70%, respectively. CONCLUSIONS The incorporation of symptom surveillance data could improve the predictive accuracy of infectious diarrhea prediction models, and symptom surveillance data was more effective than meteorological data in enhancing model performance. Using stacking to combine multiple prediction models were able to alleviate the difficulty in selecting the optimal model, and could obtain a model with better performance than base models.
Collapse
Affiliation(s)
- Pengyu Wang
- Department of Medical Statistics, School of Public Health & Center for Health Information Research & Sun Yat-sen Global Health Institute, Sun Yat-sen University, Guangzhou, China
| | - Wangjian Zhang
- Department of Medical Statistics, School of Public Health & Center for Health Information Research & Sun Yat-sen Global Health Institute, Sun Yat-sen University, Guangzhou, China
| | - Hui Wang
- Department of Infectious Disease Control and Prevention, Guangzhou Center for Disease Control and Prevention, Guangzhou, China
| | - Congxing Shi
- Department of Medical Statistics, School of Public Health & Center for Health Information Research & Sun Yat-sen Global Health Institute, Sun Yat-sen University, Guangzhou, China
| | - Zhiqiang Li
- Department of Medical Statistics, School of Public Health & Center for Health Information Research & Sun Yat-sen Global Health Institute, Sun Yat-sen University, Guangzhou, China
| | - Dahu Wang
- Department of Infectious Disease Control and Prevention, Guangzhou Center for Disease Control and Prevention, Guangzhou, China
| | - Lei Luo
- Department of Infectious Disease Control and Prevention, Guangzhou Center for Disease Control and Prevention, Guangzhou, China.
| | - Zhicheng Du
- Department of Medical Statistics, School of Public Health & Center for Health Information Research & Sun Yat-sen Global Health Institute, Sun Yat-sen University, Guangzhou, China.
- Guangzhou Joint Research Center for Disease Surveillance and Risk Assessment, Sun Yat-sen University & Guangzhou Center for Disease Control and Prevention, Guangzhou, China.
| | - Yuantao Hao
- Peking University Center for Public Health and Epidemic Preparedness & Response, Beijing, China.
- Department of Epidemiology & Biostatistics, School of Public Health, Peking University, Beijing, China.
- Key Laboratory of Epidemiology of Major Diseases (Peking University), Ministry of Education, Beijing, China.
| |
Collapse
|
3
|
Hongliang G, Zhiyao Z, Ahmadianfar I, Escorcia-Gutierrez J, Aljehane NO, Li C. Multi-step influenza forecasting through singular value decomposition and kernel ridge regression with MARCOS-guided gradient-based optimization. Comput Biol Med 2024; 169:107888. [PMID: 38157778 DOI: 10.1016/j.compbiomed.2023.107888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 11/28/2023] [Accepted: 12/18/2023] [Indexed: 01/03/2024]
Abstract
This research delves into the significance of influenza outbreaks in public health, particularly the importance of accurate forecasts using weekly Influenza-like illness (ILI) rates. The present work develops a novel hybrid machine-learning model by combining singular value decomposition with kernel ridge regression (SKRR). In this context, a novel hybrid model known as H-SKRR is developed by combining two robust forecasting approaches, SKRR and ridge regression, which aims to improve multi-step-ahead predictions for weekly ILI rates in Southern and Northern China. The study begins with feature selection via XGBoost in the preprocessing phase, identifying optimal precursor information guided by importance factors. It decomposes the original signal using multivariate variational mode decomposition (MVMD) to address non-stationarity and complexity. H-SKRR is implemented by incorporating significant lagged-time components across sub-components. The aggregated forecasted values from these sub-components generate ILI values for two horizons (i.e., 4-and 7-weekly ahead). Employing the gradient-based optimization (GBO) algorithm fine-tunes model parameters. Furthermore, the deep random vector functional link (dRVFL), Ridge regression, and gated recurrent unit neural network (GRU) models were employed to validate the MVMD-H-SKRR-GBO paradigm's effectiveness. The outcomes, assessed using the MARCOS (Measurement of alternatives and ranking according to compromise solution) method as a multi-criteria decision-making method, highlight the superior accuracy of the MVMD-H-SKRR-GBO model in predicting ILI rates. The results clearly highlight the exceptional performance of the MVMD-H-SKRR-GBO model, with outstanding precision demonstrated by impressive R, RMSE, IA, and U95 % values of 0.946, 0.388, 0.970, and 1.075, respectively, at t + 7.
Collapse
Affiliation(s)
- Guo Hongliang
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Zhang Zhiyao
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Iman Ahmadianfar
- Information and Communication Technology Research Group, Scientific Research Center, Al-Ayen University, Thi-Qar, Nasiriyah, 64001, Iraq.
| | - José Escorcia-Gutierrez
- Department of Computational Science and Electronics, Universidad de La Costa, CUC, Barranquilla, 080002, Colombia.
| | - Nojood O Aljehane
- Faculty of Computers and Information Technology, University of Tabuk, Tabuk, Saudi Arabia, Tabuk University, KSA.
| | - Chengye Li
- Department of Pulmonary and Critical Care Medicine, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, 325000, China.
| |
Collapse
|
4
|
Mellor J, Christie R, Overton CE, Paton RS, Leslie R, Tang M, Deeny S, Ward T. Forecasting influenza hospital admissions within English sub-regions using hierarchical generalised additive models. COMMUNICATIONS MEDICINE 2023; 3:190. [PMID: 38123630 PMCID: PMC10733380 DOI: 10.1038/s43856-023-00424-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 12/06/2023] [Indexed: 12/23/2023] Open
Abstract
BACKGROUND Seasonal influenza places a substantial burden annually on healthcare services. Policies during the COVID-19 pandemic limited the transmission of seasonal influenza, making the timing and magnitude of a potential resurgence difficult to ascertain and its impact important to forecast. METHODS We have developed a hierarchical generalised additive model (GAM) for the short-term forecasting of hospital admissions with a positive test for the influenza virus sub-regionally across England. The model incorporates a multi-level structure of spatio-temporal splines, weekly cycles in admissions, and spatial correlation. Using multiple performance metrics including interval score, coverage, bias, and median absolute error, the predictive performance is evaluated for the 2022-2023 seasonal wave. Performance is measured against autoregressive integrated moving average (ARIMA) and Prophet time series models. RESULTS Across the epidemic phases the hierarchical GAM shows improved performance, at all geographic scales relative to the ARIMA and Prophet models. Temporally, the hierarchical GAM has overall an improved performance at 7 and 14 day time horizons. The performance of the GAM is most sensitive to the flexibility of the smoothing function that measures the national epidemic trend. CONCLUSIONS This study introduces an approach to short-term forecasting of hospital admissions for the influenza virus using hierarchical, spatial, and temporal components. The methodology was designed for the real time forecasting of epidemics. This modelling framework was used across the 2022-2023 winter for healthcare operational planning by the UK Health Security Agency and the National Health Service in England.
Collapse
Affiliation(s)
- Jonathon Mellor
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom.
| | - Rachel Christie
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| | - Christopher E Overton
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
- University of Liverpool, Department of Mathematical Sciences, Liverpool, United Kingdom
| | - Robert S Paton
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| | - Rhianna Leslie
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| | - Maria Tang
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| | - Sarah Deeny
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| | - Thomas Ward
- UK Health Security Agency, Data Analytics and Surveillance, 10 South Colonnade, London, United Kingdom
| |
Collapse
|
5
|
Ondrikova N, Clough H, Douglas A, Vivancos R, Itturiza-Gomara M, Cunliffe N, Harris JP. Comparison of statistical approaches to predicting norovirus laboratory reports before and during COVID-19: insights to inform public health surveillance. Sci Rep 2023; 13:21457. [PMID: 38052922 PMCID: PMC10697939 DOI: 10.1038/s41598-023-48069-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Accepted: 11/22/2023] [Indexed: 12/07/2023] Open
Abstract
Social distancing interrupted transmission patterns of contact-driven infectious agents such as norovirus during the Covid-19 pandemic. Since routine surveillance of norovirus was additionally disrupted during the pandemic, traditional naïve forecasts that rely only on past public health surveillance data may not reliably represent norovirus activity. This study investigates the use of statistical modelling to predict the number of norovirus laboratory reports in England 4-weeks ahead of time before and during Covid-19 pandemic thus providing insights to inform existing practices in norovirus surveillance in England. We compare the predictive performance from three forecasting approaches that assume different underlying structure of the norovirus data and utilized various external data sources including mobility, air temperature and relative internet searches (Time Series and Regularized Generalized Linear Model, and Quantile Regression Forest). The performance of each approach was evaluated using multiple metrics, including a relative prediction error against the traditional naive forecast of a five-season mean. Our data suggest that all three forecasting approaches improve predictive performance over the naïve forecasts, especially in the 2020/21 season (30-45% relative improvement) when the number of norovirus reports reduced. The improvement ranged from 7 to 22% before the pandemic. However, performance varied: regularized regression incorporating internet searches showed the best forecasting score pre-pandemic and the time series approach achieved the best results post pandemic onset without external data. Overall, our results demonstrate that there is a significant value for public health in considering the adoption of more sophisticated forecasting tools, moving beyond traditional naïve methods, and utilizing available software to enhance the precision and timeliness of norovirus surveillance in England.
Collapse
Affiliation(s)
- Nikola Ondrikova
- Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK.
- Institute for Risk and Uncertainty, University of Liverpool, Liverpool, UK.
- NIHR Health Protection Research Unit in Gastrointestinal Infections, University of Liverpool, Liverpool, UK.
| | - Helen Clough
- Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK
- NIHR Health Protection Research Unit in Gastrointestinal Infections, University of Liverpool, Liverpool, UK
| | - Amy Douglas
- National Surveillance Gastrointestinal Pathogens Unit, UK Health Security Agency, London, UK
| | - Roberto Vivancos
- NIHR Health Protection Research Unit in Gastrointestinal Infections, University of Liverpool, Liverpool, UK
- Health Protection Operations, UK Health Security Agency, Liverpool, UK
- NIHR Health Protection Research Unit in Emerging and Zoonotic Infections, University of Liverpool, Liverpool, UK
| | | | - Nigel Cunliffe
- Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK
- NIHR Health Protection Research Unit in Gastrointestinal Infections, University of Liverpool, Liverpool, UK
| | - John P Harris
- Health Protection Operations, UK Health Security Agency, Liverpool, UK
| |
Collapse
|
6
|
Marquez E, Barrón-Palma EV, Rodríguez K, Savage J, Sanchez-Sandoval AL. Supervised Machine Learning Methods for Seasonal Influenza Diagnosis. Diagnostics (Basel) 2023; 13:3352. [PMID: 37958248 PMCID: PMC10647880 DOI: 10.3390/diagnostics13213352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 10/24/2023] [Accepted: 10/25/2023] [Indexed: 11/15/2023] Open
Abstract
Influenza has been a stationary disease in Mexico since 2009, and this causes a high cost for the national public health system, including its detection using RT-qPCR tests, treatments, and absenteeism in the workplace. Despite influenza's relevance, the main clinical features to detect the disease defined by international institutions like the World Health Organization (WHO) and the United States Centers for Disease Control and Prevention (CDC) do not follow the same pattern in all populations. The aim of this work is to find a machine learning method to facilitate decision making in the clinical differentiation between positive and negative influenza patients, based on their symptoms and demographic features. The research sample consisted of 15480 records, including clinical and demographic data of patients with a positive/negative RT-qPCR influenza tests, from 2010 to 2020 in the public healthcare institutions of Mexico City. The performance of the methods for classifying influenza cases were evaluated with indices like accuracy, specificity, sensitivity, precision, the f1-measure and the area under the curve (AUC). Results indicate that random forest and bagging classifiers were the best supervised methods; they showed promise in supporting clinical diagnosis, especially in places where performing molecular tests might be challenging or not feasible.
Collapse
Affiliation(s)
- Edna Marquez
- Genomic Medicine Department, General Hospital of México “Dr. Eduardo Liceaga”, Mexico City 06726, Mexico; (E.V.B.-P.)
| | - Eira Valeria Barrón-Palma
- Genomic Medicine Department, General Hospital of México “Dr. Eduardo Liceaga”, Mexico City 06726, Mexico; (E.V.B.-P.)
| | - Katya Rodríguez
- Institute for Research in Applied Mathematics and Systems, National Autonomous University of Mexico, Mexico City 04510, Mexico;
| | - Jesus Savage
- Signal Processing Department, Engineering School, National Autonomous University of Mexico, Mexico City 04510, Mexico;
| | - Ana Laura Sanchez-Sandoval
- Genomic Medicine Department, General Hospital of México “Dr. Eduardo Liceaga”, Mexico City 06726, Mexico; (E.V.B.-P.)
| |
Collapse
|
7
|
Chong KC, Chan PKS, Lee TC, Lau SYF, Wu P, Lai CKC, Fung KSC, Tse CWS, Leung SY, Kwok KL, Li C, Jiang X, Wei Y. Determining meteorologically-favorable zones for seasonal influenza activity in Hong Kong. INTERNATIONAL JOURNAL OF BIOMETEOROLOGY 2023; 67:609-619. [PMID: 36847884 DOI: 10.1007/s00484-023-02439-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 02/03/2023] [Accepted: 02/08/2023] [Indexed: 06/18/2023]
Abstract
Investigations of simple and accurate meteorology classification systems for influenza epidemics, particularly in subtropical regions, are limited. To assist in preparing for potential upsurges in the demand on healthcare facilities during influenza seasons, our study aims to develop a set of meteorologically-favorable zones for epidemics of influenza A and B, defined as the intervals of meteorological variables with prediction performance optimized. We collected weekly detection rates of laboratory-confirmed influenza cases from four local major hospitals in Hong Kong between 2004 and 2019. Meteorological and air quality records for hospitals were collected from their closest monitoring stations. We employed classification and regression trees to identify zones that optimize the prediction performance of meteorological data in influenza epidemics, defined as a weekly rate > 50th percentile over a year. According to the results, a combination of temperature > 25.1℃ and relative humidity > 79% was favorable to epidemics in hot seasons, whereas either temperature < 16.4℃ or a combination of < 20.4℃ and relative humidity > 76% was favorable to epidemics in cold seasons. The area under the receiver operating characteristic curve (AUC) in model training achieved 0.80 (95% confidence interval [CI], 0.76-0.83) and was kept at 0.71 (95%CI, 0.65-0.77) in validation. The meteorologically-favorable zones for predicting influenza A or A and B epidemics together were similar, but the AUC for predicting influenza B epidemics was comparatively lower. In conclusion, we established meteorologically-favorable zones for influenza A and B epidemics with a satisfactory prediction performance, even though the influenza seasonality in this subtropical setting was weak and type-specific.
Collapse
Affiliation(s)
- Ka Chun Chong
- Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
- Centre for Health Systems and Policy Research, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Paul K S Chan
- Department of Microbiology, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Tsz Cheung Lee
- Hong Kong Observatory, Hong Kong Special Administrative Region, China
| | - Steven Y F Lau
- Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Peng Wu
- School of Public Health, The University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Christopher K C Lai
- Department of Microbiology, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Kitty S C Fung
- Department of Pathology, United Christian Hospital, Hong Kong Special Administrative Region, China
| | - Cindy W S Tse
- Department of Pathology, Kwong Wah Hospital, Hong Kong Special Administrative Region, China
| | - Shuk Yu Leung
- Department of Paediatrics, Kwong Wah Hospital, Hong Kong Special Administrative Region, China
| | - Ka Li Kwok
- Department of Paediatrics, Kwong Wah Hospital, Hong Kong Special Administrative Region, China
| | - Conglu Li
- Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Xiaoting Jiang
- Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Yuchen Wei
- Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China.
- Centre for Health Systems and Policy Research, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China.
| |
Collapse
|
8
|
Moon J, Noh Y, Park S, Hwang E. Model-Agnostic Meta-Learning-based Region-Adaptive Parameter Adjustment Scheme for Influenza Forecasting. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2022. [DOI: 10.1016/j.jksuci.2022.11.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
9
|
Forecasting the Potential Number of Influenza-like Illness Cases by Fusing Internet Public Opinion. SUSTAINABILITY 2022. [DOI: 10.3390/su14052803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
As influenza viruses mutate rapidly, a prediction model for potential outbreaks of influenza-like illnesses helps detect the spread of the illnesses in real time. In order to create a better prediction model, in this study, in addition to using the traditional hydrological and atmospheric data, features, such as popular search keywords on Google Trends, public holiday information, population density, air quality indices, and the numbers of COVID-19 confirmed cases, were also used to train the model in this research. Furthermore, Random Forest and XGBoost were combined and used in the proposed prediction model to increase the prediction accuracy. The training data used in this research were the historical data taken from 2016 to 2021. In our experiments, different combinations of features were tested. The results show that features, such as popular search keywords on Google Trends, the numbers of COVID-19 confirmed cases, and air quality indices can improve the outcome of the prediction model. The evaluation results showed that the error rate between the predicted results and the actual number of influenza-like cases form Week 15 to Week 18 fell to less than 5%. The outbreak of COVID-19 in Taiwan began in Week 19 and resulted in a sharp rise in the number of clinic or hospital visits by patients of influenza-like illnesses. After that, from Week 21 to Week 26, the error rate between the predicted and actual numbers of influenza-like cases in the later period dropped down to 13%. It can be confirmed from the actual experimental results in this research that the use of the ensemble learning prediction model proposed in this research can accurately predict the trend of influenza-like cases.
Collapse
|
10
|
A Decision-Level Fusion Method for COVID-19 Patient Health Prediction. BIG DATA RESEARCH 2022; 27:100287. [PMCID: PMC8574072 DOI: 10.1016/j.bdr.2021.100287] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Revised: 08/11/2021] [Accepted: 10/28/2021] [Indexed: 06/16/2023]
Abstract
With the continuous attempts to develop effective machine learning methods, information fusion approaches play an important role in integrating data from multiple sources and improving these methods' performance. Among the different fusion techniques, decision-level fusion has unique advantages to fuse the decisions of various classifiers and getting an effective outcome. In this paper, we propose a decision-level fusion method that combines three well-calibrated ensemble classifiers, namely, a random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGB) methods. It is used to predict the COVID-19 patient health for early monitoring and efficient treatment. A soft voting technique is used to generate the final decision result from the predictions of these calibrated classifiers. The method uses the COVID-19 patient's health information, travel demographic, and geographical data to predict the possible outcome of the COVID-19 case, recovered, or death. A different set of experiments is conducted on a public novel Corona Virus 2019 dataset using a different ratio of test sets. The experimental results show that the proposed fusion method achieved an accuracy of 97.24% and an F1-score of 0.97, which is higher than the current related work that has an accuracy of 94% and an F1-score 0.86, on 20% test set taken from the dataset.
Collapse
|
11
|
A Comparison of Infectious Disease Forecasting Methods across Locations, Diseases, and Time. Pathogens 2022; 11:pathogens11020185. [PMID: 35215129 PMCID: PMC8875569 DOI: 10.3390/pathogens11020185] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 01/23/2022] [Accepted: 01/27/2022] [Indexed: 02/04/2023] Open
Abstract
Accurate infectious disease forecasting can inform efforts to prevent outbreaks and mitigate adverse impacts. This study compares the performance of statistical, machine learning (ML), and deep learning (DL) approaches in forecasting infectious disease incidences across different countries and time intervals. We forecasted three diverse diseases: campylobacteriosis, typhoid, and Q-fever, using a wide variety of features (n = 46) from public datasets, e.g., landscape, climate, and socioeconomic factors. We compared autoregressive statistical models to two tree-based ML models (extreme gradient boosted trees [XGB] and random forest [RF]) and two DL models (multi-layer perceptron and encoder–decoder model). The disease models were trained on data from seven different countries at the region-level between 2009–2017. Forecasting performance of all models was assessed using mean absolute error, root mean square error, and Poisson deviance across Australia, Israel, and the United States for the months of January through August of 2018. The overall model results were compared across diseases as well as various data splits, including country, regions with highest and lowest cases, and the forecasted months out (i.e., nowcasting, short-term, and long-term forecasting). Overall, the XGB models performed the best for all diseases and, in general, tree-based ML models performed the best when looking at data splits. There were a few instances where the statistical or DL models had minutely smaller error metrics for specific subsets of typhoid, which is a disease with very low case counts. Feature importance per disease was measured by using four tree-based ML models (i.e., XGB and RF with and without region name as a feature). The most important feature groups included previous case counts, region name, population counts and density, mortality causes of neonatal to under 5 years of age, sanitation factors, and elevation. This study demonstrates the power of ML approaches to incorporate a wide range of factors to forecast various diseases, regardless of location, more accurately than traditional statistical approaches.
Collapse
|
12
|
He Y, Zhao Y, Chen Y, Yuan H, Tsui K. Nowcasting influenza‐like illness (ILI) via a deep learning approach using google search data: An empirical study on Taiwan ILI. INT J INTELL SYST 2021. [DOI: 10.1002/int.22788] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Yuxin He
- College of Urban Transportation and Logistics Shenzhen Technology University Shenzhen China
| | - Yang Zhao
- School of Public Health (Shenzhen) Sun Yat‐Sen University Guangzhou China
| | - Yupeng Chen
- Trial Retail Engineering (T. R. E. China) Yantai China
| | - Hsiang‐Yu Yuan
- Department of Biomedical Sciences City University of Hong Kong Hong Kong China
| | - Kwok‐Leung Tsui
- Department of Industrial and Systems Engineering Virginia Polytechnic Institute and State University Blacksburg Virginia USA
| |
Collapse
|
13
|
Yang S, Bao Y. Comprehensive learning particle swarm optimization enabled modeling framework for multi-step-ahead influenza prediction. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107994] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
|
14
|
Lv CX, An SY, Qiao BJ, Wu W. Time series analysis of hemorrhagic fever with renal syndrome in mainland China by using an XGBoost forecasting model. BMC Infect Dis 2021; 21:839. [PMID: 34412581 PMCID: PMC8377883 DOI: 10.1186/s12879-021-06503-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 07/30/2021] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND Hemorrhagic fever with renal syndrome (HFRS) is still attracting public attention because of its outbreak in various cities in China. Predicting future outbreaks or epidemics disease based on past incidence data can help health departments take targeted measures to prevent diseases in advance. In this study, we propose a multistep prediction strategy based on extreme gradient boosting (XGBoost) for HFRS as an extension of the one-step prediction model. Moreover, the fitting and prediction accuracy of the XGBoost model will be compared with the autoregressive integrated moving average (ARIMA) model by different evaluation indicators. METHODS We collected HFRS incidence data from 2004 to 2018 of mainland China. The data from 2004 to 2017 were divided into training sets to establish the seasonal ARIMA model and XGBoost model, while the 2018 data were used to test the prediction performance. In the multistep XGBoost forecasting model, one-hot encoding was used to handle seasonal features. Furthermore, a series of evaluation indices were performed to evaluate the accuracy of the multistep forecast XGBoost model. RESULTS There were 200,237 HFRS cases in China from 2004 to 2018. A long-term downward trend and bimodal seasonality were identified in the original time series. According to the minimum corrected akaike information criterion (CAIC) value, the optimal ARIMA (3, 1, 0) × (1, 1, 0)12 model is selected. The index ME, RMSE, MAE, MPE, MAPE, and MASE indices of the XGBoost model were higher than those of the ARIMA model in the fitting part, whereas the RMSE of the XGBoost model was lower. The prediction performance evaluation indicators (MAE, MPE, MAPE, RMSE and MASE) of the one-step prediction and multistep prediction XGBoost model were all notably lower than those of the ARIMA model. CONCLUSIONS The multistep XGBoost prediction model showed a much better prediction accuracy and model stability than the multistep ARIMA prediction model. The XGBoost model performed better in predicting complicated and nonlinear data like HFRS. Additionally, Multistep prediction models are more practical than one-step prediction models in forecasting infectious diseases.
Collapse
Affiliation(s)
- Cai-Xia Lv
- Department of Epidemiology, School of Public Health, China Medical University, Shenyang, Liaoning China
| | - Shu-Yi An
- Liaoning Provincial Center for Disease Control and Prevention, Shenyang, Liaoning China
| | - Bao-Jun Qiao
- Liaoning Provincial Center for Disease Control and Prevention, Shenyang, Liaoning China
| | - Wei Wu
- Department of Epidemiology, School of Public Health, China Medical University, Shenyang, Liaoning China
| |
Collapse
|
15
|
Jung S, Moon J, Park S, Hwang E. Self-attention-based Deep Learning Network for Regional Influenza Forecasting. IEEE J Biomed Health Inform 2021; 26:922-933. [PMID: 34197330 DOI: 10.1109/jbhi.2021.3093897] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Early prediction of influenza plays an important role in minimizing the damage caused, as it provides the resources and time needed to formulate preventive measures. Compared to traditional mechanistic approach, deep/machine learning-based models have demonstrated excellent forecasting performance by efficiently handling various data such as weather and internet data. However, due to the limited availability and reliability of such data, many forecasting models use only historical occurrence data and formulate the influenza forecasting as a multivariate time-series task. Recently, attention mechanisms have been exploited to deal with this issue by selecting valuable data in the input data and giving them high weights. Particularly, self-attention has shown its potential in various forecasting tasks by utilizing the predictive relationship between objects from the input data describing target objects. Hence, in this study, we propose a forecasting model based on self-attention for regional influenza forecasting, called SAIFlu-Net. The model exploits a long short-term memory network for extracting time-series patterns of each region and the self-attention mechanism to find the similarities between the occurrence patterns. To evaluate its performance, we conducted extensive experiments with existing forecasting models using weekly regional influenza datasets. The results show that the proposed model outperforms other models in terms of root mean square error and Pearson correlation coefficient.
Collapse
|
16
|
Wen A, Wang L, He H, Liu S, Fu S, Sohn S, Kugel JA, Kaggal VC, Huang M, Wang Y, Shen F, Fan J, Liu H. An aberration detection-based approach for sentinel syndromic surveillance of COVID-19 and other novel influenza-like illnesses. J Biomed Inform 2021; 113:103660. [PMID: 33321199 PMCID: PMC7832634 DOI: 10.1016/j.jbi.2020.103660] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Revised: 11/06/2020] [Accepted: 12/09/2020] [Indexed: 02/08/2023]
Abstract
Coronavirus Disease 2019 has emerged as a significant global concern, triggering harsh public health restrictions in a successful bid to curb its exponential growth. As discussion shifts towards relaxation of these restrictions, there is significant concern of second-wave resurgence. The key to managing these outbreaks is early detection and intervention, and yet there is a significant lag time associated with usage of laboratory confirmed cases for surveillance purposes. To address this, syndromic surveillance can be considered to provide a timelier alternative for first-line screening. Existing syndromic surveillance solutions are however typically focused around a known disease and have limited capability to distinguish between outbreaks of individual diseases sharing similar syndromes. This poses a challenge for surveillance of COVID-19 as its active periods tend to overlap temporally with other influenza-like illnesses. In this study we explore performing sentinel syndromic surveillance for COVID-19 and other influenza-like illnesses using a deep learning-based approach. Our methods are based on aberration detection utilizing autoencoders that leverages symptom prevalence distributions to distinguish outbreaks of two ongoing diseases that share similar syndromes, even if they occur concurrently. We first demonstrate that this approach works for detection of outbreaks of influenza, which has known temporal boundaries. We then demonstrate that the autoencoder can be trained to not alert on known and well-managed influenza-like illnesses such as the common cold and influenza. Finally, we applied our approach to 2019-2020 data in the context of a COVID-19 syndromic surveillance task to demonstrate how implementation of such a system could have provided early warning of an outbreak of a novel influenza-like illness that did not match the symptom prevalence profile of influenza and other known influenza-like illnesses.
Collapse
Affiliation(s)
- Andrew Wen
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Liwei Wang
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Huan He
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Sijia Liu
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Sunyang Fu
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Sunghwan Sohn
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Jacob A Kugel
- Advanced Analytics Service Unit, Department of Information Technology, Mayo Clinic, Rochester, MN, USA
| | - Vinod C Kaggal
- Advanced Analytics Service Unit, Department of Information Technology, Mayo Clinic, Rochester, MN, USA
| | - Ming Huang
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Yanshan Wang
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Feichen Shen
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
| | - Jungwei Fan
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA.
| | - Hongfang Liu
- Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|