Forecasting of weekly patient visits to emergency department: real case study

Analyzing and Forecasting Pediatric Fever Clinic Visits in High Frequency Using Ensemble Time-Series Methods After the COVID-19 Pandemic in Hangzhou, China: Retrospective Study

BACKGROUND

The COVID-19 pandemic has significantly altered the global health and medical landscape. In response to the outbreak, Chinese hospitals have established 24-hour fever clinics to serve patients with COVID-19. The emergence of these clinics and the impact of successive epidemics have led to a surge in visits, placing pressure on hospital resource allocation and scheduling. Therefore, accurate prediction of outpatient visits is essential for informed decision-making in hospital management.

OBJECTIVE

Hourly visits to fever clinics can be characterized as a long-sequence time series in high frequency, which also exhibits distinct patterns due to the particularity of pediatric treatment behavior in an epidemic context. This study aimed to build models to forecast fever clinic visit with outstanding prediction accuracy and robust generalization in forecast horizons. In addition, this study hopes to provide a research paradigm for time-series forecasting problems, which involves an exploratory analysis revealing data patterns before model development.

METHODS

An exploratory analysis, including graphical analysis, autocorrelation analysis, and seasonal-trend decomposition, was conducted to reveal the seasonality and structural patterns of the retrospective fever clinic visit data. The data were found to exhibit multiseasonality and nonlinearity. On the basis of these results, an ensemble of time-series analysis methods, including individual models and their combinations, was validated on the data set. Root mean square error and mean absolute error were used as accuracy metrics, with the cross-validation of rolling forecasting origin conducted across different forecast horizons.

RESULTS

Hybrid models generally outperformed individual models across most forecast horizons. A novel model combination, the hybrid neural network autoregressive (NNAR)-seasonal and trend decomposition using Loess forecasting (STLF), was identified as the optimal model for our forecasting task, with the best performance in all accuracy metrics (root mean square error=20.1, mean absolute error=14.3) for the 15-days-ahead forecasts and an overall advantage for forecast horizons that were 1 to 30 days ahead.

CONCLUSIONS

Although forecast accuracy tends to decline with an increasing forecast horizon, the hybrid NNAR-STLF model is applicable for short-, medium-, and long-term forecasts owing to its ability to fit multiseasonality (captured by the STLF component) and nonlinearity (captured by the NNAR component). The model identified in this study is also applicable to hospitals in other regions with similar epidemic outpatient configurations or forecasting tasks whose data conform to long-sequence time series in high frequency exhibiting multiseasonal and nonlinear patterns. However, as external variables and disruptive events were not accounted for, the model performance declined slightly following changes in the COVID-19 containment policy in China. Future work may seek to improve accuracy by incorporating external variables that characterize moving events or other factors as well as by adding data from different organizations to enhance algorithm generalization.

Accurate Forecasting of Emergency Department Arrivals With Internet Search Index and Machine Learning Models: Model Development and Performance Evaluation

BACKGROUND

Emergency department (ED) overcrowding is a concerning global health care issue, which is mainly caused by the uncertainty of patient arrivals, especially during the pandemic. Accurate forecasting of patient arrivals can allow health resource allocation in advance to reduce overcrowding. Currently, traditional data, such as historical patient visits, weather, holiday, and calendar, are primarily used to create forecasting models. However, data from an internet search engine (eg, Google) is less studied, although they can provide pivotal real-time surveillance information. The internet data can be employed to improve forecasting performance and provide early warning, especially during the epidemic. Moreover, possible nonlinearities between patient arrivals and these variables are often ignored.

OBJECTIVE

This study aims to develop an intelligent forecasting system with machine learning models and internet search index to provide an accurate prediction of ED patient arrivals, to verify the effectiveness of the internet search index, and to explore whether nonlinear models can improve the forecasting accuracy.

METHODS

Data on ED patient arrivals were collected from July 12, 2009, to June 27, 2010, the period of the 2009 H1N1 pandemic. These included 139,910 ED visits in our collaborative hospital, which is one of the biggest public hospitals in Hong Kong. Traditional data were also collected during the same period. The internet search index was generated from 268 search queries on Google to comprehensively capture the information about potential patients. The relationship between the index and patient arrivals was verified by Pearson correlation coefficient, Johansen cointegration, and Granger causality. Linear and nonlinear models were then developed with the internet search index to predict patient arrivals. The accuracy and robustness were also examined.

RESULTS

All models could accurately predict patient arrivals. The causality test indicated internet search index as a strong predictor of ED patient arrivals. With the internet search index, the mean absolute percentage error (MAPE) and the root mean square error (RMSE) of the linear model reduced from 5.3% to 5.0% and from 24.44 to 23.18, respectively, whereas the MAPE and RMSE of the nonlinear model decreased even more, from 3.5% to 3% and from 16.72 to 14.55, respectively. Compared with each other, the experimental results revealed that the forecasting system with extreme learning machine, as well as the internet search index, had the best performance in both forecasting accuracy and robustness analysis.

CONCLUSIONS

The proposed forecasting system can make accurate, real-time prediction of ED patient arrivals. Compared with the static traditional variables, the internet search index significantly improves forecasting as a reliable predictor monitoring continuous behavior trend and sudden changes during the epidemic (P=.002). The nonlinear model performs better than the linear counterparts by capturing the dynamic relationship between the index and patient arrivals. Thus, the system can facilitate staff planning and workflow monitoring.

Forecasting daily emergency department arrivals using high-dimensional multivariate data: a feature selection approach

Background and objective

Emergency Department (ED) overcrowding is a chronic international issue that is associated with adverse treatment outcomes. Accurate forecasts of future service demand would enable intelligent resource allocation that could alleviate the problem. There has been continued academic interest in ED forecasting but the number of used explanatory variables has been low, limited mainly to calendar and weather variables. In this study we investigate whether predictive accuracy of next day arrivals could be enhanced using high number of potentially relevant explanatory variables and document two feature selection processes that aim to identify which subset of variables is associated with number of next day arrivals. Performance of such predictions over longer horizons is also shown.

Methods

We extracted numbers of total daily arrivals from Tampere University Hospital ED between the time period of June 1, 2015 and June 19, 2019. 158 potential explanatory variables were collected from multiple data sources consisting not only of weather and calendar variables but also an extensive list of local public events, numbers of website visits to two hospital domains, numbers of available hospital beds in 33 local hospitals or health centres and Google trends searches for the ED. We used two feature selection processes: Simulated Annealing (SA) and Floating Search (FS) with Recursive Least Squares (RLS) and Least Mean Squares (LMS). Performance of these approaches was compared against autoregressive integrated moving average (ARIMA), regression with ARIMA errors (ARIMAX) and Random Forest (RF). Mean Absolute Percentage Error (MAPE) was used as the main error metric.

Results

Calendar variables, load of secondary care facilities and local public events were dominant in the identified predictive features. RLS-SA and RLS-FA provided slightly better accuracy compared ARIMA. ARIMAX was the most accurate model but the difference between RLS-SA and RLS-FA was not statistically significant.

Conclusions

Our study provides new insight into potential underlying factors associated with number of next day presentations. It also suggests that predictive accuracy of next day arrivals can be increased using high-dimensional feature selection approach when compared to both univariate and nonfiltered high-dimensional approach. Performance over multiple horizons was similar with a gradual decline for longer horizons. However, outperforming ARIMAX remains a challenge when working with daily data. Future work should focus on enhancing the feature selection mechanism, investigating its applicability to other domains and in identifying other potentially relevant explanatory variables.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01878-7.

Effectiveness of cascading time series models based on meteorological factors in improving health risk prediction

Meteorological factors, which are periodic and regular in a long run, have an unignorable impact on human health. Accurate health risk prediction based on meteorological factors is essential for optimal allocation of resource in healthcare units. However, due to the non-stationary and non-linear nature of the original hospitalization sequence, traditional methods are less robust in predicting it. This study aims to investigate hospital admission prediction models using time series pre-processing algorithms and deep learning approach based on meteorological factors. Using the electronic medical record data from Panyu Central Hospital and meteorological data of Panyu district from 2003 to 2019, 46,089 eligible patients with lower respiratory tract infections (LRTIs) and four meteorological factors were identified to build and evaluate the prediction models. A novel hybrid model, Cascade GAM-CEEMDAN-LSTM Model (CGCLM), was established in combination with generalized additive model (GAM), complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), and long-short term memory (LSTM) networks for predicting daily admissions of patients with LRTIs. The experimental results show that CGCLM multistep method proposed in this paper outperforms single LSTM model in the prediction of health risk time series at different time window sizes. Moreover, our results also indicate that CGCLM has the best prediction performance when the time window is set to 61 days (RMSE = 1.12, MAE = 0.87, R² = 0.93). Adequate extraction of exposure-response relationships between meteorological factors and diseases and suitable handling of sequence pre-processing have an important role in time series prediction. This hybrid climate-based model for predicting LRTIs disease can also be extended to time series prediction of other epidemic disease.

Forecasting patient arrivals at emergency department using calendar and meteorological information

Overcrowding in emergency departments (EDs) is a serious problem in many countries. Accurate ED patient arrival forecasts can serve as a management baseline to better allocate ED personnel and medical resources. We combined calendar and meteorological information and used ten modern machine learning methods to forecast patient arrivals. For daily patient arrival forecasting, two feature selection methods are proposed. One uses kernel principal component analysis(KPCA) to reduce the dimensionality of all of the features, and the other is to use the maximal information coefficient(MIC) method to select the features related to the daily data first and then perform KPCA dimensionality reduction. The current study focuses on a public hospital ED in Hefei, China. We used the data November 1, 2019 to August 31, 2020 for model training; and patient arrival data September 1, 2020 to November 31, 2020 for model validation. The results show that for hourly patient arrival forecasting, each machine learning model has better forecasting results than the traditional autoRegressive integrated moving average (ARIMA) model, especially long short-term memory (LSTM) model. For daily patient arrival forecasting, the feature selection method based on MIC-KPCA has a better forecasting effect, and the simpler models are better than the ensemble models. The method we proposed could be used for better planning of ED personnel resources.

Performance evaluation of Emergency Department patient arrivals forecasting models by including meteorological and calendar information: A comparative study

The volume of daily patient arrivals at Emergency Departments (EDs) is unpredictable and is a significant reason of ED crowding in hospitals worldwide. Timely forecast of patients arriving at ED can help the hospital management in early planning and avoiding of overcrowding. Many different ED patient arrivals forecasting models have previously been proposed by using time series analysis methods. Even though the time series methods such as Linear and Logistic Regression, Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA (SARIMA), Exponential Smoothing (ES), and Artificial Neural Network (ANN) have been explored extensively for the ED forecasting model development, the few significant limitations of these methods associated in the analysis of time series data make the models inadequate in many practical situations. Therefore, in this paper, Machine Learning (ML)-based Random Forest (RF) regressor, and Deep Neural Network (DNN)-based Long Short-Term Memory (LSTM) and Convolutional Neural network (CNN) methods, which have not been explored to the same extent as the other time series techniques, are implemented by incorporating meteorological and calendar parameters for the development of forecasting models. The performances of the developed three models in forecasting ED patient arrivals are evaluated. Among the three models, CNN outperformed for short-term (3 days in advance) patient arrivals prediction with Mean Absolute Percentage Error (MAPE) of 9.24% and LSTM performed better for moderate-term (7 days in advance) patient arrivals prediction with MAPE of 8.91% using weather forecast information. Whereas, LSTM model outperformed with MAPE of 8.04% compared to 9.53% by CNN and 10.10% by RF model for current day prediction of patient arrivals using 3 days past weather information. Thus, for short-term ED patient arrivals forecasting, DNN-based model performed better compared to RF regressor ML-based model.

Medical service demand forecasting using a hybrid model based on ARIMA and self-adaptive filtering method

BACKGROUND

Accurate forecasting of medical service demand is beneficial for the reasonable healthcare resource planning and allocation. The daily outpatient volume is characterized by randomness, periodicity and trend, and the time series methods, like ARIMA are often used for short-term outpatient visits forecasting. Therefore, to further enlarge the prediction horizon and improve the prediction accuracy, a hybrid prediction model integrating ARIMA and self-adaptive filtering method is proposed.

METHODS

The ARIMA model is first used to identify the features like cyclicity and trend of the time series data and to estimate the model parameters. The parameters are then adjusted by the steepest descent algorithm in the adaptive filtering method to reduce the prediction error. The hybrid model is validated and compared with traditional ARIMA by several test sets from the Time Series Data Library (TSDL), a weekly emergency department (ED) visit case from literature study, and the real cases of prenatal examinations and B-ultrasounds in a maternal and child health care center (MCHCC) in Ningbo.

RESULTS

For TSDL cases the prediction accuracy of the hybrid prediction is improved by 80-99% compared with the ARIMA model. For the weekly ED visit case, the forecasting results of the hybrid model are better than those of both traditional ARIMA and ANN model, and similar to the ANN combined data decomposition model mentioned in the literature. For the actual data of MCHCC in Ningbo, the MAPE predicted by the ARIMA model in the two departments was 18.53 and 27.69%, respectively, and the hybrid models were 2.79 and 1.25%, respectively.

CONCLUSIONS

The hybrid prediction model outperforms the traditional ARIMA model in both accurate predicting result with smaller average relative error and the applicability for short-term and medium-term prediction.

Prediction of Daily Blood Sampling Room Visits Based on ARIMA and SES Model

This paper is aimed at establishing a combined prediction model to predict the demand for medical care in terms of daily visits in an outpatient blood sampling room, which provides a basis for rational arrangement of human resources and planning. On the basis of analyzing the comprehensive characteristics of the randomness, periodicity, trend, and day-of-the-week effects of the daily number of blood collections in the hospital, we firstly established an autoregressive integrated moving average model (ARIMA) model to capture the periodicity, volatility, and trend, and secondly, we constructed a simple exponential smoothing (SES) model considering the day-of-the-week effect. Finally, a combined prediction model of the residual correction is established based on the prediction results of the two models. The models are applied to data from 60 weeks of daily visits in the outpatient blood sampling room of a large hospital in Chengdu, for forecasting the daily number of blood collections about 1 week ahead. The result shows that the MAPE of the combined model is the smallest overall, of which the improvement during the weekend is obvious, indicating that the prediction error of extreme value is significantly reduced. The ARIMA model can extract the seasonal and nonseasonal components of the time series, and the SES model can capture the overall trend and the influence of regular changes in the time series, while the combined prediction model, taking into account the comprehensive characteristics of the time series data, has better fitting prediction accuracy than a single model. The new model can well realize the short-to-medium-term prediction of the daily number of blood collections one week in advance.

Forecasting emergency admissions due to respiratory diseases in high variability scenarios using time series: A case study in Chile

Respiratory diseases are ranked in the top ten group of the most frequent illness in the globe. Emergency admissions are proof of this issue, especially in the winter season. For this study, the city of Santiago de Chile was chosen because of the high variability of the time series for admissions, the quality of data collected in the governmental repository DEIS (selected period: 2014-2018), and the poor ventilation conditions of the city, which in winter contributes to increase the pollution level, and therefore, respiratory emergency admissions. Different forecasting models were reviewed using the Akaike Information Criteria (AIC) with other error estimators, such as the Root Mean Square Error (RMSE), for selecting the best approach. At the end, Seasonal Autoregressive Integrated Moving Average (SARIMA) model, with parameters (p,d,q)(P,D,Q)_s=(2,1,3)(3,0,2)₇, was selected. The Mean Average Percentage Error (MAPE) for this model was 7.81%. After selection, an investigation of its performance was made using a cross-validation through a rolling window analysis, forecasting up to 30 days ahead (testing period of one year). The results showed that error do not exceed a MAPE of 20%. This allows taking better resource managing decisions in real scenarios: reactive staff hiring is avoided given the reduction of uncertainty for the medium term forecast, which translates into lower costs. Finally, a methodology for the selection of forecasting models is proposed, which includes other constraints from resource management, as well as the different impacts for social well-being.