Designing a hybrid reinforcement learning based algorithm with application in prediction of the COVID-19 pandemic in Quebec

Multi-mitigation strategies in medical supplies for epidemic outbreaks

The outbreak of Coronavirus disease 2019 (COVID-19) highlights the importance of sufficient medical supplies stockpiling at the pre-event stage. In contrast, the potential disadvantages of maintaining adequate items at strategic locations (i.e., reserves) are considerable inventory-related costs. Unpredicted demand leads to a high degree of uncertainty. Efforts to mitigate the uncertainty should rely not only on prepositioning supplies at reserves but also on integrating various channels of medical materials. This paper proposes multi-mitigation strategies in medical supplies to ensure uninterrupted supply for hospitals and significant savings by introducing two-type suppliers, reserving and manufacturing suppliers. Thus, each hospital with uncertain demand is enabled to be served by various channels during pandemics: prepositioning in reserves, backups served by reserving suppliers, and medical commodities produced by manufacturing suppliers. Stochasticity is also incorporated into the raw materials available to produce. This research aims to develop an emergency response application that integrates preparedness action (reserve location, inventory level, and contract supplier's selection) with post-event operations (allocating medical materials from various channels). We formulate a two-stage stochastic mixed integer program to determine prepositioning strategy, including two-type suppliers' selection, and post-event allocation of multiple sources. A branch-and-Benders-cut method is developed for this problem and significantly outperforms both the classical Benders decomposition and Gurobi in the solution time. Different-sized test instances also verify the robustness of the proposed method. Based on a realistic and typical case study (inspired by the COVID-19 pandemic in Wuhan, China), significant savings, an increase in inventory utilization and an increase in demand fulfilment are obtained by our approach.

Resilient and social health service network design to reduce the effect of COVID-19 outbreak

With the severe outbreak of the novel coronavirus (COVID-19), researchers are motivated to develop efficient methods to face related issues. The present study aims to design a resilient health system to offer medical services to COVID-19 patients and prevent further disease outbreaks by social distancing, resiliency, cost, and commuting distance as decisive factors. It incorporated three novel resiliency measures (i.e., health facility criticality, patient dissatisfaction level, and dispersion of suspicious people) to promote the designed health network against potential infectious disease threats. Also, it introduced a novel hybrid uncertainty programming to resolve a mixed degree of the inherent uncertainty in the multi-objective problem, and it adopted an interactive fuzzy approach to address it. The actual data obtained from a case study in Tehran province in Iran proved the strong performance of the presented model. The findings show that the optimum use of medical centers' potential and the corresponding decisions result in a more resilient health system and cost reduction. A further outbreak of the COVID-19 pandemic is also prevented by shortening the commuting distance for patients and avoiding the increasing congestion in the medical centers. Also, the managerial insights show that establishing and evenly distributing camps and quarantine stations within the community and designing an efficient network for patients with different symptoms result in the optimum use of the potential capacity of medical centers and a decrease in the rate of bed shortage in the hospitals. Another insight drawn is that an efficient allocation of the suspect and definite cases to the nearest screening and care centers makes it possible to prevent the disease carriers from commuting within the community and increase the coronavirus transmission rate.

Efficiency of government policy during the COVID-19 pandemic

We introduce country-month indices of efficiency of government policy in dealing with the COVID-19 pandemic. Our indices cover 81 countries and the period from May 2020 to November 2021. Our framework assumes that governments impose stringent policies (listed in the Oxford COVID-19 Containment and Health Index) with the single goal of saving lives. We find that positive and significant correlates of our new indices are institutions, democratic principles, political stability, trust, high public spending in health, female participation in the workplace, and economic equality. Within the efficient jurisdictions, the most efficient ones are those with cultural characteristics of high patience.

Prescriptive Analytics-Based SIRM Model for Predicting Covid-19 Outbreak

Predicting the outbreak of a pandemic is an important measure in order to help saving people lives threatened by Covid-19. Having information about the possible spread of the pandemic, authorities and people can make better decisions. For example, such analyses help developing better strategies for distributing vaccines and medicines. This paper has modified the original Susceptible-Infectious-Recovered (SIR) model to Susceptible-Immune-Infected-Recovered (SIRM) which includes the Immunity ratio as a parameter to enhance the prediction of the pandemic. SIR is a widely used model to predict the spread of a pandemic. Many types of pandemics imply many variants of the SIR models which make it very difficult to find out the best model that matches the running pandemic. The simulation of this paper used the published data about the spread of the pandemic in order to examine our new SIRM. The results showed clearly that our new SIRM covering the aspects of vaccine and medicine is an appropriate model to predict the behavior of the pandemic.

Rationing scarce healthcare capacity: A study of the ventilator allocation guidelines during the COVID-19 pandemic

In the United States, even though national guidelines for allocating scarce healthcare resources are lacking, 26 states have specific ventilator allocation guidelines to be invoked in case of a shortage. While several states developed their guidelines in response to the recent COVID-19 pandemic, New York State developed these guidelines in 2015 as "pandemic influenza is a foreseeable threat, one that we cannot ignore." The primary objective of this study is to assess the existing procedures and priority rules in place for allocating/rationing scarce ventilator capacity and propose alternative (and improved) priority schemes. We first build machine learning models using inpatient records of COVID-19 patients admitted to New York-Presbyterian/Columbia University Irving Medical Center and an affiliated community health center to predict survival probabilities as well as ventilator length-of-use. Then, we use the resulting point estimators and their uncertainties as inputs for a multiclass priority queueing model with abandonments to assess three priority schemes: (i) SOFA-P (Sequential Organ Failure Assessment based prioritization), which most closely mimics the existing practice by prioritizing patients with sufficiently low SOFA scores; (ii) ISP (incremental survival probability), which assigns priority based on patient-level survival predictions; and (iii) ISP-LU (incremental survival probability per length-of-use), which takes into account survival predictions and resource use duration. Our findings highlight that our proposed priority scheme, ISP-LU, achieves a demonstrable improvement over the other two alternatives. Specifically, the expected number of survivals increases and death risk while waiting for ventilator use decreases. We also show that ISP-LU is a robust priority scheme whose implementation yields a Pareto-improvement over both SOFA-P and ISP in terms of maximizing saved lives after mechanical ventilation while limiting racial disparity in access to the priority queue.

A simulation-deep reinforcement learning (SiRL) approach for epidemic control optimization

In this paper, we address the controversies of epidemic control planning by developing a novel Simulation-Deep Reinforcement Learning (SiRL) model. COVID-19 reminded constituents over the world that government decision-making could change their lives. During the COVID-19 pandemic, governments were concerned with reducing fatalities as the virus spread but at the same time also maintaining a flowing economy. In this paper, we address epidemic decision-making regarding the interventions necessary given of the epidemic based on the purpose of the decision-maker. Further, we intend to compare different vaccination strategies, such as age-based and random vaccination, to shine a light on who should get priority in the vaccination process. To address these issues, we propose a simulation-deep reinforcement learning (DRL) framework. This framework is composed of an agent-based simulation model and a governor DRL agent that can enforce interventions in the agent-based simulation environment. Computational results show that our DRL agent can learn effective strategies and suggest optimal actions given a specific epidemic situation based on a multi-objective reward structure. We compare our DRL agent's decisions to government interventions at different periods of time during the COVID-19 pandemic. Our results suggest that more could have been done to control the epidemic. In addition, if a random vaccination strategy that allows super-spreaders to get vaccinated early were used, infections would have been reduced by 32% at the expense of 4% more deaths. We also show that a behavioral change of fully quarantining 10% of the risky individuals and using a random vaccination strategy leads to a reduction of the death toll by 14% and 27% compared to the age-based vaccination strategy that was implemented and the New Jersey reported data, respectively. We have also demonstrated the flexibility of our approach to be applied to other locations by validating and applying our model to the COVID-19 case in the state of Kansas.

A Robust Design-Based Expert System for Feature Selection and COVID-19 Pandemic Prediction in Japan

Expert systems are frequently used to make predictions in various areas. However, the practical robustness of expert systems is not as good as expected, mainly due to the fact that finding an ideal system configuration from a specific dataset is a challenging task. Therefore, how to optimize an expert system has become an important issue of research. In this paper, a new method called the robust design-based expert system is proposed to bridge this gap. The technical process of this system consists of data initialization, configuration generation, a genetic algorithm (GA) framework for feature selection, and a robust mechanism that helps the system find a configuration with the highest robustness. The system will finally obtain a set of features, which can be used to predict a pandemic based on given data. The robust mechanism can increase the efficiency of the system. The configuration for training is optimized by means of a genetic algorithm (GA) and the Taguchi method. The effectiveness of the proposed system in predicting epidemic trends is examined using a real COVID-19 dataset from Japan. For this dataset, the average prediction accuracy was 60%. Additionally, 10 representative features were also selected, resulting in a selection rate of 67% with a reduction rate of 33%. The critical features for predicting the epidemic trend of COVID-19 were also obtained, including new confirmed cases, ICU patients, people vaccinated, population, population density, hospital beds per thousand, middle age, aged 70 or older, and GDP per capital. The main contribution of this paper is two-fold: Firstly, this paper has bridged the gap between the pandemic research and expert systems with robust predictive performance. Secondly, this paper proposes a feature selection method for extracting representative variables and predicting the epidemic trend of a pandemic disease. The prediction results indicate that the system is valuable to healthcare authorities and can help governments get hold of the epidemic trend and strategize their use of healthcare resources.

Viable healthcare supply chain network design for a pandemic

The recent COVID-19 pandemic revealed that healthcare networks must have a flexible and effective structure. In this study, we develop a viable healthcare network design for a pandemic using a multi-stage stochastic approach. We propose a multi-level network that includes health centers, computed tomography scan centers, hospitals, and clinics. Patients have conditions to returning to normal life or quarantining at home. Three objectives are defined: maximizing the probability of patient recovery, minimizing the costs of all centers in the network, and minimizing the Coronavirus death rate. We investigate a real case study in Iran to demonstrate the model's applicability. Finally, we compare the healthcare supply chain network design in a pandemic with a normal situation to advise how the network can continue to remain viable.

Point and interval prediction of crude oil futures prices based on chaos theory and multiobjective slime mold algorithm

Crude oil is the most important energy source in the world, and fluctuations in oil prices can significantly influence investors, companies, and governments. However, crude oil prices have numerous characteristics, including randomness, sudden structural changes, intrinsic nonlinearity, volatility, and chaotic nature. This makes the accurate forecasting of crude oil prices a difficult and challenging task. In this paper, a hybrid prediction model for crude oil futures prices is proposed, the accuracy and robustness of which are demonstrated via controlled experiments and sensitivity analysis. This study uses a new data denoising method for data processing to improve the accuracy and stability of the predictions of crude oil prices. Furthermore, the chaotic time-series prediction method, shallow neural networks, linear model prediction methods, and deep learning methods are adopted as submodels. The results of interval forecasts with narrow widths and high prediction accuracies are derived by introducing a confidence interval adjustment coefficient. The results of the simulation experiments indicate that the proposed hybrid prediction model exhibits higher accuracy and efficiency, as well as better robustness of the forecasting than the control models. In summary, the proposed forecasting framework can derive accurate point and interval forecasts and provide a valuable reference for the price forecasting of crude oil futures.

Deep recurrent reinforced learning model to compare the efficacy of targeted local versus national measures on the spread of COVID-19 in the UK

OBJECTIVES

To prevent the emergence of new waves of COVID-19 caseload and associated mortalities, it is imperative to understand better the efficacy of various control measures on the national and local development of this pandemic in space-time, characterise hotspot regions of high risk, quantify the impact of under-reported measures such as international travel and project the likely effect of control measures in the coming weeks.

METHODS

We applied a deep recurrent reinforced learning based model to evaluate and predict the spatiotemporal effect of a combination of control measures on COVID-19 cases and mortality at the local authority (LA) and national scale in England, using data from week 5 to 46 of 2020, including an expert curated control measure matrix, official statistics/government data and a secure web dashboard to vary magnitude of control measures.

RESULTS

Model predictions of the number of cases and mortality of COVID-19 in the upcoming 5 weeks closely matched the actual values (cases: root mean squared error (RMSE): 700.88, mean absolute error (MAE): 453.05, mean absolute percentage error (MAPE): 0.46, correlation coefficient 0.42; mortality: RMSE 14.91, MAE 10.05, MAPE 0.39, correlation coefficient 0.68). Local lockdown with social distancing (LD_SD) (overall rank 3) was found to be ineffective in preventing outbreak rebound following lockdown easing compared with national lockdown (overall rank 2), based on prediction using simulated control measures. The ranking of the effectiveness of adjunctive measures for LD_SD were found to be consistent across hotspot and non-hotspot regions. Adjunctive measures found to be most effective were international travel and quarantine restrictions.

CONCLUSIONS

This study highlights the importance of using adjunctive measures in addition to LD_SD following lockdown easing and suggests the potential importance of controlling international travel and applying travel quarantines. Further work is required to assess the effect of variant strains and vaccination measures.

An extended robust mathematical model to project the course of COVID-19 epidemic in Iran

This research develops a regression-based Robust Optimization (RO) approach to efficiently predict the number of patients with confirmed infection caused by the recent Coronavirus Disease (COVID-19). The main idea is to study the dynamics of the COVID-19 outbreak at the first stage and then provide efficient insights to estimate the necessary resources accordingly. The convex RO with Mean Absolute Deviation (MAD) objective function is utilized to project the course of COVID-19 epidemic in Iran. To validate the performance of the suggested model, a real-case study is investigated and compared to several well-known forecasting models including Simple Moving Average, Exponential Moving Average, Weighted Moving Average and Exponential Smoothing with Trend Adjustment models. Furthermore, the effect of parameter uncertainties is examined using a set of sensitivity analyses. The results demonstrate that by increasing the degree (coefficient) of regression up to 8, MAD value decreases to 1378.12, and consequently, the corresponding equation becomes more accurate. On the other hand, from the 8th degree onwards, MAD value follows an upward trend. Furthermore, by increasing the level of regression uncertainty, MAD value follows a downward trend to reach 1309.28 and the estimation accuracy of the model increases accordingly. Finally, our proposed model achieves the least MAD and the greatest correlation coefficient against the other models.

Dynamic causality interplay from COVID-19 pandemic to oil price, stock market, and economic policy uncertainty: evidence from oil-importing and oil-exporting countries

In this study we examine the time-varying causal effect of the novel COVID-19 pandemic in the major oil-importing and oil-exporting countries on the oil price changes, stock market volatilities and the economic uncertainty using the wavelet coherence and network analysis. During the period of the pandemic, we explore such relationship by resorting to the wavelet coherence and gaussian graphical model (GGM) frameworks. Wavelet analysis enables us to measure the dynamics of the causal effect of the novel covid-19 pandemic in the time-frequency space. Regarding the findings displayed herein, we first found that the COVID-19 pandemic has a severe influence on oil prices, stock market indices, and the economic uncertainty. Second the intensity of the causality effect is stronger in the longer horizon than in the short ones, suggesting that the causality exercise continues. Our findings also provide evidence that the COVID-19 pandemic and oil price changes in oil-importing countries mirror those in oil-exporting countries and vice versa. Further, the COVID-19 pandemic has a profound immediate time-frequency effect on the US, Japanese, South Korean, Indian, and Canadian economic uncertainties. A better understanding of oil and stock market prices in the oil-importing and oil-exporting countries is vital for investors and policymakers, specially since the novel unprecedented COVID-19 crisis has been recognized among the most serious ever happened. Thus, the findings suggest that the authorities should strongly take efficient actions to minimize risk.

Investigation of robustness of hybrid artificial neural network with artificial bee colony and firefly algorithm in predicting COVID-19 new cases: case study of Iran

As an ongoing public health menace, the novel coronavirus pandemic has challenged the world. With several mutations and a high transmission rate, the virus is able to infect individuals in an exponential manner. At the same time, Iran is confronted with multiple wave peaks and the health care system is facing a major challenge. In consequence, developing a robust forecasting methodology can assist health authorities for effective planning. In that regard, with the help of Artificial Neural Network-Artificial Bee Colony (ANN-ABC) and Artificial Neural Network- Firefly Algorithm (ANN-FA) as two robust hybrid artificial intelligence-based models, the current study intends to select the optimal model with the maximum accuracy rate. To do so, first a sample of COVID-19 confirmed cases in Iran ranging from 19 February 2020 to 25 July 2021 is compiled. 75% (25%) of total observation is randomly allocated as training (testing) data. Afterwards, an ANN model is trained with Levenberg-Marquardt algorithm. Accordingly, based on R-squared and root-mean-square error criteria, the optimal number of hidden neurons is computed as 17. The proposed ANN model is employed to develop ANN-ABC and ANN-FA models for achieving the maximum accuracy rate. According to ANN-ABC, the R- squared values of the optimal model are 0.9884 and 0.9885 at train and test stages. In respect to ANN-FA, the R-squared ranged from 0.9954 to 0.9940 at the train and test phases, which indicates the outperformance of ANN-FA for predicting COVID-19 new cases in Iran. Finally, the proposed ANN-ABC and ANN-FA are applied for simulating the COVID-19 new cases data in different countries. The results revealed that both models can be used as a robust predictor of COVID-19 data and in a majority of cases ANN-FA outperforms the ANN-ABC.

How is COVID-19 altering the manufacturing landscape? A literature review of imminent challenges and management interventions

Disruption from the COVID-19 pandemic has caused major upheavals for manufacturing, and has severe implications for production networks, and the demand and supply chains underpinning manufacturing operations. This paper is the first of its kind to pull together research on both-the pandemic-related challenges and the management interventions in a manufacturing context. This systematic literature review reveals the frailty of supply chains and production networks in withstanding the pressures of lockdowns and other safety protocols, including product and workforce shortages. These, altogether, have led to closed facilities, reduced capacities, increased costs, and severe economic uncertainty for manufacturing businesses. In managing these challenges and stabilising their operations, manufacturers are urgently intervening by-investing in digital technologies, undertaking resource redistribution and repurposing, regionalizing and localizing, servitizing, and targeting policies that can help them survive in this altered economy. Based on holistic analysis of these challenges and interventions, this review proposes an extensive research agenda for future studies to pursue.

Optimization in the Context of COVID-19 Prediction and Control: A Literature Review

This paper presents an overview of some key results from a body of optimization studies that are specifically related to COVID-19, as reported in the literature during 2020-2021. As shown in this paper, optimization studies in the context of COVID-19 have been used for many aspects of the pandemic. From these studies, it is observed that since COVID-19 is a multifaceted problem, it cannot be studied from a single perspective or framework, and neither can the related optimization models. Four new and different frameworks are proposed that capture the essence of analyzing COVID-19 (or any pandemic for that matter) and the relevant optimization models. These are: (i) microscale vs. macroscale perspective; (ii) early stages vs. later stages perspective; (iii) aspects with direct vs. indirect relationship to COVID-19; and (iv) compartmentalized perspective. To limit the scope of the review, only optimization studies related to the prediction and control of COVID-19 are considered (public health focused), and which utilize formal optimization techniques or machine learning approaches. In this context and to the best of our knowledge, this survey paper is the first in the literature with a focus on the prediction and control related optimization studies. These studies include optimization of screening testing strategies, prediction, prevention and control, resource management, vaccination prioritization, and decision support tools. Upon reviewing the literature, this paper identifies current gaps and major challenges that hinder the closure of these gaps and provides some insights into future research directions.

Performance Evaluation of Soft Computing Approaches for Forecasting COVID-19 Pandemic Cases

An unexpected outbreak of deadly Covid-19 in later part of 2019 not only endangered the economies of the world but also posed threats to the cultural, social and psychological barriers of mankind. As soon as the virus emerged, scientists and researchers from all over the world started investigating the dynamics of this disease. Despite extensive investments in research, no cure has been officially found to date. This uncertain situation rises severe threats to the survival of mankind. An ultimate need of the time is to investigate the course of disease transfer and suggest a future projection of the disease transfer to be enabled to effectively tackle the always evolving situations ahead. In the present study daily new cases of COVID-19 was predicted using different forecasting techniques; Autoregressive Integrated Moving Average (ARIMA), Exponential Smoothing/Error Trend Seasonality (ETS), Artificial Neural Network Models (ANN), Gene Expression Programming (GEP), and Long Short-Term Memory (LSTM) in four countries; Pakistan, USA, India and Brazil. The dataset of new daily confirmed cases of COVID-19 from the date on which first case was registered in the respective country to 30 November 2020 is analyzed through these five forecasting models to forecast the new daily cases up to 31st January 2020. The forecasting efficiency of each model was evaluated using well known statistical parameters R 2, RMSE, and NSE. A comparative analysis of all above-mentioned models was performed. Finally, the study concluded that Long Short-Term Memory (LSTM) neural network-based forecasting model projected the future cases of COVID-19 pandemic best in all the selected four stations. The accuracy of the model ranges from coefficient of determination value of 0.85 in Brazil to 0.96 in Pakistan. NSE value for the model in India is 0. 99, 0.98 in USA and Pakistan and 0.97 in Brazil. This high-accuracy forecast of COVID-19 cases enables the projection of possible peaks in near future in the aforementioned countries and, therefore, prove to be helpful in formulating strategies to get prepared for the potential hard times ahead.