Improving the evidence base for decision making during a pandemic: the example of 2009 influenza A/H1N1

Differences in COVID-19 cyclicity and predictability among U.S. counties and states reflect the effectiveness of protective measures

During the COVID-19 pandemic, many quantitative approaches were employed to predict the course of disease spread. However, forecasting faces the challenge of inherently unpredictable spread dynamics, setting a limit to the accuracy of all models. Here, we analyze COVID-19 data from the USA to explain variation among jurisdictions in disease spread predictability (that is, the extent to which predictions are possible), using a combination of statistical and simulation models. We show that for half the counties and states the spread rate of COVID-19, r(t), was predictable at most 9 weeks and 8 weeks ahead, respectively, corresponding to at most 40% and 35% of an average cycle length of 23 weeks and 26 weeks. High predictability was associated with high cyclicity of r(t) and negatively associated with R0 values from the pandemic's onset. Our statistical evidence suggests the following explanation: jurisdictions with a severe initial outbreak, and where individuals and authorities took strong and sustained protective measures against COVID-19, successfully curbed subsequent waves of disease spread, but at the same time unintentionally decreased its predictability. Decreased predictability of disease spread should be viewed as a by-product of positive and sustained steps that people take to protect themselves and others.

Impact of COVID-19 on ports, multimodal logistics and transport sector in India: Responses and policy imperatives

The COVID-19 pandemic spread rapidly in 2020 and led to full or partial lockdowns worldwide. The restrictions led to most economies contracting in the April-June quarter of 2020. As per the IMF World Economic Outlook of June 2020, the global economy was expected to contract by 4.9% in 2020, whereas the Indian economy was expected to contract by 4.5%, affecting a population of more than 1.3 billion. This led to disruption in the global supply chain network, adversely hitting all modes of transport, including ports and multimodal logistics. Based on the survey results of 98 respondents in the Indian ports, multimodal logistics and transport ("PMT") sector conducted during the lockdown in May-June 2020, an adverse impact on profitability, employment, operations and capital expenditure was identified, which is consistent with observations from previous economic crises. Additionally, we have estimated the financial impact on different categories of organisations in the industry. Based on these impacts, this study identified key factors required to support the sector and build resilience for the future through technology interventions, developing multimodal transport systems, and providing financial support and employment support for small and medium enterprises (SMEs). By providing an analysis of the impact of COVID-19 on the different categories of organisations based on data collected during and shortly after the lockdown, the study makes a novel contribution to understanding the impact of such crises on the PMT industry at a granular level and the aggregate-level impacts with guidance for policy and managers and in particular, in an emerging economy context.

Uncomfortable science: How mathematical models, and consensus, come to be in public policy

We explore messy translations of evidence in policy as a site of 'uncomfortable science'. Drawing on the work of John Law, we follow evidence as a 'fluid object' of its situation, also enacted in relation to a hinterland of practices. Working with the qualitative interview accounts of mathematical modellers and other scientists engaged in the UK COVID-19 response, we trace how models perform as evidence. Our point of departure is a moment of controversy in the public announcement of second national lockdown in the UK, and specifically, the projected daily deaths from COVID-19 presented in support of this policy decision. We reflect on this event to trace the messy translations of "scientific consensus" in the face of uncertainty. Efforts among scientists to realise evidence-based expectation and to manage the troubled translations of models in policy, including via "scientific consensus", can extend the dis-ease of uncomfortable science rather than clean it up or close it down. We argue that the project of evidence-based policy is not so much in need of technical management or repair, but that we need to be thinking altogether differently.

Missing science: A scoping study of COVID-19 epidemiological data in the United States

Systematic approaches to epidemiologic data collection are critical for informing pandemic responses, providing information for the targeting and timing of mitigations, for judging the efficacy and efficiency of alternative response strategies, and for conducting real-world impact assessments. Here, we report on a scoping study to assess the completeness of epidemiological data available for COVID-19 pandemic management in the United States, enumerating authoritative US government estimates of parameters of infectious transmission, infection severity, and disease burden and characterizing the extent and scope of US public health affiliated epidemiological investigations published through November 2021. While we found authoritative estimates for most expected transmission and disease severity parameters, some were lacking, and others had significant uncertainties. Moreover, most transmission parameters were not validated domestically or re-assessed over the course of the pandemic. Publicly available disease surveillance measures did grow appreciably in scope and resolution over time; however, their resolution with regards to specific populations and exposure settings remained limited. We identified 283 published epidemiological reports authored by investigators affiliated with U.S. governmental public health entities. Most reported on descriptive studies. Published analytic studies did not appear to fully respond to knowledge gaps or to provide systematic evidence to support, evaluate or tailor community mitigation strategies. The existence of epidemiological data gaps 18 months after the declaration of the COVID-19 pandemic underscores the need for more timely standardization of data collection practices and for anticipatory research priorities and protocols for emerging infectious disease epidemics.

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States

The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden of COVID-19 in the United States from many contributing teams. We study methods for building an ensemble that combines forecasts from these teams. These experiments have informed the ensemble methods used by the Hub. To be most useful to policy makers, ensemble forecasts must have stable performance in the presence of two key characteristics of the component forecasts: (1) occasional misalignment with the reported data, and (2) instability in the relative performance of component forecasters over time. Our results indicate that in the presence of these challenges, an untrained and robust approach to ensembling using an equally weighted median of all component forecasts is a good choice to support public health decision makers. In settings where some contributing forecasters have a stable record of good performance, trained ensembles that give those forecasters higher weight can also be helpful.

A robust multi-objective model for healthcare resource management and location planning during pandemics

In this study, we consider the problem of healthcare resource management and location planning problem during the early stages of a pandemic/epidemic under demand uncertainty. Our main ambition is to improve the preparedness level and response effectiveness of healthcare authorities in fighting pandemics/epidemics by implementing analytical techniques. Building on lessons from the Chinese experience in the COVID-19 outbreak, we first develop a deterministic multi-objective mixed integer linear program (MILP) which determines the location and size of new pandemic hospitals (strategic level planning), periodic regional health resource re-allocations (tactical level planning) and daily patient-hospital assignments (operational level planning). Taking the forecasted number of cases along a planning horizon as an input, the model minimizes the weighted sum of the number of rejected patients, total travel distance, and installation cost of hospitals subject to real-world constraints and organizational rules. Next, accounting for the uncertainty in the spread speed of the disease, we employ an across scenario robust (ASR) model and reformulate the robust counterpart of the deterministic MILP. The ASR attains relatively more realistic solutions by considering multiple scenarios simultaneously while ensuring a predefined threshold of relative regret for the individual scenarios. Finally, we demonstrate the performance of proposed models on the case of Wuhan, China. Taking the 51 days worth of confirmed COVID-19 case data as an input, we solve both deterministic and robust models and discuss the impact of all three level decisions to the quality and performance of healthcare services during the pandemic. Our case study results show that although it is a challenging task to make strategic level decisions based on uncertain forecasted data, an immediate action can considerably improve the response effectiveness of healthcare authorities. Another important observation is that, the installation times of pandemic hospitals have significant impact on the system performance in fighting with the shortage of beds and facilities.

SARS-CoV-2 Infection Among Pregnant People at Labor and Delivery and Changes in Infection Rates in the General Population: Lessons Learned From Illinois

Objectives:

The Illinois Department of Public Health (IDPH) assessed whether increases in the SARS-CoV-2 test positivity rate among pregnant people at labor and delivery (L&D) could signal increases in SARS-CoV-2 prevalence in the general Illinois population earlier than current state metrics.

Materials and Methods:

Twenty-six birthing hospitals universally testing for SARS-CoV-2 at L&D voluntarily submitted data from June 21, 2020 through January 23, 2021, to IDPH. Hospitals reported the daily number of people who delivered, SARS-CoV-2 tests, and test results as well as symptom status. We compared the test positivity rate at L&D with the test positivity rate of the general population and the number of hospital admissions for COVID-19–like illness by quantifying correlations in trends and identifying a lead time.

Results:

Of 26 633 reported pregnant people who delivered, 96.8% (n = 25 772) were tested for SARS-CoV-2. The overall test positivity rate was 2.4% (n = 615); 77.7% (n = 478) were asymptomatic. In Chicago, the only region with a sufficient sample size for analysis, the test positivity rate at L&D (peak of 5% on December 7, 2020) was lower and more stable than the test positivity rate of the general population (peak of 14% on November 13, 2020) and lagged hospital admissions for COVID-19–like illness (peak of 118 on November 15, 2020) and the test positivity rate of the general population by about 10 days (Pearson correlation = 0.73 and 0.75, respectively).

Practice Implications:

Trends in the test positivity rate at L&D did not provide an earlier signal of increases in Illinois’s SARS-CoV-2 prevalence than current state metrics did. Nonetheless, the role of universal testing protocols in identifying asymptomatic infection is important for clinical decision making and patient education about infection prevention and control.

Deep neural networks for simultaneously capturing public topics and sentiments during a pandemic. Application to a COVID-19 tweet dataset

Background

Public engagement is a key element for mitigating pandemics, and a good understanding of public opinion could help to encourage the successful adoption of public health measures by the population. In past years, deep learning has been increasingly applied to the analysis of text from social networks. However, most of the developed approaches can only capture topics or sentiments alone but not both together.

Objective

Here, we aimed to develop a new approach, based on deep neural networks, for simultaneously capturing public topics and sentiments and applied it to tweets sent just after the announcement of the COVID-19 pandemic by the World Health Organization (WHO).

Methods

A total of 1,386,496 tweets were collected, preprocessed, and split with a ratio of 80:20 into training and validation sets, respectively. We combined lexicons and convolutional neural networks to improve sentiment prediction. The trained model achieved an overall accuracy of 81% and a precision of 82% and was able to capture simultaneously the weighted words associated with a predicted sentiment intensity score. These outputs were then visualized via an interactive and customizable web interface based on a word cloud representation. Using word cloud analysis, we captured the main topics for extreme positive and negative sentiment intensity scores.

Results

In reaction to the announcement of the pandemic by the WHO, 6 negative and 5 positive topics were discussed on Twitter. Twitter users seemed to be worried about the international situation, economic consequences, and medical situation. Conversely, they seemed to be satisfied with the commitment of medical and social workers and with the collaboration between people.

Conclusions

We propose a new method based on deep neural networks for simultaneously extracting public topics and sentiments from tweets. This method could be helpful for monitoring public opinion during crises such as pandemics.

Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers

We find that epidemic resurgence, defined as an upswing in the effective reproduction number (R) of the contagion from subcritical to supercritical values, is fundamentally difficult to detect in real time. Inherent latencies in pathogen transmission, coupled with smaller and intrinsically noisier case incidence across periods of subcritical spread, mean that resurgence cannot be reliably detected without significant delays of the order of the generation time of the disease, even when case reporting is perfect. In contrast, epidemic suppression (where R falls from supercritical to subcritical values) may be ascertained 5-10 times faster due to the naturally larger incidence at which control actions are generally applied. We prove that these innate limits on detecting resurgence only worsen when spatial or demographic heterogeneities are incorporated. Consequently, we argue that resurgence is more effectively handled proactively, potentially at the expense of false alarms. Timely responses to recrudescent infections or emerging variants of concern are more likely to be possible when policy is informed by a greater quality and diversity of surveillance data than by further optimisation of the statistical models used to process routine outbreak data.

Near real-time surveillance of the SARS-CoV-2 epidemic with incomplete data

When responding to infectious disease outbreaks, rapid and accurate estimation of the epidemic trajectory is critical. However, two common data collection problems affect the reliability of the epidemiological data in real time: missing information on the time of first symptoms, and retrospective revision of historical information, including right censoring. Here, we propose an approach to construct epidemic curves in near real time that addresses these two challenges by 1) imputation of dates of symptom onset for reported cases using a dynamically-estimated "backward" reporting delay conditional distribution, and 2) adjustment for right censoring using the NobBS software package to nowcast cases by date of symptom onset. This process allows us to obtain an approximation of the time-varying reproduction number (Rt) in real time. We apply this approach to characterize the early SARS-CoV-2 outbreak in two Spanish regions between March and April 2020. We evaluate how these real-time estimates compare with more complete epidemiological data that became available later. We explore the impact of the different assumptions on the estimates, and compare our estimates with those obtained from commonly used surveillance approaches. Our framework can help improve accuracy, quantify uncertainty, and evaluate frequently unstated assumptions when recovering the epidemic curves from limited data obtained from public health systems in other locations.

Short-Range Forecasting of COVID-19 During Early Onset at County, Health District, and State Geographic Levels Using Seven Methods: Comparative Forecasting Study

BACKGROUND

Forecasting methods rely on trends and averages of prior observations to forecast COVID-19 case counts. COVID-19 forecasts have received much media attention, and numerous platforms have been created to inform the public. However, forecasting effectiveness varies by geographic scope and is affected by changing assumptions in behaviors and preventative measures in response to the pandemic. Due to time requirements for developing a COVID-19 vaccine, evidence is needed to inform short-term forecasting method selection at county, health district, and state levels.

OBJECTIVE

COVID-19 forecasts keep the public informed and contribute to public policy. As such, proper understanding of forecasting purposes and outcomes is needed to advance knowledge of health statistics for policy makers and the public. Using publicly available real-time data provided online, we aimed to evaluate the performance of seven forecasting methods utilized to forecast cumulative COVID-19 case counts. Forecasts were evaluated based on how well they forecast 1, 3, and 7 days forward when utilizing 1-, 3-, 7-, or all prior-day cumulative case counts during early virus onset. This study provides an objective evaluation of the forecasting methods to identify forecasting model assumptions that contribute to lower error in forecasting COVID-19 cumulative case growth. This information benefits professionals, decision makers, and the public relying on the data provided by short-term case count estimates at varied geographic levels.

METHODS

We created 1-, 3-, and 7-day forecasts at the county, health district, and state levels using (1) a naïve approach, (2) Holt-Winters (HW) exponential smoothing, (3) a growth rate approach, (4) a moving average (MA) approach, (5) an autoregressive (AR) approach, (6) an autoregressive moving average (ARMA) approach, and (7) an autoregressive integrated moving average (ARIMA) approach. Forecasts relied on Virginia's 3464 historical county-level cumulative case counts from March 7 to April 22, 2020, as reported by The New York Times. Statistically significant results were identified using 95% CIs of median absolute error (MdAE) and median absolute percentage error (MdAPE) metrics of the resulting 216,698 forecasts.

RESULTS

The next-day MA forecast with 3-day look-back length obtained the lowest MdAE (median 0.67, 95% CI 0.49-0.84, P<.001) and statistically significantly differed from 39 out of 59 alternatives (66%) to 53 out of 59 alternatives (90%) at each geographic level at a significance level of .01. For short-range forecasting, methods assuming stationary means of prior days' counts outperformed methods with assumptions of weak stationarity or nonstationarity means. MdAPE results revealed statistically significant differences across geographic levels.

CONCLUSIONS

For short-range COVID-19 cumulative case count forecasting at the county, health district, and state levels during early onset, the following were found: (1) the MA method was effective for forecasting 1-, 3-, and 7-day cumulative case counts; (2) exponential growth was not the best representation of case growth during early virus onset when the public was aware of the virus; and (3) geographic resolution was a factor in the selection of forecasting methods.

Emergency preparedness for public health threats, surveillance, modelling & forecasting

In the interconnected world, safeguarding global health security is vital for maintaining public health and economic upliftment of any nation. Emergency preparedness is considered as the key to control the emerging public health challenges at both national as well as international levels. Further, the predictive information systems based on routine surveillance, disease modelling and forecasting play a pivotal role in both policy building and community participation to detect, prevent and respond to potential health threats. Therefore, reliable and timely forecasts of these untoward events could mobilize swift and effective public health responses and mitigation efforts. The present review focuses on the various aspects of emergency preparedness with special emphasis on public health surveillance, epidemiological modelling and capacity building approaches. Global coordination and capacity building, funding and commitment at the national and international levels, under the One Health framework, are crucial in combating global public health threats in a holistic manner.

Use Internet search data to accurately track state level influenza epidemics

For epidemics control and prevention, timely insights of potential hot spots are invaluable. Alternative to traditional epidemic surveillance, which often lags behind real time by weeks, big data from the Internet provide important information of the current epidemic trends. Here we present a methodology, ARGOX (Augmented Regression with GOogle data CROSS space), for accurate real-time tracking of state-level influenza epidemics in the United States. ARGOX combines Internet search data at the national, regional and state levels with traditional influenza surveillance data from the Centers for Disease Control and Prevention, and accounts for both the spatial correlation structure of state-level influenza activities and the evolution of people’s Internet search pattern. ARGOX achieves on average 28% error reduction over the best alternative for real-time state-level influenza estimation for 2014 to 2020. ARGOX is robust and reliable and can be potentially applied to track county- and city-level influenza activity and other infectious diseases.

Near real-time surveillance of the SARS-CoV-2 epidemic with incomplete data

Designing public health responses to outbreaks requires close monitoring of population-level health indicators in real-time. Thus, an accurate estimation of the epidemic curve is critical. We propose an approach to reconstruct epidemic curves in near real time. We apply this approach to characterize the early SARS-CoV-2 outbreak in two Spanish regions between March and April 2020. We address two data collection problems that affected the reliability of the available real-time epidemiological data, namely, the frequent missing information documenting when a patient first experienced symptoms, and the frequent retrospective revision of historical information (including right censoring). This is done by using a novel back-calculating procedure based on imputing patients' dates of symptom onset from reported cases, according to a dynamically-estimated "backward" reporting delay conditional distribution, and adjusting for right censoring using an existing package, NobBS , to estimate in real time (nowcast) cases by date of symptom onset. This process allows us to obtain an approximation of the time-varying reproduction number ( R t ) in real-time. At each step, we evaluate how different assumptions affect the recovered epidemiological events and compare the proposed approach to the alternative procedure of merely using curves of case counts, by report day, to characterize the time-evolution of the outbreak. Finally, we assess how these real-time estimates compare with subsequently documented epidemiological information that is considered more reliable and complete that became available later in time. Our approach may help improve accuracy, quantify uncertainty, and evaluate frequently unstated assumptions when recovering the epidemic curves from limited data obtained from public health surveillance systems in other locations.

Does Twitter Affect Stock Market Decisions? Financial Sentiment Analysis During Pandemics: A Comparative Study of the H1N1 and the COVID-19 Periods

Investors are constantly aware of the behaviour of stock markets. This affects their emotions and motivates them to buy or sell shares. Financial sentiment analysis allows us to understand the effect of social media reactions and emotions on the stock market and vice versa. In this research, we analyse Twitter data and important worldwide financial indices to answer the following question: How does the polarity generated by Twitter posts influence the behaviour of financial indices during pandemics? This study is based on the financial sentiment analysis of influential Twitter accounts and its relationship with the behaviour of important financial indices. To carry out this analysis, we used fundamental and technical financial analysis combined with a lexicon-based approach on financial Twitter accounts. We calculated the correlations between the polarities of financial market indicators and posts on Twitter by applying a date shift on tweets. In addition, correlations were identified days before and after the existing posts on financial Twitter accounts. Our findings show that the markets reacted 0 to 10 days after the information was shared and disseminated on Twitter during the COVID-19 pandemic and 0 to 15 days after the information was shared and disseminated on Twitter during the H1N1 pandemic. We identified an inverse relationship: Twitter accounts presented reactions to financial market behaviour within a period of 0 to 11 days during the H1N1 pandemic and 0 to 6 days during the COVID-19 pandemic. We also found that our method is better at detecting highly shifted correlations by using SenticNet compared with other lexicons. With SenticNet, it is possible to detect correlations even on the same day as the Twitter posts. The most influential Twitter accounts during the period of the pandemic were The New York Times, Bloomberg, CNN News and Investing.com, presenting a very high correlation between sentiments on Twitter and stock market behaviour. The combination of a lexicon-based approach is enhanced by a shifted correlation analysis, as latent or hidden correlations can be found in data.

Transforming public health using value lens and extended partner networks

INTRODUCTION

Organizational transformations have focused on creating and fulfilling value for customers, leveraging advanced technologies. Transforming public health (PH) faces an interesting challenge. The value created (preventive practices) to fulfill policy makers' desire to reduce healthcare costs is realized by several external partners with varying goals and is practiced by the public (value in use), which often places low priority on prevention.

METHODS

This paper uses value lens to argue that PH transformation strategy must align the goals of all stakeholders involved. This may include allowing partners and the public to contextualize the preventive practices to see the value in near term and as relevant. It also means extending the number of partners PH uses and helping them connect with the public to seek shared alignment in shared goals of value fulfillment and value-in-use.

RESULTS

Using lessons from Covid-19 and PH experience with partners in four different sectors: business, healthcare, public and community, the paper illustrates how PH transformation strategy can be implemented going forward.

CONCLUSIONS

We conclude the paper with five distinct directions for future research to create and sustain value using the framework of learning health systems.

A systematic assessment on COVID-19 preparedness and transition strategy in Bangladesh

RATIONALE, AIMS, AND OBJECTIVES

The COVID-19 pandemic of 2020 has overpowered the most advanced health systems worldwide with thousands of daily deaths. The current study conducted a situation analysis on the pandemic preparedness of Bangladesh and provided recommendations on the transition to the new reality and gradual restoration of normalcy.

METHOD

A complex adaptive system (CAS) framework was theorized based on four structural dimensions obtained from the crisis and complexity theory to help evaluate the health system of Bangladesh. Data sourced from published reports from the government, non-governmental organizations, and mainstream media up to June 15, 2020 were used to conduct a qualitative analysis and visualize the spatial distribution of countrywide COVID-19 cases.

RESULTS

The findings suggested that Bangladesh severely lacked the preparedness to tackle the spread of COVID-19 with both short- and long-term implications for health, the economy, and good governance. Absence of planning and coordination, disproportionate resource allocations, challenged infrastructure, adherence to bureaucratic delay, lack of synchronized risk communication, failing leadership of concerned authorities, and incoherent decision-making have led to a precarious situation that will have dire ramifications causing many uncertainties in the coming days.

CONCLUSIONS

Implementation of response protocols addressing the needs of the community and the stakeholders from the central level is urgently needed. The development of mechanisms for dynamic decision-making based on regular feedback and long-term planning for a smooth transition between the new reality and normalcy should also be urgently addressed in Bangladesh.

Key questions for modelling COVID-19 exit strategies

Combinations of intense non-pharmaceutical interventions (lockdowns) were introduced worldwide to reduce SARS-CoV-2 transmission. Many governments have begun to implement exit strategies that relax restrictions while attempting to control the risk of a surge in cases. Mathematical modelling has played a central role in guiding interventions, but the challenge of designing optimal exit strategies in the face of ongoing transmission is unprecedented. Here, we report discussions from the Isaac Newton Institute 'Models for an exit strategy' workshop (11-15 May 2020). A diverse community of modellers who are providing evidence to governments worldwide were asked to identify the main questions that, if answered, would allow for more accurate predictions of the effects of different exit strategies. Based on these questions, we propose a roadmap to facilitate the development of reliable models to guide exit strategies. This roadmap requires a global collaborative effort from the scientific community and policymakers, and has three parts: (i) improve estimation of key epidemiological parameters; (ii) understand sources of heterogeneity in populations; and (iii) focus on requirements for data collection, particularly in low-to-middle-income countries. This will provide important information for planning exit strategies that balance socio-economic benefits with public health.

Applying Machine Learning Models with An Ensemble Approach for Accurate Real-Time Influenza Forecasting in Taiwan: Development and Validation Study

Background

Changeful seasonal influenza activity in subtropical areas such as Taiwan causes problems in epidemic preparedness. The Taiwan Centers for Disease Control has maintained real-time national influenza surveillance systems since 2004. Except for timely monitoring, epidemic forecasting using the national influenza surveillance data can provide pivotal information for public health response.

Objective

We aimed to develop predictive models using machine learning to provide real-time influenza-like illness forecasts.

Methods

Using surveillance data of influenza-like illness visits from emergency departments (from the Real-Time Outbreak and Disease Surveillance System), outpatient departments (from the National Health Insurance database), and the records of patients with severe influenza with complications (from the National Notifiable Disease Surveillance System), we developed 4 machine learning models (autoregressive integrated moving average, random forest, support vector regression, and extreme gradient boosting) to produce weekly influenza-like illness predictions for a given week and 3 subsequent weeks. We established a framework of the machine learning models and used an ensemble approach called stacking to integrate these predictions. We trained the models using historical data from 2008-2014. We evaluated their predictive ability during 2015-2017 for each of the 4-week time periods using Pearson correlation, mean absolute percentage error (MAPE), and hit rate of trend prediction. A dashboard website was built to visualize the forecasts, and the results of real-world implementation of this forecasting framework in 2018 were evaluated using the same metrics.

Results

All models could accurately predict the timing and magnitudes of the seasonal peaks in the then-current week (nowcast) (ρ=0.802-0.965; MAPE: 5.2%-9.2%; hit rate: 0.577-0.756), 1-week (ρ=0.803-0.918; MAPE: 8.3%-11.8%; hit rate: 0.643-0.747), 2-week (ρ=0.783-0.867; MAPE: 10.1%-15.3%; hit rate: 0.669-0.734), and 3-week forecasts (ρ=0.676-0.801; MAPE: 12.0%-18.9%; hit rate: 0.643-0.786), especially the ensemble model. In real-world implementation in 2018, the forecasting performance was still accurate in nowcasts (ρ=0.875-0.969; MAPE: 5.3%-8.0%; hit rate: 0.582-0.782) and remained satisfactory in 3-week forecasts (ρ=0.721-0.908; MAPE: 7.6%-13.5%; hit rate: 0.596-0.904).

Conclusions

This machine learning and ensemble approach can make accurate, real-time influenza-like illness forecasts for a 4-week period, and thus, facilitate decision making.

Real-time estimation of disease activity in emerging outbreaks using internet search information

Understanding the behavior of emerging disease outbreaks in, or ahead of, real-time could help healthcare officials better design interventions to mitigate impacts on affected populations. Most healthcare-based disease surveillance systems, however, have significant inherent reporting delays due to data collection, aggregation, and distribution processes. Recent work has shown that machine learning methods leveraging a combination of traditionally collected epidemiological information and novel Internet-based data sources, such as disease-related Internet search activity, can produce meaningful "nowcasts" of disease incidence ahead of healthcare-based estimates, with most successful case studies focusing on endemic and seasonal diseases such as influenza and dengue. Here, we apply similar computational methods to emerging outbreaks in geographic regions where no historical presence of the disease of interest has been observed. By combining limited available historical epidemiological data available with disease-related Internet search activity, we retrospectively estimate disease activity in five recent outbreaks weeks ahead of traditional surveillance methods. We find that the proposed computational methods frequently provide useful real-time incidence estimates that can help fill temporal data gaps resulting from surveillance reporting delays. However, the proposed methods are limited by issues of sample bias and skew in search query volumes, perhaps as a result of media coverage.

Pandemic preparedness of community pharmacies for COVID-19

Background

Community pharmacies provide an important healthcare service, which is broadly established, and constitutes the preferred and initial contact for members of the community. The significant value of community pharmacies was further highlighted during the COVID-19 pandemic crisis.

Objective

The assessment of community pharmacies preparedness for the COVID-19 pandemic.

Methods

A cross-sectional interview survey of 1018 community pharmacies working in four regions of Egypt (South, East, Centre, and North). Data collection was conducted from 8-19 April 2020.

Results

Availability of personal protective equipment (PPE) and medication was better than alcohol (70% conc.). Home delivery services were available in 49.1% of pharmacies. Infection control measures covering interactions between staff were in place in up to 99.5% of pharmacies. Conversely, there was less frequent availability of contactless payment (29.1%), hand sanitizers (62.1%) or masks (86.5%) for customer use, or a separate area for patients with suspected COVID-19 (64%). Verbal customer education (90.4%) was used preferably to written (81.3%). Despite high clinical knowledge and awareness (97.6%-99.2%), only 8.8% of pharmacists had reported suspected COVID-19 cases, however this varied significantly with pharmacist demographics (geographic region P < 0.001; pandemic training p < 0.001; position p = 0.019; age p = 0.046).

Conclusions

Government and policymakers strive to mitigate the shortage of PPE and medication. More attention should be given to infection control measures around interactions between staff and customers to ensure community pharmacists are fit and able to provide continuity in their important role. Educating customers using regularly-updated posters, banners or signs will contribute to decreasing contact with patients, and reducing the number and duration of visits to the pharmacy. Pandemic preparedness of community pharmacists must also extend to reporting procedures. By avoiding under-reporting or over-reporting, community pharmacists will contribute to accurate monitoring of the national spread of infection.

Nowcasting by Bayesian Smoothing: A flexible, generalizable model for real-time epidemic tracking

Achieving accurate, real-time estimates of disease activity is challenged by delays in case reporting. "Nowcast" approaches attempt to estimate the complete case counts for a given reporting date, using a time series of case reports that is known to be incomplete due to reporting delays. Modeling the reporting delay distribution is a common feature of nowcast approaches. However, many nowcast approaches ignore a crucial feature of infectious disease transmission-that future cases are intrinsically linked to past reported cases-and are optimized to one or two applications, which may limit generalizability. Here, we present a Bayesian approach, NobBS (Nowcasting by Bayesian Smoothing) capable of producing smooth and accurate nowcasts in multiple disease settings. We test NobBS on dengue in Puerto Rico and influenza-like illness (ILI) in the United States to examine performance and robustness across settings exhibiting a range of common reporting delay characteristics (from stable to time-varying), and compare this approach with a published nowcasting software package while investigating the features of each approach that contribute to good or poor performance. We show that introducing a temporal relationship between cases considerably improves performance when the reporting delay distribution is time-varying, and we identify trade-offs in the role of moving windows to accurately capture changes in the delay. We present software implementing this new approach (R package "NobBS") for widespread application and provide practical guidance on implementation.

Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study

BACKGROUND

Since Dec 31, 2019, the Chinese city of Wuhan has reported an outbreak of atypical pneumonia caused by the 2019 novel coronavirus (2019-nCoV). Cases have been exported to other Chinese cities, as well as internationally, threatening to trigger a global outbreak. Here, we provide an estimate of the size of the epidemic in Wuhan on the basis of the number of cases exported from Wuhan to cities outside mainland China and forecast the extent of the domestic and global public health risks of epidemics, accounting for social and non-pharmaceutical prevention interventions.

METHODS

We used data from Dec 31, 2019, to Jan 28, 2020, on the number of cases exported from Wuhan internationally (known days of symptom onset from Dec 25, 2019, to Jan 19, 2020) to infer the number of infections in Wuhan from Dec 1, 2019, to Jan 25, 2020. Cases exported domestically were then estimated. We forecasted the national and global spread of 2019-nCoV, accounting for the effect of the metropolitan-wide quarantine of Wuhan and surrounding cities, which began Jan 23-24, 2020. We used data on monthly flight bookings from the Official Aviation Guide and data on human mobility across more than 300 prefecture-level cities in mainland China from the Tencent database. Data on confirmed cases were obtained from the reports published by the Chinese Center for Disease Control and Prevention. Serial interval estimates were based on previous studies of severe acute respiratory syndrome coronavirus (SARS-CoV). A susceptible-exposed-infectious-recovered metapopulation model was used to simulate the epidemics across all major cities in China. The basic reproductive number was estimated using Markov Chain Monte Carlo methods and presented using the resulting posterior mean and 95% credibile interval (CrI).

FINDINGS

In our baseline scenario, we estimated that the basic reproductive number for 2019-nCoV was 2·68 (95% CrI 2·47-2·86) and that 75 815 individuals (95% CrI 37 304-130 330) have been infected in Wuhan as of Jan 25, 2020. The epidemic doubling time was 6·4 days (95% CrI 5·8-7·1). We estimated that in the baseline scenario, Chongqing, Beijing, Shanghai, Guangzhou, and Shenzhen had imported 461 (95% CrI 227-805), 113 (57-193), 98 (49-168), 111 (56-191), and 80 (40-139) infections from Wuhan, respectively. If the transmissibility of 2019-nCoV were similar everywhere domestically and over time, we inferred that epidemics are already growing exponentially in multiple major cities of China with a lag time behind the Wuhan outbreak of about 1-2 weeks.

INTERPRETATION

Given that 2019-nCoV is no longer contained within Wuhan, other major Chinese cities are probably sustaining localised outbreaks. Large cities overseas with close transport links to China could also become outbreak epicentres, unless substantial public health interventions at both the population and personal levels are implemented immediately. Independent self-sustaining outbreaks in major cities globally could become inevitable because of substantial exportation of presymptomatic cases and in the absence of large-scale public health interventions. Preparedness plans and mitigation interventions should be readied for quick deployment globally.

FUNDING

Health and Medical Research Fund (Hong Kong, China).

Infectious disease pandemic planning and response: Incorporating decision analysis

Freya Shearer and co-authors discuss the use of decision analysis in planning for infectious disease pandemics.

Improving preparedness for the next flu pandemic

Pandemic influenza remains the single greatest threat to global heath security. Efforts to increase our preparedness, by improving predictions of viral emergence, spread and disease severity, by targeting reduced transmission and improved vaccination and by mitigating health impacts in low- and middle-income countries, should receive renewed urgency.

Accurate regional influenza epidemics tracking using Internet search data

Accurate, high-resolution tracking of influenza epidemics at the regional level helps public health agencies make informed and proactive decisions, especially in the face of outbreaks. Internet users' online searches offer great potential for the regional tracking of influenza. However, due to the complex data structure and reduced quality of Internet data at the regional level, few established methods provide satisfactory performance. In this article, we propose a novel method named ARGO2 (2-step Augmented Regression with GOogle data) that efficiently combines publicly available Google search data at different resolutions (national and regional) with traditional influenza surveillance data from the Centers for Disease Control and Prevention (CDC) for accurate, real-time regional tracking of influenza. ARGO2 gives very competitive performance across all US regions compared with available Internet-data-based regional influenza tracking methods, and it has achieved 30% error reduction over the best alternative method that we numerically tested for the period of March 2009 to March 2018. ARGO2 is reliable and robust, with the flexibility to incorporate additional information from other sources and resolutions, making it a powerful tool for regional influenza tracking, and potentially for tracking other social, economic, or public health events at the regional or local level.

Predicting seasonal influenza epidemics using cross-hemisphere influenza surveillance data and local internet query data

Can early warning systems be developed to predict influenza epidemics? Using Australian influenza surveillance and local internet search query data, this study investigated whether seasonal influenza epidemics in China, the US and the UK can be predicted using empirical time series analysis. Weekly national number of respiratory cases positive for influenza virus infection that were reported to the FluNet surveillance system in Australia, China, the US and the UK were obtained from World Health Organization FluNet surveillance between week 1, 2010, and week 9, 2018. We collected combined search query data for the US and the UK from Google Trends, and for China from Baidu Index. A multivariate seasonal autoregressive integrated moving average model was developed to track influenza epidemics using Australian influenza and local search data. Parameter estimates for this model were generally consistent with the observed values. The inclusion of search metrics improved the performance of the model with high correlation coefficients (China = 0.96, the US = 0.97, the UK = 0.96, p < 0.01) and low Maximum Absolute Percent Error (MAPE) values (China = 16.76, the US = 96.97, the UK = 125.42). This study demonstrates the feasibility of combining (Australia) influenza and local search query data to predict influenza epidemics a different (northern hemisphere) scales.

A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States

Accurate prediction of the size and timing of infectious disease outbreaks could help public health officials in planning an appropriate response. This paper compares approaches developed by five different research groups to forecast seasonal influenza outbreaks in real time in the United States. Many of the models show more accurate forecasts than a historical baseline. A major impediment to predictive ability was the real-time accuracy of available data. The field of infectious disease forecasting is in its infancy and we expect that innovation will spur improvements in forecasting in the coming years.

Influenza infects an estimated 9–35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually. Seasonal outbreaks of influenza are common in temperate regions of the world, with highest incidence typically occurring in colder and drier months of the year. Real-time forecasts of influenza transmission can inform public health response to outbreaks. We present the results of a multiinstitution collaborative effort to standardize the collection and evaluation of forecasting models for influenza in the United States for the 2010/2011 through 2016/2017 influenza seasons. For these seven seasons, we assembled weekly real-time forecasts of seven targets of public health interest from 22 different models. We compared forecast accuracy of each model relative to a historical baseline seasonal average. Across all regions of the United States, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness 1 wk, 2 wk, and 3 wk ahead of available data and when forecasting the timing and magnitude of the seasonal peak. In some regions, delays in data reporting were strongly and negatively associated with forecast accuracy. More timely reporting and an improved overall accessibility to novel and traditional data sources are needed to improve forecasting accuracy and its integration with real-time public health decision making.

Comparison of crowd-sourced, electronic health records based, and traditional health-care based influenza-tracking systems at multiple spatial resolutions in the United States of America

BACKGROUND

Influenza causes an estimated 3000 to 50,000 deaths per year in the United States of America (US). Timely and representative data can help local, state, and national public health officials monitor and respond to outbreaks of seasonal influenza. Data from cloud-based electronic health records (EHR) and crowd-sourced influenza surveillance systems have the potential to provide complementary, near real-time estimates of influenza activity. The objectives of this paper are to compare two novel influenza-tracking systems with three traditional healthcare-based influenza surveillance systems at four spatial resolutions: national, regional, state, and city, and to determine the minimum number of participants in these systems required to produce influenza activity estimates that resemble the historical trends recorded by traditional surveillance systems.

METHODS

We compared influenza activity estimates from five influenza surveillance systems: 1) patient visits for influenza-like illness (ILI) from the US Outpatient ILI Surveillance Network (ILINet), 2) virologic data from World Health Organization (WHO) Collaborating and National Respiratory and Enteric Virus Surveillance System (NREVSS) Laboratories, 3) Emergency Department (ED) syndromic surveillance from Boston, Massachusetts, 4) patient visits for ILI from EHR, and 5) reports of ILI from the crowd-sourced system, Flu Near You (FNY), by calculating correlations between these systems across four influenza seasons, 2012-16, at four different spatial resolutions in the US. For the crowd-sourced system, we also used a bootstrapping statistical approach to estimate the minimum number of reports necessary to produce a meaningful signal at a given spatial resolution.

RESULTS

In general, as the spatial resolution increased, correlation values between all influenza surveillance systems decreased. Influenza-like Illness rates in geographic areas with more than 250 crowd-sourced participants or with more than 20,000 visit counts for EHR tracked government-lead estimates of influenza activity.

CONCLUSIONS

With a sufficient number of reports, data from novel influenza surveillance systems can complement traditional healthcare-based systems at multiple spatial resolutions.

Preprints: An underutilized mechanism to accelerate outbreak science

In an Essay, Michael Johansson and colleagues advocate the posting of research studies addressing infectious disease outbreaks as preprints.

Resilience management during large-scale epidemic outbreaks

Assessing and managing the impact of large-scale epidemics considering only the individual risk and severity of the disease is exceedingly difficult and could be extremely expensive. Economic consequences, infrastructure and service disruption, as well as the recovery speed, are just a few of the many dimensions along which to quantify the effect of an epidemic on society's fabric. Here, we extend the concept of resilience to characterize epidemics in structured populations, by defining the system-wide critical functionality that combines an individual's risk of getting the disease (disease attack rate) and the disruption to the system's functionality (human mobility deterioration). By studying both conceptual and data-driven models, we show that the integrated consideration of individual risks and societal disruptions under resilience assessment framework provides an insightful picture of how an epidemic might impact society. In particular, containment interventions intended for a straightforward reduction of the risk may have net negative impact on the system by slowing down the recovery of basic societal functions. The presented study operationalizes the resilience framework, providing a more nuanced and comprehensive approach for optimizing containment schemes and mitigation policies in the case of epidemic outbreaks.

Accurate Influenza Monitoring and Forecasting Using Novel Internet Data Streams: A Case Study in the Boston Metropolis

Background

Influenza outbreaks pose major challenges to public health around the world, leading to thousands of deaths a year in the United States alone. Accurate systems that track influenza activity at the city level are necessary to provide actionable information that can be used for clinical, hospital, and community outbreak preparation.

Objective

Although Internet-based real-time data sources such as Google searches and tweets have been successfully used to produce influenza activity estimates ahead of traditional health care–based systems at national and state levels, influenza tracking and forecasting at finer spatial resolutions, such as the city level, remain an open question. Our study aimed to present a precise, near real-time methodology capable of producing influenza estimates ahead of those collected and published by the Boston Public Health Commission (BPHC) for the Boston metropolitan area. This approach has great potential to be extended to other cities with access to similar data sources.

Methods

We first tested the ability of Google searches, Twitter posts, electronic health records, and a crowd-sourced influenza reporting system to detect influenza activity in the Boston metropolis separately. We then adapted a multivariate dynamic regression method named ARGO (autoregression with general online information), designed for tracking influenza at the national level, and showed that it effectively uses the above data sources to monitor and forecast influenza at the city level 1 week ahead of the current date. Finally, we presented an ensemble-based approach capable of combining information from models based on multiple data sources to more robustly nowcast as well as forecast influenza activity in the Boston metropolitan area. The performances of our models were evaluated in an out-of-sample fashion over 4 influenza seasons within 2012-2016, as well as a holdout validation period from 2016 to 2017.

Results

Our ensemble-based methods incorporating information from diverse models based on multiple data sources, including ARGO, produced the most robust and accurate results. The observed Pearson correlations between our out-of-sample flu activity estimates and those historically reported by the BPHC were 0.98 in nowcasting influenza and 0.94 in forecasting influenza 1 week ahead of the current date.

Conclusions

We show that information from Internet-based data sources, when combined using an informed, robust methodology, can be effectively used as early indicators of influenza activity at fine geographic resolutions.

Quantifying Zika: Advancing the Epidemiology of Zika With Quantitative Models

When Zika virus (ZIKV) emerged in the Americas, little was known about its biology, pathogenesis, and transmission potential, and the scope of the epidemic was largely hidden, owing to generally mild infections and no established surveillance systems. Surges in congenital defects and Guillain-Barré syndrome alerted the world to the danger of ZIKV. In the context of limited data, quantitative models were critical in reducing uncertainties and guiding the global ZIKV response. Here, we review some of the models used to assess the risk of ZIKV-associated severe outcomes, the potential speed and size of ZIKV epidemics, and the geographic distribution of ZIKV risk. These models provide important insights and highlight significant unresolved questions related to ZIKV and other emerging pathogens.

The RAPIDD ebola forecasting challenge: Synthesis and lessons learnt

Infectious disease forecasting is gaining traction in the public health community; however, limited systematic comparisons of model performance exist. Here we present the results of a synthetic forecasting challenge inspired by the West African Ebola crisis in 2014-2015 and involving 16 international academic teams and US government agencies, and compare the predictive performance of 8 independent modeling approaches. Challenge participants were invited to predict 140 epidemiological targets across 5 different time points of 4 synthetic Ebola outbreaks, each involving different levels of interventions and "fog of war" in outbreak data made available for predictions. Prediction targets included 1-4 week-ahead case incidences, outbreak size, peak timing, and several natural history parameters. With respect to weekly case incidence targets, ensemble predictions based on a Bayesian average of the 8 participating models outperformed any individual model and did substantially better than a null auto-regressive model. There was no relationship between model complexity and prediction accuracy; however, the top performing models for short-term weekly incidence were reactive models with few parameters, fitted to a short and recent part of the outbreak. Individual model outputs and ensemble predictions improved with data accuracy and availability; by the second time point, just before the peak of the epidemic, estimates of final size were within 20% of the target. The 4th challenge scenario - mirroring an uncontrolled Ebola outbreak with substantial data reporting noise - was poorly predicted by all modeling teams. Overall, this synthetic forecasting challenge provided a deep understanding of model performance under controlled data and epidemiological conditions. We recommend such "peace time" forecasting challenges as key elements to improve coordination and inspire collaboration between modeling groups ahead of the next pandemic threat, and to assess model forecasting accuracy for a variety of known and hypothetical pathogens.

Key data for outbreak evaluation: building on the Ebola experience

Following the detection of an infectious disease outbreak, rapid epidemiological assessment is critical for guiding an effective public health response. To understand the transmission dynamics and potential impact of an outbreak, several types of data are necessary. Here we build on experience gained in the West African Ebola epidemic and prior emerging infectious disease outbreaks to set out a checklist of data needed to: (1) quantify severity and transmissibility; (2) characterize heterogeneities in transmission and their determinants; and (3) assess the effectiveness of different interventions. We differentiate data needs into individual-level data (e.g. a detailed list of reported cases), exposure data (e.g. identifying where/how cases may have been infected) and population-level data (e.g. size/demographics of the population(s) affected and when/where interventions were implemented). A remarkable amount of individual-level and exposure data was collected during the West African Ebola epidemic, which allowed the assessment of (1) and (2). However, gaps in population-level data (particularly around which interventions were applied when and where) posed challenges to the assessment of (3). Here we highlight recurrent data issues, give practical suggestions for addressing these issues and discuss priorities for improvements in data collection in future outbreaks.This article is part of the themed issue 'The 2013-2016 West African Ebola epidemic: data, decision-making and disease control'.

Using electronic health records and Internet search information for accurate influenza forecasting

BACKGROUND

Accurate influenza activity forecasting helps public health officials prepare and allocate resources for unusual influenza activity. Traditional flu surveillance systems, such as the Centers for Disease Control and Prevention's (CDC) influenza-like illnesses reports, lag behind real-time by one to 2 weeks, whereas information contained in cloud-based electronic health records (EHR) and in Internet users' search activity is typically available in near real-time. We present a method that combines the information from these two data sources with historical flu activity to produce national flu forecasts for the United States up to 4 weeks ahead of the publication of CDC's flu reports.

METHODS

We extend a method originally designed to track flu using Google searches, named ARGO, to combine information from EHR and Internet searches with historical flu activities. Our regularized multivariate regression model dynamically selects the most appropriate variables for flu prediction every week. The model is assessed for the flu seasons within the time period 2013-2016 using multiple metrics including root mean squared error (RMSE).

RESULTS

Our method reduces the RMSE of the publicly available alternative (Healthmap flutrends) method by 33, 20, 17 and 21%, for the four time horizons: real-time, one, two, and 3 weeks ahead, respectively. Such accuracy improvements are statistically significant at the 5% level. Our real-time estimates correctly identified the peak timing and magnitude of the studied flu seasons.

CONCLUSIONS

Our method significantly reduces the prediction error when compared to historical publicly available Internet-based prediction systems, demonstrating that: (1) the method to combine data sources is as important as data quality; (2) effectively extracting information from a cloud-based EHR and Internet search activity leads to accurate forecast of flu.

FluMob: Enabling Surveillance of Acute Respiratory Infections in Health-care Workers via Mobile Phones

Singapore is a hotspot for emerging infectious diseases and faces a constant risk of pandemic outbreaks as a major travel and health hub for Southeast Asia. With an increasing penetration of smart phone usage in this region, Singapore's pandemic preparedness framework can be strengthened by applying a mobile-based approach to health surveillance and control, and improving upon existing ideas by addressing gaps, such as a lack of health communication. FluMob is a digitally integrated syndromic surveillance system designed to assist health authorities in obtaining real-time epidemiological and surveillance data from health-care workers (HCWs) within Singapore, by allowing them to report influenza incidence using smartphones. The system, integrating a fully responsive web-based interface and a mobile interface, is made available to HCW using various types of mobile devices and web browsers. Real-time data generated from FluMob will be complementary to current health-care- and laboratory-based systems. This paper describes the development of FluMob, as well as challenges faced in the creation of the system.

The impact of stratified immunity on the transmission dynamics of influenza

•

The disease model with stratified immunity improves the accuracy on influenza epidemic reconstruction.

•

Antibody boosting in children is greater than adults during influenza outbreak.

•

Age-specific mixing pattern and the relative infectivity of children to adults are the key drivers of infection heterogeneity.

Although empirical studies show that protection against influenza infection in humans is closely related to antibody titres, influenza epidemics are often described under the assumption that individuals are either susceptible or not. Here we develop a model in which antibody titre classes are enumerated explicitly and mapped onto a variable scale of susceptibility in different age groups. Fitting only with pre- and post-wave serological data during 2009 pandemic in Hong Kong, we demonstrate that with stratified immunity, the timing and the magnitude of the epidemic dynamics can be reconstructed more accurately than is possible with binary seropositivity data. We also show that increased infectiousness of children relative to adults and age-specific mixing are required to reproduce age-specific seroprevalence observed in Hong Kong, while pre-existing immunity in the elderly is not. Overall, our results suggest that stratified immunity in an aged-structured heterogeneous population plays a significant role in determining the shape of influenza epidemics.

Model distinguishability and inference robustness in mechanisms of cholera transmission and loss of immunity

Mathematical models of cholera and waterborne disease vary widely in their structures, in terms of transmission pathways, loss of immunity, and a range of other features. These differences can affect model dynamics, with different models potentially yielding different predictions and parameter estimates from the same data. Given the increasing use of mathematical models to inform public health decision-making, it is important to assess model distinguishability (whether models can be distinguished based on fit to data) and inference robustness (whether inferences from the model are robust to realistic variations in model structure). In this paper, we examined the effects of uncertainty in model structure in the context of epidemic cholera, testing a range of models with differences in transmission and loss of immunity structure, based on known features of cholera epidemiology. We fit these models to simulated epidemic and long-term data, as well as data from the 2006 Angola epidemic. We evaluated model distinguishability based on fit to data, and whether the parameter values, model behavior, and forecasting ability can accurately be inferred from incidence data. In general, all models were able to successfully fit to all data sets, both real and simulated, regardless of whether the model generating the simulated data matched the fitted model. However, in the long-term data, the best model fits were achieved when the loss of immunity structures matched those of the model that simulated the data. Two parameters, one representing person-to-person transmission and the other representing the reporting rate, were accurately estimated across all models, while the remaining parameters showed broad variation across the different models and data sets. The basic reproduction number (R₀) was often poorly estimated even using the correct model, due to practical unidentifiability issues in the waterborne transmission pathway which were consistent across all models. Forecasting efforts using noisy data were not successful early in the outbreaks, but once the epidemic peak had been achieved, most models were able to capture the downward incidence trajectory with similar accuracy. Forecasting from noise-free data was generally successful for all outbreak stages using any model. Our results suggest that we are unlikely to be able to infer mechanistic details from epidemic case data alone, underscoring the need for broader data collection, such as immunity/serology status, pathogen dose response curves, and environmental pathogen data. Nonetheless, with sufficient data, conclusions from forecasting and some parameter estimates were robust to variations in the model structure, and comparative modeling can help to determine how realistic variations in model structure may affect the conclusions drawn from models and data.

A Likelihood Approach for Real-Time Calibration of Stochastic Compartmental Epidemic Models

Stochastic transmission dynamic models are especially useful for studying the early emergence of novel pathogens given the importance of chance events when the number of infectious individuals is small. However, methods for parameter estimation and prediction for these types of stochastic models remain limited. In this manuscript, we describe a calibration and prediction framework for stochastic compartmental transmission models of epidemics. The proposed method, Multiple Shooting for Stochastic systems (MSS), applies a linear noise approximation to describe the size of the fluctuations, and uses each new surveillance observation to update the belief about the true epidemic state. Using simulated outbreaks of a novel viral pathogen, we evaluate the accuracy of MSS for real-time parameter estimation and prediction during epidemics. We assume that weekly counts for the number of new diagnosed cases are available and serve as an imperfect proxy of incidence. We show that MSS produces accurate estimates of key epidemic parameters (i.e. mean duration of infectiousness, R0, and Reff) and can provide an accurate estimate of the unobserved number of infectious individuals during the course of an epidemic. MSS also allows for accurate prediction of the number and timing of future hospitalizations and the overall attack rate. We compare the performance of MSS to three state-of-the-art benchmark methods: 1) a likelihood approximation with an assumption of independent Poisson observations; 2) a particle filtering method; and 3) an ensemble Kalman filter method. We find that MSS significantly outperforms each of these three benchmark methods in the majority of epidemic scenarios tested. In summary, MSS is a promising method that may improve on current approaches for calibration and prediction using stochastic models of epidemics.

Cloud-based Electronic Health Records for Real-time, Region-specific Influenza Surveillance

Accurate real-time monitoring systems of influenza outbreaks help public health officials make informed decisions that may help save lives. We show that information extracted from cloud-based electronic health records databases, in combination with machine learning techniques and historical epidemiological information, have the potential to accurately and reliably provide near real-time regional estimates of flu outbreaks in the United States.

Detecting signals of seasonal influenza severity through age dynamics

BACKGROUND

Measures of population-level influenza severity are important for public health planning, but estimates are often based on case-fatality and case-hospitalization risks, which require multiple data sources, are prone to surveillance biases, and are typically unavailable in the early stages of an outbreak. To address the limitations of traditional indicators, we propose a novel severity index based on influenza age dynamics estimated from routine physician diagnosis data that can be used retrospectively and for early warning.

METHODS

We developed a quantitative 'ground truth' severity benchmark that synthesizes multiple traditional severity indicators from publicly available influenza surveillance data in the United States. Observing that the age distribution of cases may signal severity early in an epidemic, we constructed novel retrospective and early warning severity indexes based on the relative risk of influenza-like illness (ILI) among working-age adults to that among school-aged children using weekly outpatient medical claims. We compared our relative risk-based indexes to the composite benchmark and estimated seasonal severity for flu seasons from 2001-02 to 2008-09 at the national and state levels.

RESULTS

The severity classifications made by the benchmark were not uniquely captured by any single contributing metric, including pneumonia and influenza mortality; the influenza epidemics of 2003-04 and 2007-08 were correctly identified as the most severe of the study period. The retrospective index was well correlated with the severity benchmark and correctly identified the two most severe seasons. The early warning index performance varied, but it projected 2007-08 as relatively severe 10 weeks prior to the epidemic peak. Influenza severity varied significantly among states within seasons, and four states were identified as possible early warning sentinels for national severity.

CONCLUSIONS

Differences in age patterns of ILI may be used to characterize seasonal influenza severity in the United States in real-time and in a spatially resolved way. Future research on antigenic changes among circulating viruses, pre-existing immunity, and changing contact patterns may better elucidate the mechanisms underlying these indexes. Researchers and practitioners should consider the use of composite or ILI-based severity metrics in addition to traditional severity measures to inform epidemiological understanding and situational awareness in future seasonal outbreaks.

Accurate estimation of influenza epidemics using Google search data via ARGO

Accurate real-time tracking of influenza outbreaks helps public health officials make timely and meaningful decisions that could save lives. We propose an influenza tracking model, ARGO (AutoRegression with GOogle search data), that uses publicly available online search data. In addition to having a rigorous statistical foundation, ARGO outperforms all previously available Google-search-based tracking models, including the latest version of Google Flu Trends, even though it uses only low-quality search data as input from publicly available Google Trends and Google Correlate websites. ARGO not only incorporates the seasonality in influenza epidemics but also captures changes in people's online search behavior over time. ARGO is also flexible, self-correcting, robust, and scalable, making it a potentially powerful tool that can be used for real-time tracking of other social events at multiple temporal and spatial resolutions.

Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance

We present a machine learning-based methodology capable of providing real-time ("nowcast") and forecast estimates of influenza activity in the US by leveraging data from multiple data sources including: Google searches, Twitter microblogs, nearly real-time hospital visit records, and data from a participatory surveillance system. Our main contribution consists of combining multiple influenza-like illnesses (ILI) activity estimates, generated independently with each data source, into a single prediction of ILI utilizing machine learning ensemble approaches. Our methodology exploits the information in each data source and produces accurate weekly ILI predictions for up to four weeks ahead of the release of CDC's ILI reports. We evaluate the predictive ability of our ensemble approach during the 2013-2014 (retrospective) and 2014-2015 (live) flu seasons for each of the four weekly time horizons. Our ensemble approach demonstrates several advantages: (1) our ensemble method's predictions outperform every prediction using each data source independently, (2) our methodology can produce predictions one week ahead of GFT's real-time estimates with comparable accuracy, and (3) our two and three week forecast estimates have comparable accuracy to real-time predictions using an autoregressive model. Moreover, our results show that considerable insight is gained from incorporating disparate data streams, in the form of social media and crowd sourced data, into influenza predictions in all time horizons.

Early Characterization of the Severity and Transmissibility of Pandemic Influenza Using Clinical Episode Data from Multiple Populations

The potential rapid availability of large-scale clinical episode data during the next influenza pandemic suggests an opportunity for increasing the speed with which novel respiratory pathogens can be characterized. Key intervention decisions will be determined by both the transmissibility of the novel strain (measured by the basic reproductive number R0) and its individual-level severity. The 2009 pandemic illustrated that estimating individual-level severity, as described by the proportion pC of infections that result in clinical cases, can remain uncertain for a prolonged period of time. Here, we use 50 distinct US military populations during 2009 as a retrospective cohort to test the hypothesis that real-time encounter data combined with disease dynamic models can be used to bridge this uncertainty gap. Effectively, we estimated the total number of infections in multiple early-affected communities using the model and divided that number by the known number of clinical cases. Joint estimates of severity and transmissibility clustered within a relatively small region of parameter space, with 40 of the 50 populations bounded by: pC, 0.0133-0.150 and R0, 1.09-2.16. These fits were obtained despite widely varying incidence profiles: some with spring waves, some with fall waves and some with both. To illustrate the benefit of specific pairing of rapidly available data and infectious disease models, we simulated a future moderate pandemic strain with pC approximately ×10 that of 2009; the results demonstrating that even before the peak had passed in the first affected population, R0 and pC could be well estimated. This study provides a clear reference in this two-dimensional space against which future novel respiratory pathogens can be rapidly assessed and compared with previous pandemics.

Potential Biases in Estimating Absolute and Relative Case-Fatality Risks during Outbreaks

Estimating the case-fatality risk (CFR)-the probability that a person dies from an infection given that they are a case-is a high priority in epidemiologic investigation of newly emerging infectious diseases and sometimes in new outbreaks of known infectious diseases. The data available to estimate the overall CFR are often gathered for other purposes (e.g., surveillance) in challenging circumstances. We describe two forms of bias that may affect the estimation of the overall CFR-preferential ascertainment of severe cases and bias from reporting delays-and review solutions that have been proposed and implemented in past epidemics. Also of interest is the estimation of the causal impact of specific interventions (e.g., hospitalization, or hospitalization at a particular hospital) on survival, which can be estimated as a relative CFR for two or more groups. When observational data are used for this purpose, three more sources of bias may arise: confounding, survivorship bias, and selection due to preferential inclusion in surveillance datasets of those who are hospitalized and/or die. We illustrate these biases and caution against causal interpretation of differential CFR among those receiving different interventions in observational datasets. Again, we discuss ways to reduce these biases, particularly by estimating outcomes in smaller but more systematically defined cohorts ascertained before the onset of symptoms, such as those identified by forward contact tracing. Finally, we discuss the circumstances in which these biases may affect non-causal interpretation of risk factors for death among cases.

Whole-genome sequencing in outbreak analysis

In addition to the ever-present concern of medical professionals about epidemics of infectious diseases, the relative ease of access and low cost of obtaining, producing, and disseminating pathogenic organisms or biological toxins mean that bioterrorism activity should also be considered when facing a disease outbreak. Utilization of whole-genome sequencing (WGS) in outbreak analysis facilitates the rapid and accurate identification of virulence factors of the pathogen and can be used to identify the path of disease transmission within a population and provide information on the probable source. Molecular tools such as WGS are being refined and advanced at a rapid pace to provide robust and higher-resolution methods for identifying, comparing, and classifying pathogenic organisms. If these methods of pathogen characterization are properly applied, they will enable an improved public health response whether a disease outbreak was initiated by natural events or by accidental or deliberate human activity. The current application of next-generation sequencing (NGS) technology to microbial WGS and microbial forensics is reviewed.