Epidemic features affecting the performance of outbreak detection algorithms

Web-Based Apps for Responding to Acute Infectious Disease Outbreaks in the Community: Systematic Review

Background

Web-based technology has dramatically improved our ability to detect communicable disease outbreaks, with the potential to reduce morbidity and mortality because of swift public health action. Apps accessible through the internet and on mobile devices create an opportunity to enhance our traditional indicator-based surveillance systems, which have high specificity but issues with timeliness.

Objective

The aim of this study is to describe the literature on web-based apps for indicator-based surveillance and response to acute communicable disease outbreaks in the community with regard to their design, implementation, and evaluation.

Methods

We conducted a systematic search of the published literature across four databases (MEDLINE via OVID, Web of Science Core Collection, ProQuest Science, and Google Scholar) for peer-reviewed journal papers from January 1998 to October 2019 using a keyword search. Papers with the full text available were extracted for review, and exclusion criteria were applied to identify eligible papers.

Results

Of the 6649 retrieved papers, 23 remained, describing 15 web-based apps. Apps were primarily designed to improve the early detection of disease outbreaks, targeted government settings, and comprised either complex algorithmic or statistical outbreak detection mechanisms or both. We identified a need for these apps to have more features to support secure information exchange and outbreak response actions, with a focus on outbreak verification processes and staff and resources to support app operations. Evaluation studies (6 out of 15 apps) were mostly cross-sectional, with some evidence of reduction in time to notification of outbreak; however, studies lacked user-based needs assessments and evaluation of implementation.

Conclusions

Public health officials designing new or improving existing disease outbreak web-based apps should ensure that outbreak detection is automatic and signals are verified by users, the app is easy to use, and staff and resources are available to support the operations of the app and conduct rigorous and holistic evaluations.

The moving epidemic method applied to influenza surveillance in Guangdong, China

OBJECTIVES

The moving epidemic method (MEM) has been well used for assessing seasonal influenza epidemics in temperate regions. This study used the MEM to establish epidemic threshold for influenza in Guangdong, a subtropical province in China.

METHODS

Influenza virology surveillance data from 2011/2012 to 2017/2018 seasons in Guangdong were used with the MEM to calculate the epidemic thresholds and timeously detect the 2018/2019 influenza season epidemic. The weekly positive proportion of influenza A(H1N1)pdm09, A(H3N2), B/Victoria-lineage and B/Yamagata-lineage were separately adapted to calculate the subtype-specific epidemic thresholds. The performance of MEM was evaluated using a cross-validation procedure.

RESULTS

For the 2018/2019 influenza season, the epidemic threshold of a weekly positive proportion was 15.08%. Epidemic detection for the 2018/2019 season was 1 week in advance. Influenza A(H1N1)pdm09, B/Yamagata-lineage and B/Victoria-lineage prevailed during winter and spring and their epidemic thresholds were 5.12%, 4.53% and 4.38%, respectively. Influenza A(H3N2) was active in the summer, with an epidemic threshold of 11.99%.

CONCLUSIONS

Using influenza virology surveillance data stratified by types of influenza virus, the MEM was effectively used in Guangdong, China. This study provided a practical way for subtropical regions to establish local influenza epidemic thresholds.

The comparative performance of wavelet-based outbreak detector, exponential weighted moving average, and Poisson regression-based methods in detection of pertussis outbreaks in Iranian infants: A simulation-based study

BACKGROUND

Early detection of outbreaks of transmissible diseases is essential for public health. This study aimed to determine the performance of the wavelet-based outbreak detection method (WOD) in detecting outbreaks and to compare its performance with the Poisson regression-based model and exponentially weighted moving average (EWMA) using data of simulated pertussis outbreaks in Iran.

METHOD

The data on suspected cases of pertussis from 25th February 2012 to 23rd March 2018 in Iran was used. The performance of the WOD (Daubechies 10 [db10] and Haar wavelets), Poisson regression-based method, and EWMA Compared in terms of timeliness and detection of outbreak days using the simulation of different outbreaks. In the current study, two simulations were used, one based on retrospectively collected data (literature-based) on pertussis cases and another one on a synthetic dataset created by the researchers. The sensitivity, specificity, false alarm, and false-negative rate, positive and negative likelihood ratios, under receiver operating characteristics areas, and median timeliness were used to assess the performance of the methods.

RESULTS

In a literature-based outbreak simulation, the highest and lowest sensitivity, false negative in the detection of injected outbreaks were seen in db10, with sensitivity 0.59 (0.56-0.62), and Haar wavelets with 0.57 (0.54-0.60). In the researcher simulated data, the EWMA (K = 0.5) with sensitivity 0.92 (0.90-0.94) had the best performance. About timeliness, the WOD methods showed the best performance in the early warning of the outbreak in both simulation approaches.

CONCLUSION

Performance of the WOD in the early alarming outbreaks was appropriate. However, this method would be best used along with other methods of public health surveillance.

Early dengue outbreak detection modeling based on dengue incidences in Singapore during 2012 to 2017

Dengue has been as an endemic with year-round presence in Singapore. In the recent years 2013, 2014, and 2016, there were several severe dengue outbreaks, posing serious threat to the public health. To proactively control and mitigate the disease spread, early warnings of dengue outbreaks, at which there are rapid and large-scale spread of dengue incidences, are extremely helpful. In this study, a two-step framework is proposed to predict dengue outbreaks and it is evaluated based on the dengue incidences in Singapore during 2012 to 2017. First, a generalized additive model (GAM) is trained based on the weekly dengue incidence data during 2006 to 2011. The proposed GAM is a one-week-ahead forecasting model, and it inherently accounts for the possible correlation among the historical incidence data, making the residuals approximately normally distributed. Then, an exponentially weighted moving average (EWMA) control chart is proposed to sequentially monitor the weekly residuals during 2012 to 2017. Our investigation shows that the proposed two-step framework is able to give persistent signals at the early stage of the outbreaks in 2013, 2014, and 2016, which provides early alerts of outbreaks and wins time for the early interventions and the preparation of necessary public health resources. In addition, extensive simulations show that the proposed method is comparable to other potential outbreak detection methods and it is robust to the underlying data-generating mechanisms.

Establishing seasonal and alert influenza thresholds in Morocco

BACKGROUND

Several statistical methods of variable complexity have been developed to establish thresholds for influenza activity that may be used to inform public health guidance. We compared the results of two methods and explored how they worked to characterize the 2018 influenza season performance-2018 season.

METHODS

Historical data from the 2005/2006 to 2016/2018 influenza season performance seasons were provided by a network of 412 primary health centers in charge of influenza like illness (ILI) sentinel surveillance. We used the WHO averages and the moving epidemic method (MEM) to evaluate the proportion of ILI visits among all outpatient consultations (ILI%) as a proxy for influenza activity. We also used the MEM method to evaluate three seasons of composite data (ILI% multiplied by percent of ILI with laboratory-confirmed influenza) as recommended by WHO.

RESULTS

The WHO method estimated the seasonal ILI% threshold at 0.9%. The annual epidemic period began on average at week 46 and lasted an average of 18 weeks. The MEM model estimated the epidemic threshold (corresponding to the WHO seasonal threshold) at 1.5% of ILI visits among all outpatient consultations. The annual epidemic period began on week 49 and lasted on average 14 weeks. Intensity thresholds were similar using both methods. When using the composite measure, the MEM method showed a clearer estimate of the beginning of the influenza epidemic, which was coincident with a sharp increase in confirmed ILI cases.

CONCLUSIONS

We found that the threshold methodology presented in the WHO manual is simple to implement and easy to adopt for use by the Moroccan influenza surveillance system. The MEM method is more statistically sophisticated and may allow a better detection of the start of seasonal epidemics. Incorporation of virologic data into the composite parameter as recommended by WHO has the potential to increase the accuracy of seasonal threshold estimation.

A systematic review of aberration detection algorithms used in public health surveillance

The algorithms used for detecting anomalies have evolved substantially over the last decade to take advantage of advances in informatics and to accommodate changes in surveillance data. We identified 145 studies since 2007 that evaluated statistical methods used to detect aberrations in public health surveillance data. For each study, we classified the analytic methods and reviewed the evaluation metrics. We also summarized the practical usage of the detection algorithms in public health surveillance systems worldwide. Traditional methods (e.g., control charts, linear regressions) were the focus of most evaluation studies and continue to be used commonly in practice. There was, however, an increase in the number of studies using forecasting methods and studies applying machine learning methods, hidden Markov models, and Bayesian framework to multivariate datasets. Evaluation studies demonstrated improved accuracy with more sophisticated methods, but these methods do not appear to be used widely in public health practice.

FluHMM: A simple and flexible Bayesian algorithm for sentinel influenza surveillance and outbreak detection

Timely detection of the seasonal influenza epidemic is important for public health action. We introduce FluHMM, a simple but flexible Bayesian algorithm to detect and monitor the seasonal epidemic on sentinel surveillance data. No comparable historical data are required for its use. FluHMM segments a typical influenza surveillance season into five distinct phases with clear interpretation (pre-epidemic, epidemic growth, epidemic plateau, epidemic decline and post-epidemic) and provides the posterior probability of being at each phase for every week in the period under surveillance, given the available data. An alert can be raised when the probability that the epidemic has started exceeds a given threshold. An accompanying R package facilitates the application of this method in public health practice. We apply FluHMM on 12 seasons of sentinel surveillance data from Greece, and show that it achieves very good sensitivity, timeliness and perfect specificity, thereby demonstrating its usefulness. We further discuss advantages and limitations of the method, providing suggestions on how to apply it and highlighting potential future extensions such as with integrating multiple surveillance data streams.

Influence of infectious disease seasonality on the performance of the outbreak detection algorithm in the China Infectious Disease Automated-alert and Response System

Objective The Chinese Center for Disease Control and Prevention developed the China Infectious Disease Automated-alert and Response System (CIDARS) in 2008. The CIDARS can detect outbreak signals in a timely manner but generates many false-positive signals, especially for diseases with seasonality. We assessed the influence of seasonality on infectious disease outbreak detection performance. Methods Chickenpox surveillance data in Songjiang District, Shanghai were used. The optimized early alert thresholds for chickenpox were selected according to three algorithm evaluation indexes: sensitivity (Se), false alarm rate (FAR), and time to detection (TTD). Performance of selected proper thresholds was assessed by data external to the study period. Results The optimized early alert threshold for chickenpox during the epidemic season was the percentile P65, which demonstrated an Se of 93.33%, FAR of 0%, and TTD of 0 days. The optimized early alert threshold in the nonepidemic season was P50, demonstrating an Se of 100%, FAR of 18.94%, and TTD was 2.5 days. The performance evaluation demonstrated that the use of an optimized threshold adjusted for seasonality could reduce the FAR and shorten the TTD. Conclusions Selection of optimized early alert thresholds based on local infectious disease seasonality could improve the performance of the CIDARS.

'Outbreak Gold Standard' selection to provide optimized threshold for infectious diseases early-alert based on China Infectious Disease Automated-alert and Response System

The China Infectious Disease Automated-alert and Response System (CIDARS) was successfully implemented and became operational nationwide in 2008. The CIDARS plays an important role in and has been integrated into the routine outbreak monitoring efforts of the Center for Disease Control (CDC) at all levels in China. In the CIDARS, thresholds are determined using the "Mean+2SD‟ in the early stage which have limitations. This study compared the performance of optimized thresholds defined using the "Mean +2SD‟ method to the performance of 5 novel algorithms to select optimal "Outbreak Gold Standard (OGS)‟ and corresponding thresholds for outbreak detection. Data for infectious disease were organized by calendar week and year. The "Mean+2SD‟, C1, C2, moving average (MA), seasonal model (SM), and cumulative sum (CUSUM) algorithms were applied. Outbreak signals for the predicted value (Px) were calculated using a percentile-based moving window. When the outbreak signals generated by an algorithm were in line with a Px generated outbreak signal for each week, this Px was then defined as the optimized threshold for that algorithm. In this study, six infectious diseases were selected and classified into TYPE A (chickenpox and mumps), TYPE B (influenza and rubella) and TYPE C [hand foot and mouth disease (HFMD) and scarlet fever]. Optimized thresholds for chickenpox (P₅₅), mumps (P₅₀), influenza (P₄₀, P₅₅, and P₇₅), rubella (P₄₅ and P₇₅), HFMD (P₆₅ and P₇₀), and scarlet fever (P₇₅ and P₈₀) were identified. The C1, C2, CUSUM, SM, and MA algorithms were appropriate for TYPE A. All 6 algorithms were appropriate for TYPE B. C1 and CUSUM algorithms were appropriate for TYPE C. It is critical to incorporate more flexible algorithms as OGS into the CIDRAS and to identify the proper OGS and corresponding recommended optimized threshold by different infectious disease types.

Comparing the historical limits method with regression models for weekly monitoring of national notifiable diseases reports

To compare the performance of the standard Historical Limits Method (HLM), with a modified HLM (MHLM), the Farrington-like Method (FLM), and the Serfling-like Method (SLM) in detecting simulated outbreak signals. We used weekly time series data from 12 infectious diseases from the U.S. Centers for Disease Control and Prevention's National Notifiable Diseases Surveillance System (NNDSS). Data from 2006 to 2010 were used as baseline and from 2011 to 2014 were used to test the four detection methods. MHLM outperformed HLM in terms of background alert rate, sensitivity, and alerting delay. On average, SLM and FLM had higher sensitivity than MHLM. Among the four methods, the FLM had the highest sensitivity and lowest background alert rate and alerting delay. Revising or replacing the standard HLM may improve the performance of aberration detection for NNDSS standard weekly reports.

How to select a proper early warning threshold to detect infectious disease outbreaks based on the China infectious disease automated alert and response system (CIDARS)

Background

China Centre for Diseases Control and Prevention (CDC) developed the China Infectious Disease Automated Alert and Response System (CIDARS) in 2005. The CIDARS was used to strengthen infectious disease surveillance and aid in the early warning of outbreak. The CIDARS has been integrated into the routine outbreak monitoring efforts of the CDC at all levels in China. Early warning threshold is crucial for outbreak detection in the CIDARS, but CDCs at all level are currently using thresholds recommended by the China CDC, and these recommended thresholds have recognized limitations. Our study therefore seeks to explore an operational method to select the proper early warning threshold according to the epidemic features of local infectious diseases.

Methods

The data used in this study were extracted from the web-based Nationwide Notifiable Infectious Diseases Reporting Information System (NIDRIS), and data for infectious disease cases were organized by calendar week (1–52) and year (2009–2015) in Excel format; Px was calculated using a percentile-based moving window (moving window [5 week*5 year], x), where x represents one of 12 centiles (0.40, 0.45, 0.50….0.95). Outbreak signals for the 12 Px were calculated using the moving percentile method (MPM) based on data from the CIDARS. When the outbreak signals generated by the ‘mean + 2SD’ gold standard were in line with a Px generated outbreak signal for each week during the year of 2014, this Px was then defined as the proper threshold for the infectious disease. Finally, the performance of new selected thresholds for each infectious disease was evaluated by simulated outbreak signals based on 2015 data.

Results

Six infectious diseases were selected in this study (chickenpox, mumps, hand foot and mouth diseases (HFMD), scarlet fever, influenza and rubella). Proper thresholds for chickenpox (P75), mumps (P80), influenza (P75), rubella (P45), HFMD (P75), and scarlet fever (P80) were identified. The selected proper thresholds for these 6 infectious diseases could detect almost all simulated outbreaks within a shorter time period compared to thresholds recommended by the China CDC.

Conclusions

It is beneficial to select the proper early warning threshold to detect infectious disease aberrations based on characteristics and epidemic features of local diseases in the CIDARS.

Hand, foot and mouth disease in China: evaluating an automated system for the detection of outbreaks

Objective

To evaluate the performance of China’s infectious disease automated alert and response system in the detection of outbreaks of hand, foot and mouth (HFM) disease.

Methods

We estimated size, duration and delay in reporting HFM disease outbreaks from cases notified between 1 May 2008 and 30 April 2010 and between 1 May 2010 and 30 April 2012, before and after automatic alert and response included HFM disease. Sensitivity, specificity and timeliness of detection of aberrations in the incidence of HFM disease outbreaks were estimated by comparing automated detections to observations of public health staff.

Findings

The alert and response system recorded 106 005 aberrations in the incidence of HFM disease between 1 May 2010 and 30 April 2012 – a mean of 5.6 aberrations per 100 days in each county that reported HFM disease. The response system had a sensitivity of 92.7% and a specificity of 95.0%. The mean delay between the reporting of the first case of an outbreak and detection of that outbreak by the response system was 2.1 days. Between the first and second study periods, the mean size of an HFM disease outbreak decreased from 19.4 to 15.8 cases and the mean interval between the onset and initial reporting of such an outbreak to the public health emergency reporting system decreased from 10.0 to 9.1 days.

Conclusion

The automated alert and response system shows good sensitivity in the detection of HFM disease outbreaks and appears to be relatively rapid. Continued use of this system should allow more effective prevention and limitation of such outbreaks in China.

Exploring a proposed WHO method to determine thresholds for seasonal influenza surveillance

Introduction

Health authorities find thresholds useful to gauge the start and severity of influenza seasons. We explored a method for deriving thresholds proposed in an influenza surveillance manual published by the World Health Organization (WHO).

Methods

For 2002-2011, we analysed two routine influenza-like-illness (ILI) datasets, general practice sentinel surveillance and a locum medical service sentinel surveillance, plus laboratory data and hospital admissions for influenza. For each sentinel dataset, we created two composite variables from the product of weekly ILI data and the relevant laboratory data, indicating the proportion of tested specimens that were positive. For all datasets, including the composite datasets, we aligned data on the median week of peak influenza or ILI activity and assigned three threshold levels: seasonal threshold, determined by inspection; and two intensity thresholds termed average and alert thresholds, determined by calculations of means, medians, confidence intervals (CI) and percentiles. From the thresholds, we compared the seasonal onset, end and intensity across all datasets from 2002-2011. Correlation between datasets was assessed using the mean correlation coefficient.

Results

The median week of peak activity was week 34 for all datasets, except hospital data (week 35). Means and medians were comparable and the 90% upper CIs were similar to the 95^th percentiles. Comparison of thresholds revealed variations in defining the start of a season but good agreement in describing the end and intensity of influenza seasons, except in hospital admissions data after the pandemic year of 2009. The composite variables improved the agreements between the ILI and other datasets. Datasets were well correlated, with mean correlation coefficients of >0.75 for a range of combinations.

Conclusions

Thresholds for influenza surveillance are easily derived from historical surveillance and laboratory data using the approach proposed by WHO. Use of composite variables is helpful for describing influenza season characteristics.