Modeling emergency department visit patterns for infectious disease complaints: results and application to disease surveillance

Mathematical models and analysis tools for risk assessment of unnatural epidemics: a scoping review

Predicting, issuing early warnings, and assessing risks associated with unnatural epidemics (UEs) present significant challenges. These tasks also represent key areas of focus within the field of prevention and control research for UEs. A scoping review was conducted using databases such as PubMed, Web of Science, Scopus, and Embase, from inception to 31 December 2023. Sixty-six studies met the inclusion criteria. Two types of models (data-driven and mechanistic-based models) and a class of analysis tools for risk assessment of UEs were identified. The validation part of models involved calibration, improvement, and comparison. Three surveillance systems (event-based, indicator-based, and hybrid) were reported for monitoring UEs. In the current study, mathematical models and analysis tools suggest a distinction between natural epidemics and UEs in selecting model parameters and warning thresholds. Future research should consider combining a mechanistic-based model with a data-driven model and learning to pursue time-varying, high-precision risk assessment capabilities.

Use of Real-Time Information to Predict Future Arrivals in the Emergency Department

STUDY OBJECTIVE

We aimed to build prediction models for shift-level emergency department (ED) patient volume that could be used to facilitate prediction-driven staffing. We sought to evaluate the predictive power of rich real-time information and understand 1) which real-time information had predictive power and 2) what prediction techniques were appropriate for forecasting ED demand.

METHODS

We conducted a retrospective study in an ED site in a large academic hospital in New York City. We examined various prediction techniques, including linear regression, regression trees, extreme gradient boosting, and time series models. By comparing models with and without real-time predictors, we assessed the potential gain in prediction accuracy from real-time information.

RESULTS

Real-time predictors improved prediction accuracy on models without contemporary information from 5% to 11%. Among extensive real-time predictors examined, recent patient arrival counts, weather, Google trends, and concurrent patient comorbidity information had significant predictive power. Out of all the forecasting techniques explored, SARIMAX (Seasonal Autoregressive Integrated Moving Average with eXogenous factors) achieved the smallest out-of-sample the root mean square error (RMSE) of 14.656 and mean absolute prediction error (MAPE) of 8.703%. Linear regression was the second best, with out-of-sample RMSE and MAPE equal to 15.366 and 9.109%, respectively.

CONCLUSION

Real-time information was effective in improving the prediction accuracy of ED demand. Practice and policy implications for designing staffing paradigms with real-time demand forecasts to reduce ED congestion were discussed.

Hepatitis C active viremia over time in an ED-based testing programme: Impact, disparities and surveillance tool

Hepatitis C virus (HCV) surveillance is a critical component of a comprehensive strategy to prevent and control HCV infection and HCV-related chronic liver disease. The emergency department (ED) has been increasingly recognized as a vital partner in HCV testing and linkage. We sought to consider active RNA HCV viremia over time in patients participating in an ED-based testing programme as a measure of local HCV surveillance and as a barometer of ED-testing programme impact. We performed a retrospective analysis of individuals participating in our ED-based HCV testing programme between 2015 and 2021. Chi-square tests were used to compare the demographic characteristics of HCV antibody positive tests with active viremia to those without active viremia. Cox proportional hazard models were used to estimate the trend in active viremia risk over time in the overall study population as well as in key subpopulations of interest. Of 5456 HCV antibody positive individuals, 3102 (56.8%) had active viremia. In the overall study population, we found that the risk of active viremia decreased by 4.8% per year during the study period (RR: 0.95, 95% CI: 0.93-0.97|p < .0001). Baby boomers experienced a 9% decrease in active viremia risk per year over the study period while non-baby boomers only had a 2% decrease in risk per year (p = .0009). Compared with insured patients, uninsured patients had a smaller decrease in risk of active HCV viremia per year (p = .003). No significant differences in the risk of active viremia over time were observed for gender (p = .4694) or by primary care provider status (p = .2208). In conclusion, this ED-based testing and linkage programme demonstrates significantly decreased active HCV viremia over time. It also highlights subpopulations, specifically non-baby boomers and uninsured patients, who may benefit from focused interventions to improve access to and adoption of definitive HCV care.

A Survey of the Practice Status Quo of Ultrasound-Guided ECC Tip Location for Neonatal Patients in 31 Provinces of China

OBJECTIVE

To investigate the status quo of implementing ultrasound (US)-guided epicutaneo-caval catheters (ECC) tip location for neonatal patients in 31 provinces.

METHODS

The convenience sampling method was used to investigate the nursing managers and ECC (or intravenous therapy) nurses of 91 hospitals in 31 provinces from October 29 to November 10, 2021.

RESULTS

The survey involved a total of 182 medical staff, including 91 managers and 91 nurses, and 91 institutions, including 22 children's hospitals, 49 general hospitals and 21 maternal and child health care hospitals. Sixteen hospitals (17.6%) carried out US-guided ECC for neonatal patients; 176 subjects (96.7%) of the 91 hospitals had known about or heard of the technology of US-guided ECC. The low awareness of operators of the tip location of ECC catheters in children under ultrasound guidance (OR = 2.690, 95% CI = 1.163-6.221), limited conditions in existing wards (OR = 2.953, 95% CI = 1.285-6.790), and insufficient funds (OR = 2.836, 95% CI = 1.149-7.004) were the independent risk factors responsible for the failure to carry out ultrasonic-guided location of ECC tips in newborns.

CONCLUSION

The popularity of neonatal US-guided ECC location was seriously hindered by factors such as a low awareness rate of the project, the low qualification certification rate of the nursing staff, a flawed performance allocation system, and the lack of a professional team, among other factors.

Machine Learning-Based Patient Load Prediction and IoT Integrated Intelligent Patient Transfer Systems

A mismatch between staffing ratios and service demand leads to overcrowding of patients in waiting rooms of health centers. Overcrowding consequently leads to excessive patient waiting times, incomplete preventive service delivery and disgruntled medical staff. Worse, due to the limited patient load that a health center can handle, patients may leave the clinic before the medical examination is complete. It is true that as one health center may be struggling with an excessive patient load, another facility in the vicinity may have a low patient turn out. A centralized hospital management system, where hospitals are able to timely exchange patient load information would allow excess patient load from an overcrowded health center to be re-assigned in a timely way to the nearest health centers. In this paper, a machine learning-based patient load prediction model for forecasting future patient loads is proposed. Given current and historical patient load data as inputs, the model outputs future predicted patient loads. Furthermore, we propose re-assigning excess patient loads to nearby facilities that have minimal load as a way to control overcrowding and reduce the number of patients that leave health facilities without receiving medical care as a result of overcrowding. The re-assigning of patients will imply a need for transportation for the patient to move from one facility to another. To avoid putting a further strain on the already fragmented ambulatory services, we assume the existence of a scheduled bus system and propose an Internet of Things (IoT) integrated smart bus system. The developed IoT system can be tagged on buses and can be queried by patients through representation state transfer application program interfaces (APIs) to provide them with the position of the buses through web app or SMS relative to their origin and destination stop. The back end of the proposed system is based on message queue telemetry transport, which is lightweight, data efficient and scalable, unlike the traditionally used hypertext transfer protocol.

Predicting seasonal influenza epidemics using cross-hemisphere influenza surveillance data and local internet query data

Can early warning systems be developed to predict influenza epidemics? Using Australian influenza surveillance and local internet search query data, this study investigated whether seasonal influenza epidemics in China, the US and the UK can be predicted using empirical time series analysis. Weekly national number of respiratory cases positive for influenza virus infection that were reported to the FluNet surveillance system in Australia, China, the US and the UK were obtained from World Health Organization FluNet surveillance between week 1, 2010, and week 9, 2018. We collected combined search query data for the US and the UK from Google Trends, and for China from Baidu Index. A multivariate seasonal autoregressive integrated moving average model was developed to track influenza epidemics using Australian influenza and local search data. Parameter estimates for this model were generally consistent with the observed values. The inclusion of search metrics improved the performance of the model with high correlation coefficients (China = 0.96, the US = 0.97, the UK = 0.96, p < 0.01) and low Maximum Absolute Percent Error (MAPE) values (China = 16.76, the US = 96.97, the UK = 125.42). This study demonstrates the feasibility of combining (Australia) influenza and local search query data to predict influenza epidemics a different (northern hemisphere) scales.

Period of Measurement in Time-Series Predictions of Disease Counts from 2007 to 2017 in Northern Nevada: Analytics Experiment

BACKGROUND

The literature in statistics presents methods by which autocorrelation can identify the best period of measurement to improve the performance of a time-series prediction. The period of measurement plays an important role in improving the performance of disease-count predictions. However, from the operational perspective in public health surveillance, there is a limitation to the length of the measurement period that can offer meaningful and valuable predictions.

OBJECTIVE

This study aimed to establish a method that identifies the shortest period of measurement without significantly decreasing the prediction performance for time-series analysis of disease counts.

METHODS

The data used in this evaluation include disease counts from 2007 to 2017 in northern Nevada. The disease counts for chlamydia, salmonella, respiratory syncytial virus, gonorrhea, viral meningitis, and influenza A were predicted.

RESULTS

Our results showed that autocorrelation could not guarantee the best performance for prediction of disease counts. However, the proposed method with the change-point analysis suggests a period of measurement that is operationally acceptable and performance that is not significantly different from the best prediction.

CONCLUSIONS

The use of change-point analysis with autocorrelation provides the best and most practical period of measurement.

Influenza detection and prediction algorithms: comparative accuracy trial in Östergötland county, Sweden, 2008–2012

Methods for the detection of influenza epidemics and prediction of their progress have seldom been comparatively evaluated using prospective designs. This study aimed to perform a prospective comparative trial of algorithms for the detection and prediction of increased local influenza activity. Data on clinical influenza diagnoses recorded by physicians and syndromic data from a telenursing service were used. Five detection and three prediction algorithms previously evaluated in public health settings were calibrated and then evaluated over 3 years. When applied on diagnostic data, only detection using the Serfling regression method and prediction using the non-adaptive log-linear regression method showed acceptable performances during winter influenza seasons. For the syndromic data, none of the detection algorithms displayed a satisfactory performance, while non-adaptive log-linear regression was the best performing prediction method. We conclude that evidence was found for that available algorithms for influenza detection and prediction display satisfactory performance when applied on local diagnostic data during winter influenza seasons. When applied on local syndromic data, the evaluated algorithms did not display consistent performance. Further evaluations and research on combination of methods of these types in public health information infrastructures for ‘nowcasting’ (integrated detection and prediction) of influenza activity are warranted.

Simulation to Predict Effect of Citywide Events on Emergency Department Operations

Medical emergency preparedness has been an issue of medical relevance since the advent of hospital care. Studies have simulated emergency department (ED) overcrowding but not yet characterized effects of large-scale, planned events that drastically alter a city's demography, such as in Philadelphia, Pennsylvania during the 2015 World Meeting of Families. A discrete event simulation of the ED at the Children's Hospital of Philadelphia was designed and validated using past data. The model was used to predict the patient length of stay (LOS) and number of admitted patients if the arrival stream to the ED were to change by 50% from typical arrivals in either direction. We compared the model's estimations with data produced during the papal visit that had 39.65% fewer patient arrivals. For validation, the simulated mean LOS was 226.1 ± 173.3 minutes (mean ± SD) for all patients and 352.1 ± 170.3 minutes for admitted patients. Real-world mean LOSs for the fiscal year 2014 were 230.6 ± 134.8 for all patients and 345.0 ± 147.7 for admitted patients. For the estimation of the World Meeting of Families, the simulation accurately estimated the LOS of both patients overall and admitted patients within 10%. These results show that it is possible to use simulations to project the patient flow effects in EDs in case of large-scale events. Providing efficient care is essential to emergency operations, and projections of demand are crucial for targeting appropriate changes during large-scale events. Analysis of validated computer simulations allows for evidence-based decision making in a complex clinical environment.

Time Series Analysis for Forecasting Hospital Census: Application to the Neonatal Intensive Care Unit

BACKGROUND

Accurate prediction of future patient census in hospital units is essential for patient safety, health outcomes, and resource planning. Forecasting census in the Neonatal Intensive Care Unit (NICU) is particularly challenging due to limited ability to control the census and clinical trajectories. The fixed average census approach, using average census from previous year, is a forecasting alternative used in clinical practice, but has limitations due to census variations.

OBJECTIVE

Our objectives are to: (i) analyze the daily NICU census at a single health care facility and develop census forecasting models, (ii) explore models with and without patient data characteristics obtained at the time of admission, and (iii) evaluate accuracy of the models compared with the fixed average census approach.

METHODS

We used five years of retrospective daily NICU census data for model development (January 2008 - December 2012, N=1827 observations) and one year of data for validation (January - December 2013, N=365 observations). Best-fitting models of ARIMA and linear regression were applied to various 7-day prediction periods and compared using error statistics.

RESULTS

The census showed a slightly increasing linear trend. Best fitting models included a non-seasonal model, ARIMA(1,0,0), seasonal ARIMA models, ARIMA(1,0,0)x(1,1,2)7 and ARIMA(2,1,4)x(1,1,2)14, as well as a seasonal linear regression model. Proposed forecasting models resulted on average in 36.49% improvement in forecasting accuracy compared with the fixed average census approach.

CONCLUSIONS

Time series models provide higher prediction accuracy under different census conditions compared with the fixed average census approach. Presented methodology is easily applicable in clinical practice, can be generalized to other care settings, support short- and long-term census forecasting, and inform staff resource planning.

A practical method for predicting frequent use of emergency department care using routinely available electronic registration data

Background

Accurately predicting future frequent emergency department (ED) utilization can support a case management approach and ultimately reduce health care costs. This study assesses the feasibility of using routinely collected registration data to predict future frequent ED visits.

Method

Using routinely collected registration data in the state of Indiana, U.S.A., from 2008, we developed multivariable logistic regression models to predict frequent ED visits in the subsequent two years. We assessed the model’s accuracy using Receiver Operating Characteristic (ROC) curves, sensitivity, and positive predictive value (PPV).

Results

Strong predictors of frequent ED visits included age between 25 and 44 years, female gender, close proximity to the ED (less than 5 miles traveling distance), total visits in the baseline year, and respiratory and dental chief complaint syndromes. The area under ROC curve (AUC) ranged from 0.83 to 0.92 for models predicting patients with 8 or more visits to 16 or more visits in the subsequent two years, suggesting acceptable discrimination. With 25 % sensitivity, the model predicting frequent ED use as defined as 16 or more visits in 2009 and 2010 had a PPV of 59.5 % and specificity of 99.9 %. The “adjusted” PPV of this model, which includes patients having 8 or more visits, is 81.9 %.

Conclusion

We demonstrate a strong association between predictor variables present in registration data and frequent ED use. The algorithm’s performance characteristics suggest that it is technically feasible to use routinely collected registration data to predict future frequent ED use.

Practical comparison of aberration detection algorithms for biosurveillance systems

National syndromic surveillance systems require optimal anomaly detection methods. For method performance comparison, we injected multi-day signals stochastically drawn from lognormal distributions into time series of aggregated daily visit counts from the U.S. Centers for Disease Control and Prevention's BioSense syndromic surveillance system. The time series corresponded to three different syndrome groups: rash, upper respiratory infection, and gastrointestinal illness. We included a sample of facilities with data reported every day and with median daily syndromic counts ⩾1 over the entire study period. We compared anomaly detection methods of five control chart adaptations, a linear regression model and a Poisson regression model. We assessed sensitivity and timeliness of these methods for detection of multi-day signals. At a daily background alert rate of 1% and 2%, the sensitivities and timeliness ranged from 24 to 77% and 3.3 to 6.1days, respectively. The overall sensitivity and timeliness increased substantially after stratification by weekday versus weekend and holiday. Adjusting the baseline syndromic count by the total number of facility visits gave consistently improved sensitivity and timeliness without stratification, but it provided better performance when combined with stratification. The daily syndrome/total-visit proportion method did not improve the performance. In general, alerting based on linear regression outperformed control chart based methods. A Poisson regression model obtained the best sensitivity in the series with high-count data.

Emergency Department Presentations following Tropical Cyclone Yasi

Introduction

Emergency departments see an increase in cases during cyclones. The aim of this study is to describe patient presentations to the Emergency Department (ED) of a tertiary level hospital (Townsville) following a tropical cyclone (Yasi). Specific areas of focus include changes in: patient demographics (age and gender), triage categories, and classification of diseases.

Methods

Data were extracted from the Townsville Hospitals ED information system (EDIS) for three periods in 2009, 2010 and 2011 to coincide with formation of Cyclone Yasi (31 January 2011) to six days after Yasi crossed the coast line (8 February 2012). The analysis explored the changes in ICD10-AM 4-character classification and presented at the Chapter level.

Results

There was a marked increase in the number of patients attending the ED during Yasi, particularly those aged over 65 years with a maximum daily attendance of 372 patients on 4 Feb 2011. The most marked increases were in: Triage categories - 4 and 5; and ICD categories - diseases of the skin and subcutaneous tissue (L00-L99), and factors influencing health care status (Z00-Z99). The most common diagnostic presentation across all years was injury (S00-T98).

Discussion

There was an increase in presentations to the ED of TTH, which peaked in the first 24 – 48 hours following the cyclone and returned to normal over a five-day period. The changes in presentations were mostly an amplification of normal attendance patterns with some altered areas of activity. Injury patterns are similar to overseas experience.

Surveillance for emerging respiratory viruses

Several new viral respiratory tract infectious diseases with epidemic potential that threaten global health security have emerged in the past 15 years. In 2003, WHO issued a worldwide alert for an unknown emerging illness, later named severe acute respiratory syndrome (SARS). The disease caused by a novel coronavirus (SARS-CoV) rapidly spread worldwide, causing more than 8000 cases and 800 deaths in more than 30 countries with a substantial economic impact. Since then, we have witnessed the emergence of several other viral respiratory pathogens including influenza viruses (avian influenza H5N1, H7N9, and H10N8; variant influenza A H3N2 virus), human adenovirus-14, and Middle East respiratory syndrome coronavirus (MERS-CoV). In response, various surveillance systems have been developed to monitor the emergence of respiratory-tract infections. These include systems based on identification of syndromes, web-based systems, systems that gather health data from health facilities (such as emergency departments and family doctors), and systems that rely on self-reporting by patients. More effective national, regional, and international surveillance systems are required to enable rapid identification of emerging respiratory epidemics, diseases with epidemic potential, their specific microbial cause, origin, mode of acquisition, and transmission dynamics.

What's the healthiest day?: Circaseptan (weekly) rhythms in healthy considerations

BACKGROUND

Biological clocks govern numerous aspects of human health, including weekly clocks-called circaseptan rhythms-that typically include early-week spikes for many illnesses.

PURPOSE

To determine whether contemplations for healthy behaviors also follow circaseptan rhythms.

METHODS

We assessed healthy contemplations by monitoring Google search queries (2005-2012) in the U.S. that included the word healthy and were Google classified as health-related (e.g., healthy diet). A wavelet analysis was used in 2013 to isolate the circaseptan rhythm, with the resulting series compared by estimating ratios of relative query volume (healthy versus all queries) each day (e.g., (Monday-Wednesday)/Wednesday).

RESULTS

Healthy searches peaked on Monday and Tuesday, thereafter declining until rebounding modestly on Sunday. Monday and Tuesday were statistically indistinguishable (t=1.22, p=0.22), but their combined mean had 30% (99% CI=29, 32) more healthy queries than the combined mean for Wednesday-Sunday. Monday and Tuesday query volume was 3% (99% CI=2, 5) greater than Wednesday, 15% (99% CI=13, 17) greater than Thursday, 49% (99% CI=46, 52) greater than Friday, 80% (99% CI=76, 84) greater than Saturday, and 29% (99% CI=27, 31) greater than Sunday. We explored media-based (priming) motivations for these patterns and they were consistently rejected.

CONCLUSIONS

Just as many illnesses have a weekly clock, so do healthy considerations. Discovery of these rhythms opens the door for a new agenda in preventive medicine, including implications for hypothesis development, research strategies to further explore these rhythms, and interventions to exploit daily cycles in healthy considerations.

Detecting and diagnosing hotspots for the enhanced management of hospital Emergency Departments in Queensland, Australia

BACKGROUND

Predictive tools are already being implemented to assist in Emergency Department bed management by forecasting the expected total volume of patients. Yet these tools are unable to detect and diagnose when estimates fall short. Early detection of hotspots, that is subpopulations of patients presenting in unusually high numbers, would help authorities to manage limited health resources and communicate effectively about emerging risks. We evaluate an anomaly detection tool that signals when, and in what way Emergency Departments in 18 hospitals across the state of Queensland, Australia, are significantly exceeding their forecasted patient volumes.

METHODS

The tool in question is an adaptation of the Surveillance Tree methodology initially proposed in Sparks and Okugami (IntStatl 1:2-24, 2010). for the monitoring of vehicle crashes. The methodology was trained on presentations to 18 Emergency Departments across Queensland over the period 2006 to 2008. Artificial increases were added to simulated, in-control counts for these data to evaluate the tool's sensitivity, timeliness and diagnostic capability. The results were compared with those from a univariate control chart. The tool was then applied to data from 2009, the year of the H1N1 (or 'Swine Flu') pandemic.

RESULTS

The Surveillance Tree method was found to be at least as effective as a univariate, exponentially weighted moving average (EWMA) control chart when increases occurred in a subgroup of the monitored population. The method has advantages over the univariate control chart in that it allows for the monitoring of multiple disease groups while still allowing control of the overall false alarm rate. It is also able to detect changes in the makeup of the Emergency Department presentations, even when the total count remains unchanged. Furthermore, the Surveillance Tree method provides diagnostic information useful for service improvements or disease management.

CONCLUSIONS

Multivariate surveillance provides a useful tool in the management of hospital Emergency Departments by not only efficiently detecting unusually high numbers of presentations, but by providing information about which groups of patients are causing the increase.

Syndromic surveillance for influenza in the emergency department-A systematic review

The science of surveillance is rapidly evolving due to changes in public health information and preparedness as national security issues, new information technologies and health reform. As the Emergency Department has become a much more utilized venue for acute care, it has also become a more attractive data source for disease surveillance. In recent years, influenza surveillance from the Emergency Department has increased in scope and breadth and has resulted in innovative and increasingly accepted methods of surveillance for influenza and influenza-like-illness (ILI). We undertook a systematic review of published Emergency Department-based influenza and ILI syndromic surveillance systems. A PubMed search using the keywords "syndromic", "surveillance", "influenza" and "emergency" was performed. Manuscripts were included in the analysis if they described (1) data from an Emergency Department (2) surveillance of influenza or ILI and (3) syndromic or clinical data. Meeting abstracts were excluded. The references of included manuscripts were examined for additional studies. A total of 38 manuscripts met the inclusion criteria, describing 24 discrete syndromic surveillance systems. Emergency Department-based influenza syndromic surveillance has been described worldwide. A wide variety of clinical data was used for surveillance, including chief complaint/presentation, preliminary or discharge diagnosis, free text analysis of the entire medical record, Google flu trends, calls to teletriage and help lines, ambulance dispatch calls, case reports of H1N1 in the media, markers of ED crowding, admission and Left Without Being Seen rates. Syndromes used to capture influenza rates were nearly always related to ILI (i.e. fever +/- a respiratory or constitutional complaint), however, other syndromes used for surveillance included fever alone, "respiratory complaint" and seizure. Two very large surveillance networks, the North American DiSTRIBuTE network and the European Triple S system have collected large-scale Emergency Department-based influenza and ILI syndromic surveillance data. Syndromic surveillance for influenza and ILI from the Emergency Department is becoming more prevalent as a measure of yearly influenza outbreaks.

Assessment of the utility of routinely collected cattle census and disposal data for syndromic surveillance

Census and disposal data provide a multipurpose source of information on cattle mortality. The retrospective analyses we conducted on the data gathered in the National Cattle Register produced relevant information for describing and modelling the cattle mortality baseline and evaluating the impact of the 2007-2008 Blue Tongue epidemic on the French cattle population. This work was conducted retrospectively but showed that monitoring cattle mortality near real time could help detecting unexpected events. We are thus currently working on a timely and automated system to monitor cadaver disposal requests received by rendering plants, thanks to a data interchange system recently implemented between the Ministry of Agriculture and the fallen stock companies. Besides technical and methodological challenges, using these data for surveillance purposes raises epidemiological questions that still need to be answered. The question remains notably as to whether an abnormal increased mortality is a sensitive and timely signal for detecting unexpected health events. It appears also very challenging to identify the most adequate surveillance scale (time, space and population) and the most adequate anomaly detection algorithms to apply when the characteristics of the signals to be detected (shape, amplitude, etc.) are not known a priori. In Human health, similar systems have not yet proven their ability to detect unexpected events earlier than classical surveillance systems currently in place, but they have already demonstrated their value for real time assessment of identified and potentially dangerous events. Combined with traditional surveillance systems, we think that monitoring routinely collected data could improve the surveillance of the animal population health. Even if not used for detection purposes, cattle mortality monitoring could be used to rapidly produce information on the impact and evolution of identified events, what would facilitate decision-making regarding management measures and improve the communication.

Validation of a Pediatric Primary Care Network in a US Metropolitan Region as a Community-Based Infectious Disease Surveillance System

This cross-sectional study used Geographic Information System methods to compare sociodemographic and clinical characteristics of children enrolled and not enrolled in a primary care network to determine the suitability of the network to estimate population-based disease rates. We validated the network surveillance system by comparing invasive pneumococcal disease rates between network and nonnetwork children using population-based surveillance data. Among the study population of 130300 children, network children were more likely to be female, Black, non-Hispanic, younger, and receive Medicaid. These differences varied across neighborhoods, however, adjusting for neighborhood characteristics did not significantly change observed differences. Rates of invasive pneumococcal disease were not significantly different between network and non-network children. Significant demographic and clinical differences existed between network and non-network children and varied over small areas. Observed population rates of an infectious disease did not significantly differ suggesting that the network can potentially provide valid disease estimates for the community population.

Increased emergency department chief complaints of fever identified the influenza (H1N1) pandemic before outpatient symptom surveillance

OBJECTIVE

To determine whether a sentinel clinic network or an emergency department (ED) was more timely in identifying the 2009 influenza A (H1N1) pandemic.

METHODS

All reasons for presenting to the adult regional medical ED were coded online by admission secretaries, without the aid of medical personnel. Increased influenza activity defined by weekly chief complaints of fever was compared with activity defined by the Israel Center for Disease Control (viral surveillance as well as a large sentinel clinic network).

RESULTS

Influenza activity during the pandemic increased in the ED 2 weeks before outpatient sentinel clinics. During the pandemic, maximal ED activity was much higher than in previous seasons. Maximal activity during the past 5 years correlated with the timeliness of the chief complaint of fever in identifying the onset of epidemics.

CONCLUSION

Chief complaint of fever in the ED can be a sensitive marker of increased influenza activity and might replace the use of sentinel clinics.

Development and evaluation of a data-adaptive alerting algorithm for univariate temporal biosurveillance data

This paper discusses further advances in making robust predictions with the Holt-Winters forecasts for a variety of syndromic time series behaviors and introduces a control-chart detection approach based on these forecasts. Using three collections of time series data, we compare biosurveillance alerting methods with quantified measures of forecast agreement, signal sensitivity, and time-to-detect. The study presents practical rules for initialization and parameterization of biosurveillance time series. Several outbreak scenarios are used for detection comparison. We derive an alerting algorithm from forecasts using Holt-Winters-generalized smoothing for prospective application to daily syndromic time series. The derived algorithm is compared with simple control-chart adaptations and to more computationally intensive regression modeling methods. The comparisons are conducted on background data from both authentic and simulated data streams. Both types of background data include time series that vary widely by both mean value and cyclic or seasonal behavior. Plausible, simulated signals are added to the background data for detection performance testing at signal strengths calculated to be neither too easy nor too hard to separate the compared methods. Results show that both the sensitivity and the timeliness of the Holt-Winters-based algorithm proved to be comparable or superior to that of the more traditional prediction methods used for syndromic surveillance.

Public Health Syndromic Surveillance Systems

In this chapter, we summarize the key local, state, national, and international syndromic surveillance systems and related ongoing research programs of interest covered in our study. This summary provides the needed background information and application contexts. It also offers a current snapshot of syndromic surveillance practice in general. Note that as our primary focus is on public health surveillance, closely-related issues such as response planning and resource allocations strategies after an event is confirmed (e.g., Carley et al., 2003) are beyond the scope of this study.

For each system surveyed, we list its main contributors and stakeholders. We also include an overall system/project description, relevant data sources, syndromes monitored, data analysis and outbreak detection methods implemented, frequency of data collection and analysis, whether a GIS component is used, and its deployment strategy and status.

Climate change and emergency medicine: impacts and opportunities

There is scientific consensus that the climate is changing, that human activity plays a major role, and that the changes will continue through this century. Expert consensus holds that significant health effects are very likely. Public health and health care systems must understand these impacts to properly pursue preparedness and prevention activities. All of medicine will very likely be affected, and certain medical specialties are likely to be more significantly burdened based on their clinical activity, ease of public access, public health roles, and energy use profiles. These specialties have been called on to consider the likely impacts on their patients and practice and to prepare their practitioners. Emergency medicine (EM), with its focus on urgent and emergent ambulatory care, role as a safety-net provider, urban concentration, and broad-based clinical mission, will very likely experience a significant rise in demand for its services over and above current annual increases. Clinically, EM will see amplification of weather-related disease patterns and shifts in disease distribution. In EM's prehospital care and disaster response activities, both emergency medical services (EMS) activity and disaster medical assistance team (DMAT) deployment activities will likely increase. EM's public health roles, including disaster preparedness, emergency department (ED)-based surveillance, and safety-net care, are likely to face increasing demands, along with pressures to improve fuel efficiency and reduce greenhouse gas emissions. Finally, EM's roles in ED and hospital management, particularly related to building and purchasing, are likely to be impacted by efforts to reduce greenhouse gas emissions and enhance energy efficiency. Climate change thus presents multiple clinical and public health challenges to EM, but also creates numerous opportunities for research, education, and leadership on an emerging health issue of global scope.

Syndromic surveillance: STL for modeling, visualizing, and monitoring disease counts

Background

Public health surveillance is the monitoring of data to detect and quantify unusual health events. Monitoring pre-diagnostic data, such as emergency department (ED) patient chief complaints, enables rapid detection of disease outbreaks. There are many sources of variation in such data; statistical methods need to accurately model them as a basis for timely and accurate disease outbreak methods.

Methods

Our new methods for modeling daily chief complaint counts are based on a seasonal-trend decomposition procedure based on loess (STL) and were developed using data from the 76 EDs of the Indiana surveillance program from 2004 to 2008. Square root counts are decomposed into inter-annual, yearly-seasonal, day-of-the-week, and random-error components. Using this decomposition method, we develop a new synoptic-scale (days to weeks) outbreak detection method and carry out a simulation study to compare detection performance to four well-known methods for nine outbreak scenarios.

Result

The components of the STL decomposition reveal insights into the variability of the Indiana ED data. Day-of-the-week components tend to peak Sunday or Monday, fall steadily to a minimum Thursday or Friday, and then rise to the peak. Yearly-seasonal components show seasonal influenza, some with bimodal peaks.

Some inter-annual components increase slightly due to increasing patient populations. A new outbreak detection method based on the decomposition modeling performs well with 90 days or more of data. Control limits were set empirically so that all methods had a specificity of 97%. STL had the largest sensitivity in all nine outbreak scenarios. The STL method also exhibited a well-behaved false positive rate when run on the data with no outbreaks injected.

Conclusion

The STL decomposition method for chief complaint counts leads to a rapid and accurate detection method for disease outbreaks, and requires only 90 days of historical data to be put into operation. The visualization tools that accompany the decomposition and outbreak methods provide much insight into patterns in the data, which is useful for surveillance operations.

Recursive least squares background prediction of univariate syndromic surveillance data

Background

Surveillance of univariate syndromic data as a means of potential indicator of developing public health conditions has been used extensively. This paper aims to improve the performance of detecting outbreaks by using a background forecasting algorithm based on the adaptive recursive least squares method combined with a novel treatment of the Day of the Week effect.

Methods

Previous work by the first author has suggested that univariate recursive least squares analysis of syndromic data can be used to characterize the background upon which a prediction and detection component of a biosurvellance system may be built. An adaptive implementation is used to deal with data non-stationarity. In this paper we develop and implement the RLS method for background estimation of univariate data. The distinctly dissimilar distribution of data for different days of the week, however, can affect filter implementations adversely, and so a novel procedure based on linear transformations of the sorted values of the daily counts is introduced. Seven-days ahead daily predicted counts are used as background estimates. A signal injection procedure is used to examine the integrated algorithm's ability to detect synthetic anomalies in real syndromic time series. We compare the method to a baseline CDC forecasting algorithm known as the W2 method.

Results

We present detection results in the form of Receiver Operating Characteristic curve values for four different injected signal to noise ratios using 16 sets of syndromic data. We find improvements in the false alarm probabilities when compared to the baseline W2 background forecasts.

Conclusion

The current paper introduces a prediction approach for city-level biosurveillance data streams such as time series of outpatient clinic visits and sales of over-the-counter remedies. This approach uses RLS filters modified by a correction for the weekly patterns often seen in these data series, and a threshold detection algorithm from the residuals of the RLS forecasts. We compare the detection performance of this algorithm to the W2 method recently implemented at CDC. The modified RLS method gives consistently better sensitivity at multiple background alert rates, and we recommend that it should be considered for routine application in bio-surveillance systems.

Comparing syndromic surveillance detection methods: EARS' versus a CUSUM-based methodology

This paper compares the performance of three detection methods, entitled C1, C2, and C3, that are implemented in the early aberration reporting system (EARS) and other syndromic surveillance systems versus the CUSUM applied to model-based prediction errors. The cumulative sum (CUSUM) performed significantly better than the EARS' methods across all of the scenarios we evaluated. These scenarios consisted of various combinations of large and small background disease incidence rates, seasonal cycles from large to small (as well as no cycle), daily effects, and various types and levels of random daily variation. This leads us to recommend replacing the C1, C2, and C3 methods in existing syndromic surveillance systems with an appropriately implemented CUSUM method.

Increased complaints of fever in the emergency room can identify influenza epidemics

BACKGROUND

In developing countries, it may be easier to use the reasons why patients come to the emergency room (ER) instead of sentinel practices to identify influenza epidemics.

METHODS

We studied the reasons why adult patients present to the ER in order to attempt to predict increased hospital activity as a result of influenza. The daily frequency of presenting symptoms during the 30 days of maximal influenza activity was compared to the other days of the study period (335 days).

RESULTS

During the influenza period, more patients presented with fever, syncope or near syncope, cough, asthma attack, and paralysis than on the days outside of this period. On 50% of the days, eight or more patients presented with fever, an 8.36 (95% CI=4.6-15.19) higher frequency than during the rest of the year. During the subsequent year, days with excess presentations by patients with a principal complaint of fever predicted increased hospital activity due to influenza with no false-positive periods.

CONCLUSIONS

We conclude that an increase in the number of patients presenting to the ER complaining of fever can identify increased hospital influenza activity.

Validation of syndromic surveillance for respiratory pathogen activity

The studied respiratory syndromes are suitable for syndromic surveillance because they reflect respiratory pathogen activity patterns

Syndromic surveillance is increasingly used to signal unusual illness events. To validate data-source selection, we retrospectively investigated the extent to which 6 respiratory syndromes (based on different medical registries) reflected respiratory pathogen activity. These syndromes showed higher levels in winter, which corresponded with higher laboratory counts of Streptococcus pneumoniae, respiratory syncytial virus, and influenza virus. Multiple linear regression models indicated that most syndrome variations (up to 86%) can be explained by counts of respiratory pathogens. Absenteeism and pharmacy syndromes might reflect nonrespiratory conditions as well. We also observed systematic syndrome elevations in the fall, which were unexplained by pathogen counts but likely reflected rhinovirus activity. Earliest syndrome elevations were observed in absenteeism data, followed by hospital data (+1 week), pharmacy/general practitioner consultations (+2 weeks), and deaths/laboratory submissions (test requests) (+3 weeks). We conclude that these syndromes can be used for respiratory syndromic surveillance, since they reflect patterns in respiratory pathogen activity.

Forecasting daily patient volumes in the emergency department

BACKGROUND

Shifts in the supply of and demand for emergency department (ED) resources make the efficient allocation of ED resources increasingly important. Forecasting is a vital activity that guides decision-making in many areas of economic, industrial, and scientific planning, but has gained little traction in the health care industry. There are few studies that explore the use of forecasting methods to predict patient volumes in the ED.

OBJECTIVES

The goals of this study are to explore and evaluate the use of several statistical forecasting methods to predict daily ED patient volumes at three diverse hospital EDs and to compare the accuracy of these methods to the accuracy of a previously proposed forecasting method.

METHODS

Daily patient arrivals at three hospital EDs were collected for the period January 1, 2005, through March 31, 2007. The authors evaluated the use of seasonal autoregressive integrated moving average, time series regression, exponential smoothing, and artificial neural network models to forecast daily patient volumes at each facility. Forecasts were made for horizons ranging from 1 to 30 days in advance. The forecast accuracy achieved by the various forecasting methods was compared to the forecast accuracy achieved when using a benchmark forecasting method already available in the emergency medicine literature.

RESULTS

All time series methods considered in this analysis provided improved in-sample model goodness of fit. However, post-sample analysis revealed that time series regression models that augment linear regression models by accounting for serial autocorrelation offered only small improvements in terms of post-sample forecast accuracy, relative to multiple linear regression models, while seasonal autoregressive integrated moving average, exponential smoothing, and artificial neural network forecasting models did not provide consistently accurate forecasts of daily ED volumes.

CONCLUSIONS

This study confirms the widely held belief that daily demand for ED services is characterized by seasonal and weekly patterns. The authors compared several time series forecasting methods to a benchmark multiple linear regression model. The results suggest that the existing methodology proposed in the literature, multiple linear regression based on calendar variables, is a reasonable approach to forecasting daily patient volumes in the ED. However, the authors conclude that regression-based models that incorporate calendar variables, account for site-specific special-day effects, and allow for residual autocorrelation provide a more appropriate, informative, and consistently accurate approach to forecasting daily ED patient volumes.

Recombinant temporal aberration detection algorithms for enhanced biosurveillance

OBJECTIVE

Broadly, this research aims to improve the outbreak detection performance and, therefore, the cost effectiveness of automated syndromic surveillance systems by building novel, recombinant temporal aberration detection algorithms from components of previously developed detectors.

METHODS

This study decomposes existing temporal aberration detection algorithms into two sequential stages and investigates the individual impact of each stage on outbreak detection performance. The data forecasting stage (Stage 1) generates predictions of time series values a certain number of time steps in the future based on historical data. The anomaly measure stage (Stage 2) compares features of this prediction to corresponding features of the actual time series to compute a statistical anomaly measure. A Monte Carlo simulation procedure is then used to examine the recombinant algorithms' ability to detect synthetic aberrations injected into authentic syndromic time series.

RESULTS

New methods obtained with procedural components of published, sometimes widely used, algorithms were compared to the known methods using authentic datasets with plausible stochastic injected signals. Performance improvements were found for some of the recombinant methods, and these improvements were consistent over a range of data types, outbreak types, and outbreak sizes. For gradual outbreaks, the WEWD MovAvg7+WEWD Z-Score recombinant algorithm performed best; for sudden outbreaks, the HW+WEWD Z-Score performed best.

CONCLUSION

This decomposition was found not only to yield valuable insight into the effects of the aberration detection algorithms but also to produce novel combinations of data forecasters and anomaly measures with enhanced detection performance.

Online detection and quantification of epidemics

BACKGROUND

Time series data are increasingly available in health care, especially for the purpose of disease surveillance. The analysis of such data has long used periodic regression models to detect outbreaks and estimate epidemic burdens. However, implementation of the method may be difficult due to lack of statistical expertise. No dedicated tool is available to perform and guide analyses.

RESULTS

We developed an online computer application allowing analysis of epidemiologic time series. The system is available online at http://www.u707.jussieu.fr/periodic_regression/. The data is assumed to consist of a periodic baseline level and irregularly occurring epidemics. The program allows estimating the periodic baseline level and associated upper forecast limit. The latter defines a threshold for epidemic detection. The burden of an epidemic is defined as the cumulated signal in excess of the baseline estimate. The user is guided through the necessary choices for analysis. We illustrate the usage of the online epidemic analysis tool with two examples: the retrospective detection and quantification of excess pneumonia and influenza (P&I) mortality, and the prospective surveillance of gastrointestinal disease (diarrhoea).

CONCLUSION

The online application allows easy detection of special events in an epidemiologic time series and quantification of excess mortality/morbidity as a change from baseline. It should be a valuable tool for field and public health practitioners.

Modeling and detection of respiratory-related outbreak signatures

Background

Time series methods are commonly used to detect disease outbreak signatures (e.g., signals due to influenza outbreaks and anthrax attacks) from varying respiratory-related diagnostic or syndromic data sources. Typically this involves two components: (i) Using time series methods to model the baseline background distribution (the time series process that is assumed to contain no outbreak signatures), (ii) Detecting outbreak signatures using filter-based time series methods.

Methods

We consider time series models for chest radiograph data obtained from Midwest children's emergency departments. These models incorporate available covariate information such as patient visit counts and smoothed ambient temperature series, as well as time series dependencies on daily and weekly seasonal scales. Respiratory-related outbreak signature detection is based on filtering the one-step-ahead prediction errors obtained from the time series models for the respiratory-complaint background.

Results

Using simulation experiments based on a stochastic model for an anthrax attack, we illustrate the effect of the choice of filter and the statistical models upon radiograph-attributed outbreak signature detection.

Conclusion

We demonstrate the importance of using seasonal autoregressive integrated average time series models (SARIMA) with covariates in the modeling of respiratory-related time series data. We find some homogeneity in the time series models for the respiratory-complaint backgrounds across the Midwest emergency departments studied. Our simulations show that the balance between specificity, sensitivity, and timeliness to detect an outbreak signature differs by the emergency department and the choice of filter. The linear and exponential filters provide a good balance.

Automated real time constant-specificity surveillance for disease outbreaks

Background

For real time surveillance, detection of abnormal disease patterns is based on a difference between patterns observed, and those predicted by models of historical data. The usefulness of outbreak detection strategies depends on their specificity; the false alarm rate affects the interpretation of alarms.

Results

We evaluate the specificity of five traditional models: autoregressive, Serfling, trimmed seasonal, wavelet-based, and generalized linear. We apply each to 12 years of emergency department visits for respiratory infection syndromes at a pediatric hospital, finding that the specificity of the five models was almost always a non-constant function of the day of the week, month, and year of the study (p < 0.05). We develop an outbreak detection method, called the expectation-variance model, based on generalized additive modeling to achieve a constant specificity by accounting for not only the expected number of visits, but also the variance of the number of visits. The expectation-variance model achieves constant specificity on all three time scales, as well as earlier detection and improved sensitivity compared to traditional methods in most circumstances.

Conclusion

Modeling the variance of visit patterns enables real-time detection with known, constant specificity at all times. With constant specificity, public health practitioners can better interpret the alarms and better evaluate the cost-effectiveness of surveillance systems.

Automated time series forecasting for biosurveillance

For robust detection performance, traditional control chart monitoring for biosurveillance is based on input data free of trends, day-of-week effects, and other systematic behaviour. Time series forecasting methods may be used to remove this behaviour by subtracting forecasts from observations to form residuals for algorithmic input. We describe three forecast methods and compare their predictive accuracy on each of 16 authentic syndromic data streams. The methods are (1) a non-adaptive regression model using a long historical baseline, (2) an adaptive regression model with a shorter, sliding baseline, and (3) the Holt-Winters method for generalized exponential smoothing. Criteria for comparing the forecasts were the root-mean-square error, the median absolute per cent error (MedAPE), and the median absolute deviation. The median-based criteria showed best overall performance for the Holt-Winters method. The MedAPE measures over the 16 test series averaged 16.5, 11.6, and 9.7 for the non-adaptive regression, adaptive regression, and Holt-Winters methods, respectively. The non-adaptive regression forecasts were degraded by changes in the data behaviour in the fixed baseline period used to compute model coefficients. The mean-based criterion was less conclusive because of the effects of poor forecasts on a small number of calendar holidays. The Holt-Winters method was also most effective at removing serial autocorrelation, with most 1-day-lag autocorrelation coefficients below 0.15. The forecast methods were compared without tuning them to the behaviour of individual series. We achieved improved predictions with such tuning of the Holt-Winters method, but practical use of such improvements for routine surveillance will require reliable data classification methods.

Computer-aided diagnosis with potential application to rapid detection of disease outbreaks

Our objectives are to quickly interpret symptoms of emergency patients to identify likely syndromes and to improve population-wide disease outbreak detection. We constructed a database of 248 syndromes, each syndrome having an estimated probability of producing any of 85 symptoms, with some two-way, three-way, and five-way probabilities reflecting correlations among symptoms. Using these multi-way probabilities in conjunction with an iterative proportional fitting algorithm allows estimation of full conditional probabilities. Combining these conditional probabilities with misdiagnosis error rates and incidence rates via Bayes theorem, the probability of each syndrome is estimated. We tested a prototype of computer-aided differential diagnosis (CADDY) on simulated data and on more than 100 real cases, including West Nile Virus, Q fever, SARS, anthrax, plague, tularaemia and toxic shock cases. We conclude that: (1) it is important to determine whether the unrecorded positive status of a symptom means that the status is negative or that the status is unknown; (2) inclusion of misdiagnosis error rates produces more realistic results; (3) the naive Bayes classifier, which assumes all symptoms behave independently, is slightly outperformed by CADDY, which includes available multi-symptom information on correlations; as more information regarding symptom correlations becomes available, the advantage of CADDY over the naive Bayes classifier should increase; (4) overlooking low-probability, high-consequence events is less likely if the standard output summary is augmented with a list of rare syndromes that are consistent with observed symptoms, and (5) accumulating patient-level probabilities across a larger population can aid in biosurveillance for disease outbreaks.

Accounting for seasonal patterns in syndromic surveillance data for outbreak detection

Background

Syndromic surveillance (SS) can potentially contribute to outbreak detection capability by providing timely, novel data sources. One SS challenge is that some syndrome counts vary with season in a manner that is not identical from year to year.

Our goal is to evaluate the impact of inconsistent seasonal effects on performance assessments (false and true positive rates) in the context of detecting anomalous counts in data that exhibit seasonal variation.

Methods

To evaluate the impact of inconsistent seasonal effects, we injected synthetic outbreaks into real data and into data simulated from each of two models fit to the same real data. Using real respiratory syndrome counts collected in an emergency department from 2/1/94–5/31/03, we varied the length of training data from one to eight years, applied a sequential test to the forecast errors arising from each of eight forecasting methods, and evaluated their detection probabilities (DP) on the basis of 1000 injected synthetic outbreaks. We did the same for each of two corresponding simulated data sets. The less realistic, nonhierarchical model's simulated data set assumed that "one season fits all," meaning that each year's seasonal peak has the same onset, duration, and magnitude. The more realistic simulated data set used a hierarchical model to capture violation of the "one season fits all" assumption.

Results

This experiment demonstrated optimistic bias in DP estimates for some of the methods when data simulated from the nonhierarchical model was used for DP estimation, thus suggesting that at least for some real data sets and methods, it is not adequate to assume that "one season fits all."

Conclusion

For the data we analyze, the "one season fits all " assumption is violated, and DP performance claims based on simulated data that assume "one season fits all," for the forecast methods considered, except for moving average methods, tend to be optimistic. Moving average methods based on relatively short amounts of training data are competitive on all three data sets, but are particularly competitive on the real data and on data from the hierarchical model, which are the two data sets that violate the "one season fits all" assumption.

Real time spatial cluster detection using interpoint distances among precise patient locations

Background

Public health departments in the United States are beginning to gain timely access to health data, often as soon as one day after a visit to a health care facility. Consequently, new approaches to outbreak surveillance are being developed. When cases cluster geographically, an analysis of their spatial distribution can facilitate outbreak detection. Our method focuses on detecting perturbations in the distribution of pair-wise distances among all patients in a geographical region. Barring outbreaks, this distribution can be quite stable over time. We sought to exemplify the method by measuring its cluster detection performance, and to determine factors affecting sensitivity to spatial clustering among patients presenting to hospital emergency departments with respiratory syndromes.

Methods

The approach was to (1) define a baseline spatial distribution of home addresses for a population of patients visiting an emergency department with respiratory syndromes using historical data; (2) develop a controlled feature set simulation by inserting simulated outbreak data with varied parameters into authentic background noise, thereby creating semisynthetic data; (3) compare the observed with the expected spatial distribution; (4) establish the relative value of different alarm strategies so as to maximize sensitivity for the detection of clustering; and (5) measure factors which have an impact on sensitivity.

Results

Overall sensitivity to detect spatial clustering was 62%. This contrasts with an overall alarm rate of less than 5% for the same number of extra visits when the extra visits were not characterized by geographic clustering. Clusters that produced the least number of alarms were those that were small in size (10 extra visits in a week, where visits per week ranged from 120 to 472), diffusely distributed over an area with a 3 km radius, and located close to the hospital (5 km) in a region most densely populated with patients to this hospital. Near perfect alarm rates were found for clusters that varied on the opposite extremes of these parameters (40 extra visits, within a 250 meter radius, 50 km from the hospital).

Conclusion

Measuring perturbations in the interpoint distance distribution is a sensitive method for detecting spatial clustering. When cases are clustered geographically, there is clearly power to detect clustering when the spatial distribution is represented by the M statistic, even when clusters are small in size. By varying independent parameters of simulated outbreaks, we have demonstrated empirically the limits of detection of different types of outbreaks.