in silico surveillance: evaluating outbreak detection with simulation models

Fast and near-optimal monitoring for healthcare acquired infection outbreaks

According to the Centers for Disease Control and Prevention (CDC), one in twenty five hospital patients are infected with at least one healthcare acquired infection (HAI) on any given day. Early detection of possible HAI outbreaks help practitioners implement countermeasures before the infection spreads extensively. Here, we develop an efficient data and model driven method to detect outbreaks with high accuracy. We leverage mechanistic modeling of C. difficile infection, a major HAI disease, to simulate its spread in a hospital wing and design efficient near-optimal algorithms to select people and locations to monitor using an optimization formulation. Results show that our strategy detects up to 95% of "future" C. difficile outbreaks. We design our method by incorporating specific hospital practices (like swabbing for infections) as well. As a result, our method outperforms state-of-the-art algorithms for outbreak detection. Finally, a qualitative study of our result shows that the people and locations we select to monitor as sensors are intuitive and meaningful.

EnABLe: An agent-based model to understand Listeria dynamics in food processing facilities

Detection of pathogens in food processing facilities by routine environmental monitoring (EM) is essential to reduce the risk of foodborne illness but is complicated by the complexity of equipment and environment surfaces. To optimize design of EM programs, we developed EnABLe ("Environmental monitoring with an Agent-Based Model of Listeria"), a detailed and customizable agent-based simulation of a built environment. EnABLe is presented here in a model system, tracing Listeria spp. (LS) (an indicator for conditions that allow the presence of the foodborne pathogen Listeria monocytogenes) on equipment and environment surfaces in a cold-smoked salmon facility. EnABLe was parameterized by existing literature and expert elicitation and validated with historical data. Simulations revealed different contamination dynamics and risks among equipment surfaces in terms of the presence, level and persistence of LS. Grouping of surfaces by their LS contamination dynamics identified connectivity and sanitary design as predictors of contamination, indicating that these features should be considered in the design of EM programs to detect LS. The EnABLe modeling approach is particularly timely for the frozen food industry, seeking science-based recommendations for EM, and may also be relevant to other complex environments where pathogen contamination presents risks for direct or indirect human exposure.

Building test data from real outbreaks for evaluating detection algorithms

Benchmarking surveillance systems requires realistic simulations of disease outbreaks. However, obtaining these data in sufficient quantity, with a realistic shape and covering a sufficient range of agents, size and duration, is known to be very difficult. The dataset of outbreak signals generated should reflect the likely distribution of authentic situations faced by the surveillance system, including very unlikely outbreak signals. We propose and evaluate a new approach based on the use of historical outbreak data to simulate tailored outbreak signals. The method relies on a homothetic transformation of the historical distribution followed by resampling processes (Binomial, Inverse Transform Sampling Method-ITSM, Metropolis-Hasting Random Walk, Metropolis-Hasting Independent, Gibbs Sampler, Hybrid Gibbs Sampler). We carried out an analysis to identify the most important input parameters for simulation quality and to evaluate performance for each of the resampling algorithms. Our analysis confirms the influence of the type of algorithm used and simulation parameters (i.e. days, number of cases, outbreak shape, overall scale factor) on the results. We show that, regardless of the outbreaks, algorithms and metrics chosen for the evaluation, simulation quality decreased with the increase in the number of days simulated and increased with the number of cases simulated. Simulating outbreaks with fewer cases than days of duration (i.e. overall scale factor less than 1) resulted in an important loss of information during the simulation. We found that Gibbs sampling with a shrinkage procedure provides a good balance between accuracy and data dependency. If dependency is of little importance, binomial and ITSM methods are accurate. Given the constraint of keeping the simulation within a range of plausible epidemiological curves faced by the surveillance system, our study confirms that our approach can be used to generate a large spectrum of outbreak signals.

Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation

Surveillance is critical for improving population health. Public health surveillance systems generate information that drives action, and the data must be of sufficient quality and with a resolution and timeliness that matches objectives. In the context of scientific advances in public health surveillance, changing health care and public health environments, and rapidly evolving technologies, the aim of this article is to review public health surveillance systems. We consider their current use to increase the efficiency and effectiveness of the public health system, the role of system stakeholders, the analysis and interpretation of surveillance data, approaches to system monitoring and evaluation, and opportunities for future advances in terms of increased scientific rigor, outcomes-focused research, and health informatics.

Simulation training improves clinical knowledge of major haemorrhage management in foundation year doctors

OBJECTIVES

(i) To develop a major haemorrhage simulation training programme. (ii) To design an assessment tool to measure the effectiveness of this programme. (iii) To use simulation training to create more effective protocols.

BACKGROUND

Major haemorrhage is a time-critical medical emergency that can be faced by every Foundation Year (FY) doctor. Standard methods of teaching provide limited opportunity for junior doctors to improve their knowledge and practical skills for major haemorrhage situations. Simulation is increasingly used in medical training but has not been used as a means both to facilitate learning and refine hospital major haemorrhage policy.

METHODS

The effect of major haemorrhage simulation on attendees' learning was compared to a comparator group not exposed to simulation. Questionnaire pre-simulation and 3 months post-simulation training assessed knowledge of the Trust Major Haemorrhage Protocol (MHP).

RESULTS

Sixteen FY1 doctors attended simulation training. The comparator group included 47 FY1 doctors. No significant difference was found between simulation and comparator groups on baseline questionnaire scores. The simulation group showed significant improvement (total score, mean standard deviation (SD) pre-test 27.15 (4.02), post-test 39.13 (3.91), p < 0.001). The comparator group showed no significant change (total score, pre-test 25.06 (5.70), post-test 23.54 (6.85), p = 0.33) (19 lost to follow up). The study resulted in the production of a new MHP.

CONCLUSION

This study has demonstrated that simulation training improved doctors' knowledge in major haemorrhage management and that the experience of observing the simulation training allowed senior staff to undertake analysis and improvement of an existing MHP.

Comparing the accuracy of syndrome surveillance systems in detecting influenza-like illness: GUARDIAN vs. RODS vs. electronic medical record reports

BACKGROUND

A highly sensitive real-time syndrome surveillance system is critical to detect, monitor, and control infectious disease outbreaks, such as influenza. Direct comparisons of diagnostic accuracy of various surveillance systems are scarce.

OBJECTIVE

To statistically compare sensitivity and specificity of multiple proprietary and open source syndrome surveillance systems to detect influenza-like illness (ILI).

METHODS

A retrospective, cross-sectional study was conducted utilizing data from 1122 patients seen during November 1–7, 2009 in the emergency department of a single urban academic medical center. The study compared the Geographic Utilization of Artificial Intelligence in Real-time for Disease Identification and Alert Notification (GUARDIAN) system to the Complaint Coder (CoCo) of the Real-time Outbreak Detection System (RODS), the Symptom Coder (SyCo) of RODS, and to a standardized report generated via a proprietary electronic medical record (EMR) system. Sensitivity, specificity, and accuracy of each classifier's ability to identify ILI cases were calculated and compared to a manual review by a board-certified emergency physician. Chi-square and McNemar's tests were used to evaluate the statistical difference between the various surveillance systems.ResultsThe performance of GUARDIAN in detecting ILI in terms of sensitivity, specificity, and accuracy, as compared to a physician chart review, was 95.5%, 97.6%, and 97.1%, respectively. The EMR-generated reports were the next best system at identifying disease activity with a sensitivity, specificity, and accuracy of 36.7%, 99.3%, and 83.2%, respectively. RODS (CoCo and SyCo) had similar sensitivity (35.3%) but slightly different specificity (CoCo = 98.9%; SyCo = 99.3%). The GUARDIAN surveillance system with its multiple data sources performed significantly better compared to CoCo (χ2 = 130.6, p < 0.05), SyCo (χ2 = 125.2, p < 0.05), and EMR-based reports (χ2 = 121.3, p < 0.05). In addition, similar significant improvements in the accuracy (>12%) and sensitivity (>47%) were observed for GUARDIAN with only chief complaint data as compared to RODS (CoCo and SyCo) and EMR-based reports.

CONCLUSION

In our study population, the GUARDIAN surveillance system, with its ability to utilize multiple data sources from patient encounters and real-time automaticity, demonstrated a more robust performance when compared to standard EMR-based reports and the RODS systems in detecting ILI. More large-scale studies are needed to validate the study findings, and to compare the performance of GUARDIAN in detecting other infectious diseases.