A methodological framework for the evaluation of syndromic surveillance systems: a case study of England

Utilizing Wearable Device Data for Syndromic Surveillance: A Fever Detection Approach

Commercially available wearable devices (wearables) show promise for continuous physiological monitoring. Previous works have demonstrated that wearables can be used to detect the onset of acute infectious diseases, particularly those characterized by fever. We aimed to evaluate whether these devices could be used for the more general task of syndromic surveillance. We obtained wearable device data (Oura Ring) from 63,153 participants. We constructed a dataset using participants' wearable device data and participants' responses to daily online questionnaires. We included days from the participants if they (1) completed the questionnaire, (2) reported not experiencing fever and reported a self-collected body temperature below 38 °C (negative class), or reported experiencing fever and reported a self-collected body temperature at or above 38 °C (positive class), and (3) wore the wearable device the nights before and after that day. We used wearable device data (i.e., skin temperature, heart rate, and sleep) from the nights before and after participants' fever day to train a tree-based classifier to detect self-reported fevers. We evaluated the performance of our model using a five-fold cross-validation scheme. Sixteen thousand, seven hundred, and ninety-four participants provided at least one valid ground truth day; there were a total of 724 fever days (positive class examples) from 463 participants and 342,430 non-fever days (negative class examples) from 16,687 participants. Our model exhibited an area under the receiver operating characteristic curve (AUROC) of 0.85 and an average precision (AP) of 0.25. At a sensitivity of 0.50, our calibrated model had a false positive rate of 0.8%. Our results suggest that it might be possible to leverage data from these devices at a public health level for live fever surveillance. Implementing these models could increase our ability to detect disease prevalence and spread in real-time during infectious disease outbreaks.

Syndromic surveillance of respiratory infections during protracted conflict: experiences from northern Syria 2016-2021

OBJECTIVE

Northern Syria faces a large burden of influenza-like illness (ILI) and severe acute respiratory illness (SARI). This study aimed to investigate the trends of Early Warning and Response Network (EWARN) reported ILI and SARI in northern Syria between 2016 and 2021 and the potential impact of SARS-CoV-2.

METHODS

We extracted weekly EWARN data on ILI/ SARI and aggregated cases and consultations into 4-week intervals to calculate case positivity. We conducted a seasonal-trend decomposition to assess case trends in the presence of seasonal fluctuations.

RESULTS

It was observed that 4-week aggregates of ILI cases (n = 5,942,012), SARI cases (n = 114,939), ILI case positivity, and SARI case positivity exhibited seasonal fluctuations with peaks in the winter months. ILI and SARI cases in individuals aged ≥5 years surpassed those in individuals aged <5 years in late 2019. ILI cases clustered primarily in Aleppo and Idlib, whereas SARI cases clustered in Aleppo, Idlib, Deir Ezzor, and Hassakeh. SARI cases increased sharply in 2021, corresponding with a severe SARS-CoV-2 wave, compared with the steady increase in ILI cases over time.

CONCLUSION

Respiratory infections cause widespread morbidity and mortality throughout northern Syria, particularly with the emergence of SARS-CoV-2. Strengthened surveillance and access to testing and treatment are critical to manage outbreaks among conflict-affected populations.

Syndromic Surveillance Systems for Mass Gatherings: A Scoping Review

Syndromic surveillance involves the near-real-time collection of data from a potential multitude of sources to detect outbreaks of disease or adverse health events earlier than traditional forms of public health surveillance. The purpose of the present study is to elucidate the role of syndromic surveillance during mass gathering scenarios. In the present review, the use of syndromic surveillance for mass gathering scenarios is described, including characteristics such as methodologies of data collection and analysis, degree of preparation and collaboration, and the degree to which prior surveillance infrastructure is utilized. Nineteen publications were included for data extraction. The most common data source for the included syndromic surveillance systems was emergency departments, with first aid stations and event-based clinics also present. Data were often collected using custom reporting forms. While syndromic surveillance can potentially serve as a method of informing public health policy regarding specific mass gatherings based on the profile of syndromes ascertained, the present review does not indicate that this form of surveillance is a reliable method of detecting potentially critical public health events during mass gathering scenarios.

Understanding Health Communication Through Google Trends and News Coverage for COVID-19: A Multinational Study in Eight Countries

Background

Due to the COVID-19 pandemic, health information related to COVID-19 has spread across news media worldwide. Google is among the most used internet search engines, and the Google Trends tool can reflect how the public seeks COVID-19–related health information during the pandemic.

Objective

The aim of this study was to understand health communication through Google Trends and news coverage and to explore their relationship with prevention and control of COVID-19 at the early epidemic stage.

Methods

To achieve the study objectives, we analyzed the public’s information-seeking behaviors on Google and news media coverage on COVID-19. We collected data on COVID-19 news coverage and Google search queries from eight countries (ie, the United States, the United Kingdom, Canada, Singapore, Ireland, Australia, South Africa, and New Zealand) between January 1 and April 29, 2020. We depicted the characteristics of the COVID-19 news coverage trends over time, as well as the search query trends for the topics of COVID-19–related “diseases,” “treatments and medical resources,” “symptoms and signs,” and “public measures.” The search query trends provided the relative search volume (RSV) as an indicator to represent the popularity of a specific search term in a specific geographic area over time. Also, time-lag correlation analysis was used to further explore the relationship between search terms trends and the number of new daily cases, as well as the relationship between search terms trends and news coverage.

Results

Across all search trends in eight countries, almost all search peaks appeared between March and April 2020, and declined in April 2020. Regarding COVID-19–related “diseases,” in most countries, the RSV of the term “coronavirus” increased earlier than that of “covid-19”; however, around April 2020, the search volume of the term “covid-19” surpassed that of “coronavirus.” Regarding the topic “treatments and medical resources,” the most and least searched terms were “mask” and “ventilator,” respectively. Regarding the topic “symptoms and signs,” “fever” and “cough” were the most searched terms. The RSV for the term “lockdown” was significantly higher than that for “social distancing” under the topic “public health measures.” In addition, when combining search trends with news coverage, there were three main patterns: (1) the pattern for Singapore, (2) the pattern for the United States, and (3) the pattern for the other countries. In the time-lag correlation analysis between the RSV for the topic “treatments and medical resources” and the number of new daily cases, the RSV for all countries except Singapore was positively correlated with new daily cases, with a maximum correlation of 0.8 for the United States. In addition, in the time-lag correlation analysis between the overall RSV for the topic “diseases” and the number of daily news items, the overall RSV was positively correlated with the number of daily news items, the maximum correlation coefficient was more than 0.8, and the search behavior occurred 0 to 17 days earlier than the news coverage.

Conclusions

Our findings revealed public interest in masks, disease control, and public measures, and revealed the potential value of Google Trends in the face of the emergence of new infectious diseases. Also, Google Trends combined with news media can achieve more efficient health communication. Therefore, both news media and Google Trends can contribute to the early prevention and control of epidemics.

The effectiveness of syndromic surveillance for the early detection of waterborne outbreaks: a systematic review

BACKGROUND

Waterborne outbreaks are still a risk in high-income countries, and their early detection is crucial to limit their societal consequences. Although syndromic surveillance is widely used for the purpose of detecting outbreaks days earlier than traditional surveillance systems, evidence of the effectiveness of such systems is lacking. Thus, our objective was to conduct a systematic review of the effectiveness of syndromic surveillance to detect waterborne outbreaks.

METHOD

We searched the Cochrane Library, Medline/PubMed, EMBASE, Scopus, and Web of Science for relevant published articles using a combination of the keywords 'drinking water', 'surveillance', and 'waterborne disease' for the period of 1990 to 2018. The references lists of the identified articles for full-text record assessment were screened, and searches in Google Scholar using the same key words were conducted. We assessed the risk of bias in the included articles using the ROBINS-I tool and PRECEPT for the cumulative body of evidence.

RESULTS

From the 1959 articles identified, we reviewed 52 articles, of which 18 met the eligibility criteria. Twelve were descriptive/analytical studies, whereas six were simulation studies. There is no clear evidence for syndromic surveillance in terms of the ability to detect waterborne outbreaks (low sensitivity and high specificity). However, one simulation study implied that multiple sources of signals combined with spatial information may increase the timeliness in detecting a waterborne outbreak and reduce false alarms.

CONCLUSION

This review demonstrates that there is no conclusive evidence on the effectiveness of syndromic surveillance for the detection of waterborne outbreaks, thus suggesting the need to focus on primary prevention measures to reduce the risk of waterborne outbreaks. Future studies should investigate methods for combining health and environmental data with an assessment of needed financial and human resources for implementing such surveillance systems. In addition, a more critical thematic narrative synthesis on the most promising sources of data, and an assessment of the basis for arguments that joint analysis of different data or dimensions of data (e.g. spatial and temporal) might perform better, should be carried out.

TRIAL REGISTRATION

PROSPERO: International prospective register of systematic reviews. 2019. CRD42019122332 .

Probabilistic seasonal dengue forecasting in Vietnam: A modelling study using superensembles

BACKGROUND

With enough advanced notice, dengue outbreaks can be mitigated. As a climate-sensitive disease, environmental conditions and past patterns of dengue can be used to make predictions about future outbreak risk. These predictions improve public health planning and decision-making to ultimately reduce the burden of disease. Past approaches to dengue forecasting have used seasonal climate forecasts, but the predictive ability of a system using different lead times in a year-round prediction system has been seldom explored. Moreover, the transition from theoretical to operational systems integrated with disease control activities is rare.

METHODS AND FINDINGS

We introduce an operational seasonal dengue forecasting system for Vietnam where Earth observations, seasonal climate forecasts, and lagged dengue cases are used to drive a superensemble of probabilistic dengue models to predict dengue risk up to 6 months ahead. Bayesian spatiotemporal models were fit to 19 years (2002-2020) of dengue data at the province level across Vietnam. A superensemble of these models then makes probabilistic predictions of dengue incidence at various future time points aligned with key Vietnamese decision and planning deadlines. We demonstrate that the superensemble generates more accurate predictions of dengue incidence than the individual models it incorporates across a suite of time horizons and transmission settings. Using historical data, the superensemble made slightly more accurate predictions (continuous rank probability score [CRPS] = 66.8, 95% CI 60.6-148.0) than a baseline model which forecasts the same incidence rate every month (CRPS = 79.4, 95% CI 78.5-80.5) at lead times of 1 to 3 months, albeit with larger uncertainty. The outbreak detection capability of the superensemble was considerably larger (69%) than that of the baseline model (54.5%). Predictions were most accurate in southern Vietnam, an area that experiences semi-regular seasonal dengue transmission. The system also demonstrated added value across multiple areas compared to previous practice of not using a forecast. We use the system to make a prospective prediction for dengue incidence in Vietnam for the period May to October 2020. Prospective predictions made with the superensemble were slightly more accurate (CRPS = 110, 95% CI 102-575) than those made with the baseline model (CRPS = 125, 95% CI 120-168) but had larger uncertainty. Finally, we propose a framework for the evaluation of probabilistic predictions. Despite the demonstrated value of our forecasting system, the approach is limited by the consistency of the dengue case data, as well as the lack of publicly available, continuous, and long-term data sets on mosquito control efforts and serotype-specific case data.

CONCLUSIONS

This study shows that by combining detailed Earth observation data, seasonal climate forecasts, and state-of-the-art models, dengue outbreaks can be predicted across a broad range of settings, with enough lead time to meaningfully inform dengue control. While our system omits some important variables not currently available at a subnational scale, the majority of past outbreaks could be predicted up to 3 months ahead. Over the next 2 years, the system will be prospectively evaluated and, if successful, potentially extended to other areas and other climate-sensitive disease systems.

The comparative performance of wavelet-based outbreak detector, exponential weighted moving average, and Poisson regression-based methods in detection of pertussis outbreaks in Iranian infants: A simulation-based study

BACKGROUND

Early detection of outbreaks of transmissible diseases is essential for public health. This study aimed to determine the performance of the wavelet-based outbreak detection method (WOD) in detecting outbreaks and to compare its performance with the Poisson regression-based model and exponentially weighted moving average (EWMA) using data of simulated pertussis outbreaks in Iran.

METHOD

The data on suspected cases of pertussis from 25th February 2012 to 23rd March 2018 in Iran was used. The performance of the WOD (Daubechies 10 [db10] and Haar wavelets), Poisson regression-based method, and EWMA Compared in terms of timeliness and detection of outbreak days using the simulation of different outbreaks. In the current study, two simulations were used, one based on retrospectively collected data (literature-based) on pertussis cases and another one on a synthetic dataset created by the researchers. The sensitivity, specificity, false alarm, and false-negative rate, positive and negative likelihood ratios, under receiver operating characteristics areas, and median timeliness were used to assess the performance of the methods.

RESULTS

In a literature-based outbreak simulation, the highest and lowest sensitivity, false negative in the detection of injected outbreaks were seen in db10, with sensitivity 0.59 (0.56-0.62), and Haar wavelets with 0.57 (0.54-0.60). In the researcher simulated data, the EWMA (K = 0.5) with sensitivity 0.92 (0.90-0.94) had the best performance. About timeliness, the WOD methods showed the best performance in the early warning of the outbreak in both simulation approaches.

CONCLUSION

Performance of the WOD in the early alarming outbreaks was appropriate. However, this method would be best used along with other methods of public health surveillance.

The Impact of Mass Gatherings on Emergency Department Patient Presentations with Communicable Diseases Related to Syndromic Indicators: An Integrative Review

BACKGROUND

Mass-gathering events (MGEs) are commonly associated with a higher than average rate of morbidity. Spectators, workers, and the substantial number of MGE attendees can increase the spread of communicable diseases. During an MGE, emergency departments (EDs) play an important role in offering health care services to both residents of the local community and event attendees. Syndromic indicators (SIs) are widely used in an ED surveillance system for early detection of communicable diseases.

AIM

This literature review aimed to develop an understanding of the effect of MGEs on ED patient presentations with communicable diseases and their corresponding SIs.

METHOD

An integrative literature review methodology was used. Online databases were searched to retrieve relevant academic articles that focused on MGEs, EDs, and SIs. Inclusion/exclusion criteria were applied to screen articles. The Standard Quality Assessment Criteria for Evaluating Primary Research (QualSyst) assessment tool was used to assess the quality of included papers.

RESULTS

Eleven papers were included in this review; all discussed the impact of an MGE on patient presentations with communicable diseases at EDs/hospitals. Most included studies used the raw number of patients who presented or were admitted to EDs/hospitals to determine impact. Further, the majority of studies focused on either respiratory infections (n = 4) or gastrointestinal infections (n = 2); two articles reported on both. Eight articles mentioned SIs; however, such information was limited. The quality of evidence (using QualSyst) ranged from 50% to 90%.

CONCLUSIONS

Limited research exists on the impact of MGEs on ED presentations with communicable diseases and related SIs. Recommendations for future MGE studies include assessing differences in ED presentations with communicable diseases regarding demographics, clinical characteristics, and outcomes before, during, and after the event. This would benefit health care workers and researchers by offering more comprehensive knowledge for application into practice.

Multivariate syndromic surveillance for cattle diseases: Epidemic simulation and algorithm performance evaluation

Multivariate Syndromic Surveillance (SyS) systems that simultaneously assess and combine information from different data sources are especially useful for strengthening surveillance systems for early detection of infectious disease epidemics. Despite the strong motivation for implementing multivariate SyS and there being numerous methods reported, the number of operational multivariate SyS systems in veterinary medicine is still very small. One possible reason is that assessing the performance of such surveillance systems remains challenging because field epidemic data are often unavailable. The objective of this study is to demonstrate a practical multivariate event detection method (directionally sensitive multivariate control charts) that can be easily applied in livestock disease SyS, using syndrome time series data from the Swiss cattle population as an example. We present a standardized method for simulating multivariate epidemics of different diseases using four diseases as examples: Bovine Virus Diarrhea (BVD), Infectious Bovine Rhinotracheitis (IBR), Bluetongue virus (BTV) and Schmallenberg virus (SV). Two directional multivariate control chart algorithms, Multivariate Exponentially Weighted Moving Average (MEWMA) and Multivariate Cumulative Sum (MCUSUM) were compared. The two algorithms were evaluated using 12 syndrome time series extracted from two Swiss national databases. The two algorithms were able to detect all simulated epidemics around 4.5 months after the start of the epidemic, with a specificity of 95%. However, the results varied depending on the algorithm and the disease. The MEWMA algorithm always detected epidemics earlier than the MCUSUM, and epidemics of IBR and SV were detected earlier than epidemics of BVD and BTV. Our results show that the two directional multivariate control charts are promising methods for combining information from multiple time series for early detection of subtle changes in time series from a population without producing an unreasonable amount of false alarms. The approach that we used for simulating multivariate epidemics is relatively easy to implement and could be used in other situations where real epidemic data are unavailable. We believe that our study results can support the implementation and assessment of multivariate SyS systems in animal health.

Potential added value of the new emergency care dataset to ED-based public health surveillance in England: an initial concept analysis

INTRODUCTION

For the London Olympic and Paralympic Games in 2012, a sentinel ED syndromic surveillance system was established to enhance public health surveillance by obtaining data from a selected network of EDs, focusing on London. In 2017, a new national standard Emergency Care Dataset was introduced, which enabled Public Health England (PHE) to initiate the expansion of their sentinel system to national coverage. Prior to this initiative, we estimated the added value, and potential additional resource use, of an expansion of the sentinel surveillance system.

METHODS

The detection capabilities of the sentinel and national systems were compared using the aberration detection methods currently used by PHE. Different scenarios were used to measure the impact on health at a local, subnational and national level, including improvements to sensitivity and timeliness, along with changes in specificity.

RESULTS

The biggest added value was found to be for detecting local impacts, with an increase in sensitivity of over 80%. There were also improvements found at a national level with outbreaks being detected earlier and smaller impacts being detectable. However, the increased number of local sites will also increase the number of false alarms likely to be generated.

CONCLUSION

We have quantified the added value of national ED syndromic surveillance systems, showing how they will enable detection of more localised events. Furthermore, national systems add value in enabling timelier public health interventions. Finally, we have highlighted areas where extra resource may be required to manage improvements in detection coverage.