Perceived usefulness of a distributed community-based syndromic surveillance system: a pilot qualitative evaluation study

Evaluation of the collaborative integrated surveillance system (ViCo) in Guatemala: a qualitative study on lessons learned and future perspectives

BACKGROUND

The collaborative integrated surveillance system known as Vigilancia Integrada Comunitaria (ViCo) was implemented in 2007 to better understand and characterize the burden of diarrheal, respiratory and febrile illnesses in Guatemala.

METHODS

To evaluate the usefulness of ViCo and inform a redesign of the system and new surveillance activities in the Central American region, personnel from the United States Centers for Disease Control and Prevention (CDC) conducted thirty-nine in-depth interviews from June-December 2018 with key stakeholders responsible for the design and implementation of ViCo in Guatemala. A semi-structured questionnaire adapted from the Updated CDC Guidelines for Evaluating Public Health Surveillance Systems was used for data collection. We used a grounded theory approach to explore stakeholder perceptions of ViCo and generate recommendations for improvement. Primary qualitative findings were organized based on thematic areas using ATLAS.ti version 8 software.

RESULTS

Emergent themes relevant to the usefulness of ViCo were organized across strengths, weaknesses, and recommendations pertaining to the: (1) Size and Complexity of ViCo, (2) Stakeholder Expectations About the Objectives of ViCo, (3) Data Management and Structure of the Information System, (4) Local Control of Data, (5) Integration of ViCo within the Ministry of Health, and, (6) Improvement of the Operational and Design Aspects of ViCo across System, Process, and Output levels.

CONCLUSIONS

Stakeholders perceived ViCo to be useful. They recommended measures to improve system performance and quality, including simplifying the surveillance system, routine data analysis and feedback, and channeling efforts towards integrating surveillance data into the national health information system. To create a well-performing surveillance system and achieve the intended objective of surveillance for public health action, ongoing evaluation and assessment of surveillance activities are necessary.

Design and Evaluation of Low-Fidelity Visual Display Prototypes for Multiple Hospital-Acquired Conditions

Hospital-acquired conditions such as catheter-associated urinary tract infection, stage 3 or 4 hospital-acquired pressure injury, and falls with injury are common, costly, and largely preventable. This study used participatory design methods to design and evaluate low-fidelity prototypes of clinical dashboards to inform high-fidelity prototype designs to visualize integrated risks based on patient profiles. Five low-fidelity prototypes were developed through literature review and by engaging nurses, nurse managers, and providers as participants (N = 23) from two hospitals in different healthcare systems using focus groups and interviews. Five themes were identified from participatory design sessions: Need for Integrated Hospital-Acquired Condition Risk Tool, Information Needs, Sources of Information, Trustworthiness of Information, and Performance Tracking Perspectives. Participants preferred visual displays that represented patient comparative risks for hospital-acquired conditions using the familiar design metaphor of a gauge and green, yellow, and red "traffic light" colors scheme. Findings from this study were used to design a high-fidelity prototype to be tested in the next phase of the project. Visual displays of hospital-acquired conditions that are familiar in display and simplify complex information such as the green, yellow, and red dashboard are needed to assist clinicians in fast-paced clinical environments and be designed to prevent alert fatigue.

Potential added value of the new emergency care dataset to ED-based public health surveillance in England: an initial concept analysis

INTRODUCTION

For the London Olympic and Paralympic Games in 2012, a sentinel ED syndromic surveillance system was established to enhance public health surveillance by obtaining data from a selected network of EDs, focusing on London. In 2017, a new national standard Emergency Care Dataset was introduced, which enabled Public Health England (PHE) to initiate the expansion of their sentinel system to national coverage. Prior to this initiative, we estimated the added value, and potential additional resource use, of an expansion of the sentinel surveillance system.

METHODS

The detection capabilities of the sentinel and national systems were compared using the aberration detection methods currently used by PHE. Different scenarios were used to measure the impact on health at a local, subnational and national level, including improvements to sensitivity and timeliness, along with changes in specificity.

RESULTS

The biggest added value was found to be for detecting local impacts, with an increase in sensitivity of over 80%. There were also improvements found at a national level with outbreaks being detected earlier and smaller impacts being detectable. However, the increased number of local sites will also increase the number of false alarms likely to be generated.

CONCLUSION

We have quantified the added value of national ED syndromic surveillance systems, showing how they will enable detection of more localised events. Furthermore, national systems add value in enabling timelier public health interventions. Finally, we have highlighted areas where extra resource may be required to manage improvements in detection coverage.

Does locally relevant, real-time infection epidemiological data improve clinician management and antimicrobial prescribing in primary care? A systematic review

Purpose

Antimicrobial resistance is a significant threat to public health. Diagnostic uncertainty is a key driver of antimicrobial prescribing. We sought to determine whether locally relevant, real-time syndromic or microbiological infection epidemiology can improve prescribing by reducing diagnostic uncertainty.

Methods

Eligible studies investigated effects on primary care prescribing for common infections in Organisation For Economic Co-Operation And Development countries. We searched Medline, Embase, Cumulative index to nursing and allied health literature, Web of Science, grey literature sources, thesis databases and trial registries.

Results

We identified 9548 reports, of which 17 were eligible, reporting 12 studies, of which 3 reported relevant outcomes. The first (observational) showed antibacterial prescribing for upper respiratory infections reduced from 26.4% to 8.6% (P = 0.01). The second (observational) showed antibacterial prescribing reduced during influenza pandemic compared with seasonal influenza periods [odds ratio (OR) 0.72 (95% CI, 0.68 to 0.77), P < 0.001], while antiviral prescribing increased [OR 6.43 (95% CI, 5.02 to 8.25), P < 0.001]. The likelihood of prescribing also decreased as the number of infection cases a physician saw increased in the previous week [OR 0.57 (95% CI, 0.51 to 0.63), P < 0.001 for ≥12 versus ≤1 patient). The third (randomized-controlled trial) showed an absolute reduction in antibacterial prescribing of 5.1% during a period of moderate influenza activity (P < 0.05). We did not find measures of diagnostic certainty, harms or costs.

Conclusion

There is promising evidence that epidemiological syndromic and microbiological data can reduce primary care antimicrobial prescribing. Future research should use randomized designs of behaviourally informed interventions, investigate costs and harms, and establish mechanisms of behaviour change.

PROSPERO registration

CRD42016038871.

The other side of surveillance: Monitoring, application, and integration of tuberculosis data to guide and evaluate programme activities in South Africa

BACKGROUND

The importance of using surveillance data to monitor and evaluate programme activities has been emphasised in international policies for tuberculosis (TB) control.

OBJECTIVES

A survey was conducted to assess the use of TB surveillance data to monitor and guide TB programme activities in South Africa (SA).

METHODS

As part of an evaluation of the SA national TB surveillance system, semi-structured interviews were conducted among TB staff at health facilities and offices in three provinces. At each site, all persons involved with TB care, management and surveillance were invited to participate.

RESULTS

At least one person (range 1 - 4) was interviewed at 47/54 health facilities (87.0%), 11/13 subdistrict and district TB offices (84.6%), 2/3 provincial TB offices (66.7%), and at the national level (1/1, 100.0%). Of 119 TB staff, 64.7% recognised the purpose of TB surveillance as guiding programme planning, implementation and evaluation. However, only 16.0% reported using data to measure disease burden, 8.4% to monitor trends, and 9.2% to inform resource allocation. The majority reported using TB management tools provided by the national programme, but 44.5% also described using additional tools. Personnel mentioned the need for dedicated surveillance staff, training on recording and reporting, improved computer access, and methods to apply information from surveillance data to the programme.

CONCLUSIONS

The majority of TB staff understood the purpose of surveillance but did not routinely use data to guide programme planning, implementation and evaluation. Training and supporting TB staff to utilise surveillance data will help improve the TB surveillance system.

Visualizing the quality of partially accruing data for use in decision making

Secondary use of clinical health data for near real-time public health surveillance presents challenges surrounding its utility due to data quality issues. Data used for real-time surveillance must be timely, accurate and complete if it is to be useful; if incomplete data are used for surveillance, understanding the structure of the incompleteness is necessary. Such data are commonly aggregated due to privacy concerns. The Distribute project was a near real-time influenza-like-illness (ILI) surveillance system that relied on aggregated secondary clinical health data. The goal of this work is to disseminate the data quality tools developed to gain insight into the data quality problems associated with these data. These tools apply in general to any system where aggregate data are accrued over time and were created through the end-user-as-developer paradigm. Each tool was developed during the exploratory analysis to gain insight into structural aspects of data quality. Our key finding is that data quality of partially accruing data must be studied in the context of accrual lag-the difference between the time an event occurs and the time data for that event are received, i.e. the time at which data become available to the surveillance system. Our visualization methods therefore revolve around visualizing dimensions of data quality affected by accrual lag, in particular the tradeoff between timeliness and completion, and the effects of accrual lag on accuracy. Accounting for accrual lag in partially accruing data is necessary to avoid misleading or biased conclusions about trends in indicator values and data quality.

A counselor in your pocket: feasibility of mobile health tailored messages to support HIV medication adherence

PURPOSE

Medication adherence is a major challenge in HIV treatment. New mobile technologies such as smartphones facilitate the delivery of brief tailored messages to promote adherence. However, the best approach for tailoring messages is unknown. Persons living with HIV (PLWH) might be more receptive to some messages than others based on their current psychological state.

METHODS

We recruited 37 PLWH from a parent study of motivational states and adherence. Participants completed smartphone-based surveys at a random time every day for 2 weeks, then immediately received intervention or control tailored messages, depending on random assignment. After 2 weeks in the initial condition, participants received the other condition in a crossover design. Intervention messages were tailored to match PLWH's current psychological state based on five variables - control beliefs, mood, stress, coping, and social support. Control messages were tailored to create a mismatch between message framing and participants' current psychological state. We evaluated intervention feasibility based on acceptance, ease of use, and usefulness measures. We also used pilot randomized controlled trial methods to test the intervention's effect on adherence, which was measured using electronic caps that recorded pill-bottle openings.

RESULTS

Acceptance was high based on 76% enrollment and 85% satisfaction. Participants found the hardware and software easy to use. However, attrition was high at 59%, and usefulness ratings were slightly lower. The most common complaint was boredom. Unexpectedly, there was no difference between mismatched and matched messages' effects, but each group showed a 10%-15% improvement in adherence after crossing to the opposite study condition.

CONCLUSION

Although smartphone-based tailored messaging was feasible and participants had clinically meaningful improvements in adherence, the mechanisms of change require further study. Possible explanations might include novelty effects, increased receptiveness to new information after habituation, or pseudotailoring, three ways in which attentional processes can affect behavior.

A practical approach to designing syndromic surveillance systems for livestock and poultry

The field of animal syndromic surveillance (SyS) is growing, with many systems being developed worldwide. Now is an appropriate time to share ideas and lessons learned from early SyS design and implementation. Based on our practical experience in animal health SyS, with additions from the public health and animal health SyS literature, we put forward for discussion a 6-step approach to designing SyS systems for livestock and poultry. The first step is to formalise policy and surveillance goals which are considerate of stakeholder expectations and reflect priority issues (1). Next, it is important to find consensus on national priority diseases and identify current surveillance gaps. The geographic, demographic, and temporal coverage of the system must be carefully assessed (2). A minimum dataset for SyS that includes the essential data to achieve all surveillance objectives while minimizing the amount of data collected should be defined. One can then compile an inventory of the data sources available and evaluate each using the criteria developed (3). A list of syndromes should then be produced for all data sources. Cases can be classified into syndrome classes and the data can be converted into time series (4). Based on the characteristics of the syndrome-time series, the length of historic data available and the type of outbreaks the system must detect, different aberration detection algorithms can be tested (5). Finally, it is essential to develop a minimally acceptable response protocol for each statistical signal produced (6). Important outcomes of this pre-operational phase should be building of a national network of experts and collective action and evaluation plans. While some of the more applied steps (4 and 5) are currently receiving consideration, more emphasis should be put on earlier conceptual steps by decision makers and surveillance developers (1-3).

Syndromic surveillance for influenza in the emergency department-A systematic review

The science of surveillance is rapidly evolving due to changes in public health information and preparedness as national security issues, new information technologies and health reform. As the Emergency Department has become a much more utilized venue for acute care, it has also become a more attractive data source for disease surveillance. In recent years, influenza surveillance from the Emergency Department has increased in scope and breadth and has resulted in innovative and increasingly accepted methods of surveillance for influenza and influenza-like-illness (ILI). We undertook a systematic review of published Emergency Department-based influenza and ILI syndromic surveillance systems. A PubMed search using the keywords "syndromic", "surveillance", "influenza" and "emergency" was performed. Manuscripts were included in the analysis if they described (1) data from an Emergency Department (2) surveillance of influenza or ILI and (3) syndromic or clinical data. Meeting abstracts were excluded. The references of included manuscripts were examined for additional studies. A total of 38 manuscripts met the inclusion criteria, describing 24 discrete syndromic surveillance systems. Emergency Department-based influenza syndromic surveillance has been described worldwide. A wide variety of clinical data was used for surveillance, including chief complaint/presentation, preliminary or discharge diagnosis, free text analysis of the entire medical record, Google flu trends, calls to teletriage and help lines, ambulance dispatch calls, case reports of H1N1 in the media, markers of ED crowding, admission and Left Without Being Seen rates. Syndromes used to capture influenza rates were nearly always related to ILI (i.e. fever +/- a respiratory or constitutional complaint), however, other syndromes used for surveillance included fever alone, "respiratory complaint" and seizure. Two very large surveillance networks, the North American DiSTRIBuTE network and the European Triple S system have collected large-scale Emergency Department-based influenza and ILI syndromic surveillance data. Syndromic surveillance for influenza and ILI from the Emergency Department is becoming more prevalent as a measure of yearly influenza outbreaks.

Influenza forecasting with Google Flu Trends

BACKGROUND

We developed a practical influenza forecast model based on real-time, geographically focused, and easy to access data, designed to provide individual medical centers with advanced warning of the expected number of influenza cases, thus allowing for sufficient time to implement interventions. Secondly, we evaluated the effects of incorporating a real-time influenza surveillance system, Google Flu Trends, and meteorological and temporal information on forecast accuracy.

METHODS

Forecast models designed to predict one week in advance were developed from weekly counts of confirmed influenza cases over seven seasons (2004-2011) divided into seven training and out-of-sample verification sets. Forecasting procedures using classical Box-Jenkins, generalized linear models (GLM), and generalized linear autoregressive moving average (GARMA) methods were employed to develop the final model and assess the relative contribution of external variables such as, Google Flu Trends, meteorological data, and temporal information.

RESULTS

A GARMA(3,0) forecast model with Negative Binomial distribution integrating Google Flu Trends information provided the most accurate influenza case predictions. The model, on the average, predicts weekly influenza cases during 7 out-of-sample outbreaks within 7 cases for 83% of estimates. Google Flu Trend data was the only source of external information to provide statistically significant forecast improvements over the base model in four of the seven out-of-sample verification sets. Overall, the p-value of adding this external information to the model is 0.0005. The other exogenous variables did not yield a statistically significant improvement in any of the verification sets.

CONCLUSIONS

Integer-valued autoregression of influenza cases provides a strong base forecast model, which is enhanced by the addition of Google Flu Trends confirming the predictive capabilities of search query based syndromic surveillance. This accessible and flexible forecast model can be used by individual medical centers to provide advanced warning of future influenza cases.