Development and validation of an automated emergency department-based syndromic surveillance system to enhance public health surveillance in Yukon: a lower-resourced and remote setting

Using natural language processing in emergency medicine health service research: A systematic review and meta-analysis

OBJECTIVES

Natural language processing (NLP) represents one of the adjunct technologies within artificial intelligence and machine learning, creating structure out of unstructured data. This study aims to assess the performance of employing NLP to identify and categorize unstructured data within the emergency medicine (EM) setting.

METHODS

We systematically searched publications related to EM research and NLP across databases including MEDLINE, Embase, Scopus, CENTRAL, and ProQuest Dissertations & Theses Global. Independent reviewers screened, reviewed, and evaluated article quality and bias. NLP usage was categorized into syndromic surveillance, radiologic interpretation, and identification of specific diseases/events/syndromes, with respective sensitivity analysis reported. Performance metrics for NLP usage were calculated and the overall area under the summary of receiver operating characteristic curve (SROC) was determined.

RESULTS

A total of 27 studies underwent meta-analysis. Findings indicated an overall mean sensitivity (recall) of 82%-87%, specificity of 95%, with the area under the SROC at 0.96 (95% CI 0.94-0.98). Optimal performance using NLP was observed in radiologic interpretation, demonstrating an overall mean sensitivity of 93% and specificity of 96%.

CONCLUSIONS

Our analysis revealed a generally favorable performance accuracy in using NLP within EM research, particularly in the realm of radiologic interpretation. Consequently, we advocate for the adoption of NLP-based research to augment EM health care management.

Changes to Public Health Surveillance Methods Due to the COVID-19 Pandemic: Scoping Review

BACKGROUND

Public health surveillance plays a vital role in informing public health decision-making. The onset of the COVID-19 pandemic in early 2020 caused a widespread shift in public health priorities. Global efforts focused on COVID-19 monitoring and contact tracing. Existing public health programs were interrupted due to physical distancing measures and reallocation of resources. The onset of the COVID-19 pandemic intersected with advancements in technologies that have the potential to support public health surveillance efforts.

OBJECTIVE

This scoping review aims to explore emergent public health surveillance methods during the early COVID-19 pandemic to characterize the impact of the pandemic on surveillance methods.

METHODS

A scoping search was conducted in multiple databases and by scanning key government and public health organization websites from March 2020 to January 2022. Published papers and gray literature that described the application of new or revised approaches to public health surveillance were included. Papers that discussed the implications of novel public health surveillance approaches from ethical, legal, security, and equity perspectives were also included. The surveillance subject, method, location, and setting were extracted from each paper to identify trends in surveillance practices. Two public health epidemiologists were invited to provide their perspectives as peer reviewers.

RESULTS

Of the 14,238 unique papers, a total of 241 papers describing novel surveillance methods and changes to surveillance methods are included. Eighty papers were review papers and 161 were single studies. Overall, the literature heavily featured papers detailing surveillance of COVID-19 transmission (n=187). Surveillance of other infectious diseases was also described, including other pathogens (n=12). Other public health topics included vaccines (n=9), mental health (n=11), substance use (n=4), healthy nutrition (n=1), maternal and child health (n=3), antimicrobial resistance (n=2), and misinformation (n=6). The literature was dominated by applications of digital surveillance, for example, by using big data through mobility tracking and infodemiology (n=163). Wastewater surveillance was also heavily represented (n=48). Other papers described adaptations to programs or methods that existed prior to the COVID-19 pandemic (n=9). The scoping search also found 109 papers that discuss the ethical, legal, security, and equity implications of emerging surveillance methods. The peer reviewer public health epidemiologists noted that additional changes likely exist, beyond what has been reported and available for evidence syntheses.

CONCLUSIONS

The COVID-19 pandemic accelerated advancements in surveillance and the adoption of new technologies, especially for digital and wastewater surveillance methods. Given the investments in these systems, further applications for public health surveillance are likely. The literature for surveillance methods was dominated by surveillance of infectious diseases, particularly COVID-19. A substantial amount of literature on the ethical, legal, security, and equity implications of these emerging surveillance methods also points to a need for cautious consideration of potential harm.

The Use and Structure of Emergency Nurses' Triage Narrative Data: Scoping Review

BACKGROUND

Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner.

OBJECTIVE

The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted.

METHODS

We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research.

RESULTS

We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses' narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%).

CONCLUSIONS

The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055132.

Natural Language Processing for Improved Characterization of COVID-19 Symptoms: Observational Study of 350,000 Patients in a Large Integrated Health Care System

BACKGROUND

Natural language processing (NLP) of unstructured text from electronic medical records (EMR) can improve the characterization of COVID-19 signs and symptoms, but large-scale studies demonstrating the real-world application and validation of NLP for this purpose are limited.

OBJECTIVE

The aim of this paper is to assess the contribution of NLP when identifying COVID-19 signs and symptoms from EMR.

METHODS

This study was conducted in Kaiser Permanente Southern California, a large integrated health care system using data from all patients with positive SARS-CoV-2 laboratory tests from March 2020 to May 2021. An NLP algorithm was developed to extract free text from EMR on 12 established signs and symptoms of COVID-19, including fever, cough, headache, fatigue, dyspnea, chills, sore throat, myalgia, anosmia, diarrhea, vomiting or nausea, and abdominal pain. The proportion of patients reporting each symptom and the corresponding onset dates were described before and after supplementing structured EMR data with NLP-extracted signs and symptoms. A random sample of 100 chart-reviewed and adjudicated SARS-CoV-2-positive cases were used to validate the algorithm performance.

RESULTS

A total of 359,938 patients (mean age 40.4 [SD 19.2] years; 191,630/359,938, 53% female) with confirmed SARS-CoV-2 infection were identified over the study period. The most common signs and symptoms identified through NLP-supplemented analyses were cough (220,631/359,938, 61%), fever (185,618/359,938, 52%), myalgia (153,042/359,938, 43%), and headache (144,705/359,938, 40%). The NLP algorithm identified an additional 55,568 (15%) symptomatic cases that were previously defined as asymptomatic using structured data alone. The proportion of additional cases with each selected symptom identified in NLP-supplemented analysis varied across the selected symptoms, from 29% (63,742/220,631) of all records for cough to 64% (38,884/60,865) of all records with nausea or vomiting. Of the 295,305 symptomatic patients, the median time from symptom onset to testing was 3 days using structured data alone, whereas the NLP algorithm identified signs or symptoms approximately 1 day earlier. When validated against chart-reviewed cases, the NLP algorithm successfully identified signs and symptoms with consistently high sensitivity (ranging from 87% to 100%) and specificity (94% to 100%).

CONCLUSIONS

These findings demonstrate that NLP can identify and characterize a broad set of COVID-19 signs and symptoms from unstructured EMR data with enhanced detail and timeliness compared with structured data alone.

Expert consultation using the on-line Delphi method for the revision of syndromic groups compiled from emergency data (SOS Médecins and OSCOUR®) in France

Background

Consultation data from emergency general practitioners known as SOS Médecins and emergency departments (ED) from OSCOUR® network to the French syndromic surveillance system SurSaUD® (Surveillance sanitaire des urgences et décès). These data are aggregated and monitored on a daily basis through groupings of one or more medical symptoms or diagnoses (“syndromic groups” (SG)).

The objective of this study was to evaluate, revise and enrich the composition of SGs through a consensus of experts who contributed or have experience in syndromic surveillance.

Methods

Three rounds of a Delphi survey were organised, involving 15 volunteers from SOS Médecins and 64 ED physicians in the OSCOUR® network as well as 8 international epidemiologists. Thirty-four SOS Médecins and 40 OSCOUR® SGs covering major medical specialities were put to the experts, along with their diagnostic codes and their surveillance objectives. In each round, the experts could retain or reject the codes according to the surveillance objective. The panel could also put forward new diagnostic codes in the 1st round, included in subsequent rounds. Consensus was reached for a code if 80% of participants had chosen to keep it, or less than 20% to reject it.

Results

A total of 12 SOS Médecins doctors (80%), 30 ED doctors (47%) and 4 international experts (50%) participated in the three rounds. All of the SGs presented to the panel included 102 initial diagnostic codes and 73 additional codes for SOS Médecins, 272 initial diagnostic codes and 204 additional codes for OSCOUR®. At the end of the 3 rounds, 14 SOS Médecins (40%) and 11 OSCOUR® (28%) SGs achieved a consensus to maintain all of their diagnostic codes. Among these, indicators of winter seasonal surveillance (bronchiolitis and gastroenteritis) were included.

Conclusion

This study involved a panel of national experts with international representation and a good level of involvement throughout the survey. In the absence of a standard definition, the Delphi method has been shown to be useful in defining and validating syndromic surveillance indicators.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-14157-x.

Correlation-Based Discovery of Disease Patterns for Syndromic Surveillance

Early outbreak detection is a key aspect in the containment of infectious diseases, as it enables the identification and isolation of infected individuals before the disease can spread to a larger population. Instead of detecting unexpected increases of infections by monitoring confirmed cases, syndromic surveillance aims at the detection of cases with early symptoms, which allows a more timely disclosure of outbreaks. However, the definition of these disease patterns is often challenging, as early symptoms are usually shared among many diseases and a particular disease can have several clinical pictures in the early phase of an infection. As a first step toward the goal to support epidemiologists in the process of defining reliable disease patterns, we present a novel, data-driven approach to discover such patterns in historic data. The key idea is to take into account the correlation between indicators in a health-related data source and the reported number of infections in the respective geographic region. In an preliminary experimental study, we use data from several emergency departments to discover disease patterns for three infectious diseases. Our results show the potential of the proposed approach to find patterns that correlate with the reported infections and to identify indicators that are related to the respective diseases. It also motivates the need for additional measures to overcome practical limitations, such as the requirement to deal with noisy and unbalanced data, and demonstrates the importance of incorporating feedback of domain experts into the learning procedure.