Effectiveness of non-pharmaceutical interventions related to social distancing on respiratory viral infectious disease outcomes: A rapid evidence-based review and meta-analysis

OBJECTIVES

Non-pharmaceutical interventions (e.g. quarantine and isolation) are used to mitigate and control viral infectious disease, but their effectiveness has not been well studied. For COVID-19, disease control efforts will rely on non-pharmaceutical interventions until pharmaceutical interventions become widely available, while non-pharmaceutical interventions will be of continued importance thereafter.

METHODS

This rapid evidence-based review provides both qualitative and quantitative analyses of the effectiveness of social distancing non-pharmaceutical interventions on disease outcomes. Literature was retrieved from MEDLINE, Google Scholar, and pre-print databases (BioRxiv.org, MedRxiv.org, and Wellcome Open Research).

RESULTS

Twenty-eight studies met inclusion criteria (n = 28). Early, sustained, and combined application of various non-pharmaceutical interventions could mitigate and control primary outbreaks and prevent more severe secondary or tertiary outbreaks. The strategic use of non-pharmaceutical interventions decreased incidence, transmission, and/or mortality across all interventions examined. The pooled attack rates for no non-pharmaceutical intervention, single non-pharmaceutical interventions, and multiple non-pharmaceutical interventions were 42% (95% confidence interval = 30% - 55%), 29% (95% confidence interval = 23% - 36%), and 22% (95% confidence interval = 16% - 29%), respectively.

CONCLUSION

Implementation of multiple non-pharmaceutical interventions at key decision points for public health could effectively facilitate disease mitigation and suppression until pharmaceutical interventions become available. Dynamics around R 0 values, the susceptibility of certain high-risk patient groups to infection, and the probability of asymptomatic cases spreading disease should be considered.

Complete human papillomavirus vaccination coverage over a 13-year period in a large population of privately insured U.S. patients

1511

Background: In the US, Human Papillomavirus (HPV) vaccination coverage is low, particularly in adolescents aged 13-15 years with respect to the Healthy People 2020 goal of 80%. There has been variability in the definition of measuring vaccination coverage in published studies. We examined complete HPV vaccination coverage in a population of privately insured individuals in the US. Methods: This retrospective study used IBM MarketScan Commercial Database, years 2006 to 2018. Inclusion criteria were ages 9 to 45 years and continuous enrollment from age 9 years or from 2006. Complete HPV vaccination coverage was defined as receipt of 2 doses (age 9-15 years) or 3 doses (age 16-45 years) within 12 months and stratified by year, demographics, and US region. Mean vaccination costs per dose were summarized by vaccine brand and health plan type. Results: The table summarizes complete HPV vaccination coverage by selected age groups for 2006 (n=12,221,938), 2010 (n=4,692,633), 2014 (n=2,808,132), and 2018 (n=1,662,148). From 2017 to 2018, the percentage of members who received HPV vaccine increased; for females ages 13-15 by 1% and 16-17 by 5% while for males ages 13-15 by 6% and 16-17 by 15%. In 2018, by region, the highest coverage was in females aged 18-26 at 53% and males aged 16-17 at 43% in the Northeast, and mean cost for each brand was $120 (-6% from 2017), $165 (-3%) and $220 (+5%) for Cervarix (n=151), Gardasil (n=8,201) and Gardasil 9 (n=139,356), respectively. The rate of utilization of Gardasil 9 increased from 33% (2015) to 94% (2018) of all vaccines. The lowest mean HPV vaccine cost by health plan type and brand was with Point-of-Service (POS) and Cervarix at $106, and the highest was with POS with Capitation and Gardasil 9 at $243. Conclusions: In a commercially insured US population, complete HPV vaccination coverage was lower than the Healthy People 2020 goal, but increased over time. Coverage varied according to health plan type and by region. In 2018, Gardasil 9 had the highest mean cost but was the most utilized vaccine, which may be related to broader coverage of HPV types. This study was limited by the transient nature of member enrollment and complexity of measuring complete vaccination coverage. These results should inform policy makers and practicing clinicians about the gap in vaccination coverage. [Table: see text]

Artificial intelligence-based conversational agent to support medication prescribing

Objective

This article describes the system architecture, training, initial use, and performance of Watson Assistant (WA), an artificial intelligence-based conversational agent, accessible within Micromedex^®.

Materials and methods

The number and frequency of intents (target of a user’s query) triggered in WA during its initial use were examined; intents triggered over 9 months were compared to the frequency of topics accessed via keyword search of Micromedex. Accuracy of WA intents assigned to 400 queries was compared to assignments by 2 independent subject matter experts (SMEs), with inter-rater reliability measured by Cohen’s kappa.

Results

In over 126 000 conversations with WA, intents most frequently triggered involved dosing (N = 30 239, 23.9%) and administration (N = 14 520, 11.5%). SMEs with substantial inter-rater agreement (kappa = 0.71) agreed with intent mapping in 247 of 400 queries (62%), including 16 queries related to content that WA and SMEs agreed was unavailable in WA. SMEs found 57 (14%) of 400 queries incorrectly mapped by WA; 112 (28%) queries unanswerable by WA included queries that were either ambiguous, contained unrecognized typographical errors, or addressed topics unavailable to WA. Of the queries answerable by WA (288), SMEs determined 231 (80%) were correctly linked to an intent.

Discussion

A conversational agent successfully linked most queries to intents in Micromedex. Ongoing system training seeks to widen the scope of WA and improve matching capabilities.

Conclusion

WA enabled Micromedex users to obtain answers to many medication-related questions using natural language, with the conversational agent facilitating mapping to a broader distribution of topics than standard keyword searches.

Characterizing VA Users with the OMOP Common Data Model

In 2015, the VA Informatics and Computing Infrastructure, a resource center of the Department of Veterans Affairs, began to transform parts of its Corporate Data Warehouse (CDW) into the Observational Medical Outcomes Partnership) Common Data Model for use by its research and operations communities. Using the hierarchical relationships within the clinical vocabularies in OMOP we found differences in visits, disease prevalence, and medications prescribed between male and female veterans seen between VA fiscal years 2000-17.

The use of natural language processing on narrative medication schedules to compute average weekly dose

PURPOSE

Medications with non-standard dosing and unstandardized units of measurement make the estimation of prescribed dose difficult from pharmacy dispensing data. A natural language processing tool named the SIG extractor was developed to identify and extract elements from narrative medication instructions to compute average weekly doses (AWDs) for disease-modifying antirheumatic drugs. The goal of this paper is to evaluate the performance of the SIG extractor.

METHOD

This agreement study utilized Veterans Health Affairs pharmacy data from 2008 to 2012. The SIG extractor was designed to extract key elements from narrative medication schedules (SIGs) for 17 select medications to calculate AWD, and these medications were categorized by generic name and route of administration. The SIG extractor was evaluated against an annotator-derived reference standard for accuracy, which is the fraction of AWDs accurately computed.

RESULTS

The overall accuracy was 89% [95% confidence interval (CI) 88%, 90%]. The accuracy was ≥85% for all medications and route combinations, except for cyclophosphamide (oral) and cyclosporine (oral), which were 79% (95%CI 72%, 85%) and 66% (95%CI 58%, 73%), respectively.

CONCLUSIONS

The SIG extractor performed well on the majority of medications, indicating that AWD calculated by the SIG extractor can be used to improve estimation of AWD when dispensed quantity or days' supply is questionable or improbable. The working model for annotating SIGs and the SIG extractor are generalized and can easily be applied to other medications. Copyright © 2016 John Wiley & Sons, Ltd.

Extracting Concepts Related to Homelessness from the Free Text of VA Electronic Medical Records

Mining the free text of electronic medical records (EMR) using natural language processing (NLP) is an effective method of extracting information not always captured in administrative data. We sought to determine if concepts related to homelessness, a non-medical condition, were amenable to extraction from the EMR of Veterans Affairs (VA) medical records. As there were no off-the-shelf products, a lexicon of terms related to homelessness was created. A corpus of free text documents from outpatient encounters was reviewed to create the reference standard for NLP training and testing. V3NLP Framework was used to detect instances of lexical terms and was compared to the reference standard. With a positive predictive value of 77% for extracting relevant concepts, this study demonstrates the feasibility of extracting positively asserted concepts related to homelessness from the free text of medical records.

Using natural language processing on the free text of clinical documents to screen for evidence of homelessness among US veterans

Information retrieval algorithms based on natural language processing (NLP) of the free text of medical records have been used to find documents of interest from databases. Homelessness is a high priority non-medical diagnosis that is noted in electronic medical records of Veterans in Veterans Affairs (VA) facilities. Using a human-reviewed reference standard corpus of clinical documents of Veterans with evidence of homelessness and those without, an open-source NLP tool (Automated Retrieval Console v2.0, ARC) was trained to classify documents. The best performing model based on document level work-flow performed well on a test set (Precision 94%, Recall 97%, F-Measure 96). Processing of a naïve set of 10,000 randomly selected documents from the VA using this best performing model yielded 463 documents flagged as positive, indicating a 4.7% prevalence of homelessness. Human review noted a precision of 70% for these flags resulting in an adjusted prevalence of homelessness of 3.3% which matches current VA estimates. Further refinements are underway to improve the performance. We demonstrate an effective and rapid lifecycle of using an off-the-shelf NLP tool for screening targets of interest from medical records.

Extracting Surveillance Data from Templated Sections of an Electronic Medical Note: Challenges and Opportunities

Objective

To highlight the importance of templates in extracting surveillance data from the free text of electronic medical records using natural language processing (NLP) techniques.

Introduction

The main stay of recording patient data is the free text of electronic medical records (EMR). While stating the chief complaint and history of presenting illness in the patients ‘own words’, the rest of the electronic note is written by the provider in their words. Providers often use boiler-plate templates from EMR pull-downs to document information on the patient in the form of checklists, check boxes, yes/no and free text responses to questions. When these templates are used for recording symptoms, demographic information or medical, social or travel history, they represent an important source of surveillance data [1]. There is a dearth of literature on the use of natural language processing in extracting data from templates in the EMR.

Methods

A corpus of 1000 free text medical notes from the VA integrated electronic medical record (CPRS) was reviewed to identify commonly used templates. Of these, 500 were enriched for the surveillance domain of interest for this project (homelessness). The other 500 were randomly sampled from a large corpus of electronic notes. An NLP algorithm was developed to extract concepts related to our target surveillance domain. A manual review of the notes was performed by three human reviewers to generate a document-level reference standard that classified this set of documents as either demonstrating evidence of homelessness (H) or not (NH). A rule-based NLP algorithm was developed that used a combination of key word searches and negation based on an extensive lexicon of terms developed for this purpose. A random sample of 50 documents each of H and NH documents were reviewed after each iteration of the NLP algorithm to determine the false positive rate of the extracted concepts.

Results

The corpus consisted of 48% H and 52% NH documents as determined by human review. The NLP algorithm successfully extracted concepts from these documents. The H set had an average of 8 concepts related to homelessness per document (median 8, range 1 to 34). The NH set had an average 2 concepts (median 1, range 1 to 13)”. Thirteen template patterns were identified in this set of documents. The three most common were check boxes with square brackets, Yes/No and free text answer after a question. Several positively and negatively asserted concepts were noted to be in the responses to templated questions such as “Are you currently homeless: Yes or No”; “How many times have you been homeless in the past 3 years: (free text response)”; “Have you ever been in jail? [Y] or [N]”; Are you in need of substance abuse services? Yes or No”. Human review of a random sample of documents at the concept level indicated that the NLP algorithm generated 28% false positives in extracting concepts related to homelessness when templates were ignored among the H documents. When the algorithm was refined to include templates, the false positive rate declined to 22%. For the NH documents, the corresponding false positive rates were 56% and 21%.

Conclusions

To our knowledge, this is one of the first attempts to address the problem of information extraction from templates or templated sections of the EMR. A key challenge of templates is that they will most likely lead to poor performance of NLP algorithms and cause bottlenecks in processing if they are not considered. Acknowledging the presence of templates and refining NLP algorithms to handle them improves information extraction from free text medical notes, thus creating an opportunity for improved surveillance using the EMR. Algorithms will likely need to be customized to the electronic medical record and the surveillance domain of interest. A more detailed analysis of the templated sections is underway.

Sentiment Analysis of Suicide Notes: A Shared Task

This paper reports on a shared task involving the assignment of emotions to suicide notes. Two features distinguished this task from previous shared tasks in the biomedical domain. One is that it resulted in the corpus of fully anonymized clinical text and annotated suicide notes. This resource is permanently available and will (we hope) facilitate future research. The other key feature of the task is that it required categorization with respect to a large set of labels. The number of participants was larger than in any previous biomedical challenge task. We describe the data production process and the evaluation measures, and give a preliminary analysis of the results. Many systems performed at levels approaching the inter-coder agreement, suggesting that human-like performance on this task is within the reach of currently available technologies.

Combining free text and structured electronic medical record entries to detect acute respiratory infections

BACKGROUND

The electronic medical record (EMR) contains a rich source of information that could be harnessed for epidemic surveillance. We asked if structured EMR data could be coupled with computerized processing of free-text clinical entries to enhance detection of acute respiratory infections (ARI).

METHODOLOGY

A manual review of EMR records related to 15,377 outpatient visits uncovered 280 reference cases of ARI. We used logistic regression with backward elimination to determine which among candidate structured EMR parameters (diagnostic codes, vital signs and orders for tests, imaging and medications) contributed to the detection of those reference cases. We also developed a computerized free-text search to identify clinical notes documenting at least two non-negated ARI symptoms. We then used heuristics to build case-detection algorithms that best combined the retained structured EMR parameters with the results of the text analysis.

PRINCIPAL FINDINGS

An adjusted grouping of diagnostic codes identified reference ARI patients with a sensitivity of 79%, a specificity of 96% and a positive predictive value (PPV) of 32%. Of the 21 additional structured clinical parameters considered, two contributed significantly to ARI detection: new prescriptions for cough remedies and elevations in body temperature to at least 38°C. Together with the diagnostic codes, these parameters increased detection sensitivity to 87%, but specificity and PPV declined to 95% and 25%, respectively. Adding text analysis increased sensitivity to 99%, but PPV dropped further to 14%. Algorithms that required satisfying both a query of structured EMR parameters as well as text analysis disclosed PPVs of 52-68% and retained sensitivities of 69-73%.

CONCLUSION

Structured EMR parameters and free-text analyses can be combined into algorithms that can detect ARI cases with new levels of sensitivity or precision. These results highlight potential paths by which repurposed EMR information could facilitate the discovery of epidemics before they cause mass casualties.

Behavioural management in nursing and residential homes: a randomised controlled trial

BACKGROUND

As more and more elderly people are being cared for in residential and nursing homes, how best can their psychiatric needs be met? We report on evaluation of a behavioural intervention by an old-age psychiatry hospital outreach team.

METHODS

This randomised controlled trial of a training and education intervention over 6 months was done in south Manchester, UK. 12 matched nursing and residential homes were randomised to the control or intervention group and within each, the staff selected 10 residents whose behavioural problems made them difficult to care for. Care staff in the intervention homes attended seminars from the hospital outreach team and received weekly visits from a psychiatric nurse to assist in developing care planning skills. The main outcome measures were cognitive impairment and depression, behavioural disturbance, and functional ability, assessed by the geriatric mental state schedule, Crichton Royal behaviour rating scale, and Barthel index, respectively.

FINDINGS

Residents in the intervention group had significantly improved scores for depression (before-and-after change difference -0.5 [95% CI -0.8 to -0.1]) and for cognitive impairment (-0.7 [-1.1 to -0.2]) but not for behaviour rating or Barthel index.

INTERPRETATION

Elderly residents can benefit from improved quality of care achieved by training from a hospital outreach team.