Scalable Approach to Consumer Wearable Postmarket Surveillance: Development and Validation Study

Background

With the capability to render prediagnoses, consumer wearables have the potential to affect subsequent diagnoses and the level of care in the health care delivery setting. Despite this, postmarket surveillance of consumer wearables has been hindered by the lack of codified terms in electronic health records (EHRs) to capture wearable use.

Objective

We sought to develop a weak supervision-based approach to demonstrate the feasibility and efficacy of EHR-based postmarket surveillance on consumer wearables that render atrial fibrillation (AF) prediagnoses.

Methods

We applied data programming, where labeling heuristics are expressed as code-based labeling functions, to detect incidents of AF prediagnoses. A labeler model was then derived from the predictions of the labeling functions using the Snorkel framework. The labeler model was applied to clinical notes to probabilistically label them, and the labeled notes were then used as a training set to fine-tune a classifier called Clinical-Longformer. The resulting classifier identified patients with an AF prediagnosis. A retrospective cohort study was conducted, where the baseline characteristics and subsequent care patterns of patients identified by the classifier were compared against those who did not receive a prediagnosis.

Results

The labeler model derived from the labeling functions showed high accuracy (0.92; F1-score=0.77) on the training set. The classifier trained on the probabilistically labeled notes accurately identified patients with an AF prediagnosis (0.95; F1-score=0.83). The cohort study conducted using the constructed system carried enough statistical power to verify the key findings of the Apple Heart Study, which enrolled a much larger number of participants, where patients who received a prediagnosis tended to be older, male, and White with higher CHA2DS2-VASc (congestive heart failure, hypertension, age ≥75 years, diabetes, stroke, vascular disease, age 65-74 years, sex category) scores (P<.001). We also made a novel discovery that patients with a prediagnosis were more likely to use anticoagulants (525/1037, 50.63% vs 5936/16,560, 35.85%) and have an eventual AF diagnosis (305/1037, 29.41% vs 262/16,560, 1.58%). At the index diagnosis, the existence of a prediagnosis did not distinguish patients based on clinical characteristics, but did correlate with anticoagulant prescription (P=.004 for apixaban and P=.01 for rivaroxaban).

Conclusions

Our work establishes the feasibility and efficacy of an EHR-based surveillance system for consumer wearables that render AF prediagnoses. Further work is necessary to generalize these findings for patient populations at other sites.

Predictors of Incident Heart Failure Diagnosis Setting: Insights From the Veterans Affairs Healthcare System

BACKGROUND

Early recognition of heart failure (HF) can reduce morbidity, yet HF is often diagnosed only after symptoms require urgent treatment.

OBJECTIVES

The authors sought to describe predictors of HF diagnosis in the acute care vs outpatient setting within the Veterans Health Administration (VHA).

METHODS

The authors estimated whether incident HF diagnoses occurred in acute care (inpatient hospital or emergency department) vs outpatient settings within the VHA between 2014 and 2019. After excluding new-onset HF potentially caused by acute concurrent conditions, they identified sociodemographic and clinical variables associated with diagnosis setting and assessed variation across 130 VHA facilities using multivariable regression analysis.

RESULTS

The authors identified 303,632 patients with new HF, with 160,454 (52.8%) diagnosed in acute care settings. In the prior year, 44% had HF symptoms and 11% had a natriuretic peptide tested, 88% of which were elevated. Patients with housing insecurity and high neighborhood social vulnerability had higher odds of acute care diagnosis (adjusted odds ratio: 1.22 [95% CI: 1.17-1.27] and 1.17 [95% CI: 1.14-1.21], respectively) adjusting for medical comorbidities. Better outpatient quality of care (blood pressure control and cholesterol and diabetes monitoring within the prior 2 years) predicted a lower odds of acute care diagnosis. Likelihood of acute care HF diagnosis varied from 41% to 68% across facilities after adjusting for patient-level risk factors.

CONCLUSIONS

Many first HF diagnoses occur in the acute care setting, especially among socioeconomically vulnerable populations. Better outpatient care was associated with lower rates of an acute care diagnosis. These findings highlight opportunities for timelier HF diagnosis that may improve patient outcomes.

Improving chronic kidney disease detection and treatment in the United States: the chronic kidney disease cascade of care (C3) study protocol

BACKGROUND

There are major gaps in the implementation of guideline-concordant care for persons with chronic kidney disease (CKD). The CKD Cascade of Care (C³) initiative seeks to improve CKD care by improving detection and treatment of CKD in primary care.

METHODS

C³ is a multi-modal initiative deployed in three major academic medical centers within the Department of Veterans Affairs (VA) Health Care System: San Francisco VA, San Diego VA, and Houston VA. The main objective of the first phase of C³ described in this protocol is to establish the infrastructure for universal CKD detection among primary care patients at high-risk for CKD with a triple-marker screen comprising cystatin C, creatinine, and albuminuria. Across the three sites, a comprehensive educational intervention and the integration of primary care-based clinical champions will be employed with the goal of improving CKD detection and treatment. The San Francisco VA will also implement a practice-facilitation intervention leveraging telehealth and health informatics tools and capabilities for enhanced CKD detection. Parallel formative evaluation across the three sites will assess the feasibility and acceptability of integrating cystatin C as part of routine CKD detection in primary care practice. The effectiveness of the interventions will be assessed using a pre-post observational design for change in the proportion of patients tested annually for CKD. Secondary outcomes will assess change in the initiation of cardio-kidney protective therapies and in nephrology referrals of high-risk patients.

DISCUSSION

The first phase of C³ is a multi-facility multi-modal initiative that aims to improve CKD care by implementing a triple-marker screen for enhanced CKD detection in primary care.

Structural Patterns in Chronic Disease Clinical Practice Guidelines Formalized for Clinical Decision Support

Making recommendations from clinical practice guidelines (CPGs) computable for clinical decision support (CDS) has typically been a laborious and costly process. Identifying domain-specific regularities helps clinicians and knowledge engineers conceptualize, extract, and encode evidence-based recommendations. Based on our work to provide complex CDS in the management of multiple chronic diseases, we propose nine chronic disease CPG structural patterns, discuss considerations in representing the necessary knowledge, and illustrate them with the solutions that our CDS system provides.

Effect of opioid prescribing guidelines in primary care

Long-term opioid use for noncancer pain is increasingly prevalent yet controversial given the risks of addiction, diversion, and overdose. Prior literature has identified the problem and proposed management guidelines, but limited evidence exists on the actual effectiveness of implementing such guidelines in a primary care setting.A multidisciplinary working group of institutional experts assembled comprehensive guidelines for chronic opioid prescribing, including monitoring and referral recommendations. The guidelines were disseminated in September 2013 to our medical center's primary care clinics via in person and electronic education.We extracted electronic medical records for patients with noncancer pain receiving opioid prescriptions (Rxs) in seasonally matched preintervention (11/1/2012-6/1/2013) and postintervention (11/1/2013-6/1/2014) periods. For patients receiving chronic (3 or more) opioid Rxs, we assessed the rates of drug screening, specialty referrals, clinic visits, emergency room visits, and quantity of opioids prescribed.After disseminating guidelines, the percentage of noncancer clinic patients receiving any opioid Rxs dropped from 3.9% to 3.4% (P = 0.02). The percentage of noncancer patients receiving chronic opioid Rxs decreased from 2.0% to 1.6% (P = 0.03). The rate of urine drug screening increased from 9.2% to 17.3% (P = 0.005) amongst noncancer chronic opioid patients. No significant differences were detected for other metrics or demographics assessed.An educational intervention for primary care opioid prescribing is feasible and was temporally associated with a modest reduction in overall opioid Rx rates. Provider use of routine drug screening increased, but overall rates of screening and specialty referral remained low despite the intervention. Despite national pressures to introduce opioid prescribing guidelines for chronic pain, doing so alone does not necessarily yield substantial changes in clinical practice.

Learning statistical models of phenotypes using noisy labeled training data

OBJECTIVE

Traditionally, patient groups with a phenotype are selected through rule-based definitions whose creation and validation are time-consuming. Machine learning approaches to electronic phenotyping are limited by the paucity of labeled training datasets. We demonstrate the feasibility of utilizing semi-automatically labeled training sets to create phenotype models via machine learning, using a comprehensive representation of the patient medical record.

METHODS

We use a list of keywords specific to the phenotype of interest to generate noisy labeled training data. We train L1 penalized logistic regression models for a chronic and an acute disease and evaluate the performance of the models against a gold standard.

RESULTS

Our models for Type 2 diabetes mellitus and myocardial infarction achieve precision and accuracy of 0.90, 0.89, and 0.86, 0.89, respectively. Local implementations of the previously validated rule-based definitions for Type 2 diabetes mellitus and myocardial infarction achieve precision and accuracy of 0.96, 0.92 and 0.84, 0.87, respectively.We have demonstrated feasibility of learning phenotype models using imperfectly labeled data for a chronic and acute phenotype. Further research in feature engineering and in specification of the keyword list can improve the performance of the models and the scalability of the approach.

CONCLUSIONS

Our method provides an alternative to manual labeling for creating training sets for statistical models of phenotypes. Such an approach can accelerate research with large observational healthcare datasets and may also be used to create local phenotype models.

OrderRex: clinical order decision support and outcome predictions by data-mining electronic medical records

OBJECTIVE

To answer a "grand challenge" in clinical decision support, the authors produced a recommender system that automatically data-mines inpatient decision support from electronic medical records (EMR), analogous to Netflix or Amazon.com's product recommender.

MATERIALS AND METHODS

EMR data were extracted from 1 year of hospitalizations (>18K patients with >5.4M structured items including clinical orders, lab results, and diagnosis codes). Association statistics were counted for the ∼1.5K most common items to drive an order recommender. The authors assessed the recommender's ability to predict hospital admission orders and outcomes based on initial encounter data from separate validation patients.

RESULTS

Compared to a reference benchmark of using the overall most common orders, the recommender using temporal relationships improves precision at 10 recommendations from 33% to 38% (P < 10(-10)) for hospital admission orders. Relative risk-based association methods improve inverse frequency weighted recall from 4% to 16% (P < 10(-16)). The framework yields a prediction receiver operating characteristic area under curve (c-statistic) of 0.84 for 30 day mortality, 0.84 for 1 week need for ICU life support, 0.80 for 1 week hospital discharge, and 0.68 for 30-day readmission.

DISCUSSION

Recommender results quantitatively improve on reference benchmarks and qualitatively appear clinically reasonable. The method assumes that aggregate decision making converges appropriately, but ongoing evaluation is necessary to discern common behaviors from "correct" ones.

CONCLUSIONS

Collaborative filtering recommender algorithms generate clinical decision support that is predictive of real practice patterns and clinical outcomes. Incorporating temporal relationships improves accuracy. Different evaluation metrics satisfy different goals (predicting likely events vs. "interesting" suggestions).

Cohort Discovery Query Optimization via Computable Controlled Vocabulary Versioning

Self-service cohort discovery tools strive to provide intuitive interfaces to large Clinical Data Warehouses that contain extensive historic information. In those tools, controlled vocabulary (e.g., ICD-9-CM, CPT) coded clinical information is often the main search criteria used because of its ubiquity in billing processes. These tools generally require a researcher to pick specific terms from the controlled vocabulary. However, controlled vocabularies evolve over time as medical knowledge changes and can even be replaced with new versions (e.g., ICD-9 to ICD-10). These tools generally only display the current version of the controlled vocabulary. Researchers should not be expected to understand the underlying controlled vocabulary versioning issues. We propose a computable controlled vocabulary versioning system that allows cohort discovery tools to automatically expand queries to account for terminology changes.

Pharmacovigilance using clinical notes

With increasing adoption of electronic health records (EHRs), there is an opportunity to use the free-text portion of EHRs for pharmacovigilance. We present novel methods that annotate the unstructured clinical notes and transform them into a deidentified patient-feature matrix encoded using medical terminologies. We demonstrate the use of the resulting high-throughput data for detecting drug-adverse event associations and adverse events associated with drug-drug interactions. We show that these methods flag adverse events early (in most cases before an official alert), allow filtering of spurious signals by adjusting for potential confounding, and compile prevalence information. We argue that analyzing large volumes of free-text clinical notes enables drug safety surveillance using a yet untapped data source. Such data mining can be used for hypothesis generation and for rapid analysis of suspected adverse event risk.

Discretization of continuous features in clinical datasets

BACKGROUND

The increasing availability of clinical data from electronic medical records (EMRs) has created opportunities for secondary uses of health information. When used in machine learning classification, many data features must first be transformed by discretization.

OBJECTIVE

To evaluate six discretization strategies, both supervised and unsupervised, using EMR data.

MATERIALS AND METHODS

We classified laboratory data (arterial blood gas (ABG) measurements) and physiologic data (cardiac output (CO) measurements) derived from adult patients in the intensive care unit using decision trees and naïve Bayes classifiers. Continuous features were partitioned using two supervised, and four unsupervised discretization strategies. The resulting classification accuracy was compared with that obtained with the original, continuous data.

RESULTS

Supervised methods were more accurate and consistent than unsupervised, but tended to produce larger decision trees. Among the unsupervised methods, equal frequency and k-means performed well overall, while equal width was significantly less accurate.

DISCUSSION

This is, we believe, the first dedicated evaluation of discretization strategies using EMR data. It is unlikely that any one discretization method applies universally to EMR data. Performance was influenced by the choice of class labels and, in the case of unsupervised methods, the number of intervals. In selecting the number of intervals there is generally a trade-off between greater accuracy and greater consistency.

CONCLUSIONS

In general, supervised methods yield higher accuracy, but are constrained to a single specific application. Unsupervised methods do not require class labels and can produce discretized data that can be used for multiple purposes.

Managing Medical Vocabulary Updates in a Clinical Data Warehouse: An RxNorm Case Study

Use of terminology standards facilitates aggregating data from multiple sources for information retrieval, exchange and analysis. However, medical vocabularies are continuously updated and incorporating those changes consistently into clinical data warehouses requires rigorous methodology. To integrate pharmacy data from two hospital pharmacy information systems the Stanford Translational Research Integrated Database Environment (STRIDE) project mapped medication orders to RxNorm content using the RxNorm drug model. In order to keep the data relevant and up-to-date, we developed a strategy for updating to RxNorm, while preserving the original meaning and mapping of the legacy data. This case study discusses managing the vocabulary update by following the RxNorm content maintenance strategy and supplementing it with operations to retain access to its drug model information.

Automated mapping of pharmacy orders from two electronic health record systems to RxNorm within the STRIDE clinical data warehouse

The Stanford Translational Research Integrated Database Environment (STRIDE) clinical data warehouse integrates medication information from two Stanford hospitals that use different drug representation systems. To merge this pharmacy data into a single, standards-based model supporting research we developed an algorithm to map HL7 pharmacy orders to RxNorm concepts. A formal evaluation of this algorithm on 1.5 million pharmacy orders showed that the system could accurately assign pharmacy orders in over 96% of cases. This paper describes the algorithm and discusses some of the causes of failures in mapping to RxNorm.

Building an internationalized content delivery architecture: lessons learned

Increasingly, small development teams are building internationalized architectures for delivering large amounts of content. The AIM e-Learning project is one such example: in 2 years, 4 people built a system which currently delivers the print equivalent of 1500 pages of text, in 4 languages, to users in over 140 countries world wide. Here we discuss the lessons we have learned through development, including issues surrounding staffing, process, technologies and next steps.