Design of the New York City Macroscope: Innovations in Population Health Surveillance Using Electronic Health Records

Characterizing opioid overdose hotspots for place-based overdose prevention and treatment interventions: A geo-spatial analysis of Rhode Island, USA

OBJECTIVE

Examine differences in neighborhood characteristics and services between overdose hotspot and non-hotspot neighborhoods and identify neighborhood-level population factors associated with increased overdose incidence.

METHODS

We conducted a population-based retrospective analysis of Rhode Island, USA residents who had a fatal or non-fatal overdose from 2016 to 2020 using an environmental scan and data from Rhode Island emergency medical services, State Unintentional Drug Overdose Reporting System, and the American Community Survey. We conducted a spatial scan via SaTScan to identify non-fatal and fatal overdose hotspots and compared the characteristics of hotspot and non-hotspot neighborhoods. We identified associations between census block group-level characteristics using a Besag-York-Mollié model specification with a conditional autoregressive spatial random effect.

RESULTS

We identified 7 non-fatal and 3 fatal overdose hotspots in Rhode Island during the study period. Hotspot neighborhoods had higher proportions of Black and Latino/a residents, renter-occupied housing, vacant housing, unemployment, and cost-burdened households. A higher proportion of hotspot neighborhoods had a religious organization, a health center, or a police station. Non-fatal overdose risk increased in a dose responsive manner with increasing proportions of residents living in poverty. There was increased relative risk of non-fatal and fatal overdoses in neighborhoods with crowded housing above the mean (RR 1.19 [95 % CI 1.05, 1.34]; RR 1.21 [95 % CI 1.18, 1.38], respectively).

CONCLUSION

Neighborhoods with increased prevalence of housing instability and poverty are at highest risk of overdose. The high availability of social services in overdose hotspots presents an opportunity to work with established organizations to prevent overdose deaths.

Facilitation of team-based care to improve HTN management and outcomes: a protocol for a randomized stepped wedge trial

BACKGROUND

There are well-established guidelines for treating hypertension (HTN), yet only half of patients with HTN meet the defined target of < 140/90. Team-based care (TBC) is an evidence-based strategy for improving blood pressure (BP) management and control. TBC is defined as the provision of health services by at least two health professionals "who work collaboratively with patients and their caregivers to accomplish shared goals to achieve coordinated, high-quality care". However, primary care practices experience challenges to implementing TBC principles and care processes; these are more pronounced in small independent practice settings (SIPs). Practice facilitation (PF) is an implementation strategy that may overcome barriers to adopting evidence-based TBC to improve HTN management in SIPs.

METHODS

Using a stepped wedge randomized controlled trial design, we will test the effect of PF on the adoption of TBC to improve HTN management in small practices (< 5 FTE clinicians) in New York City, and the impact on BP control compared with usual care. We will enroll 90 SIPs and randomize them into one of three 12-month intervention waves. Practice facilitators will support SIPs to adopt TBC principles to improve implementation of five HTN management strategies (i.e., panel management, population health, measuring BP, supporting medication adherence, self-management). The primary outcome is the adoption of TBC for HTN management measured at baseline and 12 months. Secondary outcomes include the rate of BP control and sustainability of TBC and BP outcomes at 18 months. Aggregated data on BP measures are collected every 6 months in all clusters so that each cluster provides data points in both the control and intervention conditions. Using a mixed methods approach, we will also explore factors that influence the effectiveness of PF at the organization and team level.

DISCUSSION

This study will provide much-needed guidance on how to optimize adoption and sustainability of TBC in independent primary care settings to reduce the burden of disease related to suboptimal BP control and advance understanding of how facilitation works to improve implementation of evidence-based interventions.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT05413252 .

Utilization of Electronic Health Records for Chronic Disease Surveillance: A Systematic Literature Review

This study reviews the current utilization of electronic health records (EHRs) for chronic disease surveillance, discusses approaches that are used in obtaining EHR-derived disease prevalence estimates, and identifies health indicators that have been studied using EHR-based surveillance methods. PubMed was searched for relevant keywords: (electronic health records [Title/Abstract] AND surveillance [Title/Abstract]) OR (electronic medical records [Title/Abstract] AND surveillance [Title/Abstract]). Articles were assessed based on detailed inclusion and exclusion criteria and organized by common themes, as per the PRISMA review protocol. The study period was limited to 2015-2021 due to the wider adoption of EHR in the U.S. only since 2015. The review included only US studies and only those that focused on chronic disease surveillance. 17 studies were included in the review. The most common approaches the review identified focused on validating EHR-derived estimates against those from traditional national surveys. The most studied conditions were diabetes, obesity, and hypertension. The majority of reviewed studies demonstrated comparable prevalence estimates with traditional population health surveillance surveys. The most common approach for the estimation of chronic disease conditions was to use small-area estimation by geographic patterns, neighborhoods, or census tracts. The use of EHR-based surveillance systems for public health purposes is feasible, and the population health estimates appear comparable to those obtained through traditional surveillance surveys. The application of EHRs for public health surveillance appears promising and could offer a real-time alternative to traditional surveillance methods. A timely assessment of population health at local and regional levels would ensure a more targeted allocation of public health and healthcare resources as well as more effective intervention and prevention initiatives.

Small-area estimation for public health surveillance using electronic health record data: reducing the impact of underrepresentation

Background

Electronic Health Record (EHR) data are increasingly being used to monitor population health on account of their timeliness, granularity, and large sample sizes. While EHR data are often sufficient to estimate disease prevalence and trends for large geographic areas, the same accuracy and precision may not carry over for smaller areas that are sparsely represented by non-random samples.

Methods

We developed small-area estimation models using a combination of EHR data drawn from MDPHnet, an EHR-based public health surveillance network in Massachusetts, the American Community Survey, and state hospitalization data. We estimated municipality-specific prevalence rates of asthma, diabetes, hypertension, obesity, and smoking in each of the 351 municipalities in Massachusetts in 2016. Models were compared against Behavioral Risk Factor Surveillance System (BRFSS) state and small area estimates for 2016.

Results

Integrating progressively more variables into prediction models generally reduced mean absolute error (MAE) relative to municipality-level BRFSS small area estimates: asthma (2.24% MAE crude, 1.02% MAE modeled), diabetes (3.13% MAE crude, 3.48% MAE modeled), hypertension (2.60% MAE crude, 1.48% MAE modeled), obesity (4.92% MAE crude, 4.07% MAE modeled), and smoking (5.33% MAE crude, 2.99% MAE modeled). Correlation between modeled estimates and BRFSS estimates for the 13 municipalities in Massachusetts covered by BRFSS’s 500 Cities ranged from 81.9% (obesity) to 96.7% (diabetes).

Conclusions

Small-area estimation using EHR data is feasible and generates estimates comparable to BRFSS state and small-area estimates. Integrating EHR data with survey data can provide timely and accurate disease monitoring tools for areas with sparse data coverage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13809-2.

Addressing Hypertension Outcomes Using Telehealth and Population Health Managers: Adaptations and Implementation Considerations

PURPOSE OF REVIEW

There is a growing evidence base describing population health approaches to improve blood pressure control. We reviewed emerging trends in hypertension population health management and present implementation considerations from an intervention called Team-supported, Electronic health record-leveraged, Active Management (TEAM). By doing so, we highlight the role of population health managers, practitioners who use population level data and to proactively engage at-risk patients, in improving blood pressure control.

RECENT FINDINGS

Within a population health paradigm, we discuss telehealth-delivered approaches to equitably improve hypertension care delivery. Additionally, we explore implementation considerations and complementary features of team-based, telehealth-delivered, population health management. By leveraging the unique role and expertise of a population health manager as core member of team-based telehealth, health systems can implement a cost-effective and scalable intervention that addresses multi-level barriers to hypertension care delivery. We describe the literature of telehealth-based population health management for patients with hypertension. Using the TEAM intervention as a case study, we then present implementation considerations and intervention adaptations to integrate a population health manager within the health care team and effectively manage hypertension for a defined patient population. We emphasize practical considerations to inform implementation, scaling, and sustainability. We highlight future research directions to advance the field and support translational efforts in diverse clinical and community contexts.

The Cosmos Collaborative: A Vendor-Facilitated Electronic Health Record Data Aggregation Platform

Objective

Learning healthcare systems use routinely collected data to generate new evidence that informs future practice. While implementing an electronic health record (EHR) system can facilitate this goal for individual institutions, meaningfully aggregating data from multiple institutions can be more empowering. Cosmos is a cross-institution, single EHR vendor-facilitated data aggregation tool. This work aims to describe the initiative and illustrate its potential utility through several use cases.

Methods

Cosmos is designed to scale rapidly by leveraging preexisting agreements, clinical health information exchange networks, and data standards. Data are stored centrally as a limited dataset, but the customer facing query tool limits results to prevent patient reidentification.

Results

In 2 years, Cosmos grew to contain EHR data of more than 60 million patients. We present practical examples illustrating how Cosmos could further efforts in chronic disease surveillance (asthma and obesity), syndromic surveillance (seasonal influenza and the 2019 novel coronavirus), immunization adherence and adverse event reporting (human papilloma virus and measles, mumps, rubella, and varicella vaccination), and health services research (antibiotic usage for upper respiratory infection).

Discussion

A low barrier of entry for Cosmos allows for the rapid accumulation of multi-institutional and mostly de-duplicated EHR data to power research and quality improvement queries characteristic of learning healthcare systems. Limitations are being vendor-specific, an “all or none” contribution model, and the lack of control over queries run on an institution’s healthcare data.

Conclusion

Cosmos provides a model for within-vendor data standardization and aggregation and a steppingstone for broader intervendor interoperability.

Digital Data Sources and Their Impact on People's Health: A Systematic Review of Systematic Reviews

Background: Digital data sources have become ubiquitous in modern culture in the era of digital technology but often tend to be under-researched because of restricted access to data sources due to fragmentation, privacy issues, or industry ownership, and the methodological complexity of demonstrating their measurable impact on human health. Even though new big data sources have shown unprecedented potential for disease diagnosis and outbreak detection, we need to investigate results in the existing literature to gain a comprehensive understanding of their impact on and benefits to human health. Objective: A systematic review of systematic reviews on identifying digital data sources and their impact area on people's health, including challenges, opportunities, and good practices. Methods: A multidatabase search was performed. Peer-reviewed papers published between January 2010 and November 2020 relevant to digital data sources on health were extracted, assessed, and reviewed. Results: The 64 reviews are covered by three domains, that is, universal health coverage (UHC), public health emergencies, and healthier populations, defined in WHO's General Programme of Work, 2019-2023, and the European Programme of Work, 2020-2025. In all three categories, social media platforms are the most popular digital data source, accounting for 47% (N = 8), 84% (N = 11), and 76% (N = 26) of studies, respectively. The second most utilized data source are electronic health records (EHRs) (N = 13), followed by websites (N = 7) and mass media (N = 5). In all three categories, the most studied impact of digital data sources is on prevention, management, and intervention of diseases (N = 40), and as a tool, there are also many studies (N = 10) on early warning systems for infectious diseases. However, they could also pose health hazards (N = 13), for instance, by exacerbating mental health issues and promoting smoking and drinking behavior among young people. Conclusions: The digital data sources presented are essential for collecting and mining information about human health. The key impact of social media, electronic health records, and websites is in the area of infectious diseases and early warning systems, and in the area of personal health, that is, on mental health and smoking and drinking prevention. However, further research is required to address privacy, trust, transparency, and interoperability to leverage the potential of data held in multiple datastores and systems. This study also identified the apparent gap in systematic reviews investigating the novel big data streams, Internet of Things (IoT) data streams, and sensor, mobile, and GPS data researched using artificial intelligence, complex network, and other computer science methods, as in this domain systematic reviews are not common.

Electronic health record-based disease surveillance systems: A systematic literature review on challenges and solutions

OBJECTIVE

Disease surveillance systems are expanding using electronic health records (EHRs). However, there are many challenges in this regard. In the present study, the solutions and challenges of implementing EHR-based disease surveillance systems (EHR-DS) have been reviewed.

MATERIALS AND METHODS

We searched the related keywords in ProQuest, PubMed, Web of Science, Cochrane Library, Embase, and Scopus. Then, we assessed and selected articles using the inclusion and exclusion criteria and, finally, classified the identified solutions and challenges.

RESULTS

Finally, 50 studies were included, and 52 unique solutions and 47 challenges were organized into 6 main themes (policy and regulatory, technical, management, standardization, financial, and data quality). The results indicate that due to the multifaceted nature of the challenges, the implementation of EHR-DS is not low cost and easy to implement and requires a variety of interventions. On the one hand, the most common challenges include the need to invest significant time and resources; the poor data quality in EHRs; difficulty in analyzing, cleaning, and accessing unstructured data; data privacy and security; and the lack of interoperability standards. On the other hand, the most common solutions are the use of natural language processing and machine learning algorithms for unstructured data; the use of appropriate technical solutions for data retrieval, extraction, identification, and visualization; the collaboration of health and clinical departments to access data; standardizing EHR content for public health; and using a unique health identifier for individuals.

CONCLUSIONS

EHR systems have an important role in modernizing disease surveillance systems. However, there are many problems and challenges facing the development and implementation of EHR-DS that need to be appropriately addressed.

A step closer to nationwide electronic health record-based chronic disease surveillance: characterizing asthma prevalence and emergency department utilization from 100 million patient records through a novel multisite collaboration

OBJECTIVE

The study sought to assess the feasibility of nationwide chronic disease surveillance using data aggregated through a multisite collaboration of customers of the same electronic health record (EHR) platform across the United States.

MATERIALS AND METHODS

An independent confederation of customers of the same EHR platform proposed and guided the development of a program that leverages native EHR features to allow customers to securely contribute de-identified data regarding the prevalence of asthma and rate of asthma-associated emergency department visits to a vendor-managed repository. Data were stratified by state, age, sex, race, and ethnicity. Results were qualitatively compared with national survey-based estimates.

RESULTS

The program accumulated information from 100 million health records from over 130 healthcare systems in the United States over its first 14 months. All states were represented, with a median coverage of 22.88% of an estimated state's population (interquartile range, 12.05%-42.24%). The mean monthly prevalence of asthma was 5.27 ± 0.11%. The rate of asthma-associated emergency department visits was 1.39 ± 0.08%. Both measures mirrored national survey-based estimates.

DISCUSSION

By organizing the program around native features of a shared EHR platform, we were able to rapidly accumulate population level measures from a sizeable cohort of health records, with representation from every state. The resulting data allowed estimates of asthma prevalence that were comparable to data from traditional epidemiologic surveys at both geographic and demographic levels.

CONCLUSIONS

Our initiative demonstrates the potential of intravendor customer collaboration and highlights an organizational approach that complements other data aggregation efforts seeking to achieve nationwide EHR-based chronic disease surveillance.

Evaluating Population Coverage in a Regional Distributed Data Network: Implications for Electronic Health Record-Based Public Health Surveillance

OBJECTIVE

Electronic health records (EHRs) hold promise as a public health surveillance tool, but questions remain about how EHR patients compare with populations in health and demographic surveys. We compared population characteristics from a regional distributed data network (DDN), which securely and confidentially aggregates EHR data from multiple health care organizations in the same geographic region, with population characteristics from health and demographic surveys.

METHODS

Ten health care organizations participating in a Colorado DDN contributed data for coverage estimation. We aggregated demographic and geographic data from 2017 for patients aged ≥18 residing in 7 counties. We used a cross-sectional design to compare DDN population size, by county, with the following survey-estimated populations: the county population, estimated by the American Community Survey (ACS); residents seeking any health care, estimated by the Colorado Health Access Survey; and residents seeking routine (eg, primary) health care, estimated by the Behavioral Risk Factor Surveillance System. We also compared data on the DDN and survey populations by sex, age group, race/ethnicity, and poverty level to assess surveillance system representativeness.

RESULTS

The DDN population included 609 840 people in 7 counties, corresponding to 25% coverage of the general adult population. Population coverage ranged from 15% to 35% across counties. Demographic distributions generated by DDN and surveys were similar for many groups. Overall, the DDN and surveys assessing care-seeking populations had a higher proportion of women and older adults than the ACS population. The DDN included higher proportions of Hispanic people and people living in high-poverty neighborhoods compared with the surveys.

CONCLUSION

The DDN population is not a random sample of the regional adult population; it is influenced by health care use patterns and organizations participating in the DDN. Strengths and limitations of DDNs complement those of survey-based approaches. The regional DDN is a promising public health surveillance tool.

Developing a Regional Distributed Data Network for Surveillance of Chronic Health Conditions: The Colorado Health Observation Regional Data Service

Electronic health records (EHRs) provide an alternative to traditional public health surveillance surveys and administrative data for measuring the prevalence and impact of chronic health conditions in populations. As the infrastructure for secondary use of EHR data improves, many stakeholders are poised to benefit from data partnerships for regional access to information. Electronic health records can be transformed into a common data model that facilitates data sharing across multiple organizations and allows data to be used for surveillance. The Colorado Health Observation Regional Data Service, a regional distributed data network, has assembled diverse data partnerships, flexible infrastructure, and transparent governance practices to better understand the health of communities through EHR-based, public health surveillance. This article describes attributes of regional distributed data networks using EHR data and the history and design of Colorado Health Observation Regional Data Service as an emerging public health surveillance tool for chronic health conditions. Colorado Health Observation Regional Data Service and our experience may serve as a model for other regions interested in similar surveillance efforts. While benefits from EHR-based surveillance are described, a number of technology, partnership, and value proposition challenges remain.

Prevalence estimation by joint use of big data and health survey: a demonstration study using electronic health records in New York city

Background

Electronic Health Records (EHR) has been increasingly used as a tool to monitor population health. However, subject-level errors in the records can yield biased estimates of health indicators. There is an urgent need for methods to estimate the prevalence of health indicators using large and real-time EHR while correcting the potential bias.

Methods

We demonstrate joint analyses of EHR and a smaller gold-standard health survey. We first adopted Mosteller’s method that pools two estimators, among which one is potentially biased. It only requires knowing the prevalence estimates from two data sources and their standard errors. Then, we adopted the method of Schenker et al., which uses multiple imputations of subject-level health outcomes that are missing for the subjects in EHR. This procedure requires information to link some subjects between two sources and modeling the mechanism of misclassification in EHR as well as modeling inclusion probabilities to both sources.

Results

In a simulation study, both estimators yielded negligible bias even when EHR was biased. They performed as well as health survey estimator when EHR bias was large and better than health survey estimator when EHR bias was moderate. It may be challenging to model the misclassification mechanism in real data for the subject-level imputation estimator. We illustrated the methods analyzing six health indicators from 2013 to 14 NYC HANES and the 2013 NYC Macroscope, and a study that linked some subjects in both data sources.

Conclusions

When a small gold-standard health survey exists, it can serve as a safeguard against potential bias in EHR through the joint analysis of the two sources.

Defining Major Depressive Disorder Cohorts Using the EHR: Multiple Phenotypes Based on ICD-9 Codes and Medication Orders

Background

Major Depressive Disorder (MDD) is one of the most common mental illnesses and a leading cause of disability worldwide. Electronic Health Records (EHR) allow researchers to conduct unprecedented large-scale observational studies investigating MDD, its disease development and its interaction with other health outcomes. While there exist methods to classify patients as clear cases or controls, given specific data requirements, there are presently no simple, generalizable, and validated methods to classify an entire patient population into varying groups of depression likelihood and severity.

Methods

We have tested a simple, pragmatic electronic phenotype algorithm that classifies patients into one of five mutually exclusive, ordinal groups, varying in depression phenotype. Using data from an integrated health system on 278,026 patients from a 10-year study period we have tested the convergent validity of these constructs using measures of external validation, including patterns of psychiatric prescriptions, symptom severity, indicators of suicidality, comorbidity, mortality, health care utilization, and polygenic risk scores for MDD.

Results

We found consistent patterns of increasing morbidity and/or adverse outcomes across the five groups, providing evidence for convergent validity.

Limitations

The study population is from a single rural integrated health system which is predominantly white, possibly limiting its generalizability.

Conclusion

Our study provides initial evidence that a simple algorithm, generalizable to most EHR data sets, provides categories with meaningful face and convergent validity that can be used for stratification of an entire patient population.

Geospatial Monitoring of Body Mass Index: Use of Electronic Health Record Data Across Health Care Systems

OBJECTIVES

The Colorado BMI Monitoring System was developed to assess geographic (ie, census tract) patterns of obesity prevalence rates among children and adults in the Denver-metropolitan region. This project also sought to assess the feasibility of a surveillance system that integrates data across multiple health care and governmental organizations.

MATERIALS AND METHODS

We extracted data on height and weight measures, obtained through routine clinical care, from electronic health records (EHRs) at multiple health care sites. We selected sites from 5 Denver health care systems and collected data from visits that occurred between January 1, 2013, and December 31, 2015. We produced shaded maps showing observed obesity prevalence rates by census tract for various geographic regions across the Denver-metropolitan region.

RESULTS

We identified clearly distinguishable areas by higher rates of obesity among children than among adults, with several pockets of lower body mass index. Patterns for adults were similar to patterns for children: the highest obesity prevalence rates were concentrated around the central part of the metropolitan region. Obesity prevalence rates were moderately higher along the western and northern areas than in other parts of the study region.

PRACTICE IMPLICATIONS

The Colorado BMI Monitoring System demonstrates the feasibility of combining EHRs across multiple systems for public health and research. Challenges include ensuring de-duplication across organizations and ensuring that geocoding is performed in a consistent way that does not pose a risk for patient privacy.

Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records

INTRODUCTION

Increasing adoption of electronic health record (EHR) systems by health care providers presents an opportunity for EHR-based population health surveillance. EHR data, however, may be subject to measurement error because of factors such as data entry errors and lack of documentation by physicians. We investigated the use of a calibration model to reduce bias of prevalence estimates from the New York City (NYC) Macroscope, an EHR-based surveillance system.

METHODS

We calibrated 6 health indicators to the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) data: hypertension, diabetes, smoking, obesity, influenza vaccination, and depression. We classified indicators into having low measurement error or high measurement error on the basis of whether the proportion of misclassification (ie, false-negative or false-positive cases) was greater than 15% in 190 reviewed charts. We compared bias (ie, absolute difference between NYC Macroscope estimates and NYC HANES estimates) before and after calibration.

RESULTS

The health indicators with low measurement error had the same bias after calibration as before calibration (diabetes, 2.5 percentage points; smoking, 2.5 percentage points; obesity, 3.5 percentage points; hypertension, 1.1 percentage points). For indicators with high measurement error, bias decreased from 10.8 to 2.5 percentage points for depression, and from 26.7 to 8.4 percentage points for influenza vaccination.

CONCLUSION

The calibration model has the potential to reduce bias of prevalence estimates from EHR-based surveillance systems for indicators with high measurement errors. Further research is warranted to assess the utility of the current calibration model for other EHR data and additional indicators.

Electronic health record case studies to advance environmental public health tracking

Data from traditional public health surveillance systems can have some limitations, e.g., timeliness, geographic level, and amount of data accessible. Electronic health records (EHRs) could present an opportunity to supplement current sources of routinely collected surveillance data. The National Environmental Public Health Tracking Program (Tracking Program) sought to explore the use of EHRs for advancing environmental public health surveillance practices. The Tracking Program funded four state/local health departments to obtain and pilot the use of EHR data to address several issues including the challenges and technical requirements for accessing EHR data, and the core data elements required to integrate EHR data within their departments' Tracking Programs. The results of these pilot projects highlighted the potential of EHR data for public health surveillance of rare diseases that may lack comprehensive registries, and surveillance of prevalent health conditions or risk factors for health outcomes at a finer geographic level. EHRs therefore, may have potential to supplement traditional sources of public health surveillance data.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms

Introduction:

The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems.

Methods:

We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013–14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data.

Results:

Obesity and diabetes indicators had moderate to high sensitivity (0.81–0.96) and high specificity (0.94–0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78–0.90) and moderate to high specificity (0.88–0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use.

Discussion:

Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed.

Conclusion:

Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

State and Local Chronic Disease Surveillance Using Electronic Health Record Systems

OBJECTIVES

To assess the feasibility of chronic disease surveillance using distributed analysis of electronic health records and to compare results with Behavioral Risk Factor Surveillance System (BRFSS) state and small-area estimates.

METHODS

We queried the electronic health records of 3 independent Massachusetts-based practice groups using a distributed analysis tool called MDPHnet to measure the prevalence of diabetes, asthma, smoking, hypertension, and obesity in adults for the state and 13 cities. We adjusted observed rates for age, gender, and race/ethnicity relative to census data and compared them with BRFSS state and small-area estimates.

RESULTS

The MDPHnet population under surveillance included 1 073 545 adults (21.8% of the state adult population). MDPHnet and BRFSS state-level estimates were similar: 9.4% versus 9.7% for diabetes, 10.0% versus 12.0% for asthma, 13.5% versus 14.7% for smoking, 26.3% versus 29.6% for hypertension, and 22.8% versus 23.8% for obesity. Correlation coefficients for MDPHnet versus BRFSS small-area estimates ranged from 0.890 for diabetes to 0.646 for obesity.

CONCLUSIONS

Chronic disease surveillance using electronic health record data is feasible and generates estimates comparable with BRFSS state and small-area estimates.

Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records

INTRODUCTION

Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples.

METHODS

We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures.

RESULTS

A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins.

CONCLUSION

A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance

With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013-2014 NYC Health and Nutrition Examination Survey. NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators. We discuss NYC's experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.

Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study

Introduction:

Electronic health records (EHRs) can potentially extend chronic disease surveillance, but few EHR-based initiatives tracking population-based metrics have been validated for accuracy. We designed a new EHR-based population health surveillance system for New York City (NYC) known as NYC Macroscope. This report is the third in a 3-part series describing the development and validation of that system. The first report describes governance and technical infrastructure underlying the NYC Macroscope. The second report describes validation methods and presents validation results for estimates of obesity, smoking, depression and influenza vaccination. In this third paper we present validation findings for metabolic indicators (hypertension, hyperlipidemia, diabetes).

Methods:

We compared EHR-based estimates to those from a gold standard surveillance source - the 2013–2014 NYC Health and Nutrition Examination Survey (NYC HANES) - overall and stratified by sex and age group, using the two one-sided test of equivalence and other validation criteria.

Results:

EHR-based hypertension prevalence estimates were highly concordant with NYC HANES estimates. Diabetes prevalence estimates were highly concordant when measuring diagnosed diabetes but less so when incorporating laboratory results. Hypercholesterolemia prevalence estimates were less concordant overall. Measures to assess treatment and control of the 3 metabolic conditions performed poorly.

Discussion:

While indicator performance was variable, findings here confirm that a carefully constructed EHR-based surveillance system can generate prevalence estimates comparable to those from gold-standard examination surveys for certain metabolic conditions such as hypertension and diabetes.

Conclusions:

Standardized EHR metrics have potential utility for surveillance at lower annual costs than surveys, especially as representativeness of contributing clinical practices to EHR-based surveillance systems increases.