Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

The validity of electronic health data for measuring smoking status: a systematic review and meta-analysis

BACKGROUND

Smoking is a risk factor for many chronic diseases. Multiple smoking status ascertainment algorithms have been developed for population-based electronic health databases such as administrative databases and electronic medical records (EMRs). Evidence syntheses of algorithm validation studies have often focused on chronic diseases rather than risk factors. We conducted a systematic review and meta-analysis of smoking status ascertainment algorithms to describe the characteristics and validity of these algorithms.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed. We searched articles published from 1990 to 2022 in EMBASE, MEDLINE, Scopus, and Web of Science with key terms such as validity, administrative data, electronic health records, smoking, and tobacco use. The extracted information, including article characteristics, algorithm characteristics, and validity measures, was descriptively analyzed. Sources of heterogeneity in validity measures were estimated using a meta-regression model. Risk of bias (ROB) in the reviewed articles was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool.

RESULTS

The initial search yielded 2086 articles; 57 were selected for review and 116 algorithms were identified. Almost three-quarters (71.6%) of algorithms were based on EMR data. The algorithms were primarily constructed using diagnosis codes for smoking-related conditions, although prescription medication codes for smoking treatments were also adopted. About half of the algorithms were developed using machine-learning models. The pooled estimates of positive predictive value, sensitivity, and specificity were 0.843, 0.672, and 0.918 respectively. Algorithm sensitivity and specificity were highly variable and ranged from 3 to 100% and 36 to 100%, respectively. Model-based algorithms had significantly greater sensitivity (p = 0.006) than rule-based algorithms. Algorithms for EMR data had higher sensitivity than algorithms for administrative data (p = 0.001). The ROB was low in most of the articles (76.3%) that underwent the assessment.

CONCLUSIONS

Multiple algorithms using different data sources and methods have been proposed to ascertain smoking status in electronic health data. Many algorithms had low sensitivity and positive predictive value, but the data source influenced their validity. Algorithms based on machine-learning models for multiple linked data sources have improved validity.

Validation of a geospatial aggregation method for congressional districts and other US administrative geographies

Stakeholders need data on health and drivers of health parsed to the boundaries of essential policy-relevant geographies. US Congressional Districts are an example of a policy-relevant geography which generally lack health data. One strategy to generate Congressional District heath data metric estimates is to aggregate estimates from other geographies, for example, from counties or census tracts to Congressional Districts. Doing so requires several methodological decisions. We refine a method to aggregate health metric estimates from one geography to another, using a population weighted approach. The method's accuracy is evaluated by comparing three aggregated metric estimates to metric estimates from the US Census American Community Survey for the same years: Broadband Access, High School Completion, and Unemployment. We then conducted four sensitivity analyses testing: the effect of aggregating counts vs. percentages; impacts of component geography size and data missingness; and extent of population overlap between component and target geographies. Aggregated estimates were very similar to estimates for identical metrics drawn directly from the data source. Sensitivity analyses suggest the following best practices for Congressional district-based metrics: utilizing smaller, more plentiful geographies like census tracts as opposed to larger, less plentiful geographies like counties, despite potential for less stable estimates in smaller geographies; favoring geographies with higher percentage population overlap.

Utilization of Electronic Health Records for Chronic Disease Surveillance: A Systematic Literature Review

This study reviews the current utilization of electronic health records (EHRs) for chronic disease surveillance, discusses approaches that are used in obtaining EHR-derived disease prevalence estimates, and identifies health indicators that have been studied using EHR-based surveillance methods. PubMed was searched for relevant keywords: (electronic health records [Title/Abstract] AND surveillance [Title/Abstract]) OR (electronic medical records [Title/Abstract] AND surveillance [Title/Abstract]). Articles were assessed based on detailed inclusion and exclusion criteria and organized by common themes, as per the PRISMA review protocol. The study period was limited to 2015-2021 due to the wider adoption of EHR in the U.S. only since 2015. The review included only US studies and only those that focused on chronic disease surveillance. 17 studies were included in the review. The most common approaches the review identified focused on validating EHR-derived estimates against those from traditional national surveys. The most studied conditions were diabetes, obesity, and hypertension. The majority of reviewed studies demonstrated comparable prevalence estimates with traditional population health surveillance surveys. The most common approach for the estimation of chronic disease conditions was to use small-area estimation by geographic patterns, neighborhoods, or census tracts. The use of EHR-based surveillance systems for public health purposes is feasible, and the population health estimates appear comparable to those obtained through traditional surveillance surveys. The application of EHRs for public health surveillance appears promising and could offer a real-time alternative to traditional surveillance methods. A timely assessment of population health at local and regional levels would ensure a more targeted allocation of public health and healthcare resources as well as more effective intervention and prevention initiatives.

Global and regional prevalence of disabilities among children and adolescents: Analysis of findings from global health databases

Objective

The United Nations' Sustainable Development Goals (SDGs) require population-based data on children with disabilities to inform global policies and intervention programs. We set out to compare the prevalence estimates of disabilities among children and adolescents younger than 20 years as reported by the world's leading organizations for global health statistics.

Methods

We purposively searched the disability reports and databases of the United Nations Children's Fund (UNICEF), the World Health Organization (WHO), the World Bank and the Global Burden of Diseases (GBD) Study. We analyzed the latest disability data reported by these organizations since 2015. We examined the methodologies adopted in generating the reported prevalence estimates and evaluated the degree of agreement among the data sources using Welch's test of statistical difference, and the two one-sided t-test (TOST) for statistical equivalence.

Results

Only UNICEF and GBD provided the most comprehensive prevalence estimates of disabilities in children and adolescents. Globally, UNICEF estimated that 28.9 million (4.3%) children aged 0-4 years, 207.4 million (12.5%) children aged 5-17 years and 236.4 million (10.1%) children aged 0-17 years have moderate-to-severe disabilities based on household surveys of child functional status. Using the UNICEF estimated prevalence of 10.1%, approximately 266 million children aged 0-19 years are expected to have moderate-to-severe disabilities. In contrast, GBD 2019 estimated that 49.8 million (7.5%) children aged under 5 years, 241.5 million (12.6%) children aged 5-19 years and 291.3 million (11.3%) children younger than 20 years have mild-to-severe disabilities. In both databases, Sub-Saharan Africa and South Asia accounted for more than half of children with disabilities. A comparison of the UNICEF and GBD estimates showed that the overall mean prevalence estimates for children under 5 years were statistically different and not statistically equivalent based on ±3 percentage-point margin. However, the prevalence estimates for children 5-19 years and < 20 years were not statistically different and were statistically equivalent.

Conclusion

Prevalence estimates of disabilities among children and adolescents generated using either functional approach or statistical modeling appear to be comparable and complementary. Improved alignment of the age-groups, thresholds of disability and the estimation process across databases, particularly among children under 5 years should be considered. Children and adolescents with disabilities will be well-served by a variety of complementary data sources to optimize their health and well-being as envisioned in the SDGs.

The use of electronic health records to inform cancer surveillance efforts: a scoping review and test of indicators for public health surveillance of cancer prevention and control

INTRODUCTION

State cancer prevention and control programs rely on public health surveillance data to set objectives to improve cancer prevention and control, plan interventions, and evaluate state-level progress towards achieving those objectives. The goal of this project was to evaluate the validity of using electronic health records (EHRs) based on common data model variables to generate indicators for surveillance of cancer prevention and control for these public health programs.

METHODS

Following the methodological guidance from the PRISMA Extension for Scoping Reviews, we conducted a literature scoping review to assess how EHRs are used to inform cancer surveillance. We then developed 26 indicators along the continuum of the cascade of care, including cancer risk factors, immunizations to prevent cancer, cancer screenings, quality of initial care after abnormal screening results, and cancer burden. Indicators were calculated within a sample of patients from the New York City (NYC) INSIGHT Clinical Research Network using common data model EHR data and were weighted to the NYC population using post-stratification. We used prevalence ratios to compare these estimates to estimates from the raw EHR of NYU Langone Health to assess quality of information within INSIGHT, and we compared estimates to results from existing surveillance sources to assess validity.

RESULTS

Of the 401 identified articles, 15% had a study purpose related to surveillance. Our indicator comparisons found that INSIGHT EHR-based measures for risk factor indicators were similar to estimates from external sources. In contrast, cancer screening and vaccination indicators were substantially underestimated as compared to estimates from external sources. Cancer screenings and vaccinations were often recorded in sections of the EHR that were not captured by the common data model. INSIGHT estimates for many quality-of-care indicators were higher than those calculated using a raw EHR.

CONCLUSION

Common data model EHR data can provide rich information for certain indicators related to the cascade of care but may have substantial biases for others that limit their use in informing surveillance efforts for cancer prevention and control programs.

Using Electronic Health Records to understand the population of local children captured in a large health system in Durham County, NC, USA, and implications for population health research

Although local policies aimed at reducing childhood health inequities can benefit from local data, sample size constraints in population representative health surveys often prevent rigorous evaluations of child health disparities and health care patterns at local levels. Electronic Health Records (EHRs) offer a possible solution as they contain large amounts of information on pediatric patients within a health system. In this paper, we consider the suitability of using EHRs from a large health system to study local children's health by evaluating the extent to which the EHRs capture the county's child population. First, we compare the demographic characteristics of Duke University Health System pediatric patients who live in Durham County, NC (USA) to the child population estimates in the 2015-2019 American Community Survey. We then examine geographic variation in census tract rates of children captured in the EHR data and estimate negative binomial models to assess how tract characteristics are associated with these rates. We also perform these analyses for the subset of pediatric patients who have a well-child encounter. We find that the demographic characteristics of pediatric patients captured by the EHRs are similar to those of the county's child population. Although the county rate of children captured in the EHRs is high, there is variation across census tracts. On average, census tracts with higher concentrations of non-Hispanic Black residents have lower capture rates and tracts with higher concentrations of poverty have higher capture rates, with the poorest tracts showing the largest racial gap in rates of children captured by EHRs. Our findings suggest that EHRs from a large health system can be used to assess children's population health, but that EHR-based evaluations of children's health disparities and health care patterns should account for differences in who is captured by the EHRs based on census tract characteristics.

Reinforced risk prediction with budget constraint using irregularly measured data from electronic health records

Uncontrolled glycated hemoglobin (HbA1c) levels are associated with adverse events among complex diabetic patients. These adverse events present serious health risks to affected patients and are associated with significant financial costs. Thus, a high-quality predictive model that could identify high-risk patients so as to inform preventative treatment has the potential to improve patient outcomes while reducing healthcare costs. Because the biomarker information needed to predict risk is costly and burdensome, it is desirable that such a model collect only as much information as is needed on each patient so as to render an accurate prediction. We propose a sequential predictive model that uses accumulating patient longitudinal data to classify patients as: high-risk, low-risk, or uncertain. Patients classified as high-risk are then recommended to receive preventative treatment and those classified as low-risk are recommended to standard care. Patients classified as uncertain are monitored until a high-risk or low-risk determination is made. We construct the model using claims and enrollment files from Medicare, linked with patient Electronic Health Records (EHR) data. The proposed model uses functional principal components to accommodate noisy longitudinal data and weighting to deal with missingness and sampling bias. The proposed method demonstrates higher predictive accuracy and lower cost than competing methods in a series of simulation experiments and application to data on complex patients with diabetes.

Health and Health Determinant Metrics for Cities: A Comparison of County and City-Level Data

We evaluated whether using county-level data to characterize public health measures in cities biases the characterization of city populations. We compared 4 public health and sociodemographic measures in 447 US cities (percent of children living in poverty, percent of non-Hispanic Black population, age-adjusted cardiovascular disease mortality, life expectancy at birth) to the same measures calculated for counties that contain those cities. We found substantial and highly variable city-county differences within and across metrics, which suggests that use of county data to proxy city measures could hamper accurate allocation of public health resources and appreciation of the urgency of public health needs in specific locales.

Evaluating Population Coverage in a Regional Distributed Data Network: Implications for Electronic Health Record-Based Public Health Surveillance

OBJECTIVE

Electronic health records (EHRs) hold promise as a public health surveillance tool, but questions remain about how EHR patients compare with populations in health and demographic surveys. We compared population characteristics from a regional distributed data network (DDN), which securely and confidentially aggregates EHR data from multiple health care organizations in the same geographic region, with population characteristics from health and demographic surveys.

METHODS

Ten health care organizations participating in a Colorado DDN contributed data for coverage estimation. We aggregated demographic and geographic data from 2017 for patients aged ≥18 residing in 7 counties. We used a cross-sectional design to compare DDN population size, by county, with the following survey-estimated populations: the county population, estimated by the American Community Survey (ACS); residents seeking any health care, estimated by the Colorado Health Access Survey; and residents seeking routine (eg, primary) health care, estimated by the Behavioral Risk Factor Surveillance System. We also compared data on the DDN and survey populations by sex, age group, race/ethnicity, and poverty level to assess surveillance system representativeness.

RESULTS

The DDN population included 609 840 people in 7 counties, corresponding to 25% coverage of the general adult population. Population coverage ranged from 15% to 35% across counties. Demographic distributions generated by DDN and surveys were similar for many groups. Overall, the DDN and surveys assessing care-seeking populations had a higher proportion of women and older adults than the ACS population. The DDN included higher proportions of Hispanic people and people living in high-poverty neighborhoods compared with the surveys.

CONCLUSION

The DDN population is not a random sample of the regional adult population; it is influenced by health care use patterns and organizations participating in the DDN. Strengths and limitations of DDNs complement those of survey-based approaches. The regional DDN is a promising public health surveillance tool.

Evaluating the Representativeness of US Centricity Electronic Medical Records With Reports From the Centers for Disease Control and Prevention: Comparative Study on Office Visits and Cardiometabolic Conditions

Background

Electronic medical record (EMR)–based clinical and epidemiological research has dramatically increased over the last decade, although establishing the generalizability of such big databases for conducting epidemiological studies has been an ongoing challenge. To draw meaningful inferences from such studies, it is essential to fully understand the characteristics of the underlying population and potential biases in EMRs.

Objective

This study aimed to assess the generalizability and representativity of the widely used US Centricity Electronic Medical Record (CEMR), a primary and ambulatory care EMR for population health research, using data from the National Ambulatory Medical Care Surveys (NAMCS) and the National Health and Nutrition Examination Surveys (NHANES).

Methods

The number of office visits reported in the NAMCS, designed to meet the need for objective and reliable information about the provision and the use of ambulatory medical care services, was compared with similar data from the CEMR. The distribution of major cardiometabolic diseases in the NHANES, designed to assess the health and nutritional status of adults and children in the United States, was compared with similar data from the CEMR.

Results

Gender and ethnicity distributions were similar between the NAMCS and the CEMR. Younger patients (aged <15 years) were underrepresented in the CEMR compared with the NAMCS. The number of office visits per 100 persons per year was similar: 277.9 (95% CI 259.3-296.5) in the NAMCS and 284.6 (95% CI 284.4-284.7) in the CEMR. However, the number of visits for males was significantly higher in the CEMR (CEMR: 270.8 and NAMCS: 239.0). West and South regions were underrepresented and overrepresented, respectively, in the CEMR. The overall prevalence of diabetes along with age and gender distribution was similar in the CEMR and the NHANES: overall prevalence, 10.1% and 9.7%; male, 11.5% and 10.8%; female, 9.1% and 8.8%; age 20 to 40 years, 2.5% and 1.8%; and age 40 to 60 years, 9.4% and 11.1%, respectively. The prevalence of obesity was similar: 42.1% and 39.6%, with similar age and female distribution (41.5% and 41.1%) but different male distribution (42.7% and 37.9%). The overall prevalence of high cholesterol along with age and female distribution was similar in the CEMR and the NHANES: overall prevalence, 12.4% and 12.4%; and female, 14.8% and 13.2%, respectively. The overall prevalence of hypertension was significantly higher in the CEMR (33.5%) than in the NHANES (95% CI: 27.0%-31.0%).

Conclusions

The distribution of major cardiometabolic diseases in the CEMR is comparable with the national survey results. The CEMR represents the general US population well in terms of office visits and major chronic conditions, whereas the potential subgroup differences in terms of age and gender distribution and prevalence may differ and, therefore, should be carefully taken care of in future studies.

Prevalence estimation by joint use of big data and health survey: a demonstration study using electronic health records in New York city

Background

Electronic Health Records (EHR) has been increasingly used as a tool to monitor population health. However, subject-level errors in the records can yield biased estimates of health indicators. There is an urgent need for methods to estimate the prevalence of health indicators using large and real-time EHR while correcting the potential bias.

Methods

We demonstrate joint analyses of EHR and a smaller gold-standard health survey. We first adopted Mosteller’s method that pools two estimators, among which one is potentially biased. It only requires knowing the prevalence estimates from two data sources and their standard errors. Then, we adopted the method of Schenker et al., which uses multiple imputations of subject-level health outcomes that are missing for the subjects in EHR. This procedure requires information to link some subjects between two sources and modeling the mechanism of misclassification in EHR as well as modeling inclusion probabilities to both sources.

Results

In a simulation study, both estimators yielded negligible bias even when EHR was biased. They performed as well as health survey estimator when EHR bias was large and better than health survey estimator when EHR bias was moderate. It may be challenging to model the misclassification mechanism in real data for the subject-level imputation estimator. We illustrated the methods analyzing six health indicators from 2013 to 14 NYC HANES and the 2013 NYC Macroscope, and a study that linked some subjects in both data sources.

Conclusions

When a small gold-standard health survey exists, it can serve as a safeguard against potential bias in EHR through the joint analysis of the two sources.

Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

BACKGROUND

A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training.

OBJECTIVE

The goal of this study was to assess the completeness of electronic patient records in a student clinic.

METHODS

Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases "dry socket" and "alveolar osteitis" in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items.

RESULTS

Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement).

CONCLUSIONS

There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.

Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records

INTRODUCTION

Increasing adoption of electronic health record (EHR) systems by health care providers presents an opportunity for EHR-based population health surveillance. EHR data, however, may be subject to measurement error because of factors such as data entry errors and lack of documentation by physicians. We investigated the use of a calibration model to reduce bias of prevalence estimates from the New York City (NYC) Macroscope, an EHR-based surveillance system.

METHODS

We calibrated 6 health indicators to the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) data: hypertension, diabetes, smoking, obesity, influenza vaccination, and depression. We classified indicators into having low measurement error or high measurement error on the basis of whether the proportion of misclassification (ie, false-negative or false-positive cases) was greater than 15% in 190 reviewed charts. We compared bias (ie, absolute difference between NYC Macroscope estimates and NYC HANES estimates) before and after calibration.

RESULTS

The health indicators with low measurement error had the same bias after calibration as before calibration (diabetes, 2.5 percentage points; smoking, 2.5 percentage points; obesity, 3.5 percentage points; hypertension, 1.1 percentage points). For indicators with high measurement error, bias decreased from 10.8 to 2.5 percentage points for depression, and from 26.7 to 8.4 percentage points for influenza vaccination.

CONCLUSION

The calibration model has the potential to reduce bias of prevalence estimates from EHR-based surveillance systems for indicators with high measurement errors. Further research is warranted to assess the utility of the current calibration model for other EHR data and additional indicators.

Using the emergency department to investigate smoking in young adults

PURPOSE

Smoking in young adults identifies the population at risk for future tobacco-related disease. We investigated smoking in a young adult population and within high-risk groups using emergency department (ED) data in a metropolitan area.

METHODS

Using the electronic health record, we performed a retrospective study of smoking in adults aged 18-30 years presenting to the ED.

RESULTS

Smoking status was available for 55,777 subjects (90.9% of the total ED cohort); 60.8% were women, 55.0% were black, 35.3% were white, and 8.1% were Hispanic; 34.4% were uninsured. Most smokers used cigarettes (95.1%). Prevalence of current smoking was 21.7% for women and 42.5% for men. The electronic health record contains data about diagnosis and social history that can be used to investigate smoking status for high-risk populations. Smoking prevalence was highest for substance use disorder (58.0%), psychiatric illness (41.3%) and alcohol use (39.1%), and lowest for pregnancy (13.5%). In multivariable analyses, male gender, white race, lack of health insurance, alcohol use, and illicit drug use were independently associated with smoking. Smoking risk among alcohol and drug users varied by gender, race, and/or age.

CONCLUSIONS

The ED provides access to a large, demographically diverse population, and supports investigation of smoking risk in young adults.

Value of Neighborhood Socioeconomic Status in Predicting Risk of Outcomes in Studies That Use Electronic Health Record Data

IMPORTANCE

Data from electronic health records (EHRs) are increasingly used for risk prediction. However, EHRs do not reliably collect sociodemographic and neighborhood information, which has been shown to be associated with health. The added contribution of neighborhood socioeconomic status (nSES) in predicting health events is unknown and may help inform population-level risk reduction strategies.

OBJECTIVE

To quantify the association of nSES with adverse outcomes and the value of nSES in predicting the risk of adverse outcomes in EHR-based risk models.

DESIGN, SETTING, AND PARTICIPANTS

Cohort study in which data from 90 097 patients 18 years or older in the Duke University Health System and Lincoln Community Health Center EHR from January 1, 2009, to December 31, 2015, with at least 1 health care encounter and residence in Durham County, North Carolina, in the year prior to the index date were linked with census tract data to quantify the association between nSES and the risk of adverse outcomes. Machine learning methods were used to develop risk models and determine how adding nSES to EHR data affects risk prediction. Neighborhood socioeconomic status was defined using the Agency for Healthcare Research and Quality SES index, a weighted measure of multiple indicators of neighborhood deprivation.

MAIN OUTCOMES AND MEASURES

Outcomes included use of health care services (emergency department and inpatient and outpatient encounters) and hospitalizations due to accidents, asthma, influenza, myocardial infarction, and stroke.

RESULTS

Among the 90 097 patients in the training set of the study (57 507 women and 32 590 men; mean [SD] age, 47.2 [17.7] years) and the 122 812 patients in the testing set of the study (75 517 women and 47 295 men; mean [SD] age, 46.2 [17.9] years), those living in neighborhoods with lower nSES had a shorter time to use of emergency department services and inpatient encounters, as well as a shorter time to hospitalizations due to accidents, asthma, influenza, myocardial infarction, and stroke. The predictive value of nSES varied by outcome of interest (C statistic ranged from 0.50 to 0.63). When added to EHR variables, nSES did not improve predictive performance for any health outcome.

CONCLUSIONS AND RELEVANCE

Social determinants of health, including nSES, are associated with the health of a patient. However, the results of this study suggest that information on nSES may not contribute much more to risk prediction above and beyond what is already provided by EHR data. Although this result does not mean that integrating social determinants of health into the EHR has no benefit, researchers may be able to use EHR data alone for population risk assessment.

Adverse Childhood Experiences and Mental and Behavioral Health Conditions During Pregnancy: The Role of Resilience

INTRODUCTION

Little is known about how exposure to adverse childhood experiences (ACEs) and protective factors, such as resilience, influence prenatal mental and behavioral health. This study examined associations between exposure to ACEs and mental and behavioral health during pregnancy overall and among women with high versus low levels of resilience.

MATERIALS AND METHODS

Women in two Kaiser Permanente Northern California medical centers were screened for ACEs and resilience during prenatal care (∼14-23 weeks of gestation; N = 355). Multivariable logistic regression analyses examined associations between ACEs and prenatal mental and behavioral health conditions overall and for women with low (≤32) versus high (>32) resilience on the 10-item Connor-Davidson Resilience Scale.

RESULTS

Overall, 54% of women reported 0 ACEs, 28% 1-2 ACEs, and 18% 3+ ACEs. Relative to women with 0 ACEs, those with 1-2 ACEs had higher odds of an anxiety or depressive disorder and intimate partner violence (IPV) (odds ratios [ORs] 2.42-3.12, p < 0.05), and those with 3+ ACEs had higher odds of an anxiety or depressive disorder, depression symptoms, and IPV (ORs 3.08-4.71, p < 0.05). In stratified analyses by high (56%) and low (44%) resilience, having one or more ACEs (vs. 0 ACEs) was only associated with worse mental and behavioral health in women with low resilience.

CONCLUSIONS

ACEs predicted mental and behavioral health conditions among pregnant women, and associations were the strongest among women with low levels of current resilience. Longitudinal research is needed to understand the causal mechanisms underlying these associations.

Prevalence of Multiple Chronic Conditions Among Older Adults in Florida and the United States: Comparative Analysis of the OneFlorida Data Trust and National Inpatient Sample

Background

Older patients with multiple chronic conditions are often faced with increased health care needs and subsequent higher medical costs, posing significant financial burden to patients, their caregivers, and the health care system. The increasing adoption of electronic health record systems and the proliferation of clinical data offer new opportunities for prevalence studies and for population health assessment. The last few years have witnessed an increasing number of clinical research networks focused on building large collections of clinical data from electronic health records and claims to make it easier and less costly to conduct clinical research.

Objective

The aim of this study was to compare the prevalence of common chronic conditions and multiple chronic conditions in older adults between Florida and the United States using data from the OneFlorida Clinical Research Consortium and the Healthcare Cost and Utilization Project (HCUP) National Inpatient Sample (NIS).

Methods

We first analyzed the basic demographic characteristics of the older adults in 3 datasets—the 2013 OneFlorida data, the 2013 HCUP NIS data, and the combined 2012 to 2016 OneFlorida data. Then we analyzed the prevalence of each of the 25 chronic conditions in each of the 3 datasets. We stratified the analysis of older adults with hypertension, the most prevalent condition. Additionally, we examined trends (ie, overall trends and then by age, race, and gender) in the prevalence of discharge records representing multiple chronic conditions over time for the OneFlorida (2012-2016) and HCUP NIS cohorts (2003-2013).

Results

The rankings of the top 10 prevalent conditions are the same across the OneFlorida and HCUP NIS datasets. The most prevalent multiple chronic conditions of 2 conditions among the 3 datasets were—hyperlipidemia and hypertension; hypertension and ischemic heart disease; diabetes and hypertension; chronic kidney disease and hypertension; anemia and hypertension; and hyperlipidemia and ischemic heart disease. We observed increasing trends in multiple chronic conditions in both data sources.

Conclusions

The results showed that chronic conditions and multiple chronic conditions are prevalent in older adults across Florida and the United States. Even though slight differences were observed, the similar estimates of prevalence of chronic conditions and multiple chronic conditions across OneFlorida and HCUP NIS suggested that clinical research data networks such as OneFlorida, built from heterogeneous data sources, can provide rich data resources for conducting large-scale secondary data analyses.

Documentation of e-cigarette use and associations with smoking from 2012 to 2015 in an integrated healthcare delivery system

It is unclear whether use of electronic nicotine delivery systems (ENDS) precedes cigarette smoking initiation, relapse, and/or quitting. Healthcare systems with electronic health records (EHRs) provide unique data to examine ENDS use and changes in smoking. We examined the incidence of ENDS use (2012-2015) based on clinician documentation and tested whether EHR documented ENDS use is associated with twelve-month changes in patient smoking status using a matched retrospective cohort design. The sample was Kaiser Permanente Northern California (KPNC) patients aged ≥12 with documented ENDS use (N = 7926); 57% were current smokers, 35% former smokers, and 8% never-smokers. ENDS documentation incidence peaked in 2014 for current and former smokers and in 2015 for never-smokers. We matched patients with documented ENDS use to KPNC patients without documented ENDS use (N = 7926) on age, sex, race/ethnicity, and smoking status. Documented ENDS use predicted the likelihood of smoking in the following year. Among current smokers, ENDS use was associated with greater odds of quitting smoking (OR = 1.17, 95%CI = 1.05-1.31). Among former smokers, ENDS use was associated with greater odds of smoking relapse (OR = 1.53, 95%CI = 1.22-1.92). Among never-smokers, ENDS use was associated with greater odds of initiating smoking (OR = 7.41, 95%CI = 3.14-17.5). The overall number of current smokers at 12 months was slightly higher among patients with (N = 3931) versus without (N = 3850) documented ENDS use. Results support both potential harm reduction of ENDS use (quitting combustibles among current smokers) and potential for harm (relapse to combustibles among former smokers, initiation for never-smokers).

Electronic health record case studies to advance environmental public health tracking

Data from traditional public health surveillance systems can have some limitations, e.g., timeliness, geographic level, and amount of data accessible. Electronic health records (EHRs) could present an opportunity to supplement current sources of routinely collected surveillance data. The National Environmental Public Health Tracking Program (Tracking Program) sought to explore the use of EHRs for advancing environmental public health surveillance practices. The Tracking Program funded four state/local health departments to obtain and pilot the use of EHR data to address several issues including the challenges and technical requirements for accessing EHR data, and the core data elements required to integrate EHR data within their departments' Tracking Programs. The results of these pilot projects highlighted the potential of EHR data for public health surveillance of rare diseases that may lack comprehensive registries, and surveillance of prevalent health conditions or risk factors for health outcomes at a finer geographic level. EHRs therefore, may have potential to supplement traditional sources of public health surveillance data.

Monitoring Depression Rates in an Urban Community: Use of Electronic Health Records

Objectives:

Depression is the most common mental health disorder and mediates outcomes for many chronic diseases. Ability to accurately identify and monitor this condition, at the local level, is often limited to estimates from national surveys. This study sought to compare and validate electronic health record (EHR)-based depression surveillance with multiple data sources for more granular demographic subgroup and subcounty measurements.

Design/Setting:

A survey compared data sources for the ability to provide subcounty (eg, census tract [CT]) depression prevalence estimates. Using 2011-2012 EHR data from 2 large health care providers, and American Community Survey data, depression rates were estimated by CT for Denver County, Colorado. Sociodemographic and geographic (residence) attributes were analyzed and described. Spatial analysis assessed for clusters of higher or lower depression prevalence.

Main Outcome Measure(s):

Depression prevalence estimates by CT.

Results:

National and local survey-based depression prevalence estimates ranged from 7% to 17% but were limited to county level. Electronic health record data provided subcounty depression prevalence estimates by sociodemographic and geographic groups (CT range: 5%-20%). Overall depression prevalence was 13%; rates were higher for women (16% vs men 9%), whites (16%), and increased with age and homeless patients (18%). Areas of higher and lower EHR-based, depression prevalence were identified.

Conclusions:

Electronic health record–based depression prevalence varied by CT, gender, race/ethnicity, age, and living status. Electronic health record–based surveillance complements traditional methods with greater timeliness and granularity. Validation through subcounty-level qualitative or survey approaches should assess accuracy and address concerns about EHR selection bias. Public health agencies should consider the opportunity and evaluate EHR system data as a surveillance tool to estimate subcounty chronic disease prevalence.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms

Introduction:

The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems.

Methods:

We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013–14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data.

Results:

Obesity and diabetes indicators had moderate to high sensitivity (0.81–0.96) and high specificity (0.94–0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78–0.90) and moderate to high specificity (0.88–0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use.

Discussion:

Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed.

Conclusion:

Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

State and Local Chronic Disease Surveillance Using Electronic Health Record Systems

OBJECTIVES

To assess the feasibility of chronic disease surveillance using distributed analysis of electronic health records and to compare results with Behavioral Risk Factor Surveillance System (BRFSS) state and small-area estimates.

METHODS

We queried the electronic health records of 3 independent Massachusetts-based practice groups using a distributed analysis tool called MDPHnet to measure the prevalence of diabetes, asthma, smoking, hypertension, and obesity in adults for the state and 13 cities. We adjusted observed rates for age, gender, and race/ethnicity relative to census data and compared them with BRFSS state and small-area estimates.

RESULTS

The MDPHnet population under surveillance included 1 073 545 adults (21.8% of the state adult population). MDPHnet and BRFSS state-level estimates were similar: 9.4% versus 9.7% for diabetes, 10.0% versus 12.0% for asthma, 13.5% versus 14.7% for smoking, 26.3% versus 29.6% for hypertension, and 22.8% versus 23.8% for obesity. Correlation coefficients for MDPHnet versus BRFSS small-area estimates ranged from 0.890 for diabetes to 0.646 for obesity.

CONCLUSIONS

Chronic disease surveillance using electronic health record data is feasible and generates estimates comparable with BRFSS state and small-area estimates.

Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records

INTRODUCTION

Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples.

METHODS

We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures.

RESULTS

A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins.

CONCLUSION

A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance

With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013-2014 NYC Health and Nutrition Examination Survey. NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators. We discuss NYC's experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.