Use of practice-based research network data to measure neighborhood smoking prevalence

Electronic Health Records Versus Survey Small Area Estimates for Public Health Surveillance

INTRODUCTION

Electronic health records (EHRs) are increasingly being leveraged for public health surveillance. EHR-based small area estimates (SAEs) are often validated by comparison to survey data such as the Behavioral Risk Factor Surveillance System (BRFSS). However, survey and EHR-based SAEs are expected to differ. In this cross-sectional study, SAEs were generated using MDPHnet, a distributed EHR-based surveillance network, for all Massachusetts municipalities and zip code tabulation areas (ZCTAs), compared to BRFSS PLACES SAEs, and reasons for differences explored.

METHODS

This study delineated reasons a priori for how SAEs derived using EHRs may differ from surveys by comparing each strategy's case classification criteria and reviewing the literature. Hypertension, diabetes, obesity, asthma, and smoking EHR-based SAEs for 2021 in all ZCTAs and municipalities in Massachusetts were estimated with Bayesian mixed effects modeling and poststratification in the summer/fall of 2023. These SAEs were compared to BRFSS PLACES SAEs published by the U.S. Centers for Disease Control and Prevention.

RESULTS

Mean prevalence was higher in EHR data versus BRFSS in both municipalities and ZCTAs for all outcomes except asthma. ZCTA and municipal symmetric mean absolute percentages ranged from 12.0 to 38.2% and 13.1 to 39.8%, respectively. There was greater variability in EHR-based SAEs versus BRFSS PLACES in both municipalities and ZCTAs.

CONCLUSIONS

EHR-based SAEs tended to be higher than BRFSS and more variable. Possible explanations include detection of undiagnosed cases and over-classification using EHR data, and under-reporting within BRFSS. Both EHR and survey-based surveillance have strengths and limitations that should inform their preferred uses in public health surveillance.

Vulnerability and Colorectal screening during the pandemic

Objective

Disparities in colorectal cancer (CRC) screening prevalence across United States neighborhoods may reflect social inequities that create barriers to accessing and completing preventive health services. Our objective was to identify whether neighborhood social vulnerability was associated with a change in CRC screening prevalence in Boston neighborhoods during the COVID-19 pandemic.

Methods

Adults ages 50-74 years due for CRC screening who received primary care at one of 35 primary care practices affiliated with Massachusetts General Hospital or Brigham and Women's Hospital (Boston, MA), 3/1/2020 to 3/1/2022. The Social Vulnerability Index (SVI) is an aggregate measure of neighborhood social factors often used by public health authorities to examine neighborhood susceptibility to many health outcomes.

Results

In 2020, 74.9 % of eligible individuals were up to date with CRC screening and this fell to 67.4 % in 2022 (p < 0.001). In 2020, 36.2 % of eligible patients lived in a neighborhood above the 80th percentile of SVI, consistent with high social vulnerability, while the same value was 35.1 % in 2022. There was no association between the change in screening prevalence and SVI: a decrease of 5.5 % screened in neighborhoods with SVI ≤ 80 compared to a decrease of 3.6 % in neighborhoods with SVI > 80 (p = 0.79).

Conclusions

The COVID-19 pandemic equalized the prevalence of CRC screening across Boston-area neighborhoods despite pre-existing geographic disparities in screening prevalence and SVI. Strategies to ensure equitable participation in CRC screening to promote health equity should be considered to promote equitable pandemic recovery.

The use of electronic health records to inform cancer surveillance efforts: a scoping review and test of indicators for public health surveillance of cancer prevention and control

INTRODUCTION

State cancer prevention and control programs rely on public health surveillance data to set objectives to improve cancer prevention and control, plan interventions, and evaluate state-level progress towards achieving those objectives. The goal of this project was to evaluate the validity of using electronic health records (EHRs) based on common data model variables to generate indicators for surveillance of cancer prevention and control for these public health programs.

METHODS

Following the methodological guidance from the PRISMA Extension for Scoping Reviews, we conducted a literature scoping review to assess how EHRs are used to inform cancer surveillance. We then developed 26 indicators along the continuum of the cascade of care, including cancer risk factors, immunizations to prevent cancer, cancer screenings, quality of initial care after abnormal screening results, and cancer burden. Indicators were calculated within a sample of patients from the New York City (NYC) INSIGHT Clinical Research Network using common data model EHR data and were weighted to the NYC population using post-stratification. We used prevalence ratios to compare these estimates to estimates from the raw EHR of NYU Langone Health to assess quality of information within INSIGHT, and we compared estimates to results from existing surveillance sources to assess validity.

RESULTS

Of the 401 identified articles, 15% had a study purpose related to surveillance. Our indicator comparisons found that INSIGHT EHR-based measures for risk factor indicators were similar to estimates from external sources. In contrast, cancer screening and vaccination indicators were substantially underestimated as compared to estimates from external sources. Cancer screenings and vaccinations were often recorded in sections of the EHR that were not captured by the common data model. INSIGHT estimates for many quality-of-care indicators were higher than those calculated using a raw EHR.

CONCLUSION

Common data model EHR data can provide rich information for certain indicators related to the cascade of care but may have substantial biases for others that limit their use in informing surveillance efforts for cancer prevention and control programs.

Neighborhood-Level Chronic Disease Surveillance: Utility of Primary Care Electronic Health Records and Emergency Department Claims Data

CONTEXT

Disease burden may vary substantively across neighborhoods in an urban setting. Yet, data available for monitoring chronic conditions at the neighborhood level are scarce. Large health care data sets have potential to complement population health surveillance. Few studies have examined the utility of health care data for neighborhood-level surveillance.

OBJECTIVE

We examined the use of primary care electronic health records (EHRs) and emergency department (ED) claims for identifying neighborhoods with higher chronic disease burden and neighborhood-level prevalence estimation.

DESIGN

Comparison of hypertension and diabetes estimates from EHRs and ED claims with survey-based estimates.

SETTING

Forty-two United Hospital Fund neighborhoods in New York City.

PARTICIPANTS

The EHR sample comprised 708 452 patients from the Hub Population Health System (the Hub) in 2015, and the ED claim sample comprised 1 567 870 patients from the Statewide Planning and Research Cooperative System in 2015. We derived survey-based estimates from 2012 to 2016 Community Health Survey (n = 44 189).

MAIN OUTCOME MEASURE

We calculated hypertension and diabetes prevalence estimates by neighborhood from each data source. We obtained Pearson correlation and absolute difference between EHR-based or claims-based estimates and survey-based estimates.

RESULTS

Both EHR-based and claims-based estimates correlated strongly with survey-based estimates for hypertension (0.91 and 0.72, respectively) and diabetes (0.83 and 0.82, respectively) and identified similar neighborhoods of higher burden. For hypertension, 10 and 17 neighborhoods from the EHRs and ED claims, respectively, had an absolute difference of more than 5 percentage points from the survey-based estimate. For diabetes, 15 and 4 neighborhoods from the EHRs and ED claims, respectively, differed from the survey-based estimate by more than 5 percentage points.

CONCLUSIONS

Both EHRs and ED claims data are useful for identifying neighborhoods with greater disease burden and have potential for monitoring chronic conditions at the neighborhood level.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms

Introduction:

The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems.

Methods:

We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013–14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data.

Results:

Obesity and diabetes indicators had moderate to high sensitivity (0.81–0.96) and high specificity (0.94–0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78–0.90) and moderate to high specificity (0.88–0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use.

Discussion:

Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed.

Conclusion:

Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

State and Local Chronic Disease Surveillance Using Electronic Health Record Systems

OBJECTIVES

To assess the feasibility of chronic disease surveillance using distributed analysis of electronic health records and to compare results with Behavioral Risk Factor Surveillance System (BRFSS) state and small-area estimates.

METHODS

We queried the electronic health records of 3 independent Massachusetts-based practice groups using a distributed analysis tool called MDPHnet to measure the prevalence of diabetes, asthma, smoking, hypertension, and obesity in adults for the state and 13 cities. We adjusted observed rates for age, gender, and race/ethnicity relative to census data and compared them with BRFSS state and small-area estimates.

RESULTS

The MDPHnet population under surveillance included 1 073 545 adults (21.8% of the state adult population). MDPHnet and BRFSS state-level estimates were similar: 9.4% versus 9.7% for diabetes, 10.0% versus 12.0% for asthma, 13.5% versus 14.7% for smoking, 26.3% versus 29.6% for hypertension, and 22.8% versus 23.8% for obesity. Correlation coefficients for MDPHnet versus BRFSS small-area estimates ranged from 0.890 for diabetes to 0.646 for obesity.

CONCLUSIONS

Chronic disease surveillance using electronic health record data is feasible and generates estimates comparable with BRFSS state and small-area estimates.

Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records

INTRODUCTION

Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples.

METHODS

We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures.

RESULTS

A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins.

CONCLUSION

A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Can I Get me out of my Head? Exploring Strategies for Controlling the Self-Referential Aspects of the Mind-Wandering State during Reading

Trying to focus on a piece of text and keep unrelated thoughts at bay can be a surprisingly futile experience. The current study explored the effects of different instructions on participants’ capacity to control their mind-wandering and maximize reading comprehension, while reading. Participants were instructed to (a) enhance focus on what was read (external) or (b) enhance meta-awareness of mind-wandering (internal). To understand when these strategies were important, we induced a state of self-focus in half of our participants at the beginning of the experiment. Results replicated the negative association between mind-wandering and comprehension and demonstrated that both internal and external instructions impacted on the efficiency of reading following a period of induced self-focus. Techniques that foster meta-awareness improved task focus but did so at the detriment of reading comprehension, while promoting a deeper engagement while reading improved comprehension with no changes in reported mind-wandering. These data provide insight into how we can control mind-wandering and improve comprehension, and they underline that a state of self-focus is a condition under which they should be employed. Furthermore, these data support component process models that propose that the self-referent mental contents that arise during mind-wandering are distinguishable from those processes that interfere with comprehension.

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data

INTRODUCTION

Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia.

METHODS

NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants.

RESULTS

Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70.

DISCUSSION

Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them.

CONCLUSIONS

This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.

Local spatial clustering in youths' use of tobacco, alcohol, and marijuana in Boston

BACKGROUND

Understanding geographic variation in youth drug use is important for both identifying etiologic factors and planning prevention interventions. However, little research has examined spatial clustering of drug use among youths by using rigorous statistical methods.

OBJECTIVES

The purpose of this study was to examine spatial clustering of youth use of tobacco, alcohol, and marijuana.

METHODS

Responses on tobacco, alcohol, and marijuana use from 1,292 high school students ages 13-19 who provided complete residential addresses were drawn from the 2008 Boston Youth Survey Geospatial Dataset. Response options on past month use included "none," "1-2," "3-9," and "10 or more." The response rate for each substance was approximately 94%. Spatial clustering of youth drug use was assessed using the spatial Bernoulli model in the SatScan™ software package.

RESULTS

Approximately 12%, 36%, and 18% of youth reported any past-month use of tobacco, alcohol, and/or marijuana, respectively. Two clusters of elevated past tobacco use among Boston youths were generated, one of which was statistically significant. This cluster, located in the South Boston neighborhood, had a relative risk of 5.37 with a p-value of 0.00014. There was no significant localized spatial clustering in youth past alcohol or marijuana use in either the unadjusted or adjusted models.

CONCLUSION

Significant spatial clustering in youth tobacco use was found. Finding a significant cluster in the South Boston neighborhood provides reason for further investigation into neighborhood characteristics that may shape adolescents' substance use behaviors. This type of research can be used to evaluate the underlying reasons behind spatial clustering of youth substance and to target local drug abuse prevention interventions and use.

Urban Park Development and Pediatric Obesity Rates: A Quasi-Experiment Using Electronic Health Record Data

INTRODUCTION

Childhood obesity affects ~20% of children in the United States. Environmental influences, such as parks, are linked with increased physical activity (PA).

OBJECTIVE

To examine whether changes in Body Mass Index (BMI) z-score were associated with construction of a new park.

METHODS

A quasi-experimental design was used to determine whether living in proximity of a park was associated with a reduction in BMI z-score. Children were selected from health clinics within an 11 mile radius of the park. A repeated-measure ANOVA was employed for analysis of the relationship between exposure (new park) and BMI z-score.

RESULTS

Participants were 1443 (median age 10.3 range (2-17.9 years), BMI: z-score 0.84 ± 1.09) African American (77.4%) adolescents. Change in BMI z-score was not statistically different for children living at different distances from the park after controlling for age, gender, race, ethnicity, or payer type (p = 0.4482). We did observe a small 0.03 increase in BMI z-score from pre- to post-park (p = 0.0007). There was a significant positive association between child's baseline age and BMI z-score (p < 0.001).

CONCLUSIONS

This study found proximity to a park was not associated with reductions in BMI z-score. Additional efforts to understand the complex relationship between park proximity, access, and PA are warranted.

Comparison of practice based research network based quality improvement technical assistance and evaluation to other ongoing quality improvement efforts for changes in agency culture

BACKGROUND

Public health agencies in the USA are increasingly challenged to adopt Quality Improvement (QI) strategies to enhance performance. Many of the functional and structural barriers to effective use of QI can be found in the organizational culture of public health agencies. The purpose of this study was to assess the impact of public health practice based research network (PBRN) evaluation and technical assistance for QI interventions on the organizational culture of public health agencies in Georgia, USA.

METHODS

An online survey of key informants in Georgia's districts and county health departments was used to compare perceptions of characteristics of organizational QI culture between PBRN supported QI districts and non-PBRN supported districts before and after the QI interventions. The primary outcomes of concern were number and percentage of reported increases in characteristics of QI culture as measured by key informant responses to items assessing organizational QI practices from a validated instrument on QI Collaboratives. Survey results were analyzed using Multi-level Mixed Effects Logistic Model, which accounts for clustering/nesting.

RESULTS

Increases in QI organizational culture were consistent for all 10- items on a QI organizational culture survey related to: leadership support, use of data, on-going QI, and team collaboration. Statistically significant odds ratios were calculated for differences in increased QI organizational culture between PBRN-QI supported districts compared to Non-PBRN supported districts for 5 of the 10 items, after adjusting for District clustering of county health departments.

CONCLUSIONS

Agency culture, considered by many QI experts as the main goal of QI, is different than use of specific QI methods, such as Plan-Do-Study-Act (PDSA) cycles or root-cause analyses. The specific use of a QI method does not necessarily reflect culture change. Attempts to measure QI culture are newly emerging. This study documented significant improvements in characteristics of organizational culture and demonstrated the potential of PBRNs to support agency QI activities.

Accuracy of race, ethnicity, and language preference in an electronic health record

BACKGROUND

Collection of data on race, ethnicity, and language preference is required as part of the "meaningful use" of electronic health records (EHRs). These data serve as a foundation for interventions to reduce health disparities.

OBJECTIVE

Our aim was to compare the accuracy of EHR-recorded data on race, ethnicity, and language preference to that reported directly by patients.

DESIGN/SUBJECTS/MAIN MEASURES

Data collected as part of a tobacco cessation intervention for minority and low-income smokers across a network of 13 primary care clinics (n = 569).

KEY RESULTS

Patients were more likely to self-report Hispanic ethnicity (19.6 % vs. 16.6 %, p < 0.001) and African American race (27.0 % vs. 20.4 %, p < 0.001) than was reported in the EHR. Conversely, patients were less likely to complete the survey in Spanish than the language preference noted in the EHR suggested (5.1 % vs. 6.3 %, p < 0.001). Thirty percent of whites self-reported identification with at least one other racial or ethnic group, as did 37.0 % of Hispanics, and 41.0 % of African Americans. Over one-third of EHR-documented Spanish speakers elected to take the survey in English. One-fifth of individuals who took the survey in Spanish were recorded in the EHR as English-speaking.

CONCLUSION

We demonstrate important inaccuracies and the need for better processes to document race/ ethnicity and language preference in EHRs.

Proactive tobacco cessation outreach to smokers of low socioeconomic status: a randomized clinical trial

IMPORTANCE

Widening socioeconomic disparities in mortality in the United States are largely explained by slower declines in tobacco use among smokers of low socioeconomic status (SES) than among those of higher SES, which points to the need for targeted tobacco cessation interventions. Documentation of smoking status in electronic health records (EHRs) provides the tools for health systems to proactively offer tobacco treatment to socioeconomically disadvantaged smokers.

OBJECTIVE

To evaluate a proactive tobacco cessation strategy that addresses sociocontextual mediators of tobacco use for low-SES smokers.

DESIGN, SETTING, AND PARTICIPANTS

This prospective, randomized clinical trial included low-SES adult smokers who described their race and/or ethnicity as black, Hispanic, or white and received primary care at 1 of 13 practices in the greater Boston area (intervention group, n = 399; control group, n = 308).

INTERVENTIONS

We analyzed EHRs to identify potentially eligible participants and then used interactive voice response (IVR) techniques to reach out to them. Consenting patients were randomized to either receive usual care from their own health care team or enter an intervention program that included (1) telephone-based motivational counseling, (2) free nicotine replacement therapy (NRT) for 6 weeks, (3) access to community-based referrals to address sociocontextual mediators of tobacco use, and (4) integration of all these components into their normal health care through the EHR system.

MAIN OUTCOMES AND MEASURES

Self-reported past-7-day tobacco abstinence 9 months after randomization ("quitting"), assessed by automated caller or blinded study staff.

RESULTS

The intervention group had a higher quit rate than the usual care group (17.8% vs 8.1%; odds ratio, 2.5; 95% CI, 1.5-4.0; number needed to treat, 10). We examined whether use of intervention components was associated with quitting among individuals in the intervention group: individuals who participated in the telephone counseling were more likely to quit than those who did not (21.2% vs 10.4%; P < .001). There was no difference in quitting by use of NRT. Quitting did not differ by a request for a community referral, but individuals who used their referral were more likely to quit than those who did not (43.6% vs 15.3%; P < .001).

CONCLUSIONS AND RELEVANCE

Proactive, IVR-facilitated outreach enables engagement with low-SES smokers. Providing counseling, NRT, and access to community-based resources to address sociocontextual mediators among smokers reached in this setting is effective.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT01156610.