Feasibility of evaluating the CHIPRA care quality measures in electronic health record data

Study protocol: a mixed-methods study of women's healthcare in the safety net after Affordable Care Act implementation - EVERYWOMAN

Background

Evidence-based reproductive care reduces morbidity and mortality for women and their children, decreases health disparities and saves money. Community health centres (CHCs) are a key point of access to reproductive and primary care services for women who are publicly insured, uninsured or unable to pay for care. Women of reproductive age (15–44 years) comprise just of a quarter (26%) of the total CHC patient population, with higher than average proportions of women of colour, women with lower income and educational status and social challenges (e.g. housing). Such factors are associated with poorer reproductive health outcomes across contraceptive, preventive and pregnancy-related services. The Affordable Care Act (ACA) prioritised reproductive health as an essential component of women’s preventive services to counter these barriers and increase women’s access to care. In 2012, the United States Supreme Court ruled ACA implementation through Medicaid expansion as optional, creating a natural experiment to measure the ACA’s impact on women’s reproductive care delivery and health outcomes.

Methods

This paper describes a 5-year, mixed-methods study comparing women’s contraceptive, preventive, prenatal and postpartum care before and after ACA implementation and between Medicaid expansion and non-expansion states. Quantitative assessment will leverage electronic health record data from the ADVANCE Clinical Research Network, a network of over 130 CHCs in 24 states, to describe care and identify patient, practice and state-level factors associated with provision of recommended evidence-based care. Qualitative assessment will include patient, provider and practice level interviews to understand perceptions and utilisation of reproductive healthcare in CHC settings.

Discussion

To our knowledge, this will be the first study using patient level electronic health record data from multiple states to assess the impact of ACA implementation in conjunction with other practice and policy level factors such as Title X funding or 1115 Medicaid waivers. Findings will be relevant to policy and practice, informing efforts to enhance the provision of timely, evidence-based reproductive care, improve health outcomes and reduce disparities among women. Patient, provider and practice-level interviews will serve to contextualise our findings and develop subsequent studies and interventions to support women’s healthcare provision in CHC settings.

Feasibility testing of the Core set of quality Indicators for Paediatric Primary Care in Europe, COSI-PPC-EU

There is a need to measure and improve the quality of paediatric primary care in Europe where major differences in the delivery and outcomes of child health care exist. A collaborative panel of paediatric senior experts developed a Core Set of Indicators for Paediatric Primary Care in Europe by compiling 42 quality indicators in a modified consensus process following the RAND/UCLA appropriateness method. The aim of this study was to explore the feasibility of the quality indicator set in European paediatric primary care practices. Seventy-nine practices from eight countries participated in a detailed online interview. The practices rated the applicability, relevance, reliability and acceptance of the 42 quality indicator as well as the availability, technical feasibility and effort to retrieve the needed data from their medical records. Most quality indicators were considered applicable, available, reliable, acceptable and relevant for monitoring quality of care in paediatric primary care. Respondents rated feasibility and effort to retrieve the data lowest because of difficulties collecting the data from the medical records.Conclusion: European paediatric primary care practices generally agree with the proposed quality indicator set. They document most of the parameters. However, the collection of specific needed values from available routine patient-data is considered technically difficult and time-consuming. What is Known? • Paediatric primary care systems in Europe show striking differences in their performance. Pre-existing sets of quality indicators are predominantly limited to national populations, specific diseases and hospital care. • A Core Set of 42 quality indicators for paediatric primary care in Europe was developed by European paediatricians using a systematic literature review and a consensus process following a modified RAND/UCLA appropriateness method. What is New? • Paediatric primary care providers in Europe agree with the idea to use COSI-PPC-EU to monitor and improve the quality of care. The set was considered applicable, available, reliable, acceptable, and relevant for quality improvement. • The score for feasibility and effort to retrieve the data was low, because of technical reasons; the electronical or paper-based medical documentation in most cases does not allow convenient access to all necessary data.

Medicaid coverage accuracy in electronic health records

Health insurance coverage facilitates access to preventive screenings and other essential health care services, and is linked to improved health outcomes; therefore, it is critical to understand how well coverage information is documented in the electronic health record (EHR) and which characteristics are associated with accurate documentation. Our objective was to evaluate the validity of EHR data for monitoring longitudinal Medicaid coverage and assess variation by patient demographics, visit types, and clinic characteristics. We conducted a retrospective, observational study comparing Medicaid status agreement between Oregon community health center EHR data linked at the patient-level to Medicaid enrollment data (gold standard). We included adult patients with a Medicaid identification number and ≥1 clinic visit between 1/1/2013-12/31/2014 [>1 million visits (n = 135,514 patients)]. We estimated statistical correspondence between EHR and Medicaid data at each visit (visit-level) and for different insurance cohorts over time (patient-level). Data were collected in 2016 and analyzed 2017-2018. We observed excellent agreement between EHR and Medicaid data for health insurance information: kappa (>0.80), sensitivity (>0.80), and specificity (>0.85). Several characteristics were associated with agreement; at the visit-level, agreement was lower for patients who preferred a non-English language and for visits missing income information. At the patient-level, agreement was lower for black patients and higher for older patients seen in primary care community health centers. Community health center EHR data are a valid source of Medicaid coverage information. Agreement varied with several characteristics, something researchers and clinic staff should consider when using health insurance information from EHR data.

Electronic health record tools to assist with children's insurance coverage: a mixed methods study

Background

Children with health insurance have increased access to healthcare and receive higher quality care. However, despite recent initiatives expanding children’s coverage, many remain uninsured. New technologies present opportunities for helping clinics provide enrollment support for patients. We developed and tested electronic health record (EHR)-based tools to help clinics provide children’s insurance assistance.

Methods

We used mixed methods to understand tool adoption, and to assess impact of tool use on insurance coverage, healthcare utilization, and receipt of recommended care. We conducted intent-to-treat (ITT) analyses comparing pediatric patients in 4 intervention clinics (n = 15,024) to those at 4 matched control clinics (n = 12,227). We conducted effect-of-treatment-on-the-treated (ETOT) analyses comparing intervention clinic patients with tool use (n = 2240) to intervention clinic patients without tool use (n = 12,784).

Results

Tools were used for only 15% of eligible patients. Qualitative data indicated that tool adoption was limited by: (1) concurrent initiatives that duplicated the work associated with the tools, and (2) inability to obtain accurate insurance coverage data and end dates. The ITT analyses showed that intervention clinic patients had higher odds of gaining insurance coverage (adjusted odds ratio [aOR] = 1.32, 95% confidence interval [95%CI] 1.14–1.51) and lower odds of losing coverage (aOR = 0.77, 95%CI 0.68–0.88), compared to control clinic patients. Similarly, ETOT findings showed that intervention clinic patients with tool use had higher odds of gaining insurance (aOR = 1.83, 95%CI 1.64–2.04) and lower odds of losing coverage (aOR = 0.70, 95%CI 0.53–0.91), compared to patients without tool use. The ETOT analyses also showed higher rates of receipt of return visits, well-child visits, and several immunizations among patients for whom the tools were used.

Conclusions

This pragmatic trial, the first to evaluate EHR-based insurance assistance tools, suggests that it is feasible to create and implement tools that help clinics provide insurance enrollment support to pediatric patients. While ITT findings were limited by low rates of tool use, ITT and ETOT findings suggest tool use was associated with better odds of gaining and keeping coverage. Further, ETOT findings suggest that use of such tools may positively impact healthcare utilization and quality of pediatric care.

Trial registration

ClinicalTrials.gov, NCT02298361; retrospectively registered on November 5, 2014.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3159-x) contains supplementary material, which is available to authorized users.

Is It Feasible to Use Electronic Health Records for Quality Measurement of Adolescent Care?

OBJECTIVE

To determine the extent to which it is feasible to implement quality measures on electronic health records (EHRs) as currently implemented in pediatric health centers.

METHODS

A survey of information technology professionals at 10 institutions that provide primary care services to adolescents. The survey asked whether data about care was being captured electronically across the nine domains relevant to adolescent well care: Screening, Health Risks, Sexual Health, Diagnosis and History, Laboratory Results, Prescriptions, Referrals, Forms Management, and Patient Demographics. For each domain, we developed a scale of the extent to which the EHR makes quality measurement feasible.

RESULTS

Overall feasibility scores varied across centers from 34% to 85% and from 53% to 80% across care domains. One centre reported 100% feasibility for 8 of 10 care domains.

CONCLUSIONS

Electronic health records can facilitate quality improvement, but the feasibility of such use depends on the presence, validity, and accessibility of the quality data in the EHR. Even among the largest and most sophisticated pediatric EHR systems, quality of care measurement is not possible yet for all aspects of adolescent well care without manual effort to review and code data. Nevertheless, almost all quality measures were reported to be feasible in some systems.

Measuring Preventive Care Delivery: Comparing Rates Across Three Data Sources

INTRODUCTION

Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness.

METHODS

This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015.

RESULTS

Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96-1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83-0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67-0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services.

CONCLUSIONS

Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

A comparison of a multistate inpatient EHR database to the HCUP Nationwide Inpatient Sample

BACKGROUND

The growing availability of electronic health records (EHRs) in the US could provide researchers with a more detailed and clinically relevant alternative to using claims-based data.

METHODS

In this study we compared a very large EHR database (Health Facts©) to a well-established population estimate (Nationwide Inpatient Sample). Weighted comparisons were made using t-value and relative difference over diagnoses and procedures for the year 2010.

RESULTS

The two databases have a similar distribution pattern across all data elements, with 24 of 50 data elements being statistically similar between the two data sources. In general, differences that were found are consistent across diagnosis and procedures categories and were specific to the psychiatric-behavioral and obstetrics-gynecology services areas.

CONCLUSIONS

Large EHR databases have the potential to be a useful addition to health services researchers, although they require different analytic techniques compared to administrative databases; more research is needed to understand the differences.

Testing health information technology tools to facilitate health insurance support: a protocol for an effectiveness-implementation hybrid randomized trial

BACKGROUND

Patients with gaps in health insurance coverage often defer or forgo cancer prevention services. These delays in cancer detection and diagnoses lead to higher rates of morbidity and mortality and increased costs. Recent advances in health information technology (HIT) create new opportunities to enhance insurance support services that reduce coverage gaps through automated processes applied in healthcare settings. This study will assess the implementation of insurance support HIT tools and their effectiveness at improving patients' insurance coverage continuity and cancer screening rates.

METHODS/DESIGN

This study uses a hybrid cluster-randomized design-a combined effectiveness and implementation trial-in community health centers (CHCs) in the USA. Eligible CHC clinic sites will be randomly assigned to one of two groups in the trial's implementation component: tools + basic training (Arm I) and tools + enhanced training + facilitation (Arm II). A propensity score-matched control group of clinics will be selected to assess the tools' effectiveness. Quantitative analyses of the tools' impact will use electronic health record and Medicaid data to assess effectiveness. Qualitative data will be collected to evaluate the implementation process, understand how the HIT tools are being used, and identify facilitators and barriers to their implementation and use.

DISCUSSION

This study will test the effectiveness of HIT tools to enhance insurance support in CHCs and will compare strategies for facilitating their implementation in "real-world" practice settings. Findings will inform further development and, if indicated, more widespread implementation of insurance support HIT tools.

TRIAL REGISTRATION

Clinical trial NTC02355262.

Innovative methods for parents and clinics to create tools for kids' care (IMPACCT Kids' Care) study protocol

BACKGROUND

Despite expansions in public health insurance, many children remain uninsured or experience gaps in coverage. Community health centers (CHCs) provide primary care to many children at risk for uninsurance and are well-positioned to help families obtain and retain children's coverage. Recent advances in health information technology (HIT) capabilities provide the means to create tools that could enhance CHCs' insurance outreach efforts.

OBJECTIVE

To present the study design, baseline patient characteristics, variables, and statistical methods for the Innovative Methods for Parents And Clinics to Create Tools for Kids' Care (IMPACCT Kids' Care) study.

METHODS/DESIGN

In this mixed methods study, we will design, test and refine health insurance outreach HIT tools through a user-centered process. We will then implement the tools in four CHCs and evaluate their effectiveness and barriers and facilitators to their implementation. To measure effectiveness, we will quantitatively assess health insurance coverage continuity and utilization of healthcare services for pediatric patients in intervention CHCs compared to matched control sites using electronic health record (EHR) and Oregon Medicaid administrative data over 18months pre- and 18months post-implementation (n=34,867 children). We will also qualitatively assess the implementation process to understand how the tools fit into the clinics' workflows and the CHC staff experiences with the tools.

CONCLUSIONS

This study creates, implements, and evaluates health insurance outreach HIT tools. The use of such tools will likely improve care delivery and health outcomes, reduce healthcare disparities for vulnerable populations, and enhance overall healthcare system performance. ClinicalTrials.gov Identifier: NCT02298361.

Measuring quality in pediatrics: Florida's early experiences with the CHIPRA core measure set

Enacted in 2009, the Children's Health Insurance Program Reauthorization Act (CHIPRA) aims, among other things, to increase state's accountability for providing quality health care to all children in the United States. Although it is important for states to report on the measures, learning from their successes and failures is critical in producing the measures so that states will be prepared for future regulations. Florida covered roughly 2.59 million children in 2010. Administrative, medical record, registry, and survey data were used to report on 20 of the 24 CHIPRA core measures. Technical specifications from the Centers for Medicare and Medicaid Services were used. Approximately 10 months were needed to identify, collect, safeguard, and process the required data. Florida was able to build on its past experiences with performance measurement reporting and surveying. Conducting medical record reviews at the state level and producing measures that required registry data proved to be challenging. Although Florida was successful in its first year of reporting the CHIPRA core measures, certain populations were not included in some of the measures. The next phase of Florida's CHIPRA project will focus on developing and implementing a dissemination plan and creating opportunities to improve the measures. Florida has made significant progress in the early phases of reporting the CHIPRA measures. As Florida gains more experience in reporting the measures, and results from other states are released, it will be easier to put the statewide measure results into context. Once meaningful comparisons can be made, Florida will be able to better plan for the future of child health and health outcomes.

Using computer-extracted data from electronic health records to measure the quality of adolescent well-care

OBJECTIVE

To determine whether quality measures based on computer-extracted EHR data can reproduce findings based on data manually extracted by reviewers.

DATA SOURCES

We studied 12 measures of care indicated for adolescent well-care visits for 597 patients in three pediatric health systems.

STUDY DESIGN

Observational study.

DATA COLLECTION/EXTRACTION METHODS

Manual reviewers collected quality data from the EHR. Site personnel programmed their EHR systems to extract the same data from structured fields in the EHR according to national health IT standards.

PRINCIPAL FINDINGS

Overall performance measured via computer-extracted data was 21.9 percent, compared with 53.2 percent for manual data. Agreement measures were high for immunizations. Otherwise, agreement between computer extraction and manual review was modest (Kappa = 0.36) because computer-extracted data frequently missed care events (sensitivity = 39.5 percent). Measure validity varied by health care domain and setting. A limitation of our findings is that we studied only three domains and three sites.

CONCLUSIONS

The accuracy of computer-extracted EHR quality reporting depends on the use of structured data fields, with the highest agreement found for measures and in the setting that had the greatest concentration of structured fields. We need to improve documentation of care, data extraction, and adaptation of EHR systems to practice workflow.

Variation in outcomes of quality measurement by data source

OBJECTIVE

To evaluate selected Children's Health Insurance Program Reauthorization Act claims-based quality measures using claims data alone, electronic health record (EHR) data alone, and both data sources combined.

METHODS

Our population included pediatric patients from 46 clinics in the OCHIN network of community health centers, who were continuously enrolled in Oregon's public health insurance program during 2010. Within this population, we calculated selected pediatric care quality measures according to the Children's Health Insurance Program Reauthorization Act technical specifications within administrative claims. We then calculated these measures in the same cohort, by using EHR data, by using the technical specifications plus clinical data previously shown to enhance capture of a given measure. We used the k statistic to determine agreement in measurement when using claims versus EHR data. Finally, we measured quality of care delivered to the study population, when using a combined dataset of linked, patient-level administrative claims and EHR data.

RESULTS

When using administrative claims data, 1.0% of children (aged 3-17) had a BMI percentile recorded, compared with 71.9% based on the EHR data (k agreement [k] # 0.01), and 72.0% in the combined dataset. Among children turning 2 in 2010, 20.2% received all recommended immunizations according to the administrative claims data, 17.2% according to the EHR data (k = 0.82), and 21.4% according to the combined dataset.

CONCLUSIONS

Children's care quality measures may not be accurate when assessed using only administrative claims. Adding EHR data to administrative claims data may yield more complete measurement.

The ADVANCE network: accelerating data value across a national community health center network

The ADVANCE (Accelerating Data Value Across a National Community Health Center Network) clinical data research network (CDRN) is led by the OCHIN Community Health Information Network in partnership with Health Choice Network and Fenway Health. The ADVANCE CDRN will ‘horizontally’ integrate outpatient electronic health record data for over one million federally qualified health center patients, and ‘vertically’ integrate hospital, health plan, and community data for these patients, often under-represented in research studies. Patient investigators, community investigators, and academic investigators with diverse expertise will work together to meet project goals related to data integration, patient engagement and recruitment, and the development of streamlined regulatory policies. By enhancing the data and research infrastructure of participating organizations, the ADVANCE CDRN will serve as a ‘community laboratory’ for including disadvantaged and vulnerable patients in patient-centered outcomes research that is aligned with the priorities of patients, clinics, and communities in our network.

Variations in measurement of sexual activity based on EHR definitions

OBJECTIVE

The goal of this study was to compare the performance of 4 operational definitions of sexual activity by using data electronically abstracted from electronic health records (EHRs) and examine how documentation of Chlamydia screening and positivity vary according to definition of sexual activity.

METHODS

Extracts were created from EHRs of adolescent females 12 to 19 years old who had ≥1 visit to a primary care practice during 2011 at 4 US pediatric health care organizations. We created 4 definitions of sexual activity derived from electronically abstracted indicator variables. Percent sexually active, documentation of Chlamydia screening, and rate of positive Chlamydia test results per 1000 adolescent females according to the sexual activity definition were calculated.

RESULTS

The most commonly documented individual indicator of sexual activity was "patient report of being sexually active" (mean across 4 sites: 19.2%). The percentage of adolescent females classified as sexually active varied by site and increased as more indicator variables were included. As the definition of sexual activity expanded, the percentage of sexually active females who received at least 1 Chlamydia test decreased. Using a broader definition of sexual activity resulted in improved identification of adolescent females with Chlamydia infection. For each sexual activity definition and performance item, the difference was statistically significant (P < .0001).

CONCLUSIONS

Information about sexual activity may be gathered from a variety of data sources, and changing the configurations of these indicators results in differences in the percentage of adolescent females classified as sexually active, screened for Chlamydia infection, and Chlamydia infection rates.

The national improvement partnership network: state-based partnerships that improve primary care quality

Improvement partnerships (IPs) are a model for collaboration among public and private organizations that share interests in improving child health and the quality of health care delivered to children. Their partners typically include state public health and Medicaid agencies, the local chapter of the American Academy of Pediatrics, and an academic health care organization or children's hospital. Most IPs also engage other partners, including a variety of public, private, and professional organizations and individuals. IPs lead and support measurement-based, systems-focused quality improvement (QI) efforts that primarily target primary care practices that care for children. Their projects are most often conducted as learning collaboratives that involve a team from each of 8 to 15 participating practices over 9 to 12 months. The improvement teams typically include a clinician, office manager, clinical staff (nurses or medical assistants), and, for some projects, a parent; the IPs provide the staff and local infrastructure. The projects target clinical topics, chosen because of their importance to public health, local clinicians, and funding agencies, including asthma, attention-deficit/hyperactivity disorder, autism, developmental screening, obesity, mental health, medical home implementation, and several others. Over the past 13 years, 19 states have developed (and 5 are exploring developing) IPs. These organizations share similar aims and methods but differ substantially in leadership, structure, funding, and longevity. Their projects generally engage pediatric and family medicine practices ranging from solo private practices to community health centers to large corporate practices. The practices learn about the project topic and about QI, develop specific improvement strategies and aims that align with the project aims, perform iterative measures to evaluate and guide their improvements, and implement systems and processes to support and sustain those improvements. Since 2008, IPs have offered credit toward Part 4 of Maintenance of Certification for participants in some of their projects. To date, IPs have focused on achieving improvements in care delivery through individual projects. Rigorous measurement and evaluation of their efforts and impact will be essential to understanding, spreading, and sustaining state/regional child health care QI programs. We describe the origins, evolution to date, and hopes for the future of these partnerships and the National Improvement Partnership Network (NIPN), which was established to support existing and nurture new IPs.

Measuring prevention more broadly: an empirical assessment of CHIPRA core measures

OBJECTIVE

To assess limitations of using select Children's Health Insurance Program Reauthorization Act (CHIPRA) core claims-based measures in capturing the preventive services that may occur in the clinical setting.

METHODS

We use claims data from ALL Kids, the Alabama Children's Health Insurance Program (CHIP), to calculate each of four quality measures under two alternative definitions: (1) the formal claims-based guidelines outlined in the CMS Technical Specifications, and (2) a broader definition of appropriate claims for identifying preventive service use. Additionally, we examine the extent to which these two claims-based approaches to measuring quality differ in assessments of disparities in quality of care across subgroups of children.

RESULTS

Statistically significant differences in rates were identified when comparing the two definitions for calculating each quality measure. Measure differences ranged from a 1.9 percentage point change for measure #13 (receiving preventive dental services) to a 25.5 percentage point change for measure #12 (adolescent well-care visit). We were able to identify subgroups based upon family income, rural location, and chronic disease status with differences in quality within the core measures. However, some identified disparities were sensitive to the approach used to calculate the quality measure.

CONCLUSIONS

Differences in CHIP design and structure, across states and over time, may limit the usefulness of select claims-based core measures for detecting disparities accurately. Additional guidance and research may be necessary before reporting of the measures becomes mandatory.

Are pediatric quality care measures too stringent?

INTRODUCTION

We aimed to demonstrate the application of national pediatric quality measures, derived from claims-based data, for use with electronic medical record data, and determine the extent to which rates differ if specifications were modified to allow for flexibility in measuring receipt of care.

METHODS

We reviewed electronic medical record data for all patients up to 15 years of age with ≥1 office visit to a safety net family medicine clinic in 2010 (n = 1544). We assessed rates of appropriate well-child visits, immunizations, and body mass index (BMI) documentation, defined strictly by national guidelines versus by guidelines with clinically relevant modifications.

RESULTS

Among children aged <3 years, 52.4% attended ≥6 well-child visits by the age of 15 months; 60.8% had ≥6 visits by age 2 years. Less than 10% completed 10 vaccination series before their second birthday; with modifications, 36% were up to date. Among children aged 3 to 15 years, 63% had a BMI percentile recorded; 91% had BMI recorded within 36 months of the measurement year.

CONCLUSIONS

Applying relevant modifications to national quality measure definitions captured a substantial number of additional services. Strict adherence to measure definitions might miss the true quality of care provided, especially among populations that may have sporadic patterns of care utilization.