Using Clinical Data Standards to Measure Quality: A New Approach

Data Quality in Health Research: Integrative Literature Review

BACKGROUND

Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

OBJECTIVE

Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

METHODS

A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

RESULTS

After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

CONCLUSIONS

The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2022.05.31.22275804.

A lifecycle framework illustrates eight stages necessary for realizing the benefits of patient-centered clinical decision support

The design, development, implementation, use, and evaluation of high-quality, patient-centered clinical decision support (PC CDS) is necessary if we are to achieve the quintuple aim in healthcare. We developed a PC CDS lifecycle framework to promote a common understanding and language for communication among researchers, patients, clinicians, and policymakers. The framework puts the patient, and/or their caregiver at the center and illustrates how they are involved in all the following stages: Computable Clinical Knowledge, Patient-specific Inference, Information Delivery, Clinical Decision, Patient Behaviors, Health Outcomes, Aggregate Data, and patient-centered outcomes research (PCOR) Evidence. Using this idealized framework reminds key stakeholders that developing, deploying, and evaluating PC-CDS is a complex, sociotechnical challenge that requires consideration of all 8 stages. In addition, we need to ensure that patients, their caregivers, and the clinicians caring for them are explicitly involved at each stage to help us achieve the quintuple aim.

Generating and Reporting Electronic Clinical Quality Measures from Electronic Health Records: Strategies from EvidenceNOW Cooperatives

BACKGROUND

Electronic clinical quality measures (eCQMs) from electronic health records (EHRs) are a key component of quality improvement (QI) initiatives in small-to-medium size primary care practices, but using eCQMs for QI can be challenging. Organizational strategies are needed to effectively operationalize eCQMs for QI in these practice settings.

OBJECTIVE

This study aimed to characterize strategies that seven regional cooperatives participating in the EvidenceNOW initiative developed to generate and report EHR-based eCQMs for QI in small-to-medium size practices.

METHODS

A qualitative study comprised of 17 interviews with representatives from all seven EvidenceNOW cooperatives was conducted. Interviewees included administrators were with both strategic and cooperative-level operational responsibilities and external practice facilitators were with hands-on experience helping practices use EHRs and eCQMs. A subteam conducted 1-hour semistructured telephone interviews with administrators and practice facilitators, then analyzed interview transcripts using immersion crystallization. The analysis and a conceptual model were vetted and approved by the larger group of coauthors.

RESULTS

Cooperative strategies consisted of efforts in four key domains. First, cooperative adaptation shaped overall strategies for calculating eCQMs whether using EHRs, a centralized source, or a "hybrid strategy" of the two. Second, the eCQM generation described how EHR data were extracted, validated, and reported for calculating eCQMs. Third, practice facilitation characterized how facilitators with backgrounds in health information technology (IT) delivered services and solutions for data capture and quality and practice support. Fourth, performance reporting strategies and tools informed QI efforts and how cooperatives could alter their approaches to eCQMs.

CONCLUSION

Cooperatives ultimately generated and reported eCQMs using hybrid strategies because they determined neither EHRs alone nor centralized sources alone could operationalize eCQMs for QI. This required cooperatives to devise solutions and utilize resources that often are unavailable to typical small-to-medium-sized practices. The experiences from EvidenceNOW cooperatives provide insights into how organizations can plan for challenges and operationalize EHR-based eCQMs.

Exploring Quality Differences in Telemedicine Between Hospital Outpatient Departments and Community Clinics: A Cross-Sectional Study

Background

Telemedicine is a care delivery modality that has the potential to broaden the reach and flexibility of health care services. In the United Arab Emirates, telemedicine services are mainly delivered through either integrated hospital outpatient department (OPDs) or community clinics. However, it is unknown if patients’ perceptions of, and satisfaction with, telemedicine services differ between these two types of health care systems during the COVID-19 pandemic.

Objective

We aimed to explore the differences in patients’ perceptions of, and satisfaction with, telemedicine between hospital OPDs and community clinics during the COVID-19 pandemic. We also aimed to identify patient- or visit-related characteristics contributing to patient satisfaction with telemedicine.

Methods

In this cross-sectional study that was conducted at Abu Dhabi health care centers, we invited outpatients aged 18 years or over, who completed a telemedicine visit during the COVID-19 pandemic, to participate in our study. Patients’ perceptions of, and satisfaction with, telemedicine regarding the two system types (ie, hospital OPDs and community clinics) were assessed using an online survey that was sent as a link through the SMS system. Regression models were used to describe the association between patient- and visit-related characteristics, as well as the perception of, and satisfaction with, telemedicine services.

Results

A total of 515 patients participated in this survey. Patients’ satisfaction with telemedicine services was equally high among the settings, with no statistically significant difference between the two setting types (hospital OPDs: 253/343, 73.8%; community clinics: 114/172, 66.3%; P=.19). Video consultation was significantly associated with increased patient satisfaction (odds ratio [OR] 2.57, 95% CI 1.04-6.33; P=.04) and patients’ support of the transition to telemedicine use during and after the pandemic (OR 2.88, 95% CI 1.18-7.07; P=.02). Patients who used video consultations were more likely to report that telemedicine improved access to health care services (OR 3.06, 95% CI 1.71-8.03; P=.02), reduced waiting times and travel costs (OR 4.94, 95% CI 1.15-21.19; P=.03), addressed patients’ needs (OR 2.63, 95% CI 1.13-6.11; P=.03), and eased expression of patients’ medical concerns during the COVID-19 pandemic (OR 2.19, 95% CI 0.89-5.38; P=.09). Surprisingly, middle-aged patients were two times more likely to be satisfied with telemedicine services (OR 2.12, 95% CI 1.09-4.14; P=.03), as compared to any other age group in this study.

Conclusions

These findings suggest that patient satisfaction was unaffected by the health system setting in which patients received the teleconsultations, whether they were at hospitals or community clinics. Video consultation was associated with increased patient satisfaction with telemedicine services. Efforts should be focused on strategic planning for enhanced telemedicine services, video consultation in particular, for both emergent circumstances, such as the COVID-19 pandemic, and day-to-day health care delivery.

Harmonization of the ICHOM Quality Measures to Enable Health Outcomes Measurement in Multimorbid Patients

Objectives: To update the sets of patient-centric outcomes measures (“standard-sets”) developed by the not-for-profit organization ICHOM to become more readily applicable in patients with multimorbidity and to facilitate their implementation in health information systems. To that end we set out to (i) harmonize measures previously defined separately for different conditions, (ii) create clinical information models from the measures, and (iii) restructure the annotation to make the sets machine-readable.

Materials and Methods: First, we harmonized the semantic meaning of individual measures across all the 28 standard-sets published to date, in a harmonized measure repository. Second, measures corresponding to four conditions (Breast cancer, Cataracts, Inflammatory bowel disease and Heart failure) were expressed as logical models and mapped to reference terminologies in a pilot study.

Results: The harmonization of semantic meaning resulted in a consolidation of measures used across the standard-sets by 15%, from 3,178 to 2,712. These were all converted into a machine-readable format. 61% of the measures in the 4 pilot sets were bound to existing concepts in either SNOMED CT or LOINC.

Discussion: The harmonization of ICHOM measures across conditions is expected to increase the applicability of ICHOM standard-sets to multi-morbid patients, as well as facilitate their implementation in health information systems.

Conclusion: Harmonizing the ICHOM measures and making them machine-readable is expected to expedite the global adoption of systematic and interoperable outcomes measurement. In turn, we hope that the improved transparency on health outcomes that follows will let health systems across the globe learn from each other to the ultimate benefit of patients.

Clinical data sharing improves quality measurement and patient safety

OBJECTIVE

Accurate and robust quality measurement is critical to the future of value-based care. Having incomplete information when calculating quality measures can cause inaccuracies in reported patient outcomes. This research examines how quality calculations vary when using data from an individual electronic health record (EHR) and longitudinal data from a health information exchange (HIE) operating as a multisource registry for quality measurement.

MATERIALS AND METHODS

Data were sampled from 53 healthcare organizations in 2018. Organizations represented both ambulatory care practices and health systems participating in the state of Kansas HIE. Fourteen ambulatory quality measures for 5300 patients were calculated using the data from an individual EHR source and contrasted to calculations when HIE data were added to locally recorded data.

RESULTS

A total of 79% of patients received care at more than 1 facility during the 2018 calendar year. A total of 12 994 applicable quality measure calculations were compared using data from the originating organization vs longitudinal data from the HIE. A total of 15% of all quality measure calculations changed (P < .001) when including HIE data sources, affecting 19% of patients. Changes in quality measure calculations were observed across measures and organizations.

DISCUSSION

These results demonstrate that quality measures calculated using single-site EHR data may be limited by incomplete information. Effective data sharing significantly changes quality calculations, which affect healthcare payments, patient safety, and care quality.

CONCLUSIONS

Federal, state, and commercial programs that use quality measurement as part of reimbursement could promote more accurate and representative quality measurement through methods that increase clinical data sharing.

Quality of Neurologic Care in the United States: Initial Report From the Axon Registry

OBJECTIVE

To provide the initial description of the quality of outpatient US neurologic care as collected and reported in the Axon Registry.

METHODS

We describe characteristics of registry participants and the performance of neurology providers on 20 of the 2019 Axon Registry quality measures. From the distribution of providers' scores on a quality measure, we calculate the median performance for each quality measure. We test for associations between quality measure performance, provider characteristics, and intrinsic measure parameters.

RESULTS

There were 948 neurology providers who contributed a total of 6,480 provider-metric observations. Overall, the average quality measure performance score at the provider level was 66 (median 77). At the measure level (n = 20), the average quality measure performance score was 53 (median 55) with a range of 2 to 100 (interquartile range 20-91). Measures with a lower-complexity category (e.g., discrete orders, singular concepts) or developed through the specialty's qualified clinical data registry pathway had higher performance distributions. There was no difference in performance between Merit-Based Incentive Payment System (MIPS) and non-MIPS providers. There was no association between quality measure performance and practice size, measure clinical topic/neurologic condition, or measure year of entry.

CONCLUSIONS

This cross-sectional assessment of quality measure performance in 2019 Axon Registry data demonstrates modest performance scores and considerable variability across measures and providers. More complex measures were associated with lower performance. These findings serve as a baseline assessment of quality of ambulatory neurologic care in the United States and provide insights into future measure design.

Quality Improvement in Otolaryngology-Head and Neck Surgery: Developing Registry-Enabled Quality Measures From Guidelines for Cerumen Impaction and Allergic Rhinitis Through a Transparent and Systematic Process

BACKGROUND AND SIGNIFICANCE

Quality measurement can drive improvement in clinical care and allow for easy reporting of quality care by clinicians, but creating quality measures is a time-consuming and costly process. ECRI (formerly Emergency Care Research Institute) has pioneered a process to support systematic translation of clinical practice guidelines into electronic quality measures using a transparent and reproducible pathway. This process could be used to augment or support the development of electronic quality measures of the American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO-HNSF) and others as the Centers for Medicare and Medicaid Services transitions from the Merit-Based Incentive Payment System (MIPS) to the MIPS Value Pathways for quality reporting.

METHODS

We used a transparent and reproducible process to create electronic quality measures based on recommendations from 2 AAO-HNSF clinical practice guidelines (cerumen impaction and allergic rhinitis). Steps of this process include source material review, electronic content extraction, logic development, implementation barrier analysis, content encoding and structuring, and measure formalization. Proposed measures then go through the standard publication process for AAO-HNSF measures.

RESULTS

The 2 guidelines contained 29 recommendation statements, of which 7 were translated into electronic quality measures and published. Intermediate products of the guideline conversion process facilitated development and were retained to support review, updating, and transparency. Of the 7 initially published quality measures, 6 were approved as 2018 MIPS measures, and 2 continued to demonstrate a gap in care after a year of data collection.

CONCLUSION

Developing high-quality, registry-enabled measures from guidelines via a rigorous reproducible process is feasible. The streamlined process was effective in producing quality measures for publication in a timely fashion. Efforts to better identify gaps in care and more quickly recognize recommendations that would not translate well into quality measures could further streamline this process.

Validating the Matching of Patients in the Linkage of a Large Hospital System's EHR with State and National Death Databases

Background  Though electronic health record (EHR) data have been linked to national and state death registries, such linkages have rarely been validated for an entire hospital system's EHR.

Objectives  The aim of the study is to validate West Virginia University Medicine's (WVU Medicine) linkage of its EHR to three external death registries: the Social Security Death Masterfile (SSDMF), the national death index (NDI), the West Virginia Department of Health and Human Resources (DHHR).

Methods  Probabilistic matching was used to link patients to NDI and deterministic matching for the SSDMF and DHHR vital statistics records (WVDMF). In subanalysis, we used deaths recorded in Epic ( n  = 30,217) to further validate a subset of deaths captured by the SSDMF, NDI, and WVDMF.

Results  Of the deaths captured by the SSDMF, 59.8 and 68.5% were captured by NDI and WVDMF, respectively; for deaths captured by NDI this co-capture rate was 80 and 78%, respectively, for the SSDMF and WVDMF. Kappa statistics were strongest for NDI and WVDMF (61.2%) and NDI and SSDMF (60.6%) and weakest for SSDMF and WVDMF (27.9%). Of deaths recorded in Epic, 84.3, 85.5, and 84.4% were captured by SSDMF, NDI, and WVDMF, respectively. Less than 2% of patients' deaths recorded in Epic were not found in any of the death registries. Finally, approximately 0.2% of “decedents” in any death registry re-emerged in Epic at least 6 months after their death date, a very small percentage and thus further validating the linkages.

Conclusion  NDI had greatest validity in capturing deaths in our EHR. As a similar, though slightly less capture and agreement rate in identifying deaths is observed for SSDMF and state vital statistics records, these registries may be reasonable alternatives to NDI for research and quality assurance studies utilizing entire EHRs from large hospital systems. Investigators should also be aware that there will be a very tiny fraction of “dead” patients re-emerging in the EHR.

Igniting Harmonized Digital Clinical Quality Measurement through Terminology, CQL, and FHIR

Background  Electronic clinical quality measures (eCQMs) seek to quantify the adherence of health care to evidence-based standards. This requires a high level of consistency to reduce the effort of data collection and ensure comparisons are valid. Yet, there is considerable variability in local data capture, in the use of data standards and in implemented documentation processes, so organizations struggle to implement quality measures and extract data reliably for comparison across patients, providers, and systems.

Objective  In this paper, we discuss opportunities for harmonization within and across eCQMs; specifically, at the level of the measure concept, the logical clauses or phrases, the data elements, and the codes and value sets.

Methods  The authors, experts in measure development, quality assurance, standards and implementation, reviewed measure structure and content to describe the state of the art for measure analysis and harmonization. Our review resulted in the identification of four measure component levels for harmonization. We provide examples for harmonization of each of the four measure components based on experience with current quality measurement programs including the Centers for Medicare and Medicaid Services eCQM programs.

Results  In general, there are significant issues with lack of harmonization across measure concepts, logical phrases, and data elements. This magnifies implementation problems, confuses users, and requires more elaborate data mapping and maintenance.

Conclusion  Comparisons using semantically equivalent data are needed to accurately measure performance and reduce workflow interruptions with the aim of reducing evidence-based care gaps. It comes as no surprise that electronic health record designed for purposes other than quality improvement and used within a fragmented care delivery system would benefit greatly from common data representation, measure harmony, and consistency. We suggest that by enabling measure authors and implementers to deliver consistent electronic quality measure content in four key areas; the industry can improve quality measurement.

SNOMED CT Concept Hierarchies for Computable Clinical Phenotypes From Electronic Health Record Data: Comparison of Intensional Versus Extensional Value Sets

Background

Defining clinical phenotypes from electronic health record (EHR)–derived data proves crucial for clinical decision support, population health endeavors, and translational research. EHR diagnoses now commonly draw from a finely grained clinical terminology—either native SNOMED CT or a vendor-supplied terminology mapped to SNOMED CT concepts as the standard for EHR interoperability. Accordingly, electronic clinical quality measures (eCQMs) increasingly define clinical phenotypes with SNOMED CT value sets. The work of creating and maintaining list-based value sets proves daunting, as does insuring that their contents accurately represent the clinically intended condition.

Objective

The goal of the research was to compare an intensional (concept hierarchy-based) versus extensional (list-based) value set approach to defining clinical phenotypes using SNOMED CT–encoded data from EHRs by evaluating value set conciseness, time to create, and completeness.

Methods

Starting from published Centers for Medicare and Medicaid Services (CMS) high-priority eCQMs, we selected 10 clinical conditions referenced by those eCQMs. For each, the published SNOMED CT list-based (extensional) value set was downloaded from the Value Set Authority Center (VSAC). Ten corresponding SNOMED CT hierarchy-based intensional value sets for the same conditions were identified within our EHR. From each hierarchy-based intensional value set, an exactly equivalent full extensional value set was derived enumerating all included descendant SNOMED CT concepts. Comparisons were then made between (1) VSAC-downloaded list-based (extensional) value sets, (2) corresponding hierarchy-based intensional value sets for the same conditions, and (3) derived list-based (extensional) value sets exactly equivalent to the hierarchy-based intensional value sets. Value set conciseness was assessed by the number of SNOMED CT concepts needed for definition. Time to construct the value sets for local use was measured. Value set completeness was assessed by comparing contents of the downloaded extensional versus intensional value sets. Two measures of content completeness were made: for individual SNOMED CT concepts and for the mapped diagnosis clinical terms available for selection within the EHR by clinicians.

Results

The 10 hierarchy-based intensional value sets proved far simpler and faster to construct than exactly equivalent derived extensional value set lists, requiring a median 3 versus 78 concepts to define and 5 versus 37 minutes to build. The hierarchy-based intensional value sets also proved more complete: in comparison, the 10 downloaded 2018 extensional value sets contained a median of just 35% of the intensional value sets’ SNOMED CT concepts and 65% of mapped EHR clinical terms.

Conclusions

In the EHR era, defining conditions preferentially should employ SNOMED CT concept hierarchy-based (intensional) value sets rather than extensional lists. By doing so, clinical guideline and eCQM authors can more readily engage specialists in vetting condition subtypes to include and exclude, and streamline broad EHR implementation of condition-specific decision support promoting guideline adherence for patient benefit.