A federated EHR network data completeness tracking system

Real-World Data Quality Framework for Oncology Time to Treatment Discontinuation Use Case: Implementation and Evaluation Study

BACKGROUND

The importance of real-world evidence is widely recognized in observational oncology studies. However, the lack of interoperable data quality standards in the fragmented health information technology landscape represents an important challenge. Therefore, adopting validated systematic methods for evaluating data quality is important for oncology outcomes research leveraging real-world data (RWD).

OBJECTIVE

This study aims to implement real-world time to treatment discontinuation (rwTTD) for a systemic anticancer therapy (SACT) as a new use case for the Use Case Specific Relevance and Quality Assessment, a framework linking data quality and relevance in fit-for-purpose RWD assessment.

METHODS

To define the rwTTD use case, we mapped the operational definition of rwTTD to RWD elements commonly available from oncology electronic health record-derived data sets. We identified 20 tasks to check the completeness and plausibility of data elements concerning SACT use, line of therapy (LOT), death date, and length of follow-up. Using descriptive statistics, we illustrated how to implement the Use Case Specific Relevance and Quality Assessment on 2 oncology databases (Data sets A and B) to estimate the rwTTD of an SACT drug (target SACT) for patients with advanced head and neck cancer diagnosed on or after January 1, 2015.

RESULTS

A total of 1200 (24.96%) of 4808 patients in Data set A and 237 (5.92%) of 4003 patients in Data set B received the target SACT, suggesting better relevance of the former in estimating the rwTTD of the target SACT. The 2 data sets differed with regard to the terminology used for SACT drugs, LOT format, and target SACT LOT distribution over time. Data set B appeared to have less complete SACT records, longer lags in incorporating the latest data, and incomplete mortality data, suggesting a lack of fitness for estimating rwTTD.

CONCLUSIONS

The fit-for-purpose data quality assessment demonstrated substantial variability in the quality of the 2 real-world data sets. The data quality specifications applied for rwTTD estimation can be expanded to support a broad spectrum of oncology use cases.

Standardizing nursing data extracted from electronic health records for integration into a statewide clinical data research network

BACKGROUND

Care plans documented by nurses in electronic health records (EHR) are a rich source of data to generate knowledge and measure the impact of nursing care. Unfortunately, there is a lack of integration of these data in clinical data research networks (CDRN) data trusts, due in large part to nursing care being documented with local vocabulary, resulting in non-standardized data. The absence of high-quality nursing care plan data in data trusts limits the investigation of interdisciplinary care aimed at improving patient outcomes.

OBJECTIVE

To map local nursing care plan terms for patients' problems and goals in the EHR of one large health system to the standardized nursing terminologies (SNTs), NANDA International (NANDA-I), and Nursing Outcomes Classification (NOC).

METHODS

We extracted local problems and goals used by nurses to document care plans from two hospitals. After removing duplicates, the terms were independently mapped to NANDA-I and NOC by five mappers. Four nurses who regularly use the local vocabulary validated the mapping.

RESULTS

83% of local problem terms were mapped to NANDA-I labels and 93% of local goal terms were mapped to NOC labels. The nurses agreed with 95% of the mapping. Local terms not mapped to labels were mapped to the domains or classes of the respective terminologies.

CONCLUSION

Mapping local vocabularies used by nurses in EHRs to SNTs is a foundational step to making interoperable nursing data available for research and other secondary purposes in large data trusts. This study is the first phase of a larger project building, for the first time, a pipeline to standardize, harmonize, and integrate nursing care plan data from multiple Florida hospitals into the statewide CDRN OneFlorida+ Clinical Research Network data trust.

A broadly applicable approach to enrich electronic-health-record cohorts by identifying patients with complete data: a multisite evaluation

OBJECTIVE

Patients who receive most care within a single healthcare system (colloquially called a "loyalty cohort" since they typically return to the same providers) have mostly complete data within that organization's electronic health record (EHR). Loyalty cohorts have low data missingness, which can unintentionally bias research results. Using proxies of routine care and healthcare utilization metrics, we compute a per-patient score that identifies a loyalty cohort.

MATERIALS AND METHODS

We implemented a computable program for the widely adopted i2b2 platform that identifies loyalty cohorts in EHRs based on a machine-learning model, which was previously validated using linked claims data. We developed a novel validation approach, which tests, using only EHR data, whether patients returned to the same healthcare system after the training period. We evaluated these tools at 3 institutions using data from 2017 to 2019.

RESULTS

Loyalty cohort calculations to identify patients who returned during a 1-year follow-up yielded a mean area under the receiver operating characteristic curve of 0.77 using the original model and 0.80 after calibrating the model at individual sites. Factors such as multiple medications or visits contributed significantly at all sites. Screening tests' contributions (eg, colonoscopy) varied across sites, likely due to coding and population differences.

DISCUSSION

This open-source implementation of a "loyalty score" algorithm had good predictive power. Enriching research cohorts by utilizing these low-missingness patients is a way to obtain the data completeness necessary for accurate causal analysis.

CONCLUSION

i2b2 sites can use this approach to select cohorts with mostly complete EHR data.

Automating Electronic Health Record Data Quality Assessment

Information systems such as Electronic Health Record (EHR) systems are susceptible to data quality (DQ) issues. Given the growing importance of EHR data, there is an increasing demand for strategies and tools to help ensure that available data are fit for use. However, developing reliable data quality assessment (DQA) tools necessary for guiding and evaluating improvement efforts has remained a fundamental challenge. This review examines the state of research on operationalising EHR DQA, mainly automated tooling, and highlights necessary considerations for future implementations. We reviewed 1841 articles from PubMed, Web of Science, and Scopus published between 2011 and 2021. 23 DQA programs deployed in real-world settings to assess EHR data quality (n = 14), and a few experimental prototypes (n = 9), were identified. Many of these programs investigate completeness (n = 15) and value conformance (n = 12) quality dimensions and are backed by knowledge items gathered from domain experts (n = 9), literature reviews and existing DQ measurements (n = 3). A few DQA programs also explore the feasibility of using data-driven techniques to assess EHR data quality automatically. Overall, the automation of EHR DQA is gaining traction, but current efforts are fragmented and not backed by relevant theory. Existing programs also vary in scope, type of data supported, and how measurements are sourced. There is a need to standardise programs for assessing EHR data quality, as current evidence suggests their quality may be unknown.

The Future of Critical Care: Optimizing Technologies and a Learning Healthcare System to Potentiate a More Humanistic Approach to Critical Care

While technological innovations are the invariable crux of speculation about the future of critical care, they cannot replace the clinician at the bedside. This article summarizes the work of the Society of Critical Care Medicine–appointed multiprofessional task for the Future of Critical Care. The Task Force notes that critical care practice will be transformed by novel technologies, integration of artificial intelligence decision support algorithms, and advances in seamless data operationalization across diverse healthcare systems and geographic regions and within federated datasets. Yet, new technologies will be relevant and meaningful only if they improve the very human endeavor of caring for someone who is critically ill.

Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

INTRODUCTION

Secondary use of electronic health record (EHR) data for research requires that the data are fit for use. Data quality (DQ) frameworks have traditionally focused on structural conformance and completeness of clinical data extracted from source systems. In this paper, we propose a framework for evaluating semantic DQ that will allow researchers to evaluate fitness for use prior to analyses.

METHODS

We reviewed current DQ literature, as well as experience from recent multisite network studies, and identified gaps in the literature and current practice. Derived principles were used to construct the conceptual framework with attention to both analytic fitness and informatics practice.

RESULTS

We developed a systematic framework that guides researchers in assessing whether a data source is fit for use for their intended study or project. It combines tools for evaluating clinical context with DQ principles, as well as factoring in the characteristics of the data source, in order to develop semantic DQ checks.

CONCLUSIONS

Our framework provides a systematic process for DQ development. Further work is needed to codify practices and metadata around both structural and semantic data quality.

Evaluating Completeness of Discrete Data on Physical Functioning for Children With Cerebral Palsy in a Pediatric Rehabilitation Learning Health System

OBJECTIVE

The purpose of this study was to determine the extent that physical function discrete data elements (DDE) documented in electronic health records (EHR) are complete within pediatric rehabilitation settings.

METHODS

A descriptive analysis on completeness of EHR-based DDEs detailing physical functioning for children with cerebral palsy was conducted. Data from an existing pediatric rehabilitation research learning health system data network, consisting of EHR data from 20 care sites in a pediatric specialty health care system, were leveraged. Completeness was calculated for unique data elements, unique outpatient visits, and unique outpatient records.

RESULTS

Completeness of physical function DDEs was low across 5766 outpatient records (10.5%, approximately 2 DDEs documented). The DDE for Gross Motor Function Classification System level was available for 21% (n = 3746) outpatient visits and 38% of patient records. Ambulation level was the most frequently documented DDE. Intercept only mixed effects models demonstrated that 21.4% and 45% of the variance in completeness for DDEs and the Gross Motor Function Classification System, respectively, across unique patient records could be attributed to factors at the individual care site level.

CONCLUSION

Values of physical function DDEs are missing in designated fields of the EHR infrastructure for pediatric rehabilitation providers. Although completeness appears limited for these DDEs, our observations indicate that data are not missing at random and may be influenced by system-level standards in clinical documentation practices between providers and factors specific to individual care sites. The extent of missing data has significant implications for pediatric rehabilitation quality measurement. More research is needed to understand why discrete data are missing in EHRs and to further elucidate the professional and system-level factors that influence completeness and missingness.

IMPACT

Completeness of DDEs reported in this study is limited and presents a significant opportunity to improve documentation and standards to optimize EHR data for learning health system research and quality measurement in pediatric rehabilitation settings.

Assessing real-world medication data completeness

OBJECTIVE

Analysis of healthcare Real-World Data (RWD) provides an opportunity to observe actual patient diagnostic, treatment and outcomes events. However, researchers should understand the possible limitations of RWD. In particular, these data may be incomplete, which would affect the validity of study conclusions.

MATERIALS AND METHODS

The completeness of medication RWD was investigated by analyzing the incidence of various diagnosis-medication couplets: the occurrence of a certain medication in the RWD for a patient having a certain diagnosis. Diagnosis and medication data were obtained from 61 U.S. medical data provider organizations, members of the TriNetX global research network. The number of patients having 22 diagnoses and expected medications were obtained at each institution, and the percent completion of each diagnosis-medication couplet calculated. The study hypothesis is that the degree of couplet completeness can serve as a proxy for overall completeness of medication data for a given organization.

RESULTS

Five diagnosis-medication couplets were found to be reliable proxies, having at least a peak 87% observed completeness for the organizations studied: Type 1 diabetes mellitus and insulin; asthma and albuterol; congestive heart failure and diuretics; cardiovascular disease and aspirin; hypothyroidism and levothyroxine.

DISCUSSION

These couplets were validated as reliable indicators by determining their status as standards of care. The degree to which patients with these five diagnoses had the specified associated medication was consistent within an organization data set.

CONCLUSION

The overall degree of medication data completeness for an organization can be assessed by measuring the completeness of certain indicator diagnosis-medication couplets.

Towards a European health research and innovation cloud (HRIC)

The European Union (EU) initiative on the Digital Transformation of Health and Care (Digicare) aims to provide the conditions necessary for building a secure, flexible, and decentralized digital health infrastructure. Creating a European Health Research and Innovation Cloud (HRIC) within this environment should enable data sharing and analysis for health research across the EU, in compliance with data protection legislation while preserving the full trust of the participants. Such a HRIC should learn from and build on existing data infrastructures, integrate best practices, and focus on the concrete needs of the community in terms of technologies, governance, management, regulation, and ethics requirements. Here, we describe the vision and expected benefits of digital data sharing in health research activities and present a roadmap that fosters the opportunities while answering the challenges of implementing a HRIC. For this, we put forward five specific recommendations and action points to ensure that a European HRIC: i) is built on established standards and guidelines, providing cloud technologies through an open and decentralized infrastructure; ii) is developed and certified to the highest standards of interoperability and data security that can be trusted by all stakeholders; iii) is supported by a robust ethical and legal framework that is compliant with the EU General Data Protection Regulation (GDPR); iv) establishes a proper environment for the training of new generations of data and medical scientists; and v) stimulates research and innovation in transnational collaborations through public and private initiatives and partnerships funded by the EU through Horizon 2020 and Horizon Europe.