Evaluating Foundational Data Quality in the National Patient-Centered Clinical Research Network (PCORnet®)

Concordance With Screening and Treatment Guidelines for Chronic Kidney Disease in Type 2 Diabetes

Importance

Chronic kidney disease (CKD) is an often-asymptomatic complication of type 2 diabetes (T2D) that requires annual screening to diagnose. Patient-level factors linked to inadequate screening and treatment can inform implementation strategies to facilitate guideline-recommended CKD care.

Objective

To identify risk factors for nonconcordance with guideline-recommended CKD screening and treatment in patients with T2D.

Design, Setting, and Participants

This retrospective cohort study was performed at 20 health care systems contributing data to the US National Patient-Centered Clinical Research Network. To evaluate concordance with CKD screening guidelines, adults with an outpatient clinician visit linked to T2D diagnosis between January 1, 2015, and December 31, 2020, and without known CKD were included. A separate analysis reviewed prescription of angiotensin-converting enzyme inhibitors (ACEIs) or angiotensin receptor blockers (ARBs) and sodium-glucose cotransporter 2 (SGLT2) inhibitors in adults with CKD (estimated glomerular filtration rate [eGFR] of 30-90 mL/min/1.73 m2 and urinary albumin-to-creatinine ratio [UACR] of 200-5000 mg/g) and an outpatient clinician visit for T2D between October 1, 2019, and December 31, 2020. Data were analyzed from July 8, 2022, through June 22, 2023.

Exposures

Demographics, lifestyle factors, comorbidities, medications, and laboratory results.

Main Outcomes and Measures

Screening required measurement of creatinine levels and UACR within 15 months of the index visit. Treatment reflected prescription of ACEIs or ARBs and SGLT2 inhibitors within 12 months before or 6 months following the index visit.

Results

Concordance with CKD screening guidelines was assessed in 316 234 adults (median age, 59 [IQR, 50-67] years), of whom 51.5% were women; 21.7%, Black; 10.3%, Hispanic; and 67.6%, White. Only 24.9% received creatinine and UACR screening, 56.5% received 1 screening measurement, and 18.6% received neither. Hispanic ethnicity was associated with lack of screening (relative risk [RR], 1.16 [95% CI, 1.14-1.18]). In contrast, heart failure, peripheral arterial disease, and hypertension were associated with a lower risk of nonconcordance. In 4215 patients with CKD and albuminuria, 3288 (78.0%) received an ACEI or ARB; 194 (4.6%), an SGLT2 inhibitor; and 885 (21.0%), neither therapy. Peripheral arterial disease and lower eGFR were associated with lack of CKD treatment, while diuretic or statin prescription and hypertension were associated with treatment.

Conclusions and Relevance

In this cohort study of patients with T2D, fewer than one-quarter received recommended CKD screening. In patients with CKD and albuminuria, 21.0% did not receive an SGLT2 inhibitor or an ACEI or an ARB, despite compelling indications. Patient-level factors may inform implementation strategies to improve CKD screening and treatment in people with T2D.

Cardiology Encounters for Underrepresented Racial and Ethnic Groups with Human Immunodeficiency Virus and Borderline Cardiovascular Disease Risk

BACKGROUND

Underrepresented racial and ethnic groups (UREGs) with HIV have a higher risk of cardiovascular disease (CVD) compared with the general population. Referral to a cardiovascular specialist improves CVD risk factor management in high-risk individuals. However, patient and provider factors impacting the likelihood of UREGs with HIV to have an encounter with a cardiologist are unknown.

METHODS

We evaluated a cohort of UREGs with HIV and borderline CVD risk (10-year risk ≥ 5% by the pooled cohort equations or ≥ 7.5% by Framingham risk score). Participants received HIV-related care from 2014-2020 at four academic medical centers in the United States (U.S.). Adjusted Cox proportional hazards regression was used to estimate the association of patient and provider characteristics with time to first ambulatory cardiology encounter.

RESULTS

A total of 2,039 people with HIV (PWH) and borderline CVD risk were identified. The median age was 45 years (IQR: 36-50); 52% were female; and 94% were Black. Of these participants, 283 (14%) had an ambulatory visit with a cardiologist (17% of women vs. 11% of men, p < .001). In fully adjusted models, older age, higher body mass index (BMI), atrial fibrillation, multimorbidity, urban residence, and no recent insurance were associated with a greater likelihood of an encounter with a cardiologist.

CONCLUSION

In UREGs with HIV and borderline CVD risk, the strongest determinants of a cardiology encounter were diagnosed CVD, insurance type, and urban residence. Future research is needed to determine the extent to which these encounters impact CVD care practices and outcomes in this population.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04025125.

Trends of Antihypertensive Prescription Among US Adults From 2010 to 2019 and Changes Following Treatment Guidelines: Analysis of Multicenter Electronic Health Records

BACKGROUND

Guidelines for the use of antihypertensives changed in 2014 and 2017. To understand the effect of these guidelines, we examined trends in antihypertensive prescriptions in the United States from 2010 to 2019 using a repeated cross-sectional design.

METHODS AND RESULTS

Using electronic health records from 15 health care institutions for adults (20-85 years old) who had ≥1 antihypertensive prescription, we assessed whether (1) prescriptions of beta blockers decreased after the 2014 Eighth Joint National Committee (JNC 8) report discouraged use for first-line treatment, (2) prescriptions for calcium channel blockers and thiazide diuretics increased among Black patients after the JNC 8 report encouraged use as first-line therapy, and (3) prescriptions for dual therapy and fixed-dose combination among patients with blood pressure ≥140/90 mm Hg increased after recommendations in the 2017 Hypertension Clinical Practice Guidelines. The study included 1 074 314 patients with 2 133 158 prescription episodes. After publication of the JNC 8 report, prescriptions for beta blockers decreased (3% lower in 2018-2019 compared to 2010-2014), and calcium channel blockers increased among Black patients (20% higher in 2015-2017 and 41% higher in 2018-2019, compared to 2010-2014), in accordance with guideline recommendations. However, contrary to guidelines, dual therapy and fixed-dose combination decreased after publication of the 2017 Hypertension Clinical Practice Guidelines (9% and 11% decrease in 2018-2019 for dual therapy and fixed-dose combination, respectively, compared to 2015-2017), and thiazide diuretics decreased among Black patients after the JNC 8 report (6% lower in 2018-2019 compared to 2010-2014).

CONCLUSIONS

Adherence to guidelines on prescribing antihypertensive medication was inconsistent, presenting an opportunity for interventions to achieve better blood pressure control in the US population.

A large, multi-center survey assessing health, social support, literacy, and self-management resources in patients with heart failure

BACKGROUND

Most patients with heart failure (HF) have multimorbidity which may cause difficulties with self-management. Understanding the resources patients draw upon to effectively manage their health is fundamental to designing new practice models to improve outcomes in HF. We describe the rationale, conceptual framework, and implementation of a multi-center survey of HF patients, characterize differences between responders and non-responders, and summarize patient characteristics and responses to the survey constructs among responders.

METHODS

This was a multi-center cross-sectional survey study with linked electronic health record (EHR) data. Our survey was guided by the Chronic Care Model to understand the distribution of patient-centric factors, including health literacy, social support, self-management, and functional and mental status in patients with HF. Most questions were from existing validated questionnaires. The survey was administered to HF patients aged ≥ 30 years from 4 health systems in PCORnet® (the National Patient-Centered Clinical Research Network): Essentia Health, Intermountain Health, Mayo Clinic, and The Ohio State University. Each health system mapped their EHR data to a standardized PCORnet Common Data Model, which was used to extract demographic and clinical data on survey responders and non-responders.

RESULTS

Across the 4 sites, 10,662 patients with HF were invited to participate, and 3330 completed the survey (response rate: 31%). Responders were older (74 vs. 71 years; standardized difference (95% CI): 0.18 (0.13, 0.22)), less racially diverse (3% vs. 12% non-White; standardized difference (95% CI): -0.32 (-0.36, -0.28)), and had higher prevalence of many chronic conditions than non-responders, and thus may not be representative of all HF patients. The internal reliability of the validated questionnaires in our survey was good (range of Cronbach's alpha: 0.50-0.96). Responders reported their health was generally good or fair, they frequently had cardiovascular comorbidities, > 50% had difficulty climbing stairs, and > 10% reported difficulties with bathing, preparing meals, and using transportation. Nearly 80% of patients had family or friends sit with them during a doctor visit, and 54% managed their health by themselves. Patients reported generally low perceived support for self-management related to exercise and diet.

CONCLUSIONS

More than half of patients with HF managed their health by themselves. Increased understanding of self-management resources may guide the development of interventions to improve HF outcomes.

Comparative antimicrobial use in coronavirus disease 2019 (COVID-19) and non-COVID-19 inpatients from 2019 to 2020: A multicenter ecological study

OBJECTIVE

We sought to determine whether increased antimicrobial use (AU) at the onset of the coronavirus disease 2019 (COVID-19) pandemic was driven by greater AU in COVID-19 patients only, or whether AU also increased in non-COVID-19 patients.

DESIGN

In this retrospective observational ecological study from 2019 to 2020, we stratified inpatients by COVID-19 status and determined relative percentage differences in median monthly AU in COVID-19 patients versus non-COVID-19 patients during the COVID-19 period (March-December 2020) and the pre-COVID-19 period (March-December 2019). We also determined relative percentage differences in median monthly AU in non-COVID-19 patients during the COVID-19 period versus the pre-COVID-19 period. Statistical significance was assessed using Wilcoxon signed-rank tests.

SETTING

The study was conducted in 3 acute-care hospitals in Chicago, Illinois.

PATIENTS

Hospitalized patients.

RESULTS

Facility-wide AU for broad-spectrum antibacterial agents predominantly used for hospital-onset infections was significantly greater in COVID-19 patients versus non-COVID-19 patients during the COVID-19 period (with relative increases of 73%, 66%, and 91% for hospitals A, B, and C, respectively), and during the pre-COVID-19 period (with relative increases of 52%, 64%, and 66% for hospitals A, B, and C, respectively). In contrast, facility-wide AU for all antibacterial agents was significantly lower in non-COVID-19 patients during the COVID-19 period versus the pre-COVID-19 period (with relative decreases of 8%, 7%, and 8% in hospitals A, B, and C, respectively).

CONCLUSIONS

AU for broad-spectrum antimicrobials was greater in COVID-19 patients compared to non-COVID-19 patients at the onset of the pandemic. AU for all antibacterial agents in non-COVID-19 patients decreased in the COVID-19 period compared to the pre-COVID-19 period.

Mapping Clinical Documents to the Logical Observation Identifiers, Names and Codes (LOINC) Document Ontology using Electronic Health Record Systems Structured Metadata

As Electronic Health Record (EHR) systems increase in usage, organizations struggle to maintain and categorize clinical documentation so it can be used for clinical care and research. While prior research has often employed natural language processing techniques to categorize free text documents, there are shortcomings relative to computational scalability and the lack of key metadata within notes' text. This study presents a framework that can allow institutions to map their notes to the LOINC document ontology using a Bag of Words approach. After preliminary manual value- set mapping, an automated pipeline that leverages key dimensions of metadata from structured EHR fields aligns the notes with the dimensions of the document ontology. This framework resulted in 73.4% coverage of EHR documents, while also mapping 132 million notes in less than 2 hours; an order of magnitude more efficient than NLP based methods.

Association of prior treatment with nitrogen-containing bisphosphonates on outcomes of COVID-19 positive patients

COVID-19 infection has resulted in significant morbidity and mortality globally, especially among older adults. Repurposed drugs have demonstrated activity in respiratory illnesses, including nitrogen-containing bisphosphonates. In this retrospective longitudinal study at 4 academic medical centers, we show no benefit of nitrogen-containing bisphosphonates regarding ICU admission, ventilator use, and mortality among older adults with COVID-19 infection. We specifically evaluated the intravenous bisphosphonate zoledronic acid and found no difference compared to oral bisphosphonates.

BACKGROUND

Widely used in osteoporosis treatment, nitrogen-containing bisphosphonates (N-BP) have been associated with reduced mortality and morbidity among older adults. Based on prior studies, we hypothesized that prior treatment with N-BP might reduce intensive care unit (ICU) admission, ventilator use, and death among older adults diagnosed with COVID-19.

METHODS

This retrospective analysis of the PCORnet Common Data Model across 4 academic medical centers through 1 September 2021 identified individuals age >50 years with a diagnosis of COVID-19. The composite outcome included ICU admission, ventilator use, or death within 15, 30, and 180 days of COVID-19 diagnosis. Use of N-BP was defined as a prescription within 3 years prior. ICU admission and ventilator use were determined using administrative codes. Death included both in-hospital and out-of-hospital events. Patients treated with N-BP were matched 1:1 by propensity score to patients without prior N-BP use. Secondary analysis compared outcomes among those prescribed zoledronic acid (ZOL) to those prescribed oral N-BPs.

RESULTS

Of 76,223 COVID-19 patients identified, 1,853 were previously prescribed N-BP, among whom 559 were prescribed ZOL. After propensity score matching, there were no significant differences in the composite outcome at 15 days (HR 1.22, 95% CI: 0.89-1.67), 30 days (HR 1.24, 95% CI: 0.93-1.66), or 180 days (HR 1.17, 95% CI: 0.93-1.48), comparing those prescribed and not prescribed N-BP. Compared to those prescribed oral N-BP, there were no significant differences in outcomes among those prescribed ZOL.

CONCLUSION

Among older COVID-19 patients, prior exposure to N-BP including ZOL was not associated with a reduction in ICU admission, ventilator use, or death.

Racial disparities in septic shock mortality: a retrospective cohort study

Background

Patients with septic shock have the highest risk of death from sepsis, however, racial disparities in mortality outcomes in this cohort have not been rigorously investigated. Our objective was to describe the association between race/ethnicity and mortality in patients with septic shock.

Methods

Our study is a retrospective cohort study of adult patients in the OneFlorida Data Trust (Florida, United States of America) admitted with septic shock between January 2012 and July 2018. We identified patients as having septic shock if they received vasopressors during their hospital encounter and had either an explicit International Classification of Disease (ICD) code for sepsis, or had an infection ICD code and received intravenous antibiotics. Our primary outcome was 90-day mortality. Our secondary outcome was in-hospital mortality. Multiple logistic regression with Least Absolute Shrinkage and Selection Operator (LASSO) for variable selection was used to assess associations.

Findings

There were 13,932 patients with septic shock in our cohort. The mean age was 61 years (SD 16), 68% of the cohort identified as White (n = 9419), 28% identified as Black (n = 3936), 2% (n = 294) identified as Hispanic ethnicity, and 2% as other races not specified in the previous groups (n = 283). In our logistic regression model for 90-day mortality, patients identified as Black had 1.57 times the odds of mortality (95% CI 1.07-2.29, p = 0.02) compared to White patients. Other significant predictors included mechanical ventilation (OR 3.66, 95% CI 3.35-4.00, p < 0.01), liver disease (OR 1.75, 95% CI 1.59-1.93, p < 0.01), laboratory components of the Sequential Organ Failure Assessment score (OR 1.18, 95% CI 1.16-1.21, p < 0.01), lactate (OR 1.10, 95% CI 1.08-1.12, p < 0.01), congestive heart failure (OR 1.19, 95% CI 1.10-1.30, p < 0.01), human immunodeficiency virus (OR 1.35, 95% CI 1.04-1.75, p = 0.03), age (OR 1.04, 95% CI 1.04-1.04, p < 0.01), and the interaction between age and race (OR 0.99, 95% CI 0.99-1.00, p < 0.01). Among younger patients (<45 years), patients identified as Black accounted for a higher proportion of the deaths. Results were similar in the in-hospital mortality model.

Interpretation

In this retrospective study of septic shock patients, we found that patients identified as Black had higher odds of mortality compared to patients identified as non-Hispanic White. Our findings suggest that the greatest disparities in mortality are among younger Black patients with septic shock.

Funding

National Institutes of Health National Center for Advancing Translational Sciences (1KL2TR001429); National Institute of Health National Institute of General Medical Sciences (1K23GM144802).

Alcohol Misuse and Sexually Transmitted Infections: Using the CAGE Questionnaire as a Screening Tool

Background: While the connection between alcohol and risky behavior is well known, a clear correlation between alcohol misuse and contracting sexually transmitted infections (STIs) has not been determined. The 4-question CAGE questionnaire-the acronym stands for attitudes and activities related to alcohol use-is often administered at primary care annual visits to screen patients for alcohol abuse. This study assessed the relationship between CAGE scores and STI results to determine if the CAGE questionnaire could help determine the need for STI screening at annual visits. Methods: All patients who received a CAGE screening from 2015 to 2022 at a Gulf South health system were included in the analysis. The primary outcome of the study was the relationship between a positive CAGE score (a score ≥2) and a positive STI result. STIs included in the primary analysis were human immunodeficiency virus (HIV), hepatitis B, syphilis, chlamydia, gonorrhea, and trichomoniasis. The correlation between a positive CAGE score and hepatitis C was examined as a secondary outcome. Results: A total of 40,022 patients received a CAGE screening during the study period, and 757 (1.9%) scored ≥2 on the CAGE questionnaire. Significant associations were found between a positive CAGE score and hepatitis B (odds ratio [OR]=2.69, 95% CI 1.91, 3.80; P<0.001), gonorrhea (OR=5.43, 95% CI 1.80, 16.39; P=0.003), and hepatitis C (OR=2.10, 95% CI 1.57, 2.80; P<0.001). No associations were found between a positive CAGE score and HIV, chlamydia, or trichomoniasis. No patients with a CAGE score ≥2 had a syphilis diagnosis; therefore, no syphilis analysis was possible. Conclusion: Based on the results of this study, patients with a CAGE score ≥2 may benefit from screening for hepatitis B, hepatitis C, and gonorrhea at their primary care annual visit. Early STI detection could lead to prompt treatment and prevent further transmission and complications.

Avenues for Strengthening PCORnet's Capacity to Advance Patient-Centered Economic Outcomes in Patient-Centered Outcomes Research (PCOR)

PCORnet, the National Patient-Centered Clinical Research Network, provides the ability to conduct prospective and observational pragmatic research by leveraging standardized, curated electronic health records data together with patient and stakeholder engagement. PCORnet is funded by the Patient-Centered Outcomes Research Institute (PCORI) and is composed of 8 Clinical Research Networks that incorporate at total of 79 health system "sites." As the network developed, linkage to commercial health plans, federal insurance claims, disease registries, and other data resources demonstrated the value in extending the networks infrastructure to provide a more complete representation of patient's health and lived experiences. Initially, PCORnet studies avoided direct economic comparative effectiveness as a topic. However, PCORI's authorizing law was amended in 2019 to allow studies to incorporate patient-centered economic outcomes in primary research aims. With PCORI's expanded scope and PCORnet's phase 3 beginning in January 2022, there are opportunities to strengthen the network's ability to support economic patient-centered outcomes research. This commentary will discuss approaches that have been incorporated to date by the network and point to opportunities for the network to incorporate economic variables for analysis, informed by patient and stakeholder perspectives. Topics addressed include: (1) data linkage infrastructure; (2) commercial health plan partnerships; (3) Medicare and Medicaid linkage; (4) health system billing-based benchmarking; (5) area-level measures; (6) individual-level measures; (7) pharmacy benefits and retail pharmacy data; and (8) the importance of transparency and engagement while addressing the biases inherent in linking real-world data sources.

Optimizing data integration in trials that use EHR data: lessons learned from a multi-center randomized clinical trial

BACKGROUND

Despite great promise, trials that ascertain patient clinical data from electronic health records (EHR), referred to here as "EHR-sourced" trials, are limited by uncertainty about how existing trial sites and infrastructure can be best used to operationalize study goals. Evidence is needed to support the practical use of EHRs in contemporary clinical trial settings.

MAIN TEXT

We describe a demonstration project that used EHR data to complement data collected for a contemporary multi-center pharmaceutical industry outcomes trial, and how a central coordinating center supported participating sites through the technical, governance, and operational aspects of this type of activity. We discuss operational considerations related to site selection, data extraction, site performance, and data transfer and quality review, and we outline challenges and lessons learned. We surveyed potential sites and used their responses to assess feasibility, determine the potential capabilities of sites and choose an appropriate data extraction strategy. We designed a flexible, multimodal approach for data extraction, enabling each site to either leverage an existing data source, create a new research datamart, or send all data to the central coordinating center to produce the requisite data elements. We evaluated site performance, as reflected by the speed of contracting and IRB approval, total patients enrolled, enrollment yield, data quality, and compared performance by data collection strategy.

CONCLUSION

While broadening the type of sites able to participate in EHR-sourced trials may lead to greater generalizability and improved enrollment, sites with fewer technical resources may require additional support to participate. Central coordinating center support is essential to facilitate the execution of operational processes. Future work should focus on sharing lessons learned and creating reusable tools to facilitate participation of heterogeneous trial sites.

Electronic health record data quality variability across a multistate clinical research network

Background

Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems.

Methods

To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality.

Results

The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs.

Conclusion

Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.

Disruption in Blood Pressure Control With the COVID-19 Pandemic: The PCORnet Blood Pressure Control Laboratory

OBJECTIVE

To explore trends in blood pressure (BP) control before and during the COVID-19 pandemic.

PATIENTS AND METHODS

Health systems participating in the National Patient-Centered Clinical Research Network (PCORnet) Blood Pressure Control Laboratory Surveillance System responded to data queries, producing 9 BP control metrics. Averages of the BP control metrics (weighted by numbers of observations in each health system) were calculated and compared between two 1-year measurement periods (January 1, 2019, through December 31, 2019, and January 1, 2020, through December 31, 2020).

RESULTS

Among 1,770,547 hypertensive persons in 2019, BP control to <140/<90 mm Hg varied across 24 health systems (range, 46%-74%). Reduced BP control occurred in most health systems with onset of the COVID-19 pandemic; the weighted average BP control was 60.5% in 2019 and 53.3% in 2020. Reductions were also evident for BP control to <130/<80 mm Hg (29.9% in 2019 and 25.4% in 2020) and improvement in BP (reduction of 10 mm Hg in systolic BP or achievement of systolic BP <140 mm Hg; 29.7% in 2019 and 23.8% in 2020). Two BP control process metrics exhibited pandemic-associated disruption: repeat visit in 4 weeks after a visit with uncontrolled hypertension (36.7% in 2019 and 31.7% in 2020) and prescription of fixed-dose combination medications among those with 2 or more drug classes (24.6% in 2019 and 21.5% in 2020).

CONCLUSION

BP control decreased substantially during the COVID-19 pandemic, with a corresponding reduction in follow-up health care visits among persons with uncontrolled hypertension. It is unclear whether the observed decline in BP control during the pandemic will contribute to future cardiovascular events.

Data quality considerations for evaluating COVID-19 treatments using real world data: learnings from the National COVID Cohort Collaborative (N3C)

BACKGROUND

Multi-institution electronic health records (EHR) are a rich source of real world data (RWD) for generating real world evidence (RWE) regarding the utilization, benefits and harms of medical interventions. They provide access to clinical data from large pooled patient populations in addition to laboratory measurements unavailable in insurance claims-based data. However, secondary use of these data for research requires specialized knowledge and careful evaluation of data quality and completeness. We discuss data quality assessments undertaken during the conduct of prep-to-research, focusing on the investigation of treatment safety and effectiveness.

METHODS

Using the National COVID Cohort Collaborative (N3C) enclave, we defined a patient population using criteria typical in non-interventional inpatient drug effectiveness studies. We present the challenges encountered when constructing this dataset, beginning with an examination of data quality across data partners. We then discuss the methods and best practices used to operationalize several important study elements: exposure to treatment, baseline health comorbidities, and key outcomes of interest.

RESULTS

We share our experiences and lessons learned when working with heterogeneous EHR data from over 65 healthcare institutions and 4 common data models. We discuss six key areas of data variability and quality. (1) The specific EHR data elements captured from a site can vary depending on source data model and practice. (2) Data missingness remains a significant issue. (3) Drug exposures can be recorded at different levels and may not contain route of administration or dosage information. (4) Reconstruction of continuous drug exposure intervals may not always be possible. (5) EHR discontinuity is a major concern for capturing history of prior treatment and comorbidities. Lastly, (6) access to EHR data alone limits the potential outcomes which can be used in studies.

CONCLUSIONS

The creation of large scale centralized multi-site EHR databases such as N3C enables a wide range of research aimed at better understanding treatments and health impacts of many conditions including COVID-19. As with all observational research, it is important that research teams engage with appropriate domain experts to understand the data in order to define research questions that are both clinically important and feasible to address using these real world data.

The IeDEA harmonist data toolkit: A data quality and data sharing solution for a global HIV research consortium

We describe the design, implementation, and impact of a data harmonization, data quality checking, and dynamic report generation application in an international observational HIV research network. The IeDEA Harmonist Data Toolkit is a web-based application written in the open source programming language R, employs the R/Shiny and RMarkdown packages, and leverages the REDCap data collection platform for data model definition and user authentication. The Toolkit performs data quality checks on uploaded datasets, checks for conformance with the network's common data model, displays the results both interactively and in downloadable reports, and stores approved datasets in secure cloud storage for retrieval by the requesting investigator. Including stakeholders and users in the design process was key to the successful adoption of the application. A survey of regional data managers as well as initial usage metrics indicate that the Toolkit saves time and results in improved data quality, with a 61% mean reduction in the number of error records in a dataset. The generalized application design allows the Toolkit to be easily adapted to other research networks.

Global Regulatory and Public Health Initiatives to Advance Pediatric Drug Development for Rare Diseases

The literature thoroughly describes the challenges of pediatric drug development for rare diseases. This includes (1) generating interest from sponsors, (2) small numbers of children affected by a particular disease, (3) difficulties with study design, (4) lack of definitive outcome measures and assessment tools, (5) the need for additional safeguards for children as a vulnerable population, and (6) logistical hurdles to completing trials, especially with the need for longer term follow-up to establish safety and efficacy. There has also been an increasing awareness of the need to engage patients and their families in drug development processes and to address inequities in access to pediatric clinical trials. The year 2020 ushered in yet another challenge—the COVID-19 pandemic. The pediatric drug development ecosystem continues to evolve to meet these challenges. This article will focus on several key factors including recent regulatory approaches and public health policies to facilitate pediatric rare disease drug development, emerging trends in product development (biologics, molecularly targeted therapies), innovations in trial design/endpoints and data collection, and current efforts to increase patient engagement and promote equity. Finally, lessons learned from COVID-19 about building adaptable pediatric rare disease drug development processes will be discussed.

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations

OBJECTIVE

To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.

METHODS

We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments.

RESULTS

Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments.

DISCUSSION

These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators.

CONCLUSION

These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.

Data Integration Challenges for Machine Learning in Precision Medicine

A main goal of Precision Medicine is that of incorporating and integrating the vast corpora on different databases about the molecular and environmental origins of disease, into analytic frameworks, allowing the development of individualized, context-dependent diagnostics, and therapeutic approaches. In this regard, artificial intelligence and machine learning approaches can be used to build analytical models of complex disease aimed at prediction of personalized health conditions and outcomes. Such models must handle the wide heterogeneity of individuals in both their genetic predisposition and their social and environmental determinants. Computational approaches to medicine need to be able to efficiently manage, visualize and integrate, large datasets combining structure, and unstructured formats. This needs to be done while constrained by different levels of confidentiality, ideally doing so within a unified analytical architecture. Efficient data integration and management is key to the successful application of computational intelligence approaches to medicine. A number of challenges arise in the design of successful designs to medical data analytics under currently demanding conditions of performance in personalized medicine, while also subject to time, computational power, and bioethical constraints. Here, we will review some of these constraints and discuss possible avenues to overcome current challenges.

Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

INTRODUCTION

Secondary use of electronic health record (EHR) data for research requires that the data are fit for use. Data quality (DQ) frameworks have traditionally focused on structural conformance and completeness of clinical data extracted from source systems. In this paper, we propose a framework for evaluating semantic DQ that will allow researchers to evaluate fitness for use prior to analyses.

METHODS

We reviewed current DQ literature, as well as experience from recent multisite network studies, and identified gaps in the literature and current practice. Derived principles were used to construct the conceptual framework with attention to both analytic fitness and informatics practice.

RESULTS

We developed a systematic framework that guides researchers in assessing whether a data source is fit for use for their intended study or project. It combines tools for evaluating clinical context with DQ principles, as well as factoring in the characteristics of the data source, in order to develop semantic DQ checks.

CONCLUSIONS

Our framework provides a systematic process for DQ development. Further work is needed to codify practices and metadata around both structural and semantic data quality.

Electronic Health Record Network Research in Infectious Diseases

With the marked increases in electronic health record (EHR) use for providing clinical care, there have been parallel efforts to leverage EHR data for research. EHR repositories offer the promise of vast amounts of clinical data not easily captured with traditional research methods and facilitate clinical epidemiology and comparative effectiveness research, including analyses to identify patients at higher risk for complications or who are better candidates for treatment. These types of studies have been relatively slow to penetrate the field of infectious diseases, but the need for rapid turnaround during the COVID-19 global pandemic has accelerated the uptake. This review discusses the rationale for her network projects, opportunities and challenges that such networks present, and some prior studies within the field of infectious diseases.

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

Background  Many research initiatives aim at using data from electronic health records (EHRs) in observational studies. Participating sites of the German Medical Informatics Initiative (MII) established data integration centers to integrate EHR data within research data repositories to support local and federated analyses. To address concerns regarding possible data quality (DQ) issues of hospital routine data compared with data specifically collected for scientific purposes, we have previously presented a data quality assessment (DQA) tool providing a standardized approach to assess DQ of the research data repositories at the MIRACUM consortium's partner sites.

Objectives  Major limitations of the former approach included manual interpretation of the results and hard coding of analyses, making their expansion to new data elements and databases time-consuming and error prone. We here present an enhanced version of the DQA tool by linking it to common data element definitions stored in a metadata repository (MDR), adopting the harmonized DQA framework from Kahn et al and its application within the MIRACUM consortium.

Methods  Data quality checks were consequently aligned to a harmonized DQA terminology. Database-specific information were systematically identified and represented in an MDR. Furthermore, a structured representation of logical relations between data elements was developed to model plausibility-statements in the MDR.

Results  The MIRACUM DQA tool was linked to data element definitions stored in a consortium-wide MDR. Additional databases used within MIRACUM were linked to the DQ checks by extending the respective data elements in the MDR with the required information. The evaluation of DQ checks was automated. An adaptable software implementation is provided with the R package DQAstats .

Conclusion  The enhancements of the DQA tool facilitate the future integration of new data elements and make the tool scalable to other databases and data models. It has been provided to all ten MIRACUM partners and was successfully deployed and integrated into their respective data integration center infrastructure.

Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

OBJECTIVE

Large clinical databases are increasingly used for research and quality improvement. We describe an approach to data quality assessment from the General Medicine Inpatient Initiative (GEMINI), which collects and standardizes administrative and clinical data from hospitals.

METHODS

The GEMINI database contained 245 559 patient admissions at 7 hospitals in Ontario, Canada from 2010 to 2017. We performed 7 computational data quality checks and iteratively re-extracted data from hospitals to correct problems. Thereafter, GEMINI data were compared to data that were manually abstracted from the hospital's electronic medical record for 23 419 selected data points on a sample of 7488 patients.

RESULTS

Computational checks flagged 103 potential data quality issues, which were either corrected or documented to inform future analysis. For example, we identified the inclusion of canceled radiology tests, a time shift of transfusion data, and mistakenly processing the chemical symbol for sodium ("Na") as a missing value. Manual validation identified 1 important data quality issue that was not detected by computational checks: transfusion dates and times at 1 site were unreliable. Apart from that single issue, across all data tables, GEMINI data had high overall accuracy (ranging from 98%-100%), sensitivity (95%-100%), specificity (99%-100%), positive predictive value (93%-100%), and negative predictive value (99%-100%) compared to the gold standard.

DISCUSSION AND CONCLUSION

Computational data quality checks with iterative re-extraction facilitated reliable data collection from hospitals but missed 1 critical quality issue. Combining computational and manual approaches may be optimal for assessing the quality of large multisite clinical databases.

Comparative incidence and outcomes of COVID-19 in kidney or kidney-pancreas transplant recipients versus kidney or kidney-pancreas waitlisted patients: A single-center study

BACKGROUND

COVID-19 epidemiologic studies comparing immunosuppressed and immunocompetent patients may provide insight into the impact of immunosuppressants on outcomes.

METHODS

In this retrospective cohort study, we assembled kidney or kidney-pancreas transplant recipients who underwent transplant from January 1, 2010, to June 30, 2020, and kidney or kidney-pancreas waitlisted patients who were ever on the waitlist from January 1, 2019, to June 30, 2020. We identified laboratory-confirmed COVID-19 until January 31, 2021, and tracked its outcomes by leveraging informatics infrastructure developed for an outcomes research network.

RESULTS

COVID-19 was identified in 62 of 887 kidney or kidney-pancreas transplant recipients and 20 of 434 kidney or kidney-pancreas waitlisted patients (7.0% vs. 4.6%, p = .092). Of these patients with COVID-19, hospitalization occurred in 48 of 62 transplant recipients and 8 of 20 waitlisted patients (77% vs. 40%, p = .002); intensive care unit admission occurred in 18 of 62 transplant recipients and 2 of 20 waitlisted patients (29% vs. 10%, p = .085); and 7 transplant recipients were mechanically ventilated and died, whereas no waitlisted patients were mechanically ventilated or died (11% vs. 0%, p = .116).

CONCLUSIONS

Our study provides single-center data and an informatics approach that can be used to inform the design of multicenter studies.

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data

Objective

To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).

Materials and Methods

We started with 3 widely cited DQ literature—2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)—and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.

Results

We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.

Discussion

Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.

Conclusion

The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.

Recommendations for improving national clinical datasets for health equity research

Health and healthcare disparities continue despite clinical, research, and policy efforts. Large clinical datasets may not contain data relevant to healthcare disparities and leveraging these for research may be crucial to improve health equity. The Health Disparities Collaborative Research Group was commissioned by the Patient-Centered Outcomes Research Institute to examine the data science needs for quality and complete data and provide recommendations for improving data science around health disparities. The group convened content experts, researchers, clinicians, and patients to produce these recommendations and suggestions for implementation. Our desire was to produce recommendations to improve the usability of healthcare datasets for health equity research. The recommendations are summarized in 3 primary domains: patient voice, accurate variables, and data linkage. The implementation of these recommendations in national datasets has the potential to accelerate health disparities research and promote efforts to reduce health inequities.

Using UMLS for electronic health data standardization and database design

OBJECTIVE

Patients that undergo medical transfer represent 1 patient population that remains infrequently studied due to challenges in aggregating data across multiple domains and sources that are necessary to capture the entire episode of patient care. To facilitate access to and secondary use of transport patient data, we developed the Transport Data Repository that combines data from 3 separate domains and many sources within our health system.

METHODS

The repository is a relational database anchored by the Unified Medical Language System unique concept identifiers to integrate, map, and standardize the data into a common data model. Primary data domains included sending and receiving hospital encounters, medical transport record, and custom hospital transport log data. A 4-step mapping process was developed: 1) automatic source code match, 2) exact text match, 3) fuzzy matching, and 4) manual matching.

RESULTS

431 090 total mappings were generated in the Transport Data Repository, consisting of 69 010 unique concepts with 77% of the data being mapped automatically. Transport Source Data yielded significantly lower mapping results with only 8% of data entities automatically mapped and a significant amount (43%) remaining unmapped.

DISCUSSION

The multistep mapping process resulted in a majority of data been automatically mapped. Poor matching of transport medical record data is due to the third-party vendor data being generated and stored in a nonstandardized format.

CONCLUSION

The multistep mapping process developed and implemented is necessary to normalize electronic health data from multiple domains and sources into a common data model to support secondary use of data.

Quality assessment of real-world data repositories across the data life cycle: A literature review

OBJECTIVE

Data quality (DQ) must be consistently defined in context. The attributes, metadata, and context of longitudinal real-world data (RWD) have not been formalized for quality improvement across the data production and curation life cycle. We sought to complete a literature review on DQ assessment frameworks, indicators and tools for research, public health, service, and quality improvement across the data life cycle.

MATERIALS AND METHODS

The review followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Databases from health, physical and social sciences were used: Cinahl, Embase, Scopus, ProQuest, Emcare, PsycINFO, Compendex, and Inspec. Embase was used instead of PubMed (an interface to search MEDLINE) because it includes all MeSH (Medical Subject Headings) terms used and journals in MEDLINE as well as additional unique journals and conference abstracts. A combined data life cycle and quality framework guided the search of published and gray literature for DQ frameworks, indicators, and tools. At least 2 authors independently identified articles for inclusion and extracted and categorized DQ concepts and constructs. All authors discussed findings iteratively until consensus was reached.

RESULTS

The 120 included articles yielded concepts related to contextual (data source, custodian, and user) and technical (interoperability) factors across the data life cycle. Contextual DQ subcategories included relevance, usability, accessibility, timeliness, and trust. Well-tested computable DQ indicators and assessment tools were also found.

CONCLUSIONS

A DQ assessment framework that covers intrinsic, technical, and contextual categories across the data life cycle enables assessment and management of RWD repositories to ensure fitness for purpose. Balancing security, privacy, and FAIR principles requires trust and reciprocity, transparent governance, and organizational cultures that value good documentation.

A federated EHR network data completeness tracking system

OBJECTIVE

The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network.

MATERIALS AND METHODS

The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018.

RESULTS

The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base.

DISCUSSION

Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating.

CONCLUSIONS

The CTX has increased the network's capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.

Leveraging Differential Privacy in Geospatial Analyses of Standardized Healthcare Data

We present a collection of geodatabase functions which expedite utilizing differential privacy for privacy-aware geospatial analysis of healthcare data. The healthcare domain has a long history of standardization and research communities have developed open-source common data models to support the larger goals of interoperability, reproducibility, and data sharing; these models also standardize geospatial patient data. However, patient privacy laws and institutional regulations complicate geospatial analyses and dissemination of research findings due to protective restrictions in how data and results are shared. This results in infrastructures with great abilities to organize and store healthcare data, yet which lack the innate ability to produce shareable results that preserve privacy and conform to regulatory requirements. Differential privacy is a model for performing privacy-preserving analytics. We detail our process and findings in inserting an open-source library for differential privacy into a workflow for leveraging a geodatabase for geocoding and analyzing geospatial data stored as part of the Observational Medical Outcomes Partnership (OMOP) common data model. We pilot this process using an open big data repository of addresses.

Cross-site transportability of an explainable artificial intelligence model for acute kidney injury prediction

Artificial intelligence (AI) has demonstrated promise in predicting acute kidney injury (AKI), however, clinical adoption of these models requires interpretability and transportability. Non-interoperable data across hospitals is a major barrier to model transportability. Here, we leverage the US PCORnet platform to develop an AKI prediction model and assess its transportability across six independent health systems. Our work demonstrates that cross-site performance deterioration is likely and reveals heterogeneity of risk factors across populations to be the cause. Therefore, no matter how accurate an AI model is trained at the source hospital, whether it can be adopted at target hospitals is an unanswered question. To fill the research gap, we derive a method to predict the transportability of AI models which can accelerate the adaptation process of external AI models in hospitals.

Artificial intelligence (AI) has demonstrated promise in predicting acutekidney injury (AKI), however, clinical adoption of these models requires interpretability and transportability across sites. Here, the authors develop an AKI prediction model and a measure for model transportability across six independent health systems.

PCORnet® 2020: current state, accomplishments, and future directions

OBJECTIVE

To describe PCORnet, a clinical research network developed for patient-centered outcomes research on a national scale.

STUDY DESIGN AND SETTING

Descriptive study of the current state and future directions for PCORnet. We conducted cross-sectional analyses of the health systems and patient populations of the 9 Clinical Research Networks and 2 Health Plan Research Networks that are part of PCORnet.

RESULTS

Within the Clinical Research Networks, electronic health data are currently collected from 337 hospitals, 169,695 physicians, 3,564 primary care practices, 338 emergency departments, and 1,024 community clinics. Patients can be recruited for prospective studies from any of these clinical sites. The Clinical Research Networks have accumulated data from 80 million patients with at least one visit from 2009 to 2018. The PCORnet Health Plan Research Network population of individuals with a valid enrollment segment from 2009 to 2019 exceeds 60 million individuals, who on average have 2.63 years of follow-up.

CONCLUSION

PCORnet's infrastructure comprises clinical data from a diverse cohort of patients and has the capacity to rapidly access these patient populations for pragmatic clinical trials, epidemiological research, and patient-centered research on rare diseases.

Electronic Measurement of a Clinical Quality Measure for Inpatient Hypoglycemic Events: A Multicenter Validation Study

BACKGROUND

Hypoglycemia related to antidiabetic drugs (ADDs) is important iatrogenic harm in hospitalized patients. Electronic identification of ADD-related hypoglycemia may be an efficient, reliable method to inform quality improvement.

OBJECTIVE

Develop electronic queries of electronic health records for facility-wide and unit-specific inpatient hypoglycemia event rates and validate query findings with manual chart review.

METHODS

Electronic queries were created to associate blood glucose (BG) values with ADD administration and inpatient location in 3 tertiary care hospitals with Patient-Centered Outcomes Research Network (PCORnet) databases. Queries were based on National Quality Forum criteria with hypoglycemia thresholds <40 and <54 mg/dL, and validated using a stratified random sample of 321 BG events. Sensitivity and specificity were calculated with manual chart review as the reference standard.

RESULTS

The sensitivity and specificity of queries for hypoglycemia events were 97.3% [95% confidence interval (CI), 90.5%-99.7%] and 100.0% (95% CI, 92.6%-100.0%), respectively for BG <40 mg/dL, and 97.7% (95% CI, 93.3%-99.5%) and 100.0% (95% CI, 95.3%-100.0%), respectively for <54 mg/dL. The sensitivity and specificity of the query for identifying ADD days were 91.8% (95% CI, 89.2%-94.0%) and 99.0% (95% CI, 97.5%-99.7%). Of 48 events missed by the queries, 37 (77.1%) were due to incomplete identification of insulin administered by infusion. Facility-wide hypoglycemia rates were 0.4%-0.8% (BG <40 mg/dL) and 1.9%-3.0% (BG <54 mg/dL); rates varied by patient care unit.

CONCLUSIONS

Electronic queries can accurately identify inpatient hypoglycemia. Implementation in non-PCORnet-participating facilities should be assessed, with particular attention to patient location and insulin infusions.

Development of comparable algorithms to measure primary care indicators using administrative health data across three Canadian provinces

INTRODUCTION

Performance measurement has been recognized as key to transforming primary care (PC). Yet, performance reporting in PC lags behind even though high-performing PC is foundational to an effective and efficient health care system.

OBJECTIVES

We used administrative data from three Canadian provinces, British Columbia, Ontario and Nova Scotia, to: 1) identify and develop a core set of PC performance indicators using administrative data and 2) examine their ability to capture PC performance.

METHODS

Administrative data used included Physician Billings, Discharge Abstract Database, the National Ambulatory Care and Reporting System database, Census and Vital Statistics. Indicators were compiled based on a literature review of PC indicators previously developed with administrative data available in Canada (n=158). We engaged in iterative discussions to assess data conformity, completeness, and plausibility of results in all jurisdictions. Challenges to creating comparable algorithms were examined through content analysis and research team discussions, which included clinicians, analysts, and health services researchers familiar with PC.

RESULTS

Our final list included 21 PC performance indicators pertaining to 1) technical care (n=4), 2) continuity of care (n=6), and 3) health services utilization (n=11). Establishing comparable algorithms across provinces was possible though time intensive. A major challenge was inconsistent data elements. Ease of data access, and a deep understanding of the data and practice context, was essential for selecting the most appropriate data elements.

CONCLUSIONS

This project is unique in creating algorithms to measure PC performance across provinces. It was essential to balance internal validity of the indicators within a province and external validity across provinces. The intuitive desire of having the exact same coding across provinces was infeasible due to lack of standardized PC data. Rather, a context-tailored definition was developed for each jurisdiction. This work serves as an example for developing comparable PC performance indicators across different provincial/territorial jurisdictions.

Big Data for Nutrition Research in Pediatric Oncology: Current State and Framework for Advancement

Recognition and treatment of malnutrition in pediatric oncology patients is crucial because it is associated with increased morbidity and mortality. Nutrition-relevant data collected from cancer clinical trials and nutrition-specific studies are insufficient to drive high-impact nutrition research without augmentation from additional data sources. To date, clinical big data resources are underused for nutrition research in pediatric oncology. Health-care big data can be broadly subclassified into three clinical data categories: administrative, electronic health record (including clinical data research networks and learning health systems), and mobile health. Along with -omics data, each has unique applications and limitations. We summarize the potential use of clinical big data to drive pediatric oncology nutrition research and identify key scientific gaps. A framework for advancement of big data utilization for pediatric oncology nutrition research is presented and focuses on transdisciplinary teams, data interoperability, validated cohort curation, data repurposing, and mobile health applications.

Computable Phenotype Implementation for a National, Multicenter Pragmatic Clinical Trial: Lessons Learned From ADAPTABLE

BACKGROUND

Many large-scale cardiovascular clinical trials are plagued with escalating costs and low enrollment. Implementing a computable phenotype, which is a set of executable algorithms, to identify a group of clinical characteristics derivable from electronic health records or administrative claims records, is essential to successful recruitment in large-scale pragmatic clinical trials. This methods paper provides an overview of the development and implementation of a computable phenotype in ADAPTABLE (Aspirin Dosing: a Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness)-a pragmatic, randomized, open-label clinical trial testing the optimal dose of aspirin for secondary prevention of atherosclerotic cardiovascular disease events.

METHODS AND RESULTS

A multidisciplinary team developed and tested the computable phenotype to identify adults ≥18 years of age with a history of atherosclerotic cardiovascular disease without safety concerns around using aspirin and meeting trial eligibility criteria. Using the computable phenotype, investigators identified over 650 000 potentially eligible patients from the 40 participating sites from Patient-Centered Outcomes Research Network-a network of Clinical Data Research Networks, Patient-Powered Research Networks, and Health Plan Research Networks. Leveraging diverse recruitment methods, sites enrolled 15 076 participants from April 2016 to June 2019. During the process of developing and implementing the ADAPTABLE computable phenotype, several key lessons were learned. The accuracy and utility of a computable phenotype are dependent on the quality of the source data, which can be variable even with a common data model. Local validation and modification were required based on site factors, such as recruitment strategies, data quality, and local coding patterns. Sustained collaboration among a diverse team of researchers is needed during computable phenotype development and implementation.

CONCLUSIONS

The ADAPTABLE computable phenotype served as an efficient method to recruit patients in a multisite pragmatic clinical trial. This process of development and implementation will be informative for future large-scale, pragmatic clinical trials. Registration: URL: https://www.clinicaltrials.gov; Unique identifier: NCT02697916.

Comparing the 5-Year Diabetes Outcomes of Sleeve Gastrectomy and Gastric Bypass: The National Patient-Centered Clinical Research Network (PCORNet) Bariatric Study

Question

How do type 2 diabetes (T2DM) outcomes compare across the 2 most common bariatric procedures?

Findings

In this cohort study of 9710 adults with T2DM who underwent bariatric surgery, most patients who had Roux-en-Y gastric bypass or sleeve gastrectomy experienced T2DM remission at some point over 5 years of follow-up. Patients who had Roux-en-Y gastric bypass showed slightly higher T2DM remission rates, better glycemic control, and fewer T2DM relapse events than patients who had sleeve gastrectomy.

Meaning

Understanding diabetes outcomes of different bariatric procedures will help surgeons and patients with diabetes make informed health care choices.

Importance

Bariatric surgery can lead to substantial improvements in type 2 diabetes (T2DM), but outcomes vary across procedures and populations. It is unclear which bariatric procedure has the most benefits for patients with T2DM.

Objective

To evaluate associations of bariatric surgery with T2DM outcomes.

Design, Setting, and Participants

This cohort study was conducted in 34 US health system sites in the National Patient-Centered Clinical Research Network Bariatric Study. Adult patients with T2DM who had bariatric surgery between January 1, 2005, and September 30, 2015, were included. Data analysis was conducted from April 2017 to August 2019.

Interventions

Roux-en-Y gastric bypass (RYGB) or sleeve gastrectomy (SG).

Main Outcome and Measures

Type 2 diabetes remission, T2DM relapse, percentage of total weight lost, and change in glycosylated hemoglobin (hemoglobin A_1c).

Results

A total of 9710 patients were included (median [interquartile range] follow-up time, 2.7 [2.9] years; 7051 female patients [72.6%]; mean [SD] age, 49.8 [10.5] years; mean [SD] BMI, 49.0 [8.4]; 6040 white patients [72.2%]). Weight loss was significantly greater with RYGB than SG at 1 year (mean difference, 6.3 [95% CI, 5.8-6.7] percentage points) and 5 years (mean difference, 8.1 [95% CI, 6.6-9.6] percentage points). The T2DM remission rate was approximately 10% higher in patients who had RYGB (hazard ratio, 1.10 [95% CI, 1.04-1.16]) than those who had SG. Estimated adjusted cumulative T2DM remission rates for patients who had RYGB and SG were 59.2% (95% CI, 57.7%-60.7%) and 55.9% (95% CI, 53.9%-57.9%), respectively, at 1 year and 86.1% (95% CI, 84.7%-87.3%) and 83.5% (95% CI, 81.6%-85.1%) at 5 years postsurgery. Among 6141 patients who experienced T2DM remission, the subsequent T2DM relapse rate was lower for those who had RYGB than those who had SG (hazard ratio, 0.75 [95% CI, 0.67-0.84]). Estimated relapse rates for those who had RYGB and SG were 8.4% (95% CI, 7.4%-9.3%) and 11.0% (95% CI, 9.6%-12.4%) at 1 year and 33.1% (95% CI, 29.6%-36.5%) and 41.6% (95% CI, 36.8%-46.1%) at 5 years after surgery. At 5 years, compared with baseline, hemoglobin A_1c was reduced 0.45 (95% CI, 0.27-0.63) percentage points more for patients who had RYGB vs patients who had SG.

Conclusions and Relevance

In this large multicenter study, patients who had RYGB had greater weight loss, a slightly higher T2DM remission rate, less T2DM relapse, and better long-term glycemic control compared with those who had SG. These findings can help inform patient-centered surgical decision-making.

Collaborative data familiarisation and quality assessment: Reflections from use of a national dataset to investigate palliative care for Indigenous Australians

BACKGROUND

Data quality is fundamental to the integrity of quantitative research. The role of external researchers in data quality assessment (DQA) remains ill-defined in the context of secondary use for research of large, centrally curated health datasets. In order to investigate equity of palliative care provided to Indigenous Australian patients, researchers accessed a now-historical version of a national palliative care dataset developed primarily for the purpose of continuous quality improvement.

OBJECTIVES

(i) To apply a generic DQA framework to the dataset and (ii) to report the process and results of this assessment and examine the consequences for conducting the research.

METHOD

The data were systematically examined for completeness, consistency and credibility. Data quality issues relevant to the Indigenous identifier and framing of research questions were of particular interest.

RESULTS

The dataset comprised 477,518 records of 144,951 patients (Indigenous N = 1515; missing Indigenous identifier N = 4998) collected from participating specialist palliative care services during a period (1 January 2010-30 June 2015) in which data-checking systems underwent substantial upgrades. Progressive improvement in completeness of data over the study period was evident. The data were error-free with respect to many credibility and consistency checks, with anomalies detected reported to data managers. As the proportion of missing values remained substantial for some clinical care variables, multiple imputation procedures were used in subsequent analyses.

CONCLUSION AND IMPLICATIONS

In secondary use of large curated datasets, DQA by external researchers may both influence proposed analytical methods and contribute to improvement of data curation processes through feedback to data managers.

Development of Evaluation System for Iranian Health Research Networks: Challenges and Lessons Learned

BACKGROUND

Health research networks (HRNs) are critical components of large-scale systems of production and validation of scientific evidence. As evaluation of research systems is a reliable process to measure efficiency and effectiveness of their activities, we aimed to report the processes of development of evaluation indicators' for Iranian health research networks and the results of conducted assessment.

METHODS

In 2017, for the first time, aim to develop the evaluation framework for national HRNs, following the qualitative approach to assess the quality of research we designed the peer review method as one of the most important tools. This qualitative method was conducted according to experts' views in specific fields. Key policy makers and stakeholders collaboratively developed a number of criteria for evaluation of research performance of Iranian HRNs. Following the review of conducted studies, benefitting from published guide line, these indicators were defined under 4 main axes of governance and leadership; infrastructures; research products and research impact.

RESULTS

Based on requirements of developed protocol for evaluation of HRNs in Iran, 18 HRNs completed the processes of evaluation. Results show a progressive need for more attention to precise planning of HRNs for achieving to goals. Another point to consider is the attention to documenting processes. The observational system for researches for detection of latest research priority was the most important issues that need to be more addressed by all of networks.

CONCLUSION

Research evaluation of Iranian HRNs more over creating of constructive positive competition provide an overview of the shortcomings and research challenges could be used for better planning and promotion of the health research system.

Incrementally Transforming Electronic Medical Records into the Observational Medical Outcomes Partnership Common Data Model: A Multidimensional Quality Assurance Approach

BACKGROUND

The development and adoption of health care common data models (CDMs) has addressed some of the logistical challenges of performing research on data generated from disparate health care systems by standardizing data representations and leveraging standardized terminology to express clinical information consistently. However, transforming a data system into a CDM is not a trivial task, and maintaining an operational, enterprise capable CDM that is incrementally updated within a data warehouse is challenging.

OBJECTIVES

To develop a quality assurance (QA) process and code base to accompany our incremental transformation of the Department of Veterans Affairs Corporate Data Warehouse health care database into the Observational Medical Outcomes Partnership (OMOP) CDM to prevent incremental load errors.

METHODS

We designed and implemented a multistage QA) approach centered on completeness, value conformance, and relational conformance data-quality elements. For each element we describe key incremental load challenges, our extract, transform, and load (ETL) solution of data to overcome those challenges, and potential impacts of incremental load failure.

RESULTS

Completeness and value conformance data-quality elements are most affected by incremental changes to the CDW, while updates to source identifiers impact relational conformance. ETL failures surrounding these elements lead to incomplete and inaccurate capture of clinical concepts as well as data fragmentation across patients, providers, and locations.

CONCLUSION

Development of robust QA processes supporting accurate transformation of OMOP and other CDMs from source data is still in evolution, and opportunities exist to extend the existing QA framework and tools used for incremental ETL QA processes.

Applying a Commercialization-Readiness Framework to Optimize Value for Achieving Sustainability of an Electronic Health Data Research Network and Its Data Capabilities: The SAFTINet Experience

CONTEXT

Sustaining electronic health data networks and maximizing return on federal investment in their development is essential for achieving national data insight goals for transforming health care. However, crossing the business model chasm from grant funding to self-sustaining viability is challenging.

CASE DESCRIPTION

This paper presents lessons learned in seeking the sustainability of the Scalable Architecture for Federated Translational Inquiries Network (SAFTINet), and electronic health data network involving over 50 primary care practices in three states. SAFTINet was developed with funding from the Agency for Healthcare Research and Quality to create a multi-state network for comparative effectiveness research (CER) involving safety-net patients.

METHODS

Three analyses were performed: (1) a product gap analysis of alternative data sources; (2) a Strengths-Weaknesses-Opportunities-Threat (SWOT) analysis of SAFTINet in the context of competing alternatives; and (3) a customer discovery process involving approximately 150 SAFTINet stakeholders to identify SAFTINet's sustaining value proposition for health services researchers, clinical data partners, and policy makers.

FINDINGS

The results of this business model analysis informed SAFTINet's sustainability strategy. The fundamental high-level product needs were similar between the three primary customer segments: credible data, efficient and easy to use, and relevance to their daily work or 'jobs to be done'. However, how these benefits needed to be minimally demonstrated varied by customer such that different supporting evidence was required.

MAJOR THEMES

The SAFTINet experience illustrates that commercialization-readiness and business model methods can be used to identify multi-sided value propositions for sustaining electronic health data networks and their data capabilities as drivers of health care transformation.

Design and Refinement of a Data Quality Assessment Workflow for a Large Pediatric Research Network

Background:

Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research. A critical operation toward building a CDRN is conducting continual evaluations to optimize data quality. The key challenges include determining the assessment coverage on big datasets, handling data variability over time, and facilitating communication with data teams. This study presents the evolution of a systematic workflow for data quality assessment in CDRNs.

Implementation:

Using a specific CDRN as use case, the workflow was iteratively developed and packaged into a toolkit. The resultant toolkit comprises 685 data quality checks to identify any data quality issues, procedures to reconciliate with a history of known issues, and a contemporary GitHub-based reporting mechanism for organized tracking.

Results:

During the first two years of network development, the toolkit assisted in discovering over 800 data characteristics and resolving over 1400 programming errors. Longitudinal analysis indicated that the variability in time to resolution (15day mean, 24day IQR) is due to the underlying cause of the issue, perceived importance of the domain, and the complexity of assessment.

Conclusions:

In the absence of a formalized data quality framework, CDRNs continue to face challenges in data management and query fulfillment. The proposed data quality toolkit was empirically validated on a particular network, and is publicly available for other networks. While the toolkit is user-friendly and effective, the usage statistics indicated that the data quality process is very time-intensive and sufficient resources should be dedicated for investigating problems and optimizing data for research.

A Data Element-Function Conceptual Model for Data Quality Checks

Introduction:

In aggregate, existing data quality (DQ) checks are currently represented in heterogeneous formats, making it difficult to compare, categorize, and index checks. This study contributes a data element-function conceptual model to facilitate the categorization and indexing of DQ checks and explores the feasibility of leveraging natural language processing (NLP) for scalable acquisition of knowledge of common data elements and functions from DQ checks narratives.

Methods:

The model defines a “data element”, the primary focus of the check, and a “function”, the qualitative or quantitative measure over a data element. We applied NLP techniques to extract both from 172 checks for Observational Health Data Sciences and Informatics (OHDSI) and 3,434 checks for Kaiser Permanente’s Center for Effectiveness and Safety Research (CESR).

Results:

The model was able to classify all checks. A total of 751 unique data elements and 24 unique functions were extracted. The top five frequent data element-function pairings for OHDSI were Person-Count (55 checks), Insurance-Distribution (17), Medication-Count (16), Condition-Count (14), and Observations-Count (13); for CESR, they were Medication-Variable Type (175), Medication-Missing (172), Medication-Existence (152), Medication-Count (127), and Socioeconomic Factors-Variable Type (114).

Conclusions:

This study shows the efficacy of the data element-function conceptual model for classifying DQ checks, demonstrates early promise of NLP-assisted knowledge acquisition, and reveals the great heterogeneity in the focus in DQ checks, confirming variation in intrinsic checks and use-case specific “fitness-for-use” checks.

Comparing Prescribing and Dispensing Data of the PCORnet Common Data Model Within PCORnet Antibiotics and Childhood Growth Study

Researchers often use prescribing data from electronic health records (EHR) or dispensing data from medication or medical claims to determine medication utilization. However, neither source has complete information on medication use. We compared antibiotic prescribing and dispensing records for 200,395 patients in the National Patient-Centered Clinical Research Network (PCORnet) Antibiotics and Childhood Growth Study. We stratified analyses by delivery system type [closed integrated (cIDS) and non-cIDS]; 90.5 percent and 39.4 percent of prescribing records had matching dispensing records, and 92.7 percent and 64.0 percent of dispensing records had matching prescribing records at cIDS and non-cIDS, respectively. Most of the dispensings without a matching prescription did not have same-day encounters in the EHR, suggesting they were medications given outside the institution providing data, such as those from urgent care or retail clinics. The sensitivity of prescriptions in the EHR, using dispensings as a gold standard, was 99.1 percent and 89.9 percent for cIDS and non-cIDS, respectively. Only 0.7 percent and 6.1 percent of patients at cIDS and non-cIDS, respectively, were classified as false-negative, i.e. entirely unexposed to antibiotics when they in fact had dispensings. These patients were more likely to have a complex chronic condition or asthma. Overall, prescription records worked well to identify exposure to antibiotics. EHR data, such as the data available in PCORnet, is a unique and vital resource for clinical research. Closing data gaps by understanding why prescriptions may not be captured can improve this type of data, making it more robust for observational research.

Collaborating on Data, Science, and Infrastructure: The 20-Year Journey of the Cancer Research Network

The Cancer Research Network (CRN) is a consortium of 12 research groups, each affiliated with a nonprofit integrated health care delivery system, that was first funded in 1998. The overall goal of the CRN is to support and facilitate collaborative cancer research within its component delivery systems. This paper describes the CRN's 20-year experience and evolution. The network combined its members' scientific capabilities and data resources to create an infrastructure that has ultimately supported over 275 projects. Insights about the strengths and limitations of electronic health data for research, approaches to optimizing multidisciplinary collaboration, and the role of a health services research infrastructure to complement traditional clinical trials and large observational datasets are described, along with recommendations for other research consortia.

PCORnet Antibiotics and Childhood Growth Study: Process for Cohort Creation and Cohort Description

OBJECTIVES

The National Patient-Centered Clinical Research Network (PCORnet) supports observational and clinical research using health care data. The PCORnet Antibiotics and Childhood Growth Study is one of PCORnet's inaugural observational studies. We sought to describe the processes used to integrate and analyze data from children across 35 participating institutions, the cohort characteristics, and prevalence of antibiotic use.

METHODS

We included children in the cohort if they had at least one same-day height and weight measured in each of 3 age periods: 1) before 12 months, 2) 12 to 30 months, and 3) after 24 months. We distributed statistical queries that each institution ran on its local version of the PCORnet Common Data Model, with aggregate data returned for analysis. We defined overweight or obesity as age- and sex-specific body mass index ≥85th percentile, obesity ≥95th percentile, and severe obesity ≥120% of the 95th percentile.

RESULTS

A total of 681,739 children met the cohort inclusion criteria, and participants were racially/ethnically diverse (24.9% black, 17.5% Hispanic). Before 24 months of age, 55.2% of children received at least one antibiotic prescription; 21.3% received a single antibiotic prescription; 14.3% received 4 or more; and 33.3% received a broad-spectrum antibiotic. Overweight and obesity prevalence was 27.6% at age 4 to <6 years (n = 362,044) and 36.2% at 9 to <11 years (n = 58,344).

CONCLUSIONS

The PCORnet Antibiotics and Childhood Growth Study is a large national longitudinal observational study in a diverse population that will examine the relationship between early antibiotic use and subsequent growth patterns in children.