Synergies between centralized and federated approaches to data quality: a report from the national COVID cohort collaborative

Value sets and the problem of redundancy in value set repositories

OBJECTIVE

Crafting high-quality value sets is time-consuming and requires a range of clinical, terminological, and informatics expertise. Despite widespread agreement on the importance of reusing value sets, value set repositories suffer from clutter and redundancy, greatly complicating efforts at reuse. When users encounter multiple value sets with the same name or ostensibly representing the same clinical condition, it can be difficult to choose amongst them or determine if any differences among them are due to error or intentional decision.

METHODS

This paper offers a view of value set development and reuse based on a field study of researchers and informaticists. The results emerge from an analysis of relevant literature, reflective practice, and the field research data.

RESULTS

Qualitative analysis of our study data, the relevant literature, and our own professional experience led us to three dichotomous concepts that frame an understanding of diverse practices and perspectives surrounding value set development: Permissible values versus analytic value sets;Prescriptive versus descriptive approaches to controlled medical vocabulary use; andSemantic and empirical types of value set development and evaluation practices and the data they rely on.This three-fold framework opens up the redundancy problem, explaining why multiple value sets may or may not be needed and advancing academic understanding of value set development.

CONCLUSION

In order for value set repositories to become more rather than less useful over time, software must channel user efforts into either improving existing value sets or making new ones only when absolutely necessary. This would require major, innovative changes to value set repository platforms. We believe the most direct path to giving value set developers the ability to leverage prior work is by encouraging them to compare existing value sets using advanced interfaces like VS-Hub, and by collecting and using metadata about code inclusion and exclusion decisions during the authoring process.

Critical Data for Critical Care: A Primer on Leveraging Electronic Health Record Data for Research From Society of Critical Care Medicine's Panel on Data Sharing and Harmonization

A growing body of critical care research draws on real-world data from electronic health records (EHRs). The bedside clinician has myriad data sources to aid in clinical decision-making, but the lack of data sharing and harmonization standards leaves much of this data out of reach for multi-institution critical care research. The Society of Critical Care Medicine (SCCM) Discovery Data Science Campaign convened a panel of critical care and data science experts to explore and document unique advantages and opportunities for leveraging EHR data in critical care research. This article reviews and illustrates six organizing topics (data domains and common data elements; data harmonization; data quality; data interoperability and digital infrastructure; data access, sharing, and governance; and ethics and equity) as a data science primer for critical care researchers, laying a foundation for future publications from the SCCM Discovery Data Harmonization and Sharing Guiding Principles Panel.

Finding Long-COVID: temporal topic modeling of electronic health records from the N3C and RECOVER programs

Post-Acute Sequelae of SARS-CoV-2 infection (PASC), also known as Long-COVID, encompasses a variety of complex and varied outcomes following COVID-19 infection that are still poorly understood. We clustered over 600 million condition diagnoses from 14 million patients available through the National COVID Cohort Collaborative (N3C), generating hundreds of highly detailed clinical phenotypes. Assessing patient clinical trajectories using these clusters allowed us to identify individual conditions and phenotypes strongly increased after acute infection. We found many conditions increased in COVID-19 patients compared to controls, and using a novel method to associate patients with clusters over time, we additionally found phenotypes specific to patient sex, age, wave of infection, and PASC diagnosis status. While many of these results reflect known PASC symptoms, the resolution provided by this unprecedented data scale suggests avenues for improved diagnostics and mechanistic understanding of this multifaceted disease.

Bridging Data Models in Health Care With a Novel Intermediate Query Format for Feasibility Queries: Mixed Methods Study

Background

To advance research with clinical data, it is essential to make access to the available data as fast and easy as possible for researchers, which is especially challenging for data from different source systems within and across institutions. Over the years, many research repositories and data standards have been created. One of these is the Fast Healthcare Interoperability Resources (FHIR) standard, used by the German Medical Informatics Initiative (MII) to harmonize and standardize data across university hospitals in Germany. One of the first steps to make these data available is to allow researchers to create feasibility queries to determine the data availability for a specific research question. Given the heterogeneity of different query languages to access different data across and even within standards such as FHIR (eg, CQL and FHIR Search), creating an intermediate query syntax for feasibility queries reduces the complexity of query translation and improves interoperability across different research repositories and query languages.

Objective

This study describes the creation and implementation of an intermediate query syntax for feasibility queries and how it integrates into the federated German health research portal (Forschungsdatenportal Gesundheit) and the MII.

Methods

We analyzed the requirements for feasibility queries and the feasibility tools that are currently available in research repositories. Based on this analysis, we developed an intermediate query syntax that can be easily translated into different research repository-specific query languages.

Results

The resulting Clinical Cohort Definition Language (CCDL) for feasibility queries combines inclusion criteria in a conjunctive normal form and exclusion criteria in a disjunctive normal form, allowing for additional filters like time or numerical restrictions. The inclusion and exclusion results are combined via an expression to specify feasibility queries. We defined a JSON schema for the CCDL, generated an ontology, and demonstrated the use and translatability of the CCDL across multiple studies and real-world use cases.

Conclusions

We developed and evaluated a structured query syntax for feasibility queries and demonstrated its use in a real-world example as part of a research platform across 39 German university hospitals.

A national unmet needs assessment for CTSA-affiliated electronic health record data networks: A customer discovery approach

Introduction

The expansion of electronic health record (EHR) data networks over the last two decades has significantly improved the accessibility and processes around data sharing. However, there lies a gap in meeting the needs of Clinical and Translational Science Award (CTSA) hubs, particularly related to real-world data (RWD) and real-world evidence (RWE).

Methods

We adopted a mixed-methods approach to construct a comprehensive needs assessment that included: (1) A Landscape Context analysis to understand the competitive environment; and (2) Customer Discovery to identify stakeholders and the value proposition related to EHR data networks. Methods included surveys, interviews, and a focus group.

Results

Thirty-two CTSA institutions contributed data for analysis. Fifty-four interviews and one focus group were conducted. The synthesis of our findings pivots around five emergent themes: (1) CTSA segmentation needs vary according to resources; (2) Team science is key for success; (3) Quality of data generates trust in the network; (4) Capacity building is defined differently by researcher career stage and CTSA existing resources; and (5) Researchers' unmet needs.

Conclusions

Based on the results, EHR data networks like ENACT that would like to meet the expectations of academic research centers within the CTSA consortium need to consider filling the gaps identified by our study: foster team science, improve workforce capacity, achieve data governance trust and efficiency of operation, and aid Learning Health Systems with validating, applying, and scaling the evidence to support quality improvement and high-value care. These findings align with the NIH NCATS Strategic Plan for Data Science.

Evolution of a Graph Model for the OMOP Common Data Model

OBJECTIVE

 Graph databases for electronic health record (EHR) data have become a useful tool for clinical research in recent years, but there is a lack of published methods to transform relational databases to a graph database schema. We developed a graph model for the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) that can be reused across research institutions.

METHODS

 We created and evaluated four models, representing two different strategies, for converting the standardized clinical and vocabulary tables of OMOP into a property graph model within the Neo4j graph database. Taking the Successful Clinical Response in Pneumonia Therapy (SCRIPT) and Collaborative Resource for Intensive care Translational science, Informatics, Comprehensive Analytics, and Learning (CRITICAL) cohorts as test datasets with different sizes, we compared two of the resulting graph models with respect to database performance including database building time, query complexity, and runtime for both cohorts.

RESULTS

 Utilizing a graph schema that was optimized for storing critical information as topology rather than attributes resulted in a significant improvement in both data creation and querying. The graph database for our larger cohort, CRITICAL, can be built within 1 hour for 134,145 patients, with a total of 749,011,396 nodes and 1,703,560,910 edges.

DISCUSSION

 To our knowledge, this is the first generalized solution to convert the OMOP CDM to a graph-optimized schema. Despite being developed for studies at a single institution, the modeling method can be applied to other OMOP CDM v5.x databases. Our evaluation with the SCRIPT and CRITICAL cohorts and comparison between the current and previous versions show advantages in code simplicity, database building, and query speed.

CONCLUSION

 We developed a method for converting OMOP CDM databases into graph databases. Our experiments revealed that the final model outperformed the initial relational-to-graph transformation in both code simplicity and query efficiency, particularly for complex queries.

Data Quality-Driven Improvement in Health Care: Systematic Literature Review

BACKGROUND

The promise of real-world evidence and the learning health care system primarily depends on access to high-quality data. Despite widespread awareness of the prevalence and potential impacts of poor data quality (DQ), best practices for its assessment and improvement are unknown.

OBJECTIVE

This review aims to investigate how existing research studies define, assess, and improve the quality of structured real-world health care data.

METHODS

A systematic literature search of studies in the English language was implemented in the Embase and PubMed databases to select studies that specifically aimed to measure and improve the quality of structured real-world data within any clinical setting. The time frame for the analysis was from January 1945 to June 2023. We standardized DQ concepts according to the Data Management Association (DAMA) DQ framework to enable comparison between studies. After screening and filtering by 2 independent authors, we identified 39 relevant articles reporting DQ improvement initiatives.

RESULTS

The studies were characterized by considerable heterogeneity in settings and approaches to DQ assessment and improvement. Affiliated institutions were from 18 different countries and 18 different health domains. DQ assessment methods were largely manual and targeted completeness and 1 other DQ dimension. Use of DQ frameworks was limited to the Weiskopf and Weng (3/6, 50%) or Kahn harmonized model (3/6, 50%). Use of standardized methodologies to design and implement quality improvement was lacking, but mainly included plan-do-study-act (PDSA) or define-measure-analyze-improve-control (DMAIC) cycles. Most studies reported DQ improvements using multiple interventions, which included either DQ reporting and personalized feedback (24/39, 61%), IT-related solutions (21/39, 54%), training (17/39, 44%), improvements in workflows (5/39, 13%), or data cleaning (3/39, 8%). Most studies reported improvements in DQ through a combination of these interventions. Statistical methods were used to determine significance of treatment effect (22/39, 56% times), but only 1 study implemented a randomized controlled study design. Variability in study designs, approaches to delivering interventions, and reporting DQ changes hindered a robust meta-analysis of treatment effects.

CONCLUSIONS

There is an urgent need for standardized guidelines in DQ improvement research to enable comparison and effective synthesis of lessons learned. Frameworks such as PDSA learning cycles and the DAMA DQ framework can facilitate this unmet need. In addition, DQ improvement studies can also benefit from prioritizing root cause analysis of DQ issues to ensure the most appropriate intervention is implemented, thereby ensuring long-term, sustainable improvement. Despite the rise in DQ improvement studies in the last decade, significant heterogeneity in methodologies and reporting remains a challenge. Adopting standardized frameworks for DQ assessment, analysis, and improvement can enhance the effectiveness, comparability, and generalizability of DQ improvement initiatives.

Long-Term Mortality Following SARS-CoV-2 Infection in Rural Versus Urban Dwellers With Autoimmune or Inflammatory Rheumatic Disease: A Retrospective Cohort Analysis From the National COVID Cohort Collaborative

OBJECTIVE

Autoimmune or inflammatory rheumatic diseases (AIRDs) increase the risk for poor COVID-19 outcomes. Although rurality is associated with higher post-COVID-19 mortality in the general population, whether rurality elevates this risk among people with AIRD is unknown. We assessed associations between rurality and post-COVID-19 all-cause mortality, up to two years post infection, among people with AIRD using a large nationally sampled US cohort.

METHODS

This retrospective study used the National COVID Cohort Collaborative, a medical records repository containing COVID-19 patient data. We included adults with two or more AIRD diagnostic codes and a COVID-19 diagnosis documented between April 2020 and March 2023. Rural residency was categorized using patient residential zip codes. We adjusted for AIRD medications and glucocorticoid prescription, age, sex, race and ethnicity, tobacco or substance use, comorbid burden, and SARS-CoV-2 variant-dominant periods. Multivariable Cox proportional hazards with inverse probability treatment weighting assessed associations between rurality and two-year all-cause mortality.

RESULTS

Among the 86,467 SARS-CoV-2-infected persons with AIRD, we observed a higher risk for two-year post-COVID-19 mortality in rural versus urban dwellers. Rural-residing persons with AIRD had higher two-year all-cause mortality risk (adjusted hazard ratio 1.24, 95% confidence interval 1.19-1.29). Glucocorticoid, immunosuppressive, and rituximab prescriptions were associated with a higher risk for two-year post-COVID-19 mortality, whereas risk with nonbiologic or biologic disease-modifying antirheumatic drugs was lower.

CONCLUSION

Rural residence in people with AIRD was independently associated with higher two-year post-COVID-19 mortality in a large US cohort after adjusting for background risk factors. Policymakers and health care providers should consider these findings when designing interventions to improve outcomes in people with AIRD following SARS-CoV-2 infection, especially among high-risk rural residents.

Effect of Paxlovid Treatment During Acute COVID-19 on Long COVID Onset: An EHR-Based Target Trial Emulation from the N3C and RECOVER Consortia

Preventing and treating post-acute sequelae of SARS-CoV-2 infection (PASC), commonly known as Long COVID, has become a public health priority. In this study, we examined whether treatment with Paxlovid in the acute phase of COVID-19 helps prevent the onset of PASC. We used electronic health records from the National Covid Cohort Collaborative (N3C) to define a cohort of 426,352 patients who had COVID-19 since April 1, 2022, and were eligible for Paxlovid treatment due to risk for progression to severe COVID-19. We used the target trial emulation (TTE) framework to estimate the effect of Paxlovid treatment on PASC incidence. We estimated overall PASC incidence using a computable phenotype. We also measured the onset of novel cognitive, fatigue, and respiratory symptoms in the post-acute period. Paxlovid treatment did not have a significant effect on overall PASC incidence (relative risk [RR] = 0.98, 95% confidence interval [CI] 0.95-1.01). However, it had a protective effect on cognitive (RR = 0.90, 95% CI 0.84-0.96) and fatigue (RR = 0.95, 95% CI 0.91-0.98) symptom clusters, which suggests that the etiology of these symptoms may be more closely related to viral load than that of respiratory symptoms.

Understanding enterprise data warehouses to support clinical and translational research: impact, sustainability, demand management, and accessibility

OBJECTIVES

Healthcare organizations, including Clinical and Translational Science Awards (CTSA) hubs funded by the National Institutes of Health, seek to enable secondary use of electronic health record (EHR) data through an enterprise data warehouse for research (EDW4R), but optimal approaches are unknown. In this qualitative study, our goal was to understand EDW4R impact, sustainability, demand management, and accessibility.

MATERIALS AND METHODS

We engaged a convenience sample of informatics leaders from CTSA hubs (n = 21) for semi-structured interviews and completed a directed content analysis of interview transcripts.

RESULTS

EDW4R have created institutional capacity for single- and multi-center studies, democratized access to EHR data for investigators from multiple disciplines, and enabled the learning health system. Bibliometrics have been challenging due to investigator non-compliance, but one hub's requirement to link all study protocols with funding records enabled quantifying an EDW4R's multi-million dollar impact. Sustainability of EDW4R has relied on multiple funding sources with a general shift away from the CTSA grant toward institutional and industry support. To address EDW4R demand, institutions have expanded staff, used different governance approaches, and provided investigator self-service tools. EDW4R accessibility can benefit from improved tools incorporating user-centered design, increased data literacy among scientists, expansion of informaticians in the workforce, and growth of team science.

DISCUSSION

As investigator demand for EDW4R has increased, approaches to tracking impact, ensuring sustainability, and improving accessibility of EDW4R resources have varied.

CONCLUSION

This study adds to understanding of how informatics leaders seek to support investigators using EDW4R across the CTSA consortium and potentially elsewhere.

Finding Long-COVID: Temporal Topic Modeling of Electronic Health Records from the N3C and RECOVER Programs

Post-Acute Sequelae of SARS-CoV-2 infection (PASC), also known as Long-COVID, encompasses a variety of complex and varied outcomes following COVID-19 infection that are still poorly understood. We clustered over 600 million condition diagnoses from 14 million patients available through the National COVID Cohort Collaborative (N3C), generating hundreds of highly detailed clinical phenotypes. Assessing patient clinical trajectories using these clusters allowed us to identify individual conditions and phenotypes strongly increased after acute infection. We found many conditions increased in COVID-19 patients compared to controls, and using a novel method to associate patients with clusters over time, we additionally found phenotypes specific to patient sex, age, wave of infection, and PASC diagnosis status. While many of these results reflect known PASC symptoms, the resolution provided by this unprecedented data scale suggests avenues for improved diagnostics and mechanistic understanding of this multifaceted disease.

Association of post-COVID phenotypic manifestations with new-onset psychiatric disease

Acute COVID-19 infection can be followed by diverse clinical manifestations referred to as Post Acute Sequelae of SARS-CoV2 Infection (PASC). Studies have shown an increased risk of being diagnosed with new-onset psychiatric disease following a diagnosis of acute COVID-19. However, it was unclear whether non-psychiatric PASC-associated manifestations (PASC-AMs) are associated with an increased risk of new-onset psychiatric disease following COVID-19. A retrospective electronic health record (EHR) cohort study of 2,391,006 individuals with acute COVID-19 was performed to evaluate whether non-psychiatric PASC-AMs are associated with new-onset psychiatric disease. Data were obtained from the National COVID Cohort Collaborative (N3C), which has EHR data from 76 clinical organizations. EHR codes were mapped to 151 non-psychiatric PASC-AMs recorded 28-120 days following SARS-CoV-2 diagnosis and before diagnosis of new-onset psychiatric disease. Association of newly diagnosed psychiatric disease with age, sex, race, pre-existing comorbidities, and PASC-AMs in seven categories was assessed by logistic regression. There were significant associations between a diagnosis of any psychiatric disease and five categories of PASC-AMs with odds ratios highest for neurological, cardiovascular, and constitutional PASC-AMs with odds ratios of 1.31, 1.29, and 1.23 respectively. Secondary analysis revealed that the proportions of 50 individual clinical features significantly differed between patients diagnosed with different psychiatric diseases. Our study provides evidence for association between non-psychiatric PASC-AMs and the incidence of newly diagnosed psychiatric disease. Significant associations were found for features related to multiple organ systems. This information could prove useful in understanding risk stratification for new-onset psychiatric disease following COVID-19. Prospective studies are needed to corroborate these findings.

Systematic data quality assessment of electronic health record data to evaluate study-specific fitness: Report from the PRESERVE research study

Study-specific data quality testing is an essential part of minimizing analytic errors, particularly for studies making secondary use of clinical data. We applied a systematic and reproducible approach for study-specific data quality testing to the analysis plan for PRESERVE, a 15-site, EHR-based observational study of chronic kidney disease in children. This approach integrated widely adopted data quality concepts with healthcare-specific evaluation methods. We implemented two rounds of data quality assessment. The first produced high-level evaluation using aggregate results from a distributed query, focused on cohort identification and main analytic requirements. The second focused on extended testing of row-level data centralized for analysis. We systematized reporting and cataloguing of data quality issues, providing institutional teams with prioritized issues for resolution. We tracked improvements and documented anomalous data for consideration during analyses. The checks we developed identified 115 and 157 data quality issues in the two rounds, involving completeness, data model conformance, cross-variable concordance, consistency, and plausibility, extending traditional data quality approaches to address more complex stratification and temporal patterns. Resolution efforts focused on higher priority issues, given finite study resources. In many cases, institutional teams were able to correct data extraction errors or obtain additional data, avoiding exclusion of 2 institutions entirely and resolving 123 other gaps. Other results identified complexities in measures of kidney function, bearing on the study's outcome definition. Where limitations such as these are intrinsic to clinical data, the study team must account for them in conducting analyses. This study rigorously evaluated fitness of data for intended use. The framework is reusable and built on a strong theoretical underpinning. Significant data quality issues that would have otherwise delayed analyses or made data unusable were addressed. This study highlights the need for teams combining subject-matter and informatics expertise to address data quality when working with real world data.

Mobilizing data during a crisis: Building rapid evidence pipelines using multi-institutional real world data

The COVID-19 pandemic generated tremendous interest in using real world data (RWD). Many consortia across the public and private sectors formed in 2020 with the goal of rapidly producing high-quality evidence from RWD to guide medical decision-making, public health priorities, and more. Experiences were gathered from five large consortia on rapid multi-institutional evidence generation during the COVID-19 pandemic. Insights have been compiled across five dimensions: consortium composition, governance structure and alignment of priorities, data sharing, data analysis, and evidence dissemination. The purpose of this piece is to offer guidance on building large-scale multi-institutional RWD analysis pipelines for future public health issues. The composition of each consortium was largely influenced by existing collaborations. A central set of priorities for evidence generation guided each consortium, however different approaches to governance emerged. Challenges surrounding limited access to clinical data due to various contributors were overcome in unique ways. While all consortia used different methods to construct and analyze patient cohorts ranging from centralized to federated approaches, all proved effective for generating meaningful real-world evidence. Actionable recommendations for clinical practice and public health agencies were made from translating insights from consortium analyses. Each consortium was successful in rapidly answering questions about COVID-19 diagnosis and treatment despite all taking slightly different approaches to data sharing and analysis. Leveraging RWD, leveraged in a manner that applies scientific rigor and transparency, can complement higher-level evidence and serve as an important adjunct to clinical trials to quickly guide policy and critical care, especially for a pandemic response.

Prior COVID-19 vaccination and reduced risk of cerebrovascular diseases among COVID-19 survivors

The effects of COVID-19 vaccination on short-term and long-term cerebrovascular risks among COVID-19 survivors remained unknown. We conducted a national multi-center retrospective cohort study with 151 597 vaccinated and 151 597 unvaccinated COVID-19 patients using the TriNetX database, from January 1, 2020 to December 31, 2023. Patients baseline characteristics were balanced with propensity score matching (PSM). The outcomes were incident cerebrovascular diseases occurred between 1st and 30th days (short-term) after COVID-19 diagnosis. Nine subgroup analyses were conducted to explore potential effect modifications. We performed six sensitivity analyses, including evaluation of outcomes between 1st to 180th days, accounting for competing risk, and incorporating different variant timeline to test the robustness of our results. Kaplan-Meier curves and Log-Rank tests were performed to evaluate survival difference. Cox proportional hazards regressions were adopted to estimate the PSM-adjusted hazard ratios (HR). The overall short-term cerebrovascular risks were lower in the vaccinated group compared to the unvaccinated group (HR: 0.66, 95% CI: 0.56-0.77), specifically cerebral infarction (HR: 0.62, 95% CI: 0.48-0.79), occlusion and stenosis of precerebral arteries (HR: 0.74, 95% CI: 0.53-0.98), other cerebrovascular diseases (HR: 0.57, 95% CI: 0.42-0.77), and sequelae of cerebrovascular disease (HR: 0.39, 95% CI:0.23-0.68). Similarly, the overall cerebrovascular risks were lower in those vaccinated among most subgroups. The long-term outcomes, though slightly attenuated, were consistent (HR: 0.80, 95% CI: 0.73-0.87). Full 2-dose vaccination was associated with a further reduced risk of cerebrovascular diseases (HR: 0.63, 95% CI: 0.50-0.80) compared to unvaccinated patients. Unvaccinated COVID-19 survivors have significantly higher cerebrovascular risks than their vaccinated counterparts. Thus, clinicians are recommended to monitor this population closely for stroke events during postinfection follow-up.

Chiropractic spinal manipulation and likelihood of tramadol prescription in adults with radicular low back pain: a retrospective cohort study using US data

OBJECTIVES

Patients receiving chiropractic spinal manipulation (CSM) for low back pain (LBP) are less likely to receive any opioid prescription for subsequent pain management. However, the likelihood of specifically being prescribed tramadol, a less potent opioid, has not been explored. We hypothesised that adults receiving CSM for newly diagnosed radicular LBP would be less likely to receive a tramadol prescription over 1-year follow-up, compared with those receiving usual medical care.

DESIGN

Retrospective cohort study.

SETTING

US medical records-based dataset including >115 million patients attending academic health centres (TriNetX, Inc), queried 9 November 2023.

PARTICIPANTS

Opioid-naive adults aged 18-50 with a new diagnosis of radicular LBP were included. Patients with serious pathology and tramadol use contraindications were excluded. Variables associated with tramadol prescription were controlled via propensity matching.

INTERVENTIONS

Patients were divided into two cohorts dependent on treatment received on the index date of radicular LBP diagnosis (CSM or usual medical care).

PRIMARY AND SECONDARY OUTCOME MEASURES

Risk ratio (RR) for tramadol prescription (primary); markers of usual medical care utilisation (secondary).

RESULTS

After propensity matching, there were 1171 patients per cohort (mean age 35 years). Tramadol prescription was significantly lower in the CSM cohort compared with the usual medical care cohort, with an RR (95% CI) of 0.32 (0.18 to 0.57; p<0.0001). A cumulative incidence graph demonstrated that the reduced incidence of tramadol prescription in the CSM cohort relative to the usual medical care cohort was maintained throughout 1-year follow-up. Utilisation of NSAIDs, physical therapy evaluation and lumbar imaging was similar between cohorts.

CONCLUSIONS

This study found that US adults initially receiving CSM for radicular LBP had a reduced likelihood of receiving a tramadol prescription over 1-year follow-up. These findings should be corroborated by a prospective study to minimise residual confounding.

The Epidemiology of Posttraumatic Osteoarthritis of the Knee in the United States: An Analysis of 948,853 Patients From 2000 to 2022

INTRODUCTION

Posttraumatic osteoarthritis of the knee (PTOAK) is a known sequela of bony and soft-tissue articular knee injuries, although its historically reported prevalence is highly variable with no recent population-based studies.

METHODS

The TriNetX/US Collaborative Network database was queried to identify adult patients diagnosed with a history of knee trauma using ICD-10-CM coding. Primary outcomes measured were yearly incidence proportion (IP), incidence rate (IR), and prevalence of knee osteoarthritis in the United States from 2000 to 2022. Chi square analyses were conducted to compare outcomes across categorical data. Regression modeling was performed to project PTOAK epidemiology to 2030. Statistical significance was held at P < 0.05 for all analyses.

RESULTS

Nine hundred forty-eight thousand eight hundred fifty-three patients meeting criteria were identified. As of 2022, the IP of PTOAK was 5.93%, IR was 2.26 × 10 -4 cases/person-day, and prevalence was 21.1%. By strata in 2022, posttraumatic knee OA is most prevalent among the 54 to 59-year-old age group (50.9%), 60 to 64-year-old age group (50.3%), 50 to 54-year-old age group (49.7%), female patients (24.2%), and White patients (23.1%). Regression analyses revealed that the IP, IR, and prevalence of PTOAK have increased exponentially from 2000 to 2022. By 2030, the model predicts that the IP will further increase to 10.7% (95% PI = 9.79% to 11.7%), IR will be 3.79 × 10 -4 cases/person-day (95% PI = 3.28 × 10 -4 to 4.29 × 10 -4 ), and prevalence of PTOAK in the United States will be 40.6% (95% PI = 39.1% to 42.0%).

DISCUSSION

These findings echo earlier, smaller scale studies but reveal an alarming rise in PTOAK prevalence, potentially doubling by 2030. The financial burden of knee OA treatment in the United States is already substantial, costing between $5.7 and $15 billion USD annually. This projected increase in prevalence could further increase healthcare expenditures by $1 to 3 billion by 2030. These results emphasize the need for additional research into factors contributing to PTOAK, evidence-based preventive public health interventions, and the development of multidisciplinary system-based care delivery optimization pathways.

Privacy-preserving record linkage across disparate institutions and datasets to enable a learning health system: The national COVID cohort collaborative (N3C) experience

Introduction

Research driven by real-world clinical data is increasingly vital to enabling learning health systems, but integrating such data from across disparate health systems is challenging. As part of the NCATS National COVID Cohort Collaborative (N3C), the N3C Data Enclave was established as a centralized repository of deidentified and harmonized COVID-19 patient data from institutions across the US. However, making this data most useful for research requires linking it with information such as mortality data, images, and viral variants. The objective of this project was to establish privacy-preserving record linkage (PPRL) methods to ensure that patient-level EHR data remains secure and private when governance-approved linkages with other datasets occur.

Methods

Separate agreements and approval processes govern N3C data contribution and data access. The Linkage Honest Broker (LHB), an independent neutral party (the Regenstrief Institute), ensures data linkages are robust and secure by adding an extra layer of separation between protected health information and clinical data. The LHB's PPRL methods (including algorithms, processes, and governance) match patient records using "deidentified tokens," which are hashed combinations of identifier fields that define a match across data repositories without using patients' clear-text identifiers.

Results

These methods enable three linkage functions: Deduplication, Linking Multiple Datasets, and Cohort Discovery. To date, two external repositories have been cross-linked. As of March 1, 2023, 43 sites have signed the LHB Agreement; 35 sites have sent tokens generated for 9 528 998 patients. In this initial cohort, the LHB identified 135 037 matches and 68 596 duplicates.

Conclusion

This large-scale linkage study using deidentified datasets of varying characteristics established secure methods for protecting the privacy of N3C patient data when linked for research purposes. This technology has potential for use with registries for other diseases and conditions.

The N3C governance ecosystem: A model socio-technical partnership for the future of collaborative analytics at scale

The National COVID Cohort Collaborative (N3C) is a public-private-government partnership established during the Coronavirus pandemic to create a centralized data resource called the "N3C data enclave." This resource contains individual-level health data from participating healthcare sites nationwide to support rapid collaborative analytics. N3C has enabled analytics within a cloud-based enclave of data from electronic health records from over 17 million people (with and without COVID-19) in the USA. To achieve this goal of a shared data resource, N3C implemented a shared governance strategy involving stakeholders in decision-making. The approach leveraged best practices in data stewardship and team science to rapidly enable COVID-19-related research at scale while respecting the privacy of data subjects and participating institutions. N3C balanced equitable access to data, team-based scientific productivity, and individual professional recognition - a key incentive for academic researchers. This governance approach makes N3C research sustainable and effective beyond the initial days of the pandemic. N3C demonstrated that shared governance can overcome traditional barriers to data sharing without compromising data security and trust. The governance innovations described herein are a helpful framework for other privacy-preserving data infrastructure programs and provide a working model for effective team science beyond COVID-19.

COVID-19 Does Not Increase the Risk of Spontaneous Cervical Artery Dissection

Background Case reports have raised the possibility of an association between coronavirus disease 2019 (COVID-19) and spontaneous cervical artery dissection (sCeAD), yet no large studies have examined this association. We hypothesized that adults with confirmed COVID-19 would have an increased risk of sCeAD over the subsequent six months compared to test-negative controls after adjusting for confounding variables. Methods We obtained data from a United States medical records network (TriNetX, Inc., Cambridge, MA) of >106 million patients, providing adequate power needed for this rare outcome. We identified two cohorts of adults meeting the criteria of (1) test-confirmed COVID-19 or (2) non-COVID-19 test-negative controls, from April 1, 2020, to December 31, 2022. Patients with previous COVID-19 or conditions predisposing to sCeAD were excluded. Propensity matching was used to control for variables associated with sCeAD and markers of healthcare utilization. Results The number of patients reduced from before matching (COVID-19: 491,592; non-COVID-19: 1,472,895) to after matching, resulting in 491,115 patients per cohort. After matching, there were 22 cases of sCeAD in the COVID-19 cohort (0.0045%) and 20 cases in the non-COVID-19 cohort (0.0041%), yielding a risk ratio of 1.10 (95% CI: 0.60-2.02; P = 0.7576). Both cohorts had a median of five healthcare visits during follow-up. Conclusions Our results suggest that COVID-19 is not a risk factor for sCeAD. This null finding alleviates the concern raised by initial case reports and may better direct future research efforts on this topic.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Preexisting Autoimmunity Is Associated With Increased Severity of Coronavirus Disease 2019: A Retrospective Cohort Study Using Data From the National COVID Cohort Collaborative (N3C)

BACKGROUND

Identifying individuals with a higher risk of developing severe coronavirus disease 2019 (COVID-19) outcomes will inform targeted and more intensive clinical monitoring and management. To date, there is mixed evidence regarding the impact of preexisting autoimmune disease (AID) diagnosis and/or immunosuppressant (IS) exposure on developing severe COVID-19 outcomes.

METHODS

A retrospective cohort of adults diagnosed with COVID-19 was created in the National COVID Cohort Collaborative enclave. Two outcomes, life-threatening disease and hospitalization, were evaluated by using logistic regression models with and without adjustment for demographics and comorbidities.

RESULTS

Of the 2 453 799 adults diagnosed with COVID-19, 191 520 (7.81%) had a preexisting AID diagnosis and 278 095 (11.33%) had a preexisting IS exposure. Logistic regression models adjusted for demographics and comorbidities demonstrated that individuals with a preexisting AID (odds ratio [OR], 1.13; 95% confidence interval [CI]: 1.09-1.17; P < .001), IS exposure (OR, 1.27; 95% CI: 1.24-1.30; P < .001), or both (OR, 1.35; 95% CI: 1.29-1.40; P < .001) were more likely to have a life-threatening disease. These results were consistent when hospitalization was evaluated. A sensitivity analysis evaluating specific IS revealed that tumor necrosis factor inhibitors were protective against life-threatening disease (OR, 0.80; 95% CI: .66-.96; P = .017) and hospitalization (OR, 0.80; 95% CI: .73-.89; P < .001).

CONCLUSIONS

Patients with preexisting AID, IS exposure, or both are more likely to have a life-threatening disease or hospitalization. These patients may thus require tailored monitoring and preventative measures to minimize negative consequences of COVID-19.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Association between chiropractic spinal manipulation and gabapentin prescription in adults with radicular low back pain: retrospective cohort study using US data

OBJECTIVES

Radicular low back pain (rLBP) is often treated off-label with gabapentin or by chiropractors using chiropractic spinal manipulative therapy (CSMT). To date, no studies have examined the association between these interventions. We hypothesised that adults under 50 years of age receiving CSMT for newly diagnosed rLBP would have reduced odds of receiving a gabapentin prescription over 1 year-follow-up.

DESIGN

Retrospective cohort study.

SETTING

US network including linked medical records, medical claims and pharmacy claims of >122 million patients attending large healthcare organisations (TriNetX), queried 15 June 2023, yielding data from 2017 to 2023.

PARTICIPANTS

Adults aged 18-49 were included at their first occurrence of rLBP diagnosis. Exclusions were severe pathology, other spinal conditions, on-label gabapentin indications and gabapentin contraindications. Propensity score matching controlled for variables associated with gabapentin use and receipt of prescription medication over the preceding year.

INTERVENTIONS

Patients were divided into CSMT or usual medical care cohorts based on the care received on the index date of rLBP diagnosis.

PRIMARY AND SECONDARY OUTCOME MEASURES

OR for gabapentin prescription.

RESULTS

After propensity matching, there were 1635 patients per cohort (mean age 36.3±8.6 years, 60% women). Gabapentin prescription over 1-year follow-up was significantly lower in the CSMT cohort compared with the usual medical care cohort, with an OR (95% CI) of 0.53 (0.40 to 0.71; p<0.0001). Sensitivity analyses revealed early divergence in cumulative incidence of prescription; and no significant between-cohort difference in a negative control outcome (gastrointestinal medication) suggesting adequate control for pharmacological care preference.

CONCLUSIONS

Our findings suggest that US adults receiving CSMT for newly diagnosed rLBP have significantly reduced odds of receiving a gabapentin prescription over 1-year follow-up compared with those receiving usual medical care. Results may not be generalisable and should be replicated in other healthcare settings and corroborated by a prospective study to reduce confounding.

De-black-boxing health AI: demonstrating reproducible machine learning computable phenotypes using the N3C-RECOVER Long COVID model in the All of Us data repository

Machine learning (ML)-driven computable phenotypes are among the most challenging to share and reproduce. Despite this difficulty, the urgent public health considerations around Long COVID make it especially important to ensure the rigor and reproducibility of Long COVID phenotyping algorithms such that they can be made available to a broad audience of researchers. As part of the NIH Researching COVID to Enhance Recovery (RECOVER) Initiative, researchers with the National COVID Cohort Collaborative (N3C) devised and trained an ML-based phenotype to identify patients highly probable to have Long COVID. Supported by RECOVER, N3C and NIH's All of Us study partnered to reproduce the output of N3C's trained model in the All of Us data enclave, demonstrating model extensibility in multiple environments. This case study in ML-based phenotype reuse illustrates how open-source software best practices and cross-site collaboration can de-black-box phenotyping algorithms, prevent unnecessary rework, and promote open science in informatics.

Developing a linked electronic health record derived data platform to support research into healthy ageing

Introduction

Digitalisation of Electronic Health Record (EHR) data has created unique opportunities for research. However, these data are routinely collected for operational purposes and so are not curated to the standard required for research. Harnessing such routine data at large scale allows efficient and long-term epidemiological and health services research.

Objectives

To describe the establishment a linked EHR derived data platform in the National Centre for Healthy Ageing, Melbourne, Australia, aimed at enabling research targeting national health priority areas in ageing.

Methods

Our approach incorporated: data validation, curation and warehousing to ensure quality and completeness; end-user engagement and consensus on the platform content; implementation of an artificial intelligence (AI) pipeline for extraction of text-based data items; early consumer involvement; and implementation of routine collection of patient reported outcome measures, in a multisite public health service.

Results

Data for a cohort of >800,000 patients collected over a 10-year period have been curated within the platform's research data warehouse. So far 117 items have been identified as suitable for inclusion, from 11 research relevant datasets held within the health service EHR systems. Data access, extraction and release processes, guided by the Five Safes Framework, are being tested through project use-cases. A natural language processing (NLP) pipeline has been implemented and a framework for the routine collection and incorporation of patient reported outcome measures developed.

Conclusions

We highlight the importance of establishing comprehensive processes for the foundations of a data platform utilising routine data not collected for research purposes. These robust foundations will facilitate future expansion through linkages to other datasets for the efficient and cost-effective study of health related to ageing at a large scale.

Relative Effectiveness of Coronavirus Disease 2019 Vaccination and Booster Dose Combinations Among 18.9 Million Vaccinated Adults During the Early Severe Acute Respiratory Syndrome Coronavirus 2 Omicron Period-United States, 1 January 2022 to 31 March 2022

BACKGROUND

Small sample sizes have limited prior studies' ability to capture severe COVID-19 outcomes, especially among Ad26.COV2.S vaccine recipients. This study of 18.9 million adults aged ≥18 years assessed relative vaccine effectiveness (rVE) in three recipient cohorts: (1) primary Ad26.COV2.S vaccine and Ad26.COV2.S booster (2 Ad26.COV2.S), (2) primary Ad26.COV2.S vaccine and mRNA booster (Ad26.COV2.S+mRNA), (3) two doses of primary mRNA vaccine and mRNA booster (3 mRNA).

METHODS

We analyzed two de-identified datasets linked using privacy-preserving record linkage (PPRL): insurance claims and retail pharmacy COVID-19 vaccination data. We assessed the presence of COVID-19 diagnosis during January 1-March 31, 2022 in: (1) any claim, (2) outpatient claim, (3) emergency department (ED) claim, (4) inpatient claim, and (5) inpatient claim with intensive care unit (ICU) admission. rVE for each outcome comparing three recipient cohorts (reference: two Ad26.COV2.S doses) was estimated from adjusted Cox proportional hazards models.

RESULTS

Compared with two Ad26.COV2.S doses, Ad26.COV2.S+mRNA and three mRNA doses were more effective against all COVID-19 outcomes, including 57% (95% CI: 52-62) and 62% (95% CI: 58-65) rVE against an ED visit; 44% (95% CI: 34-52) and 54% (95% CI: 48-59) rVE against hospitalization; and 48% (95% CI: 22-66) and 66% (95% CI: 53-75) rVE against ICU admission, respectively.

CONCLUSIONS

This study demonstrated that Ad26.COV2.S + mRNA doses were as good as three doses of mRNA, and better than two doses of Ad26.COV2.S. Vaccination continues to be an important preventive measure for reducing the public health impact of COVID-19.

Long COVID risk and pre-COVID vaccination in an EHR-based cohort study from the RECOVER program

Long COVID, or complications arising from COVID-19 weeks after infection, has become a central concern for public health experts. The United States National Institutes of Health founded the RECOVER initiative to better understand long COVID. We used electronic health records available through the National COVID Cohort Collaborative to characterize the association between SARS-CoV-2 vaccination and long COVID diagnosis. Among patients with a COVID-19 infection between August 1, 2021 and January 31, 2022, we defined two cohorts using distinct definitions of long COVID-a clinical diagnosis (n = 47,404) or a previously described computational phenotype (n = 198,514)-to compare unvaccinated individuals to those with a complete vaccine series prior to infection. Evidence of long COVID was monitored through June or July of 2022, depending on patients' data availability. We found that vaccination was consistently associated with lower odds and rates of long COVID clinical diagnosis and high-confidence computationally derived diagnosis after adjusting for sex, demographics, and medical history.

Clinical encounter heterogeneity and methods for resolving in networked EHR data: a study from N3C and RECOVER programs

OBJECTIVE

Clinical encounter data are heterogeneous and vary greatly from institution to institution. These problems of variance affect interpretability and usability of clinical encounter data for analysis. These problems are magnified when multisite electronic health record (EHR) data are networked together. This article presents a novel, generalizable method for resolving encounter heterogeneity for analysis by combining related atomic encounters into composite "macrovisits."

MATERIALS AND METHODS

Encounters were composed of data from 75 partner sites harmonized to a common data model as part of the NIH Researching COVID to Enhance Recovery Initiative, a project of the National Covid Cohort Collaborative. Summary statistics were computed for overall and site-level data to assess issues and identify modifications. Two algorithms were developed to refine atomic encounters into cleaner, analyzable longitudinal clinical visits.

RESULTS

Atomic inpatient encounters data were found to be widely disparate between sites in terms of length-of-stay (LOS) and numbers of OMOP CDM measurements per encounter. After aggregating encounters to macrovisits, LOS and measurement variance decreased. A subsequent algorithm to identify hospitalized macrovisits further reduced data variability.

DISCUSSION

Encounters are a complex and heterogeneous component of EHR data and native data issues are not addressed by existing methods. These types of complex and poorly studied issues contribute to the difficulty of deriving value from EHR data, and these types of foundational, large-scale explorations, and developments are necessary to realize the full potential of modern real-world data.

CONCLUSION

This article presents method developments to manipulate and resolve EHR encounter data issues in a generalizable way as a foundation for future research and analysis.

Effect of Nirmatrelvir/Ritonavir (Paxlovid) on Hospitalization among Adults with COVID-19: an EHR-based Target Trial Emulation from N3C

This study leverages electronic health record data in the National COVID Cohort Collaborative's (N3C) repository to investigate disparities in Paxlovid treatment and to emulate a target trial assessing its effectiveness in reducing COVID-19 hospitalization rates. From an eligible population of 632,822 COVID-19 patients seen at 33 clinical sites across the United States between December 23, 2021 and December 31, 2022, patients were matched across observed treatment groups, yielding an analytical sample of 410,642 patients. We estimate a 65% reduced odds of hospitalization among Paxlovid-treated patients within a 28-day follow-up period, and this effect did not vary by patient vaccination status. Notably, we observe disparities in Paxlovid treatment, with lower rates among Black and Hispanic or Latino patients, and within socially vulnerable communities. Ours is the largest study of Paxlovid's real-world effectiveness to date, and our primary findings are consistent with previous randomized control trials and real-world studies.

Complications Associated With Transesophageal Echocardiography in Transcatheter Structural Cardiac Interventions

BACKGROUND

Transesophageal echocardiograms (TEEs) performed during transcatheter structural cardiac interventions may result in greater complications than those performed in the nonoperative setting or even those performed during cardiac surgery. However, there are limited data on complications associated with TEE during these procedures. We evaluated the prevalence of major complications among these patients in the United States.

METHODS

A retrospective cohort study was conducted using an electronic health record database (TriNetX Research Network) from large academic medical centers across the United States for patients undergoing TEE during transcatheter structural interventions from January 2012 to January 2022. Using the American Society of Echocardiography-endorsed International Statistical Classification of Diseases and Related Health Problems Clinical Modifications (10th edition) codes, patients undergoing TEE during a transcatheter structural cardiac intervention, including transaortic, mitral or tricuspid valve repair, left atrial appendage occlusion, atrial septal defect closure, patent foramen ovale closure, and paravalvular leak repair, were identified. The primary outcome was major complications within 72 hours of the procedure (composite of bleeding and esophageal and upper respiratory tract injury). The secondary aim was the frequency of major complications, death, or cardiac arrest within 72 hours in patients who completed intraoperative TEE during surgical valve replacement.

RESULTS

Among 12,043 adult patients (mean age, 74 years old; 42% female) undergoing TEE for transcatheter structural cardiac interventions, 429 (3.6%) patients had a major complication. Complication frequency was higher in patients on anticoagulation or antiplatelet therapy compared with those not on therapy (3.9% vs 0.5%; risk ratio [RR] = 8.09, P < .001). Compared with those patients <65 years of age, patients ≥65 years of age had a higher frequency of major complications (3.9% vs 2.2%; RR = 1.75, P < .001). Complication frequency was similar among male and female patients (3.5% vs 3.7%; RR = 0.96, P = .67). Among 28,848 patients who completed surgical valve replacement with TEE guidance, 728 (2.5%) experienced a major complication.

CONCLUSIONS

This study found that more than 3% of patients undergoing TEE during transcatheter structural cardiac interventions have a major complication, which is more common among those on anticoagulant or antiplatelet therapy or who are elderly. With a shift of poor surgical candidates to less invasive percutaneous procedures, the future of TEE-guided procedures relies on comprehensive risk discussion and updating practices beyond conventional methods to minimize risk for TEE-related complications.

Data quality considerations for evaluating COVID-19 treatments using real world data: learnings from the National COVID Cohort Collaborative (N3C)

BACKGROUND

Multi-institution electronic health records (EHR) are a rich source of real world data (RWD) for generating real world evidence (RWE) regarding the utilization, benefits and harms of medical interventions. They provide access to clinical data from large pooled patient populations in addition to laboratory measurements unavailable in insurance claims-based data. However, secondary use of these data for research requires specialized knowledge and careful evaluation of data quality and completeness. We discuss data quality assessments undertaken during the conduct of prep-to-research, focusing on the investigation of treatment safety and effectiveness.

METHODS

Using the National COVID Cohort Collaborative (N3C) enclave, we defined a patient population using criteria typical in non-interventional inpatient drug effectiveness studies. We present the challenges encountered when constructing this dataset, beginning with an examination of data quality across data partners. We then discuss the methods and best practices used to operationalize several important study elements: exposure to treatment, baseline health comorbidities, and key outcomes of interest.

RESULTS

We share our experiences and lessons learned when working with heterogeneous EHR data from over 65 healthcare institutions and 4 common data models. We discuss six key areas of data variability and quality. (1) The specific EHR data elements captured from a site can vary depending on source data model and practice. (2) Data missingness remains a significant issue. (3) Drug exposures can be recorded at different levels and may not contain route of administration or dosage information. (4) Reconstruction of continuous drug exposure intervals may not always be possible. (5) EHR discontinuity is a major concern for capturing history of prior treatment and comorbidities. Lastly, (6) access to EHR data alone limits the potential outcomes which can be used in studies.

CONCLUSIONS

The creation of large scale centralized multi-site EHR databases such as N3C enables a wide range of research aimed at better understanding treatments and health impacts of many conditions including COVID-19. As with all observational research, it is important that research teams engage with appropriate domain experts to understand the data in order to define research questions that are both clinically important and feasible to address using these real world data.

Coding long COVID: characterizing a new disease through an ICD-10 lens

BACKGROUND

Naming a newly discovered disease is a difficult process; in the context of the COVID-19 pandemic and the existence of post-acute sequelae of SARS-CoV-2 infection (PASC), which includes long COVID, it has proven especially challenging. Disease definitions and assignment of a diagnosis code are often asynchronous and iterative. The clinical definition and our understanding of the underlying mechanisms of long COVID are still in flux, and the deployment of an ICD-10-CM code for long COVID in the USA took nearly 2 years after patients had begun to describe their condition. Here, we leverage the largest publicly available HIPAA-limited dataset about patients with COVID-19 in the US to examine the heterogeneity of adoption and use of U09.9, the ICD-10-CM code for "Post COVID-19 condition, unspecified."

METHODS

We undertook a number of analyses to characterize the N3C population with a U09.9 diagnosis code (n = 33,782), including assessing person-level demographics and a number of area-level social determinants of health; diagnoses commonly co-occurring with U09.9, clustered using the Louvain algorithm; and quantifying medications and procedures recorded within 60 days of U09.9 diagnosis. We stratified all analyses by age group in order to discern differing patterns of care across the lifespan.

RESULTS

We established the diagnoses most commonly co-occurring with U09.9 and algorithmically clustered them into four major categories: cardiopulmonary, neurological, gastrointestinal, and comorbid conditions. Importantly, we discovered that the population of patients diagnosed with U09.9 is demographically skewed toward female, White, non-Hispanic individuals, as well as individuals living in areas with low poverty and low unemployment. Our results also include a characterization of common procedures and medications associated with U09.9-coded patients.

CONCLUSIONS

This work offers insight into potential subtypes and current practice patterns around long COVID and speaks to the existence of disparities in the diagnosis of patients with long COVID. This latter finding in particular requires further research and urgent remediation.

Pre-existing autoimmunity is associated with increased severity of COVID-19: A retrospective cohort study using data from the National COVID Cohort Collaborative (N3C)

Importance

Identifying individuals with a higher risk of developing severe COVID-19 outcomes will inform targeted or more intensive clinical monitoring and management.

Objective

To examine, using data from the National COVID Cohort Collaborative (N3C), whether patients with pre-existing autoimmune disease (AID) diagnosis and/or immunosuppressant (IS) exposure are at a higher risk of developing severe COVID-19 outcomes.

Design setting and participants

A retrospective cohort of 2,453,799 individuals diagnosed with COVID-19 between January 1 st , 2020, and June 30 th , 2022, was created from the N3C data enclave, which comprises data of 15,231,849 patients from 75 USA data partners. Patients were stratified as those with/without a pre-existing diagnosis of AID and/or those with/without exposure to IS prior to COVID-19.

Main outcomes and measures

Two outcomes of COVID-19 severity, derived from the World Health Organization severity score, were defined, namely life-threatening disease and hospitalization. Odds ratios (ORs) with 95% confidence intervals (CIs) were calculated using logistic regression models with and without adjustment for demographics (age, BMI, gender, race, ethnicity, smoking status), and comorbidities (cardiovascular disease, dementia, pulmonary disease, liver disease, type 2 diabetes mellitus, kidney disease, cancer, and HIV infection).

Results

In total, 2,453,799 (16.11% of the N3C cohort) adults (age> 18 years) were diagnosed with COVID-19, of which 191,520 (7.81%) had a prior AID diagnosis, and 278,095 (11.33%) had a prior IS exposure. Logistic regression models adjusted for demographic factors and comorbidities demonstrated that individuals with a prior AID (OR = 1.13, 95% CI 1.09 - 1.17; p =2.43E-13), prior exposure to IS (OR= 1.27, 95% CI 1.24 - 1.30; p =3.66E-74), or both (OR= 1.35, 95% CI 1.29 - 1.40; p =7.50E-49) were more likely to have a life-threatening COVID-19 disease. These results were confirmed after adjusting for exposure to antivirals and vaccination in a cohort subset with COVID-19 diagnosis dates after December 2021 (AID OR = 1.18, 95% CI 1.02 - 1.36; p =2.46E-02; IS OR= 1.60, 95% CI 1.41 - 1.80; p =5.11E-14; AID+IS OR= 1.93, 95% CI 1.62 - 2.30; p =1.68E-13). These results were consistent when evaluating hospitalization as the outcome and also when stratifying by race and sex. Finally, a sensitivity analysis evaluating specific IS revealed that TNF inhibitors were protective against life-threatening disease (OR = 0.80, 95% CI 0.66-0.96; p =1.66E-2) and hospitalization (OR = 0.80, 95% CI 0.73 - 0.89; p =1.06E-05).

Conclusions and Relevance

Patients with pre-existing AID, exposure to IS, or both are more likely to have a life-threatening disease or hospitalization. These patients may thus require tailored monitoring and preventative measures to minimize negative consequences of COVID-19.

Association between chiropractic spinal manipulation and lumbar discectomy in adults with lumbar disc herniation and radiculopathy: retrospective cohort study using United States' data

OBJECTIVES

Chiropractic spinal manipulative therapy (CSMT) and lumbar discectomy are both used for lumbar disc herniation (LDH) and lumbosacral radiculopathy (LSR); however, limited research has examined the relationship between these therapies. We hypothesised that adults receiving CSMT for newly diagnosed LDH or LSR would have reduced odds of lumbar discectomy over 1-year and 2-year follow-up compared with those receiving other care.

DESIGN

Retrospective cohort study.

SETTING

101 million patient US health records network (TriNetX), queried on 24 October 2022, yielding data from 2012 query.

PARTICIPANTS

Adults age 18-49 with newly diagnosed LDH/LSR (first date of diagnosis) were included. Exclusions were prior lumbar surgery, absolute indications for surgery, trauma, spondylolisthesis and scoliosis. Propensity score matching controlled for variables associated with the likelihood of discectomy (eg, demographics, medications).

INTERVENTIONS

Patients were divided into cohorts according to receipt of CSMT.

PRIMARY AND SECONDARY OUTCOME MEASURES

ORs for lumbar discectomy; calculated by dividing odds in the CSMT cohort by odds in the cohort receiving other care.

RESULTS

After matching, there were 5785 patients per cohort (mean age 36.9±8.2). The ORs (95% CI) for discectomy were significantly reduced in the CSMT cohort compared with the cohort receiving other care over 1-year (0.69 (0.52 to 0.90), p=0.006) and 2-year follow-up (0.77 (0.60 to 0.99), p=0.040). E-value sensitivity analysis estimated the strength in terms of risk ratio an unmeasured confounding variable would need to account for study results, yielding point estimates for each follow-up (1 year: 2.26; 2 years: 1.92), which no variables in the literature reached.

CONCLUSIONS

Our findings suggest receiving CSMT compared with other care for newly diagnosed LDH/LSR is associated with significantly reduced odds of discectomy over 2-year follow-up. Given socioeconomic variables were unavailable and an observational design precludes inferring causality, the efficacy of CSMT for LDH/LSR should be examined via randomised controlled trial to eliminate residual confounding.

Post-COVID Phenotypic Manifestations are Associated with New-Onset Psychiatric Disease: Findings from the NIH N3C and RECOVER Studies

Acute COVID-19 infection can be followed by diverse clinical manifestations referred to as Post Acute Sequelae of SARS-CoV2 Infection (PASC). Studies have shown an increased risk of being diagnosed with new-onset psychiatric disease following a diagnosis of acute COVID-19. However, it was unclear whether non-psychiatric PASC-associated manifestations (PASC-AMs) are associated with an increased risk of new-onset psychiatric disease following COVID-19. A retrospective EHR cohort study of 1,603,767 individuals with acute COVID-19 was performed to evaluate whether non-psychiatric PASC-AMs are associated with new-onset psychiatric disease. Data were obtained from the National COVID Cohort Collaborative (N3C), which has EHR data from 65 clinical organizations. EHR codes were mapped to 151 non-psychiatric PASC-AMs recorded 28-120 days following SARS-CoV-2 diagnosis and before diagnosis of new-onset psychiatric disease. Association of newly diagnosed psychiatric disease with age, sex, race, pre-existing comorbidities, and PASC-AMs in seven categories was assessed by logistic regression. There was a significant association between six categories and newly diagnosed anxiety, mood, and psychotic disorders, with odds ratios highest for cardiovascular (1.35, 1.27-1.42) PASC-AMs. Secondary analysis revealed that the proportions of 95 individual clinical features significantly differed between patients diagnosed with different psychiatric disorders. Our study provides evidence for association between non-psychiatric PASC-AMs and the incidence of newly diagnosed psychiatric disease. Significant associations were found for features related to multiple organ systems. This information could prove useful in understanding risk stratification for new-onset psychiatric disease following COVID-19. Prospective studies are needed to corroborate these findings.

FUNDING

NCATS U24 TR002306.

Long COVID Risk and Pre-COVID Vaccination: An EHR-Based Cohort Study from the RECOVER Program

Importance

Characterizing the effect of vaccination on long COVID allows for better healthcare recommendations.

Objective

To determine if, and to what degree, vaccination prior to COVID-19 is associated with eventual long COVID onset, among those a documented COVID-19 infection.

Design Settings and Participants

Retrospective cohort study of adults with evidence of COVID-19 between August 1, 2021 and January 31, 2022 based on electronic health records from eleven healthcare institutions taking part in the NIH Researching COVID to Enhance Recovery (RECOVER) Initiative, a project of the National Covid Cohort Collaborative (N3C).

Exposures

Pre-COVID-19 receipt of a complete vaccine series versus no pre-COVID-19 vaccination.

Main Outcomes and Measures

Two approaches to the identification of long COVID were used. In the clinical diagnosis cohort (n=47,752), ICD-10 diagnosis codes or evidence of a healthcare encounter at a long COVID clinic were used. In the model-based cohort (n=199,498), a computable phenotype was used. The association between pre-COVID vaccination and long COVID was estimated using IPTW-adjusted logistic regression and Cox proportional hazards.

Results

In both cohorts, when adjusting for demographics and medical history, pre-COVID vaccination was associated with a reduced risk of long COVID (clinic-based cohort: HR, 0.66; 95% CI, 0.55-0.80; OR, 0.69; 95% CI, 0.59-0.82; model-based cohort: HR, 0.62; 95% CI, 0.56-0.69; OR, 0.70; 95% CI, 0.65-0.75).

Conclusions and Relevance

Long COVID has become a central concern for public health experts. Prior studies have considered the effect of vaccination on the prevalence of future long COVID symptoms, but ours is the first to thoroughly characterize the association between vaccination and clinically diagnosed or computationally derived long COVID. Our results bolster the growing consensus that vaccines retain protective effects against long COVID even in breakthrough infections.

Key Points

Question: Does vaccination prior to COVID-19 onset change the risk of long COVID diagnosis?Findings: Four observational analyses of EHRs showed a statistically significant reduction in long COVID risk associated with pre-COVID vaccination (first cohort: HR, 0.66; 95% CI, 0.55-0.80; OR, 0.69; 95% CI, 0.59-0.82; second cohort: HR, 0.62; 95% CI, 0.56-0.69; OR, 0.70; 95% CI, 0.65-0.75).Meaning: Vaccination prior to COVID onset has a protective association with long COVID even in the case of breakthrough infections.

An ordinal severity scale for COVID-19 retrospective studies using Electronic Health Record data

Objectives

Although the World Health Organization (WHO) Clinical Progression Scale for COVID-19 is useful in prospective clinical trials, it cannot be effectively used with retrospective Electronic Health Record (EHR) datasets. Modifying the existing WHO Clinical Progression Scale, we developed an ordinal severity scale (OS) and assessed its usefulness in the analyses of COVID-19 patient outcomes using retrospective EHR data.

Materials and Methods

An OS was developed to assign COVID-19 disease severity using the Observational Medical Outcomes Partnership common data model within the National COVID Cohort Collaborative (N3C) data enclave. We then evaluated usefulness of the developed OS using heterogenous EHR data from January 2020 to October 2021 submitted to N3C by 63 healthcare organizations across the United States. Principal component analysis (PCA) was employed to characterize changes in disease severity among patients during the 28-day period following COVID-19 diagnosis.

Results

The data set used in this analysis consists of 2 880 456 patients. PCA of the day-to-day variation in OS levels over the totality of the 28-day period revealed contrasting patterns of variation in disease severity within the first and second 14 days and illustrated the importance of evaluation over the full 28-day period.

Discussion

An OS with well-defined, robust features, based on discrete EHR data elements, is useful for assessments of COVID-19 patient outcomes, providing insights on the progression of COVID-19 disease severity over time.

Conclusions

The OS provides a framework that can facilitate better understanding of the course of acute COVID-19, informing clinical decision-making and resource allocation.

Coding Long COVID: Characterizing a new disease through an ICD-10 lens

Background

Naming a newly discovered disease is a difficult process; in the context of the COVID-19 pandemic and the existence of post-acute sequelae of SARS-CoV-2 infection (PASC), which includes Long COVID, it has proven especially challenging. Disease definitions and assignment of a diagnosis code are often asynchronous and iterative. The clinical definition and our understanding of the underlying mechanisms of Long COVID are still in flux, and the deployment of an ICD-10-CM code for Long COVID in the US took nearly two years after patients had begun to describe their condition. Here we leverage the largest publicly available HIPAA-limited dataset about patients with COVID-19 in the US to examine the heterogeneity of adoption and use of U09.9, the ICD-10-CM code for "Post COVID-19 condition, unspecified."

Methods

We undertook a number of analyses to characterize the N3C population with a U09.9 diagnosis code ( n = 21,072), including assessing person-level demographics and a number of area-level social determinants of health; diagnoses commonly co-occurring with U09.9, clustered using the Louvain algorithm; and quantifying medications and procedures recorded within 60 days of U09.9 diagnosis. We stratified all analyses by age group in order to discern differing patterns of care across the lifespan.

Results

We established the diagnoses most commonly co-occurring with U09.9, and algorithmically clustered them into four major categories: cardiopulmonary, neurological, gastrointestinal, and comorbid conditions. Importantly, we discovered that the population of patients diagnosed with U09.9 is demographically skewed toward female, White, non-Hispanic individuals, as well as individuals living in areas with low poverty, high education, and high access to medical care. Our results also include a characterization of common procedures and medications associated with U09.9-coded patients.

Conclusions

This work offers insight into potential subtypes and current practice patterns around Long COVID, and speaks to the existence of disparities in the diagnosis of patients with Long COVID. This latter finding in particular requires further research and urgent remediation.

DQAgui: a graphical user interface for the MIRACUM data quality assessment tool

Background

With the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.

Methods

The aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.’s DQ categories conformance, completeness and plausibility.

Results

With DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.

Conclusions

As part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01961-z.

Increased risk of psychiatric sequelae of COVID-19 is highest early in the clinical course

Background

COVID-19 has been shown to increase the risk of adverse mental health consequences. A recent electronic health record (EHR)-based observational study showed an almost two-fold increased risk of new-onset mental illness in the first 90 days following a diagnosis of acute COVID-19.

Methods

We used the National COVID Cohort Collaborative, a harmonized EHR repository with 2,965,506 COVID-19 positive patients, and compared cohorts of COVID-19 patients with comparable controls. Patients were propensity score-matched to control for confounding factors. We estimated the hazard ratio (COVID-19:control) for new-onset of mental illness for the first year following diagnosis. We additionally estimated the change in risk for new-onset mental illness between the periods of 21-120 and 121-365 days following infection.

Findings

We find a significant increase in incidence of new-onset mental disorders in the period of 21-120 days following COVID-19 (3.8%, 3.6-4.0) compared to patients with respiratory tract infections (3%, 2.8-3.2). We further show that the risk for new-onset mental illness decreases over the first year following COVID-19 diagnosis compared to other respiratory tract infections and demonstrate a reduced (non-significant) hazard ratio over the period of 121-365 days following diagnosis. Similar findings are seen for new-onset anxiety disorders but not for mood disorders.

Interpretation

Patients who have recovered from COVID-19 are at an increased risk for developing new-onset mental illness, especially anxiety disorders. This risk is most prominent in the first 120 days following infection.

Funding

National Center for Advancing Translational Sciences (NCATS).