Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment

Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force

This ISPOR Good Practices report provides a framework for assessing the suitability of electronic health records data for use in health technology assessments (HTAs). Although electronic health record (EHR) data can fill evidence gaps and improve decisions, several important limitations can affect its validity and relevance. The ISPOR framework includes 2 components: data delineation and data fitness for purpose. Data delineation provides a complete understanding of the data and an assessment of its trustworthiness by describing (1) data characteristics; (2) data provenance; and (3) data governance. Fitness for purpose comprises (1) data reliability items, ie, how accurate and complete the estimates are for answering the question at hand and (2) data relevance items, which assess how well the data are suited to answer the particular question from a decision-making perspective. The report includes a checklist specific to EHR data reporting: the ISPOR SUITABILITY Checklist. It also provides recommendations for HTA agencies and policy makers to improve the use of EHR-derived data over time. The report concludes with a discussion of limitations and future directions in the field, including the potential impact from the substantial and rapid advances in the diffusion and capabilities of large language models and generative artificial intelligence. The report's immediate audiences are HTA evidence developers and users. We anticipate that it will also be useful to other stakeholders, particularly regulators and manufacturers, in the future.

Real-World Evidence to Reinforce Clinical Trial Evidence in Health Technology Assessment: A Critical Review of Real-World Evidence Requirements from Seven Countries and Recommendations to Improve Acceptance

BACKGROUND

Real-world evidence (RWE) can reinforce clinical trial evidence in health technology assessment (HTA).

OBJECTIVES

Review HTA bodies' (HTAbs) requirements for RWE, real uses, and acceptance across seven countries (Brazil, Canada, France, Germany, Italy, Spain, and the United Kingdom) and outline recommendations that may improve acceptance of RWE in efficacy/effectiveness assessments and appraisals processes.

METHODS

RWE requirements were summarized based on HTAbs' guidelines. Acceptance by HTAbs was evaluated based on industry experience and case studies.

RESULTS

As of June 2022, RWE methodological guidelines were in place in three of the seven countries. HTAbs typically requested analyses based on local data sources, but the preferred study design and data sources differed. HTAbs had individual submission, assessment, and appraisal processes; some allowed early meetings for the protocol and/or results validation, though few involved external experts or medical societies to provide input to assessment and appraisal. The extent of submission, assessment, and appraisal requirements did not necessarily reflect the degree of acceptance.

CONCLUSION

All the countries reviewed face common challenges regarding the use of RWE. Our proposals address the need to facilitate collaboration and communication with industry and regulatory agencies and the need for specific guidelines describing RWE design and criteria of acceptance throughout the assessment and appraisal processes.

Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

OBJECTIVES

Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future.

METHODS

Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies.

RESULTS

Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD.

CONCLUSION

NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Markov modeling for cost-effectiveness using federated health data network

OBJECTIVE

To introduce 2 R-packages that facilitate conducting health economics research on OMOP-based data networks, aiming to standardize and improve the reproducibility, transparency, and transferability of health economic models.

MATERIALS AND METHODS

We developed the software tools and demonstrated their utility by replicating a UK-based heart failure data analysis across 5 different international databases from Estonia, Spain, Serbia, and the United States.

RESULTS

We examined treatment trajectories of 47 163 patients. The overall incremental cost-effectiveness ratio (ICER) for telemonitoring relative to standard of care was 57 472 €/QALY. Country-specific ICERs were 60 312 €/QALY in Estonia, 58 096 €/QALY in Spain, 40 372 €/QALY in Serbia, and 90 893 €/QALY in the US, which surpassed the established willingness-to-pay thresholds.

DISCUSSION

Currently, the cost-effectiveness analysis lacks standard tools, is performed in ad-hoc manner, and relies heavily on published information that might not be specific for local circumstances. Published results often exhibit a narrow focus, central to a single site, and provide only partial decision criteria, limiting their generalizability and comprehensive utility.

CONCLUSION

We created 2 R-packages to pioneer cost-effectiveness analysis in OMOP CDM data networks. The first manages state definitions and database interaction, while the second focuses on Markov model learning and profile synthesis. We demonstrated their utility in a multisite heart failure study, comparing telemonitoring and standard care, finding telemonitoring not cost-effective.

The OMOP common data model in Australian primary care data: Building a quality research ready harmonised dataset

BACKGROUND

The use of routinely collected health data for secondary research purposes is increasingly recognised as a methodology that advances medical research, improves patient outcomes, and guides policy. This secondary data, as found in electronic medical records (EMRs), can be optimised through conversion into a uniform data structure to enable analysis alongside other comparable health metric datasets. This can be achieved with the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM), which employs a standardised vocabulary to facilitate systematic analysis across various observational databases. The concept behind the OMOP-CDM is the conversion of data into a common format through the harmonisation of terminologies, vocabularies, and coding schemes within a unique repository. The OMOP model enhances research capacity through the development of shared analytic and prediction techniques; pharmacovigilance for the active surveillance of drug safety; and 'validation' analyses across multiple institutions across Australia, the United States, Europe, and the Asia Pacific. In this research, we aim to investigate the use of the open-source OMOP-CDM in the PATRON primary care data repository.

METHODS

We used standard structured query language (SQL) to construct, extract, transform, and load scripts to convert the data to the OMOP-CDM. The process of mapping distinct free-text terms extracted from various EMRs presented a substantial challenge, as many terms could not be automatically matched to standard vocabularies through direct text comparison. This resulted in a number of terms that required manual assignment. To address this issue, we implemented a strategy where our clinical mappers were instructed to focus only on terms that appeared with sufficient frequency. We established a specific threshold value for each domain, ensuring that more than 95% of all records were linked to an approved vocabulary like SNOMED once appropriate mapping was completed. To assess the data quality of the resultant OMOP dataset we utilised the OHDSI Data Quality Dashboard (DQD) to evaluate the plausibility, conformity, and comprehensiveness of the data in the PATRON repository according to the Kahn framework.

RESULTS

Across three primary care EMR systems we converted data on 2.03 million active patients to version 5.4 of the OMOP common data model. The DQD assessment involved a total of 3,570 individual evaluations. Each evaluation compared the outcome against a predefined threshold. A 'FAIL' occurred when the percentage of non-compliant rows exceeded the specified threshold value. In this assessment of the primary care OMOP database described here, we achieved an overall pass rate of 97%.

CONCLUSION

The OMOP CDM's widespread international use, support, and training provides a well-established pathway for data standardisation in collaborative research. Its compatibility allows the sharing of analysis packages across local and international research groups, which facilitates rapid and reproducible data comparisons. A suite of open-source tools, including the OHDSI Data Quality Dashboard (Version 1.4.1), supports the model. Its simplicity and standards-based approach facilitates adoption and integration into existing data processes.

Advancing the use of real world evidence in health technology assessment: insights from a multi-stakeholder workshop

Introduction: Real-world evidence (RWE) in health technology assessment (HTA) holds significant potential for informing healthcare decision-making. A multistakeholder workshop was organised by the European Health Data and Evidence Network (EHDEN) and the GetReal Institute to explore the status, challenges, and opportunities in incorporating RWE into HTA, with a focus on learning from regulatory initiatives such as the European Medicines Agency (EMA) Data Analysis and Real World Interrogation Network (DARWIN EU®). Methods: The workshop gathered key stakeholders from regulatory agencies, HTA organizations, academia, and industry for three panel discussions on RWE and HTA integration. Insights and recommendations were collected through panel discussions and audience polls. The workshop outcomes were reviewed by authors to identify key themes, challenges, and recommendations. Results: The workshop discussions revealed several important findings relating to the use of RWE in HTA. Compared with regulatory processes, its adoption in HTA to date has been slow. Barriers include limited trust in RWE, data quality concerns, and uncertainty about best practices. Facilitators include multidisciplinary training, educational initiatives, and stakeholder collaboration, which could be facilitated by initiatives like EHDEN and the GetReal Institute. Demonstrating the impact of "driver projects" could promote RWE adoption in HTA. Conclusion: To enhance the integration of RWE in HTA, it is crucial to address known barriers through comprehensive training, stakeholder collaboration, and impactful exemplar research projects. By upskilling users and beneficiaries of RWE and those that generate it, promoting collaboration, and conducting "driver projects," can strengthen the HTA evidence base for more informed healthcare decisions.

Advancing Toward a Common Data Model in Ophthalmology: Gap Analysis of General Eye Examination Concepts to Standard Observational Medical Outcomes Partnership (OMOP) Concepts

Purpose

Evaluate the degree of concept coverage of the general eye examination in one widely used electronic health record (EHR) system using the Observational Health Data Sciences and Informatics Observational Medical Outcomes Partnership (OMOP) common data model (CDM).

Design

Study of data elements.

Participants

Not applicable.

Methods

Data elements (field names and predefined entry values) from the general eye examination in the Epic foundation system were mapped to OMOP concepts and analyzed. Each mapping was given a Health Level 7 equivalence designation-equal when the OMOP concept had the same meaning as the source EHR concept, wider when it was missing information, narrower when it was overly specific, and unmatched when there was no match. Initial mappings were reviewed by 2 graders. Intergrader agreement for equivalence designation was calculated using Cohen's kappa. Agreement on the mapped OMOP concept was calculated as a percentage of total mappable concepts. Discrepancies were discussed and a final consensus created. Quantitative analysis was performed on wider and unmatched concepts.

Main Outcome Measures

Gaps in OMOP concept coverage of EHR elements and intergrader agreement of mapped OMOP concepts.

Results

A total of 698 data elements (210 fields, 488 values) from the EHR were analyzed. The intergrader kappa on the equivalence designation was 0.88 (standard error 0.03, P < 0.001). There was a 96% agreement on the mapped OMOP concept. In the final consensus mapping, 25% (1% fields, 31% values) of the EHR to OMOP concept mappings were considered equal, 50% (27% fields, 60% values) wider, 4% (8% fields, 2% values) narrower, and 21% (52% fields, 8% values) unmatched. Of the wider mapped elements, 46% were missing the laterality specification, 24% had other missing attributes, and 30% had both issues. Wider and unmatched EHR elements could be found in all areas of the general eye examination.

Conclusions

Most data elements in the general eye examination could not be represented precisely using the OMOP CDM. Our work suggests multiple ways to improve the incorporation of important ophthalmology concepts in OMOP, including adding laterality to existing concepts. There exists a strong need to improve the coverage of ophthalmic concepts in source vocabularies so that the OMOP CDM can better accommodate vision research.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Transforming Estonian health data to the Observational Medical Outcomes Partnership (OMOP) Common Data Model: lessons learned

Objective

To describe the reusable transformation process of electronic health records (EHR), claims, and prescriptions data into Observational Medical Outcome Partnership (OMOP) Common Data Model (CDM), together with challenges faced and solutions implemented.

Materials and Methods

We used Estonian national health databases that store almost all residents' claims, prescriptions, and EHR records. To develop and demonstrate the transformation process of Estonian health data to OMOP CDM, we used a 10% random sample of the Estonian population (n = 150 824 patients) from 2012 to 2019 (MAITT dataset). For the sample, complete information from all 3 databases was converted to OMOP CDM version 5.3. The validation was performed using open-source tools.

Results

In total, we transformed over 100 million entries to standard concepts using standard OMOP vocabularies with the average mapping rate 95%. For conditions, observations, drugs, and measurements, the mapping rate was over 90%. In most cases, SNOMED Clinical Terms were used as the target vocabulary.

Discussion

During the transformation process, we encountered several challenges, which are described in detail with concrete examples and solutions.

Conclusion

For a representative 10% random sample, we successfully transferred complete records from 3 national health databases to OMOP CDM and created a reusable transformation process. Our work helps future researchers to transform linked databases into OMOP CDM more efficiently, ultimately leading to better real-world evidence.

Multiple modes of data sharing can facilitate secondary use of sensitive health data for research

Evidence-based healthcare relies on health data from diverse sources to inform decision-making across different domains, including disease prevention, aetiology, diagnostics, therapeutics and prognosis. Increasing volumes of highly granular data provide opportunities to leverage the evidence base, with growing recognition that health data are highly sensitive and onward research use may create privacy issues for individuals providing data. Concerns are heightened for data without explicit informed consent for secondary research use. Additionally, researchers-especially from under-resourced environments and the global South-may wish to participate in onward analysis of resources they collected or retain oversight of onward use to ensure ethical constraints are respected. Different data-sharing approaches may be adopted according to data sensitivity and secondary use restrictions, moving beyond the traditional Open Access model of unidirectional data transfer from generator to secondary user. We describe collaborative data sharing, facilitating research by combining datasets and undertaking meta-analysis involving collaborating partners; federated data analysis, where partners undertake synchronous, harmonised analyses on their independent datasets and then combine their results in a coauthored report, and trusted research environments where data are analysed in a controlled environment and only aggregate results are exported. We review how deidentification and anonymisation methods, including data perturbation, can reduce risks specifically associated with health data secondary use. In addition, we present an innovative modularised approach for building data sharing agreements incorporating a more nuanced approach to data sharing to protect privacy, and provide a framework for building the agreements for each of these data-sharing scenarios.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Integration of real-world evidence from different data sources in health technology assessment

Real-world evidence (RWE) is being increasingly used by a wide range of stakeholders involved in the therapeutic product lifecycle but remains underutilized in the health technology assessment (HTA) process. RWE aims to fill the current evidence gaps, reduce the uncertainty around the benefits of medical technologies, and better understand the long-term impact of health technologies in real-world conditions. Despite the minimal use of RWE in some elements of HTA, there has been a larger push to further utilize RWE in the HTA processes. HTA bodies, as other stakeholders, work towards developing more robust means to leverage RWE from various data sources in the HTA processes. However, these agencies need to overcome important challenges before the broader incorporation of RWE into their routine practice. This paper aims to explore the extensive integration of RWE utilizing diverse sources of RWD. We discuss the utilization of RWE in HTA processes, considering aspects such as when, where, and how RWE can be effectively applied. Additionally, we seek the potential challenges and barriers associated with the utilization of different data sources.

Common data model for sickle cell disease surveillance: considerations and implications

Objective

Population-level data on sickle cell disease (SCD) are sparse in the United States. The Centers for Disease Control and Prevention (CDC) is addressing the need for SCD surveillance through state-level Sickle Cell Data Collection Programs (SCDC). The SCDC developed a pilot common informatics infrastructure to standardize processes across states.

Materials and Methods

We describe the process for establishing and maintaining the proposed common informatics infrastructure for a rare disease, starting with a common data model and identify key data elements for public health SCD reporting.

Results

The proposed model is constructed to allow pooling of table shells across states for comparison. Core Surveillance Data reports are compiled based on aggregate data provided by states to CDC annually.

Discussion and Conclusion

We successfully implemented a pilot SCDC common informatics infrastructure to strengthen our distributed data network and provide a blueprint for similar initiatives in other rare diseases.

Guidance on using real-world evidence from Western Europe in Central and Eastern European health policy decision making

Aim: Real-world data and real-world evidence (RWE) are becoming more important for healthcare decision making and health technology assessment. We aimed to propose solutions to overcome barriers preventing Central and Eastern European (CEE) countries from using RWE generated in Western Europe. Materials & methods: To achieve this, following a scoping review and a webinar, the most important barriers were selected through a survey. A workshop was held with CEE experts to discuss proposed solutions. Results: Based on survey results, we selected the nine most important barriers. Multiple solutions were proposed, for example, the need for a European consensus, and building trust in using RWE. Conclusion: Through collaboration with regional stakeholders, we proposed a list of solutions to overcome barriers on transferring RWE from Western Europe to CEE countries.

Integrating real-world data from Brazil and Pakistan into the OMOP common data model and standardized health analytics framework to characterize COVID-19 in the Global South

OBJECTIVES

The aim of this work is to demonstrate the use of a standardized health informatics framework to generate reliable and reproducible real-world evidence from Latin America and South Asia towards characterizing coronavirus disease 2019 (COVID-19) in the Global South.

MATERIALS AND METHODS

Patient-level COVID-19 records collected in a patient self-reported notification system, hospital in-patient and out-patient records, and community diagnostic labs were harmonized to the Observational Medical Outcomes Partnership common data model and analyzed using a federated network analytics framework. Clinical characteristics of individuals tested for, diagnosed with or tested positive for, hospitalized with, admitted to intensive care unit with, or dying with COVID-19 were estimated.

RESULTS

Two COVID-19 databases covering 8.3 million people from Pakistan and 2.6 million people from Bahia, Brazil were analyzed. 109 504 (Pakistan) and 921 (Brazil) medical concepts were harmonized to Observational Medical Outcomes Partnership common data model. In total, 341 505 (4.1%) people in the Pakistan dataset and 1 312 832 (49.2%) people in the Brazilian dataset were tested for COVID-19 between January 1, 2020 and April 20, 2022, with a median [IQR] age of 36 [25, 76] and 38 (27, 50); 40.3% and 56.5% were female in Pakistan and Brazil, respectively. 1.2% percent individuals in the Pakistan dataset had Afghan ethnicity. In Brazil, 52.3% had mixed ethnicity. In agreement with international findings, COVID-19 outcomes were more severe in men, elderly, and those with underlying health conditions.

CONCLUSIONS

COVID-19 data from 2 large countries in the Global South were harmonized and analyzed using a standardized health informatics framework developed by an international community of health informaticians. This proof-of-concept study demonstrates a potential open science framework for global knowledge mobilization and clinical translation for timely response to healthcare needs in pandemics and beyond.

Recommendations to overcome barriers to the use of artificial intelligence-driven evidence in health technology assessment

Background

Artificial intelligence (AI) has attracted much attention because of its enormous potential in healthcare, but uptake has been slow. There are substantial barriers that challenge health technology assessment (HTA) professionals to use AI-generated evidence for decision-making from large real-world databases (e.g., based on claims data). As part of the European Commission-funded HTx H2020 (Next Generation Health Technology Assessment) project, we aimed to put forward recommendations to support healthcare decision-makers in integrating AI into the HTA processes. The barriers, addressed by the paper, are particularly focusing on Central and Eastern European (CEE) countries, where the implementation of HTA and access to health databases lag behind Western European countries.

Methods

We constructed a survey to rank the barriers to using AI for HTA purposes, completed by respondents from CEE jurisdictions with expertise in HTA. Using the results, two members of the HTx consortium from CEE developed recommendations on the most critical barriers. Then these recommendations were discussed in a workshop by a wider group of experts, including HTA and reimbursement decision-makers from both CEE countries and Western European countries, and summarized in a consensus report.

Results

Recommendations have been developed to address the top 15 barriers in areas of (1) human factor-related barriers, focusing on educating HTA doers and users, establishing collaborations and best practice sharing; (2) regulatory and policy-related barriers, proposing increasing awareness and political commitment and improving the management of sensitive information for AI use; (3) data-related barriers, suggesting enhancing standardization and collaboration with data networks, managing missing and unstructured data, using analytical and statistical approaches to address bias, using quality assessment tools and quality standards, improving reporting, and developing better conditions for the use of data; and (4) technological barriers, suggesting sustainable development of AI infrastructure.

Conclusion

In the field of HTA, the great potential of AI to support evidence generation and evaluation has not yet been sufficiently explored and realized. Raising awareness of the intended and unintended consequences of AI-based methods and encouraging political commitment from policymakers is necessary to upgrade the regulatory and infrastructural environment and knowledge base required to integrate AI into HTA-based decision-making processes better.

A Hybrid Architecture (CO-CONNECT) to Facilitate Rapid Discovery and Access to Data Across the United Kingdom in Response to the COVID-19 Pandemic: Development Study

BACKGROUND

COVID-19 data have been generated across the United Kingdom as a by-product of clinical care and public health provision, as well as numerous bespoke and repurposed research endeavors. Analysis of these data has underpinned the United Kingdom's response to the pandemic, and informed public health policies and clinical guidelines. However, these data are held by different organizations, and this fragmented landscape has presented challenges for public health agencies and researchers as they struggle to find relevant data to access and interrogate the data they need to inform the pandemic response at pace.

OBJECTIVE

We aimed to transform UK COVID-19 diagnostic data sets to be findable, accessible, interoperable, and reusable (FAIR).

METHODS

A federated infrastructure model (COVID - Curated and Open Analysis and Research Platform [CO-CONNECT]) was rapidly built to enable the automated and reproducible mapping of health data partners' pseudonymized data to the Observational Medical Outcomes Partnership Common Data Model without the need for any data to leave the data controllers' secure environments, and to support federated cohort discovery queries and meta-analysis.

RESULTS

A total of 56 data sets from 19 organizations are being connected to the federated network. The data include research cohorts and COVID-19 data collected through routine health care provision linked to longitudinal health care records and demographics. The infrastructure is live, supporting aggregate-level querying of data across the United Kingdom.

CONCLUSIONS

CO-CONNECT was developed by a multidisciplinary team. It enables rapid COVID-19 data discovery and instantaneous meta-analysis across data sources, and it is researching streamlined data extraction for use in a Trusted Research Environment for research and public health analysis. CO-CONNECT has the potential to make UK health data more interconnected and better able to answer national-level research questions while maintaining patient confidentiality and local governance procedures.

Risks of long-term use of proton pump inhibitor on ischemic vascular events: A distributed network analysis of 5 real-world observational Korean databases using a common data model

BACKGROUND

Proton pump inhibitors (PPIs) are frequently prescribed drugs. However, it has been suggested that they are associated with an increased risk of ischemic vascular events (IVE) including stroke, although the data are inconsistent.

AIMS

We investigated the association between PPIs use and IVE in five observational Korean databases using a common data model (CDM).

METHODS

This study included patient-based retrospective, observational cohort data of subjects aged over 18 years between January 1, 2004, and December 31, 2020, from five medical centers as part of the Observational Medical Outcomes Partnership (OMOP) CDM. Subjects who were included in both cohorts or had a previous history of ischemic stroke were excluded. After propensity matching, 8007 propensity-matched pairs between the PPIs and H₂ receptor antagonist (H₂RA) users were included in this study.

RESULTS

In the 1:1 propensity score matching with 8007 in each group, long-term PPIs use (⩾365 days) was not associated with ischemic stroke (odds ratio (OR) = 1.05, 95% confidence interval (CI) 0.71-1.56; I² = 57%), ischemic stroke and transient ischemic attack (OR = 1.02, 95% CI 0.71-1.48; I² = 53%), and net adverse clinical events (OR = 1.08, 95% CI 0.83-1.40; I² = 47%) compared with H2RAs users.

CONCLUSIONS

Our analysis in a large dataset found no evidence that long-term use of PPIs was associated with an increased risk of ischemic stroke.

Developing a common data model approach for DISCOVER CKD: A retrospective, global cohort of real-world patients with chronic kidney disease

Objectives

To describe a flexible common data model (CDM) approach that can be efficiently tailored to study-specific needs to facilitate pooled patient-level analysis and aggregated/meta-analysis of routinely collected retrospective patient data from disparate data sources; and to detail the application of this CDM approach to the DISCOVER CKD retrospective cohort, a longitudinal database of routinely collected (secondary) patient data of individuals with chronic kidney disease (CKD).

Methods

The flexible CDM approach incorporated three independent, exchangeable components that preceded data mapping and data model implementation: (1) standardized code lists (unifying medical events from different coding systems); (2) laboratory unit harmonization tables; and (3) base cohort definitions. Events between different coding vocabularies were not mapped code-to-code; for each data source, code lists of labels were curated at the entity/event level. A study team of epidemiologists, clinicians, informaticists, and data scientists were included within the validation of each component.

Results

Applying the CDM to the DISCOVER CKD retrospective cohort, secondary data from 1,857,593 patients with CKD were harmonized from five data sources, across three countries, into a discrete database for rapid real-world evidence generation.

Conclusions

This flexible CDM approach facilitates evidence generation from real-world data within the DISCOVER CKD retrospective cohort, providing novel insights into the epidemiology of CKD that may expedite improvements in diagnosis, prognosis, early intervention, and disease management. The adaptable architecture of this CDM approach ensures scalable, fast, and efficient application within other therapy areas to facilitate the combined analysis of different types of secondary data from multiple, heterogeneous sources.

Alignment of two standard terminologies for dosage form: RxNorm from the National Library of Medicine for the United States and EDQM from the European Directorate for the Quality in Medicines and Healthcare for Europe

BACKGROUND

There is currently no system that aligns pharmaceutically equivalent medicinal products across nations, creating obstacles to transnational medication prescribing and medical research. EDQM has been internationally recognized as the leading system in systematic pharmaceutical product descriptions. RxNorm is a critical terminology based in the US and used widely in applications internationally that would benefit from alignment with EDQM-based dosage form descriptions.

GOAL

Demonstrate a method for alignment of RxNorm dosage forms with EDQM terminologies and with EDQM dosage forms. Describe obstacles and advantages of such an alignment for ultimate application in calculating universal Pharmaceutical Product Identifiers.

METHODS

A pharmaceutical sciences student and a clinical pharmacology expert in dosage forms used definitions supplied by RxNorm and EDQM technical documentation to align the 120 RxNorm dose forms to EDQM-based dosage form description terms. The alignment of RxNorm to EDQM was then used to fit the RxNorm dose forms into an ontology based on EDQM.

RESULTS AND CONCLUSIONS

The alignment of RxNorm and EDQM requires further validation but provides a potential method of establishing interoperability between the two terminologies without cumbersome manual reclassification. There remain ambiguities within each dosage form nomenclature that create obstacles to integrating medication databases rooted in EDQM and RxNorm into a single worldwide database.

Machine learning for improving high-dimensional proxy confounder adjustment in healthcare database studies: an overview of the current literature

Controlling for large numbers of variables that collectively serve as 'proxies' for unmeasured factors can often improve confounding control in pharmacoepidemiologic studies utilizing administrative healthcare databases. There is a growing body of evidence showing that data-driven machine learning algorithms for high-dimensional proxy confounder adjustment can supplement investigator-specified variables to improve confounding control compared to adjustment based on investigator-specified variables alone. Consequently, there has been a recent focus on the development of data-driven methods for high-dimensional proxy confounder adjustment. In this paper, we discuss the considerations underpinning three areas for data-driven high-dimensional proxy confounder adjustment: 1) feature generation-transforming raw data into covariates (or features) to be used for proxy adjustment; 2) covariate prioritization, selection and adjustment; and 3) diagnostic assessment. We survey current approaches and recent advancements within each area, including the most widely used approach to proxy confounder adjustment in healthcare database studies (the high-dimensional propensity score or hdPS). We also discuss limitations of the hdPS and outline recent advancements that incorporate the principles of proxy adjustment with machine learning extensions to improve performance. We further discuss challenges and avenues of future development within each area. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE). This article is protected by copyright. All rights reserved.

Target Trial Emulation for Transparent and Robust Estimation of Treatment Effects for Health Technology Assessment Using Real-World Data: Opportunities and Challenges

Evidence about the relative effects of new treatments is typically collected in randomised controlled trials (RCTs). In many instances, evidence from RCTs falls short of the needs of health technology assessment (HTA). For example, RCTs may not be able to capture longer-term treatment effects, or include all relevant comparators and outcomes required for HTA purposes. Information routinely collected about patients and the care they receive have been increasingly used to complement RCT evidence on treatment effects. However, such routine (or real-world) data are not collected for research purposes, so investigators have little control over the way patients are selected into the study or allocated to the different treatment groups, introducing biases for example due to selection or confounding. A promising approach to minimise common biases in non-randomised studies that use real-world data (RWD) is to apply design principles from RCTs. This approach, known as 'target trial emulation' (TTE), involves (1) developing the protocol with respect to core study design and analysis components of the hypothetical RCT that would answer the question of interest, and (2) applying this protocol to the RWD so that it mimics the data that would have been gathered for the RCT. By making the 'target trial' explicit, TTE helps avoid common design flaws and methodological pitfalls in the analysis of non-randomised studies, keeping each step transparent and accessible. It provides a coherent framework that embeds existing analytical methods to minimise confounding and helps identify potential limitations of RWD and the extent to which these affect the HTA decision. This paper provides a broad overview of TTE and discusses the opportunities and challenges of using this approach in HTA. We describe the basic principles of trial emulation, outline some areas where TTE using RWD can help complement RCT evidence in HTA, identify potential barriers to its adoption in the HTA setting and highlight some priorities for future work.

Big data analysis and artificial intelligence in epilepsy - common data model analysis and machine learning-based seizure detection and forecasting

There has been significant interest in big data analysis and artificial intelligence (AI) in medicine. Ever-increasing medical data and advanced computing power have enabled the number of big data analyses and AI studies to increase rapidly. Here we briefly introduce epilepsy, big data, and AI and review big data analysis using a common data model. Studies in which AI has been actively applied, such as those of electroencephalography epileptiform discharge detection, seizure detection, and forecasting, will be reviewed. We will also provide practical suggestions for pediatricians to understand and interpret big data analysis and AI research and work together with technical expertise.

Unraveling COVID-19: A Large-Scale Characterization of 4.5 Million COVID-19 Cases Using CHARYBDIS

Purpose

Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) Characterizing Health Associated Risks and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD.

Patients and Methods

We conducted a descriptive retrospective database study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub. We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services.

Results

We aggregated over 22,000 unique characteristics describing patients with COVID-19. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts and are readily available online. Globally, we observed similarities in the USA and Europe: more women diagnosed than men but more men hospitalized than women, most diagnosed cases between 25 and 60 years of age versus most hospitalized cases between 60 and 80 years of age. South Korea differed with more women than men hospitalized. Common comorbidities included type 2 diabetes, hypertension, chronic kidney disease and heart disease. Common presenting symptoms were dyspnea, cough and fever. Symptom data availability was more common in hospitalized cohorts than diagnosed.

Conclusion

We constructed a global, multi-centre view to describe trends in COVID-19 progression, management and evolution over time. By characterising baseline variability in patients and geography, our work provides critical context that may otherwise be misconstrued as data quality issues. This is important as we perform studies on adverse events of special interest in COVID-19 vaccine surveillance.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to HTA: part 5

In the latest update we focus on recent publications which have provided insights into the importance of focusing on the development and consideration of a body of real-world evidence, and an approach to evaluating the complex area of treatment sequencing.

Applying Machine Learning in Distributed Data Networks for Pharmacoepidemiologic and Pharmacovigilance Studies: Opportunities, Challenges, and Considerations

Increasing availability of electronic health databases capturing real-world experiences with medical products has garnered much interest in their use for pharmacoepidemiologic and pharmacovigilance studies. The traditional practice of having numerous groups use single databases to accomplish similar tasks and address common questions about medical products can be made more efficient through well-coordinated multi-database studies, greatly facilitated through distributed data network (DDN) architectures. Access to larger amounts of electronic health data within DDNs has created a growing interest in using data-adaptive machine learning (ML) techniques that can automatically model complex associations in high-dimensional data with minimal human guidance. However, the siloed storage and diverse nature of the databases in DDNs create unique challenges for using ML. In this paper, we discuss opportunities, challenges, and considerations for applying ML in DDNs for pharmacoepidemiologic and pharmacovigilance studies. We first discuss major types of activities performed by DDNs and how ML may be used. Next, we discuss practical data-related factors influencing how DDNs work in practice. We then combine these discussions and jointly consider how opportunities for ML are affected by practical data-related factors for DDNs, leading to several challenges. We present different approaches for addressing these challenges and highlight efforts that real-world DDNs have taken or are currently taking to help mitigate them. Despite these challenges, the time is ripe for the emerging interest to use ML in DDNs, and the utility of these data-adaptive modeling techniques in pharmacoepidemiologic and pharmacovigilance studies will likely continue to increase in the coming years.

The Challenges of Outcomes-Based Contract Implementation for Medicines in Europe

OBJECTIVES

The aim was to outline the challenges of implementing outcomes-based contracts (OBCs) in Europe.

METHODS

A scoping review was conducted, building on the searches of a previous systematic review and updating them for December 2017 until May 2021. The combined results were screened, based on inclusion and exclusion criteria. All identified studies published in the English language that described specific OBC schemes for medicines in European countries were included. Insights into the challenges of OBCs were extracted and analysed to develop a conceptual framework.

RESULTS

Ten articles from the previous systematic review matched our inclusion criteria, along with 14 articles from electronic searches. Analysis of these 24 articles and classification of the challenges revealed that there are multiple barriers that must be overcome if OBCs that benefit all stakeholders are going to be adopted widely across Europe. These challenges were grouped according to five key themes: negotiation framework; outcomes; data; administration and implementation; and laws and regulation.

CONCLUSIONS

If the promise of OBCs is to be fully realised in Europe, there remain major challenges that need to be overcome by all stakeholders working in partnership. The overlapping and interconnected nature of these challenges highlights the complexity of OBC arrangements.

Can we use existing guidance to support the development of robust real-world evidence for health technology assessment/payer decision-making?

Advances in the digitization of health systems and expedited regulatory approvals of innovative treatments have led to increased potential for the use of real-world data (RWD) to generate real-world evidence (RWE) to complement evidence from clinical trials. However, health technology assessment (HTA) bodies and payers have concerns about the ability to generate RWE of sufficient quality to be pivotal evidence of relative treatment effectiveness. Consequently, there is a growing need for HTA bodies and payers to develop guidance for the industry and other stakeholders about the use of RWD/RWE to support access, reimbursement, and pricing. We therefore sought to (i) understand barriers to the use of RWD/RWE by HTA bodies and payers; (ii) review potential solutions in the form of published guidance; and (iii) review findings with selected HTA/payer bodies. Four themes considered key to shaping the generation of robust RWE for HTA bodies and payers were identified as: (i) data (availability, governance, and quality); (ii) methodology (design and analytics); (iii) trust (transparency and reproducibility); and (iv) policy and partnerships. A range of guidance documents were found from trusted sources that could address these themes. These were discussed with HTA experts. This commentary summarizes the potential guidance solutions available to help resolve issues faced by HTA decision-makers in the adoption of RWD/RWE. It shows that there is alignment among stakeholders about the areas that need improvement in the development of RWE and that the key priority to move forward is better collaboration to make data usable for multiple purposes.

Using Iterative Pairwise External Validation to Contextualize Prediction Model Performance: A Use Case Predicting 1-Year Heart Failure Risk in Patients with Diabetes Across Five Data Sources

INTRODUCTION

External validation of prediction models is increasingly being seen as a minimum requirement for acceptance in clinical practice. However, the lack of interoperability of healthcare databases has been the biggest barrier to this occurring on a large scale. Recent improvements in database interoperability enable a standardized analytical framework for model development and external validation. External validation of a model in a new database lacks context, whereby the external validation can be compared with a benchmark in this database. Iterative pairwise external validation (IPEV) is a framework that uses a rotating model development and validation approach to contextualize the assessment of performance across a network of databases. As a use case, we predicted 1-year risk of heart failure in patients with type 2 diabetes mellitus.

METHODS

The method follows a two-step process involving (1) development of baseline and data-driven models in each database according to best practices and (2) validation of these models across the remaining databases. We introduce a heatmap visualization that supports the assessment of the internal and external model performance in all available databases. As a use case, we developed and validated models to predict 1-year risk of heart failure in patients initializing a second pharmacological intervention for type 2 diabetes mellitus. We leveraged the power of the Observational Medical Outcomes Partnership common data model to create an open-source software package to increase the consistency, speed, and transparency of this process.

RESULTS

A total of 403,187 patients from five databases were included in the study. We developed five models that, when assessed internally, had a discriminative performance ranging from 0.73 to 0.81 area under the receiver operating characteristic curve with acceptable calibration. When we externally validated these models in a new database, three models achieved consistent performance and in context often performed similarly to models developed in the database itself. The visualization of IPEV provided valuable insights. From this, we identified the model developed in the Commercial Claims and Encounters (CCAE) database as the best performing model overall.

CONCLUSION

Using IPEV lends weight to the model development process. The rotation of development through multiple databases provides context to model assessment, leading to improved understanding of transportability and generalizability. The inclusion of a baseline model in all modelling steps provides further context to the performance gains of increasing model complexity. The CCAE model was identified as a candidate for clinical use. The use case demonstrates that IPEV provides a huge opportunity in a new era of standardised data and analytics to improve insight into and trust in prediction models at an unprecedented scale.

Characterization of Medication Trends for Chronic Kidney Disease: Mineral and Bone Disorder Treatment Using Electronic Health Record-Based Common Data Model

Chronic kidney disease–mineral bone disorder (CKD-MBD) is the most common complication in CKD patients. Although there is a consensus on treatment guidelines for CKD-MBD, it remains uncertain whether these treatment recommendations reflect actual practice. Therefore, the aim of this study was to investigate the CKD-MBD medication trend in real-world practice. This was a retrospective and observational study using a 12-year period database transformed into a common data model from three tertiary university hospitals. Study populations were subjects initially diagnosed as CKD. The date of diagnosis was designated as the index date. New patients were categorized year to year from 2008 to 2019 with a fixed observation period of 365 days to check the prescription of CKD-MBD medications including calcium-containing phosphate binder, noncalcium-containing phosphate binder, aluminium hydroxide, vitamin D receptor activator (VDRA), and cinacalcet. The numbers of CKD patients in the three hospitals were 7555, 2424, and 5351, respectively. The proportion for patients with CKD-MBD medication prescription decreased yearly regardless of hospital and CKD stage (p for trend < 0.05). The use of aluminium hydroxide disappeared steadily while the use of VDRA increased annually in all settings. Despite these changes in prescription patterns, the mean value for CKD-MBD-related serologic markers was almost within target range. The proportion of the population within the target value was not significantly changed. Irrespective of hospital and CKD stage, similar trends of prescription for CKD-MBD medications were observed in real-world practice. Further research with a distributed network study may be helpful to understand medication trends in CKD-MBD treatment.

From Inception to ConcePTION: Genesis of a Network to Support Better Monitoring and Communication of Medication Safety During Pregnancy and Breastfeeding

In 2019, the Innovative Medicines Initiative (IMI) funded the ConcePTION project-Building an ecosystem for better monitoring and communicating safety of medicines use in pregnancy and breastfeeding: validated and regulatory endorsed workflows for fast, optimised evidence generation-with the vision that there is a societal obligation to rapidly reduce uncertainty about the safety of medication use in pregnancy and breastfeeding. The present paper introduces the set of concepts used to describe the European data sources involved in the ConcePTION project and illustrates the ConcePTION Common Data Model (CDM), which serves as the keystone of the federated ConcePTION network. Based on data availability and content analysis of 21 European data sources, the ConcePTION CDM has been structured with six tables designed to capture data from routine healthcare, three tables for data from public health surveillance activities, three curated tables for derived data on population (e.g., observation time and mother-child linkage), plus four metadata tables. By its first anniversary, the ConcePTION CDM has enabled 13 data sources to run common scripts to contribute to major European projects, demonstrating its capacity to facilitate effective and transparent deployment of distributed analytics, and its potential to address questions about utilization, effectiveness, and safety of medicines in special populations, including during pregnancy and breastfeeding, and, more broadly, in the general population.

GA4GH: International policies and standards for data sharing across genomic research and healthcare

The Global Alliance for Genomics and Health (GA4GH) aims to accelerate biomedical advances by enabling the responsible sharing of clinical and genomic data through both harmonized data aggregation and federated approaches. The decreasing cost of genomic sequencing (along with other genome-wide molecular assays) and increasing evidence of its clinical utility will soon drive the generation of sequence data from tens of millions of humans, with increasing levels of diversity. In this perspective, we present the GA4GH strategies for addressing the major challenges of this data revolution. We describe the GA4GH organization, which is fueled by the development efforts of eight Work Streams and informed by the needs of 24 Driver Projects and other key stakeholders. We present the GA4GH suite of secure, interoperable technical standards and policy frameworks and review the current status of standards, their relevance to key domains of research and clinical care, and future plans of GA4GH. Broad international participation in building, adopting, and deploying GA4GH standards and frameworks will catalyze an unprecedented effort in data sharing that will be critical to advancing genomic medicine and ensuring that all populations can access its benefits.

Covid-19 Vaccine Surveillance in Saudi Arabia: Opportunities for Real-time Assessment

After months of confronting COVID-19 pandemic, several countries, including Saudi Arabia, have recently approved newly developed vaccines to prevent COVID-19 infection. With the new technology utilized to develop some vaccines, questions arise about their long-term safety. To provide rapid response to emerging safety issues, robust surveillance programs that provide near real-time analysis of vaccines effects are required. Saudi Arabia has a well-established passive pharmacovigilance system that monitors drugs and vaccines safety. However, recent development in health digitalization in Saudi Arabia may provide a unique opportunity to harvest existing resources to generate high-quality evidence. This commentary provides an overview of the available systems that can be utilized to monitor the COVID-19 vaccines’ safety and discusses opportunities for data integration to improve data quality and generate real-world evidence on COVID-19 vaccine safety.

Unraveling COVID-19: a large-scale characterization of 4.5 million COVID-19 cases using CHARYBDIS

Background: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response [1,2]. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) [3] Characterizing Health Associated Risks, and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD. Methods: We conducted a descriptive cohort study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11 th June 2020 and are iteratively updated via GitHub [4]. Findings: We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19 , and 113,627 hospitalized with COVID-19 requiring intensive services . All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts, and are available in an interactive website: https://data.ohdsi.org/Covid19CharacterizationCharybdis/. Interpretation: CHARYBDIS findings provide benchmarks that contribute to our understanding of COVID-19 progression, management and evolution over time. This can enable timely assessment of real-world outcomes of preventative and therapeutic options as they are introduced in clinical practice.