Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Transparency, reproducibility, and replicability of pharmacoepidemiology studies in a distributed network environment

PURPOSE

Our objective is to describe how the U.S. Food and Drug Administration (FDA)'s Sentinel System implements best practices to ensure trust in drug safety studies using real-world data from disparate sources.

METHODS

We present a stepwise schematic for Sentinel's data harmonization, data quality check, query design and implementation, and reporting practices, and describe approaches to enhancing the transparency, reproducibility, and replicability of studies at each step.

CONCLUSIONS

Each Sentinel data partner converts its source data into the Sentinel Common Data Model. The transformed data undergoes rigorous quality checks before it can be used for Sentinel queries. The Sentinel Common Data Model framework, data transformation codes for several data sources, and data quality assurance packages are publicly available. Designed to run against the Sentinel Common Data Model, Sentinel's querying system comprises a suite of pre-tested, parametrizable computer programs that allow users to perform sophisticated descriptive and inferential analysis without having to exchange individual-level data across sites. Detailed documentation of capabilities of the programs as well as the codes and information required to execute them are publicly available on the Sentinel website. Sentinel also provides public trainings and online resources to facilitate use of its data model and querying system. Its study specifications conform to established reporting frameworks aimed at facilitating reproducibility and replicability of real-world data studies. Reports from Sentinel queries and associated design and analytic specifications are available for download on the Sentinel website. Sentinel is an example of how real-world data can be used to generate regulatory-grade evidence at scale using a transparent, reproducible, and replicable process.

Metric identification and mapping of reimbursable implant codes in Morocco to the global nomenclature (GMDN) and European (EUDAMED) of medical devices

BACKGROUND

The list of medical devices (MD) eligible for reimbursement under the Compulsory Health Insurance in Morocco is set by Ministerial Order comprising 869 items between life-support equipment, external prostheses, and implants. The objective of the present study is to analyze the nomenclature of implantable medical devices (IMD) appearing on this list and compare them with the global nomenclature of MD (GMDN) and the European nomenclature of MD (EMDN).

METHODS

The study deals with (i) the mapping of the codes of the IMD list with 170 DM per cardinality and (ii) a metric identification by Sørensen-Dice coefficient of terminological similarity of the IMD with respect to the two databases.

RESULTS

The 170 IMD codes are mapped onto 493 terms in the GMDN and 344 terms in the EMDN. The 37.7% of implants are mapped to more than or equal to 2 terms of GMDN while 36.5% are mapped to more than or equal to 2 terms to the EMDN. The comparison of cardinality distributions has revealed no significant difference (P=0.430) between the two databases. The implants examined are divided into 11 categories whose strong similarity is given to active cardiovascular implants in the EMDN database with simDice=0.534.

CONCLUSION

Healthcare authorities need to align with nomenclature standards to improve interoperability and rely on a more efficient and rational regulatory process.

Using natural language processing to analyze unstructured patient-reported outcomes data derived from electronic health records for cancer populations: a systematic review

INTRODUCTION

Patient-reported outcomes (PROs; symptoms, functional status, quality-of-life) expressed in the 'free-text' or 'unstructured' format within clinical notes from electronic health records (EHRs) offer valuable insights beyond biological and clinical data for medical decision-making. However, a comprehensive assessment of utilizing natural language processing (NLP) coupled with machine learning (ML) methods to analyze unstructured PROs and their clinical implementation for individuals affected by cancer remains lacking.

AREAS COVERED

This study aimed to systematically review published studies that used NLP techniques to extract and analyze PROs in clinical narratives from EHRs for cancer populations. We examined the types of NLP (with and without ML) techniques and platforms for data processing, analysis, and clinical applications.

EXPERT OPINION

Utilizing NLP methods offers a valuable approach for processing and analyzing unstructured PROs among cancer patients and survivors. These techniques encompass a broad range of applications, such as extracting or recognizing PROs, categorizing, characterizing, or grouping PROs, predicting or stratifying risk for unfavorable clinical results, and evaluating connections between PROs and adverse clinical outcomes. The employment of NLP techniques is advantageous in converting substantial volumes of unstructured PRO data within EHRs into practical clinical utilities for individuals with cancer.

Antidiabetic Therapy during Pregnancy: The Prescription Pattern in Italy

Pregestational and gestational diabetes mellitus are relevant complications of pregnancy, and antidiabetic drugs are prescribed to obtain glycemic control and improve perinatal outcomes. The objective of this study was to describe the prescription pattern of antidiabetics before, during and after pregnancy in Italy and to evaluate its concordance with the Italian guideline on treatment of diabetes mellitus. A multi-database cross-sectional population study using a Common Data Model was performed. In a cohort of about 450,000 women, the prescribing profile of antidiabetics seemed to be in line with the Italian guideline, which currently does not recommend the use of oral antidiabetics and non-insulin injection, even if practice is still heterogeneous (up to 3.8% in the third trimester used oral antidiabetics). A substantial variability in the prescription pattern was observed among the Italian regions considered: the highest increase was registered in Tuscany (4.2%) while the lowest was in Lombardy (1.5%). Women with multiple births had a higher proportion of antidiabetic prescriptions than women with singleton births both in the preconception period and during pregnancy (1.3% vs. 0.7%; 3.4% vs. 2.6%) and used metformin more frequently. The consumption of antidiabetics in foreign women was higher than Italians (second trimester: 1.8% vs. 0.9%, third trimester: 3.6% vs. 1.8%).

External validity in distributed data networks

PURPOSE

While much has been written about how distributed networks address internal validity, external validity is rarely discussed. We aimed to define key terms related to external validity, discuss how they relate to distributed networks, and identify how three networks (the US Food and Drug Administration's Sentinel System, the Canadian Network for Observational Drug Effect Studies [CNODES], and the National Patient Centered Clinical Research Network [PCORnet]) deal with external validity.

METHODS

We define external validity, target populations, target validity, generalizability, and transportability and describe how each relates to distributed networks. We then describe Sentinel, CNODES, and PCORnet and how each approaches these concepts, including a sample case study.

RESULTS

Each network approaches external validity differently. As its target population is US citizens and it includes only US data, Sentinel primarily worries about lack of external validity by not including some segments of the population. The fact that CNODES includes Canadian, United States, and United Kingdom data forces them to seriously consider whether the United States and United Kingdom data will be transportable to Canadian citizens when they meta-analyze database-specific estimates. PCORnet, with its focus on study-specific cohorts and pragmatic trials, conducts more case-by-case explorations of external validity for each new analytic data set it generates.

CONCLUSIONS

There is no one-size-fits-all approach to external validity within distributed networks. With these networks and comparisons between their findings becoming a key part of pharmacoepidemiology, there is a need to adapt tools for improving external validity to the distributed network setting.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Hydrophilic and lipophilic statin use and risk of hearing loss in hyperlipidemia using a Common Data Model: multicenter cohort study

Hearing impairment, the third largest health burden worldwide, currently lacks definitive treatments or preventive drugs. This study compared the effects of hydrophilic and lipophilic statin on hearing loss using a common database model. This retrospective multicenter study was conducted in three hospitals in South Korea (Anam, Guro, Ansan). We enrolled patients with hyperlipidemia with an initial hearing loss diagnosis. Data were collected during January 1, 2022-December 31, 2021 using the Observational Health Data Science and Informatics open-source software and Common Data Model database. The primary outcome was the occurrence of first-time hearing loss following a hyperlipidemia diagnosis, as documented in the Common Data Model cohort database. The measures of interest were hearing loss risk between hydrophilic and lipophilic statin use. Variables were compared using propensity score matching, Cox proportional regression, and meta-analysis. Among 37,322 patients with hyperlipidemia, 13,751 (7669 men and 6082 women) and 23,631 (11,390 men and 12,241 women) were treated with hydrophilic and lipophilic statins, respectively. After propensity score matching, according to the Kaplan-Meier curve, hearing loss risk did not significantly differ among the hospitals. The hazard ratio (HR) of the male patients from Anam (0.29, [95% confidence interval (CI), 0.05-1.51]), Guro (HR, 0.56, [95% CI 0.18-1.71]), and Ansan (hazard ratio, 0.29, [95% CI 0.05-1.51]) hospitals were analyzed using Cox proportional regression. Overall effect size (HR, 0.40, [95% CI 0.18-0.91]) was estimated using meta-analysis, which indicated that hearing loss risk among hydrophilic statin users was less than that among lipophilic statin users and was statistically significant. Men in the hydrophilic statin group had a lower risk of hearing impairment than those in the lipophilic statin group.

A Multicenter Cohort Study on the Association between Metformin Use and Hearing Loss in Patients with Type 2 Diabetes Mellitus Using a Common Data Model

We attempted to explore the association between metformin use and hearing loss in in a large-scale study. This retrospective multicenter cohort study assessed the data of patients with type 2 diabetes mellitus (DM) aged over 40 years using the Observational Health Data Science and Informatics open-source software and the Common Data Model database from 1 January 2002 to 31 December 2019. Each participant was selected using the ICD-10-CM diagnosis code E11 for type 2 DM with sensorineural hearing loss. The participants were divided into metformin and non-metformin users. The outcome measure was the first occurrence of hearing loss after the diagnosis of DM as measured by the CDM cohort study. A total of 80,596 patients, including 46,152 metformin users and 34,444 non-metformin users from three hospitals were assessed. After calibration, we compared the risk of hearing loss using Kaplan-Meier curves, and found significant differences between the groups. The calibrated hazard ratio in the three hospitals (0.79 [95% confidence interval, 0.57-1.12]) was summarized. These findings suggest that the probability of hearing loss-free survival in the metformin user group is higher than that in the non-metformin user group.

Identifying new drugs associated with pulmonary arterial hypertension: A WHO pharmacovigilance database disproportionality analysis

Since the 1960s, several drugs have been linked to the onset or aggravation of pulmonary arterial hypertension (PAH): dasatinib, some amphetamine-like appetite suppressants (aminorex, fenfluramine, dexfenfluramine, benfluorex) and recreational drugs (methamphetamine). Moreover, in numerous cases, the implication of other drugs with PAH have been suggested, but the precise identification of iatrogenic aetiologies of PAH is challenging given the scarcity of this disease and the potential long latency period between drug intake and PAH onset. In this context, we used the World Health Organization's pharmacovigilance database, VigiBase, to generate new hypotheses about drug associated PAH.

METHODS

We used VigiBase, the largest pharmacovigilance database worldwide to generate disproportionality signals through the Bayesian neural network method. All disproportionality signals were further independently reviewed by experts in pulmonary arterial hypertension, pharmacovigilance and vascular pharmacology and their plausibility ranked according to World Health Organization causality categories.

RESULTS

We included 2184 idiopathic PAH cases, yielding a total of 93 disproportionality signals. Among them, 25 signals were considered very likely, 15 probable, 28 possible and 25 unlikely. Notably, we identified 4 new protein kinases inhibitors (lapatinib, lorlatinib, ponatinib and ruxolitinib), 1 angiogenesis inhibitor (bevacizumab), and several chemotherapeutics (etoposide, trastuzumab), antimetabolites (cytarabine, fludarabine, fluorouracil, gemcitabine) and immunosuppressants (leflunomide, thalidomide, ciclosporin).

CONCLUSION

Such signals represent plausible adverse drug reactions considering the knowledge of iatrogenic PAH, the drugs' biological and pharmacological activity and the characteristics of the reported case. Although confirmatory studies need to be performed, the signals identified may help clinicians envisage an iatrogenic aetiology when faced with a patient who develops PAH.

Integrating real-world data to accelerate and guide drug development: A clinical pharmacology perspective

Pharmaceutical products in the current accelerated drug development landscape can benefit from tools beyond data generated from randomized control trials. We have seen an abundance of real-world data (RWD) and real-world evidence, driven by the digitalization of healthcare systems and an increased awareness that has inspired a heightened interest in their potential use. Literature review suggest leveraging RWD as a promising tool to answer key questions in the areas of clinical pharmacology and translational science. RWD may increase our understanding regarding the impact of intrinsic (e.g., liver, renal impairment, or genetic polymorphisms) and extrinsic (e.g., food consumption or concomitant medications) factors on the clearance of administered drugs. Changes in clearance may lead to clinically relevant changes in drug exposure that may require clinical management strategies, such as change in dose or dosing regimen. RWD can be leveraged to potentially bridge the gaps among research, development, and clinical care. This paper highlights promising areas of how RWD have been used to complement clinical pharmacology throughout various phases of drug development; case examples will include dose/regimen extrapolation, dose adjustments for special populations (organ impairment, pediatrics, etc.), and pharmacokinetic/pharmacodynamic models to assess impact of prognostic factors on outcomes. In addition, this paper will also juxtapose limitations and promises of utilizing RWD to answer key scientific questions in drug development and articulate challenges posed by quality issues, data availability, and integration from various sources as well as the increased need for multidimensional-omics data that can better guide the development of personalized and predictive medicine.

Developing a common data model approach for DISCOVER CKD: A retrospective, global cohort of real-world patients with chronic kidney disease

Objectives

To describe a flexible common data model (CDM) approach that can be efficiently tailored to study-specific needs to facilitate pooled patient-level analysis and aggregated/meta-analysis of routinely collected retrospective patient data from disparate data sources; and to detail the application of this CDM approach to the DISCOVER CKD retrospective cohort, a longitudinal database of routinely collected (secondary) patient data of individuals with chronic kidney disease (CKD).

Methods

The flexible CDM approach incorporated three independent, exchangeable components that preceded data mapping and data model implementation: (1) standardized code lists (unifying medical events from different coding systems); (2) laboratory unit harmonization tables; and (3) base cohort definitions. Events between different coding vocabularies were not mapped code-to-code; for each data source, code lists of labels were curated at the entity/event level. A study team of epidemiologists, clinicians, informaticists, and data scientists were included within the validation of each component.

Results

Applying the CDM to the DISCOVER CKD retrospective cohort, secondary data from 1,857,593 patients with CKD were harmonized from five data sources, across three countries, into a discrete database for rapid real-world evidence generation.

Conclusions

This flexible CDM approach facilitates evidence generation from real-world data within the DISCOVER CKD retrospective cohort, providing novel insights into the epidemiology of CKD that may expedite improvements in diagnosis, prognosis, early intervention, and disease management. The adaptable architecture of this CDM approach ensures scalable, fast, and efficient application within other therapy areas to facilitate the combined analysis of different types of secondary data from multiple, heterogeneous sources.

Perceived Risk of Re-Identification in OMOP-CDM Database: A Cross-Sectional Survey

BACKGROUND

The advancement of information technology has immensely increased the quality and volume of health data. This has led to an increase in observational study, as well as to the threat of privacy invasion. Recently, a distributed research network based on the common data model (CDM) has emerged, enabling collaborative international medical research without sharing patient-level data. Although the CDM database for each institution is built inside a firewall, the risk of re-identification requires management. Hence, this study aims to elucidate the perceptions CDM users have towards CDM and risk management for re-identification.

METHODS

The survey, targeted to answer specific in-depth questions on CDM, was conducted from October to November 2020. We targeted well-experienced researchers who actively use CDM. Basic statistics (total number and percent) were computed for all covariates.

RESULTS

There were 33 valid respondents. Of these, 43.8% suggested additional anonymization was unnecessary beyond, "minimum cell count" policy, which obscures a cell with a value lower than certain number (usually 5) in shared results to minimize the liability of re-identification due to rare conditions. During extract-transform-load processes, 81.8% of respondents assumed structured data is under control from the risk of re-identification. However, respondents noted that date of birth and death were highly re-identifiable information. The majority of respondents (n = 22, 66.7%) conceded the possibility of identifier-contained unstructured data in the NOTE table.

CONCLUSION

Overall, CDM users generally attributed high reliability for privacy protection to the intrinsic nature of CDM. There was little demand for additional de-identification methods. However, unstructured data in the CDM were suspected to have risks. The necessity for a coordinating consortium to define and manage the re-identification risk of CDM was urged.

Broadening the reach of the FDA Sentinel system: A roadmap for integrating electronic health record data in a causal analysis framework

The Sentinel System is a major component of the United States Food and Drug Administration's (FDA) approach to active medical product safety surveillance. While Sentinel has historically relied on large quantities of health insurance claims data, leveraging longitudinal electronic health records (EHRs) that contain more detailed clinical information, as structured and unstructured features, may address some of the current gaps in capabilities. We identify key challenges when using EHR data to investigate medical product safety in a scalable and accelerated way, outline potential solutions, and describe the Sentinel Innovation Center's initiatives to put solutions into practice by expanding and strengthening the existing system with a query-ready, large-scale data infrastructure of linked EHR and claims data. We describe our initiatives in four strategic priority areas: (1) data infrastructure, (2) feature engineering, (3) causal inference, and (4) detection analytics, with the goal of incorporating emerging data science innovations to maximize the utility of EHR data for medical product safety surveillance.

From Inception to ConcePTION: Genesis of a Network to Support Better Monitoring and Communication of Medication Safety During Pregnancy and Breastfeeding

In 2019, the Innovative Medicines Initiative (IMI) funded the ConcePTION project-Building an ecosystem for better monitoring and communicating safety of medicines use in pregnancy and breastfeeding: validated and regulatory endorsed workflows for fast, optimised evidence generation-with the vision that there is a societal obligation to rapidly reduce uncertainty about the safety of medication use in pregnancy and breastfeeding. The present paper introduces the set of concepts used to describe the European data sources involved in the ConcePTION project and illustrates the ConcePTION Common Data Model (CDM), which serves as the keystone of the federated ConcePTION network. Based on data availability and content analysis of 21 European data sources, the ConcePTION CDM has been structured with six tables designed to capture data from routine healthcare, three tables for data from public health surveillance activities, three curated tables for derived data on population (e.g., observation time and mother-child linkage), plus four metadata tables. By its first anniversary, the ConcePTION CDM has enabled 13 data sources to run common scripts to contribute to major European projects, demonstrating its capacity to facilitate effective and transparent deployment of distributed analytics, and its potential to address questions about utilization, effectiveness, and safety of medicines in special populations, including during pregnancy and breastfeeding, and, more broadly, in the general population.

Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments

Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.

The Italian Network for Monitoring Medication Use During Pregnancy (MoM-Net): Experience and Perspectives

There is an acute need for research to acquire high-quality information on the use of medicines in pregnancy, both in terms of appropriateness and safety. For this purpose, the Italian Medicines Agency established a Network for Monitoring Medication use in pregnancy (MoM-Net) through the conduction of population-based studies using administrative data available at regional level. This paper aimed to describe the experiences and challenges within the network. MoM-Net currently involves eight regions and several experts from public and academic institutions. The first study conducted aimed to identify drug use before, during and after pregnancy investigating specific therapeutic categories, analysing regional variability and monitoring drug use in specific subpopulations (i.e. foreign women/multiple pregnancies). Aggregated demographic, clinical, and prescription data were analysed using a distributed network approach based on common data model. The study population included all women delivering during 2016–2018 in the participating regions (n = 449,012), and corresponding to 59% of deliveries in Italy. Seventy-three per cent of the cohort had at least one drug prescription during pregnancy, compared to 57% before and 59% after pregnancy. In general, a good adherence to guidelines for pregnant women was found although some drug categories at risk of inappropriateness, such as progestins and antibiotics, were prescribed. A strong variability in the use of drugs among regions and in specific subpopulations was observed. The MoM-Net represents a valuable surveillance system on the use of medicines in pregnancy, available to monitor drug categories at high risk of inappropriateness and to investigate health needs in specific regions or subpopulations.

Using and improving distributed data networks to generate actionable evidence: the case of real-world outcomes in the Food and Drug Administration's Sentinel system

The US Food and Drug Administration (FDA) Sentinel System uses a distributed data network, a common data model, curated real-world data, and distributed analytic tools to generate evidence for FDA decision-making. Sentinel system needs include analytic flexibility, transparency, and reproducibility while protecting patient privacy. Based on over a decade of experience, a critical system limitation is the inability to identify enough medical conditions of interest in observational data to a satisfactory level of accuracy. Improving the system's ability to use computable phenotypes will require an "all of the above" approach that improves use of electronic health data while incorporating the growing array of complementary electronic health record data sources. FDA recently funded a Sentinel System Innovation Center and a Community Building and Outreach Center that will provide a platform for collaboration across disciplines to promote better use of real-world data for decision-making.

National health information technology priorities for research: A policy and development agenda

The growth of digitized health data presents exciting opportunities to leverage the health information technology (IT) infrastructure for advancing biomedical and health services research. However, challenges impede use of those resources effectively and at scale to improve outcomes. The Office of the National Coordinator for Health Information Technology (ONC) led a collaborative effort to identify challenges, priorities, and actions to leverage health IT and electronic health data for research. Specifically, ONC led a review of relevant literature and programs, key informant interviews, and a stakeholder workshop to identify electronic health data and health IT infrastructure gaps. This effort resulted in the National Health IT Priorities for Research: A Policy and Development Agenda, which articulates an optimized health information ecosystem for scientific discovery. This article outlines 9 priorities and recommended actions to be implemented in collaboration with the research and informatics communities for realizing this vision.

Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment

There is growing interest in using observational data to assess the safety, effectiveness, and cost effectiveness of medical technologies, but operational, technical, and methodological challenges limit its more widespread use. Common data models and federated data networks offer a potential solution to many of these problems. The open-source Observational and Medical Outcomes Partnerships (OMOP) common data model standardises the structure, format, and terminologies of otherwise disparate datasets, enabling the execution of common analytical code across a federated data network in which only code and aggregate results are shared. While common data models are increasingly used in regulatory decision making, relatively little attention has been given to their use in health technology assessment (HTA). We show that the common data model has the potential to facilitate access to relevant data, enable multidatabase studies to enhance statistical power and transfer results across populations and settings to meet the needs of local HTA decision makers, and validate findings. The use of open-source and standardised analytics improves transparency and reduces coding errors, thereby increasing confidence in the results. Further engagement from the HTA community is required to inform the appropriate standards for mapping data to the common data model and to design tools that can support evidence generation and decision making.

Real-world reproducibility study characterizing patients newly diagnosed with multiple myeloma using Clinical Practice Research Datalink, a UK-based electronic health records database

Purpose

We evaluated the reproducibility of a study characterizing newly‐diagnosed multiple myeloma (MM) patients within an electronic health records (EHR) database using different analytic tools.

Methods

We reproduced the findings of a descriptive cohort study using an iterative two‐phase approach. In Phase I, a common protocol and statistical analysis plan (SAP) were implemented by independent investigators using the Aetion Evidence Platform® (AEP), a rapid‐cycle analytics tool, and SAS statistical software as a gold standard for statistical analyses. Using the UK Clinical Practice Research Datalink (CPRD) dataset, the study included patients newly diagnosed with MM within primary care setting and assessed baseline demographics, conditions, drug exposure, and laboratory procedures. Phase II incorporated analysis revisions based on our initial comparison of the Phase I findings. Reproducibility of findings was evaluate by calculating the match rate and absolute difference in prevalence between the SAS and AEP study results.

Results

Phase I yielded slightly discrepant results, prompting amendments to SAP to add more clarity to operational decisions. After detailed specification of data and operational choices, exact concordance was achieved for the number of eligible patients (N = 2646), demographics, comorbidities (i.e., osteopenia, osteoporosis, cardiovascular disease [CVD], and hypertension), bone pain, skeletal‐related events, drug exposure, and laboratory investigations in the Phase II analyses.

Conclusions

In this reproducibility study, a rapid‐cycle analytics tool and traditional statistical software achieved near‐exact findings after detailed specification of data and operational choices. Transparency and communication of the study design, operational and analytical choices between independent investigators were critical to achieve this reproducibility.

Different Strategies to Execute Multi-Database Studies for Medicines Surveillance in Real-World Setting: A Reflection on the European Model

Although postmarketing studies conducted in population‐based databases often contain information on patients in the order of millions, they can still be underpowered if outcomes or exposure of interest is rare, or the interest is in subgroup effects. Combining several databases might provide the statistical power needed. A multi‐database study (MDS) uses at least two healthcare databases, which are not linked with each other at an individual person level, with analyses carried out in parallel across each database applying a common study protocol. Although many MDSs have been performed in Europe in the past 10 years, there is a lack of clarity on the peculiarities and implications of the existing strategies to conduct them. In this review, we identify four strategies to execute MDSs, classified according to specific choices in the execution: (A) local analyses, where data are extracted and analyzed locally, with programs developed by each site; (B) sharing of raw data, where raw data are locally extracted and transferred without analysis to a central partner, where all the data are pooled and analyzed; (C) use of a common data model with study‐specific data, where study‐specific data are locally extracted, loaded into a common data model, and processed locally with centrally developed programs; and (D) use of general common data model, where all local data are extracted and loaded into a common data model, prior to and independent of any study protocol, and protocols are incorporated in centrally developed programs that run locally. We illustrate differences between strategies and analyze potential implications.

Can We Rely on Results From IQVIA Medical Research Data UK Converted to the Observational Medical Outcome Partnership Common Data Model?: A Validation Study Based on Prescribing Codeine in Children

Exploring and combining results from more than one real‐world data (RWD) source might be necessary in order to explore variability and demonstrate generalizability of the results or for regulatory requirements. However, the heterogeneous nature of RWD poses challenges when working with more than one source, some of which can be solved by analyzing databases converted into a common data model (CDM). The main objective of the study was to evaluate the implementation of the Observational Medical Outcome Partnership (OMOP) CDM on IQVIA Medical Research Data (IMRD)‐UK data. A drug utilization study describing the prescribing of codeine for pain in children was used as a case study to be replicated in IMRD‐UK and its corresponding OMOP CDM transformation. Differences between IMRD‐UK source and OMOP CDM were identified and investigated. In IMRD‐UK updated to May 2017, results were similar between source and transformed data with few discrepancies. These were the result of different conventions applied during the transformation regarding the date of birth for children younger than 15 years and the start of the observation period, and of a misclassification of two drug treatments. After the initial analysis and feedback provided, a rerun of the analysis in IMRD‐UK updated to September 2018 showed almost identical results for all the measures analyzed. For this study, the conversion to OMOP CDM was adequate. Although some decisions and mapping could be improved, these impacted on the absolute results but not on the study inferences. This validation study supports six recommendations for good practice in transforming to CDMs.