Strengthening standards, transparency, and collaboration to support medicine evaluation: Ten years of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP)

Building transparency and reproducibility into the practice of pharmacoepidemiology and outcomes research

Real-world evidence (RWE) studies are increasingly used to inform policy and clinical decisions. However, there remain concerns about the credibility and reproducibility of RWE studies. While there is universal agreement on the critical importance of transparent and reproducible science, the building blocks of open science practice that are common across many disciplines have not yet been built into routine workflows for pharmacoepidemiology and outcomes researchers. Observational researchers should highlight the level of transparency of their studies by providing a succinct statement addressing study transparency with the publication of every paper, poster, or presentation that reports on an RWE study. In this paper, we propose a framework for an explicit transparency statement that declares the level of transparency a given RWE study has achieved across 5 key domains: (1) protocol, (2) preregistration, (3) data, (4) code-sharing, and (5) reporting checklists. The transparency statement outlined in the present paper can be used by research teams to proudly display the open science practices that were used to generate evidence designed to inform public health policy and practice. While transparency does not guarantee validity, such a statement signals confidence from the research team in the scientific choices that were made.

Pharmacoepidemiology: An Overview

The aims of this review are to provide a comprehensive overview of the definition and scope of pharmacoepidemiology, to summarize the study designs and methodologies used in the field, to discuss the future trends in the field and new methodologies to address bias and confounding, and finally to give some recommendations to clinicians interested in pharmacoepidemiologic research. Because drug efficacy and safety from randomized clinical trials do not reflect the real-world situation, pharmacoepidemiological studies on drug safety monitoring and drug effectiveness in large numbers of people are needed by healthcare professionals and regulatory institutions. We aim to highlight the importance of pharmacoepidemiologic research in informing evidence-based medicine and public health policy. The development of new designs and methodologies for the generation of valid evidence, as well as new initiatives to provide guidance and recommendations on how to incorporate real-world evidence into the drug development process, are reported on. In addition, we have touched on the implication of artificial intelligence in the management of real-world data. This overview aims to summarize all important aspects to consider when conducting or interpreting a pharmacoepidemiologic study.

Reproducible variability: assessing investigator discordance across 9 research teams attempting to reproduce the same observational study

OBJECTIVE

Observational studies can impact patient care but must be robust and reproducible. Nonreproducibility is primarily caused by unclear reporting of design choices and analytic procedures. This study aimed to: (1) assess how the study logic described in an observational study could be interpreted by independent researchers and (2) quantify the impact of interpretations' variability on patient characteristics.

MATERIALS AND METHODS

Nine teams of highly qualified researchers reproduced a cohort from a study by Albogami et al. The teams were provided the clinical codes and access to the tools to create cohort definitions such that the only variable part was their logic choices. We executed teams' cohort definitions against the database and compared the number of subjects, patient overlap, and patient characteristics.

RESULTS

On average, the teams' interpretations fully aligned with the master implementation in 4 out of 10 inclusion criteria with at least 4 deviations per team. Cohorts' size varied from one-third of the master cohort size to 10 times the cohort size (2159-63 619 subjects compared to 6196 subjects). Median agreement was 9.4% (interquartile range 15.3-16.2%). The teams' cohorts significantly differed from the master implementation by at least 2 baseline characteristics, and most of the teams differed by at least 5.

CONCLUSIONS

Independent research teams attempting to reproduce the study based on its free-text description alone produce different implementations that vary in the population size and composition. Sharing analytical code supported by a common data model and open-source tools allows reproducing a study unambiguously thereby preserving initial design choices.

Use of Structured Template and Reporting Tool for Real-World Evidence for Critical Appraisal of the Quality of Reporting of Real-World Evidence Studies: A Systematic Review

OBJECTIVES

Real-world evidence (RWE) studies are increasingly being used to support healthcare decisions. Various frameworks, tools, and checklists exist for ensuring quality of real-world data, designing robust studies, and assessing potential for bias. In January 2021, Structured Template and Reporting Tool for RWE (STaRT-RWE) was released to further reduce ambiguity, assumptions, and misinterpretation while planning, implementing, and reporting RWE studies of the safety and effectiveness of treatments. The objective of this study was to identify gaps in the reporting quality of published RWE studies by using this template for critical appraisal.

METHODS

Two reviewers conducted a keyword search on PubMed for free-full-text research articles using real-world data, RWE design, and safety with or without effectiveness outcomes of a medicinal product or intervention in humans of any age or gender, published in English between January 13, 2021, and January 13, 2022. Assessment of risk of bias was done using Assessment of Real-World Observational Studies critical appraisal tool. Deficiencies in methods and findings as per STaRT-RWE template were reported as frequencies.

RESULTS

A total of 54 of 2374 retrieved studies were included in the review. Based on the STaRT-RWE template, the studies inadequately reported empirically defined covariates, power and sample size calculation, attrition, sensitivity analyses, index date (day 0) defining criterion, predefined covariates, outcome, metadata about data source and software, objective, inclusion and exclusion criteria, analysis specifications, and follow-up.

CONCLUSIONS

The use of STaRT-RWE template along with its tables, design diagram, and library of published studies has a potential of improving robustness of RWE studies.

Safety of biological and targeted synthetic disease-modifying antirheumatic drugs for rheumatoid arthritis as used in clinical practice: results from the ARTIS programme

OBJECTIVE

Longitudinal clinical registry-infrastructures such as Anti-Rheumatic Therapies in Sweden (ARTIS) allow simultaneous comparison of the safety of individual immunomodulatory drugs used in clinical practice, with consistent definitions of treatment cohorts, follow-up and outcomes. Our objective was to assess and compare incidence rates of key safety outcomes for individual targeted synthetic or biological disease-modifying antirheumatic drugs (b/ts DMARDs) in rheumatoid arthritis (RA), updating previous reports and including newer treatments including Janus Kinase inhibitors (JAKi).

METHODS

Nationwide register-based cohort study including all patients with RA in Sweden registered as starting any b/tsDMARD 1 January 2010 through 31 December 2020, followed until 30 June 2021 (N=20 117). The incidence rates of selected outcomes, identified through national healthcare registers, were compared between individual b/tsDMARDs, adjusted for confounding by demographics, RA disease characteristics and comorbidity.

RESULTS

There were marked differences in treatment discontinuations due to adverse events (rates per 1000 person-years ranged from 18 on rituximab to 57 on tofacitinib), but few significant differences were observed for the serious adverse events under study. Neither cardiovascular events nor general serious infections were more frequent on baricitinib or tofacitinib versus bDMARDs, but JAKi were associated with higher rates of hospital-treated herpes zoster (HR vs etanercept, 3.82 (95% CI 2.05 to 7.09) and 4.00 (1.59 to 10.06)). Low number of events limited some comparisons, in particular for sarilumab and tofacitinib.

CONCLUSION

Data from ARTIS supports that the b/tsDMARDs currently used to treat RA have acceptable and largely similar safety profiles, but differences exist in particular concerning tolerability and specific infection risks.

HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force

PROBLEM

Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias.

WHAT WE DID

The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The overarching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility.

STRATEGIES TO DISSEMINATE AND FACILITATE USE

Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years.

CONCLUSION

The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

HARmonized Protocol Template to Enhance Reproducibility of Hypothesis Evaluating Real-World Evidence Studies on Treatment Effects: A Good Practices Report of a Joint ISPE/ISPOR Task Force

OBJECTIVES

Ambiguity in communication of key study parameters limits the utility of real-world evidence (RWE) studies in healthcare decision-making. Clear communication about data provenance, design, analysis, and implementation is needed. This would facilitate reproducibility, replication in independent data, and assessment of potential sources of bias.

METHODS

The International Society for Pharmacoepidemiology (ISPE) and ISPOR-The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a joint task force, including representation from key international stakeholders, to create a harmonized protocol template for RWE studies that evaluate a treatment effect and are intended to inform decision-making. The template builds on existing efforts to improve transparency and incorporates recent insights regarding the level of detail needed to enable RWE study reproducibility. The over-arching principle was to reach for sufficient clarity regarding data, design, analysis, and implementation to achieve 3 main goals. One, to help investigators thoroughly consider, then document their choices and rationale for key study parameters that define the causal question (e.g., target estimand), two, to facilitate decision-making by enabling reviewers to readily assess potential for biases related to these choices, and three, to facilitate reproducibility.

STRATEGIES TO DISSEMINATE AND FACILITATE USE

Recognizing that the impact of this harmonized template relies on uptake, we have outlined a plan to introduce and pilot the template with key international stakeholders over the next 2 years.

CONCLUSION

The HARmonized Protocol Template to Enhance Reproducibility (HARPER) helps to create a shared understanding of intended scientific decisions through a common text, tabular and visual structure. The template provides a set of core recommendations for clear and reproducible RWE study protocols and is intended to be used as a backbone throughout the research process from developing a valid study protocol, to registration, through implementation and reporting on those implementation decisions.

Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions

Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson's correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratiooriginal/hazard ratioreproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.

Characterization of Asthma by Age of Onset: A Multi-Database Cohort Study

BACKGROUND

Asthma can occur at any age but the differences in patient characteristics between childhood-, adult-, and late-onset asthma are not well understood.

OBJECTIVE

To investigate differences in patients' characteristics by age at asthma onset.

METHODS

From 5 European electronic databases, we created a cohort encompassing adult patients with doctor-diagnosed asthma in 2008 to 2013. Patients were categorized based on their age at asthma onset: childhood-onset (age at onset < 18 y), adult-onset (age at onset 18-40 y), and late-onset asthma (age at onset ≥ 40 y). Comorbidities were assessed at study entry. For each characteristic and comorbidity, odds ratios and age- and sex-adjusted odds ratios (OR_adj) comparing asthma-onset categories were estimated per database and combined in a meta-analysis using a random effect model.

RESULTS

In total, 586,436 adult asthma patients were included, 81,691 had childhood-onset, 218,184 adult-onset, and 286,561 late-onset asthma. Overall, 7.3% had severe asthma. Subjects with adult-onset compared with childhood-asthma had higher risks for overweight/obesity (OR_adj 1.4; 95% CI 1.1-1.8) and lower risks for atopic disorders (OR_adj 0.8; 95% CI 0.7-0.95). Patients with late-onset compared with adult-onset asthma had higher risks for nasal polyposis (OR_adj 1.8; 95% CI 1.2-2.6), overweight/obesity (OR_adj 1.3; 95% CI 1.2-1.4), gastroesophageal reflux disease (OR_adj 1.4; 95% CI 1.2-1.7), and diabetes (OR_adj 2.3; 95% CI 1.8-2.9). A significant association between late-onset asthma and uncontrolled asthma was observed (OR_adj 2.8; 95% CI 1.7-4.5).

CONCLUSIONS

This international study demonstrates clear differences in comorbidities between childhood-, adult-, and late-onset asthma phenotypes in adults. Furthermore, patients with late-onset asthma had more frequent uncontrolled asthma.

The Multiple Sclerosis Data Alliance Catalogue: Enabling Web-Based Discovery of Metadata from Real-World Multiple Sclerosis Data Sources

Background

One of the major objectives of the Multiple Sclerosis Data Alliance (MSDA) is to enable better discovery of multiple sclerosis (MS) real-world data (RWD).

Methods

We implemented the MSDA Catalogue, which is available worldwide. The current version of the MSDA Catalogue collects descriptive information on governance, purpose, inclusion criteria, procedures for data quality control, and how and which data are collected, including the use of e-health technologies and data on collection of COVID-19 variables. The current cataloguing procedure is performed in several manual steps, securing an effective catalogue.

Results

Herein we summarize the status of the MSDA Catalogue as of January 6, 2021. To date, 38 data sources across five continents are included in the MSDA Catalogue. These data sources differ in purpose, maturity, and variables collected, but this landscaping effort shows that there is substantial alignment on some domains. The MSDA Catalogue shows that personal data and basic disease data are the most collected categories of variables, whereas data on fatigue measurements and cognition scales are the least collected in MS registries/cohorts.

Conclusions

The Web-based MSDA Catalogue provides strategic overview and allows authorized end users to browse metadata profiles of data cohorts and data sources. There are many existing and arising RWD sources in MS. Detailed cataloguing of MS RWD is a first and useful step toward reducing the time needed to discover MS RWD sets and promoting collaboration.

Learning from disease registries during a pandemic: Moving toward an international federation of patient registries

High-quality dermatology patient registries often require considerable time to develop and produce meaningful data. Development time is influenced by registry complexity and regulatory hurdles that vary significantly nationally and institutionally. The rapid emergence of the coronavirus disease 2019 (COVID-19) global pandemic has challenged health services in an unprecedented manner. Mobilization of the dermatology community in response has included rapid development and deployment of multiple, partially harmonized, international patient registries, reinventing established patient registry timelines. Partnership with patient organizations has demonstrated the critical nature of inclusive patient involvement. This global effort has demonstrated the value, capacity, and necessity for the dermatology community to adopt a more cohesive approach to patient registry development and data sharing that can lead to myriad benefits. These include improved utilization of limited resources, increased data interoperability, improved ability to rapidly collect meaningful data, and shortened response times to generate real-world evidence. We call on the global dermatology community to support the development of an international federation of patient registries to consolidate and operationalize the lessons learned during this pandemic. This will provide an enduring means of applying this knowledge to the maintenance and development of sustainable, coherent, and impactful patient registries of benefit now and in the future.

A review of the sampling methodology used in studies evaluating the effectiveness of risk minimisation measures in Europe

Purpose

This review aims to describe the sampling methodology used in studies assessing effectiveness of risk minimisation measures (RMMs) in the European Union.

Methods

The European Union electronic Register of Post‐Authorization Studies (EU PAS Register) was searched to identify studies that assessed the effectiveness of RMMs and recruited a target population of healthcare professionals (HCPs), sites or patients. Studies with both protocol and report were included and data was extracted from these documents to describe study characteristics and variables involved in the sampling methodology.

Results

Out of 1092 studies finalised between June 2017 and May 2019, 17 studies were eligible for review. Thirteen were surveys, three chart reviews and one combined both methodologies. All the 17 studies recruited HCPs/sites and 8 of them also recruited patients. The most common rationale for country sampling was market uptake (10/17), while for HCP/site sampling, it was representativeness of the prescribing practices (14/17). Only a minority of the studies (4/17) provided supporting evidence to inform this theoretical framework. HCP/site sampling frames were mainly network of physicians (5/17) or HCP databases (5/17), with only one study providing a detailed description of the sampling frame. HCPs were selected mainly using probabilistic sampling (10/17) and patients using non‐probabilistic sampling (6/8). Only a few studies compared participating with non‐participating HCPs/sites (5/17) and patients (3/8). Eight studies reported that their results were generalisable.

Conclusions

Overall, the study documents provided insufficient details to understand the rationale behind the sampling decisions. More standardisation and guidance in reporting the sampling strategy and operational considerations applicable to these types of studies would support transparency and facilitate the evaluation of representativeness of the study results.

Integrating Real-World Evidence in the Regulatory Decision-Making Process: A Systematic Analysis of Experiences in the US, EU, and China Using a Logic Model

Real world evidence (RWE) and real-world data (RWD) are drawing ever-increasing attention in the pharmaceutical industry and drug regulatory authorities (DRAs) all over the world due to their paramount role in supporting drug development and regulatory decision making. However, there is little systematic documentary analysis about how RWE was integrated for the use by the DRAs in evaluating new treatment approaches and monitoring post-market safety. This study aimed to analyze and discuss the integration of RWE into regulatory decision-making process from the perspective of DRAs. Different development strategies to develop and adopt RWE by the DRAs in the US, Europe, and China were reviewed and compared, and the challenges encountered were discussed. It was found that different strategies on development of RWE were applied by FDA, EMA, and NMPA. The extent to which RWE was adopted in China was relatively limited compared to that in the US and EU, which was highly related to the national pharmaceutical environment and development stages. A better understanding of the overall goals, inputs, activities, outputs, and outcomes in developing RWE will help inform actions to harness RWD and leverage RWE for better health care decisions.

Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment

There is growing interest in using observational data to assess the safety, effectiveness, and cost effectiveness of medical technologies, but operational, technical, and methodological challenges limit its more widespread use. Common data models and federated data networks offer a potential solution to many of these problems. The open-source Observational and Medical Outcomes Partnerships (OMOP) common data model standardises the structure, format, and terminologies of otherwise disparate datasets, enabling the execution of common analytical code across a federated data network in which only code and aggregate results are shared. While common data models are increasingly used in regulatory decision making, relatively little attention has been given to their use in health technology assessment (HTA). We show that the common data model has the potential to facilitate access to relevant data, enable multidatabase studies to enhance statistical power and transfer results across populations and settings to meet the needs of local HTA decision makers, and validate findings. The use of open-source and standardised analytics improves transparency and reduces coding errors, thereby increasing confidence in the results. Further engagement from the HTA community is required to inform the appropriate standards for mapping data to the common data model and to design tools that can support evidence generation and decision making.

Improving transparency to build trust in real-world secondary data studies for hypothesis testing-Why, what, and how: recommendations and a road map from the real-world evidence transparency initiative

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Using real-world evidence for pharmacovigilance and drug safety-related decision making by a resource-limited health authority: 10 years of experience in Taiwan

PURPOSE

Real-world evidence has become increasingly relevant in regulatory decision making. Compared to large regulatory bodies, the national pharmacovigilance system in Taiwan is still under development, and the aim of this study is to demonstrate how a resource-limited health authority utilizes real-world evidence in decision making.

METHODS

We described different sources of real-world data available in Taiwan and illustrated the structural framework that integrates real-world evidence into Taiwan's national pharmacovigilance system. Additionally, we reviewed real-world studies conducted in the past 10 years and provided examples to show how these studies influenced drug safety-related decision making in Taiwan.

RESULTS

During the past 10 years, real-world evidence used when making drug safety-related regulatory decisions in Taiwan was mainly generated from nationwide claims databases, but other sources of real-world data, such as national registries and large electronic hospital databases, also became available recently. Different types of real-world evidence, including drug utilization studies, risk evaluation studies, and risk minimization measure evaluation studies, have been used to support regulatory decisions in Taiwan.

CONCLUSIONS

Through collaborations between the government and academics, Taiwan has started to integrate real-world evidence into the national pharmacovigilance system. However, future efforts, including linkages between different sources of real-world data and improvements in procedural and methodological practices, are needed to generate more regulatory-quality real-world evidence.

Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Existing Data Sources in Clinical Epidemiology: The Danish COVID-19 Cohort

Background

To facilitate research on severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a prospective cohort of all Danish residents tested for SARS-CoV-2 in Denmark is established.

Data Structure

All Danish residents tested by reverse transcriptase polymerase chain reactions (RT-PCR) for SARS-CoV-2 in Denmark are included. The cohort is identified using the Danish Microbiology Database. Individual-level record linkage between administrative and health-care registries is facilitated by the Danish Civil Registration System. Information on outcomes related to SARS-CoV-2 infection includes hospital admission, intensive care unit admission, mechanical ventilation, and death and is retrieved from the five administrative Danish regions, the Danish National Patient Registry, and the Danish Register of Causes of Death. The Patient Registry further provides a complete hospital contact history of somatic and psychiatric conditions and procedures. Data on all prescriptions filled at community pharmacies are available from the Danish National Prescription Registry. Health-care authorization status is obtained from the Danish Register of Healthcare Professionals. Finally, selected laboratory values are obtained from the Register of Laboratory Results for Research. The cohort is governed by a steering committee with representatives from the Danish Medicines Agency, Statens Serum Institut, the Danish Health Authority, the Danish Health Data Authority, Danish Patients, the Faculties of Health Sciences at the Danish universities, and Danish regions. The steering committee welcomes suggestions for research studies and collaborations. Research proposals will be prioritized based on timeliness and potential clinical and public health implications. All research protocols assessing specific hypotheses for medicines will be made publicly available using the European Union electronic Register of Post-Authorisation Studies.

Conclusion

The Danish COVID-19 cohort includes all Danish residents with an RT-PCR test for SARS-CoV-2. Through individual-level linkage with existing Danish health and administrative registries, this is a valuable data source for epidemiological research on SARS-CoV-2.

Screening Medications for Association with Progression to Wet Age-Related Macular Degeneration

PURPOSE

There is an urgent need for treatments that prevent or delay development to advanced age-related macular degeneration (AMD). Drugs already on the market for other conditions could affect progression to neovascular AMD (nAMD). If identified, these drugs could provide insights for drug development targets. The objective of this study was to use a novel data mining method that can simultaneously evaluate thousands of correlated hypotheses, while adjusting for multiple testing, to screen for associations between drugs and delayed progression to nAMD.

DESIGN

We applied a nested case-control study to administrative insurance claims data to identify cases with nAMD and risk-set sampled controls that were 1:4 variable ratio matched on age, gender, and recent healthcare use.

PARTICIPANTS

The study population included cases with nAMD and risk set matched controls.

METHODS

We used a tree-based scanning method to evaluate associations between hierarchical classifications of drugs that patients were exposed to within 6 months, 7 to 24 months, or ever before their index date. The index date was the date of first nAMD diagnosis in cases. Risk-set sampled controls were assigned the same index date as the case to which they were matched. The study was implemented using Medicare data from New Jersey and Pennsylvania, and national data from IBM MarketScan Research Database. We set an a priori threshold for statistical alerting at P ≤ 0.01 and focused on associations with large magnitude (relative risks ≥ 2.0).

MAIN OUTCOME MEASURES

Progression to nAMD.

RESULTS

Of approximately 4000 generic drugs and drug classes evaluated, the method detected 19 distinct drug exposures with statistically significant, large relative risks indicating that cases were less frequently exposed than controls. These included (1) drugs with prior evidence for a causal relationship (e.g., megestrol); (2) drugs without prior evidence for a causal relationship, but potentially worth further exploration (e.g., donepezil, epoetin alfa); (3) drugs with alternative biologic explanations for the association (e.g., sevelamer); and (4) drugs that may have resulted in statistical alerts due to their correlation with drugs that alerted for other reasons.

CONCLUSIONS

This exploratory drug-screening study identified several potential targets for follow-up studies to further evaluate and determine if they may prevent or delay progression to advanced AMD.

Different Strategies to Execute Multi-Database Studies for Medicines Surveillance in Real-World Setting: A Reflection on the European Model

Although postmarketing studies conducted in population-based databases often contain information on patients in the order of millions, they can still be underpowered if outcomes or exposure of interest is rare, or the interest is in subgroup effects. Combining several databases might provide the statistical power needed. A multi-database study (MDS) uses at least two healthcare databases, which are not linked with each other at an individual person level, with analyses carried out in parallel across each database applying a common study protocol. Although many MDSs have been performed in Europe in the past 10 years, there is a lack of clarity on the peculiarities and implications of the existing strategies to conduct them. In this review, we identify four strategies to execute MDSs, classified according to specific choices in the execution: (A) local analyses, where data are extracted and analyzed locally, with programs developed by each site; (B) sharing of raw data, where raw data are locally extracted and transferred without analysis to a central partner, where all the data are pooled and analyzed; (C) use of a common data model with study-specific data, where study-specific data are locally extracted, loaded into a common data model, and processed locally with centrally developed programs; and (D) use of general common data model, where all local data are extracted and loaded into a common data model, prior to and independent of any study protocol, and protocols are incorporated in centrally developed programs that run locally. We illustrate differences between strategies and analyze potential implications.

Are Novel, Nonrandomized Analytic Methods Fit for Decision Making? The Need for Prospective, Controlled, and Transparent Validation

Real-world data and patient-level data from completed randomized controlled trials are becoming available for secondary analysis on an unprecedented scale. A range of novel methodologies and study designs have been proposed for their analysis or combination. However, to make novel analytical methods acceptable for regulators and other decision makers will require their testing and validation in broadly the same way one would evaluate a new drug: prospectively, well-controlled, and according to a pre-agreed plan. From a European regulators' perspective, the established methods qualification advice procedure with active participation of patient groups and other decision makers is an efficient and transparent platform for the development and validation of novel study designs.

Generating comparative evidence on new drugs and devices after approval

Certain limitations of evidence available on drugs and devices at the time of market approval often persist in the post-marketing period. Often, post-marketing research landscape is fragmented. When regulatory agencies require pharmaceutical and device manufacturers to conduct studies in the post-marketing period, these studies might remain incomplete many years after approval. Even when completed, many post-marketing studies lack meaningful active comparators, have observational designs, and might not collect patient-relevant outcomes. Regulators, in collaboration with the industry and patients, ought to ensure that the key questions unanswered at the time of drug and device approval are resolved in a timely fashion during the post-marketing phase. We propose a set of seven key guiding principles that we believe will provide the necessary incentives for pharmaceutical and device manufacturers to generate comparative data in the post-marketing period. First, regulators (for drugs and devices), notified bodies (for devices in Europe), health technology assessment organisations, and payers should develop customised evidence generation plans, ensuring that future post-approval studies address any limitations of the data available at the time of market entry impacting the benefit-risk profiles of drugs and devices. Second, post-marketing studies should be designed hierarchically: priority should be given to efforts aimed at evaluating a product's net clinical benefit in randomised trials compared with current known effective therapy, whenever possible, to address common decisional dilemmas. Third, post-marketing studies should incorporate active comparators as appropriate. Fourth, use of non-randomised studies for the evaluation of clinical benefit in the post-marketing period should be limited to instances when the magnitude of effect is deemed to be large or when it is possible to reasonably infer the comparative benefits or risks in settings, in which doing a randomised trial is not feasible. Fifth, efficiency of randomised trials should be improved by streamlining patient recruitment and data collection through innovative design elements. Sixth, governments should directly support and facilitate the production of comparative post-marketing data by investing in the development of collaborative research networks and data systems that reduce the complexity, cost, and waste of rigorous post-marketing research efforts. Last, financial incentives and penalties should be developed or more actively reinforced.

Data Rich, Information Poor: Can We Use Electronic Health Records to Create a Learning Healthcare System for Pharmaceuticals?

Judicious use of real-world data (RWD) is expected to make all steps in the development and use of pharmaceuticals more effective and efficient, including research and development, regulatory decision making, health technology assessment, pricing, and reimbursement decisions and treatment. A "learning healthcare system" based on electronic health records and other routinely collected data will be required to harness the full potential of RWD to complement evidence based on randomized controlled trials. We describe and illustrate with examples the growing demand for a learning healthcare system; we contrast the exigencies of an efficient pharmaceutical ecosystem in the future with current deficiencies highlighted in recently published Organisation for Economic Co-operation and Development (OECD) reports; and we reflect on the steps necessary to enable the transition from healthcare data to actionable information. A coordinated effort from all stakeholders and international cooperation will be required to increase the speed of implementation of the learning healthcare system, to everybody's benefit.

The ENCePP Code of Conduct: A best practise for scientific independence and transparency in noninterventional postauthorisation studies

PURPOSE

The ENCePP Code of Conduct provides a framework for scientifically independent and transparent pharmacoepidemiological research. Despite becoming a landmark reference, practical implementation of key provisions was still limited. The fourth revision defines scientific independence and clarifies uncertainties on the applicability to postauthorisation safety studies requested by regulators. To separate the influence of the funder from the investigator's scientific responsibility, the Code now requires that the lead investigator is not employed by the funding institution.

METHOD

To assess how the revised Code fits the ecosystem of noninterventional pharmacoepidemiology research in Europe, we first mapped key recommendations of the revised Code against ISPE Good Pharmacoepidemiology Practices and the ADVANCE Code of Conduct. We surveyed stakeholders to understand perceptions on its value and practical applicability. Representatives from the different stakeholders' groups described their experience and expectations.

RESULTS

Unmet needs in pharmacoepidemiological research are fulfilled by providing unique guidance on roles and responsibilities to support scientific independence. The principles of scientific independence and transparency are well understood and reinforce trust in study results; however, around 70% of survey respondents still found some provisions difficult to apply. Representatives from stakeholders' groups found the new version promising, although limitations still exist.

CONCLUSION

By clarifying definitions and roles, the latest revision of the Code sets a new standard in the relationship between investigators and funders to support scientific independence of pharmacoepidemiological research. Disseminating and training on the provisions of the Code would help stakeholders to better understand its advantages and promote its adoption in noninterventional research.

Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies

Purpose

The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context.

Methods

Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context.

Results

A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing.

Conclusion

A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.

Electronic healthcare databases in Europe: descriptive analysis of characteristics and potential for use in medicines regulation

OBJECTIVE

Electronic healthcare databases (EHDs) are useful tools for drug development and safety evaluation but their heterogeneity of structure, validity and access across Europe complicates the conduct of multidatabase studies. In this paper, we provide insight into available EHDs to support regulatory decisions on medicines.

METHODS

EHDs were identified from publicly available information from the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance resources database, textbooks and web-based searches. Databases were selected using criteria related to accessibility, longitudinal dimension, recording of exposure and outcomes, and generalisability. Extracted information was verified with the database owners.

RESULTS

A total of 34 EHDs were selected after applying key criteria relevant for regulatory purposes. The most represented regions were Northern, Central and Western Europe. The most frequent types of data source were electronic medical records (44.1%) and record linkage systems (29.4%). The median number of patients registered in the 34 data sources was 5 million (range 0.07-15 million) while the median time covered by a database was 18.5 years. Paediatric patients were included in 32 databases (94%). Completeness of information on drug exposure was variable. Published validation studies were found for only 17 databases (50%). Some level of access exists for 25 databases (73.5%), and 23 databases (67.6%) can be linked through a personal identification number to other databases with parent-child linkage possible in 7 (21%) databases. Eight databases (23.5%) were already transformed or were in the process of being transformed into a common data model that could facilitate multidatabase studies.

CONCLUSION

A Few European databases meet minimal regulatory requirements and are readily available to be used in a regulatory context. Accessibility and validity information of the included information needs to be improved. This study confirmed the fragmentation, heterogeneity and lack of transparency existing in many European EHDs.

Learning the effects of psychotropic drugs during pregnancy using real-world safety data: a paradigm shift toward modern pharmacovigilance

The growing evidence on psychotropic drug safety in pregnancy has been possible thanks to the increasing availability of real-world data, i.e. data not collected in conventional randomised controlled trials. Use of these data is a key to establish psychotropic drug effects on foetal, child, and maternal health. Despite the inherent limitations and pitfalls of observational data, these can still be informative after a critical appraisal of the collective body of evidence has been done. By valuing real-world safety data, and making these a larger part of the regulatory decision-making process, we move toward a modern pregnancy pharmacovigilance. The recent uptake of real-world safety data by health authorities has set the basis for an important paradigm shift, which is integrating such data into drug labelling. The recent safety assessment of sodium valproate in pregnant and childbearing women is probably one of the first examples of modern pregnancy pharmacovigilance.