1
|
Coste A, Wong AY, Warren-Gash C, Matthewman J, Bate A, Douglas IJ. Implementation of a Taxonomy-Based Framework for the Selection of Appropriate Drugs and Outcomes for Real-World Data Signal Detection Studies. Drug Saf 2024; 47:183-192. [PMID: 38093083 PMCID: PMC10821990 DOI: 10.1007/s40264-023-01382-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/14/2023] [Indexed: 01/28/2024]
Abstract
INTRODUCTION For signal detection studies investigating either drug safety or method evaluation, the choice of drug-outcome pairs needs to be tailored to the planned study design and vice versa. While this is well understood in hypothesis-testing epidemiology, it should be as important in signal detection, but this has not widely been considered. There is a need for a taxonomy framework to provide guidance and a systematic reproducible approach to the selection of appropriate drugs and outcomes for signal detection studies either investigating drug safety or assessing method performance using real-world data. OBJECTIVE The aim was to design a general framework for the selection of appropriate drugs and outcomes for signal detection studies given a study design of interest. As a motivating example, we illustrate how the framework is applied to build a reference set for a study aiming to assess the performance of the self-controlled case series with active comparators. METHODS We reviewed criteria presented in two published studies which aimed to provide practical advice for choosing the appropriate signal evaluation methodology, and assessed their relevance for signal detection. Further characteristics specific to signal detection were added. The final framework is based on: the application of study design requirements, the database(s) of interest, and the clinical importance of the drug(s) and outcome(s) under consideration. This structure was applied by selecting drug-outcome pairs as a reference set (i.e. list of drug-outcome pairs classified as positive or negative controls) for which the method is expected to work well for a signal detection study aiming to assess the performance of self-controlled case series. Eight criteria were used, related to the application of self-controlled case series assumptions, choice of active comparators, coverage in the database of interest and clinical importance of the outcomes. RESULTS After application of the framework, two classes of antibiotics (seven drugs) were selected for the study, and 28 outcomes from all organ classes were chosen from the drug labels, out of the 273 investigated. In total, this corresponds to 104 positive controls (drug-outcome pairs) and 58 negative controls. CONCLUSIONS We proposed and applied a framework for the selection of drugs and outcomes for both drug safety signal detection and method assessment used in signal detection to optimise their performance given a study design. This framework will eliminate part of the bias relating to drugs and outcomes not being suited to the method or database. The main difficulty lies in the choice of the criteria and their application to ensure systematic selection, especially as some information remains unknown in signal detection, and clinical judgement was needed on occasions. The same framework could be adapted for other methods.
Collapse
Affiliation(s)
- Astrid Coste
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK.
| | - Angel Ys Wong
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Charlotte Warren-Gash
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Julian Matthewman
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| | - Andrew Bate
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
- GlaxoSmithKline, Brentford, UK
| | - Ian J Douglas
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, Keppel St, WC1E 7HT, London, UK
| |
Collapse
|
2
|
Abdullah SS, Rostamzadeh N, Muanda FT, McArthur E, Weir MA, Sontrop JM, Kim RB, Kamran S, Garg AX. High-Throughput Computing to Automate Population-Based Studies to Detect the 30-Day Risk of Adverse Outcomes After New Outpatient Medication Use in Older Adults with Chronic Kidney Disease: A Clinical Research Protocol. Can J Kidney Health Dis 2024; 11:20543581231221891. [PMID: 38186562 PMCID: PMC10771740 DOI: 10.1177/20543581231221891] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 11/20/2023] [Indexed: 01/09/2024] Open
Abstract
Background Safety issues are detected in about one third of prescription drugs in the years following regulatory agency approval. Older adults, especially those with chronic kidney disease, are at particular risk of adverse reactions to prescription drugs. This protocol describes a new approach that may identify credible drug-safety signals more efficiently using administrative health care data. Objective To use high-throughput computing and automation to conduct 700+ drug-safety cohort studies in older adults in Ontario, Canada. Each study will compare 74 acute (30-day) outcomes in patients who start a new prescription drug (new users) to a group of nonusers with similar baseline health characteristics. Risks will be assessed within strata of baseline kidney function. Design and setting The studies will be population-based, new-user cohort studies conducted using linked administrative health care databases in Ontario, Canada (January 1, 2008, to March 1, 2020). The source population for these studies will be residents of Ontario aged 66 years or older who filled at least one outpatient prescription through the Ontario Drug Benefit (ODB) program during the study period (all residents have universal health care, and those aged 65+ have universal prescription drug coverage through the ODB). Patients We identified 3.2 million older adults in the source population during the study period and built 700+ initial medication cohorts, each containing mutually exclusive groups of new users and nonusers. Nonusers were randomly assigned cohort entry dates that followed the same distribution of prescription start dates as new users. Eligibility criteria included a baseline estimated glomerular filtration rate (eGFR) measurement within 12 months before the cohort entry date (median time was 71 days before cohort entry in the new user group), no prior receipt of maintenance dialysis or a kidney transplant, and no prior prescriptions for drugs in the same subclass as the study drug. New users and nonusers will be balanced on ~400 baseline health characteristics using inverse probability of treatment weighting on propensity scores within 3 strata of baseline eGFR: ≥60, 45 to <60, <45 mL/min per 1.73 m2. Outcomes We will compare new user and nonuser groups on 74 clinically relevant outcomes (17 composites and 57 individual outcomes) in the 30 days after cohort entry. We used a prespecified approach to identify these 74 outcomes. Statistical analysis plan In each cohort, we will obtain eGFR-stratum-specific weighted risk ratios and risk differences using modified Poisson regression and binomial regression, respectively. Additive and multiplicative interaction by eGFR category will be examined. Drug-outcome associations that meet prespecified criteria (identified signals) will be further examined in additional analyses (including survival, negative-control exposure, and E-value analyses) and visualizations. Results The initial medication cohorts had a median of 6120 new users per cohort (interquartile range: 1469-38 839) and a median of 1 088 301 nonusers (interquartile range: 751 697-1 267 009). Medications with the largest number of new users were amoxicillin trihydrate (n = 1 000 032), cephalexin (n = 571 566), prescription acetaminophen (n = 571 563), and ciprofloxacin (n = 504,374); 19% to 29% of new users in these cohorts had an eGFR <60 mL/min per 1.73 m2. Limitations Despite our use of robust techniques to balance baseline indicators and to control for confounding by indication, residual confounding will remain a possibility. Only acute (30-day) outcomes will be examined. Our data sources do not include nonprescription (over-the-counter) drugs or drugs prescribed in hospitals and do not include outpatient prescription drug use in children or adults <65 years. Conclusion This accelerated approach to conducting postmarket drug-safety studies has the potential to more efficiently detect drug-safety signals in a vulnerable population. The results of this protocol may ultimately help improve medication safety.
Collapse
Affiliation(s)
| | - Neda Rostamzadeh
- Insight Lab, Western University and ICES Western, London, ON, Canada
| | - Flory T. Muanda
- London Health Sciences Centre, Western University and ICES Western, London, ON, Canada
| | - Eric McArthur
- London Health Sciences Centre and ICES Western, London, ON, Canada
| | - Matthew A. Weir
- London Health Sciences Centre, Western University and ICES Western, London, ON, Canada
| | | | | | - Sedig Kamran
- Insight Lab, Western University, London, ON, Canada
| | - Amit X. Garg
- London Health Sciences Centre, Western University and ICES Western, London, ON, Canada
- Victoria Hospital, London Health Sciences Centre, London, ON, Canada
| |
Collapse
|
3
|
Aakjaer M, De Bruin ML, Andersen M. Epidemiological surveillance of drug safety using cumulative sequential analysis in electronic healthcare data. Basic Clin Pharmacol Toxicol 2024; 134:129-140. [PMID: 37897140 DOI: 10.1111/bcpt.13955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 10/16/2023] [Accepted: 10/23/2023] [Indexed: 10/29/2023]
Abstract
BACKGROUND Methods for safety signal detection in electronic healthcare data analysing data sequentially are being developed to meet the limitations of spontaneous reporting systems. OBJECTIVES This study aims to provide an overview of the literature on sequential analysis of electronic healthcare data and describe the development and testing of a novel epidemiological surveillance system. METHODS We searched Medline, Embase, PubMed, Scopus, Web of Science, and the Cochrane Library applying similar in- and exclusion criteria as those of a previous systematic review. The proposed system consisted of repeated cohort studies and was tested in an emulated prospective setting. Two signal evaluations were performed with several sensitivity analyses and a target trial emulation. FINDINGS In the literature, 11 studies analysed the data sequentially of which two applied traditional epidemiological methods. Epidemiological surveillance of several exposures and outcomes can be successfully conducted with the newly proposed sequential analysis of electronic healthcare data. Signal evaluation studies confirmed the results of the system. CONCLUSIONS Very few studies in the literature analysed data at multiple time points, although this seems to be a prerequisite for testing the methods in a realistic setting. We demonstrated the feasibility of a sequential surveillance system using electronic healthcare data.
Collapse
Affiliation(s)
- Mia Aakjaer
- Pharmacovigilance Research Center, Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark
| | - Marie Louise De Bruin
- Utrecht Institute for Pharmaceutical Sciences (UIPS), Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University, Utrecht, the Netherlands
| | - Morten Andersen
- Pharmacovigilance Research Center, Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
4
|
Gauffin O, Brand JS, Vidlin SH, Sartori D, Asikainen S, Català M, Chalabi E, Dedman D, Danilovic A, Duarte-Salles T, García Morales MT, Hiltunen S, Jödicke AM, Lazarevic M, Mayer MA, Miladinovic J, Mitchell J, Pistillo A, Ramírez-Anguita JM, Reyes C, Rudolph A, Sandberg L, Savage R, Schuemie M, Spasic D, Trinh NTH, Veljkovic N, Vujovic A, de Wilde M, Zekarias A, Rijnbeek P, Ryan P, Prieto-Alhambra D, Norén GN. Supporting Pharmacovigilance Signal Validation and Prioritization with Analyses of Routinely Collected Health Data: Lessons Learned from an EHDEN Network Study. Drug Saf 2023; 46:1335-1352. [PMID: 37804398 PMCID: PMC10684396 DOI: 10.1007/s40264-023-01353-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/17/2023] [Indexed: 10/09/2023]
Abstract
INTRODUCTION Individual case reports are the main asset in pharmacovigilance signal management. Signal validation is the first stage after signal detection and aims to determine if there is sufficient evidence to justify further assessment. Throughout signal management, a prioritization of signals is continually made. Routinely collected health data can provide relevant contextual information but are primarily used at a later stage in pharmacoepidemiological studies to assess communicated signals. OBJECTIVE The aim of this study was to examine the feasibility and utility of analysing routine health data from a multinational distributed network to support signal validation and prioritization and to reflect on key user requirements for these analyses to become an integral part of this process. METHODS Statistical signal detection was performed in VigiBase, the WHO global database of individual case safety reports, targeting generic manufacturer drugs and 16 prespecified adverse events. During a 5-day study-a-thon, signal validation and prioritization were performed using information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from 10 partners of the European Health Data and Evidence Network (EHDEN). Databases included in the study were from the UK, Spain, Norway, the Netherlands and Serbia, capturing records from primary care and/or hospitals. RESULTS Ninety-five statistical signals were subjected to signal validation, of which eight were considered for descriptive analyses in the routine health data. Design, execution and interpretation of results from these analyses took up to a few hours for each signal (of which 15-60 minutes were for execution) and informed decisions for five out of eight signals. The impact of insights from the routine health data varied and included possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological studies. Three signals were selected for signal assessment, two of these decisions were supported by insights from the routine health data. Standardization of analytical code, availability of adverse event phenotypes including bridges between different source vocabularies, and governance around the access and use of routine health data were identified as important aspects for future development. CONCLUSIONS Analyses of routine health data from a distributed network to support signal validation and prioritization are feasible in the given time limits and can inform decision making. The cost-benefit of integrating these analyses at this stage of signal management requires further research.
Collapse
Affiliation(s)
| | | | | | | | | | - Martí Català
- Pharmaco- and Device Epidemiology, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | | | - Daniel Dedman
- Clinical Practice Research Datalink (CPRD), The Medicines and Healthcare Products Regulatory Agency, London, UK
| | | | - Talita Duarte-Salles
- Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, Spain
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Maria Teresa García Morales
- Instituto de Investigación Sanitaria Hospital 12 de Octubre, CIBER de Epidemiología y Salud Pública, Facultad de Medicina, Universidad Complutense de Madrid, Madrid, Spain
| | | | - Annika M Jödicke
- Pharmaco- and Device Epidemiology, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Milan Lazarevic
- Clinic for cardiac and transplant surgery, University Clinical Center Nis, Nis, Serbia
| | - Miguel A Mayer
- Hospital del Mar Medical Research Institute, Parc de Salut Mar, Barcelona, Spain
| | - Jelena Miladinovic
- Clinic for infectious diseases, University Clinical Center Nis, University Clinical Center Nis, Nis, Serbia
| | | | - Andrea Pistillo
- Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, Spain
| | | | - Carlen Reyes
- Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, Spain
| | | | | | - Ruth Savage
- Uppsala Monitoring Centre, Uppsala, Sweden
- Department of General Practice, University of Otago, Christchurch, New Zealand
| | - Martijn Schuemie
- Epidemiology Department, Johnson & Johnson, Titusville, NJ, USA
- Department of Biostatistics, UCLA, Los Angeles, CA, USA
| | - Dimitrije Spasic
- Clinic for cardiac and transplant surgery, University Clinical Center Nis, Nis, Serbia
| | - Nhung T H Trinh
- PharmacoEpidemiology and Drug Safety Research Group, Department of Pharmacy, Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
| | - Nevena Veljkovic
- Heliant Ltd, Belgrade, Serbia
- Vinca Institute of Nuclear Sciences, National Institute of the Republic of Serbia, University of Belgrade, Belgrade, Serbia
| | - Ankica Vujovic
- Clinic for Infectious and Tropical Diseases, University Clinical Center of Serbia, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Marcel de Wilde
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands
| | | | - Peter Rijnbeek
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Patrick Ryan
- Epidemiology Department, Johnson & Johnson, Titusville, NJ, USA
- Department of Biomedical Informatics, Columbia University Medical Center, New York, NY, USA
| | - Daniel Prieto-Alhambra
- Pharmaco- and Device Epidemiology, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands
| | | |
Collapse
|
5
|
Leviton A, Loddenkemper T. Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review. BMC Med Res Methodol 2023; 23:271. [PMID: 37974111 PMCID: PMC10652539 DOI: 10.1186/s12874-023-02102-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 11/08/2023] [Indexed: 11/19/2023] Open
Abstract
Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.
Collapse
Affiliation(s)
- Alan Leviton
- Department of Neurology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
| | - Tobias Loddenkemper
- Department of Neurology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|