1
|
Brock K, Chen C, Ho S, Fuller G, Woolfolk J, McShea C, Penard N. A Bayesian method for safety signal detection in ongoing blinded randomised controlled trials. Pharm Stat 2023; 22:378-395. [PMID: 36510711 DOI: 10.1002/pst.2278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 10/21/2022] [Accepted: 11/19/2022] [Indexed: 12/15/2022]
Abstract
Sponsors have a responsibility to minimise risk to participants in clinical studies through safety monitoring. The FDA Final Rule for IND Safety Reporting requires routine aggregate safety evaluation, including in ongoing blinded studies. We are interested in estimating the probability that the true adverse event rate in the experimental arm exceeds that in the control arm. We developed a Bayesian approach that specifies an informative meta-analytic predictive prior on the event probability in the control arm and an uninformative prior on that in the experimental arm. We combined these priors with a mixture likelihood that considers each patient in the ongoing blinded study may belong to the experimental or control arm. This allowed us to estimate the quantity of interest without unblinding. We evaluated our method by simulation, pairing scenarios that differed only in whether a safety signal was present or missing, and quantifying the ability of our model to discriminate using signal detection theory. Our approach shows benefit. It detects safety signals more reliably with greater sample sizes and for common rather than rare events. Performance does not deteriorate markedly when historical studies exhibit heterogeneous hazards or non-constant hazards. Our method will allow us to monitor safety signals in ongoing blinded studies with the goal of earlier identification and risk mitigation. Our method could be adapted to use informative priors on both arms or predictive covariates where pertinent data exist. We stress that ongoing safety monitoring should involve a multi-disciplinary team where statistical methods are paired with medical judgement.
Collapse
Affiliation(s)
- Kristian Brock
- Centre for Excellence in Statistical Innovation, UCB Pharma, Slough, UK
| | - Chen Chen
- Safety Standards, UCB Pharma, Brussels, Belgium
| | - Shuyen Ho
- Safety Standards, UCB Biosciences, Morrisville, North Carolina, USA
| | | | | | - Cindy McShea
- Safety Standards, UCB Biosciences, Morrisville, North Carolina, USA
| | - Nils Penard
- Safety Standards, UCB France, Colombes, France
| |
Collapse
|
2
|
Carragher R, Robertson C. Assessing safety at the end of clinical trials using system organ classes: A case and comparative study. Pharm Stat 2021; 20:1278-1287. [PMID: 34169636 DOI: 10.1002/pst.2148] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 06/14/2021] [Indexed: 11/11/2022]
Abstract
Recent approaches to the statistical analysis of adverse event (AE) data in clinical trials have proposed the use of groupings of related AEs, such as by system organ class (SOC). These methods have opened up the possibility of scanning large numbers of AEs while controlling for multiple comparisons, making the comparative performance of the different methods in terms of AE detection and error rates of interest to investigators. We apply two Bayesian models and two procedures for controlling the false discovery rate (FDR), which use groupings of AEs, to real clinical trial safety data. We find that while the Bayesian models are appropriate for the full data set, the error controlling methods only give similar results to the Bayesian methods when low incidence AEs are removed. A simulation study is used to compare the relative performances of the methods. We investigate the differences between the methods over full trial data sets, and over data sets with low incidence AEs and SOCs removed. We find that while the removal of low incidence AEs increases the power of the error controlling procedures, the estimated power of the Bayesian methods remains relatively constant over all data sizes. Automatic removal of low-incidence AEs however does have an effect on the error rates of all the methods, and a clinically guided approach to their removal is needed. Overall we found that the Bayesian approaches are particularly useful for scanning the large amounts of AE data gathered.
Collapse
Affiliation(s)
- Raymond Carragher
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow, UK.,Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK.,Health Data Research (UK), University of Strathclyde, Glasgow, UK
| | - Chris Robertson
- Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK.,Public Health Scotland, NHS National Services Scotland, Glasgow, UK
| |
Collapse
|
3
|
Hendrickson BA, Wang W, Ball G, Bennett D, Bhattacharyya A, Fries M, Kuebler J, Kurek R, McShea C, Tremmel L. Aggregate Safety Assessment Planning for the Drug Development Life-Cycle. Ther Innov Regul Sci 2021; 55:717-732. [PMID: 33755928 DOI: 10.1007/s43441-021-00271-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2020] [Accepted: 02/25/2021] [Indexed: 10/21/2022]
Abstract
The Program Safety Analysis Plan (PSAP) was proposed previously as a tool to proactively plan for integrated analyses of product safety data. Building on the PSAP and taking into consideration the evolving regulatory landscape, the Drug Information Association-American Statistical Association (DIA-ASA) Interdisciplinary Safety Evaluation scientific working group herein proposes the Aggregate Safety Assessment Plan (ASAP) process. The ASAP evolves over a product's life-cycle and promotes interdisciplinary, systematic safety planning as well as ongoing data review and characterization of the emerging product safety profile. Objectives include alignment on the safety topics of interest, identification of safety knowledge gaps, planning for aggregate safety evaluation of the clinical trial data and preparing for safety communications. The ASAP seeks to tailor the analyses for a drug development program while standardizing the analyses across studies within the program. The document is intended to be modular and flexible in nature, depending on the program complexity, phase of development and existing sponsor processes. Implementation of the ASAP process will facilitate early safety signal detection, improve characterization of product risks, harmonize safety messaging, and inform program decision-making.
Collapse
Affiliation(s)
| | - William Wang
- Clinical Safety Statistics, Biostatistics and Research Decision Sciences, Merck Research Laboratories, North Wales, PA, USA
| | - Greg Ball
- Clinical Safety Statistics, Biostatistics and Research Decision Sciences, Merck Research Laboratories, Rahway, NJ, USA
| | - Dimitri Bennett
- Takeda Pharmaceutical Company Ltd., Cambridge, MA, USA.,Perelman School of Medicine, Adjunct, University of Pennsylvania, Philadelphia, PA, USA
| | | | - Michael Fries
- Quantitative Clinical Sciences and Reporting, CSL Behring, King of Prussia, PA, USA
| | - Juergen Kuebler
- QSciCon, Quantitative Scientific Consulting, Marburg, Germany
| | - Raffael Kurek
- Early Oncology Clinical Group, Oncology R&D, AstraZeneca, Cambridge, UK
| | - Cynthia McShea
- Statistical Sciences and Innovation, UCB BioSciences, Inc., Raleigh, NC, USA
| | - Lothar Tremmel
- Quantitative Clinical Sciences and Reporting, CSL Behring, King of Prussia, PA, USA
| |
Collapse
|
4
|
Phillips R, Sauzet O, Cornelius V. Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy. BMC Med Res Methodol 2020; 20:288. [PMID: 33256641 PMCID: PMC7708917 DOI: 10.1186/s12874-020-01167-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 11/19/2020] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs). METHODS A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification. RESULTS Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods). CONCLUSIONS This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed. TRIAL REGISTRATION PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.
Collapse
Affiliation(s)
- Rachel Phillips
- Imperial Clinical Trials Unit, Imperial College London, 1st Floor Stadium House, 68 Wood Lane, London, W12 7RH, United Kingdom.
| | - Odile Sauzet
- School of Public Health / AG 3 Epidemiologie & International Public Health, Bielefeld University, Bielefeld, Germany
| | - Victoria Cornelius
- Imperial Clinical Trials Unit, Imperial College London, 1st Floor Stadium House, 68 Wood Lane, London, W12 7RH, United Kingdom
| |
Collapse
|
5
|
Liu J, Wick J, Martin RH, Meinzer C, Roy D, Gajewski B. Two-stage Bayesian hierarchical modeling for blinded and unblinded safety monitoring in randomized clinical trials. BMC Med Res Methodol 2020; 20:211. [PMID: 32807102 PMCID: PMC7433072 DOI: 10.1186/s12874-020-01097-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2020] [Accepted: 08/10/2020] [Indexed: 11/10/2022] Open
Abstract
Background Monitoring and reporting of drug safety during a clinical trial is essential to its success. More recent attention to drug safety has encouraged statistical methods development for monitoring and detecting potential safety signals. This paper investigates the potential impact of the process of the blinded investigator identifying a potential safety signal, which should be further investigated by the Data and Safety Monitoring Board with an unblinded safety data analysis. Methods In this paper, two-stage Bayesian hierarchical models are proposed for safety signal detection following a pre-specified set of interim analyses that are applied to efficacy. At stage 1, a hierarchical blinded model uses blinded safety data to detect a potential safety signal and at stage 2, a hierarchical logistic model is applied to confirm the signal with unblinded safety data. Results Any interim safety monitoring analysis is usually scheduled via negotiation between the trial sponsor and the Data and Safety Monitoring Board. The proposed safety monitoring process starts once 53 subjects have been enrolled into an eight-arm phase II clinical trial for the first interim analysis. Operating characteristics describing the performance of this proposed workflow are investigated using simulations based on the different scenarios. Conclusions The two-stage Bayesian safety procedure in this paper provides a statistical view to monitor safety during the clinical trials. The proposed two-stage monitoring model has an excellent accuracy of detecting and flagging a potential safety signal at stage 1, and with the most important feature that further action at stage 2 could confirm the safety issue.
Collapse
Affiliation(s)
- Junhao Liu
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Mail Stop 1026, 3901 Rainbow Blvd, Kansas City, KS, 66160, USA.,Novartis, East Hanover, NJ, 07936, USA
| | - Jo Wick
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Mail Stop 1026, 3901 Rainbow Blvd, Kansas City, KS, 66160, USA
| | - Renee' H Martin
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, 29425, USA
| | - Caitlyn Meinzer
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, 29425, USA
| | - Dooti Roy
- Boehringer Ingelheim, Ridgefield, CT, 06877, USA
| | - Byron Gajewski
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Mail Stop 1026, 3901 Rainbow Blvd, Kansas City, KS, 66160, USA.
| |
Collapse
|
6
|
Lin LA, Yuan SS, Li L, Ball G. Meta-analysis of blinded and unblinded studies for ongoing aggregate safety monitoring and evaluation. Contemp Clin Trials 2020; 95:106068. [PMID: 32603785 DOI: 10.1016/j.cct.2020.106068] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 06/02/2020] [Accepted: 06/11/2020] [Indexed: 11/18/2022]
Abstract
During the course of clinical development, ongoing aggregate safety monitoring and evaluation are needed to understand the evolving safety profile and to ensure effective risk-management strategies for medicinal products. CIOMS reports and global regulatory guidance (including from ICH, US FDA, and EMA) compel sponsors for assessment of safety based on aggregate data. To identify and characterize the risks of medicinal products at a program level in a more timely and informed manner, aggregate safety evaluations should combine all available information, including from ongoing blinded trials, completed unblinded trials, and other data sources. In this article, we propose two Bayesian meta-analytic approaches for synthesizing blinded and unblinded studies in order to characterize the evolving safety profile of medicinal products at the program level. With the proposed approaches, sponsors can dynamically update knowledge of their product safety profiles as data accrue. Application of the procedures to a real and a hypothetical clinical trial program are provided to illustrate how the proposed approaches can be used to analyze a pre-specified event of interest and to screen for risk-elevated events.
Collapse
Affiliation(s)
- Li-An Lin
- Clinical Safety Statistics, Merck & Co., Inc., Rahway, NJ, USA.
| | - Shuai Sammy Yuan
- Biometrics Team, Kite Pharma, a Gilead Company, Santa Monica, CA, USA
| | - Lie Li
- Clinical Safety Statistics, Merck & Co., Inc., North Wales, PA, USA
| | - Greg Ball
- Clinical Safety Statistics, Merck & Co., Inc., Rahway, NJ, USA
| |
Collapse
|
7
|
Wang W, Revis R, Nilsson M, Crowe B. Clinical Trial Drug Safety Assessment With Interactive Visual Analytics. Stat Biopharm Res 2020. [DOI: 10.1080/19466315.2020.1736142] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Wei Wang
- Eli Lilly Canada Inc., Toronto, ON, Canada
| | | | | | | |
Collapse
|
8
|
Diao G, Liu GF, Zeng D, Wang W, Tan X, Heyse JF, Ibrahim JG. Efficient methods for signal detection from correlated adverse events in clinical trials. Biometrics 2019; 75:1000-1008. [PMID: 30690717 DOI: 10.1111/biom.13031] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2019] [Accepted: 01/15/2019] [Indexed: 11/27/2022]
Abstract
It is an important and yet challenging task to identify true signals from many adverse events that may be reported during the course of a clinical trial. One unique feature of drug safety data from clinical trials, unlike data from post-marketing spontaneous reporting, is that many types of adverse events are reported by only very few patients leading to rare events. Due to the limited study size, the p-values of testing whether the rate is higher in the treatment group across all types of adverse events are in general not uniformly distributed under the null hypothesis that there is no difference between the treatment group and the placebo group. A consequence is that typically fewer than 100 α percent of the hypotheses are rejected under the null at the nominal significance level of α . The other challenge is multiplicity control. Adverse events from the same body system may be correlated. There may also be correlations between adverse events from different body systems. To tackle these challenging issues, we develop Monte-Carlo-based methods for the signal identification from patient-reported adverse events in clinical trials. The proposed methodologies account for the rare events and arbitrary correlation structures among adverse events within and/or between body systems. Extensive simulation studies demonstrate that the proposed method can accurately control the family-wise error rate and is more powerful than existing methods under many practical situations. Application to two real examples is provided.
Collapse
Affiliation(s)
- Guoqing Diao
- Department of Statistics, George Mason University, Fairfax, Virginia
| | | | - Donglin Zeng
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | | | - Xianming Tan
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | | | - Joseph G Ibrahim
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| |
Collapse
|
9
|
Seltzer JH, Li J, Wang W. Interdisciplinary Safety Evaluation and Quantitative Safety Monitoring: Introduction to a Series of Papers. Ther Innov Regul Sci 2019:2168479018793130. [PMID: 30925080 DOI: 10.1177/2168479018793130] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
The American Statistical Association and DIA have created an interdisciplinary working group of drug safety experts from academia, industry and regulatory backgrounds to explore the future direction for safety monitoring. This introduction to the series explains the background and rationale for this special section.
Collapse
Affiliation(s)
| | - Judy Li
- 2 Biostatistics, Celgene Corporation, San Diego, CA, USA
| | - William Wang
- 3 Clinical Safety Statistics, Biostatistics and Research Decision Sciences (BARDS), Merck Research Laboratories, North Wales, PA, USA
| |
Collapse
|