On Quantitative Methods for Clinical Safety Monitoring in Drug Development

A Bayesian method for safety signal detection in ongoing blinded randomised controlled trials

Sponsors have a responsibility to minimise risk to participants in clinical studies through safety monitoring. The FDA Final Rule for IND Safety Reporting requires routine aggregate safety evaluation, including in ongoing blinded studies. We are interested in estimating the probability that the true adverse event rate in the experimental arm exceeds that in the control arm. We developed a Bayesian approach that specifies an informative meta-analytic predictive prior on the event probability in the control arm and an uninformative prior on that in the experimental arm. We combined these priors with a mixture likelihood that considers each patient in the ongoing blinded study may belong to the experimental or control arm. This allowed us to estimate the quantity of interest without unblinding. We evaluated our method by simulation, pairing scenarios that differed only in whether a safety signal was present or missing, and quantifying the ability of our model to discriminate using signal detection theory. Our approach shows benefit. It detects safety signals more reliably with greater sample sizes and for common rather than rare events. Performance does not deteriorate markedly when historical studies exhibit heterogeneous hazards or non-constant hazards. Our method will allow us to monitor safety signals in ongoing blinded studies with the goal of earlier identification and risk mitigation. Our method could be adapted to use informative priors on both arms or predictive covariates where pertinent data exist. We stress that ongoing safety monitoring should involve a multi-disciplinary team where statistical methods are paired with medical judgement.

Assessing safety at the end of clinical trials using system organ classes: A case and comparative study

Recent approaches to the statistical analysis of adverse event (AE) data in clinical trials have proposed the use of groupings of related AEs, such as by system organ class (SOC). These methods have opened up the possibility of scanning large numbers of AEs while controlling for multiple comparisons, making the comparative performance of the different methods in terms of AE detection and error rates of interest to investigators. We apply two Bayesian models and two procedures for controlling the false discovery rate (FDR), which use groupings of AEs, to real clinical trial safety data. We find that while the Bayesian models are appropriate for the full data set, the error controlling methods only give similar results to the Bayesian methods when low incidence AEs are removed. A simulation study is used to compare the relative performances of the methods. We investigate the differences between the methods over full trial data sets, and over data sets with low incidence AEs and SOCs removed. We find that while the removal of low incidence AEs increases the power of the error controlling procedures, the estimated power of the Bayesian methods remains relatively constant over all data sizes. Automatic removal of low-incidence AEs however does have an effect on the error rates of all the methods, and a clinically guided approach to their removal is needed. Overall we found that the Bayesian approaches are particularly useful for scanning the large amounts of AE data gathered.

Aggregate Safety Assessment Planning for the Drug Development Life-Cycle

The Program Safety Analysis Plan (PSAP) was proposed previously as a tool to proactively plan for integrated analyses of product safety data. Building on the PSAP and taking into consideration the evolving regulatory landscape, the Drug Information Association-American Statistical Association (DIA-ASA) Interdisciplinary Safety Evaluation scientific working group herein proposes the Aggregate Safety Assessment Plan (ASAP) process. The ASAP evolves over a product's life-cycle and promotes interdisciplinary, systematic safety planning as well as ongoing data review and characterization of the emerging product safety profile. Objectives include alignment on the safety topics of interest, identification of safety knowledge gaps, planning for aggregate safety evaluation of the clinical trial data and preparing for safety communications. The ASAP seeks to tailor the analyses for a drug development program while standardizing the analyses across studies within the program. The document is intended to be modular and flexible in nature, depending on the program complexity, phase of development and existing sponsor processes. Implementation of the ASAP process will facilitate early safety signal detection, improve characterization of product risks, harmonize safety messaging, and inform program decision-making.

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

BACKGROUND

Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).

METHODS

A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.

RESULTS

Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).

CONCLUSIONS

This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.

TRIAL REGISTRATION

PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

Two-stage Bayesian hierarchical modeling for blinded and unblinded safety monitoring in randomized clinical trials

Background

Monitoring and reporting of drug safety during a clinical trial is essential to its success. More recent attention to drug safety has encouraged statistical methods development for monitoring and detecting potential safety signals. This paper investigates the potential impact of the process of the blinded investigator identifying a potential safety signal, which should be further investigated by the Data and Safety Monitoring Board with an unblinded safety data analysis.

Methods

In this paper, two-stage Bayesian hierarchical models are proposed for safety signal detection following a pre-specified set of interim analyses that are applied to efficacy. At stage 1, a hierarchical blinded model uses blinded safety data to detect a potential safety signal and at stage 2, a hierarchical logistic model is applied to confirm the signal with unblinded safety data.

Results

Any interim safety monitoring analysis is usually scheduled via negotiation between the trial sponsor and the Data and Safety Monitoring Board. The proposed safety monitoring process starts once 53 subjects have been enrolled into an eight-arm phase II clinical trial for the first interim analysis. Operating characteristics describing the performance of this proposed workflow are investigated using simulations based on the different scenarios.

Conclusions

The two-stage Bayesian safety procedure in this paper provides a statistical view to monitor safety during the clinical trials. The proposed two-stage monitoring model has an excellent accuracy of detecting and flagging a potential safety signal at stage 1, and with the most important feature that further action at stage 2 could confirm the safety issue.

Meta-analysis of blinded and unblinded studies for ongoing aggregate safety monitoring and evaluation

During the course of clinical development, ongoing aggregate safety monitoring and evaluation are needed to understand the evolving safety profile and to ensure effective risk-management strategies for medicinal products. CIOMS reports and global regulatory guidance (including from ICH, US FDA, and EMA) compel sponsors for assessment of safety based on aggregate data. To identify and characterize the risks of medicinal products at a program level in a more timely and informed manner, aggregate safety evaluations should combine all available information, including from ongoing blinded trials, completed unblinded trials, and other data sources. In this article, we propose two Bayesian meta-analytic approaches for synthesizing blinded and unblinded studies in order to characterize the evolving safety profile of medicinal products at the program level. With the proposed approaches, sponsors can dynamically update knowledge of their product safety profiles as data accrue. Application of the procedures to a real and a hypothetical clinical trial program are provided to illustrate how the proposed approaches can be used to analyze a pre-specified event of interest and to screen for risk-elevated events.

Efficient methods for signal detection from correlated adverse events in clinical trials

It is an important and yet challenging task to identify true signals from many adverse events that may be reported during the course of a clinical trial. One unique feature of drug safety data from clinical trials, unlike data from post-marketing spontaneous reporting, is that many types of adverse events are reported by only very few patients leading to rare events. Due to the limited study size, the p-values of testing whether the rate is higher in the treatment group across all types of adverse events are in general not uniformly distributed under the null hypothesis that there is no difference between the treatment group and the placebo group. A consequence is that typically fewer than 100 α percent of the hypotheses are rejected under the null at the nominal significance level of α . The other challenge is multiplicity control. Adverse events from the same body system may be correlated. There may also be correlations between adverse events from different body systems. To tackle these challenging issues, we develop Monte-Carlo-based methods for the signal identification from patient-reported adverse events in clinical trials. The proposed methodologies account for the rare events and arbitrary correlation structures among adverse events within and/or between body systems. Extensive simulation studies demonstrate that the proposed method can accurately control the family-wise error rate and is more powerful than existing methods under many practical situations. Application to two real examples is provided.

Interdisciplinary Safety Evaluation and Quantitative Safety Monitoring: Introduction to a Series of Papers

The American Statistical Association and DIA have created an interdisciplinary working group of drug safety experts from academia, industry and regulatory backgrounds to explore the future direction for safety monitoring. This introduction to the series explains the background and rationale for this special section.