Tree-based scan statistic - Application in manufacturing-related safety signal detection

Tree-temporal scan statistics for safety signal detection in vaccine clinical trials

The evaluation of safety is critical in all clinical trials. However, the quantitative analysis of safety data in clinical trials poses statistical difficulties because of multiple potentially overlapping endpoints. Tree-temporal scan statistic approaches address this issue and have been widely employed in other data sources, but not to date in clinical trials. We evaluated the performance of three complementary scan statistical methods for routine quantitative safety signal detection: the self-controlled tree-temporal scan (SCTTS), a tree-temporal scan based on group comparison (BGTTS), and a log-rank based tree-temporal scan (LgRTTS). Each method was evaluated using data from two phase III clinical trials, and simulated data (simulation study). In the case study, the reference set was adverse events (AEs) in the Reference Safety Information of the evaluated vaccine. The SCTTS method had higher sensitivity than other methods, and after dose 1 detected 80 true positives (TP) with a positive predictive value (PPV) of 60%. The LgRTTS detected 49 TPs with 69% PPV. The BGTTS had 90% of PPV with 38 TPs. In the simulation study, with simulated reference sets of AEs, the SCTTS method had good sensitivity to detect transient effects. The LgRTTS method showed the best performance for the detection of persistent effects, with high sensitivity and expected probability of type I error. These three methods provide complementary approaches to safety signal detection in clinical trials or across clinical development programmes. All three methods formally adjust for multiple testing of large numbers of overlapping endpoints without being excessively conservative.

Development and Application of a Data-Driven Signal Detection Method for Surveillance of Adverse Event Variability Across Manufacturing Lots of Biologics

INTRODUCTION

Postmarketing drug safety surveillance research has focused on the product-patient interaction as the primary source of variability in clinical outcomes. However, the inherent complexity of pharmaceutical manufacturing and distribution, especially of biologic drugs, also underscores the importance of risks related to variability in manufacturing and supply chain conditions that could potentially impact clinical outcomes. We propose a data-driven signal detection method called HMMScan to monitor for manufacturing lot-dependent changes in adverse event (AE) rates, and herein apply it to a biologic drug.

METHODS

The HMMScan method chooses the best-fitting candidate from a family of probabilistic Hidden Markov Models to detect temporal correlations in per lot AE rates that could signal clinically relevant variability in manufacturing and supply chain conditions. Additionally, HMMScan indicates the particular lots most likely to be related to risky states of the manufacturing or supply chain condition. The HMMScan method was validated on extensive simulated data and applied to three actual lot sequences of a major biologic drug by combining lot metadata from the manufacturer with AE reports from the US FDA Adverse Event Reporting System (FAERS).

RESULTS

Extensive method validation on simulated data indicated that HMMScan is able to correctly detect the presence or absence of variable manufacturing and supply chain conditions for contiguous sequences of 100 lots or more when changes in these conditions have a meaningful impact on AE rates. Applying the HMMScan method to FAERS data, two of the three actual lot sequences examined exhibited evidence of potential manufacturing or supply chain-related variability.

CONCLUSIONS

HMMScan could be utilized by both manufacturers and regulators to automate lot variability monitoring and inform targeted root-cause analysis. Broad application of HMMScan would rely on a well-developed data input pipeline. The proposed method is implemented in an open-source GitHub repository.

Post-Marketing Safety Surveillance for the Adjuvanted Recombinant Zoster Vaccine: Methodology

A diligent, systematic, regular review of aggregate safety data is essential, particularly early after vaccine introduction, as this is when safety signals not identified during clinical development may emerge. In October 2017, the US Centers for Disease Control and Prevention Advisory Committee on Immunization Practices recommended the adjuvanted recombinant zoster vaccine (RZV; Shingrix, GSK) as the preferred vaccine for preventing herpes zoster (HZ) and related complications in immunocompetent adults aged ≥ 50 years. Subsequently, GSK experienced an unprecedented high demand for RZV. In this methodology paper, we summarize the enhanced measures undertaken to assess RZV safety during its early post-marketing experience in the USA, Canada and Germany. In addition to the routine signal-detection methods already in place for all vaccines, GSK established tailored and enhanced safety monitoring for RZV based on aggregate data of spontaneous reports and manufacturing data. Proactive, near real-time detection and evaluation of signals was a key objective. A dedicated in-house signal-detection tool customized for RZV was employed on a weekly (rather than the routine monthly) basis, allowing for a centralized, more frequent review of data on a single web-based platform. We also identified the background incidence rates of preselected medical events of interest in the first countries to introduce RZV (USA, Canada and Germany) to perform observed-to-expected analyses. This approach may offer a solution to the challenges associated with the assessment and monitoring of vaccine safety in an efficient and timely manner in the context of high vaccine uptake.