Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

BACKGROUND

Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).

METHODS

A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.

RESULTS

Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).

CONCLUSIONS

This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.

TRIAL REGISTRATION

PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies

The analysis of adverse events (AEs) is a key component in the assessment of a drug's safety profile. Inappropriate analysis methods may result in misleading conclusions about a therapy's safety and consequently its benefit-risk ratio. The statistical analysis of AEs is complicated by the fact that the follow-up times can vary between the patients included in a clinical trial. This paper takes as its focus the analysis of AE data in the presence of varying follow-up times within the benefit assessment of therapeutic interventions. Instead of approaching this issue directly and solely from an analysis point of view, we first discuss what should be estimated in the context of safety data, leading to the concept of estimands. Although the current discussion on estimands is mainly related to efficacy evaluation, the concept is applicable to safety endpoints as well. Within the framework of estimands, we present statistical methods for analysing AEs with the focus being on the time to the occurrence of the first AE of a specific type. We give recommendations which estimators should be used for the estimands described. Furthermore, we state practical implications of the analysis of AEs in clinical trials and give an overview of examples across different indications. We also provide a review of current practices of health technology assessment (HTA) agencies with respect to the evaluation of safety data. Finally, we describe problems with meta-analyses of AE data and sketch possible solutions.

Statistical Practices of Safety Monitoring: An Industry Survey

The Biopharmaceutical Section of the American Statistical Association (ASA) formed a Safety Monitoring Working Group to strengthen collaborations between biostatisticians and safety scientists. The task began by surveying current needs and practices regarding available statistical safety tools and methods, regulatory guidance, and processes needed to support their implementation. The goal is for biostatisticians to become fully engaged safety team members by having the necessary safety skill set including appropriate methodology, regulatory guidance and access to appropriate tools. In this publication, we will discuss our survey results that reveal current practices at 22 pharmaceutical companies and demonstrate how the survey instrument can be used to map an action plan for meeting the demand for improved quantitative safety monitoring.

Informative graphing of continuous safety variables relative to normal reference limits

BACKGROUND

Interpreting graphs of continuous safety variables can be complicated because differences in age, gender, and testing site methodologies data may give rise to multiple reference limits. Furthermore, data below the lower limit of normal are compressed relative to those points above the upper limit of normal. The objective of this study is to develop a graphing technique that addresses these issues and is visually intuitive.

METHODS

A mock dataset with multiple reference ranges is initially used to develop the graphing technique. Formulas are developed for conditions where data are above the upper limit of normal, normal, below the lower limit of normal, and below the lower limit of normal when the data value equals zero. After the formulae are developed, an anonymized dataset from an actual set of trials for an approved drug is evaluated comparing the technique developed in this study to standard graphical methods.

RESULTS

Formulas are derived for the novel graphing method based on multiples of the normal limits. The formula for values scaled between the upper and lower limits of normal is a novel application of a readily available scaling formula. The formula for the lower limit of normal is novel and addresses the issue of this value potentially being indeterminate when the result to be scaled as a multiple is zero.

CONCLUSIONS

The formulae and graphing method described in this study provides a visually intuitive method to graph continuous safety data including laboratory values, vital sign data.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Clinical Trials

BACKGROUND

There has been an increased emphasis on the proactive and comprehensive evaluation of safety endpoints to ensure patient well-being throughout the medical product life cycle. In fact, depending on the severity of the underlying disease, it is important to plan for a comprehensive safety evaluation at the start of any development program. Statisticians should be intimately involved in this process and contribute their expertise to study design, safety data collection, analysis, reporting (including data visualization), and interpretation.

METHODS

In this manuscript, we review the challenges associated with the analysis of safety endpoints and describe the safety data that are available to influence the design and analysis of premarket clinical trials.

RESULTS

We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from clinical trials compared to other sources.

CONCLUSIONS

Clinical trials are an important source of safety data that contribute to the totality of safety information available to generate evidence for regulators, sponsors, payers, physicians, and patients. This work is a result of the efforts of the American Statistical Association Biopharmaceutical Section Safety Working Group.

Detecting potential safety issues in large clinical or observational trials by Bayesian screening when event counts arise from poisson distributions

Patients in large clinical trials and in studies employing large observational databases report many different adverse events, most of which will not have been anticipated at the outset. Conventional hypothesis testing of between group differences for each adverse event can be problematic: Lack of significance does not mean lack of risk, the tests usually are not adjusted for multiplicity, and the data determine which hypotheses are tested. This article describes a Bayesian screening approach that does not test hypotheses, is self-adjusting for multiplicity, provides a direct assessment of the likelihood of no material drug-event association, and quantifies the strength of the observed association. The criteria for assessing drug-event associations can be determined by clinical or regulatory considerations. In contrast to conventional approaches, the diagnostic properties of this new approach can be evaluated analytically. Application of the method to findings from a vaccine trial yields results similar to those found by methods using a false discovery rate argument or a hierarchical Bayes approach. [Supplemental materials are available for this article. Go to the publisher's online edition of Journal of Biopharmaceutical Statistics for the following free supplemental resource: Appendix R: Code for calculations.].

The practice of pre-marketing safety assessment in drug development

The last 15 years have seen a substantial increase in efforts devoted to safety assessment by statisticians in the pharmaceutical industry. While some of these efforts were driven by regulations and public demand for safer products, much of the motivation came from the realization that there is a strong need for a systematic approach to safety planning, evaluation, and reporting at the program level throughout the drug development life cycle. An efficient process can help us identify safety signals early and afford us the opportunity to develop effective risk minimization plan early in the development cycle. This awareness has led many pharmaceutical sponsors to set up internal systems and structures to effectively conduct safety assessment at all levels (patient, study, and program). In addition to process, tools have emerged that are designed to enhance data review and pattern recognition. In this paper, we describe advancements in the practice of safety assessment during the premarketing phase of drug development. In particular, we share examples of safety assessment practice at our respective companies, some of which are based on recommendations from industry-initiated working groups on best practice in recent years.

Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team

Background The Safety Planning, Evaluation and Reporting Team (SPERT) was formed in 2006 by the Pharmaceutical Research and Manufacturers of America.

Purpose SPERT’s goal was to propose a pharmaceutical industry standard for safety planning, data collection, evaluation, and reporting, beginning with planning first-in-human studies and continuing through the planning of the post-product-approval period.

Methods SPERT’s recommendations are based on our review of relevant literature and on consensus reached in our discussions.

Results An important recommendation is that sponsors create a Program Safety Analysis Plan early in development. We also give recommendations for the planning of repeated, cumulative meta-analyses of the safety data obtained from the studies conducted within the development program. These include clear definitions of adverse events of special interest and standardization of many aspects of data collection and study design. We describe a 3-tier system for signal detection and analysis of adverse events and highlight proposals for reducing "false positive" safety findings. We recommend that sponsors review the aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission.

Limitations We recognize that there may be other valid approaches.

Conclusions The proactive approach we advocate has the potential to benefit patients and health care providers by providing more comprehensive safety information at the time of new product marketing and beyond. Clinical Trials 2009; 6: 430—440. http://ctj.sagepub.com

Graphical approaches to the analysis of safety data from clinical trials

Patient safety has always been a primary focus in the development of new pharmaceutical products. The predominant method for statistical evaluation and interpretation of safety data collected in a clinical trial is the tabular display of descriptive statistics. There is a great opportunity to enhance evaluation of drug safety through the use of graphical displays, which can convey multiple pieces of information concisely and more effectively than can tables. Graphs can be used in an exploratory setting to help identify emerging safety signals, or in a confirmatory setting as a tool to elucidate known safety issues. We developed several graphical displays for routine safety data collected during a clinical trial, covering a broad range of graphical techniques, and illustrate here 10 specific graphical designs, many of which display the data along with statistics derived from them. Two are simple plots, comparing distributions in the form of boxplots or cumulative plots, and four more display data and summaries over time, comparing information from two groups in terms of distribution (with boxplots), cumulative incidence, hazard, or simply means with error bars. The other four are multi-panel displays: one-dimensional and two-dimensional arrays of scatterplots, a trellis of individual profiles, and a paired dotplot displaying risk together with relative risk. The displays focus on key safety endpoints in clinical trials including the QT interval from electrocardiograms, laboratory measurements for detecting hepatotoxicity, and adverse events of special interest. We discuss in detail the statistical and graphical principles underlying the production and interpretation of the displays.

New developments in medical clinical trials

This paper reviews several new developments and long-standing good practices for conducting clinical trials. Discussion starts with the need for clear statements of study objectives, proceeds to clarify target and sample population, and elaborates on primary vs. secondary variables with the need for alpha adjustment in the presence of multiple outcomes. Here we also review the issue of surrogate endpoints. Study design issues--including blinding, randomization, and multicenter studies--come next. Then we discuss the current trend of the replacement of placebo-controlled trials by active controlled non-inferiority trials, the increasing use of Independent Data Monitoring Committees, the prominence of analysis on Intention-to-Treat samples, and the importance of imputation of missing data. We close with a brief discussion of the unit of analysis, the role of newer statistical analysis methods, safety issues, subset analysis, and, most importantly, clinical significance.