Summarizing the incidence of adverse events using volcano plots and time intervals

[Special issue PRO] Considering endpoints for comparative tolerability of cancer treatments using patient report given the estimand framework

Regulatory agencies are advancing the use of systematic approaches to collect patient experience data, including patient-reported outcomes (PROs), in cancer clinical trials to inform regulatory decision-making. Due in part to clinician under-reporting of symptomatic adverse events, there is a growing recognition that evaluation of cancer treatment tolerability should include the patient experience, both in terms of the overall side effect impact and symptomatic adverse events. Methodologies around implementation, analysis, and interpretation of "patient" reported tolerability are under development, and current approaches are largely descriptive. There is robust guidance for use of PROs as efficacy endpoints to compare cancer treatments, but it is unclear to what extent this can be relied-upon to develop tolerability endpoints. An important consideration when developing endpoints to compare tolerability between treatments is the linkage of trial design, objectives, and statistical analysis. Despite interest in and frequent collection of PRO data in oncology trials, heterogeneity in analyses and unclear PRO objectives mean that design, objectives, and analysis may not be aligned, posing substantial challenges for the interpretation of results. The recent ICH E9 (R1) estimand framework represents an opportunity to help address these challenges. Efforts to apply the estimand framework in the context of PROs have primarily focused on efficacy outcomes. In this paper, we discuss considerations for comparing the patient-reported tolerability of different treatments in an oncology trial context.

DMC reports in the 21st century: towards better tools for decision-making

Data Monitoring Committees (DMCs) have the important task to protect the safety of current and future patients during the conduct of a clinical study. Unfortunately, their work is often made difficult by voluminous DMC reports that are poorly structured and difficult to digest. In this article, we suggest improved solutions. Starting from a principled approach and building upon previous proposals, we offer concrete and easily understood displays, including related computer code. While leveraging modern tools, the most important is that these displays support the DMC's workflow in answering the relevant questions of interest. We hope that the adoption of these proposals can ease the task of DMCs, and importantly, lead to better decision-making for the benefit of patients.

Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials

In clinical trials, harms (i.e., adverse events) are often reported by simply counting the number of people who experienced each event. Reporting only frequencies ignores other dimensions of the data that are important for stakeholders, including severity, seriousness, rate (recurrence), timing, and groups of related harms. Additionally, application of selection criteria to harms prevents most from being reported. Visualization of data could improve communication of multidimensional data. We replicated and compared the characteristics of 6 different approaches for visualizing harms: dot plot, stacked bar chart, volcano plot, heat map, treemap, and tendril plot. We considered binary events using individual participant data from a randomized trial of gabapentin for neuropathic pain. We assessed their value using a heuristic approach and a group of content experts. We produced all figures using R and share the open-source code on GitHub. Most original visualizations propose presenting individual harms (e.g., dizziness, somnolence) alone or alongside higher level (e.g., by body systems) summaries of harms, although they could be applied at either level. Visualizations can present different dimensions of all harms observed in trials. Except for the tendril plot, all other plots do not require individual participant data. The dot plot and volcano plot are favored as visualization approaches to present an overall summary of harms data. Our value assessment found the dot plot and volcano plot were favored by content experts. Using visualizations to report harms could improve communication. Trialists can use our provided code to easily implement these approaches.

Drugs-associated with red man syndrome: An integrative approach using disproportionality analysis and Pharmip

WHAT IS KNOWN AND OBJECTIVE

Red man syndrome (RMS) is a non-IgE-mediated anaphylactoid adverse event frequently witnessed after a rapid infusion of vancomycin. This study aims to unravel drugs and associated off-label targets that induce RMS by exploiting FDA Adverse Event Reporting System (FAERS) and Pharmacovigilance/Pharmacogenomics Insilico Pipeline (PHARMIP).

METHODS

The case/non-case retrospective observational study was conducted in the FAERS database. Reporting odds ratio (ROR) and proportional reporting ratio (PRR) data mining algorithms were used to evaluate the strength of the signal. The off-label targets of the drugs with potential signals were obtained using online servers by applying a similarity ensemble approach and a reverse pharmacophore database, which was further validated by molecular docking studies.

RESULTS AND DISCUSSION

Oritavancin exhibited a strong positive signal (PRR:1185.20 and ROR:1256), which suggests a higher risk for causing RMS. The literature search revealed the involvement of the MRGPRX2 gene in the development of RMS. PHARMIP study unearthed Carbonic anhydrase II (CA2) as the common off-label target among the drugs causing RMS. The results obtained from molecular docking studies reinforced the findings as mentioned earlier, wherein the highest docking score was disinterred for oritavancin (-9.4 for MRGPRX2 and - 8.7 for CA2).

WHAT IS NEW AND CONCLUSION

Many antibiotics and other classes of medications have been discovered in the quest for drugs that may induce RMS, although a causal relationship could not be established. The implication of MRGPX2 and CA2 in the initial stages of pathogenesis necessitates the development of inhibitors that could be used as potential therapeutic agents against RMS.

Visualizing adverse events in clinical trials using correspondence analysis with R-package visae

Background

Graphical displays and data visualization are essential components of statistical analysis that can lead to improved understanding of clinical trial adverse event (AE) data. Correspondence analysis (CA) has been introduced decades ago as a multivariate technique that can communicate AE contingency tables using two-dimensional plots, while quantifying the loss of information as other dimension reduction techniques such as principal components and factor analysis.

Methods

We propose the application of stacked CA using contribution biplots as a tool to explore differences in AE data among treatments in clinical trials. We defined five levels of refinement for the analysis based on data derived from the Common Terminology Criteria for Adverse Events (CTCAE) grades, domains, terms and their combinations. In addition, we developed a Shiny app built in an R-package, visae, publicly available on Comprehensive R Archive Network (CRAN), to interactively investigate CA configurations based on the contribution to the explained variance and relative frequency of AEs. Data from two randomized controlled trials (RCT) were used to illustrate the proposed methods: NSABP R-04, a neoadjuvant rectal 2 × 2 factorial trial comparing radiation therapy with either capecitabine (Cape) or 5-fluorouracil (5-FU) alone with or without oxaliplatin (Oxa), and NSABP B-35, a double-blind RCT comparing tamoxifen to anastrozole in postmenopausal women with hormone-positive ductal carcinoma in situ.

Results

In the R04 trial (n = 1308), CA biplots displayed the discrepancies between single agent treatments and their combinations with Oxa at all levels of AE classes, such that these discrepancies were responsible for the largest portion of the explained variability among treatments. In addition, an interaction effect when adding Oxa to Cape/5-FU was identified when the distance between Cape+Oxa and 5-FU + Oxa was observed to be larger than the distance between 5-FU and Cape, with Cape+Oxa and 5-FU + Oxa in different quadrants of the CA biplots. In the B35 trial (n = 3009), CA biplots showed different patterns for non-adherent Anastrozole and Tamoxifen compared with their adherent counterparts.

Conclusion

CA with contribution biplot is an effective tool that can be used to summarize AE data in a two-dimensional display while minimizing the loss of information and interpretation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01368-w.

A multicentre, randomized, parallel group, superiority study to compare the clinical effectiveness and cost-effectiveness of external frame versus internal locking plate for complete articular pilon fracture fixation in adults

AIMS

A pilon fracture is a severe ankle joint injury caused by high-energy trauma, typically affecting men of working age. Although relatively uncommon (5% to 7% of all tibial fractures), this injury causes among the worst functional and health outcomes of any skeletal injury, with a high risk of serious complications and long-term disability, and with devastating consequences on patients' quality of life and financial prospects. Robust evidence to guide treatment is currently lacking. This study aims to evaluate the clinical and cost-effectiveness of two surgical interventions that are most commonly used to treat pilon fractures.

METHODS

A randomized controlled trial (RCT) of 334 adult patients diagnosed with a closed type C pilon fracture will be conducted. Internal locking plate fixation will be compared with external frame fixation. The primary outcome and endpoint will be the Disability Rating Index (a patient self-reported assessment of physical disability) at 12 months. This will also be measured at baseline, three, six, and 24 months after randomization. Secondary outcomes include the Olerud and Molander Ankle Score (OMAS), the five-level EuroQol five-dimenison score (EQ-5D-5L), complications (including bone healing), resource use, work impact, and patient treatment preference. The acceptability of the treatments and study design to patients and health care professionals will be explored through qualitative methods.

DISCUSSION

The two treatments being compared are the most commonly used for this injury, however there is uncertainty over which is most clinically and cost-effective. The Articular Pilon Fracture (ACTIVE) Trial is a sufficiently powered and rigorously designed study to inform clinical decisions for the treatment of adults with this injury. Cite this article: Bone Jt Open 2021;2(3):150-163.

Advantages of visualisations to evaluate and communicate adverse event information in randomised controlled trials

BACKGROUND

Randomised controlled trials (RCTs) provide valuable information and inform the development of harm profiles of new treatments. Harms are typically assessed through the collection of adverse events (AEs). Despite AEs being routine outcomes collected in trials, analysis and reporting of AEs in journal articles are continually shown to be suboptimal. One key challenge is the large volume of AEs, which can make evaluation and communication problematic. Prominent practice is to report frequency tables of AEs by arm. Visual displays offer an effective solution to assess and communicate complex information; however, they are rarely used and there is a lack of practical guidance on what and how to visually display complex AE data.

METHODS

In this article, we demonstrate the use of two plots identified to be beneficial for wide use in RCTs, since both can display multiple AEs and are suitable to display point estimates for binary, count, or time-to-event AE data: the volcano and dot plots. We compare and contrast the use of data visualisations against traditional frequency table reporting, using published AE information in two placebo-controlled trials, of remdesivir for COVID-19 and GDNF for Parkinson disease. We introduce statistical programmes for implementation in Stata.

RESULTS/CASE STUDY

Visualisations of AEs in the COVID-19 trial communicated a risk profile for remdesivir which differed from the main message in the published authors' conclusion. In the Parkinson's disease trial of GDNF, the visualisation provided immediate communication of harm signals, which had otherwise been contained within lengthy descriptive text and tables. Asymmetry in the volcano plot helped flag extreme events that were less obvious from review of the frequency table and dot plot. The dot plot allowed a more comprehensive representation by means of a more detailed summary.

CONCLUSIONS

Visualisations can better support investigators to assimilate large volumes of data and enable improved informal between-arm comparisons compared to tables. We endorse increased uptake for use in trial publications. Care in construction of visual displays needs to be taken as there can be potential to overemphasise treatment effects in some circumstances.

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy

BACKGROUND

Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).

METHODS

A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.

RESULTS

Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).

CONCLUSIONS

This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.

TRIAL REGISTRATION

PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

Enhancing clarity of clinical trial safety reports for data monitoring committees

A Data Monitoring Committee (DMC) evaluates patient safety in a clinical trial of an investigational intervention through periodic review of adverse events (AEs) and clinical safety assessments. Our aim was to construct DMC report displays to enhance the DMC safety review through use of graphics and clear identification and adjustment for missing data caused by early discontinuations and ongoing study participation. Suggested displays include a study snapshot graph, enhanced adverse event incidence tables including the incidence density and plotted incidence proportions, line graphs in place of by-patient listings, and trend plots in place of tables for continuous assessments.

A randomised placebo controlled trial of anakinra for treating pustular psoriasis: statistical analysis plan for stage two of the APRICOT trial

BACKGROUND

Current treatment options for Palmoplantar Pustulosis (PPP), a debilitating chronic skin disease which affects the hands and feet, are limited. The Anakinra for Pustular psoriasis: Response in a Controlled Trial (APRICOT) aims to determine the efficacy of anakinra in the treatment of PPP. This article describes the statistical analysis plan for the final analysis of this two-staged trial, which was determined prior to unblinding and database lock. This is an update to the published protocol and stage one analysis plan.

METHODS

APRICOT is a randomised, double-blind, placebo-controlled trial of anakinra versus placebo, with two stages and an adaptive element. Stage one compared treatment arms to ensure proof-of-concept and determined the primary outcome for stage two of the trial. The primary outcome was selected to be the change in Palmoplantar Pustulosis Psoriasis Area and Severity Index (PPPASI) at 8 weeks. Secondary outcomes include other investigator-assessed efficacy measures of disease severity, participant-reported measures of efficacy and safety measures. This manuscript describes in detail the outcomes, sample size, general analysis principles, the pre-specified statistical analysis plan for each of the outcomes, the handling of missing outcome data and the planned sensitivity and supplementary analyses for the second stage of the APRICOT trial.

DISCUSSION

This statistical analysis plan was developed in compliance with international trial guidelines and is published to increase transparency of the trial analysis. The results of the trial analysis will indicate whether anakinra has a role in the treatment of PPP.

TRIAL REGISTRATION

ISCRTN, ISCRTN13127147. Registered on 1 August 2016. EudraCT Number 2015-003600-23. Registered on 1 April 2016.

The Tendril Plot-a novel visual summary of the incidence, significance and temporal aspects of adverse events in clinical trials

Background

In contrast to efficacy, safety hypotheses of clinical trials are not always pre-specified, and therefore, the safety interpretation work of a trial tends to be more exploratory, often reactive, and the analysis more statistically and graphically challenging.

Methods

We introduce a new means of visualizing the adverse event data across an entire clinical trial.

Results

The approach overcomes some of the current limitations of adverse event analysis and streamlines the way safety data can be explored, interpreted and analyzed. Using a phase II study, we describe and exemplify how the tendril plot effectively summarizes the time-resolved safety profile of two treatment arms in a single plot and how that can provide scientists with a trial safety overview that can support medical decision making.

Conclusion

To our knowledge, the tendril plot is the only way to graphically show important treatment differences with preserved temporal information, across an entire clinical trial, in a single view.

Analysis and reporting of adverse events in randomised controlled trials: a review

OBJECTIVE

To ascertain contemporary approaches to the collection, reporting and analysis of adverse events (AEs) in randomised controlled trials (RCTs) with a primary efficacy outcome.

DESIGN

A review of clinical trials of drug interventions from four high impact medical journals.

DATA SOURCES

Electronic contents table of the BMJ, the Journal of the American Medical Association (JAMA), the Lancet and the New England Journal of Medicine (NEJM) were searched for reports of original RCTs published between September 2015 and September 2016.

METHODS

A prepiloted checklist was used and single data extraction was performed by three reviewers with independent check of a randomly sampled subset to verify quality. We extracted data on collection methods, assessment of severity and causality, reporting criteria, analysis methods and presentation of AE data.

RESULTS

We identified 184 eligible reports (BMJ n=3; JAMA n=38, Lancet n=62 and NEJM n=81). Sixty-two per cent reported some form of spontaneous AE collection but only 29% included details of specific prompts used to ascertain AE data. Numbers that withdrew from the trial were well reported (80%), however only 35% of these reported whether withdrawals were due to AEs. Results presented and analysis performed was predominantly on 'patients with at least one event' with 84% of studies ignoring repeated events. Despite a lack of power to undertake formal hypothesis testing, 47% performed such tests for binary outcomes.

CONCLUSIONS

This review highlighted that the collection, reporting and analysis of AE data in clinical trials is inconsistent and RCTs as a source of safety data are underused. Areas to improve include reducing information loss when analysing at patient level and inappropriate practice of underpowered multiple hypothesis testing. Implementation of standard reporting practices could enable a more accurate synthesis of safety data and development of guidance for statistical methodology to assess causality of AEs could facilitate better statistical practice.

Understanding the influence of individual variables contributing to multivariate outliers in assessments of data quality

Mahalanobis distance is often recommended to identify patients or clinical sites that are considered unusual in clinical trials. Patients extreme in one or more covariates may be considered outliers in that they reside some distance from the multivariate mean, which can be thought of as the center of the data cloud. Less often discussed, patients whose data are believed to be "too good to be true" are located near the centroid as inliers. In order to efficiently investigate these anomalies for potential lapses in data quality, it is important to understand how the individual variables contribute to each multivariate outlier. There is a lack of literature describing a reasonable workflow for identification of outliers and their subsequent investigation to understand how each variable contributes to an observation being considered extreme. We describe how to identify multivariate inliers and outliers, classify outliers according to varying levels of severity, and summarize the contributions of variables using principal components in a manner that is accessible to a wide audience with straightforward interpretation. We illustrate how numerous data visualizations, including Pareto plots, can facilitate further review even in studies containing numerous observations and variables. We illustrate these methodologies using data from a multicenter clinical trial.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Clinical Trials

BACKGROUND

There has been an increased emphasis on the proactive and comprehensive evaluation of safety endpoints to ensure patient well-being throughout the medical product life cycle. In fact, depending on the severity of the underlying disease, it is important to plan for a comprehensive safety evaluation at the start of any development program. Statisticians should be intimately involved in this process and contribute their expertise to study design, safety data collection, analysis, reporting (including data visualization), and interpretation.

METHODS

In this manuscript, we review the challenges associated with the analysis of safety endpoints and describe the safety data that are available to influence the design and analysis of premarket clinical trials.

RESULTS

We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from clinical trials compared to other sources.

CONCLUSIONS

Clinical trials are an important source of safety data that contribute to the totality of safety information available to generate evidence for regulators, sponsors, payers, physicians, and patients. This work is a result of the efforts of the American Statistical Association Biopharmaceutical Section Safety Working Group.

Statistical challenges for central monitoring in clinical trials: a review

Recently, the complexity and costs of clinical trials have increased dramatically, especially in the area of new drug development. Risk-based monitoring (RBM) has been attracting attention as an efficient and effective trial monitoring approach, which can be applied irrespectively of the trial sponsor, i.e., academic institution or pharmaceutical company. In the RBM paradigm, it is expected that a statistical approach to central monitoring can help improve the effectiveness of on-site monitoring by prioritizing and guiding site visits according to central statistical data checks, as evidenced by examples of actual trial datasets. In this review, several statistical methods for central monitoring are presented. It is important to share knowledge about the role and performance capabilities of statistical methodology among clinical trial team members (i.e., sponsors, investigators, data managers, monitors, and biostatisticians) in order to adopt central statistical monitoring for assessing data quality in the actual clinical trial.

Carbohydrate Microarrays

Carbohydrates, like nucleic acids and proteins, are essential biological molecules. Owing to their intrinsic physicochemical properties, carbohydrates are capable of generating structural diversity in a multitude of ways and are prominently displayed on the surfaces of cell membranes or on the exposed regions of macromolecules. Recent studies highlight that carbohydrate moieties are critical for molecular recognition, cell-cell interactions, and cell signaling in many physiological and pathological processes, and for biocommunication between microbes and host species. Modern carbohydrate microarrays emerged in 2002 and brought in new high-throughput tools for “glyco code” exploration. In this section, some basic concepts of sugar chain diversity, glyco-epitope recognition, and the evolving area of glyco-epitomics and biomarker discovery are discussed. Two complementary technologies, carbohydrate antigen arrays and photogenerated glyco-chips, serve as models to illustrate how to apply carbohydrate microarrays to address biomedical questions.

Uncovering cryptic glycan markers in multiple sclerosis (MS) and experimental autoimmune encephalomyelitis (EAE)

Using an integrated antigen microarray approach, we observed epitope-spreading of autoantibody responses to a variety of antigenic structures in the cerebrospinal fluid (CSF) of patients with multiple sclerosis (MS) and in the serum of mice with experimental autoimmune encephalomyelitis (EAE). These included previously described protein- and lipid-based antigenic targets and newly discovered autoimmunogenic sugar moieties, notably, autoantibodies specific for the oligomannoses in both MS patient CSF and the sera of mice with EAE. These glycans are often masked by other sugar moieties and belong to a class of cryptic autoantigens. We further determined that these targets are highly expressed on multiple cell types in MS and EAE lesions. Co-immunization of SJL/J mice with a Man9-KLH conjugate at the time of EAE induction elicited highly significant levels of anti-Man9-cluster autoantibodies. Nevertheless, this anti-glycan autoantibody response was associated with a significantly reduced clinical severity of EAE. The potential of these cryptic glycan markers and targeting antibodies for diagnostic and therapeutic interventions of neurological disorders has yet to be explored.