Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): Rationale and statistical concept of a meta-analytic study

Correlation of safety and efficacy of atezolizumab therapy across indications

BACKGROUND

The association between safety and efficacy of immune checkpoint inhibitors is known, but the correlation between severity and impact of specific organ involvement by immune-related adverse events (irAE) and cancer outcomes is poorly understood. Most irAEs are mild-to-moderate but severe irAEs may pose clinical management challenges and affect patient outcomes.

METHODS

We assessed the association between irAE grade (G) and specific organ involvement with overall survival (OS) in 9,521 patients across 14 studies involving atezolizumab as mono (IO) or with chemo/targeted (C-IO) therapy as compared with chemo/targeted therapy (C) in advanced non-small cell lung, small-cell lung, renal cell, urothelial, and triple-negative breast cancers. We used a mixed-effect Cox proportional hazard model for time-varying covariates to address immortal-time bias; adjusted for baseline factors associated with irAEs and OS to control for confounding bias; and focused on five common irAEs (dermatologic, thyroid dysfunction, hepatitis, pneumonitis, and colitis) to avoid low statistical power for rare events.

RESULTS

For patients treated with IO or C-IO, G1-2 irAEs were associated with improved OS (HR=0.65, p<0.01) and G3-4 irAEs showed a slight increased risk of death (HR=1.18, p=0.10) versus patients without irAEs. By specific irAE, G1-2 cutaneous irAEs, thyroid dysfunction, or pneumonitis were associated with improved OS (p<0.05), while G3-4 pneumonitis and colitis were associated with worse OS (p<0.01). There was no association between hepatitis and OS by any grade. Findings were consistent across indications.

CONCLUSIONS

This analysis demonstrates a correlation between irAEs and improved OS with atezolizumab by severity grade and the most common irAEs by organ involvement. Low-grade irAEs are significantly associated with improved OS, while specific high-grade irAEs are associated with poorer OS, underscoring the importance of early recognition and management of toxicity to optimize benefit/risk balance.

Evaluating Primary Treatment for People with Advanced Glaucoma: Five-Year Results of the Treatment of Advanced Glaucoma Study

PURPOSE

To determine whether primary trabeculectomy or medical treatment produces better outcomes in terms of quality of life (QoL), clinical effectiveness, and safety in patients with advanced glaucoma.

DESIGN

Multicenter randomized controlled trial.

PARTICIPANTS

Between June 3, 2014, and May 31, 2017, 453 adults with newly diagnosed advanced open-angle glaucoma in at least 1 eye (Hodapp classification) were recruited from 27 secondary care glaucoma departments in the United Kingdom. Two hundred twenty-seven were allocated to trabeculectomy, and 226 were allocated medical management.

METHODS

Participants were randomized on a 1:1 basis to have either mitomycin C-augmented trabeculectomy or escalating medical management with intraocular pressure (IOP)-reducing drops as the primary intervention and were followed up for 5 years.

MAIN OUTCOME MEASURES

The primary outcome was vision-specific QoL measured with the 25-item Visual Function Questionnaire (VFQ-25) at 5 years. Secondary outcomes were general health status, glaucoma-related QoL, clinical effectiveness (IOP, visual field, and visual acuity), and safety.

RESULTS

At 5 years, the mean ± standard deviation VFQ-25 scores in the trabeculectomy and medication arms were 83.3 ± 15.5 and 81.3 ± 17.5, respectively, and the mean difference was 1.01 (95% confidence interval [CI], -1.99 to 4.00; P = 0.51). The mean IOPs were 12.07 ± 5.18 mmHg and 14.76 ± 4.14 mmHg, respectively, and the mean difference was -2.56 (95% CI, -3.80 to -1.32; P < 0.001). Glaucoma severity measured with visual field mean deviation were -14.30 ± 7.14 dB and -16.74 ± 6.78 dB, respectively, with a mean difference of 1.87 (95% CI, 0.87-2.87 dB; P < 0.001). Safety events occurred in 115 (52.2%) of patients in the trabeculectomy arm and 124 (57.9%) of patients in the medication arm (relative risk, 0.92; 95% CI, 0.72-1.19; P = 0.54). Serious adverse events were rare.

CONCLUSIONS

At 5 years, the Treatment of Advanced Glaucoma Study demonstrated that primary trabeculectomy surgery is more effective in lowering IOP and preventing disease progression than primary medical treatment in patients with advanced disease and has a similar safety profile.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): summary of findings and assessment of existing guidelines

BACKGROUND

The SAVVY project aims to improve the analyses of adverse events (AEs) in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). This paper summarizes key features and conclusions from the various SAVVY papers.

METHODS

Summarizing several papers reporting theoretical investigations using simulations and an empirical study including randomized clinical trials from several sponsor organizations, biases from ignoring varying follow-up times or CEs are investigated. The bias of commonly used estimators of the absolute (incidence proportion and one minus Kaplan-Meier) and relative (risk and hazard ratio) AE risk is quantified. Furthermore, we provide a cursory assessment of how pertinent guidelines for the analysis of safety data deal with the features of varying follow-up time and CEs.

RESULTS

SAVVY finds that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard.

CONCLUSIONS

The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. Whenever varying follow-up times and/or CEs are present in the assessment of AEs, SAVVY recommends using the Aalen-Johansen estimator (AJE) with an appropriate definition of CEs to quantify AE risk. There is an urgent need to improve pertinent clinical trial reporting guidelines for reporting AEs so that incidence proportions or one minus Kaplan-Meier estimators are finally replaced by the AJE with appropriate definition of CEs.

Adverse events in single-arm clinical trials with non-fatal time-to-event efficacy endpoint: from clinical questions to methods for statistical analysis

BACKGROUND

In any single-arm trial on novel treatments, assessment of toxicity plays an important role as occurrence of adverse events (AEs) is relevant for application in clinical practice. In the presence of a non-fatal time-to-event(s) efficacy endpoint, the analysis should be broadened to consider AEs occurrence in time. The AEs analysis could be tackled with two approaches, depending on the clinical question of interest. Approach 1 focuses on the occurrence of AE as first event. Treatment ability to protect from the efficacy endpoint event(s) has an impact on the chance of observing AEs due to competing risks action. Approach 2 considers how treatment affects the occurrence of AEs in the potential framework where the efficacy endpoint event(s) could not occur.

METHODS

In the first part of the work we review the strategy of analysis for these two approaches. We identify theoretical quantities and estimators consistent with the following features: (a) estimators should address for the presence of right censoring; (b) theoretical quantities and estimators should be functions of time. In the second part of the work we propose the use of alternative methods (regression models, stratified Kaplan-Meier curves, inverse probability of censoring weighting) to relax the assumption of independence between the potential times to AE and to event(s) in the efficacy endpoint for addressing Approach 2.

RESULTS

We show through simulations that the proposed methods overcome the bias due to the dependence between the two potential times and related to the use of standard estimators.

CONCLUSIONS

We demonstrated through simulations that one can handle patients selection in the risk sets due to the competing event, and thus obtain conditional independence between the two potential times, adjusting for all the observed covariates that induce dependence.

On the role of benchmarking data sets and simulations in method comparison studies

Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favor a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data for such method comparison studies. Most manuscripts on statistical methodology rely on simulation studies and provide a single real-world data set as an example to motivate and illustrate the methodology investigated. In the context of supervised learning, in contrast, methods are often evaluated using so-called benchmarking data sets, that is, real-world data that serve as gold standard in the community. Simulation studies, on the other hand, are much less common in this context. The aim of this paper is to investigate differences and similarities between these approaches, to discuss their advantages and disadvantages, and ultimately to develop new approaches to the evaluation of methods picking the best of both worlds. To this aim, we borrow ideas from different contexts such as mixed methods research and Clinical Scenario Evaluation.

Adverse events after left ventricular assist device implantation linked to psychosocial risk in women and men

BACKGROUND

Reasons for women's increased probability to experience adverse events (AEs) after left ventricular assist device (LVAD) implantation compared with men's remain uncertain. We explored the role of psychosocial risk in the experience of AEs in women and men.

METHODS

INTERMACS patients receiving a primary continuous-flow LVAD between July 2006 and December 2017, median follow-up 13.6 months, were included (n = 20,123, 21.3% women). Time-to-event was calculated with cumulative incidence functions for 10 types of AEs separately (e.g., infection, device malfunction), each time accounting for the competing outcomes death, heart transplant, and device explant due to recovery. Event-specific Cox proportional hazard models were run with a binary psychosocial risk variable (including substance abuse, psychiatric diagnosis, limited social support, limited cognition, repeated noncompliance), controlled for covariates.

RESULTS

Psychosocial risk was more prevalent in men than in women (21.4% vs 17.5%, p < 0.001). Seven out of 10 AEs were more likely in women than in men (e.g., infection 44.5% vs 39.2%, p < 0.001). The association of psychosocial risk with each AE was either stronger in women than in men (e.g., device malfunction HRadj 1.29, 95% confidence interval (CI) (1.06-1.56) vs HRadj 1.10, 95% CI (0.97-1.25); rehospitalization HRadj 1.15, 95% CI (1.02-1.29) vs HRadj 1.03, 95% CI (0.97-1.10) or similar between sexes.

CONCLUSIONS

Independent of clinical parameters, the presence of psychosocial risk is associated with increases in AEs. This suggests that early modification of psychosocial risk factors may have the potential to lower the risk for AEs in this patient population.

Safety analysis of new medications in clinical trials: a simulation study to assess the differences between cause-specific and subdistribution frameworks in the presence of competing events

Safety is an essential part of the evaluation of new medications and competing risks that occur in most clinical trials are a well identified challenge in the analysis of adverse events. Two statistical frameworks exist to consider competing risks: the cause-specific and the subdistribution framework. To date, the application of the cause-specific framework is the standard practice in safety analyses. Here we analyze how the safety analysis results of new medications would be affected if instead of the cause-specific the subdistribution framework was chosen. We conducted a simulation study with 600 participants, equally allocated to verum and control groups and a 30 months follow-up period. Simulated trials were analyzed for safety in a competing risk (death) setting using both the cause-specific and subdistribution frameworks. Results show that comparing safety profiles in a subdistribution setting is always more pessimistic than in a cause-specific setting. For the group with the longest survival and a safety advantage in a cause-specific setting, the advantage either disappeared or a disadvantage was found in the subdistribution analysis setting. These observations are not contradictory but show different perspectives. To evaluate the safety of a new medication over its comparator, one needs to understand the origin of both the risks and the benefits associated with each therapy. These requirements are best met with a cause-specific framework. The subdistribution framework seems better suited for clinical prediction, and therefore more relevant for providers or payers, for example.

Important Considerations for Signal Detection and Evaluation

Safety clinicians have a wealth of resources describing how to perform signal detection. Nevertheless, there are some nuances concerning approaches taken by regulatory authorities and statistical considerations that should be appreciated. New approaches, such as the FDA Medical Queries, illustrate the value of considering medical concepts over individual adverse events. One area which would benefit from further clarity is how safety signals may be evaluated for evidence of a causal relationship to the drug of interest. Just as such safety signals can take many forms, the types of tools and methods required to interrogate these signals are equally as diverse. An understanding of the complexity of this process can aid the safety reviewer in successfully characterizing the emerging safety profile of a drug during the pre-marketing phase of development.

Considerations of Competing Risks Analysis in Cardio-Oncology Studies: JACC: CardioOncology State-of-the-Art Review

Cardio-oncology research studies often require consideration of potential competing risks, as the occurrence of other events (eg, cancer-related death) may preclude the primary event of interest (eg, cardiovascular outcome). However, the decision to conduct competing risks analysis is not always straightforward, and even when deemed necessary, misconceptions exist about the appropriate choice of analytical methods to address the competing risks. R researchers are encouraged to consider competing risks at the study design stage and are provided provide an assessment tool to guide decisions on analytical approach on the basis of study objectives. The existing statistical methods for competing risks analysis, including cumulative incidence estimations and regression modeling are also reviewed. Cardio-oncology-specific examples are used to illustrate these concepts and highlight potential pitfalls and misinterpretations. R code is also provided for these analyses.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY)-estimation of adverse event risks

Background

The SAVVY project aims to improve the analyses of adverse events (AEs), whether prespecified or emerging, in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses, often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator are used, which ignore either censoring or CEs. In an empirical study including randomized clinical trials from several sponsor organizations, these potential sources of bias are investigated. The main purpose is to compare the estimators that are typically used to quantify AE risk within trial arms to the non-parametric Aalen-Johansen estimator as the gold-standard for estimating cumulative AE probabilities. A follow-up paper will consider consequences when comparing safety between treatment groups.

Methods

Estimators are compared with descriptive statistics, graphical displays, and a more formal assessment using a random effects meta-analysis. The influence of different factors on the size of deviations from the gold-standard is investigated in a meta-regression. Comparisons are conducted at the maximum follow-up time and at earlier evaluation times. CEs definition does not only include death before AE but also end of follow-up for AEs due to events related to the disease course or safety of the treatment.

Results

Ten sponsor organizations provided 17 clinical trials including 186 types of investigated AEs. The one minus Kaplan-Meier estimator was on average about 1.2-fold larger than the Aalen-Johansen estimator and the probability transform of the incidence density ignoring CEs was even 2-fold larger. The average bias using the incidence proportion was less than 5%. Assuming constant hazards using incidence densities was hardly an issue provided that CEs were accounted for. The meta-regression showed that the bias depended mainly on the amount of censoring and on the amount of CEs.

Conclusions

The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. We recommend using the Aalen-Johansen estimator with an appropriate definition of CEs whenever the risk for AEs is to be quantified and to change the guidelines accordingly.

Estimating and comparing adverse event probabilities in the presence of varying follow-up times and competing events

Safety analyses of adverse events (AEs) are important in assessing benefit-risk of therapies but are often rather simplistic compared to efficacy analyses. AE probabilities are typically estimated by incidence proportions, sometimes incidence densities or Kaplan-Meier estimation are proposed. These analyses either do not account for censoring, rely on a too restrictive parametric model, or ignore competing events. With the non-parametric Aalen-Johansen estimator as the "gold standard", that is, reference estimator, potential sources of bias are investigated in an example from oncology and in simulations, for both one-sample and two-sample scenarios. The Aalen-Johansen estimator serves as a reference, because it is the proper non-parametric generalization of the Kaplan-Meier estimator to multiple outcomes. Because of potential large variances at the end of follow-up, comparisons also consider further quantiles of the observed times. To date, consequences for safety comparisons have hardly been investigated, the impact of using different estimators for group comparisons being unclear. For example, the ratio of two both underestimating or overestimating estimators may not be comparable to the ratio of the reference, and our investigation also considers the ratio of AE probabilities. We find that ignoring competing events is more of a problem than falsely assuming constant hazards by the use of the incidence density and that the choice of the AE probability estimator is crucial for group comparisons.

Clinical Trial Drug Safety Assessment for Studies and Submissions Impacted by COVID-19

Abstract-In this article, we provide guidance on how safety analyses and reporting of clinical trial safety data may need to be modified, given potential impact from the COVID-19 pandemic. Impact could include missed visits, alternative methods for assessments (such as virtual visits), alternative locations for assessments (such as local labs), and study drug interruptions. Starting from the safety analyses typically included in Clinical Study Reports for Phase 2-4 clinical trials and integrated submission documents, we assess what modifications might be needed. If the impact from COVID-19 affects treatment arms equally, analyses of adverse events from controlled data can, to a large extent, remain unchanged. However, interpretation of summaries from uncontrolled data (summaries that include open-label extension data) will require even more caution than usual. Special consideration will be needed for safety topics of interest, especially events expected to have a higher incidence due to a COVID-19 infection or due to quarantine or travel restrictions (e.g., depression). Analyses of laboratory measurements may need to be modified to account for the combination of measurements from local and central laboratories.

Assessing the Impact of COVID-19 on the Clinical Trial Objective and Analysis of Oncology Clinical Trials-Application of the Estimand Framework

Abstract-Coronavirus disease 2019 (COVID-19) outbreak has rapidly evolved into a global pandemic. The impact of COVID-19 on patient journeys in oncology represents a new risk to interpretation of trial results and its broad applicability for future clinical practice. We identify key intercurrent events (ICEs) that may occur due to COVID-19 in oncology clinical trials with a focus on time-to-event endpoints and discuss considerations pertaining to the other estimand attributes introduced in the ICH E9 addendum. We propose strategies to handle COVID-19 related ICEs, depending on their relationship with malignancy and treatment and the interpretability of data after them. We argue that the clinical trial objective from a world without COVID-19 pandemic remains valid. The estimand framework provides a common language to discuss the impact of COVID-19 in a structured and transparent manner. This demonstrates that the applicability of the framework may even go beyond what it was initially intended for.