Bayesian hierarchical meta-analytic methods for modeling surrogate relationships that vary across treatment classes using aggregate data

Assessing heterogeneity in surrogacy using censored data

Determining whether a surrogate marker can be used to replace a primary outcome in a clinical study is complex. While many statistical methods have been developed to formally evaluate a surrogate marker, they generally do not provide a way to examine heterogeneity in the utility of a surrogate marker. Similar to treatment effect heterogeneity, where the effect of a treatment varies based on a patient characteristic, heterogeneity in surrogacy means that the strength or utility of the surrogate marker varies based on a patient characteristic. The few methods that have been recently developed to examine such heterogeneity cannot accommodate censored data. Studies with a censored outcome are typically the studies that could most benefit from a surrogate because the follow-up time is often long. In this paper, we develop a robust nonparametric approach to assess heterogeneity in the utility of a surrogate marker with respect to a baseline variable in a censored time-to-event outcome setting. In addition, we propose and evaluate a testing procedure to formally test for heterogeneity at a single time point or across multiple time points simultaneously. Finite sample performance of our estimation and testing procedure are examined in a simulation study. We use our proposed method to investigate the complex relationship between change in fasting plasma glucose, diabetes, and sex hormones using data from the diabetes prevention program study.

Comparing Bayesian hierarchical meta-regression methods and evaluating the influence of priors for evaluations of surrogate endpoints on heterogeneous collections of clinical trials

BACKGROUND

Surrogate endpoints, such as those of interest in chronic kidney disease (CKD), are often evaluated using Bayesian meta-regression. Trials used for the analysis can evaluate a variety of interventions for different sub-classifications of disease, which can introduce two additional goals in the analysis. The first is to infer the quality of the surrogate within specific trial subgroups defined by disease or intervention classes. The second is to generate more targeted subgroup-specific predictions of treatment effects on the clinical endpoint.

METHODS

Using real data from a collection of CKD trials and a simulation study, we contrasted surrogate endpoint evaluations under different hierarchical Bayesian approaches. Each approach we considered induces different assumptions regarding the relatedness (exchangeability) of trials within and between subgroups. These include partial-pooling approaches, which allow subgroup-specific meta-regressions and, yet, facilitate data adaptive information sharing across subgroups to potentially improve inferential precision. Because partial-pooling models come with additional parameters relative to a standard approach assuming one meta-regression for the entire set of studies, we performed analyses to understand the impact of the parameterization and priors with the overall goals of comparing precision in estimates of subgroup-specific meta-regression parameters and predictive performance.

RESULTS

In the analyses considered, partial-pooling approaches to surrogate endpoint evaluation improved accuracy of estimation of subgroup-specific meta-regression parameters relative to fitting separate models within subgroups. A random rather than fixed effects approach led to reduced bias in estimation of meta-regression parameters and in prediction in subgroups where the surrogate was strong. Finally, we found that subgroup-specific meta-regression posteriors were robust to use of constrained priors under the partial-pooling approach, and that use of constrained priors could facilitate more precise prediction for clinical effects in trials of a subgroup not available for the initial surrogacy evaluation.

CONCLUSION

Partial-pooling modeling strategies should be considered for surrogate endpoint evaluation on collections of heterogeneous studies. Fitting these models comes with additional complexity related to choosing priors. Constrained priors should be considered when using partial-pooling models when the goal is to predict the treatment effect on the clinical endpoint.

Flexible evaluation of surrogacy in platform studies

Trial-level surrogates are useful tools for improving the speed and cost effectiveness of trials but surrogates that have not been properly evaluated can cause misleading results. The evaluation procedure is often contextual and depends on the type of trial setting. There have been many proposed methods for trial-level surrogate evaluation, but none, to our knowledge, for the specific setting of platform studies. As platform studies are becoming more popular, methods for surrogate evaluation using them are needed. These studies also offer a rich data resource for surrogate evaluation that would not normally be possible. However, they also offer a set of statistical issues including heterogeneity of the study population, treatments, implementation, and even potentially the quality of the surrogate. We propose the use of a hierarchical Bayesian semiparametric model for the evaluation of potential surrogates using nonparametric priors for the distribution of true effects based on Dirichlet process mixtures. The motivation for this approach is to flexibly model relationships between the treatment effect on the surrogate and the treatment effect on the outcome and also to identify potential clusters with differential surrogate value in a data-driven manner so that treatment effects on the surrogate can be used to reliably predict treatment effects on the clinical outcome. In simulations, we find that our proposed method is superior to a simple, but fairly standard, hierarchical Bayesian method. We demonstrate how our method can be used in a simulated illustrative example (based on the ProBio trial), in which we are able to identify clusters where the surrogate is, and is not useful. We plan to apply our method to the ProBio trial, once it is completed.

Handling missing within-study correlations in the evaluation of surrogate endpoints

Rigorous evaluation of surrogate endpoints is performed in a trial-level analysis in which the strength of the association between treatment effects on the clinical and surrogate endpoints is quantified across a collection of previously conducted trials. To reduce bias in measures of the performance of the surrogate, the statistical model must account for the sampling error in each trial's estimated treatment effects and their potential correlation. Unfortunately, these within-study correlations can be difficult to obtain, especially for meta-analysis of published trial results where individual patient data is not available. As such, these terms are frequently partially or completely missing in the analysis. We show that improper handling of these missing terms can meaningfully alter the perceived quality of the surrogate and we introduce novel strategies to handle the missingness.

Evaluation of Variation in the Performance of GFR Slope as a Surrogate End Point for Kidney Failure in Clinical Trials that Differ by Severity of CKD

BACKGROUND

The GFR slope has been evaluated as a surrogate end point for kidney failure in meta-analyses on a broad collection of randomized controlled trials (RCTs) in CKD. These analyses evaluate how accurately a treatment effect on GFR slope predicts a treatment effect on kidney failure. We sought to determine whether severity of CKD in the patient population modifies the performance of GFR slope.

METHODS

We performed Bayesian meta-regression analyses on 66 CKD RCTs to evaluate associations between effects on GFR slope (the chronic slope and the total slope over 3 years, expressed as mean differences in ml/min per 1.73 m2/yr) and those of the clinical end point (doubling of serum creatinine, GFR <15 ml/min per 1.73 m2, or kidney failure, expressed as a log-hazard ratio), where models allow interaction with variables defining disease severity. We evaluated three measures (baseline GFR in 10 ml/min per 1.73 m2, baseline urine albumin-to-creatinine ratio [UACR] per doubling in mg/g, and CKD progression rate defined as the control arm chronic slope, in ml/min per 1.73 m2/yr) and defined strong evidence for modification when 95% posterior credible intervals for interaction terms excluded zero.

RESULTS

There was no evidence for modification by disease severity when evaluating 3-year total slope (95% credible intervals for the interaction slope: baseline GFR [-0.05 to 0.03]; baseline UACR [-0.02 to 0.04]; CKD progression rate [-0.07 to 0.02]). There was strong evidence for modification in evaluations of chronic slope (95% credible intervals: baseline GFR [0.02 to 0.11]; baseline UACR [-0.11 to -0.02]; CKD progression rate [0.01 to 0.15]).

CONCLUSIONS

These analyses indicate consistency of the performance of total slope over 3 years, which provides further evidence for its validity as a surrogate end point in RCTs representing varied CKD populations.

A Guide to Selecting Flexible Survival Models to Inform Economic Evaluations of Cancer Immunotherapies

OBJECTIVES

Parametric models are routinely used to estimate the benefit of cancer drugs beyond trial follow-up. The advent of immune checkpoint inhibitors has challenged this paradigm, and emerging evidence suggests that more flexible survival models, which can better capture the shapes of complex hazard functions, might be needed for these interventions. Nevertheless, there is a need for an algorithm to help analysts decide whether flexible models are required and, if so, which should be chosen for testing. This position article has been produced to bridge this gap.

METHODS

A virtual advisory board comprising 7 international experts with in-depth knowledge of survival analysis and health technology assessment was held in summer 2021. The experts discussed 24 questions across 6 topics: the current survival model selection procedure, data maturity, heterogeneity of treatment effect, cure and mortality, external evidence, and additions to existing guidelines. Their responses culminated in an algorithm to inform selection of flexible survival models.

RESULTS

The algorithm consists of 8 steps and 4 questions. Key elements include the systematic identification of relevant external data, using clinical expert input at multiple points in the selection process, considering the future and the observed hazard functions, assessing the potential for long-term survivorship, and presenting results from all plausible models.

CONCLUSIONS

This algorithm provides a systematic, evidence-based approach to justify the selection of survival extrapolation models for cancer immunotherapies. If followed, it should reduce the risk of selecting inappropriate models, partially addressing a key area of uncertainty in the economic evaluation of these agents.

The Validity of Surrogate Endpoints in Sub Groups of Metastatic Colorectal Cancer Patients Defined by Treatment Class and KRAS Status

Background and Aim: Findings from the literature suggest that the validity of surrogate endpoints in metastatic colorectal cancer (mCRC) may depend on a treatments' mechanism of action. We explore this and the impact of Kirsten rat sarcoma (KRAS) status on surrogacy patterns in mCRC. Methods: A systematic review was undertaken to identify randomized controlled trials (RCTs) for pharmacological therapies in mCRC. Bayesian meta-analytic methods for surrogate endpoint evaluation were used to evaluate surrogate relationships across all RCTs, by KRAS status and treatment class. Surrogate endpoints explored were progression free survival (PFS) as a surrogate endpoint for overall survival (OS), and tumour response (TR) as a surrogate for PFS and OS. Results: 66 RCTs were identified from the systematic review. PFS showed a strong surrogate relationship with OS across all data and in subgroups by KRAS status. The relationship appeared stronger within individual treatment classes compared to the overall analysis. The TR-PFS and TR-OS relationships were found to be weak overall but stronger within the Epidermal Growth Factor Receptor + Chemotherapy (EGFR + Chemo) treatment class; both overall and in the wild type (WT) patients for TR-PFS, but not in patients with the mutant (MT) KRAS status where data were limited. Conclusions: PFS appeared to be a good surrogate endpoint for OS. TR showed a moderate surrogate relationship with PFS and OS for the EGFR + Chemo treatment class. There was some evidence of impact of the mechanism of action on the strength of the surrogacy patterns in mCRC, but little evidence of the impact of KRAS status on the validity of surrogate endpoints.

Scoping and targeted reviews to support development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: protocol

INTRODUCTION

Using a surrogate endpoint as a substitute for a primary patient-relevant outcome enables randomised controlled trials (RCTs) to be conducted more efficiently, that is, with shorter time, smaller sample size and lower cost. However, there is currently no consensus-driven guideline for the reporting of RCTs using a surrogate endpoint as a primary outcome; therefore, we seek to develop SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extensions to improve the design and reporting of these trials. As an initial step, scoping and targeted reviews will identify potential items for inclusion in the extensions and participants to contribute to a Delphi consensus process.

METHODS AND ANALYSIS

The scoping review will search and include literature reporting on the current understanding, limitations and guidance on using surrogate endpoints in trials. Relevant literature will be identified through: (1) bibliographic databases; (2) grey literature; (3) handsearching of reference lists and (4) solicitation from experts. Data from eligible records will be thematically analysed into potential items for inclusion in extensions. The targeted review will search for RCT reports and protocols published from 2017 to 2021 in six high impact general medical journals. Trial corresponding author contacts will be listed as potential participants for the Delphi exercise.

ETHICS AND DISSEMINATION

Ethical approval is not required. The reviews will support the development of SPIRIT and CONSORT extensions for reporting surrogate primary endpoints (surrogate endpoint as the primary outcome). The findings will be published in open-access publications.This review has been prospectively registered in the OSF Registration DOI: 10.17605/OSF.IO/WP3QH.

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

INTRODUCTION

Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome.

METHODS AND ANALYSIS

The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases.

ETHICS AND DISSEMINATION

The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums.

Development of a framework and decision tool for the evaluation of health technologies based on surrogate endpoint evidence

In the drive toward faster patient access to treatments, health technology assessment (HTA) agencies and payers are increasingly faced with reliance on evidence based on surrogate endpoints, increasing decision uncertainty. Despite the development of a small number of evaluation frameworks, there remains no consensus on the detailed methodology for handling surrogate endpoints in HTA practice. This research overviews the methods and findings of four empirical studies undertaken as part of COMED (Pushing the Boundaries of Cost and Outcome Analysis of Medical Technologies) program work package 2 with the aim of analyzing international HTA practice of the handling and considerations around the use of surrogate endpoint evidence. We have synthesized the findings of these empirical studies, in context of wider contemporary body of methodological and policy-related literature on surrogate endpoints, to develop a web-based decision tool to support HTA agencies and payers when faced with surrogate endpoint evidence. Our decision tool is intended for use by HTA agencies and their decision-making committees together with the wider community of HTA stakeholders (including clinicians, patient groups, and healthcare manufacturers). Having developed this tool, we will monitor its use and we welcome feedback on its utility.

Correlation Between Early Time-to-Event Outcomes and Overall Survival in Patients With Locally Advanced Head and Neck Squamous Cell Carcinoma Receiving Definitive Chemoradiation Therapy: Systematic Review and Meta-Analysis

Background

Overall survival (OS) is the most patient-relevant outcome in oncology; however, in early cancers, large sample sizes and extended follow-up durations are needed to detect statistically significant differences in OS between interventions. Use of early time-to-event outcomes as surrogates for OS can help facilitate faster approval of cancer therapies. In locally advanced head and neck squamous cell carcinoma (LA-HNSCC), event-free survival (EFS) was previously evaluated as a surrogate outcome (Michiels 2009) and demonstrated a strong correlation with OS. The current study aimed to further assess the correlation between EFS and OS in LA-HNSCC using an updated systematic literature review (SLR) focusing on patients receiving definitive chemoradiation therapy (CRT).

Methods

An SLR was conducted on May 27, 2021 to identify randomized controlled trials assessing radiotherapy alone or CRT in the target population. Studies assessing CRT and reporting hazard ratios (HRs) or Kaplan-Meier data for OS and EFS were eligible for the analysis. CRT included any systemic treatments administered concurrently or sequentially with radiation therapy. Trial-level EFS/OS correlations were assessed using regression models, and the relationship strength was measured with Pearson correlation coefficient (R). Correlations were assessed across all CRT trials and in trial subsets assessing concurrent CRT, sequential CRT, RT+cisplatin, targeted therapies and intensity-modulated RT. Subgroup analysis was conducted among trials with similar EFS definitions (i.e. EFS including disease progression and/or death as events) and longer length of follow-up (i.e.≥ 5 years).

Results

The SLR identified 149 trials of which 31 were included in the analysis. A strong correlation between EFS and OS was observed in the overall analysis of all CRT trials (R=0.85, 95% confidence interval: 0.72-0.93). Similar results were obtained in the sensitivity analyses of trials assessing concurrent CRT (R=0.88), sequential CRT (R=0.83), RT+cisplatin (R=0.82), targeted therapies (R=0.83) and intensity-modulated RT (R=0.86), as well as in trials with similar EFS definitions (R=0.87), with longer follow-up (R=0.81).

Conclusion

EFS was strongly correlated with OS in this trial-level analysis. Future research using individual patient-level data can further investigate if EFS could be considered a suitable early clinical endpoint for evaluation of CRT regimens in LA-HNSCC patients receiving definitive CRT.

A flexible Bayesian framework for individualized inference via adaptive borrowing

The explosion in high-resolution data capture technologies in health has increased interest in making inferences about individual-level parameters. While technology may provide substantial data on a single individual, how best to use multisource population data to improve individualized inference remains an open research question. One possible approach, the multisource exchangeability model (MEM), is a Bayesian method for integrating data from supplementary sources into the analysis of a primary source. MEM was originally developed to improve inference for a single study by asymmetrically borrowing information from a set of similar previous studies and was further developed to apply a more computationally intensive symmetric borrowing in the context of basket trial; however, even for asymmetric borrowing, its computational burden grows exponentially with the number of supplementary sources, making it unsuitable for applications where hundreds or thousands of supplementary sources (i.e., individuals) could contribute to inference on a given individual. In this article, we propose the data-driven MEM (dMEM), a two-stage approach that includes both source selection and clustering to enable the inclusion of an arbitrary number of sources to contribute to individualized inference in a computationally tractable and data-efficient way. We illustrate the application of dMEM to individual-level human behavior and mental well-being data collected via smartphones, where our approach increases individual-level estimation precision by 84% compared with a standard no-borrowing method and outperforms recently proposed competing methods in 80% of individuals.

A survival mediation model with Bayesian model averaging

Determining the extent to which a patient is benefiting from cancer therapy is challenging. Criteria for quantifying the extent of "tumor response" observed within a few cycles of treatment have been established for various types of solids as well as hematologic malignancies. These measures comprise the primary endpoints of phase II trials. Regulatory approvals of new cancer therapies, however, are usually contingent upon the demonstration of superior overall survival with randomized evidence acquired with a phase III trial comparing the novel therapy to an appropriate standard of care treatment. With nearly two-thirds of phase III oncology trials failing to achieve statistically significant results, researchers continue to refine and propose new surrogate endpoints. This article presents a Bayesian framework for studying relationships among treatment, patient subgroups, tumor response, and survival. Combining classical components of a mediation analysis with Bayesian model averaging, the methodology is robust to model misspecification among various possible relationships among the observable entities. A posterior inference is demonstrated via an application to a randomized controlled phase III trial in metastatic colorectal cancer. Moreover, the article details posterior predictive distributions of survival and statistical metrics for quantifying the extent of direct and indirect, or tumor response mediated treatment effects.

Bayesian hierarchical meta-analytic methods for modeling surrogate relationships that vary across treatment classes using aggregate data

Surrogate endpoints play an important role in drug development when they can be used to measure treatment effect early compared to the final clinical outcome and to predict clinical benefit or harm. Such endpoints are assessed for their predictive value of clinical benefit by investigating the surrogate relationship between treatment effects on the surrogate and final outcomes using meta‐analytic methods. When surrogate relationships vary across treatment classes, such validation may fail due to limited data within each treatment class. In this paper, two alternative Bayesian meta‐analytic methods are introduced which allow for borrowing of information from other treatment classes when exploring the surrogacy in a particular class. The first approach extends a standard model for the evaluation of surrogate endpoints to a hierarchical meta‐analysis model assuming full exchangeability of surrogate relationships across all the treatment classes, thus facilitating borrowing of information across the classes. The second method is able to relax this assumption by allowing for partial exchangeability of surrogate relationships across treatment classes to avoid excessive borrowing of information from distinctly different classes. We carried out a simulation study to assess the proposed methods in nine data scenarios and compared them with subgroup analysis using the standard model within each treatment class. We also applied the methods to an illustrative example in colorectal cancer which led to obtaining the parameters describing the surrogate relationships with higher precision.