The Role of Expert Opinion in Projecting Long-Term Survival Outcomes Beyond the Horizon of a Clinical Trial

An Evaluation of an Algorithm for the Selection of Flexible Survival Models for Cancer Immunotherapies: Pass or Fail?

BACKGROUND AND OBJECTIVE

Accurately extrapolating survival beyond trial follow-up is essential in a health technology assessment where model choice often substantially impacts estimates of clinical and cost effectiveness. Evidence suggests standard parametric models often provide poor fits to long-term data from immuno-oncology trials. Palmer et al. developed an algorithm to aid the selection of more flexible survival models for these interventions. We assess the usability of the algorithm, identify areas for improvement and evaluate whether it effectively identifies models capable of accurate extrapolation.

METHODS

We applied the Palmer algorithm to the CheckMate-649 trial, which investigated nivolumab plus chemotherapy versus chemotherapy alone in patients with gastroesophageal adenocarcinoma. We evaluated the algorithm's performance by comparing survival estimates from identified models using the 12-month data cut to survival observed in the 48-month data cut.

RESULTS

The Palmer algorithm offers a systematic procedure for model selection, encouraging detailed analyses and ensuring that crucial stages in the selection process are not overlooked. In our study, a range of models were identified as potentially appropriate for extrapolating survival, but only flexible parametric non-mixture cure models provided extrapolations that were plausible and accurately predicted subsequently observed survival. The algorithm could be improved with minor additions around the specification of hazard plots and setting out plausibility criteria.

CONCLUSIONS

The Palmer algorithm provides a systematic framework for identifying suitable survival models, and for defining plausibility criteria for extrapolation validity. Using the algorithm ensures that model selection is based on explicit justification and evidence, which could reduce discordance in health technology appraisals.

MPES-R: Multi-Parameter Evidence Synthesis in R for Survival Extrapolation-A Tutorial

Survival extrapolation often plays an important role in health technology assessment (HTA), and there are a range of different approaches available. Approaches that can leverage external evidence (i.e. data or information collected outside the main data source of interest) may be helpful, given the extent of uncertainty often present when determining a suitable survival extrapolation. One of these methods is the multi-parameter evidence synthesis (MPES) approach, first proposed for use in HTA by Guyot et al., and more recently by Jackson. While MPES has potential benefits over conventional extrapolation approaches (such as simple or flexible parametric models), it is more computationally complex and requires use of specialist software. This tutorial presents an introduction to MPES for HTA, alongside a user-friendly, publicly available operationalisation of Guyot's original MPES that can be executed using the statistical software package R. Through two case studies, both Guyot's and Jackson's MPES approaches are explored, along with sensitivity analyses relevant to HTA. Finally, the discussion section of the tutorial details important considerations for analysts considering use of an MPES approach, along with potential further developments. MPES has not been used often in HTA, and so there are limited examples of how it has been used and perceived. However, this tutorial may aid future research efforts exploring the use of MPES further.

Algorithmic benchmark modulation: A novel method to develop success rates for clinical studies

BACKGROUND

High-quality decision-making in the pharmaceutical industry requires accurate assessments of the Probability of Technical Success of clinical trials. Failure to do so will lead to lost opportunities for both patients and investors. Pharmaceutical companies employ different methodologies to determine Probability of Technical Success values. Some companies use power and assurance calculations; others prefer to use industry benchmarks with or without the overlay of subjective modulations. At AstraZeneca, both assurance calculations and industry benchmarks are used, and both methods are combined with modulations.

METHODS

AstraZeneca has recently implemented a simple algorithm that allows for modulation of a Probability of Technical Success value. The algorithm is based on a set of multiple-choice questions. These questions cover a comprehensive set of issues that have historically been considered by AstraZeneca when subjective modulations to Probability of Technical Success values were made but do so in a much more structured way.

RESULTS

A set of 57 phase 3 Probability of Technical Success assessments suggests that AstraZeneca's historical estimation of Probability of Technical Success has been reasonably accurate. A good correlation between the subjective modulation and the modulation algorithm was found. This latter observation, combined with the finding that historically AstraZeneca has been reasonably accurate in its estimation of Probability of Technical Success, gives confidence in the validity of the novel method.

DISCUSSION

Although it is too early to demonstrate whether the method has improved the accuracy of company's Probability of Technical Success assessments, we present our data and analysis here in the hope that it may assist the pharmaceutical industry in addressing this key challenge. This new methodology, developed for pivotal studies, enables AstraZeneca to develop more consistent Probability of Technical Success assessments with less effort and can be used to adjust benchmarks as well as assurance calculations.

CONCLUSION

The Probability of Technical Success modulation algorithm addresses several concerns generally associated with assurance calculations or benchmark without modulation: selection biases, situations where little relevant prior data are available and the difficulty to model many factors affecting study outcomes. As opposed to using industry benchmarks, the Probability of Technical Success modulation algorithm allows to accommodate project-specific considerations.