Adapting Health Technology Assessment agency standards for surrogate outcomes in early stage cancer trials: what needs to happen?

INTRODUCTION

An avalanche of early stage cancer clinical trials is coming. The majority of these solely use surrogate outcomes that have not been validated against a target outcome of interest (e.g. overall survival). Current HTA guidance on surrogate outcome validation are not methodologically or practically conducive to this scenario.

AREAS COVERED

We provide a high-level overview of methods, approaches, and conceptual thinking for making better use of limited evidence within early stage cancer HTA submissions. We outline regulatory and HTA issues and emphasize how evidence transitions from one to another, what major gaps currently exist, and how these may be bridged. We summarize current methodologies and practices, their pros and cons. We outline how complementary measurements strengthen evaluations and address fallacies and biases of conventional statistical methods for surrogate outcomes validation. The value of real-world data to support some of the necessary validity components is discussed. Lastly, we address the importance of the patient voice for better understanding which surrogate outcomes may appropriately inform HTA.

EXPERT OPINION

Conventional surrogate outcome validation represents a fraught and sub-optimal framework for HTA purposes, particularly for early stage cancer. Tools for optimizing use of limited evidence exist. Education of stakeholders is highly needed.

Surrogate endpoints: a key concept in clinical epidemiology

Surrogate endpoints are biomarkers or intermediate outcomes that are used as substitutes for clinical outcomes of interest, often to expedite research or decision-making. In contrast, patient-important (or patient-centered) outcomes are health outcomes that are of direct relevance and importance to patients themselves; clinical trials may have measured the impact of the intervention on other endpoints related to, but different from, those of primary importance to patients. This article aims to elaborate on the use and understanding of surrogate endpoints. There should be a well-understood and scientifically grounded relationship between the surrogate (replacement) and the patient-important (target) endpoint it is intended to represent. It should be biologically plausible that changes in the surrogate will consistently and predictably reflect changes in the patient-important endpoint. The surrogate endpoint should show a threshold effect, meaning that a specific change (or state) in the surrogate with an intervention (relative to the comparator) is associated with a predictable (change in the) patient-important outcome. This helps establish a meaningful cutoff or target for the treatment effect on the surrogate endpoint. While surrogate endpoints offer advantages in certain situations, it is important to remember that their use requires careful validation to ensure they reliably predict the true clinical outcome. The validity of "surrogate endpoints" should be supported by robust scientific evidence and rigorous evaluation before these can be considered and labeled as surrogate endpoints.

Impact of surrogate outcomes in randomized controlled trials for shoulder rotator cuff tears

PURPOSE

Surrogate outcomes are clinical endpoints that are used as substitutes for direct measures of how a patient feels, functions, or survives. The present study aims to analyze the impact of surrogate outcomes on the results of randomized controlled trials on shoulder rotator cuff tears disorders.

METHODS

Randomized controlled trials (RCTs) related to rotator cuff tear conditions published up until 2021 were retrieved from the PubMed and ACCESSSS databases. The primary outcome of the article was considered a surrogate outcome when the authors used radiological, physiologic, or functional variables. The result of the article was considered positive when results supported the intervention based on the trial's primary outcome. We recorded the sample size, the mean follow-up, and the type of funding. Statistical significance was set at p < 0.05.

RESULTS

A total of 112 papers were included in the analysis. The mean sample size was 87.6 patients; mean follow-up period was 25.97 months. Thirty-six out of 112 RCTs used a surrogate outcome as a primary endpoint. More than half of papers using surrogate outcomes reported a positive finding (20 out of 36), while 10 out of 71 RCTs using patient-centered outcomes favored the intervention (14.08%, p < 0.001) [RR = 3.94 (95% CI 2.07-7.51)]. The mean sample size was smaller in trials using surrogate endpoints (75.11 vs 92.35 patients, respectively, p = 0.049), while the follow-up was shorter (14.12 m vs. 31.9 m, p < 0.001). Approximately 25% of the papers that reported surrogate endpoints (22.58%) were industry-funded projects.

CONCLUSIONS

The substitution of surrogate endpoints for patient-important outcomes in shoulder rotator cuff trials quadruplicates the chances of obtaining a favorable result that favors the analyzed intervention.

How do IVF patients interpret claims about fertility treatments? A randomised survey experiment

Trials evaluating the efficacy of IVF and various treatment options often focus on upstream outcome measures, improvements which may not translate into clinical outcome improvements. A cross-sectional online survey was distributed globally among IVF patients. Respondents were randomised to view one of 16 statements about a hypothetical IVF treatment option called 'FertiSure', stated to improve one of four upstream outcomes. Statements varied in whether they contained information stating that FertiSure was not proven to improve live-birth rates and about potential risks. Many patients inferred that improvements in upstream outcomes would result in improvements in the probability of live-birth. Nearly 80% of respondents were willing to use FertiSure. Respondents told that FertiSure was not proven to improve live-birth rates and were less willing to use FertiSure. More respondents agreed that FertiSure may pose a risk to patients when they were told this was the case. However, this did not affect their willingness to use FertiSure. Interestingly, 34% of respondents believed FertiSure would not improve the probability of live-birth but were still willing to use it. These results have implications for IVF clinic websites and information about treatment options which may not routinely contain statements about the limited evidence-base and possible risks.

Potential surrogate outcomes in individuals at high risk for incident knee osteoarthritis

OBJECTIVE

To study potential surrogate outcomes for osteoarthritis (OA) incidence by evaluating the association of short-term changes in clinical and imaging biomarkers with long-term clinical knee OA incidence.

DESIGN

Middle-aged women with overweight/obesity, but free of knee symptoms were recruited through their general practitioners. At baseline, after 2.5 years, and after 6.5 years, questionnaires, physical examination, radiographs, and Magnetic resonance imaging (MRI) scans were obtained. The percentage of knees with a minimal clinically important difference for knee pain severity, Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) pain/stiffness/function, and joint space narrowing, and of those with progression/regression of medial knee alignment, chronic knee pain, radiographic osteophytes, and cartilage defects, bone marrow lesions, osteophytes, and effusion/synovitis on MRI were determined. For each of these potential surrogate outcomes with ≥10% improvement or progression in the population over 2.5 years, the association with incident clinical knee OA, defined using the combined ACR-criteria, after 6.5 years was determined.

RESULTS

Most pre-defined potential surrogate outcomes showed ≥10% change in the population over 2.5 years, but only worsening of TF cartilage defects, worsening of TF osteophytes on MRI, and an increase in pain severity were significantly associated with greater clinical knee OA incidence after 6.5 years. These potential surrogate outcomes had high specificity and negative predictive value (89-91%) and low sensitivity and positive predictive value (20-28%) CONCLUSIONS: Worsening of TF cartilage defects and TF osteophytes on MRI, and increased pain severity could be seen as surrogate outcomes for long-term OA incidence. However, higher positive predictive values seem warranted for the applicability of these factors in future preventive trials.

Metabolomic predictors of phenotypic traits can replace and complement measured clinical variables in population-scale expression profiling studies

Population-scale expression profiling studies can provide valuable insights into biological and disease-underlying mechanisms. The availability of phenotypic traits is essential for studying clinical effects. Therefore, missing, incomplete, or inaccurate phenotypic information can make analyses challenging and prevent RNA-seq or other omics data to be reused. A possible solution are predictors that infer clinical or behavioral phenotypic traits from molecular data. While such predictors have been developed based on different omics data types and are being applied in various studies, metabolomics-based surrogates are less commonly used than predictors based on DNA methylation profiles.In this study, we inferred 17 traits, including diabetes status and exposure to lipid medication, using previously trained metabolomic predictors. We evaluated whether these metabolomic surrogates can be used as an alternative to reported information for studying the respective phenotypes using expression profiling data of four population cohorts. For the majority of the 17 traits, the metabolomic surrogates performed similarly to the reported phenotypes in terms of effect sizes, number of significant associations, replication rates, and significantly enriched pathways.The application of metabolomics-derived surrogate outcomes opens new possibilities for reuse of multi-omics data sets. In studies where availability of clinical metadata is limited, missing or incomplete information can be complemented by these surrogates, thereby increasing the size of available data sets. Additionally, the availability of such surrogates could be used to correct for potential biological confounding. In the future, it would be interesting to further investigate the use of molecular predictors across different omics types and cohorts.

Why clinical trial outcomes fail to translate into benefits for patients

Clinical research should ultimately improve patient care. For this to be possible, trials must evaluate outcomes that genuinely reflect real-world settings and concerns. However, many trials continue to measure and report outcomes that fall short of this clear requirement. We highlight problems with trial outcomes that make evidence difficult or impossible to interpret and that undermine the translation of research into practice and policy. These complex issues include the use of surrogate, composite and subjective endpoints; a failure to take account of patients' perspectives when designing research outcomes; publication and other outcome reporting biases, including the under-reporting of adverse events; the reporting of relative measures at the expense of more informative absolute outcomes; misleading reporting; multiplicity of outcomes; and a lack of core outcome sets. Trial outcomes can be developed with patients in mind, however, and can be reported completely, transparently and competently. Clinicians, patients, researchers and those who pay for health services are entitled to demand reliable evidence demonstrating whether interventions improve patient-relevant clinical outcomes.

Why clinical trial outcomes fail to translate into benefits for patients

Clinical research should ultimately improve patient care. For this to be possible, trials must evaluate outcomes that genuinely reflect real-world settings and concerns. However, many trials continue to measure and report outcomes that fall short of this clear requirement. We highlight problems with trial outcomes that make evidence difficult or impossible to interpret and that undermine the translation of research into practice and policy. These complex issues include the use of surrogate, composite and subjective endpoints; a failure to take account of patients' perspectives when designing research outcomes; publication and other outcome reporting biases, including the under-reporting of adverse events; the reporting of relative measures at the expense of more informative absolute outcomes; misleading reporting; multiplicity of outcomes; and a lack of core outcome sets. Trial outcomes can be developed with patients in mind, however, and can be reported completely, transparently and competently. Clinicians, patients, researchers and those who pay for health services are entitled to demand reliable evidence demonstrating whether interventions improve patient-relevant clinical outcomes.

Accounting for treatment by center interaction in sample size determinations and the use of surrogate outcomes in the pessary for the prevention of preterm birth trial: a simulation study

Background

The Pessary for the Prevention of Preterm Birth Study (PS3) is an international, multicenter, randomized clinical trial designed to examine the effectiveness of the Arabin pessary in preventing preterm birth in pregnant women with a short cervix. During the design of the study two methodological issues regarding power and sample size were raised. Since treatment in the Standard Arm will vary between centers, it is anticipated that so too will the probability of preterm birth in that arm. This will likely result in a treatment by center interaction, and the issue of how this will affect the sample size requirements was raised. The sample size requirements to examine the effect of the pessary on the baby’s clinical outcome was prohibitively high, so the second issue is how best to examine the effect on clinical outcome. The approaches taken to address these issues are presented.

Results

Simulation and sensitivity analysis were used to address the sample size issue. The probability of preterm birth in the Standard Arm was assumed to vary between centers following a Beta distribution with a mean of 0.3 and a coefficient of variation of 0.3. To address the second issue a Bayesian decision model is proposed that combines the information regarding the between-treatment difference in the probability of preterm birth from PS3 with the data from the Multiple Courses of Antenatal Corticosteroids for Preterm Birth Study that relate preterm birth and perinatal mortality/morbidity. The approach provides a between-treatment comparison with respect to the probability of a bad clinical outcome. The performance of the approach was assessed using simulation and sensitivity analysis.

Accounting for a possible treatment by center interaction increased the sample size from 540 to 700 patients per arm for the base case. The sample size requirements increase with the coefficient of variation and decrease with the number of centers. Under the same assumptions used for determining the sample size requirements, the simulated mean probability that pessary reduces the risk of perinatal mortality/morbidity is 0.98. The simulated mean decreased with coefficient of variation and increased with the number of clinical sites.

Conclusion

Employing simulation and sensitivity analysis is a useful approach for determining sample size requirements while accounting for the additional uncertainty due to a treatment by center interaction. Using a surrogate outcome in conjunction with a Bayesian decision model is an efficient way to compare important clinical outcomes in a randomized clinical trial in situations where the direct approach requires a prohibitively high sample size.