Statistical issues in the validation of prognostic, predictive, and surrogate biomarkers

Development and validation of a novel combinational index of liquid biopsy biomarker for longitudinal lung cancer patient management

Objectives

Many cancer biomarkers such as the circulating tumor cells/microemboli (CTCs/CTM) have been reported significant associations with clinical outcomes. However, different biomarkers have different sensitivities and specificities for cancer types and cohort patients, and synergistic effects between certain biomarkers have also been observed, leading to the inaccurate, fluctuating, and even controversial results when multiple biomarkers are analyzed together. In this paper, a novel combinational index, P-score, was developed for monitoring and predicting the disease condition of lung cancer patients during follow-up visits.

Materials and methods

There were totally 13 return patients with 54 blood samples involved in this study to examine the number of CTC and CTM. Information from one group of 7 patients including 27 blood samples with published clinical data was employed to develop while those from another group of 4 patients containing 14 blood samples with unpublished clinical data were used to validate the P score in prediction. Enumerations were based on immunofluorescent staining images. Distributions of CTC/CTM and their frequencies in stratified patients were carefully examined and analyzed the ROC curve and AUC value to develop the P score and P score-based prediction model.

Results and conclusion

We found that the predictive power of P-score was not only comparable to the traditional cancer marker, in comparison with individual CTC/CTM, more false positives could be corrected by using P-score, thereby to improve the accuracy of analysis. From our preliminary validation tests, the prognosis and disease progression monitored longitudinally by P-score were further confirmed by clinical outcome data from physicians and its sensitivity was even better than those from individual biomarkers. We believe that this novel combinational indicator could be a promising tool to interpret clinical outcomes more accurately from multiple factors, particularly useful for the early prognosis and longitudinal monitoring in cancer patient management.

Criteria for assessing evidence for biomarker-targeted therapies in rare cancers-an extrapolation framework

Background

Advances in targeted therapy development and tumor sequencing technology are reclassifying cancers into smaller biomarker-defined diseases. Randomized controlled trials (RCTs) are often impractical in rare diseases, leading to calls for single-arm studies to be sufficient to inform clinical practice based on a strong biological rationale. However, without RCTs, favorable outcomes are often attributed to therapy but may be due to a more indolent disease course or other biases. When the clinical benefit of targeted therapy in a common cancer is established in RCTs, this benefit may extend to rarer cancers sharing the same biomarker. However, careful consideration of the appropriateness of extending the existing trial evidence beyond specific cancer types is required. A framework for extrapolating evidence for biomarker-targeted therapies to rare cancers is needed to support transparent decision-making.

Objectives

To construct a framework outlining the breadth of criteria essential for extrapolating evidence for a biomarker-targeted therapy generated from RCTs in common cancers to different rare cancers sharing the same biomarker.

Design

A series of questions articulating essential criteria for extrapolation.

Methods

The framework was developed from the core topics for extrapolation identified from a previous scoping review of methodological guidance. Principles for extrapolation outlined in guidance documents from the European Medicines Agency, the US Food and Drug Administration, and Australia's Medical Services Advisory Committee were incorporated.

Results

We propose a framework for assessing key assumptions of similarity of the disease and treatment outcomes between the common and rare cancer for five essential components: prognosis of the biomarker-defined cancer, biomarker test analytical validity, biomarker actionability, treatment efficacy, and safety. Knowledge gaps identified can be used to prioritize future studies.

Conclusion

This framework will allow systematic assessment, standardize regulatory, reimbursement and clinical decision-making, and facilitate transparent discussions between key stakeholders in drug assessment for rare biomarker-defined cancers.

Biomarkers: how to consolidate them in clinical practice

An inadequate biomarker validation can affect many patients' diagnosis, treatment, and follow-up. Therefore, special interest should be placed on performing these analyses correctly so that biomarkers can be applicable to patients and evidence of their clinical usefulness can be generated. A methodological work on the concept of biomarkers is presented, as well as the difficulties associated with the methodological approach to their development, validation, and implementation in clinical practice.

Olaparib Addition to Maintenance Bevacizumab Therapy in Ovarian Carcinoma With BRCA-Like Genomic Aberrations

Importance

Testing for homologous recombination deficiency is required for the optimal treatment of high-grade epithelial ovarian cancer. The search for accurate biomarkers is ongoing.

Objective

To investigate whether progression-free survival (PFS) and overall survival (OS) of patients with high-grade epithelial ovarian cancer treated with maintenance olaparib or placebo differed between patients with a tumor BRCA-like genomic profile and patients without a tumor BRCA-like profile.

Design, Setting, and Participants

This cohort study was a secondary analysis of the PAOLA-1 randomized clinical trial that compared olaparib plus bevacizumab with placebo plus bevacizumab as maintenance treatment in patients with advanced high-grade ovarian cancer after a good response to first-line platinum with taxane chemotherapy plus bevacizumab, irrespective of germline or tumor BRCA1/2 mutation status. All patients with available tumor DNA were included in the analysis. The current analysis tested for an interaction between BRCA-like status and olaparib treatment on survival outcomes. The original trial was conducted between July 2015 and September 2017; at the time of data extraction for analysis in March 2022, a median follow-up of 54.1 months (IQR, 28.5-62.2 months) and a total follow-up time of 21 711 months was available, with 336 PFS and 245 OS events.

Exposures

Tumor homologous recombination deficiency was assessed using the BRCA-like copy number aberration profile classifier. Myriad MyChoice CDx was previously measured. The trial was randomized between the olaparib and bevacizumab and placebo plus bevacizumab groups.

Main Outcomes and Measures

This secondary analysis assessed hazard ratios (HRs) of olaparib vs placebo among biomarker strata and tested for interaction between BRCA-like status and olaparib treatment on PFS and OS, using Cox proportional hazards regression.

Results

A total of 469 patients (median age, 60 [range 26-80] years) were included in this study. The patient cohort consisted of women with International Federation of Gynaecology and Obstetrics stage III (76%) high-grade serous (95%) ovarian cancer who had no evaluable disease or complete remission at initial or interval debulking surgery (76%). Thirty-one percent of the tumor samples (n = 138) harbored a pathogenic BRCA mutation, and BRCA-like classification was performed for 442 patients. Patients with a BRCA-like tumor had a longer PFS after olaparib treatment than after placebo (36.4 vs 18.6 months; HR, 0.49; 95% CI, 0.37-0.65; P < .001). No association of olaparib with PFS was found in patients with a non-BRCA-like tumor (17.6 vs 16.6 months; HR, 1.02; 95% CI, 0.68-1.51; P = .93). The interaction was significant (P = .004), and HRs and P values (for interaction) were similar in the relevant subgroups, OS, and multivariable analyses.

Conclusions and Relevance

In this secondary analysis of the PAOLA-1 randomized clinical trial, patients with a BRCA-like tumor, but not those with a non-BRCA-like tumor, had a significantly longer survival after olaparib plus bevacizumab treatment than placebo plus bevacizumab treatment. Thus, the BRCA1-like classifier could be used as a biomarker for olaparib plus bevacizumab as a maintenance treatment.

Meaningful response criteria for myelodysplastic syndromes

Standardizing response criteria for myelodysplastic syndromes (MDS), a clinically and biologically heterogeneous group of disorders, has been historically challenging. The International Working Group (IWG) response criteria, first proposed in 2000 and modified in 2006 and 2018, represent the best effort by a group of international experts to define a set of clinically meaningful end-points in MDS. These criteria have been adopted in many MDS clinical trials, allowing for comparisons of response across trials. However, clinical experience has also revealed some limitations of these criteria, and most of the end-points proposed by the IWG require further validation. In this review, we present a critical analysis of the current MDS response criteria from both a practical standpoint and based on currently available clinical trial data. Potential areas for improvement in the criteria are highlighted, which may be considered in future iterations of the response criteria.

Circulating Biomarkers in Migraine: New Opportunities for Precision Medicine

Background:

Migraine is the most common neurological disorder and the second most disabling human condition, whose pathogenesis is favored by a combination of genetic, epigenetic, and environmental factors. In recent years, several efforts have been made to identify reliable biomarker(s) useful to monitor disease activity and/or ascertain the response to a specific treatment.

Objective:

To review the current evidence on the potential biological markers associated with migraine.

Methods:

A structured search of peer-reviewed research literature was performed by searching major publications databases up to December 2017.

Results:

Several circulating biomarkers have been proposed as diagnostic or therapeutic tools in migraine, mostly related to migraine’s inflammatory pathophysiological aspects. Nonetheless, their detection is still a challenge for the scientific community, reflecting, at least in part, disease complexity and clinical diagnostic limitations. At the present time, calcitonin generelated peptide (CGRP) represents probably the most promising candidate as a diagnostic and/or therapeutic biomarker, as its plasma levels are elevated during migraine attack and decrease during successful treatment. Other molecules (including some neuropeptides, cytokines, adipokines, or vascular activation markers) despite promising, do not possess the sufficient prerequisites to be considered as migraine biomarkers.

Conclusion:

The characterization of migraine-specific biomarkers would be fundamental in a perspective of precision medicine, enabling risk assessment and tailored treatments. However, speculating on the clinical validity of migraine biomarkers may be premature and controlled clinical trials are presently needed to investigate both the diagnostic and therapeutic value of these biomarkers in migraine.

Choosing primary endpoints for clinical trials of health care interventions

The purpose of late phase clinical trials is to generate evidence of sufficient validity and generalisability to be translated into practice and policy to improve health outcomes. It is therefore crucial that the chosen endpoints are meaningful to the clinicians, patients and policymakers that are the end-users of evidence generated by these trials. The choice of endpoints may be improved by understanding their characteristics and properties. This narrative review describes the evolution, range and relative strengths and weaknesses of endpoints used in late phase trials. It is intended to serve as a reference to assist those designing trials when choosing primary endpoint(s), and for the end-users charged with interpreting these trials to inform practice and policy.

Power estimation in biomarker studies where events are already observed

Background The clinical utility of a new biomarker should ideally be established in a prospective randomized clinical trial. However, such trials are not always practical. As such, it is common for investigators to identify promising biomarkers using archived specimens and clinical data collected from previously completed therapeutic trials. Simon et al. defined such biomarker studies as prospective-retrospective studies and proposed specific conditions to satisfy for such evaluations to be more than hypothesis generating. One condition they proposed is that archived tissues must be available on a sufficiently large number of patients from the pivotal trials to ensure adequately powered analyses. Purpose The purpose of this article is to provide a new perspective on how to estimate power for assessing the prognostic and predictive values of a single binary biomarker in prospective-retrospective biomarker studies. Computer programs are provided to facilitate the use of these methods in practice. Methods The proposed methods utilize additional information that becomes available during the course of the treatment trial including sample size, accrual time, additional follow-up time, and the observed number of events at time of biomarker analysis. These methods involve solving for the exponential hazard rates that give rise to the event numbers that are consistent with those observed while satisfying other design parameter constraints. Conclusion Simon et al. proposed a new paradigm for biomarker design, conduct, analysis, and evaluation in prospective-retrospective studies that offers a more efficient alternative than fully prospective biomarker studies. As a general rule, they suggest that samples from at least two-thirds of the patients be available for analysis. In this article, I expand on this issue and provide a methodological tool useful for estimating study power in prospective-retrospective biomarker studies. It is my hope that these incremental efforts to improve the quality and statistical rigor in biomarker studies will hasten the introduction of useful tumor biomarkers into clinical practice.

Biomarkers in Veterinary Medicine

This article summarizes the relevant definitions related to biomarkers; reviews the general processes related to biomarker discovery and ultimate acceptance and use; and finally summarizes and reviews, to the extent possible, examples of the types of biomarkers used in animal species within veterinary clinical practice and human and veterinary drug development. We highlight opportunities for collaboration and coordination of research within the veterinary community and leveraging of resources from human medicine to support biomarker discovery and validation efforts for veterinary medicine.

Evaluating Markers for Guiding Treatment

BACKGROUND

The subpopulation treatment effect pattern plot (STEPP) is an appealing method for assessing the clinical impact of a predictive marker on patient outcomes and identifying a promising subgroup for further study. However, its original formulation lacked a decision analytic justification and applied only to a single marker.

METHODS

We derive a decision-analytic result that motivates STEPP. We discuss the incorporation of multiple predictive markers into STEPP using risk difference, cadit, and responders-only benefit functions.

RESULTS

Applying STEPP to data from a breast cancer treatment trial with multiple markers, we found that none of the three benefit functions identified a promising subgroup for further study. Applying STEPP to hypothetical data from a trial with 100 markers, we found that all three benefit functions identified promising subgroups as evidenced by the large statistically significant treatment effect in these subgroups.

CONCLUSIONS

Because the method has desirable decision-analytic properties and yields an informative plot, it is worth applying to randomized trials on the chance there is a large treatment effect in a subgroup determined by the predictive markers.

Evaluating surrogate endpoints, prognostic markers, and predictive markers: Some simple themes

BACKGROUND

A surrogate endpoint is an endpoint observed earlier than the true endpoint (a health outcome) that is used to draw conclusions about the effect of treatment on the unobserved true endpoint. A prognostic marker is a marker for predicting the risk of an event given a control treatment; it informs treatment decisions when there is information on anticipated benefits and harms of a new treatment applied to persons at high risk. A predictive marker is a marker for predicting the effect of treatment on outcome in a subgroup of patients or study participants; it provides more rigorous information for treatment selection than a prognostic marker when it is based on estimated treatment effects in a randomized trial.

METHODS

We organized our discussion around a different theme for each topic.

RESULTS

"Fundamentally an extrapolation" refers to the non-statistical considerations and assumptions needed when using surrogate endpoints to evaluate a new treatment. "Decision analysis to the rescue" refers to use the use of decision analysis to evaluate an additional prognostic marker because it is not possible to choose between purely statistical measures of marker performance. "The appeal of simplicity" refers to a straightforward and efficient use of a single randomized trial to evaluate overall treatment effect and treatment effect within subgroups using predictive markers.

CONCLUSION

The simple themes provide a general guideline for evaluation of surrogate endpoints, prognostic markers, and predictive markers.

Regularized outcome weighted subgroup identification for differential treatment effects

To facilitate comparative treatment selection when there is substantial heterogeneity of treatment effectiveness, it is important to identify subgroups that exhibit differential treatment effects. Existing approaches model outcomes directly and then define subgroups according to interactions between treatment and covariates. Because outcomes are affected by both the covariate-treatment interactions and covariate main effects, direct modeling outcomes can be hard due to model misspecification, especially in presence of many covariates. Alternatively one can directly work with differential treatment effect estimation. We propose such a method that approximates a target function whose value directly reflects correct treatment assignment for patients. The function uses patient outcomes as weights rather than modeling targets. Consequently, our method can deal with binary, continuous, time-to-event, and possibly contaminated outcomes in the same fashion. We first focus on identifying only directional estimates from linear rules that characterize important subgroups. We further consider estimation of comparative treatment effects for identified subgroups. We demonstrate the advantages of our method in simulation studies and in analyses of two real data sets.

Adaptive designs for subpopulation analysis optimizing utility functions

If the response to treatment depends on genetic biomarkers, it is important to identify predictive biomarkers that define (sub-)populations where the treatment has a positive benefit risk balance. One approach to determine relevant subpopulations are subgroup analyses where the treatment effect is estimated in biomarker positive and biomarker negative groups. Subgroup analyses are challenging because several types of risks are associated with inference on subgroups. On the one hand, by disregarding a relevant subpopulation a treatment option may be missed due to a dilution of the treatment effect in the full population. Furthermore, even if the diluted treatment effect can be demonstrated in an overall population, it is not ethical to treat patients that do not benefit from the treatment when they can be identified in advance. On the other hand, selecting a spurious subpopulation increases the risk to restrict an efficacious treatment to a too narrow fraction of a potential benefiting population. We propose to quantify these risks with utility functions and investigate nonadaptive study designs that allow for inference on subgroups using multiple testing procedures as well as adaptive designs, where subgroups may be selected in an interim analysis. The characteristics of such adaptive and nonadaptive designs are compared for a range of scenarios.

Biomarker validation: common data analysis concerns

Biomarker validation, like any other confirmatory process based on statistical methodology, must discern associations that occur by chance from those reflecting true biological relationships. Validity of a biomarker is established by authenticating its correlation with clinical outcome. Validated biomarkers can lead to targeted therapy, improve clinical diagnosis, and serve as useful prognostic and predictive factors of clinical outcome. Statistical concerns such as confounding and multiplicity are common in biomarker validation studies. This article discusses four major areas of concern in the biomarker validation process and some of the proposed solutions. Because present-day statistical packages enable the researcher to address these common concerns, the purpose of this discussion is to raise awareness of these statistical issues in the hope of improving the reproducibility of validation study findings.

Biomarker enrichment strategies: matching trial design to biomarker credentials

The use of biomarkers to identify patients who can benefit from treatment with a specific anticancer agent has the potential to both improve patient care and accelerate drug development. The development of targeted agents and their accompanying biomarkers frequently occurs contemporaneously, and confidence in a putative biomarker's performance might, therefore, be insufficient to restrict the definitive testing of a new agent to the subgroup of biomarker-positive patients. This Review considers which clinical trial designs and analysis strategies are appropriate for use in phase III, biomarker-driven, randomized clinical trials, on the basis of pre-existing evidence that the biomarker can successfully identify patients who will respond to the treatment in question. The types of interim monitoring that are appropriate for these trials are also discussed. In addition, enrichment strategies based on the use of prognostic biomarkers to separate a population into subgroups with better and worse outcomes, regardless of treatment, are described. Finally, the possibility of formally using a biomarker during phase II drug development, to select what type of biomarker-driven strategy should be used in the phase III trial, is discussed.