An approach to evaluating and comparing biomarkers for patient treatment selection

Nonparametric Biomarker Based Treatment Selection With Reproducibility Data

We consider evaluating biomarkers for treatment selection under assay modification. Survival outcome, treatment, and Affymetrix gene expression data were attained from cancer patients. Consider migrating a gene expression biomarker to the Illumina platform. A recent novel approach allows a quick evaluation of the migrated biomarker with only a reproducibility study needed to compare the two platforms, achieved by treating the original biomarker as an error-contaminated observation of the migrated biomarker. However, its assumptions of a classical measurement error model and a linear predictor for the outcome may not hold. Ignoring such model deviations may lead to sub-optimal treatment selection or failure to identify effective biomarkers. To overcome such limitations, we adopt a nonparametric logistic regression to model the relationship between the event rate and the biomarker, and the deduced marker-based treatment selection is optimal. We further assume a nonparametric relationship between the migrated and original biomarkers and show that the error-contaminated biomarker leads to sub-optimal treatment selection compared to the error-free biomarker. We obtain the estimation via B-spline approximation. The approach is assessed by simulation studies and demonstrated through application to lung cancer data.

Physicians' Hierarchy of Tumor Biomarkers for Optimizing Chemotherapy in Breast Cancer Care

BACKGROUND

Tumor biomarkers are regularly used to guide breast cancer treatment and clinical trial enrollment. However, there remains a lack of knowledge regarding physicians' perspectives towards biomarkers and their role in treatment optimization, where treatment intensity is reduced to minimize toxicity.

METHODS

Thirty-nine academic and community oncologists participated in semi-structured qualitative interviews, providing perspectives on optimization approaches to chemotherapy treatment. Interviews were audio-recorded, transcribed, and analyzed by 2 independent coders utilizing a constant comparative method in NVivo. Major themes and exemplary quotes were extracted. A framework outlining physicians' conception of biomarkers, and their comfortability with their use in treatment optimization, was developed.

RESULTS

In the hierarchal model of biomarkers, level 1 is comprised of standard-of-care (SoC) biomarkers, defined by a strong level of evidence, alignment with national guidelines, and widespread utilization. Level 2 includes SoC biomarkers used in alternative contexts, in which physicians expressed confidence, yet less certainty, due to a lack of data in certain subgroups. Level 3, or experimental, biomarkers created the most diverse concerns related to quality and quantity of evidence, with several additional modulators.

CONCLUSION

This study demonstrates that physicians conceptualize the use of biomarkers for treatment optimization in successive levels. This hierarchy can be used to guide trialists in the development of novel biomarkers and design of future trials.

Finding the (biomarker-defined) subgroup of patients who benefit from a novel therapy: No time for a game of hide and seek

An important element of precision medicine is the ability to identify, for a specific therapy, those patients for whom benefits of that therapy meaningfully exceed the risks. To achieve this goal, treatment effect usually is examined across subgroups defined by a variety of factors, including demographic, clinical, or pathologic characteristics or by molecular attributes of patients or their disease. Frequently such subgroups are defined by the measurement of biomarkers. Even though such examination is necessary when pursuing this goal, the evaluation of treatment effect across a variety of subgroups is statistically fraught due to both the danger of inflated false-positive error rate from multiple testing and the inherent insensitivity to how treatment effects differ across subgroups.Pre-specification of subgroup analyses with appropriate control of false-positive (i.e. type I) error is recommended when possible. However, when subgroups are specified by biomarkers, which could be measured by different assays and might lack established interpretation criteria, such as cut-offs, it might not be possible to fully specify those subgroups at the time a new therapy is ready for definitive evaluation in a Phase 3 trial. In these situations, further refinement and evaluation of treatment effect in biomarker-defined subgroups might have to take place within the trial. A common scenario is that evidence suggests that treatment effect is a monotone function of a biomarker value, but optimal cut-offs for therapy decisions are not known. In this setting, hierarchical testing strategies are widely used, where testing is first conducted in a particular biomarker-positive subgroup and then is conducted in the expanded pool of biomarker-positive and biomarker-negative patients, with control for multiple testing. A serious limitation of this approach is the logical inconsistency of excluding the biomarker-negatives when evaluating effects in the biomarker-positives, yet allowing the biomarker-positives to drive the assessment of whether a conclusion of benefit could be extrapolated to the biomarker-negative subgroup.Examples from oncology and cardiology are described to illustrate the challenges and pitfalls. Recommendations are provided for statistically valid and logically consistent subgroup testing in these scenarios as alternatives to reliance on hierarchical testing alone, and approaches for exploratory assessment of continuous biomarkers as treatment effect modifiers are discussed.

A scoping review of statistical methods in studies of biomarker-related treatment heterogeneity for breast cancer

BACKGROUND

Many scientific papers are published each year and substantial resources are spent to develop biomarker-based tests for precision oncology. However, only a handful of tests is currently used in daily clinical practice, since development is challenging. In this situation, the application of adequate statistical methods is essential, but little is known about the scope of methods used.

METHODS

A PubMed search identified clinical studies among women with breast cancer comparing at least two different treatment groups, one of which chemotherapy or endocrine treatment, by levels of at least one biomarker. Studies presenting original data published in 2019 in one of 15 selected journals were eligible for this review. Clinical and statistical characteristics were extracted by three reviewers and a selection of characteristics for each study was reported.

RESULTS

Of 164 studies identified by the query, 31 were eligible. Over 70 different biomarkers were evaluated. Twenty-two studies (71%) evaluated multiplicative interaction between treatment and biomarker. Twenty-eight studies (90%) evaluated either the treatment effect in biomarker subgroups or the biomarker effect in treatment subgroups. Eight studies (26%) reported results for one predictive biomarker analysis, while the majority performed multiple evaluations, either for several biomarkers, outcomes and/or subpopulations. Twenty-one studies (68%) claimed to have found significant differences in treatment effects by biomarker level. Fourteen studies (45%) mentioned that the study was not designed to evaluate treatment effect heterogeneity.

CONCLUSIONS

Most studies evaluated treatment heterogeneity via separate analyses of biomarker-specific treatment effects and/or multiplicative interaction analysis. There is a need for the application of more efficient statistical methods to evaluate treatment heterogeneity in clinical studies.

Pool adjacent violators algorithm-assisted learning with application on estimating optimal individualized treatment regimes

Personalized medicine allows individuals to choose the best fit of their treatments based on their characteristics through an individualized treatment regime. In this paper, we develop a pool adjacent violators algorithm-assisted learning method to find the optimal individualized treatment regime under the monotone single-index outcome gain model. The proposed estimator is more efficient than peers, and it is robust to the misspecification of the propensity score model or the baseline regression model. The optimal treatment regime is also robust to the misspecification of the functional form of the expected outcome gain model. Simulation studies verified our theoretical results. We also provide an estimate of the expected outcome gain model. Plotting the expected outcome gain versus an individual's characteristics index can visualize how significant the treatment effect is over the control. We apply the proposed method to an AIDS study.

A comparison of approaches for combining predictive markers for personalised treatment recommendations

Background

In the presence of heterogeneous treatment effects, it is desirable to divide patients into subgroups based on their expected response to treatment. This is formalised via a personalised treatment recommendation: an algorithm that uses biomarker measurements to select treatments. It could be that multiple, rather than single, biomarkers better predict these subgroups. However, finding the optimal combination of multiple biomarkers can be a difficult prediction problem.

Methods

We described three parametric methods for finding the optimal combination of biomarkers in a personalised treatment recommendation, using randomised trial data: a regression approach that models outcome using treatment by biomarker interactions; an approach proposed by Kraemer that forms a combined measure from individual biomarker weights, calculated on all treated and control pairs; and a novel modification of Kraemer’s approach that utilises a prognostic score to sample matched treated and control subjects. Using Monte Carlo simulations under multiple data-generating models, we compare these approaches and draw conclusions based on a measure of improvement under a personalised treatment recommendation compared to a standard treatment. The three methods are applied to data from a randomised trial of home-delivered pragmatic rehabilitation versus treatment as usual for patients with chronic fatigue syndrome (the FINE trial). Prior analysis of this data indicated some treatment effect heterogeneity from multiple, correlated biomarkers.

Results

The regression approach outperformed Kraemer’s approach across all data-generating scenarios. The modification of Kraemer’s approach leads to improved treatment recommendations, except in the case where there was a strong unobserved prognostic biomarker. In the FINE example, the regression method indicated a weak improvement under its personalised treatment recommendation algorithm.

Conclusions

The method proposed by Kraemer does not perform better than a regression approach for combining multiple biomarkers. All methods are sensitive to misspecification of the parametric models.

BioPETsurv: Methodology and open source software to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials

Biomarkers can be used to enrich a clinical trial for patients at higher risk for an outcome, a strategy termed "prognostic enrichment." Methodology is needed to evaluate biomarkers for prognostic enrichment of trials with time-to-event endpoints such as survival. Key considerations when considering prognostic enrichment include: clinical trial sample size; the number of patients one must screen to enroll the trial; and total patient screening costs and total per-patient trial costs. The Biomarker Prognostic Enrichment Tool for Survival Outcomes (BioPETsurv) is a suite of methods for estimating these elements to evaluate a prognostic enrichment biomarker and/or plan a prognostically enriched clinical trial with a time-to-event primary endpoint. BioPETsurv allows investigators to analyze data on a candidate biomarker and potentially censored survival times. Alternatively, BioPETsurv can simulate data to match a particular clinical setting. BioPETsurv's data simulator enables investigators to explore the potential utility of a prognostic enrichment biomarker for their clinical setting. Results demonstrate that both modestly prognostic and strongly prognostic biomarkers can improve trial metrics such as reducing sample size or trial costs. In addition to the quantitative analysis provided by BioPETsurv, investigators should consider the generalizability of trial results and evaluate the ethics of trial eligibility criteria. BioPETsurv is freely available as a package for the R statistical computing platform, and as a webtool at www.prognosticenrichment.com/surv.

Fit-For-Purpose PD-L1 Biomarker Testing For Patient Selection in Immuno-Oncology: Guidelines For Clinical Laboratories From the Canadian Association of Pathologists-Association Canadienne Des Pathologistes (CAP-ACP)

Since 2014, programmed cell death protein 1 (PD-1)/programmed cell death ligand 1 (PD-L1) checkpoint inhibitors have been approved by various regulatory agencies for the treatment of multiple cancers including melanoma, lung cancer, urothelial carcinoma, renal cell carcinoma, head and neck cancer, classical Hodgkin lymphoma, colorectal cancer, gastroesophageal cancer, hepatocellular cancer, and other solid tumors. Of these approved drug/disease combinations, a subset also has regulatory agency-approved, commercially available companion/complementary diagnostic assays that were clinically validated using data from their corresponding clinical trials. The objective of this document is to provide evidence-based guidance to assist clinical laboratories in establishing fit-for-purpose PD-L1 biomarker assays that can accurately identify patients with specific tumor types who may respond to specific approved immuno-oncology therapies targeting the PD-1/PD-L1 checkpoint. These recommendations are issued as 38 Guideline Statements that address (i) assay development for surgical pathology and cytopathology specimens, (ii) reporting elements, and (iii) quality assurance (including validation/verification, internal quality assurance, and external quality assurance). The intent of this work is to provide recommendations that are relevant to any tumor type, are universally applicable and can be implemented by any clinical immunohistochemistry laboratory performing predictive PD-L1 immunohistochemistry testing.

Evaluating biomarkers for treatment selection from reproducibility studies

We consider evaluating new or more accurately measured predictive biomarkers for treatment selection based on a previous clinical trial involving standard biomarkers. Instead of rerunning the clinical trial with the new biomarkers, we propose a more efficient approach which requires only either conducting a reproducibility study in which the new biomarkers and standard biomarkers are both measured on a set of patient samples, or adopting replicated measures of the error-contaminated standard biomarkers in the original study. This approach is easier to conduct and much less expensive than studies that require new samples from patients randomized to the intervention. In addition, it makes it possible to perform the estimation of the clinical performance quickly, since there will be no requirement to wait for events to occur as would be the case with prospective validation. The treatment selection is assessed via a working model, but the proposed estimator of the mean restricted lifetime is valid even if the working model is misspecified. The proposed approach is assessed through simulation studies and applied to a cancer study.

The area between ROC curves, a non-parametric method to evaluate a biomarker for patient treatment selection

Treatment selection markers are generally sought for when the benefit of an innovative treatment in comparison with a reference treatment is considered, and this benefit is suspected to vary according to the characteristics of the patients. Classically, such quantitative markers are detected through testing a marker-by-treatment interaction in a parametric regression model. Most alternative methods rely on modeling the risk of event occurrence in each treatment arm or the benefit of the innovative treatment over the marker values, but with assumptions that may be difficult to verify. Herein, a simple non-parametric approach is proposed to detect and assess the general capacity of a quantitative marker for treatment selection when no overall difference in efficacy could be demonstrated between two treatments in a clinical trial. This graphical method relies on the area between treatment-arm-specific receiver operating characteristic curves (ABC), which reflects the treatment selection capacity of the marker. A simulation study assessed the inference properties of the ABC estimator and compared them with other parametric and non-parametric indicators. The simulations showed that the estimate of the ABC had low bias, power comparable to parametric indicators, and that its confidence interval had a good coverage probability (better than the other non-parametric indicator in some cases). Thus, the ABC is a good alternative to parametric indicators. The ABC method was applied to data of the PETACC-8 trial that investigated FOLFOX4 versus FOLFOX4 + cetuximab in stage III colon adenocarcinoma. It enabled the detection of a treatment selection marker: the DDR2 gene.

On evaluating how well a biomarker can predict treatment response with survival data

One of the objectives of personalized medicine is to take treatment decisions based on a biomarker measurement. Therefore, it is often interesting to evaluate how well a biomarker can predict the response to a treatment. To do so, a popular methodology consists of using a regression model and testing for an interaction between treatment assignment and biomarker. However, the existence of an interaction is not sufficient for a biomarker to be predictive. It is only necessary. Hence, the use of the marker-by-treatment predictiveness curve has been recommended. In addition to evaluate how well a single continuous biomarker predicts treatment response, it can further help to define an optimal threshold. This curve displays the risk of a binary outcome as a function of the quantiles of the biomarker, for each treatment group. Methods that assume a binary outcome or rely on a proportional hazard model for a time-to-event outcome have been proposed to estimate this curve. In this work, we propose some extensions for censored data. They rely on a time-dependent logistic model, and we propose to estimate this model via inverse probability of censoring weighting. We present simulations results and three applications to prostate cancer, liver cirrhosis, and lung cancer data. They suggest that a large number of events need to be observed to define a threshold with sufficient accuracy for clinical usefulness. They also illustrate that when the treatment effect varies with the time horizon which defines the outcome, then the optimal threshold also depends on this time horizon.

Evaluating the impact of policies recommending PrEP to subpopulations of men and transgender women who have sex with men based on demographic and behavioral risk factors

INTRODUCTION

Developing guidelines to inform the use of antiretroviral pre-exposure prophylaxis (PrEP) for HIV prevention in resource-limited settings must necessarily be informed by considering the resources and infrastructure needed for PrEP delivery. We describe an approach that identifies subpopulations of cisgender men who have sex with men (MSM) and transgender women (TGW) to prioritize for the rollout of PrEP in resource-limited settings.

METHODS

We use data from the iPrEx study, a multi-national phase III study of PrEP for HIV prevention in MSM/TGW, to build statistical models that identify subpopulations at high risk of HIV acquisition without PrEP, and with high expected PrEP benefit. We then evaluate empirically the population impact of policies recommending PrEP to these subpopulations, and contrast these with existing policies.

RESULTS

A policy recommending PrEP to a high risk subpopulation of MSM/TGW reporting condomless receptive anal intercourse over the last 3 months (estimated 3.3% 1-year HIV incidence) yields an estimated 1.95% absolute reduction in 1-year HIV incidence at the population level, and 3.83% reduction over 2 years. Importantly, such a policy requires rolling PrEP out to just 59.7% of MSM/TGW in the iPrEx population. We find that this policy is identical to that which prioritizes MSM/TGW with high expected PrEP benefit. It is estimated to achieve nearly the same reduction in HIV incidence as the PrEP guideline put forth by the US Centers for Disease Control, which relies on the measurement of more behavioral risk factors and which would recommend PrEP to a larger subset of the MSM/TGW population (86% vs. 60%).

CONCLUSIONS

These findings may be used to focus future mathematical modelling studies of PrEP in resource-limited settings on prioritizing PrEP for high-risk subpopulations of MSM/TGW. The statistical approach we took could be employed to develop PrEP policies for other at-risk populations and resource-limited settings.

Constructing treatment selection rules based on an estimated treatment effect function: different approaches to take stochastic uncertainty into account have a substantial effect on performance

BACKGROUND

Today we are often interested in the predictive value of a continuous marker with respect to the expected difference in outcome between a new treatment and a standard treatment. We can investigate this in a randomized control trial, allowing us to assess interactions between treatment and marker and to construct a treatment selection rule. A first step is often to estimate the treatment effect as a function of the marker value. A variety of approaches have been suggested for the second step to define explicitly the rule to select the treatment, varying in the way to take uncertainty into account. Little is known about the merits of the different approaches.

METHODS

Four construction principles for the second step are compared. They are based on the root of the estimated function, on confidence intervals for the root, or on pointwise or simultaneous confidence bands. All of them have been used implicitly or explicitly in the literature. As performance characteristics we consider the probability to select at least some patients, the probability to classify patients with and without a benefit correctly, and the gain in expected outcome at the population level. These characteristics are investigated in a simulation study.

RESULTS

As to be expected confidence interval/band based approaches reduce the risk to select patients who do not benefit from the new treatment, but they tend to overlook patients who can benefit. Simply using positivity of the estimated treatment effect function for selection implies often a larger gain in expected outcome.

CONCLUSIONS

The use of 95% confidence intervals/bands in constructing treatment selection rules is a rather conservative approach. There is a need for better construction principles for treatment selection rules aiming to maximize the gain in expected outcome at the population level. Choosing a confidence level of 80% may be a first step in this direction.

A Bayesian method to estimate the optimal threshold of a marker used to select patients' treatment

The use of a quantitative treatment selection marker to choose between two treatment options requires the estimate of an optimal threshold above which one of these two treatments is preferred. Herein, the optimal threshold expression is based on the definition of a utility function which aims to quantify the expected utility of the population (e.g. life expectancy, quality of life) by taking into account both efficacy (success or failure) and toxicity of each treatment option. Therefore, the optimal threshold is the marker value that maximizes the expected utility of the population. A method modelling the marker distribution in patient subgroups defined by the received treatment and the outcome is proposed to calculate the parameters of the utility function so as to estimate the optimal threshold and its 95% credible interval using the Bayesian inference. The simulation study found that the method had low bias and coverage probability close to 95% in multiple settings, but also the need of large sample size to estimate the optimal threshold in some settings. The method is then applied to the PETACC-8 trial that compares the efficacy of chemotherapy with a combined chemotherapy + anti-epidermal growth factor receptor in stage III colorectal cancer.

Identifying treatment responders using counterfactual modeling and potential outcomes

Individualizing treatment according to patients' characteristics is central for personalized or precision medicine. There has been considerable recent research in developing statistical methods to determine optimal personalized treatment strategies by modeling the outcome of patients according to relevant covariates under each of the alternative treatments, and then relying on so-called predicted individual treatment effects. In this paper, we use potential outcomes and principal stratification frameworks and develop a multinomial model for left and right-censored data to estimate the probability that a patient is a responder given a set of baseline covariates. The model can apply to RCT or observational study data. This method is based on the monotonicity assumption, which implies that no patients would respond to the control treatment but not to the experimental one. We conduct a simulation study to evaluate the properties of the proposed estimation method. Results showed that the predictions of the probability of being a responder were well calibrated even if we observed variability and a small bias when many parameters were estimated. We finally applied the method to a cohort study on the selection of patients for additional radiotherapy after resection of a soft-tissue sarcoma.

Auxiliary variable-enriched biomarker-stratified design

Clinical trials in the era of precision medicine require assessment of biomarkers to identify appropriate subgroups of patients for targeted therapy. In a biomarker-stratified design (BSD), biomarkers are measured on all patients and used as stratification variables. However, such a trial can be both inefficient and costly, especially when the prevalence of the subgroup of primary interest is low and the cost of assessing the biomarkers is high. Efficiency can be improved and costs reduced by using enriched biomarker-stratified designs, in which patients of primary interest, typically the biomarker-positive patients, are oversampled. We consider a special type of enrichment design, an auxiliary variable-enriched design (AEBSD), in which enrichment is based on some inexpensive auxiliary variable that is positively correlated with the true biomarker. The proposed AEBSD reduces the total cost of the trial compared with a standard BSD when the prevalence rate of true biomarker positivity is small and the positive predictive value (PPV) of the auxiliary biomarker is larger than the prevalence rate. In addition, for an AEBSD, we can immediately randomize the patients selected in the screening process without waiting for the result of the true biomarker test, reducing the treatment waiting time. We propose an adaptive Bayesian method to adjust the assumed PPV while the trial is ongoing. Numerical studies and an example illustrate the approach. An R package is available.

Estimating mean local posterior predictive benefit for biomarker-guided treatment strategies

Precision medicine has emerged from the awareness that many human diseases are intrinsically heterogeneous with respect to their pathogenesis and composition among patients as well as dynamic over the course therapy. Its successful application relies on our understanding of distinct molecular profiles and their biomarkers which can be used as targets to devise treatment strategies that exploit current understanding of the biological mechanisms of the disease. Precision medicine present challenges to traditional paradigms of clinical translational, however, for which estimates of population-averaged effects from large randomized trials are used as the basis for demonstrating improvements comparative effectiveness. A general approach for estimating the relative effectiveness of biomarker-guided therapeutic strategies is presented herein. The statistical procedure attempts to define the local benefit of a given biomarker-guided therapeutic strategy in consideration of the treatment response surfaces, selection rule, and inter-cohort balance of prognostic determinants. Theoretical and simulation results are provided. Additionally, the methodology is demonstrated through a proteomic study of lower grade glioma.

Development and evaluating multimarker models for guiding treatment decisions

Background

Despite the growing interest in developing markers for predicting treatment response and optimizing treatment decisions, an appropriate methodology to identify, combine and evaluate such markers has been slow to develop. We propose a step-by-step strategy for analysing data from existing randomised trials with the aim of identifying a multi-marker model for guiding decisions about treatment.

Methods

We start with formulating the treatment selection problem, continue with defining the treatment threshold, prepare a list of candidate markers, develop the model, apply the model to estimate individual treatment effects, and evaluate model performance in the study group of patients who meet the trial eligibility criteria. In this process, we rely on some well-known techniques for multivariable prediction modelling, but focus on predicting benefit from treatment, rather than outcome itself. We present our approach using data from a randomised trial in which 808 women with multiple pregnancy were assigned to cervical pessary or control, to prevent adverse perinatal outcomes. Overall, cervical pessary did not reduce the risk of adverse perinatal outcomes.

Results

The treatment threshold was zero. We had a preselected list of 5 potential markers and developed a logistic model including the markers, treatment and all marker-by-treatment interaction terms. The model was well calibrated and identified 35% (95% confidence interval (CI) 32 to 39%) of the trial participants as benefitting from pessary insertion. We estimated that the risk of adverse outcome could be reduced from 13.5 to 8.1% (5.4% risk reduction; 95% CI 2.1 to 8.6%) through model-based selective pessary insertion. The next step is external validation upon existence of independent trial data.

Conclusions

We suggest revisiting existing trials data to explore whether differences in treatment benefit can be explained by differences in baseline characteristics of patients. This could lead to treatment selection tools which, after validation in comparable existing trials, can be introduced into clinical practice for guiding treatment decisions in future patients.

Case-only approach to identifying markers predicting treatment effects on the relative risk scale

Retrospectively measuring markers on stored baseline samples from participants in a randomized controlled trial (RCT) may provide high quality evidence as to the value of the markers for treatment selection. Originally developed for approximating gene-environment interactions in the odds ratio scale, the case-only method has recently been advocated for assessing gene-treatment interactions on rare disease endpoints in randomized clinical trials. In this article, the case-only approach is shown to provide a consistent and efficient estimator of marker by treatment interactions and marker-specific treatment effects on the relative risk scale. The prohibitive rare-disease assumption is no longer needed, broadening the utility of the case-only approach. The case-only method is resource-efficient as markers only need to be measured in cases only. It eliminates the need to model the marker's main effect, and can be used with any parametric or nonparametric learning method. The utility of this approach is illustrated by an application to genetic data in the Women's Health Initiative (WHI) hormone therapy trial.

Evaluation of biomarkers for treatment selection using individual participant data from multiple clinical trials

Biomarkers that predict treatment effects may be used to guide treatment decisions, thus improving patient outcomes. A meta-analysis of individual participant data (IPD) is potentially more powerful than a single-study data analysis in evaluating markers for treatment selection. Our study was motivated by the IPD that were collected from 2 randomized controlled trials of hypertension and preeclampsia among pregnant women to evaluate the effect of labor induction over expectant management of the pregnancy in preventing progression to severe maternal disease. The existing literature on statistical methods for biomarker evaluation in IPD meta-analysis have evaluated a marker's performance in terms of its ability to predict risk of disease outcome, which do not directly apply to the treatment selection problem. In this study, we propose a statistical framework for evaluating a marker for treatment selection given IPD from a small number of individual clinical trials. We derive marker-based treatment rules by minimizing the average expected outcome across studies. The application of the proposed methods to the IPD from 2 studies in women with hypertension in pregnancy is presented.

Tree-based Ensemble Methods For Individualized Treatment Rules

There is a growing interest in development of statistical methods for personalized medicine or precision medicine, especially for deriving optimal individualized treatment rules (ITRs). An ITR recommends a patient to a treatment based on the patient's characteristics. The common parametric methods for deriving an optimal ITR, which model the clinical endpoint as a function of the patient's characteristics, can have suboptimal performance when the conditional mean model is misspecified. Recent methodology development has cast the problem of deriving optimal ITR under a weighted classification framework. Under this weighted classification framework, we develop a weighted random forests (W-RF) algorithm that derives an optimal ITR nonparametrically. In addition, with the W-RF algorithm, we propose the variable importance measures for quantifying relative relevance of the patient's characteristics to treatment selection, and the out-of-bag estimator for the population average outcome under the estimated optimal ITR. Our proposed methods are evaluated through intensive simulation studies. We illustrate the application of our methods using data from Clinical Antipsychotic Trials of Intervention Effectiveness Alzheimers Disease Study (CATIE-AD).

Quantitative assessment of a prognostic or predictive biomarker panel

Methods for assessing whether a single biomarker is prognostic or predictive in the context of a control and experimental treatment are well known. With a panel of biomarkers, each component biomarker potentially measuring sensitivity to a different drug, it is not obvious how to extend these methods. We consider two situations, which lead to different ways of defining whether a biomarker panel is prognostic or predictive. In one, there are multiple experimental targeted treatments, each with an associated biomarker assay of the relevant target in the panel, along with a control treatment; the extension of the single-biomarker scenario to this situation is straightforward. In the other situation, there are many (nontargeted) treatments and a single assay that can be used to assess the sensitivity of the patient's tumor to the different treatments. In addition to evaluating previous approaches to this situation, we propose using regression models with varying assumptions to assess such panel biomarkers. Missing biomarker data can be problematic with the regression models, and, after demonstrating that a multiple imputation procedure does not work, we suggest a modified regression model that can accommodate some forms of missing data. We also address the notions of qualitative interactions in the biomarker panel setting.

On Enrichment Strategies for Biomarker Stratified Clinical Trials

In the era of precision medicine, drugs are increasingly developed to target subgroups of patients with certain biomarkers. In large all-comer trials using a biomarker stratified design, the cost of treating and following patients for clinical outcomes may be prohibitive. With a fixed number of randomized patients, the efficiency of testing certain treatments parameters, including the treatment effect among biomarker-positive patients and the interaction between treatment and biomarker, can be improved by increasing the proportion of biomarker positives on study, especially when the prevalence rate of biomarker positives is low in the underlying patient population. When the cost of assessing the true biomarker is prohibitive, one can further improve the study efficiency by oversampling biomarker positives with a cheaper auxiliary variable or a surrogate biomarker that correlates with the true biomarker. To improve efficiency and reduce cost, we can adopt an enrichment strategy for both scenarios by concentrating on testing and treating patient subgroups that contain more information about specific treatment parameters of primary interest to the investigators. In the first scenario, an enriched biomarker stratified design enriches the cohort of randomized patients by directly oversampling the relevant patients with the true biomarker, while in the second scenario, an auxiliary-variable-enriched biomarker stratified design enriches the randomized cohort based on an inexpensive auxiliary variable, thereby avoiding testing the true biomarker on all screened patients and reducing treatment waiting time. For both designs, we discuss how to choose the optimal enrichment proportion when testing a single hypothesis or two hypotheses simultaneously. At a requisite power, we compare the two new designs with the BSD design in terms of the number of randomized patients and the cost of trial under scenarios mimicking real biomarker stratified trials. The new designs are illustrated with hypothetical examples for designing biomarker-driven cancer trials.

IVF or IUI as first-line treatment in unexplained subfertility: the conundrum of treatment selection markers

STUDY QUESTION

Are there treatment selection markers that could aid in identifying couples, with unexplained or mild male subfertility, who would have better chances of a healthy child with IVF with single embryo transfer (IVF-SET) than with IUI with ovarian stimulation (IUI-OS)?

SUMMARY ANSWER

We did not find any treatment selection markers that were associated with better chances of a healthy child with IVF-SET instead of IUI-OS in couples with unexplained or mild male subfertility.

WHAT IS KNOWN ALREADY

A recent trial, comparing IVF-SET to IUI-OS, found no evidence of a difference between live birth rates and multiple pregnancy rates. It was suggested that IUI-OS should remain the first-line treatment instead of IVF-SET in couples with unexplained or mild male subfertility and female age between 18 and 38 years. The question remains whether there are some couples that may have higher pregnancy chances if treated with IVF-SET instead of IUI.

STUDY DESIGN, SIZE, DURATION

We performed our analyses on data from the INeS trial, where couples with unexplained or mild male subfertility and an unfavourable prognosis for natural conception were randomly allocated to IVF-SET, IVF in a modified natural cycle or IUI-OS. In view of the aim of this study, we only used data of the comparison between IVF-SET (201 couples) and IUI-OS (207 couples).

PARTICIPANTS/MATERIALS, SETTING, METHODS

We pre-defined the following baseline characteristics as potential treatment selection markers: female age, ethnicity, smoking status, type of subfertility (primary/secondary), duration of subfertility, BMI, pre-wash total motile count and Hunault prediction score. For each potential treatment selection marker, we explored the association with the chances of a healthy child after IVF-SET and IUI-OS and tested if there was an interaction with treatment. Given the exploratory nature of our analysis, we used a P-value of 0.1.

MAIN RESULTS AND THE ROLE OF CHANCE

None of the markers were associated with higher chances of a healthy child from IVF-SET compared to IUI-OS (P-value for interaction >0.10).

LIMITATIONS, REASONS FOR CAUTION

Since this is the first large study that looked at potential treatment selection markers for IVF-SET compared to IUI-OS, we had no data on which to base a power calculation. The sample size was limited, making it difficult to detect any smaller associations.

WIDER IMPLICATIONS OF THE FINDINGS

We could not identify couples with unexplained or mild male subfertility who would have had higher chances of a healthy child from immediate IVF-SET than from IUI-OS. As in the original trial IUI-OS had similar effectiveness and was less costly compared to IVF-SET, IUI-OS should remain the preferred first-line treatment in these couples.

STUDY FUNDING/COMPETING INTEREST(S)

The study was supported by a grant from the Netherlands Organization for Health Research and Development, and a grant from the Netherlands' association of health care insurers. There are no conflicts of interest.

TRIAL REGISTRATION NUMBER

The trial was registered at the Dutch trial registry (NTR939).

Adjusting for covariates in evaluating markers for selecting treatment, with application to guiding chemotherapy for treating estrogen-receptor-positive, node-positive breast cancer

In many clinical contexts, biomarkers that predict treatment efficacy are highly sought after. Such treatment selection or predictive biomarkers have the potential to identify subgroups most likely to benefit from the treatment, and may therefore be used to improve clinical outcomes and reduce medical costs. A methodological challenge in evaluating these biomarkers is determining how to take into account other variables that predict clinical outcomes, or that influence the biomarker distribution, generically termed covariates. We distinguish between two questions that arise when evaluating markers in the context of covariates. First, what is the biomarker's added value for selecting treatment, over and above the covariates? Second, what is the marker's performance within covariate strata-does performance vary? We lay out statistical methodology for addressing each of these questions. We argue that the common approach of testing for the marker's statistical interaction with treatment, in the context of a multivariate model that includes the covariates as predictors, does not directly address either question. We illustrate the methodology in new analyses of the Oncotype DX Recurrence Score, a marker used to select adjuvant chemotherapy for the treatment of estrogen-receptor-positive breast cancer.

The SUPER study: protocol for a randomised controlled trial comparing follicle-stimulating hormone and clomiphene citrate for ovarian stimulation in intrauterine insemination

OBJECTIVE

To study the effectiveness of four cycles of intrauterine insemination (IUI) with ovarian stimulation (OS) by follicle-stimulating hormone (FSH) or by clomiphene citrate (CC), and adherence to strict cancellation criteria.

SETTING

Randomised controlled trial among 22 secondary and tertiary fertility clinics in the Netherlands.

PARTICIPANTS

732 women from couples diagnosed with unexplained or mild male subfertility and an unfavourable prognosis according to the model of Hunault of natural conception.

INTERVENTIONS

Four cycles of IUI-OS within a time horizon of 6 months comparing FSH 75 IU with CC 100 mg. The primary outcome is ongoing pregnancy conceived within 6 months after randomisation, defined as a positive heartbeat at 12 weeks of gestation. Secondary outcomes are cancellation rates, number of cycles with a monofollicular or with multifollicular growth, number of follicles >14 mm at the time of ovulation triggering, time to ongoing pregnancy, clinical pregnancy, miscarriage, live birth and multiple pregnancy. We will also assess if biomarkers such as female age, body mass index, smoking status, antral follicle count and endometrial aspect and thickness can be used as treatment selection markers.

ETHICS AND DISSEMINATION

The study has been approved by the Medical Ethical Committee of the Academic Medical Centre and from the Dutch Central Committee on Research involving Human Subjects (CCMO NL 43131-018-13). Results will be disseminated through peer-reviewed publications and presentations at international scientific meetings.

TRIAL REGISTRATION NUMBER

NTR4057.

Graphical Presentation of Patient-Treatment Interaction Elucidated by Continuous Biomarkers. Current Practice and Scope for Improvement

BACKGROUND

Biomarkers providing evidence for patient-treatment interaction are key in the development and practice of personalized medicine. Knowledge that a patient with a specific feature - as demonstrated through a biomarker - would have an advantage under a given treatment vs. a competing treatment can aid immensely in medical decision-making. Statistical strategies to establish evidence of continuous biomarkers are complex and their formal results are thus not easy to communicate. Good graphical representations would help to translate such findings for use in the clinical community. Although general guidelines on how to present figures in clinical reports are available, there remains little guidance for figures elucidating the role of continuous biomarkers in patient-treatment interaction (CBPTI).

OBJECTIVES

To combat the current lack of comprehensive reviews or adequate guides on graphical presentation within this topic, our study proposes presentation principles for CBPTI plots. In order to understand current practice, we review the development of CBPTI methodology and how CBPTI plots are currently used in clinical research.

METHODS

The quality of a CBPTI plot is determined by how well the presentation provides key information for clinical decision-making. Several criteria for a good CBPTI plot are proposed, including general principles of visual display, use of units presenting absolute outcome measures, appropriate quantification of statistical uncertainty, correct display of benchmarks, and informative content for answering clinical questions especially on the quantitative advantage for an individual patient with regard to a specific treatment. We examined the development of CBPTI methodology from the years 2000 - 2014, and reviewed how CBPTI plots were currently used in clinical research in six major clinical journals from 2013 - 2014 using the principle of theoretical saturation. Each CBPTI plot found was assessed for appropriateness of its presentation and clinical utility.

RESULTS

In our review, a total of seven methodological papers and five clinical reports used CBPTI plots which we categorized into four types: those that distinguish the outcome effect for each treatment group; those that show the outcome differences between treatment groups (by either partitioning all individuals into subpopulations or modelling the functional form of the interaction); those that evaluate the proportion of population impact of the biomarker; and those that show the classification accuracy of the biomarker. The current practice of utilizing CBPTI plots in clinical reports suffers from methodological shortcomings: the lack of presentation of statistical uncertainty, the outcome measure scaled by relative unit instead of absolute unit, incorrect use of benchmarks, and being non-informative in answering clinical questions.

CONCLUSIONS

There is considerable scope for improvement in the graphical representation of CBPTI in clinical reports. The current challenge is to develop instruments for high-quality graphical plots which not only convey quantitative concepts to readers with limited statistical knowledge, but also facilitate medical decision-making.

Value-based and benefit-based strategies in deciding to bring a test into use should be distinguished

BACKGROUND

Regulatory and health technology assessment agencies have commented differently on the question whether results from enrichment studies can be used to justify to bring a test into use. We try to provide a framework to discuss this issue.

RESULTS

Mathematical definitions for the value and the benefit of a new diagnostic test are given. The possible conclusions about value and benefit from enrichment studies and interaction studies are explored. The terms benefit-based strategy and value-based strategy are introduced. Several potential consequences of using one of the two strategies in deciding to bring a test into use are identified and quantified. Interaction designs allow to assess benefit and value. Enrichment designs allow only to assess benefit. However, it is often probable that interaction studies allow no firm conclusions about the value. The advantage of a benefit-based strategy stems mainly from allowing test-positive patients earlier or even ever to benefit. The main disadvantage is a potential delay in detecting tests of no value.

CONCLUSIONS

Benefit-based strategies are preferable if the risk of off-label use and of delayed decisions on the value of a test can be limited. Otherwise, the superiority depends highly on research practice.

A multivariable model to guide the decision for pessary placement to prevent preterm birth in women with a multiple pregnancy: a secondary analysis of the ProTWIN trial

OBJECTIVE

The ProTWIN Trial (NTR1858) showed that, in women with a multiple pregnancy and a cervical length < 25(th) percentile (38 mm), prophylactic use of a cervical pessary reduced the risk of adverse perinatal outcome. We investigated whether other maternal or pregnancy characteristics collected at baseline can improve identification of women most likely to benefit from pessary placement.

METHODS

ProTWIN is a multicenter randomized trial in which 808 women with a multiple pregnancy were assigned to pessary or control. Using these data we developed a multivariable logistic model comprising treatment, cervical length, chorionicity, pregnancy history and number of fetuses, and the interaction of these variables with treatment as predictors of adverse perinatal outcome.

RESULTS

Short cervix, monochorionicity and nulliparity were predictive factors for a benefit from pessary insertion. History of previous preterm birth and triplet pregnancy were predictive factors of possible harm from pessary. The model identified 35% of women as benefiting (95% CI, 32-39%), which is 10% more than using cervical length only (25%) for pessary decisions. The model had acceptable calibration. We estimated that using the model to guide the choice of pessary placement would reduce the risk of adverse perinatal outcome significantly from 13.5% when no pessary is inserted to 8.1% (absolute risk reduction, 5.4% (95% CI, 2.1-8.6%)).

CONCLUSIONS

We developed and internally validated a multivariable treatment selection model, with cervical length, chorionicity, pregnancy history and number of fetuses. If externally validated, it could be used to identify women with a twin pregnancy who would benefit from a pessary, and lead to a reduction in adverse perinatal outcomes in these women. Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.

Designing a study to evaluate the benefit of a biomarker for selecting patient treatment

Biomarkers that predict the efficacy of treatment can potentially improve clinical outcomes and decrease medical costs by allowing treatment to be provided only to those most likely to benefit. We consider the design of a randomized clinical trial in which one objective is to evaluate a treatment selection marker. The marker may be measured prospectively or retrospectively using samples collected at baseline. We describe and contrast criteria around which the trial can be designed. An existing approach focuses on determining if there is a statistical interaction between the marker and treatment. We propose three alternative approaches based on estimating clinically relevant measures of improvement in outcomes with use of the marker. Importantly, our approaches accommodate the common scenario in which the marker-based rule for recommending treatment is developed with data from the trial. Sample sizes are calculated for powering a trial to assess these criteria in the context of adjuvant chemotherapy for the treatment of estrogen-receptor-positive, node-positive breast cancer. In this example, we find that larger sample sizes are generally required for assessing clinical impact than for simply evaluating if there is a statistical interaction between marker and treatment. We also find that retrospectively selecting a case-control subset of subjects for marker evaluation can lead to large efficiency gains, especially if cases and controls are matched on treatment assignment.

Immune monitoring technology primer

Background

Recent biotechnological developments have resulted in increasing interest in immunology biomarkers. These biomarkers have potential clinical utility in the near future as predictors of treatment response. Hence, clinical validation of these predictive markers is critical.

Findings

The process of clinically validating a predictive biomarker is reviewed. Validation of a predictive biomarker requires quantifying the strength of a statistical interaction between marker and a treatment. Different study designs are considered.

Conclusions

Clinical validation of immunology biomarkers can be demanding both in terms of time and resources, and careful planning and study design are critical.

Evaluating surrogate endpoints, prognostic markers, and predictive markers: Some simple themes

BACKGROUND

A surrogate endpoint is an endpoint observed earlier than the true endpoint (a health outcome) that is used to draw conclusions about the effect of treatment on the unobserved true endpoint. A prognostic marker is a marker for predicting the risk of an event given a control treatment; it informs treatment decisions when there is information on anticipated benefits and harms of a new treatment applied to persons at high risk. A predictive marker is a marker for predicting the effect of treatment on outcome in a subgroup of patients or study participants; it provides more rigorous information for treatment selection than a prognostic marker when it is based on estimated treatment effects in a randomized trial.

METHODS

We organized our discussion around a different theme for each topic.

RESULTS

"Fundamentally an extrapolation" refers to the non-statistical considerations and assumptions needed when using surrogate endpoints to evaluate a new treatment. "Decision analysis to the rescue" refers to use the use of decision analysis to evaluate an additional prognostic marker because it is not possible to choose between purely statistical measures of marker performance. "The appeal of simplicity" refers to a straightforward and efficient use of a single randomized trial to evaluate overall treatment effect and treatment effect within subgroups using predictive markers.

CONCLUSION

The simple themes provide a general guideline for evaluation of surrogate endpoints, prognostic markers, and predictive markers.

The Fundamental Difficulty With Evaluating the Accuracy of Biomarkers for Guiding Treatment

Developing biomarkers that can predict whether patients are likely to benefit from an intervention is a pressing objective in many areas of medicine. Recent guidance documents have recommended that the accuracy of predictive biomarkers, ie, sensitivity, specificity, and positive and negative predictive values, should be assessed. We clarify the meanings of these entities for predictive markers and demonstrate that generally they cannot be estimated from data without making strong untestable assumptions. Language suggesting that predictive biomarkers can identify patients who benefit from an intervention is also widespread. We show that in general one cannot estimate the chance that a patient will benefit from treatment. We recommend instead that predictive biomarkers be evaluated with respect to their ability to predict clinical outcomes among patients treated and among patients receiving standard of care, and the population impact of treatment rules based on those predictions. Ideally these entities are estimated from a randomized trial comparing the experimental intervention with standard of care.

Evaluating Biomarkers for Guiding Treatment Decisions

The genetic revolution is expected to lead to improved targeting of new and existing forms of treatment. Rather than a one-size-fits-all blockbuster strategy in battling disease with drugs and other interventions, a more precise approach is becoming available, one in which treatment is only offered to those likely to benefit. The identification of those likely to benefit from treatment could be based on one or more biomarkers, but in an era where medical decisions aim to be evidence-based, the use of treatment selection markers should not just be based on hope and optimism, but on solid data from sound research. The performance of the treatment selection marker should be expressed in quantitative terms, similar to the way we express the clinical performance of diagnostic markers, or the performance of prognostic markers.

Personalized Evaluation of Biomarker Value: A Cost-Benefit Perspective

For a patient who is facing a treatment decision, the added value of information provided by a biomarker depends on the individual patient's expected response to treatment with and without the biomarker, as well as his/her tolerance of disease and treatment harm. However, individualized estimators of the value of a biomarker are lacking. We propose a new graphical tool named the subject-specific expected benefit curve for quantifying the personalized value of a biomarker in aiding a treatment decision. We develop semiparametric estimators for two general settings: (i) when biomarker data are available from a randomized trial; and (ii) when biomarker data are available from a cohort or a cross-sectional study, together with external information about a multiplicative treatment effect. We also develop adaptive bootstrap confidence intervals for consistent inference in the presence of nonregularity. The proposed method is used to evaluate the individualized value of the serum creatinine marker in informing treatment decisions for the prevention of renal artery stenosis.

Characterizing expected benefits of biomarkers in treatment selection

Biomarkers associated with heterogeneity in subject responses to treatment hold potential for treatment selection. In practice, the decision regarding whether to adopt a treatment-selection marker depends on the effect of using the marker on the rate of targeted disease and on the cost associated with treatment. We propose an expected benefit measure that incorporates both effects to quantify a marker's treatment-selection capacity. This measure builds upon an existing decision-theoretic framework, but is expanded to account for the fact that optimal treatment absent marker information varies with the cost of treatment. In addition, we establish upper and lower bounds on the expected benefit for a perfect treatment-selection rule which provides the basis for a standardized expected benefit measure. We develop model-based estimators for these measures in a randomized trial setting and evaluate their asymptotic properties. An adaptive bootstrap confidence interval is proposed for inference in the presence of non-regularity. Alternative estimators robust to risk model misspecification are also investigated. We illustrate our methods using the Diabetes Control and Complications Trial where we evaluate the expected benefit of baseline hemoglobin A1C in selecting diabetes treatment.

Combining biomarkers to optimize patient treatment recommendations

Markers that predict treatment effect have the potential to improve patient outcomes. For example, the OncotypeDX® RecurrenceScore® has some ability to predict the benefit of adjuvant chemotherapy over and above hormone therapy for the treatment of estrogen-receptor-positive breast cancer, facilitating the provision of chemotherapy to women most likely to benefit from it. Given that the score was originally developed for predicting outcome given hormone therapy alone, it is of interest to develop alternative combinations of the genes comprising the score that are optimized for treatment selection. However, most methodology for combining markers is useful when predicting outcome under a single treatment. We propose a method for combining markers for treatment selection which requires modeling the treatment effect as a function of markers. Multiple models of treatment effect are fit iteratively by upweighting or "boosting" subjects potentially misclassified according to treatment benefit at the previous stage. The boosting approach is compared to existing methods in a simulation study based on the change in expected outcome under marker-based treatment. The approach improves upon methods in some settings and has comparable performance in others. Our simulation study also provides insights as to the relative merits of the existing methods. Application of the boosting approach to the breast cancer data, using scaled versions of the original markers, produces marker combinations that may have improved performance for treatment selection.

Dynamic treatment regimes: technical challenges and applications

Dynamic treatment regimes are of growing interest across the clinical sciences because these regimes provide one way to operationalize and thus inform sequential personalized clinical decision making. Formally, a dynamic treatment regime is a sequence of decision rules, one per stage of clinical intervention. Each decision rule maps up-to-date patient information to a recommended treatment. We briefly review a variety of approaches for using data to construct the decision rules. We then review a critical inferential challenge that results from nonregularity, which often arises in this area. In particular, nonregularity arises in inference for parameters in the optimal dynamic treatment regime; the asymptotic, limiting, distribution of estimators are sensitive to local perturbations. We propose and evaluate a locally consistent Adaptive Confidence Interval (ACI) for the parameters of the optimal dynamic treatment regime. We use data from the Adaptive Pharmacological and Behavioral Treatments for Children with ADHD Trial as an illustrative example. We conclude by highlighting and discussing emerging theoretical problems in this area.