IMPROVING EFFICIENCY IN BIOMARKER INCREMENTAL VALUE EVALUATION UNDER TWO-PHASE DESIGNS

Biomarker evaluation under imperfect nested case-control design

The nested case-control (NCC) design has been widely adopted as a cost-effective sampling design for biomarker research. Under the NCC design, markers are only measured for the NCC subcohort consisting of all cases and a fraction of the controls selected randomly from the matched risk sets of the cases. Robust methods for evaluating prediction performance of risk models have been derived under the inverse probability weighting framework. The probabilities of samples being included in the NCC cohort can be calculated based on the study design ``a previous study'' or estimated non-parametrically ``a previous study''. Neither strategy works well due to model mis-specification and the curse of dimensionality in practical settings where the sampling does not entirely follow the study design or depends on many factors. In this paper, we propose an alternative strategy to estimate the sampling probabilities based on a varying coefficient model, which attains a balance between robustness and the curse of dimensionality. The complex correlation structure induced by repeated finite risk set sampling makes the standard resampling procedure for variance estimation fail. We propose a perturbation resampling procedure that provides valid interval estimation for the proposed estimators. Simulation studies show that the proposed method performs well in finite samples. We apply the proposed method to the Nurses' Health Study II to develop and evaluate prediction models using clinical biomarkers for cardiovascular risk.

Evaluating the time-dependent predictive accuracy for event-to-time outcome with a cure fraction

In medical studies, it is often observed that a portion of subjects will never experience the event of interest and thus can be treated as cured or long-term survivors. Many populations of early-stage cancer patients contain both uncured and cured individuals that should be modeled using cure models. In prognostic studies, the cure status (uncure or cure) is an issue of interest for medical practitioners, and the disease status (death or alive) of an individual is not a fixed characteristic and it varies along the time. These statuses are usually predicted by a prognostic risk score. The time-dependent receiver operating characteristic (ROC) curve is a powerful tool to evaluate these predicting performances dynamically. In the context with a cure fraction, quantifying and estimating the predictive performances of the risk score is a challenge since the disease status and cure status are both unknown among individuals who are censored. In this paper, to assess the predictive accuracy for the survival outcome with a cure fraction, we propose a time-dependent ROC curve semi-parametric estimator based on the sieve maximum likelihood (ML) estimation under the mixture cure model. We also apply a Bernstein-based smoothing method in the estimation procedure, and this estimator can lead to substantial gain in efficiency. In addition, we derive the time-dependent area under the ROC curve (AUC) to summarize the discriminatory capacity of the risk score globally. Finally, we evaluate the finite sample performance of the proposed methods by extensive simulations and illustrate the estimation using two real data sets, one from a melanoma study and the other from stomach cancer.

Evaluating Discrimination of a Lung Cancer Risk Prediction Model Using Partial Risk-Score in a Two-Phase Study

BACKGROUND

Independent validation of risk prediction models in prospective cohorts is required for risk-stratified cancer prevention. Such studies often have a two-phase design, where information on expensive biomarkers are ascertained in a nested substudy of the original cohort.

METHODS

We propose a simple approach for evaluating model discrimination that accounts for incomplete follow-up and gains efficiency by using data from all individuals in the cohort irrespective of whether they were sampled in the substudy. For evaluating the AUC, we estimated probabilities of risk-scores for cases being larger than those in controls conditional on partial risk-scores, computed using partial covariate information. The proposed method was compared with an inverse probability weighted (IPW) approach that used information only from the subjects in the substudy. We evaluated age-stratified AUC of a model including questionnaire-based risk factors and inflammation biomarkers to predict 10-year risk of lung cancer using data from the Prostate, Lung, Colorectal, and Ovarian Cancer (1993-2009) trial (30,297 ever-smokers, 1,253 patients with lung cancer).

RESULTS

For estimating age-stratified AUC of the combined lung cancer risk model, the proposed method was 3.8 to 5.3 times more efficient compared with the IPW approach across the different age groups. Extensive simulation studies also demonstrated substantial efficiency gain compared with the IPW approach.

CONCLUSIONS

Incorporating information from all individuals in a two-phase cohort study can substantially improve precision of discrimination measures of lung cancer risk models.

IMPACT

Novel, simple, and practically useful methods are proposed for evaluating risk models, a critical step toward risk-stratified cancer prevention.

Study Design Considerations for Cancer Biomarker Discoveries

Background

Biomarker discovery studies have generated an array of omic data, however few novel biomarkers have reached clinical use. Guidelines for rigorous study designs are needed.

Content

Biases frequently occur in sample selection, outcome ascertainment, or unblinded sample handling and assaying process. The principles of a prospective-specimen collection and retrospective-blinded-evaluation (PRoBE) design can be adapted to mitigate various sources of biases in discovery. We recommend establishing quality biospecimen repositories using matched two-phase designs to minimize biases and maximize efficiency. We also highlight the importance of taking the clinical context into consideration in both sample selection and power calculation for discovery studies.

Summary

Biomarker discovery research should follow rigorous design principles in sample se- lection to avoid biases. Consideration of clinical application and the corresponding biomarker performance characteristics in study designs will lead to a more fruitful discovery study.

Impact

Appropriate study designs will improve the quality and clinical rigor of biomarker discovery studies.

IMPROVING EFFICIENCY IN BIOMARKER INCREMENTAL VALUE EVALUATION UNDER TWO-PHASE DESIGNS

Cost-effective yet efficient designs are critical to the success of biomarker evaluation research. Two-phase sampling designs, under which expensive markers are only measured on a subsample of cases and non-cases within a prospective cohort, are useful in novel biomarker studies for preserving study samples and minimizing cost of biomarker assaying. Statistical methods for quantifying the predictiveness of biomarkers under two-phase studies have been proposed (Cai and Zheng, 2012; Liu, Cai and Zheng, 2012). These methods are based on a class of inverse probability weighted (IPW) estimators where weights are 'true' sampling weights that simply reflect the sampling strategy of the study. While simple to implement, existing IPW estimators are limited by lack of practicality and efficiency. In this manuscript, we investigate a variety of two-phase design options and provide statistical approaches aimed at improving the efficiency of simple IPW estimators by incorporating auxiliary information available for the entire cohort. We consider accuracy summary estimators that accommodate auxiliary information in the context of evaluating the incremental values of novel biomarkers over existing prediction tools. In addition, we evaluate the relative efficiency of a variety of sampling and estimation options under two-phase studies, shedding light on issues pertaining to both the design and analysis of biomarker validation studies. We apply our methods to the evaluation of a novel biomarker for liver cancer risk conducted with a two-phase nested case control design (Lok et al., 2010).