Group sequential design for comparative diagnostic accuracy studies: implications and guidelines for practitioners

Group sequential comparison of positive predictive value curves for correlated biomarker data

Clinical studies of predictive diagnostic tests consider the evaluation of a single test and comparison of two tests regarding their predictive accuracy of disease status. The positive predictive value (PPV) curve is used for assessing the probability of predicting the disease given a positive test result. The sequential property of one PPV curve had been studied. However, in later stages of diagnostic test development, it is more interesting to compare predictive accuracy of two tests. In this article, we propose a group sequential test for the comparison of PPV curves for paired designs when both diagnostic tests are applied to the same subject. We first derive asymptotic properties of the sequential differences of two correlated empirical PPV curves under the common case-control sampling. We then apply these results to develop a group sequential test procedure. The asymptotic results are also critical for deriving both the optimal sample size ratio and minimal required sample sizes for the proposed procedure. Our simulation studies show that the proposed sequential testing maintains the nominal type I error rate in finite samples. The proposed design is illustrated in a hypothetical lung cancer predictive trial and in a cancer diagnostic trial.

Adaptive trial designs in diagnostic accuracy research

The aim of diagnostic accuracy studies is to evaluate how accurately a diagnostic test can distinguish diseased from nondiseased individuals. Depending on the research question, different study designs and accuracy measures are appropriate. As the prior knowledge in the planning phase is often very limited, modifications of design aspects such as the sample size during the ongoing trial could increase the efficiency of diagnostic trials. In intervention studies, group sequential and adaptive designs are well established. Such designs are characterized by preplanned interim analyses, giving the opportunity to stop early for efficacy or futility or to modify elements of the study design. In contrast, in diagnostic accuracy studies, such flexible designs are less common, even if they are as important as for intervention studies. However, diagnostic accuracy studies have specific features, which may require adaptations of the statistical methods or may lead to specific advantages or limitations of sequential and adaptive designs. In this article, we summarize the current status of methodological research and applications of flexible designs in diagnostic accuracy research. Furthermore, we indicate and advocate future development of adaptive design methodology and their use in diagnostic accuracy trials from an interdisciplinary viewpoint. The term "interdisciplinary viewpoint" describes the collaboration of experts of the academic and nonacademic research.

Group sequential testing of the predictive accuracy of a continuous biomarker with unknown prevalence

Group sequential testing procedures have been proposed as an approach to conserving resources in biomarker validation studies. Previously, we derived the asymptotic properties of the sequential empirical positive predictive value (PPV) and negative predictive value (NPV) curves, which summarize the predictive accuracy of a continuous marker, under case-control sampling. A limitation of this approach is that the prevalence cannot be estimated from a case-control study and must be assumed known. In this paper, we consider group sequential testing of the predictive accuracy of a continuous biomarker with unknown prevalence. First, we develop asymptotic theory for the sequential empirical PPV and NPV curves when the prevalence must be estimated, rather than assumed known in a case-control study. We then discuss how our results can be combined with standard group sequential methods to develop group sequential testing procedures and bias-adjusted estimators for the PPV and NPV curve. The small sample properties of the proposed group sequential testing procedures and estimators are evaluated by simulation, and we illustrate our approach in the context of a study to validate a novel biomarker for prostate cancer.

Early termination of a two-stage study to develop and validate a panel of biomarkers

Two-stage designs to develop and validate a panel of biomarkers present a natural setting for the inclusion of stopping rules for futility in the event of poor preliminary estimates of performance. We consider the design of a two-stage study to develop and validate a panel of biomarkers where a predictive model is developed using a subset of the samples in stage 1 and the model is validated using the remainder of the samples in stage 2. First, we illustrate how we can implement a stopping rule for futility in a standard, two-stage study for developing and validating a predictive model where samples are separated into a training sample and a validation sample. Simulation results indicate that our design has type I error rate and power similar to the fixed-sample design but with a substantially reduced sample size under the null hypothesis. We then illustrate how we can include additional interim analyses in stage 2 by applying existing group sequential methodology, which results in even greater savings in the number of samples required under both the null and the alternative hypotheses. Our simulation results also illustrate that the operating characteristics of our design are robust to changes in the underlying marker distribution.

Conditional estimation after a two-stage diagnostic biomarker study that allows early termination for futility

Many biomarkers identified in marker discovery are shown to have inadequate performance in validation studies. This motivates the use of group sequential designs that allow early termination for futility. However, an option for early termination will lead to biased estimates for studies that reach full enrollment. We propose conditional estimators and confidence intervals that correct for this bias assuming that an unadjusted estimator exists that has an independent increments covariance structure. The proposed estimators and confidence intervals are applied to conditional estimation of the receiver operating characteristic curve and the positive predictive value curve after a two-stage study that allows early termination for futility, and their performance is evaluated by simulation.

Conditional estimation of sensitivity and specificity from a phase 2 biomarker study allowing early termination for futility

Development of a disease screening biomarker involves several phases. In phase 2 its sensitivity and specificity is compared with established thresholds for minimally acceptable performance. Since we anticipate that most candidate markers will not prove to be useful and availability of specimens and funding is limited, early termination of a study is appropriate, if accumulating data indicate that the marker is inadequate. Yet, for markers that complete phase 2, we seek estimates of sensitivity and specificity to proceed with the design of subsequent phase 3 studies. We suggest early stopping criteria and estimation procedures that adjust for bias caused by the early termination option. An important aspect of our approach is to focus on properties of estimates conditional on reaching full study enrollment. We propose the conditional-UMVUE and contrast it with other estimates, including naïve estimators, the well-studied unconditional-UMVUE and the mean and median Whitehead-adjusted estimators. The conditional-UMVUE appears to be a very good choice.