Semiparametric estimation of the covariate-specific ROC curve in presence of ignorable verification bias

Significance tests for covariates in the diagnostic accuracy index of a biomarker against a continuous gold standard

Receiver operating characteristic (ROC) curve is a popular tool to describe and compare the diagnostic accuracy of biomarkers when a binary-scale gold standard is available. However, there are many examples of diagnostic tests whose gold standards are continuous. Hence, Several extensions of receiver operating characteristic (ROC) curve are proposed to evaluate the diagnostic potential of biomarkers when the gold standard is continuous-scale. Moreover, in evaluating these biomarkers, it is often necessary to consider the effects of covariates on the diagnostic accuracy of the biomarker of interest. Covariates may include subject characteristics, expertise of the test operator, test procedures or aspects of specimen handling. Applying the covariate adjustment to the case that the gold standard is continuous is challenging and has not been addressed in the literature. To fill the gap, we propose two general testing frameworks to account for the covariates effect on diagnostic accuracy. Simulation studies are conducted to compare the proposed tests. Data from a study that assessed three types of imaging modalities with the purpose of detecting neoplastic colon polyps and cancers are used to illustrate the proposed methods.

Statistical inference for decision curve analysis, with applications to cataract diagnosis

Statistical learning methods are widely used in medical literature for the purpose of diagnosis or prediction. Conventional accuracy assessment via sensitivity, specificity, and ROC curves does not fully account for clinical utility of a specific model. Decision curve analysis (DCA) becomes a novel complement as it incorporates a clinical judgment of the relative value of benefits (treating a true positive case) and harms (treating a false positive case) associated with prediction models. The preference of a patient or a policy-maker is formulated statistically as the underlying threshold probability, above which the patient would choose to be treated. Net benefit is then calculated for possible threshold probability, which places benefits and harms on the same scale. We consider the inference problems for DCA in this paper. Interval estimation procedure and inference methodology are provided after we derive the relevant asymptotic properties. Our formulation can accommodate the classification problems with multiple categories. We carry out numerical studies to assess the performance of the proposed methods. An eye disease dataset is analyzed to illustrate our proposals.

Evaluating Discrimination of a Lung Cancer Risk Prediction Model Using Partial Risk-Score in a Two-Phase Study

BACKGROUND

Independent validation of risk prediction models in prospective cohorts is required for risk-stratified cancer prevention. Such studies often have a two-phase design, where information on expensive biomarkers are ascertained in a nested substudy of the original cohort.

METHODS

We propose a simple approach for evaluating model discrimination that accounts for incomplete follow-up and gains efficiency by using data from all individuals in the cohort irrespective of whether they were sampled in the substudy. For evaluating the AUC, we estimated probabilities of risk-scores for cases being larger than those in controls conditional on partial risk-scores, computed using partial covariate information. The proposed method was compared with an inverse probability weighted (IPW) approach that used information only from the subjects in the substudy. We evaluated age-stratified AUC of a model including questionnaire-based risk factors and inflammation biomarkers to predict 10-year risk of lung cancer using data from the Prostate, Lung, Colorectal, and Ovarian Cancer (1993-2009) trial (30,297 ever-smokers, 1,253 patients with lung cancer).

RESULTS

For estimating age-stratified AUC of the combined lung cancer risk model, the proposed method was 3.8 to 5.3 times more efficient compared with the IPW approach across the different age groups. Extensive simulation studies also demonstrated substantial efficiency gain compared with the IPW approach.

CONCLUSIONS

Incorporating information from all individuals in a two-phase cohort study can substantially improve precision of discrimination measures of lung cancer risk models.

IMPACT

Novel, simple, and practically useful methods are proposed for evaluating risk models, a critical step toward risk-stratified cancer prevention.

A pair-wise meta-analysis highlights circular RNAs as potential biomarkers for colorectal cancer

Background

Circular RNAs (circRNAs) have emerged as a special subset of endogenous RNAs that are implicated in tumorigenesis and cancer progression. Herein we aim to carry out a meta-analysis to evaluate the clinicopathologic, diagnostic and prognostic significance of circRNA expression in colorectal cancer (CRC).

Methods

A systematic search of online databases was performed for original articles published in English, which investigated the diagnostic accuracy, prognostic utility, and clinicopathologic association of circRNA(s) in CRC. Data were strictly extracted and study bias was judged using the Quality Assessment for Studies of Diagnostic Accuracy II (QUADAS II) and Newcastle-Ottawa Scale (NOS) checklists.

Results

A total of 13 studies, involving 1430 patients with CRC, were included in the meta-analysis. The clinicopathologic study showed that abnormally expressed circRNAs were correlated with tumor diameter (P = 0.0350), differentiation (P = 0.0038), lymphatic metastasis (P = 0.0119), distant metastasis (P < 0.0001), TNM stage (P = 0.0002), and depth of invasion (P = 0.001) in patients with CRC. The summary area under the curve (AUC) of circRNA for the discriminative efficacy between patients with and without CRC was estimated to be 0.79, corresponding to a weighted sensitivity of 0.77 [95% confidence interval (CI): 0.74–0.79], specificity of 0.67 (95%CI: 0.64–0.70), and diagnostic odds ratio (DOR) of 7.52 (95%CI: 4.66–12.12). Survival analysis showed that highly expressed circRNAs were correlated with significantly worse overall survival (OS) [hazard ratio (HR) = 2.66, 95%CI: 2.03–3.50, P = 0.000; X² = 4.34, P = 0.740, I² = 0.0%], whereas lower expression of circRNAs was associated with prolonged OS (weighted HR = 0.30, 95%CI: 0.17–0.53, P = 0.000; X² = 1.34, P = 0.909, I² = 0.0%). Stratified analysis in circRNA expression status, and test matrix also showed robust results.

Conclusion

Abnormally expressed circRNAs may be auxiliary biomarkers facilitating CRC diagnosis, and promising prognostic biomarkers in predicting the survival of CRC patients.

Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update

OBJECTIVE

To systematically review methods developed and employed to evaluate the diagnostic accuracy of medical test when there is a missing or no gold standard.

STUDY DESIGN AND SETTINGS

Articles that proposed or applied any methods to evaluate the diagnostic accuracy of medical test(s) in the absence of gold standard were reviewed. The protocol for this review was registered in PROSPERO (CRD42018089349).

RESULTS

Identified methods were classified into four main groups: methods employed when there is a missing gold standard; correction methods (which make adjustment for an imperfect reference standard with known diagnostic accuracy measures); methods employed to evaluate a medical test using multiple imperfect reference standards; and other methods, like agreement studies, and a mixed group of alternative study designs. Fifty-one statistical methods were identified from the review that were developed to evaluate medical test(s) when the true disease status of some participants is unverified with the gold standard. Seven correction methods were identified and four methods were identified to evaluate medical test(s) using multiple imperfect reference standards. Flow-diagrams were developed to guide the selection of appropriate methods.

CONCLUSION

Various methods have been proposed to evaluate medical test(s) in the absence of a gold standard for some or all participants in a diagnostic accuracy study. These methods depend on the availability of the gold standard, its' application to the participants in the study and the availability of alternative reference standard(s). The clinical application of some of these methods, especially methods developed when there is missing gold standard is however limited. This may be due to the complexity of these methods and/or a disconnection between the fields of expertise of those who develop (e.g. mathematicians) and those who employ the methods (e.g. clinical researchers). This review aims to help close this gap with our classification and guidance tools.

Quantitative Multiparametric MRI May Augment the Response to Radiotherapy in Mid-Treatment Assessment of Patients with Esophageal Carcinoma

OBJECTIVE

The purpose of this study was to assess the mid-treatment response to radiotherapy (RT) using dynamic contrast-enhanced MRI (DCE-MRI) and diffusion-weighted imaging (DWI) in patients with esophageal cancer (EC).

METHODS

42 patients with squamous EC were prepared for DCE-MRI and DWI scans both before treatment (NRT) and after the fifth radiotherapy (5th RT). The patients were classified into two groups (complete response [CR] and partial response [PR]) according to tumor regression after treatment. The quantitative parameters of DCE-MRI (Ktrans, Kep, Ve, and ADC) were measured. A receiver operating characteristic curve (ROC) was used to detect the efficiency of the above parameters.

RESULTS

After 1 month of RT, 29 patients were classified as CR and 11 patients were classified as PR. In the NRT group, the p values of Ktrans, Kep, Ve, and ADC were 0.004, 0.078, 0.0008, and <0.0001, respectively. After the 5th RT, the p values of the above parameters were <0.001, 0.005, 0.108, and 0.365, respectively. In the NRT group, the areas under the ROC curves of Ktrans, Ve, and ADC were 0.790, 0.617, and 0.737; the sensitivity values were 89.3, 92.5, and 90.0%; the specificity values were 69.4, 27.5, and 50.0%. In the 5th RT group, the areas under the ROC curves of Ktrans and Kep were 0.816 and 0.804; the sensitivity values were 71.2 and 95.0%; the specificity values were 81.6 and 50.0%.

CONCLUSION

DCE-MRI combined with DWI is effective in the early prediction of radiotherapeutic response of EC after the 5th RT other than after the traditional final treatment.

Combining biomarkers for classification with covariate adjustment

Combining multiple markers can improve classification accuracy compared with using a single marker. In practice, covariates associated with markers or disease outcome can affect the performance of a biomarker or biomarker combination in the population. The covariate-adjusted receiver operating characteristic (ROC) curve has been proposed as a tool to tease out the covariate effect in the evaluation of a single marker; this curve characterizes the classification accuracy solely because of the marker of interest. However, research on the effect of covariates on the performance of marker combinations and on how to adjust for the covariate effect when combining markers is still lacking. In this article, we examine the effect of covariates on classification performance of linear marker combinations and propose to adjust for covariates in combining markers by maximizing the nonparametric estimate of the area under the covariate-adjusted ROC curve. The proposed method provides a way to estimate the best linear biomarker combination that is robust to risk model assumptions underlying alternative regression-model-based methods. The proposed estimator is shown to be consistent and asymptotically normally distributed. We conduct simulations to evaluate the performance of our estimator in cohort and case/control designs and compare several different weighting strategies during estimation with respect to efficiency. Our estimator is also compared with alternative regression-model-based estimators or estimators that maximize the empirical area under the ROC curve, with respect to bias and efficiency. We apply the proposed method to a biomarker study from an human immunodeficiency virus vaccine trial. Copyright © 2017 John Wiley & Sons, Ltd.

Estimation of diagnostic test accuracy without full verification: a review of latent class methods

The performance of a diagnostic test is best evaluated against a reference test that is without error. For many diseases, this is not possible, and an imperfect reference test must be used. However, diagnostic accuracy estimates may be biased if inaccurately verified status is used as the truth. Statistical models have been developed to handle this situation by treating disease as a latent variable. In this paper, we conduct a systematized review of statistical methods using latent class models for estimating test accuracy and disease prevalence in the absence of complete verification.

ROC analysis in biomarker combination with covariate adjustment

RATIONAL AND OBJECTIVES

Receiver operating characteristic (ROC) analysis is often used to find the optimal combination of biomarkers. When the subject level covariates affect the magnitude and/or accuracy of the biomarkers, the combination rule should take into account of the covariate adjustment. The authors propose two new biomarker combination methods that make use of the covariate information.

MATERIALS AND METHODS

The first method is to maximize the area under the covariate-adjusted ROC curve (AAUC). To overcome the limitations of the AAUC measure, the authors further proposed the area under covariate-standardized ROC curve (SAUC), which is an extension of the covariate-specific ROC curve. With a series of simulation studies, the proposed optimal AAUC and SAUC methods are compared with the optimal AUC method that ignores the covariates. The biomarker combination methods are illustrated by an example from Alzheimer's disease research.

RESULTS

The simulation results indicate that the optimal AAUC combination performs well in the current study population. The optimal SAUC method is flexible to choose any reference populations, and allows the results to be generalized to different populations.

CONCLUSIONS

The proposed optimal AAUC and SAUC approaches successfully address the covariate adjustment problem in estimating the optimal marker combination. The optimal SAUC method is preferred for practical use, because the biomarker combination rule can be easily evaluated for different population of interest.

Covariate adjustment in estimating the area under ROC curve with partially missing gold standard

In ROC analysis, covariate adjustment is advocated when the covariates impact the magnitude or accuracy of the test under study. Meanwhile, for many large scale screening tests, the true condition status may be subject to missingness because it is expensive and/or invasive to ascertain the disease status. The complete-case analysis may end up with a biased inference, also known as "verification bias." To address the issue of covariate adjustment with verification bias in ROC analysis, we propose several estimators for the area under the covariate-specific and covariate-adjusted ROC curves (AUCx and AAUC). The AUCx is directly modeled in the form of binary regression, and the estimating equations are based on the U statistics. The AAUC is estimated from the weighted average of AUCx over the covariate distribution of the diseased subjects. We employ reweighting and imputation techniques to overcome the verification bias problem. Our proposed estimators are initially derived assuming that the true disease status is missing at random (MAR), and then with some modification, the estimators can be extended to the not missing at random (NMAR) situation. The asymptotic distributions are derived for the proposed estimators. The finite sample performance is evaluated by a series of simulation studies. Our method is applied to a data set in Alzheimer's disease research.