Maximizing an ROC-type measure via linear combination of markers when the gold reference is continuous

Significance tests for covariates in the diagnostic accuracy index of a biomarker against a continuous gold standard

Receiver operating characteristic (ROC) curve is a popular tool to describe and compare the diagnostic accuracy of biomarkers when a binary-scale gold standard is available. However, there are many examples of diagnostic tests whose gold standards are continuous. Hence, Several extensions of receiver operating characteristic (ROC) curve are proposed to evaluate the diagnostic potential of biomarkers when the gold standard is continuous-scale. Moreover, in evaluating these biomarkers, it is often necessary to consider the effects of covariates on the diagnostic accuracy of the biomarker of interest. Covariates may include subject characteristics, expertise of the test operator, test procedures or aspects of specimen handling. Applying the covariate adjustment to the case that the gold standard is continuous is challenging and has not been addressed in the literature. To fill the gap, we propose two general testing frameworks to account for the covariates effect on diagnostic accuracy. Simulation studies are conducted to compare the proposed tests. Data from a study that assessed three types of imaging modalities with the purpose of detecting neoplastic colon polyps and cancers are used to illustrate the proposed methods.

A Multiplex Assay for the Stratification of Patients with Primary Central Nervous System Lymphoma Using Targeted Mass Spectrometry

Primary central nervous system lymphomas (PCNSL) account for approximately 2% to 3% of all primary brain tumors. Until now, neuropathological tumor tissue analysis, most frequently gained by stereotactic biopsy, is still the diagnostic gold standard. Here, we rigorously analyzed two independent patient cohorts comprising the clinical entities PCNSL (n = 47), secondary central nervous system lymphomas (SCNSL; n = 13), multiple sclerosis (MS, n = 23), glioma (n = 10), other tumors (n = 17) and tumor-free controls (n = 21) by proteomic approaches. In total, we identified more than 1220 proteins in the cerebrospinal fluid (CSF) and validated eight candidate biomarkers by a peptide-centric approach in an independent patient cohort (n = 63). Thus, we obtained excellent diagnostic accuracy for the stratification between PCNSL, MS and glioma patients as well as tumor-free controls for three peptides originating from the three proteins VSIG4, GPNMB4 and APOC2. The combination of all three biomarker candidates resulted in diagnostic accuracy with an area under the curve (AUC) of 0.901 (PCNSL vs. MS), AUC of 0.953 (PCNSL vs. glioma) and AUC 0.850 (PCNSL vs. tumor-free control). In summary, the determination of VSIG4, GPNMB4 and APOC2 in CSF as novel biomarkers for supporting the diagnosis of PCNSL is suggested.

A machine learning approach for the prediction of pulmonary hypertension

BACKGROUND

Machine learning (ML) is a powerful tool for identifying and structuring several informative variables for predictive tasks. Here, we investigated how ML algorithms may assist in echocardiographic pulmonary hypertension (PH) prediction, where current guidelines recommend integrating several echocardiographic parameters.

METHODS

In our database of 90 patients with invasively determined pulmonary artery pressure (PAP) with corresponding echocardiographic estimations of PAP obtained within 24 hours, we trained and applied five ML algorithms (random forest of classification trees, random forest of regression trees, lasso penalized logistic regression, boosted classification trees, support vector machines) using a 10 times 3-fold cross-validation (CV) scheme.

RESULTS

ML algorithms achieved high prediction accuracies: support vector machines (AUC 0.83; 95% CI 0.73-0.93), boosted classification trees (AUC 0.80; 95% CI 0.68-0.92), lasso penalized logistic regression (AUC 0.78; 95% CI 0.67-0.89), random forest of classification trees (AUC 0.85; 95% CI 0.75-0.95), random forest of regression trees (AUC 0.87; 95% CI 0.78-0.96). In contrast to the best of several conventional formulae (by Aduen et al.), this ML algorithm is based on several echocardiographic signs and feature selection, with estimated right atrial pressure (RAP) being of minor importance.

CONCLUSIONS

Using ML, we were able to predict pulmonary hypertension based on a broader set of echocardiographic data with little reliance on estimated RAP compared to an existing formula with non-inferior performance. With the conceptual advantages of a broader and unbiased selection and weighting of data our ML approach is suited for high level assistance in PH prediction.

Influence Analysis for the Area Under the Receiver Operating Characteristic Curve

Classification measures play essential roles in the assessment and construction of classifiers. Hence, determining how to prevent these measures from being affected by individual observations has become an important problem. In this paper, we propose several indexes based on the influence function and the concept of local influence to identify influential observations that affect the estimate of the area under the receiver operating characteristic curve (AUC), an important and commonly used measure. Cumulative lift charts are also used to equipoise the disagreements among the proposed indexes. Both the AUC indexes and the graphical tools only rely on the classification scores, and both are applicable to classifiers that can produce real-valued classification scores. A real data set is used for illustration.

Model-free scoring system for risk prediction with application to hepatocellular carcinoma study

There is an increasing need to construct a risk-prediction scoring system for survival data and identify important risk factors (e.g., biomarkers) for patient screening and treatment recommendation. However, most existing methodologies either rely on strong model assumptions (e.g., proportional hazards) or only handle binary outcomes. In this article, we propose a flexible method that simultaneously selects important risk factors and identifies the optimal linear combination of risk factors by maximizing a pseudo-likelihood function based on the time-dependent area under the receiver operating characteristic curve. Our method is particularly useful for risk evaluation and recommendation of optimal subsequent treatments. We show that the proposed method has desirable theoretical properties, including asymptotic normality and the oracle property after variable selection. Numerical performance is evaluated on several simulation data sets and an application to hepatocellular carcinoma data.

Linear combination methods to improve diagnostic/prognostic accuracy on future observations

Multiple diagnostic tests or biomarkers can be combined to improve diagnostic accuracy. The problem of finding the optimal linear combinations of biomarkers to maximise the area under the receiver operating characteristic curve has been extensively addressed in the literature. The purpose of this article is threefold: (1) to provide an extensive review of the existing methods for biomarker combination; (2) to propose a new combination method, namely, the nonparametric stepwise approach; (3) to use leave-one-pair-out cross-validation method, instead of re-substitution method, which is overoptimistic and hence might lead to wrong conclusion, to empirically evaluate and compare the performance of different linear combination methods in yielding the largest area under receiver operating characteristic curve. A data set of Duchenne muscular dystrophy was analysed to illustrate the applications of the discussed combination methods.