General nonparametric ROC curve comparison

Confidence intervals for the length of the receiver-operating characteristic curve based on a smooth estimator

A good diagnostic test should show different behavior on both the positive and the negative populations. However, this is not enough for having a good classification system. The binary classification problem is a complex task, which implies to define decision criteria. The knowledge of the level of dissimilarity between the two involved distributions is not enough. We also have to know how to define those decision criteria. The length of the receiver-operating characteristic curve has been proposed as an index of the optimal discriminatory capacity of a biomarker. It is related not with the actual but with the optimal classification capacity of the considered diagnostic test. One particularity of this index is that its estimation should be based on parametric or smoothed models. We explore here the behavior of a kernel density estimator-based approximation for estimating the length of the receiver-operating characteristic curve. We prove the asymptotic distribution of the resulting statistic, propose a parametric bootstrap algorithm for confidence intervals construction, discuss the role that the bandwidth parameter plays in the quality of the provided estimations and, via Monte Carlo simulations, study its finite-sample behavior considering four different criteria for the bandwidth selection. The practical use of the length of the receiver-operating characteristic curve is illustrated through two real-world examples.

A test for comparing conditional ROC curves with multidimensional covariates

The comparison of Receiver Operating Characteristic (ROC) curves is frequently used in the literature to compare the discriminatory capability of different classification procedures based on diagnostic variables. The performance of these variables can be sometimes influenced by the presence of other covariates, and thus they should be taken into account when making the comparison. A new non-parametric test is proposed here for testing the equality of two or more dependent ROC curves conditioned to the value of a multidimensional covariate. Projections are used for transforming the problem into a one-dimensional approach easier to handle. Simulations are carried out to study the practical performance of the new methodology. The procedure is then used to analyse a real data set of patients with Pleural Effusion to compare the diagnostic capability of different markers.

Machine Learning Models and Statistical Complexity to Analyze the Effects of Posture on Cerebral Hemodynamics

The mechanism of cerebral blood flow autoregulation can be of great importance in diagnosing and controlling a diversity of cerebrovascular pathologies such as vascular dementia, brain injury, and neurodegenerative diseases. To assess it, there are several methods that use changing postures, such as sit-stand or squat-stand maneuvers. However, the evaluation of the dynamic cerebral blood flow autoregulation (dCA) in these postures has not been adequately studied using more complex models, such as non-linear ones. Moreover, dCA can be considered part of a more complex mechanism called cerebral hemodynamics, where others (CO2 reactivity and neurovascular-coupling) that affect cerebral blood flow (BF) are included. In this work, we analyzed postural influences using non-linear machine learning models of dCA and studied characteristics of cerebral hemodynamics under statistical complexity using eighteen young adult subjects, aged 27 ± 6.29 years, who took the systemic or arterial blood pressure (BP) and cerebral blood flow velocity (BFV) for five minutes in three different postures: stand, sit, and lay. With models of a Support Vector Machine (SVM) through time, we used an AutoRegulatory Index (ARI) to compare the dCA in different postures. Using wavelet entropy, we estimated the statistical complexity of BFV for three postures. Repeated measures ANOVA showed that only the complexity of lay-sit had significant differences.

Using complexity-entropy planes to detect Parkinson's disease from short segments of haemodynamic signals

Objective. There is emerging evidence that analysing the entropy and complexity of biomedical signals can detect underlying changes in physiology which may be reflective of disease pathology. This approach can be used even when only short recordings of biomedical signals are available. This study aimed to determine whether entropy and complexity measures can detect differences between subjects with Parkinsons disease and healthy controls (HCs).Approach. A method based on a diagram of entropy versus complexity, named complexity-entropy plane, was used to re-analyse a dataset of cerebral haemodynamic signals from subjects with Parkinsons disease and HCs obtained under poikilocapnic conditions. A probability distribution for a set of ordinal patterns, designed to capture regularities in a time series, was computed from each signal under analysis. Four types of entropy and ten types of complexity measures were estimated from these distributions. Mean values of entropy and complexity were compared and their classification power was assessed by evaluating the best linear separator on the corresponding complexity-entropy planes.Main results. Few linear separators obtained significantly better classification, evaluated as the area under the receiver operating characteristic curve, than signal mean values. However, significant differences in both entropy and complexity were detected between the groups of participants.Significance. Measures of entropy and complexity were able to detect differences between healthy volunteers and subjects with Parkinson's disease, in poikilocapnic conditions, even though only short recordings were available for analysis. Further work is needed to refine this promising approach, and to help understand the findings in the context of specific pathophysiological changes.

Statistical inference for the difference between two maximized Youden indices obtained from correlated biomarkers

Currently, there is global interest in deriving new promising cancer biomarkers that could complement or substitute the conventional ones. Clinical decisions can often be based on the cutoff that corresponds to the maximized Youden index when maximum accuracy drives decisions. When more than one classification criteria are measured within the same individuals, correlated measurements arise. In this work, we propose hypothesis tests and confidence intervals for the comparison of two correlated receiver operating characteristic (ROC) curves in terms of their corresponding maximized Youden indices. We explore delta-based techniques under parametric assumptions, or power transformations. Nonparametric kernel-based methods are also examined. We evaluate our approaches through simulations and illustrate them using data from a metabolomic study referring to the detection of pancreatic cancer.

The area under the generalized receiver-operating characteristic curve

The receiver operating-characteristic (ROC) curve is a well-known graphical tool routinely used for evaluating the discriminatory ability of continuous markers, referring to a binary characteristic. The area under the curve (AUC) has been proposed as a summarized accuracy index. Higher values of the marker are usually associated with higher probabilities of having the characteristic under study. However, there are other situations where both, higher and lower marker scores, are associated with a positive result. The generalized ROC (gROC) curve has been proposed as a proper extension of the ROC curve to fit these situations. Of course, the corresponding area under the gROC curve, gAUC, has also been introduced as a global measure of the classification capacity. In this paper, we study in deep the gAUC properties. The weak convergence of its empirical estimator is provided while deriving an explicit and useful expression for the asymptotic variance. We also obtain the expression for the asymptotic covariance of related gAUCs and propose a non-parametric procedure to compare them. The finite-samples behavior is studied through Monte Carlo simulations under different scenarios, presenting a real-world problem in order to illustrate its practical application. The R code functions implementing the procedures are provided as Supplementary Material.

Predictive validity of the HCR-20V3 in a sample of Australian forensic psychiatric patients

The Historical Clinical Risk Management-20 Version 3 is the latest iteration in the HCR-20 series, adopting novel changes such as the addition of Relevance ratings and non-requirement to include the Psychopathy Checklist-Revised. This study aimed to examine these changes and compare the predictive validity of the HCR-20^V3 to the HCR-20^V2. The sample comprised of 100 forensic psychiatric patients, retrospectively followed up for a maximum period of approximately 13 years post-discharge from the Thomas Embling Hospital. Recidivism data were sourced from official police records. Results indicated good to excellent inter-rater reliability. The HCR-20^V3 significantly predicted violent recidivism (area under the curve = .70 to .77), levels of accuracy that were not significantly different from the HCR-20^V2. HCR-20^V3 Relevance ratings failed to add incremental validity above Presence ratings; however, the PCL-R improved upon the HCR-20^V3's validity. The study represented one of the first evaluations of the HCR-20^V3 in Australia.

The Youden Index in the Generalized Receiver Operating Characteristic Curve Context

The receiver operating characteristic (ROC) curve and their associated summary indices, such as the Youden index, are statistical tools commonly used to analyze the discrimination ability of a (bio)marker to distinguish between two populations. This paper presents the concept of Youden index in the context of the generalized ROC (gROC) curve for non-monotone relationships. The interval estimation of the Youden index and the associated cutoff points in a parametric (binormal) and a non-parametric setting is considered. Monte Carlo simulations and a real-world application illustrate the proposed methodology.

Efficient nonparametric confidence bands for receiver operating-characteristic curves

Receiver operating-characteristic curve is a popular graphical method frequently used in order to study the diagnostic capacity of continuous (bio)markers. In spite of the existence of a huge number of papers devoted to both theoretical and practical aspects of this topic, the construction of confidence bands has had little impact in the specialized literature. As far as the authors know, in the CRAN there are only three R packages providing receiver operating-characteristic curve confidence regions: plotROC, pROC and fbroc. This work tries to fill this gap studying and proposing a new nonparametric method to build confidence bands for both the standard receiver operating-characteristic curve and its generalization for nonmonotone relationships. The behavior of the proposed procedure is studied via Monte Carlo simulations and the methodology is applied on two real-world biomedical problems. In addition, an R function to compute the proposed and some of the previously existing methodologies is provided as online supplementary material.

Fully non-parametric receiver operating characteristic curve estimation for random-effects meta-analysis

Meta-analyses, broadly defined as the quantitative review and synthesis of the results of related but independent comparable studies, allow to know the state of the art of one considered topic. Since the amount of available bibliography has enhanced in almost all fields and, specifically, in biomedical research, its popularity has drastically increased during the last decades. In particular, different methodologies have been developed in order to perform meta-analytic studies of diagnostic tests for both fixed- and random-effects models. From a parametric point of view, these techniques often compute a bivariate estimation for the sensitivity and the specificity by using only one threshold per included study. Frequently, an overall receiver operating characteristic curve based on a bivariate normal distribution is also provided. In this work, the author deals with the problem of estimating an overall receiver operating characteristic curve from a fully non-parametric approach when the data come from a meta-analysis study i.e. only certain information about the diagnostic capacity is available. Both fixed- and random-effects models are considered. In addition, the proposed methodology lets to use the information of all cut-off points available (not only one of them) in the selected original studies. The performance of the method is explored through Monte Carlo simulations. The observed results suggest that the proposed estimator is better than the reference one when the reported information is related to a threshold based on the Youden index and when information for two or more points are provided. Real data illustrations are included.