The ROC region of a regression tree

Obtaining optimal cutoff values for tree classifiers using multiple biomarkers

In biomedical practices, multiple biomarkers are often combined using a prespecified classification rule with tree structure for diagnostic decisions. The classification structure and cutoff point at each node of a tree are usually chosen on an ad hoc basis, depending on decision makers' experience. There is a lack of analytical approaches that lead to optimal prediction performance, and that guide the choice of optimal cutoff points in a pre-specified classification tree. In this paper, we propose to search for and estimate the optimal decision rule through an approach of rank correlation maximization. The proposed method is flexible, theoretically sound, and computationally feasible when many biomarkers are available for classification or prediction. Using the proposed approach, for a prespecified tree-structured classification rule, we can guide the choice of optimal cutoff points at tree nodes and estimate optimal prediction performance from multiple biomarkers combined.

The Optimal Cut-off of BIPSS in Differential Diagnosis of ACTH-dependent Cushing's Syndrome: Is Stimulation Necessary?

CONTEXTS

Bilateral inferior petrosal sinus sampling (BIPSS) can differentiate Cushing's disease (CD) and ectopic adrenocorticotropin (ACTH) syndrome (EAS). The traditional cutoff of inferior petrosal sinus to peripheral (IPS:P) ACTH gradient was 2 before stimulation and 3 after stimulation, which yielded unsatisfactory sensitivity in some studies.

OBJECTIVES

To determine the optimal cutoff in BIPSS before or after desmopressin stimulation and to evaluate the necessity of stimulation.

DESIGN AND SETTING

Single-center retrospective study (2011-2018) along with meta-analysis.

PATIENTS

226 CD and 24 EAS patients with confirmed diagnosis who underwent BIPSS with desmopressin stimulation.

RESULTS

In the meta-analysis of 25 studies with 1249 CD and 152 EAS patients, the traditional cutoff yielded sensitivity of 86% and 97% and specificity of 98% and 100% before and after stimulation, respectively. We then analyzed the data from our center. With the traditional cutoff, the sensitivity was 87.2% (197/226) and 96.5% (218/226) before and after stimulation, and specificity was both 100% (25/25), which were close to the results of meta-analysis. Receiver operating characteristic analysis revealed that the optimal cutoff was 1.4 before stimulation and 2.8 after stimulation. With the new cutoff, the sensitivity was 94.7% (214/226) and 97.8% (221/226) while the specificity remained 100% (25/25) before and after stimulation. Among the 7 CD patients (7/226; 3.1%) for whom stimulation was necessary to get correct diagnosis, none has a pituitary lesion >6 mm by magnetic resonance imaging, and their sampling lateralization rate (P = .007) and peak ACTH level at dominant inferior petrosal sinus (P = .011) were lower than those among CD patients with IPS:P >1.4 before stimulation.

CONCLUSIONS

The optimal cutoff for IPS:P in BIPSS is different from the commonly-used one. The optimal cutoff value can yield satisfactory accuracy even without stimulation, and stimulation may be unnecessary for those with pituitary adenoma >6 mm.

Local axonal morphology guides the topography of interneuron myelination in mouse and human neocortex

GABAergic fast-spiking parvalbumin-positive (PV) interneurons are frequently myelinated in the cerebral cortex. However, the factors governing the topography of cortical interneuron myelination remain incompletely understood. Here, we report that segmental myelination along neocortical interneuron axons is strongly predicted by the joint combination of interbranch distance and local axon caliber. Enlargement of PV+ interneurons increased axonal myelination, while reduced cell size led to decreased myelination. Next, we considered regular-spiking SOM+ cells, which normally have relatively shorter interbranch distances and thinner axon diameters than PV+ cells, and are rarely myelinated. Consistent with the importance of axonal morphology for guiding interneuron myelination, enlargement of SOM+ cell size dramatically increased the frequency of myelinated axonal segments. Lastly, we confirm that these findings also extend to human neocortex by quantifying interneuron axonal myelination from ex vivo surgical tissue. Together, these findings establish a predictive model of neocortical GABAergic interneuron myelination determined by local axonal morphology.

Partial Youden index and its inferences

In medical diagnostic research, medical tests with continuous values are widely employed to distinguish between diseased and non-diseased subjects. The diagnostic accuracy of a medical test can be assessed by using the receiver operating characteristic (ROC) curve of the test. To summarize the ROC curve and determine an optimal cut-off point for test results, the Youden index is commonly used. In particular, the Youden index is optimized over the entire range of values for sensitivity and specificity, which determine the ROC space. However in clinical practice, one may only be interested in the regions of the ROC curve that correspond to low false-positive rates or/and high sensitivities. In this paper, a new summary index for the ROC curve, called the "partial Youden index", is defined on regions of the ROC space in which sensitivity/specificity values are of practical interest. The traditional Youden index is a special case of the partial Youden index. Various parametric and non-parametric interval estimation methods are proposed for the partial Youden index. Extensive simulation studies are conducted to evaluate the finite sample performances of the proposed methods. A real example is used to illustrate the application of the new methods.

Methylation marker analysis of self-sampled cervico-vaginal lavage specimens to triage high-risk HPV-positive women for colposcopy

Methylation markers were studied for their suitability to triage human papillomavirus (HPV)-positive women by testing self-collected cervico-vaginal lavage specimens. For this purpose, we analyzed 355 hrHPV-positive self-collected specimens with three methylation markers, that is, CADM1-m18, MAL-m1 and miR-124-2 by quantitative methylation-specific PCR. The areas under the receiver-operating characteristic (ROC) curve for end-point cervical intraepithelial neoplasia grade 3 or worse (CIN3+) were 0.637 for CADM1-m18, 0.767 for MAL-m1 and 0.762 for miR-124-2. This indicates that CADM1-m18 is not suitable as single marker. By varying the thresholds of both markers in the bi-marker panels CADM1-m18/MAL-m1, CADM1-m18/miR-124-2 and MAL-m1/miR-124-2 upper and lower ROC curves were obtained, depicting the maximum and minimum CIN3+ sensitivity, respectively, at given specificity. For all these bi-marker combinations, the upper curves were similar. However, for the MAL-m1/miR-124-2 panel, the distance between upper and lower ROC curves was closest and this panel displayed the highest assay thresholds, indicating that this combination was most robust. At clinical specificities of 50 and 70%, the MAL-m1/miR-124-2 sensitivity for detection of CIN3+ ranged from 77.0 to 87.8% and from 64.9 to 71.6%, respectively. At 70% specificity thresholds no carcinomas were missed. By comparison, the CIN3+ sensitivity of HPV16/18 genotyping on the self-sampled lavage specimens was 58.1% (95%CI: 46.6-68.8) at a specificity of 87.7% (95%CI: 83.2-91.2). In conclusion, methylation analysis is a promising triage tool that in combination with HPV-DNA testing offers feasible, full molecular screening on self-collected cervico-vaginal lavage specimens.

ROC analysis for multiple markers with tree-based classification

Multiple biomarkers are frequently observed or collected for detecting or understanding a disease. The research interest of this article is to extend tools of receiver operating characteristic (ROC) analysis from univariate marker setting to multivariate marker setting for evaluating predictive accuracy of biomarkers using a tree-based classification rule. Using an arbitrarily combined and-or classifier, an ROC function together with a weighted ROC function (WROC) and their conjugate counterparts are introduced for examining the performance of multivariate markers. Specific features of the ROC and WROC functions and other related statistics are discussed in comparison with those familiar properties for univariate marker. Nonparametric methods are developed for estimating the ROC and WROC functions, and area under curve and concordance probability. With emphasis on population average performance of markers, the proposed procedures and inferential results are useful for evaluating marker predictability based on multivariate marker measurements with different choices of markers, and for evaluating different and-or combinations in classifiers.

Bivariate marker measurements and ROC analysis

This article considers receiver operating characteristic (ROC) analysis for bivariate marker measurements. The research interest is to extend tools and rules from univariate marker to bivariate marker setting for evaluating predictive accuracy of markers using a tree-based classification rule. Using an and-or classifier, an ROC function together with a weighted ROC function (WROC) and their conjugate counterparts are proposed for examining the performance of bivariate markers. The proposed functions evaluate the performance of and-or classifiers among all possible combinations of marker values, and are ideal measures for understanding the predictability of biomarkers in target population. Specific features of ROC and WROC functions and other related statistics are discussed in comparison with those familiar properties for univariate marker. Nonparametric methods are developed for estimating ROC-related functions (partial) area under curve and concordance probability. With emphasis on average performance of markers, the proposed procedures and inferential results are useful for evaluating marker predictability based on a single or bivariate marker (or test) measurements with different choices of markers, and for evaluating different and-or combinations in classifiers. The inferential results developed in this article also extend to multivariate markers with a sequence of arbitrarily combined and-or classifier.