Corrected ROC analysis for misclassified binary outcomes

Creating accurate risk prediction models from Big Data resources such as Electronic Health Records (EHRs) is a critical step toward achieving precision medicine. A major challenge in developing these tools is accounting for imperfect aspects of EHR data, particularly the potential for misclassified outcomes. Misclassification, the swapping of case and control outcome labels, is well known to bias effect size estimates for regression prediction models. In this paper, we study the effect of misclassification on accuracy assessment for risk prediction models and find that it leads to bias in the area under the curve (AUC) metric from standard ROC analysis. The extent of the bias is determined by the false positive and false negative misclassification rates as well as disease prevalence. Notably, we show that simply correcting for misclassification while building the prediction model is not sufficient to remove the bias in AUC. We therefore introduce an intuitive misclassification-adjusted ROC procedure that accounts for uncertainty in observed outcomes and produces bias-corrected estimates of the true AUC. The method requires that misclassification rates are either known or can be estimated, quantities typically required for the modeling step. The computational simplicity of our method is a key advantage, making it ideal for efficiently comparing multiple prediction models on very large datasets. Finally, we apply the correction method to a hospitalization prediction model from a cohort of over 1 million patients from the Veterans Health Administrations EHR. Implementations of the ROC correction are provided for Stata and R. Published 2017. This article is a U.S. Government work and is in the public domain in the USA.

Optimal Cutoff Scores for Alzheimer's Disease Using the Chinese Version of Mini-Mental State Examination Among Chinese Population Living in Rural Areas

To explore the optimal cutoff score for initial detection of Alzheimer's Disease (AD) through the Chinese version of Mini-Mental State Examination (CMMSE) in rural areas in China, we conducted a cross-sectional study within the Linxian General Population Nutritional Follow-up study. 16,488 eligible cohort members participated in the survey and 881 completed the CMMSE. Among 881 participants, the median age (Interquartile range) was 69.00 (10.00), 634 (71.92%) were female, 657 (74.57%) were illiterate, 35 (3.97%) had 6 years of education or higher, and 295 (33.48%) were diagnosed with AD. By reducing the CMMSE criteria for illiterate to 16 points, primary school to 19 points, and middle school or higher to 23 points, the efficiency of Chinese version of Mini-Mental State Examination can be significantly improved for initial detection of AD in rural areas in China, especially in those nutrition deficient areas.

BAP1 immunohistochemistry and p16 FISH results in combination provide higher confidence in malignant pleural mesothelioma diagnosis: ROC analysis of the two tests

Differentiation of malignant pleural mesothelioma (MPM) from benign mesothelial proliferation remains problematic. Loss of nuclear staining of BRCA1-associated protein 1 (BAP1; detected using immunohistochemistry (IHC)) and homozygous deletion (HD) of p16 (detected using fluorescence in situ hybridization (FISH)) are useful for differentiation of MPM from reactive mesothelial hyperplasia (RMH), but the correlation between BAP1 expression loss and p16 HD has not been fully described. We performed BAP1 IHC and p16-specific FISH for 40 MPM and 20 RMH cases, and measured proportions of cells showing BAP1 expression and p16 HD for each case. The diagnostic accuracy for MPM and the cut-off values of the two methods were assessed using receiver operating characteristic (ROC) analysis. BAP1 expression loss, p16 HD and coexistence of both were present in 27 (67.5 %), 27 (67.5 %) and 17 (42.5 %) MPM cases, respectively. Three MPM cases (7.5 %) and all 20 RMH cases had neither BAP1 loss nor p16 HD. There was no correlation between the results of the two methods. Their combination showed higher sensitivity (92.5 %, 37/40) and estimated probability than BAP1 IHC and p16-specific FISH used alone. BAP1 IHC and p16-specific FISH have independent diagnostic value, and have increased reliability when used in combination, for MPM diagnosis.

Identifying key parent-reported symptoms for detecting depression in high risk adolescents

Adolescent offspring of depressed parents are at particularly heightened risk of developing early onset Major Depressive Disorder (MDD) yet are unlikely to access services. We therefore aimed to identify a parsimonious combination of parent-reported symptoms that accurately detected offspring MDD. We used a multi-sample study comprising a development sample of 335 offspring of adults with recurrent MDD assessed on three occasions (mean age 12.4-14.8 years) and an independent validation sub-sample of 807 adolescents from a general population cohort (mean age 13.1 years). Parent ratings of psychiatric symptoms in adolescent offspring were assessed using established questionnaires. The best performing four-item combination of symptoms was identified. Accuracy in detecting concurrent DSM-IV MDD diagnosis, assessed by direct adolescent and parent interviews, was compared to the well-established 13-item short Moods and Feelings Questionnaire (sMFQ) using ROC curve analysis. The combination identified (concentration problems, anhedonia, worrying excessively and feeling unloved) performed equivalently to the sMFQ both in the development dataset and in the validation dataset. We concluded that a combination of four parent-reported mental health items performs equivalently to an established, longer depression questionnaire measure in detecting a diagnosis of adolescent major depressive disorder among offspring of parents with recurrent MDD and needs further evaluation.

Classify vertebrate hemoglobin proteins by incorporating the evolutionary information into the general PseAAC with the hybrid approach

Hemoglobin is an oxygen-binding protein widely present in all kingdoms of life from prokaryotic to eukaryotic, but well established in the vertebrate system. An attempt was made to determine the Vertebrate hemoglobin (VerHb) protein on their animal classifications, based on general pseudo amino acid composition (PseAAC)'s evolutionary profiles and hybrid approach. The support vector machine (SVM) has been applied to develop all models, the prediction results further compared according to their animal classification. The performance of the approaches estimated using five-fold cross-validation techniques. The prediction performance was further investigated by receiver operating characteristic (ROC) and prediction score graphs. The prediction accuracy (ACC), sensitivity (SN) and specificity (SP) were examined to find the accurate predictions on the threshold level. Based on the approach, a web-tool has been developed for identifying the VerHb proteins.

Inter-observer variation and diagnostic efficacy of apparent diffusion coefficient (ADC) measurements obtained by diffusion-weighted imaging (DWI) in small renal masses

BACKGROUND

Diffusion-weighted imaging (DWI) is increasingly used to diagnose renal lesion subtypes. Especially in small renal masses, identification of less aggressive tumor types is of clinical interest, as active surveillance strategies can be applied.

PURPOSE

To evaluate the inter-observer variation and diagnostic efficacy of apparent diffusion coefficient (ADC) measurements obtained by DWI in small renal masses ≤4 cm (SRM).

MATERIAL AND METHODS

This retrospective IRB-approved study included 39 patients (46 SRM: 12 benign, 34 malignant). All underwent a 3 T DWI of SRM prior to surgery. Two radiologists independently analyzed all imaging data by three measurements. Limits of agreement, intraclass correlation coefficients (ICC), group comparisons by t-tests, and ROC analysis were performed.

RESULTS

Reliability of ADC measurements was very high with an ICC of >0.9 for both observers. Inter-rater reliability was high with an ICC of 0.82. Limits of agreement for average ADC values between both observers were -23.5% to 38.3% with a mean difference of 7.5% between both observers. No significant differences were found between benign and malignant lesions (P value Observer 1: 0.362, Observer 2: 0.622). Papillary carcinoma showed lower ADC values compared to non-papillary carcinoma (P value Observer 1: 0.008, Observer 2: 0.012). Consequently, ROC analysis revealed a significant (P < 0.001, respectively) area under the ROC curve of 0.853 (Observer 1) and 0.837 (Observer 2) without significant differences between both readers (P = 0.772).

CONCLUSION

ADC measurements of SRM at 3 T show a high reproducibility and differentiate papillary from non-papillary carcinoma subtypes. However, measurement variability may limit the application of fixed ADC thresholds for lesion diagnosis.

Development and validation of a blocking ELISA test for the detection of avian influenza antibodies in poultry species

Avian influenza (AI) is an infectious viral disease usually asymptomatic in wild birds that can spread to domestic poultry and cause large-scale outbreaks. Some of the AI viruses (AIV) can cross the species barrier and induce fatal disease to humans, a matter of great concern worldwide. Early detection of AIV is of major importance for disease control, since prompt implementation of adequate measures can prevent spread of the virus and therefore further outbreaks. This paper reports the development and validation of a blocking ELISA using a monoclonal antibody against a conserved structural protein for the serologic diagnosis of all AI virus subtypes from domestic bird species, allowing the quick, easily automated and low-cost screening of a high number of farms. The test will be of great value not only for surveillance, but also for monitoring the efficiency of vaccination programs. Cut-off values were established in 20% of inhibition for turkey sera and in 50% of inhibition for chicken and duck sera. Estimations of 100% specificity and 100% sensitivity were obtained based on the results of known AI positive (n=130) and negative (n=208) sera, including serum samples from birds infected with other common avian viral pathogens (n=7). ROC analysis showed an area under curve (AUC) of 1.0 for chicken, duck and turkey sera, indicating that the blocking ELISA was able to perfectly discriminate between negative and positive samples of any of the poultry species tested. Inter- and intra-assay coefficients of variation were above the acceptance threshold. Furthermore, the ELISA titers were similar to the known hemagglutination inhibition titers of three positive reference sera indicating sensitivity comparable with the golden standard HI method. The method here described is an economically attractive alternative to the commercial ELISAs currently available.

VoxelStats: A MATLAB Package for Multi-Modal Voxel-Wise Brain Image Analysis

In healthy individuals, behavioral outcomes are highly associated with the variability on brain regional structure or neurochemical phenotypes. Similarly, in the context of neurodegenerative conditions, neuroimaging reveals that cognitive decline is linked to the magnitude of atrophy, neurochemical declines, or concentrations of abnormal protein aggregates across brain regions. However, modeling the effects of multiple regional abnormalities as determinants of cognitive decline at the voxel level remains largely unexplored by multimodal imaging research, given the high computational cost of estimating regression models for every single voxel from various imaging modalities. VoxelStats is a voxel-wise computational framework to overcome these computational limitations and to perform statistical operations on multiple scalar variables and imaging modalities at the voxel level. VoxelStats package has been developed in Matlab(®) and supports imaging formats such as Nifti-1, ANALYZE, and MINC v2. Prebuilt functions in VoxelStats enable the user to perform voxel-wise general and generalized linear models and mixed effect models with multiple volumetric covariates. Importantly, VoxelStats can recognize scalar values or image volumes as response variables and can accommodate volumetric statistical covariates as well as their interaction effects with other variables. Furthermore, this package includes built-in functionality to perform voxel-wise receiver operating characteristic analysis and paired and unpaired group contrast analysis. Validation of VoxelStats was conducted by comparing the linear regression functionality with existing toolboxes such as glim_image and RMINC. The validation results were identical to existing methods and the additional functionality was demonstrated by generating feature case assessments (t-statistics, odds ratio, and true positive rate maps). In summary, VoxelStats expands the current methods for multimodal imaging analysis by allowing the estimation of advanced regional association metrics at the voxel level.

Functional specificity of rat vibrissal primary afferents

In this study, we propose to analyze the peripheral vibrissal system specificity through its neuronal responses. Receiver operating characteristics (ROC) curve analyses were used, which required the implementation of a binary classifier (artificial neural network) trained to identify the applied stimulus. The training phase consisted of the observation of a predetermined amount of vibrissal sweeps on two surfaces of different texture and similar roughness. Our results suggest that the specificity of the peripheral vibrissal system easily permits the discrimination between perceived stimuli, quantified through neuronal responses, and that it can be evaluated through an ROC curve analysis. We found that such specificity makes a linear binary classifier capable of detecting differences between stimuli with five sweeps at most.

Impairment due to eating disorder pathology: Identifying the cut-off score on the Clinical Impairment Assessment in a clinical and community sample

OBJECTIVE

The Clinical Impairment Assessment (CIA) is a self-report measure of impairment secondary to eating disorder (ED) features. The purpose of this study was to identify the global CIA cut-off score that maximized sensitivity and specificity to discriminate impairment due to eating disorder pathology in a community versus clinical ED sample using receiver operating characteristic (ROC) analyses.

METHOD

Participants were 1,468 female community participants and 552 eating disorder patients.

RESULTS

Mean global CIA scores were 5.17 (SD 7.61) and 32.50 (SD 10.20). The ROC analysis demonstrated excellent accuracy of the global CIA score (AUC = 0.97; 95% CI: 0.96-0.98). A cut-off score of 16.0 yielded a sensitivity of 0.91 (95% CI: 0.88-0.93) and a specificity of 0.91 (95% CI: 0.89-0.93).

DISCUSSION

This study is the first to replicate findings from the original development study of the CIA and lends support to initial recommendations. Data provide strong evidence of the discriminant validity of the CIA and suggest the utility of assessing eating-disorder related impairment for classification purposes. © 2016 Wiley Periodicals, Inc. (Int J Eat Disord 2016; 49:635-638).

Contrast-enhanced ultrasonography for detecting histological carotid plaque rupture: Quantitative analysis of ulcer

BACKGROUND

Few studies have evaluated the distinct ability of contrast-enhanced ultrasonography for detecting carotid plaque rupture versus histological observations.

AIMS

The aim of this study was to quantitatively assess the ability of contrast-enhanced ultrasonography to detect plaque rupture compared to ultrasonographic and histological images in terms of geometric accordance.

METHODS

Carotid plaque morphology was classified as "smooth," "irregular," or "ulcerated" on 45 conventional ultrasonography and contrast-enhanced ultrasonography images from consecutive patients undergoing endarterectomy, and 55 regions of interests were captured on contrast-enhanced ultrasonography. A comparative study with a receiver operating characteristic analysis was performed using histological findings for reference.

RESULTS

Contrast-enhanced ultrasonography exhibited a higher percentage of "ulcerated" findings in patients with plaque rupture compared to conventional ultrasonography (P = 0.002) as well as an association with thrombus formation (P = 0.048) and fibrous cap disruption (P < 0.0001). On contrast-enhanced ultrasonography, "ulcerated" were significantly more likely than "smooth" findings when the fibrous cap was disrupted (odds ratio (OR), 41.5). The receiver operating characteristic areas under the curve for the orifice, depth, and width of the concavities on contrast-enhanced ultrasonography were significantly greater than 0.5, while their optimal cut-off values were 1.40 mm, 1.30 mm, and 1.88 mm, respectively. When one of these variables was greater than the optimal cut-off value, the sensitivity, negative hit rate, and odds ratio for detecting fibrous cap disruption were 91.3%, 91.6%, and 23.1, respectively.

CONCLUSIONS

In our study, contrast-enhanced ultrasonography has high sensitivity for identifying histological plaque rupture, and the measurement of concavity on contrast-enhanced ultrasonography may enable the accurate detection of fibrous cap disruption.

Heart-Type Fatty Acid-Binding Protein, in Early Detection of Acute Myocardial Infarction: Comparison with CK-MB, Troponin I and Myoglobin

The study aimed to investigate whether heart-type fatty acid binding protein (H-FABP) measurement provides additional diagnostic value to that of conventional cardiac markers in acute myocardial infarction (AMI) within first 6 h after the onset of symptoms. The study included 120 subjects: 60 AMI cases and 60 age and sex matched controls. The cases and controls were further divided into 2 subgroups depending on the time since onset of chest pain as (1) subjects within 3 h and (2) between 3 and 6 h of onset of chest pain. In all the cases and controls, serum H-FABP concentration was measured by Immunoturbidimetric method, serum Troponin I and myoglobin concentrations by Chemiluminescence immunoassay and serum CK-MB concentration by Immuno-inhibition method. The sensitivity, specificity, positive and negative predictive values of H-FABP were significantly greater than CK-MB and myoglobin but were lesser than Troponin I in patients with suspected AMI in both within 3 h and 3-6 h groups. Receiver operating characteristic curves demonstrated greatest diagnostic ability for Troponin I (AUC = 0.99, p < 0.001) followed by H-FABP (AUC = 0.906, p < 0.001) within 3 h and 3-6 h after the onset of chest pain. In conclusion, the diagnostic value of H-FABP is greater than CK-MB and myoglobin but slightly lesser than troponin I for the early diagnosis of AMI within first 6 h of chest pain. H-FABP can be used as an additional diagnostic tool for the early diagnosis of AMI along with troponin I.

Improved nonparametric estimation of the optimal diagnostic cut-off point associated with the Youden index under different sampling schemes

A diagnostic cut-off point of a biomarker measurement is needed for classifying a random subject to be either diseased or healthy. However, the cut-off point is usually unknown and needs to be estimated by some optimization criteria. One important criterion is the Youden index, which has been widely adopted in practice. The Youden index, which is defined as the maximum of (sensitivity + specificity -1), directly measures the largest total diagnostic accuracy a biomarker can achieve. Therefore, it is desirable to estimate the optimal cut-off point associated with the Youden index. Sometimes, taking the actual measurements of a biomarker is very difficult and expensive, while ranking them without the actual measurement can be relatively easy. In such cases, ranked set sampling can give more precise estimation than simple random sampling, as ranked set samples are more likely to span the full range of the population. In this study, kernel density estimation is utilized to numerically solve for an estimate of the optimal cut-off point. The asymptotic distributions of the kernel estimators based on two sampling schemes are derived analytically and we prove that the estimators based on ranked set sampling are relatively more efficient than that of simple random sampling and both estimators are asymptotically unbiased. Furthermore, the asymptotic confidence intervals are derived. Intensive simulations are carried out to compare the proposed method using ranked set sampling with simple random sampling, with the proposed method outperforming simple random sampling in all cases. A real data set is analyzed for illustrating the proposed method.

Wheezing recognition algorithm using recordings of respiratory sounds at the mouth in a pediatric population

BACKGROUND

Respiratory diseases in children are a common reason for physician visits. A diagnostic difficulty arises when parents hear wheezing that is no longer present during the medical consultation. Thus, an outpatient objective tool for recognition of wheezing is of clinical value.

METHOD

We developed a wheezing recognition algorithm from recorded respiratory sounds with a Smartphone placed near the mouth. A total of 186 recordings were obtained in a pediatric emergency department, mostly in toddlers (mean age 20 months). After exclusion of recordings with artefacts and those with a single clinical operator auscultation, 95 recordings with the agreement of two operators on auscultation diagnosis (27 with wheezing and 68 without) were subjected to a two phase algorithm (signal analysis and pattern classifier using machine learning algorithms) to classify records.

RESULTS

The best performance (71.4% sensitivity and 88.9% specificity) was observed with a Support Vector Machine-based algorithm. We further tested the algorithm over a set of 39 recordings having a single operator and found a fair agreement (kappa=0.28, CI95% [0.12, 0.45]) between the algorithm and the operator.

CONCLUSIONS

The main advantage of such an algorithm is its use in contact-free sound recording, thus valuable in the pediatric population.

Accuracy, Precision, Sensitivity, and Specificity of Noninvasive ICP Absolute Value Measurements

An innovative absolute intracranial pressure (ICP) value measurement method has been validated by multicenter comparative clinical studies. The method is based on two-depth transcranial Doppler (TCD) technology and uses intracranial and extracranial segments of the ophthalmic artery as pressure sensors. The ophthalmic artery is used as a natural pair of "scales" that compares ICP with controlled pressure Pe, which is externally applied to the orbit. To balance the scales, ICP = Pe a special two-depth TCD device was used as a pressure balance indicator. The proposed method is the only noninvasive ICP measurement method that does not need patient-specific calibration.

IgA Nephropathy in Children: A Multicenter Study in Poland

IgA nephropathy (IgAN) is the most common form of glomerulonephritis in pediatric population. The clinical presentation of the disease in children ranges from microscopic hematuria to end-stage kidney disease. The aim of the study was to retrospectively assess clinical and kidney biopsy features in children with IgAN. We assessed a cohort of 140 children, 88 boys, 52 girls with the diagnosis of IgAN in the period of 2000-2015, entered into the national Polish pediatric IgAN registry. The assessment included the following: proteinuria, hematuria, glomerular filtration rate (GFR), arterial blood pressure, and the renal pathological changes according to the Oxford classification and crescents formation, as modifiable and unmodifiable risk factors. The incidence of IgAN in Poland was set at 9.3 new cases per year. The mean age at onset of IgAN was 11.9 ± 4.3 years, and the most common presentation of the disease was the nephritic syndrome, recognized in 52 % of patients. Kidney biopsy was performed, on average, 1.3 ± 2.0 years after onset of disease. Based on the ROC analysis, a cut-off age at onset of disease for GFR <90 mL/min/1.73 m² (risk factor of progression) was calculated as 13.9 years. Unmodifiable lesions: segmental sclerosis, tubular atrophy/interstitial fibrosis (S1, T1-2) in the Oxford classification and crescents in kidney biopsy were significantly more common in Gr 1 (>13.9 years) compared with Gr 2 (<13.9 years), despite a significantly shorter time to kidney biopsy in the former. We conclude that IgAN in children may be an insidious disease. A regular urine analysis, especially after respiratory tract infections, seems the best way for an early detection of the disease.

α-Tocopherol Stereoisomers in Human Plasma Are Affected by the Level and Form of the Vitamin E Supplement Used

BACKGROUND

Studies examining vitamin E intake and the percentage of the population meeting dietary guidelines do not distinguish between natural (RRR-α-tocopherol) and synthetic (all-rac-α-tocopherol) intake, even though these different isomeric forms differ in bioactivity.

OBJECTIVE

This study aimed to determine the effect of RRR-α-tocopherol vs. all-rac-α-tocopherol intake on the percentage of the population meeting the vitamin E recommendation and on plasma α-tocopherol stereoisomer distribution.

METHODS

With the use of data from the Irish National Adult Nutrition Survey (NANS), this study examined the percentage of the Irish population meeting the European Union (EU) RDA for vitamin E of 12 mg/d, correcting for a bioactivity difference in all-rac- vs. RRR-α-tocopherol, where 1 mg of all-rac-α-tocopherol is considered to be equivalent to 1:1.36 (0.74) mg in the EU RDA. In a subcohort of supplement users and nonusers, plasma α- and γ-tocopherol concentrations and α-tocopherol stereoisomer distribution were measured. Receiver operating characteristic (ROC) curve analysis was conducted to determine ability to discriminate supplement user types.

RESULTS

Analysis of the NANS showed that 100% of participants still met the recommended intake of 12 mg/d, after all-rac-α-tocopherol intake was corrected for α-tocopherol equivalent bioactivity. In the subcohort analysis, the percentage of plasma RRR-α-tocopherol was significantly lower in high all-rac-α-tocopherol supplement (>11 mg/d) users (82%) compared with nonusers and with high RRR-α-tocopherol supplement (>35 mg/d) users (91% and 93% respectively, P < 0.01). High RRR-α-tocopherol supplement users had a significantly higher plasma α-tocopherol than low all-rac-α-tocopherol supplement (<2.5 mg/d) users (34 vs. 25 μmol/L, P = 0.01). ROC analysis demonstrated an ability to distinguish between RRR- and all-rac-α-tocopherol consumers, which may be useful in investigating the potential effect of RRR- and all-rac-α-tocopherol intake on health.

CONCLUSIONS

This study demonstrated that the percentage of the population meeting the vitamin E recommendation was unaffected when all-rac-α-tocopherol intake was corrected for α-tocopherol equivalent bioactivity. all-rac-α-Tocopherol intake led to a decrease in the percentage of plasma RRR-α-tocopherol relative to RRR-α-tocopherol intake.

Heart-Type Fatty Acid Binding Protein: A Better Cardiac Biomarker than CK-MB and Myoglobin in the Early Diagnosis of Acute Myocardial Infarction

BACKGROUND

Early diagnosis and therapeutic intervention can improve the outcome of acute myocardial infarction (AMI). However, there are no satisfactory cardiac biomarkers for the diagnosis of AMI within 6 hours of onset of symptoms. Among novel biochemical markers of AMI, heart-type fatty acid binding protein (H-FABP) is of particular interest.

AIM

To compare the diagnostic value of H-FABP with that of CK-MB and myoglobin in suspected AMI patients within first 6 hours after the onset of symptoms.

SETTINGS AND DESIGN

The study includes 40 AMI cases and 40 non-cardiac chest pain otherwise healthy controls. The cases and controls were further divided into 2 groups depending on the time since chest pain as those subjects within 3 hours and those between 3-6 hours of onset of chest pain.

MATERIALS AND METHODS

In all the cases and controls, serum H-FABP, CK-MB and myoglobin concentrations were measured by Immunoturbidimetric method, immuno-inhibition method and Chemiluminescence immunoassay respectively.

STATISTICAL ANALYSIS

Data is presented as mean ± SD values. Differences between means of two groups were assessed by Student t-test. Sensitivity, Specificity, Positive predictive value, Negative predictive values were calculated and ROC curve analysis was done to assess the diagnostic validity of each study parameter.

RESULTS

The sensitivity, specificity, PPV, NPV of H-FABP were greater than CK-MB and myoglobin and ROC curve analysis demonstrated highest area under curve for H-FABP followed by myoglobin and CK-MB in patients with suspected AMI both within 3 hours and 3-6 hours after the onset of chest pain.

CONCLUSION

The diagnostic efficiency of H-FABP is greater than CK-MB and myoglobin for the early diagnosis of AMI within first 6 hours of chest pain. H-FABP can be used as an additional diagnostic tool for the early diagnosis of AMI.

Validation of Nonparametric Two-Sample Bootstrap in ROC Analysis on Large Datasets

The nonparametric two-sample bootstrap is applied to computing uncertainties of measures in ROC analysis on large datasets in areas such as biometrics, speaker recognition, etc., when the analytical method cannot be used. Its validation was studied by computing the SE of the area under ROC curve using the well-established analytical Mann-Whitney-statistic method and also using the bootstrap. The analytical result is unique. The bootstrap results are expressed as a probability distribution due to its stochastic nature. The comparisons were carried out using relative errors and hypothesis testing. They match very well. This validation provides a sound foundation for such computations.

Positive impact of elastography in breast cancer diagnosis: an institutional experience

Elastography (ES) is a technique that, when associated with traditional B mode ultrasound (US), allows the degree of elasticity of tissue to be evaluated according to a color scale system. The aims of the study were to compare the diagnostic characteristics of two widely used techniques adopted in breast cancer screening; US and color Doppler (CD), with those of the same two techniques plus ES, and assessment of the same diagnostic characteristics when the three methods were applied to lesions < or >1 cm. Methods used included subjecting 212 women to investigations aimed at the early diagnosis of breast cancer outside the screening model, whereby 395 lesions were detected by US, ES, and CD, with a definitive diagnosis proved by histological exam. The diagnostic performance of US, ES, CD, and their combinations was calculated. The results showed that comparing the diagnostic characteristics of the three methods with reference to the definitive histological results for malignant breast lesions, the best diagnostic accuracy was obtained when US, ES, and CD were combined (0.837). For lesions <1 cm, diagnostic accuracy was 0.782, and for those >1 cm, it was 0.886. In the lesions <1 cm, which were more difficult to study, a positive ES score (>4) appeared to be sufficient to deepen the diagnosis, even though 35 % of the ES or US positive lesions were not malignant. By contrast, in lesions >1 cm, the probability of having a malignant lesion when all three tests were positive was very high (97 %). It was concluded that early diagnosis is a key factor in breast cancer, so an economically sustainable, non-invasive pathway is the target of diagnostic breast imaging.

Semiparametric transformation models for multiple continuous biomarkers in ROC analysis

Recent technological advances continue to provide noninvasive and more accurate biomarkers for evaluating disease status. One standard tool for assessing the accuracy of diagnostic tests is the receiver operating characteristic (ROC) curve. Few statistical methods exist to accommodate multiple continuous-scale biomarkers in the framework of ROC analysis. In this paper, we propose a method to integrate continuous-scale biomarkers to optimize classification accuracy. Specifically, we develop semiparametric transformation models for multiple biomarkers. We assume that unknown and marker-specific transformations of biomarkers follow a multivariate normal distribution. Our models accommodate biomarkers subject to limits of detection and account for the dependence among biomarkers by including a subject-specific random effect. We also propose a diagnostic measure using an optimal linear combination of the transformed biomarkers. Our diagnostic rule does not depend on any monotone transformation of biomarkers and is not sensitive to extreme biomarker values. Nonparametric maximum likelihood estimation (NPMLE) is used for inference. We show that the parameter estimators are asymptotically normal and efficient. We illustrate our semiparametric approach using data from the Endometriosis, Natural History, Diagnosis, and Outcomes (ENDO) study.

Independent replication of a melanoma subtype gene signature and evaluation of its prognostic value and biological correlates in a population cohort

Development and validation of robust molecular biomarkers has so far been limited in melanoma research. In this paper we used a large population-based cohort to replicate two published gene signatures for melanoma classification. We assessed the signatures prognostic value and explored their biological significance by correlating them with factors known to be associated with survival (vitamin D) or etiological routes (nevi, sun sensitivity and telomere length). Genomewide microarray gene expressions were profiled in 300 archived tumors (224 primaries, 76 secondaries). The two gene signatures classified up to 96% of our samples and showed strong correlation with melanoma specific survival (P=3 x 10(-4)), Breslow thickness (P=5 x 10(-10)), ulceration (P=9.x10-8) and mitotic rate (P=3 x 10(-7)), adding prognostic value over AJCC stage (adjusted hazard ratio 1.79, 95%CI 1.13-2.83), as previously reported. Furthermore, molecular subtypes were associated with season-adjusted serum vitamin D at diagnosis (P=0.04) and genetically predicted telomere length (P=0.03). Specifically, molecular high-grade tumors were more frequent in patients with lower vitamin D levels whereas high immune tumors came from patients with predicted shorter telomeres. Our data confirm the utility of molecular biomarkers in melanoma prognostic estimation using tiny archived specimens and shed light on biological mechanisms likely to impact on cancer initiation and progression.

Evaluation of Early Reperfusion Criteria in Acute Ischemic Stroke

BACKGROUND AND PURPOSE

Though still debated, early reperfusion is increasingly used as a biomarker for clinical outcome. However, the lack of a standard definition hinders the assessment of reperfusion therapies and study comparisons. The objective was to determine the optimal early reperfusion criteria that predicts clinical outcome in ischemic stroke.

METHODS

Early reperfusion was assessed voxel-wise in 57 patients within 6 hours of symptom onset. The performance of the time to peak (TTP), the mean transit time (MTT), and the time to maximum of residue function (Tmax ) at various delays thresholds in predicting the neurological response (based on the National Institutes of Health Stroke Scale) and the functional outcome (modified Rankin scale ≤1) at 1 month were compared. A receiver operating characteristics (ROC) analysis determined the optimal extent of reperfusion. A novel unsupervised classification of reperfusion using group-based trajectory modeling (GBTM) was evaluated.

RESULTS

MTT had a lower performance than TTP and Tmax in predicting the neurological response (P = .008 vs. TTP and P = .006 vs. Tmax ) or the functional outcome (P = .0006 vs. TTP; P = .002 vs. Tmax ). No delay threshold had a significantly higher predictive value than another. The optimal percentage of reperfusion was dependent on the outcome scale (P < .001). The GBTM-based classification of reperfusion was closely associated with the clinical outcome and had a similar accuracy compared to ROC-based classification.

CONCLUSIONS

TTP and Tmax should be preferred to MTT in defining early reperfusion. GBTM provided a clinically relevant reperfusion classification that does not require prespecified delay thresholds or clinical outcomes.

Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms: a simulation study

RATIONALE AND OBJECTIVES

To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies.

MATERIALS AND METHODS

We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6% of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10%-80%; and (4) SRS-D and multiple imputation (SRS-D + MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6% proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95% confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25% were cancer cases) versus from the large case population.

RESULTS

AUC estimates were unbiased from RS, SRS-P, and SRS-D + MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D + MI had 10% smaller root-mean-square error than RS.

CONCLUSIONS

Both SRS-P and SRS-D + MI can be used to obtain unbiased and 10% more efficient estimate of screening-mammography ROC curves.

Aciduric microbial taxa including Scardovia wiggsiae and Bifidobacterium spp. in caries and caries free subjects

Actinobacteria came into focus of being potential caries-associated pathogens and could, together with the established Streptococcus mutans and lactobacilli thus function as caries indicator species. Here we analyzed the role and diagnostic predictive value of the acidogenic-aciduric species Scardovia wiggsiae and Bifidobacterium dentium together with S. mutans, lactobacilli and bifidobacteria in biofilm of non-cavitated (n = 20) and cavitated (n = 6) caries lesions versus controls (n = 30). For the genus Bifidobacterium and for B. dentium new sets of primers were designed. Based on real-time quantitative PCR and confirmed by DNA sequencing we found a higher prevalence (61.5%) of S. wiggsiae in caries lesions than in controls (40%). However, among the controls we found three individuals with both the highest absolute and relative S. wiggsiae numbers. Testing for S. mutans revealed the same prevalence as S. wiggsiae in caries lesions (61.5%) but in controls its prevalence was only 10%. B. dentium was never found in healthy plaque but in 30.8% of clinical cases, with the highest numbers in cavitated lesions. The Bifidobacterium-genus specific PCR had less discriminative power as more control samples were positive. We calculated the relative abundances and applied receiver operating characteristic analyses. The top results of specificity (93% and 87%) and sensitivity (100% and 88%) were found when the constraint set was "Lactobacillus relative abundance ≥0.02%" and "two aciduric species with a relative abundance of each ≥0.007%". Combinatory measurement of several aciduric taxa may be useful to reveal caries activity or even to predict caries progression.