Applications of ROC analysis in medical research: recent developments and future directions

Unveiling novel metabolic alterations in postmenopausal osteoporosis and type 2 diabetes mellitus through NMR-based metabolomics: A pioneering approach for identifying early diagnostic markers

BACKGROUND AND AIMS

Postmenopausal osteoporosis (PMO) and type 2 diabetes mellitus (T2DM) frequently coexist in postmenopausal women. The study aimed to explore metabolic variations linked to these circumstances and their simultaneous presence through proton nuclear magnetic resonance metabolomics (1H NMR).

MATERIALS AND METHODS

Serum samples from 80 postmenopausal women, including 20 PMO individuals, 20 T2DM, 20 T2DM + PMO, and 20 healthy postmenopausal women, were analyzed using 1H NMR spectroscopy.

RESULTS

Our study revealed significant metabolic profile differences among the four groups. Notably, the T2DM + PMO group showed elevated levels of alanine, pyruvate, glutamate, lactate, and aspartate, indicating their involvement in lipid metabolism, energy, and amino acids. Importantly, our multivariate statistical analysis identified a metabolite set that accurately distinguished the groups, suggesting its potential as an early diagnostic marker.

CONCLUSION

The 1H NMR metabolomics approach uncovered metabolic biomarkers intricately linked to postmenopausal osteoporosis (PMO), type 2 diabetes mellitus (T2DM), and their concurrent presence. Among these biomarkers, alanine emerged as a pivotal player, showing its significant role in the metabolic landscape associated with PMO and T2DM. These findings shed light on the pathophysiological mechanisms underlying these conditions and underscore alanine's potential as a diagnostic biomarker.

A multi-omics study to monitor senescence-associated secretory phenotypes of Alzheimer's disease

OBJECTIVE

Alzheimer's disease (AD) is characterized by the progressive degeneration and damage of neurons in the brain. However, developing an accurate diagnostic assay using blood samples remains a challenge in clinic practice. The aim of this study was to explore senescence-associated secretory phenotypes (SASPs) in peripheral blood using mass spectrometry based multi-omics approach and to establish diagnostic assays for AD.

METHODS

This retrospective study included 88 participants, consisting of 29 AD patients and 59 cognitively normal (CN) individuals. Plasma and serum samples were examined using high-resolution mass spectrometry to identify proteomic and metabolomic profiles. Receiver operating characteristic (ROC) analysis was employed to screen biomarkers with diagnostic potential. K-nearest neighbors (KNN) algorithm was utilized to construct a multi-dimensional model for distinguishing AD from CN.

RESULTS

Proteomics analysis revealed upregulation of five plasma proteins in AD, including RNA helicase aquarius (AQR), zinc finger protein 587B (ZNF587B), C-reactive protein (CRP), fibronectin (FN1), and serum amyloid A-1 protein (SAA1), indicating their potential for AD classification. Interestingly, KNN-based three-dimensional model, comprising AQR, ZNF587B, and CRP, demonstrated its high accuracy in AD recognition, with evaluation possibilities of 0.941, 1.000, and 1.000 for the training, testing, and validation datasets, respectively. Besides, metabolomics analysis suggested elevated levels of serum phenylacetylglutamine (PAGIn) in AD.

INTERPRETATION

The multi-omics outcomes highlighted the significance of the SASPs, specifically AQR, ZNF587B, CRP, and PAGIn, in terms of their potential for diagnosing AD and suggested neuronal aging-associated pathophysiology.

Validity of ten analytical heat stress indices in predicting the physiological parameters of people under various occupational and meteorological conditions

Until now, only a few comprehensive studies have validated analytical heat stress indices in different conditions. The present study aims to investigate the validity of these indicators in predicting the physiological parameters of workers. This cross-sectional study was conducted with 194 male employees working in warm environments. First, demographic information was collected. After participants rested for 30 min, their heart rate and tympanic temperature were measured. The subjects then performed their routine tasks. At the end of 90 min, their heart rate and tympanic temperature were again measured. Additionally, their metabolism rate and clothing thermal insulation were estimated. Environmental parameters were also measured at 30-, 60-, and 90-min time points. Additional information required to compute the indices was recorded. Then, the values of each of the indices were computed. Finally, the validity of the indices was assessed under different conditions. The results indicated that the highest regression coefficients with tympanic temperature were assigned to modified physiologically equivalent temperature (mPET) (0.7515), predicted heat strain (PHS) (0.7201), and predicted mean vote (PMV) (0.7082), index, respectively. Also, the greatest regression coefficients with heart rate belonged to mPET (0.7773), PMV (0.7624), and PHS (0.6479) index, respectively. Based on the results, the highest diagnostic accuracies of receiver operating characteristic (ROC) curves for tympanic temperature were related to indices of mPET, PHS, and PMV with the area under the ROC curve (AUC) of 0.945, 0.931, and 0.930, respectively. Of the studied indices, it was observed that mPET, PHS, PMV, and PPD showed more validity compared to others.

Accuracy of the Exeter Hospitalizations-Office Visits-Medical Conditions-Extra Care-Social Concerns Index for Identifying Children With Complex Chronic Medical Conditions in the Clinical Setting

OBJECTIVE

Our objective was to determine the accuracy of a point-of-care instrument, the Hospitalizations-Office Visits-Medical Conditions-Extra Care-Social Concerns (HOMES) instrument, in identifying patients with complex chronic conditions (CCCs) compared to an algorithm used to identify patients with CCCs within large administrative data sets.

METHODS

We compared the HOMES to Feudtner's CCCs classification system. Using administrative algorithms, we categorized primary care patients at a children's hospital into 3 categories: no chronic conditions, non-complex chronic conditions, and CCCs. We randomly selected 100 patients from each category. HOMES scoring was completed for each patient. We performed an optimal cut-point analysis on 80% of the sample to determine which total HOMES score best identified children with ≥1 CCC and ≥2 CCCs. Using the optimal cut points and the remaining 20% of the study population, we determined the odds and area under the curve (AUC) of having ≥1 CCC and ≥2 CCCs.

RESULTS

The median (interquartile range [IQR]) age was 4 (IQR: 0, 8). Using optimal cut points of ≥7 for ≥1 CCC and ≥11 for ≥2 CCCs, the odds of having ≥1 CCC was 19 times higher than lower scores (odds ratio [OR] 19.1 [95% confidence interval [CI]: 9.75, 37.5]) and of having ≥2 CCCs was 32 times higher (OR 32.3 [95% CI: 12.9, 50.6]). The AUCs were 0.76 for ≥1 CCC (sensitivity 0.82, specificity 0.80) and 0.74 for ≥2 CCCs (sensitivity 0.92, specificity 0.74).

CONCLUSIONS

The HOMES accurately identified patients with CCCs.

A Survey on the Role of Artificial Intelligence in Biobanking Studies: A Systematic Review

Introduction: In biobanks, participants' biological samples are stored for future research. The application of artificial intelligence (AI) involves the analysis of data and the prediction of any pathological outcomes. In AI, models are used to diagnose diseases as well as classify and predict disease risks. Our research analyzed AI's role in the development of biobanks in the healthcare industry, systematically. Methods: The literature search was conducted using three digital reference databases, namely PubMed, CINAHL, and WoS. Guidelines for preferred reporting elements for systematic reviews and meta-analyses (PRISMA)-2020 in conducting the systematic review were followed. The search terms included "biobanks", "AI", "machine learning", and "deep learning", as well as combinations such as "biobanks with AI", "deep learning in the biobanking field", and "recent advances in biobanking". Only English-language papers were included in the study, and to assess the quality of selected works, the Newcastle-Ottawa scale (NOS) was used. The good quality range (NOS ≥ 7) is only considered for further review. Results: A literature analysis of the above entries resulted in 239 studies. Based on their relevance to the study's goal, research characteristics, and NOS criteria, we included 18 articles for reviewing. In the last decade, biobanks and artificial intelligence have had a relatively large impact on the medical system. Interestingly, UK biobanks account for the highest percentage of high-quality works, followed by Qatar, South Korea, Singapore, Japan, and Denmark. Conclusions: Translational bioinformatics probably represent a future leader in precision medicine. AI and machine learning applications to biobanking research may contribute to the development of biobanks for the utility of health services and citizens.

Predictive risk-score model for selection of patients with high-risk stage II colon cancer for adjuvant systemic therapy

BACKGROUND

Adjuvant systemic therapy is selectively considered for high-risk stage II colon cancer, but which patients benefit most from adjuvant systemic therapy is unclear.

METHODS

Patients who underwent resection of stage II colon cancer were identified from the National Cancer Database (2010-2016). Risk-factors for decreased overall survival on multivariable analysis were used to establish a predictive risk-score model for all-cause mortality. After propensity matching within each risk group, 5-year overall survival was estimated based on receipt of adjuvant systemic therapy.

RESULTS

Of the 15,241 patients evaluated, 2,857 (18.8%) received adjuvant systemic therapy. Risk factors for decreased overall survival included age >75 (hazard ratio 3.3, P < .001), male sex (hazard ratio 1.2, P < .001), White/Black race (hazard ratio 1.4, P = .020), preoperative carcinoembryonic antigen >3.5 ng/mL (hazard ratio 1.6, P < .001), T4a T-stage (hazard ratio 2.0, P < .001), T4b T-stage (hazard ratio 2.4, P < .001), lymphovascular invasion (hazard ratio 1.2, P = .003), perineural invasion (hazard ratio 1.3, P = .003), and non-R0 proximal/distal resection margins (hazard ratio 1.7, P < .001). An internally validated risk-score model using these factors was developed composed of low-risk (n = 8,489), moderate-risk (n = 4,623), and high-risk (n = 2,129) groups; within each group, 19.9%, 15.7%, and 20.8% of patients, respectively, received adjuvant systemic therapy. After propensity matching, adjuvant systemic therapy was not associated with improved 5-year overall survival for low-risk patients (89.8% vs 88.3%, P = .280), but was for moderate-risk (80.5% vs 70.8%, P < .001), and high-risk (65.2% vs 45.7%, P < .001) patients.

CONCLUSION

A predictive risk-score model incorporating patient and tumor factors identifies a high-risk cohort of stage II colon cancer patients who may benefit from adjuvant systemic therapy, although the minority of these patients appear to be receiving treatment.

The Neutrophil to Lymphocyte Ratio as a Triage Tool in Criticality Accidents

ABSTRACT

During triage of possibly irradiated individuals after a criticality accident or nuclear weapon event, it is necessary to decide whether a patient has experienced a clinically significant dose (> 2 Gy) that would require referral for additional evaluation and medical treatment. This is a binary decision: yes or no. The neutrophil-to-lymphocyte ratio (NLR) is an appropriate decision parameter, is simple to obtain in field operations, and is recognized in clinical medicine as an independent marker of systemic inflammation. NLR is evaluated for usefulness in triage using data from the Radiation Accident Registry at the Radiation Emergency Assistance Center/Training Site (REAC/TS). A criticality accident data set has been prepared using historic complete blood counts from 12 criticality events with 33 patients. In addition, a cohort of 125 normal controls has been assembled for comparison with the radiation accident data. In the control set, NLR is found to be 2.1 ± 0.06 (mean ± SEM) and distributed consistent with a Gaussian distribution. A patient from the 1958 Y-12 criticality accident is presented as an example of the time dependence of NLR after an event. In this case, NLR is statistically elevated above controls from <4 h until ~20 d post-event, and for times >20 d post-event, NLR is less than the control value, returning to baseline > ~40 d. The latter result has been confirmed using late hematological data taken from patients at Hiroshima and Nagasaki, and this appears to be a general finding. Since triage is a binary decision, analyzing NLR with receiver operating characteristic (ROC) statistics is appropriate. Maximizing the Youden J statistic (sensitivity + specificity -1) determines an appropriate decision point. For this data set, the decision point for NLR is found to be 3.33, with area under the curve (AUC) 0.865, sensitivity 0.67, specificity 0.97, positive predictive value (PPV) 0.85, and negative predictive value (NPV) 0.92. Therefore, when a known criticality accident or nuclear weapon event has occurred and if the patient's NLR is greater than 3.33 early post-event, then that person should be referred for further health physics and medical evaluation.

Association of Red Blood Cell Indices with Mild Cognitive Impairment in Chinese Elderly Individuals: A Matched Case-control Study

BACKGROUND

Mild cognitive impairment (MCI) represents an intermediate and modifiable stage between normal aging and dementia. There is an urgent need for simple, non-invasive testing of MCI by blood biomarkers.

OBJECTIVE

This study aimed to retrospectively evaluate the association of red blood cell (RBC) indices with MCI, and select the best hematologic characteristic for detection of MCI in elderly Chinese.

METHODS

Matched case-control study was carried out with 85 pairs of MCI subjects and healthy controls. The matching criteria was age, gender and education attainment. All samples were analyzed for RBC indices, including hemoglobin (HGB), hematocrit (HCT), mean corpuscular volume (MCV), mean corpuscular hemoglobin concentration (MCHC) and red cell distribution width-standard deviation (RDW-SD). A conditional logistic regression model was used to evaluate the association between RBC indices and MCI. The diagnostic efficacy of the biomarkers was evaluated by receiver operating characteristics (ROC).

RESULTS

Among all RBC indices, there were significant differences in HGB (124.82 ± 7.89 vs. 133.93 ± 4.52, P < 0.001) and RDW-SD (45.29 ± 2.03 vs. 41.34 ± 4.41, P < 0.001) between two groups. In the logistic regression model, after adjustment for lifestyle factors and comorbidities, significant statistically associations have been found between higher HGB and lower risk of MCI (adjusted OR: 0.831; 95% CI: 0.773-0.893), higher RDW-SD and a higher risk of MCI (adjusted OR: 1.575; 95% CI: 1.326- 1.872). ROC analysis suggested that the largest area under the ROC curve (AUC) was found with the combination of HGB and RDW-SD (AUC = 0.842), followed by HGB(AUC = 0.795), and finally by modest RDW-SD (AUC = 0.777). Combination of HGB <131 g/L and RDW-SD >43.4 fL yielded a sensitivity of 92% and a specificity of 89%, overall diagnosis efficiency of which were better than HBG and RDW-SD alone.

CONCLUSION

Lower HGB and higher RDW-SD alone were significantly found to be associated with increased risk of MCI, and offered modest sensitivity and specificity as a diagnostic marker. The combination of HGB and RDW-SD was more sensitive and had higher classification accuracy for differentiating MCI from healthy controls. Further prospective research is needed to clarify whether HGB in combination with RDW-SD may be a potential diagnostic tool for early diagnosis of AD.

Defining Abdominal Obesity as a Risk Factor for Coronary Heart Disease in the U.S.: Results From the Hispanic Community Health Study/Study of Latinos (HCHS/SOL)

OBJECTIVE

Various organizations have highlighted the need to examine whether abdominal obesity cut points are appropriate for identification of cardiovascular risk among ethnic minority adults, particularly Hispanic/Latino adults living in Western societies. This study aimed 1) to establish optimal definitions for abdominal obesity among Hispanics/Latinos and 2) to determine the level of agreement between the presence of metabolic syndrome diagnosed by the current Joint Interim Statement (JIS) definition and an updated definition with optimal abdominal obesity cut points.

RESEARCH DESIGN AND METHODS

The sample included 16,289 adults who self-identified as Hispanic/Latino ages 18-74 years enrolled in the Hispanic Community Health Study/Study of Latinos (HCHS/SOL). Receiver operating characteristic curves were used to derive sensitivity and specificity values. The largest sum of sensitivity plus specificity was used to determine appropriate cut points.

RESULTS

Among U.S. Hispanic/Latino adults, waist circumference cut points of >102 cm in men (in line with current JIS criteria) and >97 cm (9 points higher than JIS criteria) in women provide optimal discrimination for cardiovascular risk as judged by the presence of coronary heart disease. When using these cut points to create an updated metabolic syndrome definition among women, we found disagreement between our updated definition and the current JIS criteria. The prevalence of the metabolic syndrome was overestimated by ∼5 percentage points among women based on JIS criteria in comparison with our definition.

CONCLUSIONS

Our results suggest that the current recommendations for waist circumference cut points may not be appropriate for U.S. Hispanic/Latino women.

Predictors of intracranial hypertension in children undergoing ICP monitoring after severe traumatic brain injury

PURPOSE

Intracranial hypertension (ICH) is a common and treatable complication after severe traumatic brain injury (sTBI) in children. Describing the incidence and risk factors for developing ICH after sTBI could impact clinical practice.

METHODS

Retrospective cohort study from 2006 to 2015 at two university-affiliated level I pediatric trauma centers of children admitted with accidental or abusive TBI, a post-resuscitation Glasgow Coma Score (GCS) of 8 or less, and an invasive intracranial pressure (ICP) monitor. Bivariate and multivariable logistic regression analysis were performed to identify demographic, injury, and imaging characteristics in patients who received ICP directed therapies for ICH (ICP > 20 mmHg).

RESULTS

Eight to 5% (271/321) of monitored patients received ICP directed therapy for ICH during their PICU stay. Ninety-seven percent of patients had an abnormality on CT scan by either the Marshall or the Rotterdam score. Of the analyzed clinical and radiologic variables, only presence of hypoxia prior to PICU arrival, female sex, and a higher Injury Severity Score (ISS) were associated with increased risk of ICH (p < 0.05).

CONCLUSIONS

In this retrospective study of clinical practice of ICP monitoring in children after sTBI, the vast majority of children had an abnormal CT scan and experienced ICH requiring clinical intervention. Commonly measured clinical variables and radiologic classification scores did not significantly add to the prediction for developing of ICH and further efforts are needed to define low-risk populations that would not develop ICH.

Application of automatic statistical post-processing method for analysis of ultrasonic and digital dermatoscopy images

Ultrasonic and digital dermatoscopy diagnostic methods are used in order to estimate the changes of structure, as well as to non-invasively measure the changes of parameters of lesions of human tissue. These days, it is very actual to perform the quantitative analysis of medical data, which allows to achieve the reliable early-stage diagnosis of lesions and help to save more lives. The proposed automatic statistical post-processing method based on integration of ultrasonic and digital dermatoscopy measurements is intended to estimate the parameters of malignant tumours, measure spatial dimensions (e.g. thickness) and shape, and perform faster diagnostics by increasing the accuracy of tumours differentiation. It leads to optimization of time-consuming analysis procedures of medical images and could be used as a reliable decision support tool in the field of dermatology.

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.

Measuring adherence to oral antidiabetic multi-drug treatment: Comparative validity of prescription claims-based adherence measures against hospitalization

BACKGROUND

The use of prescription claims data to measure adherence to diabetes treatment is very common in research. However, there is no clear evidence regarding the accuracy of the methods used to assess adherence to multi-drug treatments for the many patients using multiple antidiabetic drugs.

PURPOSE

To assess the validity of prescription claims-based adherence measures in the context of multiple oral antidiabetic drug treatment.

METHODS

A cohort of patients who began their antidiabetic drug treatment with at least two oral antidiabetic drugs (new users) was created using Quebec medico-administrative data. Four different prescription claims-based adherence measures were assessed: proportion of days covered (PDC) by at least one class of drugs, mean PDC, PDC by all classes and daily polypharmacy possession ratio (DPPR). All-cause and diabetes-related hospitalizations were the validation criteria. To assess the validity of the measures, receiver operating characteristic (ROC) curves were plotted for each measure and each criterion.

RESULTS

A total of 5982 individuals were included. The areas under the ROC curves for the PDC by at least one class of drugs, the mean PDC, the PDC by all classes of drugs and the DPPR were respectively 0.54 (95% CI: 0.52-0.56), 0.51 (0.49-0.53), 0.50 (0.48-0.52) and 0.51 (0.49-0.53) with all-cause hospitalization as criterion and 0.55 (0.53-0.57), 0.53 (0.51-0.55), 0.51 (0.49-0.53) and 0.53 (0.51-0.55) using diabetes-related hospitalization as criterion.

CONCLUSIONS

The results suggest that all measures have poor validity in predicting hospitalizations thus raising concerns about their utility in the assessment of adherence to multi-drug treatment. Future research should assess the capacity of these measures to predict other outcomes more closely related to medication adherence.

Analysis of which local set-up errors can be covered by a 5-mm margin for cone beam CT-guided radiotherapy for nasopharyngeal carcinoma

Objective:

To analyse which local set-up errors can be covered by a 5-mm margin for cone beam computed tomography (CBCT)-guided radiotherapy in nasopharyngeal carcinoma (NPC).

Methods:

11 regions of interest (ROIs) were registered for 24 NPC patients, with a total of 323 CBCT scans. According to the registration results, clinical target volume–planning target volume (CTV–PTV)/organs at risk-planning risk volume (OAR-PRV) margin analysis; Pearson correlation analysis; Bland–Altman plots; and a receiver operating characteristic (ROC) analysis were used to investigate which local set-up errors of substructure can be represented by the PTV_ROI.

Results:

The clinical target volume-PTV/OAR-planning risk volume margins were less than 5 mm for C1_ROI-C4_ROI, mandible (M_ROI), and sphenoid sinus (S_ROI) with respect to PTV_ROI. C1_ROI-C4_ROI, M_ROI, and S_ROI exhibited significant correlations and consistencies in the mediolateral, superior–inferior, and anteroposterior (AP) directions and significant receiver operating characteristic analysis results in the anteroposterior direction.

Conclusion:

Only the upper local set-up error of C1_ROI-C4_ROI, M_ROI, and S_ROI can be covered by a 5-mm margin for CBCT-guided NPC radiotherapy with a large ROI. Using these ROIs as an integral reference ROI is better than individual bony landmark.

Advances in knowledge:

This report is helpful to CBCT registration for NPC radiotherapy in clinical practice.

α-Synuclein Heterocomplexes with β-Amyloid Are Increased in Red Blood Cells of Parkinson's Disease Patients and Correlate with Disease Severity

Neurodegenerative disorders (NDs) are characterized by abnormal accumulation/misfolding of specific proteins, primarily α-synuclein (α-syn), β-amyloid1-42 (Aβ1-42) and tau, in both brain and peripheral tissues. In addition to oligomers, the role of the interactions of α-syn with Aβ or tau has gradually emerged. Nevertheless, despite intensive research, NDs have no accepted peripheral markers for biochemical diagnosis. In this respect, Red Blood Cells (RBCs) are emerging as a valid peripheral model for the study of aging-related pathologies. Herein, a small cohort (N = 28) of patients affected by Parkinson's disease (PD) and age-matched controls were enrolled to detect the content of α-syn (total and oligomeric), Aβ1-42 and tau (total and phosphorylated) in RBCs. Moreover, the presence of α-syn association with tau and Aβ1-42 was explored by co-immunoprecipitation/western blotting in the same cells, and quantitatively confirmed by immunoenzymatic assays. For the first time, PD patients were demonstrated to exhibit α-syn heterocomplexes with Aβ1-42 and tau in peripheral tissues; interestingly, α-syn-Aβ1-42 concentrations were increased in PD subjects with respect to healthy controls (HC), and directly correlated with disease severity and motor deficits. Moreover, total-α-syn levels were decreased in PD subjects and inversely related to their motor deficits. Finally, an increase of oligomeric-α-syn and phosphorylated-tau was observed in RBCs of the enrolled patients. The combination of three parameters (total-α-syn, phosphorylated-tau and α-syn-Aβ1-42 concentrations) provided the best fitting predictive index for discriminating PD patients from controls. Nevertheless further investigations should be required, overall, these data suggest α-syn hetero-aggregates in RBCs as a putative tool for the diagnosis of PD.

Does tumor size improve the accuracy of prognostic prediction in patients with esophageal squamous cell carcinoma after surgical resection?

This study aimed to investigate whether the inclusion of tumor size could improve the prognostic accuracy in patients with esophageal squamous cell cancer (ESCC). A total of 387 patients with ESCC who underwent curative resection were enrolled in this analysis. The patients were categorized into small-sized tumors (SSTs) and large-sized tumors (LSTs) using an appropriate cut-off point for tumor size. Kaplan–Meier survival curve and log–rank test were used to evaluate the prognostic value of tumor size. A Cox regression model was adopted for multivariate analysis. Their accuracy was compared based on the presence or absence of tumor size. Using 3.5 cm as the optimal cut-off point, 228 and 159 patients presented with LSTs (≥ 3.5 cm) and SSTs (< 3.5 cm), respectively. The patients with LSTs had significantly worse prognoses than patients with SSTs (23.9% vs. 43.2%, P < 0.001). Multivariate analysis revealed that tumor size, histological type, invasion depth, and lymph node metastasis were independent predictors of overall survival. The addition of tumor size to the AJCC TNM staging improved the predictive accuracy of the 5-year survival rate by 3.9%. Further study showed that tumor size and T stage were independent predictors of the prognosis of node-negative patients, and the combination of tumor size and T stage improved the predictive accuracy by 3.7%. In conclusion, tumor size is indeed a simple and practical prognostic factor in patients with ESCC. It can be used to improve the prognostic accuracy of the current TNM staging, especially for patients with node-negative disease.

Serum Growth Differentiation Factor 15 in Parkinson Disease

BACKGROUND

Growth differentiation factor 15 (GDF15) has been shown to be protective for dopaminergic neurons in animal and ex vivo experiments. However, little is known about its effect on the human body.

OBJECTIVE

This study investigated associations between serum GDF15 levels and clinical parameters in patients with Parkinson disease (PD).

METHODS

Idiopathic PD patients (n = 104) and age-matched controls (n = 88) were enrolled. Serum GDF15 levels were measured by human enzyme-linked immunosorbent assay. Univariate and multivariate analyses investigated correlations between GDF15 and clinical characteristics, including disease severity by the Unified PD Rating Scale (UPDRS)-III. The diagnostic value of GDF15 was evaluated by receiver-operating characteristic curve (ROC) analysis.

RESULTS

The serum GDF15 levels of the PD patients were significantly higher than those of the healthy controls. In PD patients, serum GDF15 levels in men were significantly higher than in women. GDF15 levels correlated with age, gender, disease duration, and UPDRS-III score. After adjusting for confounding factors, multiple linear regression analysis showed that the serum GDF15 level (β = 0.015, p = 0.001) was an independent risk factor for UPDRS-III score. In ROC analysis, GDF15 achieved an area under the curve of 0.86 for the identification of PD, with a sensitivity of 71.15% and a specificity of 87.50%.

CONCLUSION

GDF15 may be a potential biomarker for the diagnosis and monitoring of motor severity in PD.

DTI measures identify mild and moderate TBI cases among patients with complex health problems: A receiver operating characteristic analysis of U.S. veterans

Standard MRI methods are often inadequate for identifying mild traumatic brain injury (TBI). Advances in diffusion tensor imaging now provide potential biomarkers of TBI among white matter fascicles (tracts). However, it is still unclear which tracts are most pertinent to TBI diagnosis. This study ranked fiber tracts on their ability to discriminate patients with and without TBI. We acquired diffusion tensor imaging data from military veterans admitted to a polytrauma clinic (Overall n = 109; Age: M = 47.2, SD = 11.3; Male: 88%; TBI: 67%). TBI diagnosis was based on self-report and neurological examination. Fiber tractography analysis produced 20 fiber tracts per patient. Each tract yielded four clinically relevant measures (fractional anisotropy, mean diffusivity, radial diffusivity, and axial diffusivity). We applied receiver operating characteristic (ROC) analyses to identify the most diagnostic tract for each measure. The analyses produced an optimal cutpoint for each tract. We then used kappa coefficients to rate the agreement of each cutpoint with the neurologist's diagnosis. The tract with the highest kappa was most diagnostic. As a check on the ROC results, we performed a stepwise logistic regression on each measure using all 20 tracts as predictors. We also bootstrapped the ROC analyses to compute the 95% confidence intervals for sensitivity, specificity, and the highest kappa coefficients. The ROC analyses identified two fiber tracts as most diagnostic of TBI: the left cingulum (LCG) and the left inferior fronto-occipital fasciculus (LIF). Like ROC, logistic regression identified LCG as most predictive for the FA measure but identified the right anterior thalamic tract (RAT) for the MD, RD, and AD measures. These findings are potentially relevant to the development of TBI biomarkers. Our methods also demonstrate how ROC analysis may be used to identify clinically relevant variables in the TBI population.

Plasma D-dimer Can Effectively Predict the Prospective Occurrence of Ascites in Advanced Schistosomiasis Japonica Patients

China still has more than 30,000 patients of advanced schistosomiasis while new cases being reported consistently. D-dimer is a fibrin degradation product. As ascites being the dominating symptom in advanced schistosomiasis, the present study aimed to explore a prediction model of ascites with D-dimer and other clinical easy-achievable indicators. A case-control study nested in a prospective cohort was conducted in schistosomiasis-endemic area of southern China. A total of 291 patients of advanced schistosomiasis were first investigated in 2013 and further followed in 2014. Information on clinical history, physical examination, and abdominal ultrasonography, including the symptom of ascites was repeatedly collected. Result showed 44 patients having ascites. Most of the patients’ ascites were confined in the kidney area with median area of 20 mm². The level of plasma D-dimer and pertinent liver function indicators were measured at the initial investigation in 2013. Compared with those without ascites, cases with ascites had significantly higher levels of D-dimer (0.71±2.44 μg/L vs 0.48±2.12 μg/L, P=0.005), as well ALB (44.5 vs 46.2, g/L) and Type IV collagen (50.04 vs 44.50 μg/L). Receiver operating characteristic curve analyses indicated a moderate predictive value of D-dimer by its own area under curve (AUC) of 0.64 (95% CI: 0.54–0.73) and the cutoff value as 0.81 μg/L. Dichotomized by the cutoff level, D-dimer along with other categorical variables generated a prediction model with AUC of 0.76 (95% CI: 0.68–0.89). Risks of patients with specific characteristics in the prediction model were summarized. Our study suggests that the plasma D-dimer level is a reliable predictor for incident ascites in advanced schistosomiasis japonica patients.

Diagnostic value of serum procalcitonin, lactate, and high-sensitivity C-reactive protein for predicting bacteremia in adult patients in the emergency department

BACKGROUND

Few studies compared the diagnostic value of procalcitonin with a combination of other tests including lactate and high-sensitivity C-reactive protein in the prediction of pathogenic bacteremia in emergency department adult patients.

METHODS

We performed a retrospective study assessing the differences in performances of procalcitonin at a cutoff of 0.5 ng/mL, lactate at a cutoff of 19.8 mg/dL, high-sensitivity C-reactive protein at a cutoff of 0.8 mg/dL and their combinations for predicting bacteremia in emergency department adult patients. Sensitivity, specificity, overall accuracy, positive-test and negative-test likelihood, and diagnostic odds ratio with 95% confidence interval for each test combination were calculated for comparison. The receiver operating characteristic curve for every single test were compared using DeLong's method. We also performed a sensitivity analysis in two expanded patient cohorts to assess the discriminative ability of procalcitonin or test combination.

RESULTS

A total of 886 patients formed the initial patient cohort. The area under the receiver operating characteristic curve for discriminating positive blood culture was: procalcitonin = 0.72 (95% CI [0.69-0.75]) with a derived optimal cutoff at 3.9 ng/mL; lactate 0.69 (0.66-0.72) with an optimal cutoff at 17.9 mg/dL; high-sensitivity C-reactive protein 0.56 (0.53-0.59) with an optimal cutoff of 13 mg/dL; with pairwise comparisons showing statistically significant better performance of either procalcitonin or lactate outperforming high-sensitivity C-reactive protein. To predict positive blood cultures, the diagnostic odds ratio for procalcitonin was 3.64 (95% CI [2.46-5.51]), lactate 2.93 (2.09-4.14), and high-sensitivity C-reactive protein 0.91 (0.55-1.55; P = 0.79). About combined tests, the diagnostic odds ratio for procalcitonin and lactate increases were 3.98 (95% CI [2.81-5.63]) for positive blood culture prediction. Elevated procalcitonin level rendered a six-fold increased risk of positive gram-negative bacteremia with a diagnostic odds ratio of 6.44 (95% CI [3.65-12.15]), which showed no further improvement in any test combinations. In the sensitivity analysis, as a single test to predict unspecified, gram-negative and gram-positive bacteremia, procalcitonin performed even better in an expanded cohort of 2,234 adult patients in terms of the diagnostic odds ratio.

DISCUSSIONS

For adult emergency patients, procalcitonin has an acceptable discriminative ability for bacterial blood culture and a better discriminative ability for gram-negative bacteremia when compared with lactate and high-sensitivity C-reactive protein. High-sensitivity C-reactive protein at a cutoff of 0.8 mg/dL performed poorly for the prediction of positive bacterial culture.

Plasma complement biomarkers distinguish multiple sclerosis and neuromyelitis optica spectrum disorder

Background:

Multiple sclerosis (MS) and neuromyelitis optica spectrum disorder (NMOSD) are autoimmune inflammatory demyelinating diseases of the central nervous system. Although distinguished by clinicoradiological and demographic features, early manifestations can be similar complicating management. Antibodies against aquaporin-4 support the diagnosis of NMOSD but are negative in some patients. Therefore, there is unmet need for biomarkers that enable early diagnosis and disease-specific intervention.

Objective:

We investigated whether plasma complement proteins are altered in MS and NMOSD and provide biomarkers that distinguish these diseases.

Methods:

Plasma from 54 NMOSD, 40 MS and 69 control donors was tested in multiplex assays measuring complement activation products and proteins. Using logistic regression, we tested whether combinations of complement analytes distinguished NMOSD from controls and MS.

Results:

All activation products were elevated in NMOSD compared to either control or MS. Four complement proteins (C1inh, C1s, C5 and FH) were higher in NMOSD compared to MS or controls. A model comprising C1inh and terminal complement complex (TCC) distinguished NMOSD from MS (area under the curve (AUC): 0.98), while C1inh and C5 distinguished NMOSD from controls (AUC: 0.94).

Conclusion:

NMOSD is distinguished from MS by plasma complement biomarkers. Selected complement analytes enable differential diagnosis. Findings support trials of anti-complement therapies in NMOSD.

Predictive Validity of Self-Reported Measures of Adherence to Noninsulin Antidiabetes Medication against Control of Glycated Hemoglobin Levels

OBJECTIVE

To assess and compare the predictive validity of 4 self-reported adherence measures.

METHODS

A convenience sample of 153 patients with type 2 diabetes completed a self-report with 4 items (SR-4) and a French version of the Morisky Medication Adherence Scale with 8 items (MMAS-8), reported the proportion of pills missed, and answered a single-item scale regarding their antidiabetes drug treatments. They also provided measures of glycated hemoglobin (A1C) taken between 3 and 6 months after the adherence measurements. We examined the relationship between self-reported adherence and glycemic control using the area under the receiver operating characteristics curve (AUC) and linear regression analyses.

RESULTS

AUCs were 0.51, 0.52, 0.53 and 0.52 for the SR-4, MMAS-8, self-reported proportion of pills missed and single-item scale, respectively. AUCs stratified according to median duration of diabetes ranged from 0.55 to 0.63. Based on linear regression analyses adjusted for diabetes duration, the association measured in the total sample between adherence measures and A1C levels was not statistically significant. When regression analyses were performed among participants with A1C levels ≥7% only, SR-4, MMAS-8 and the single-item scale scores were significantly associated with A1C levels, and beta coefficients were associated with a 1-unit increase in adherence scores of -0.46, -0.20 and 0.38, respectively.

CONCLUSION

The results support the predictive validity of all measures except the self-reported proportion of missed pills.

Translational biomarkers of acetaminophen-induced acute liver injury

Acetaminophen (APAP) is a commonly used analgesic drug that can cause liver injury, liver necrosis and liver failure. APAP-induced liver injury is associated with glutathione depletion, the formation of APAP protein adducts, the generation of reactive oxygen and nitrogen species and mitochondrial injury. The systems biology omics technologies (transcriptomics, proteomics and metabolomics) have been used to discover potential translational biomarkers of liver injury. The following review provides a summary of the systems biology discovery process, analytical validation of biomarkers and translation of omics biomarkers from the nonclinical to clinical setting in APAP-induced liver injury.

Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach

OBJECTIVES

The objectives are to describe the disadvantages of the area under the receiver operating characteristic curve (ROC AUC) to measure diagnostic test performance and to propose an alternative based on net benefit.

METHODS

We use a narrative review supplemented by data from a study of computer-assisted detection for CT colonography.

RESULTS

We identified problems with ROC AUC. Confidence scoring by readers was highly non-normal, and score distribution was bimodal. Consequently, ROC curves were highly extrapolated with AUC mostly dependent on areas without patient data. AUC depended on the method used for curve fitting. ROC AUC does not account for prevalence or different misclassification costs arising from false-negative and false-positive diagnoses. Change in ROC AUC has little direct clinical meaning for clinicians. An alternative analysis based on net benefit is proposed, based on the change in sensitivity and specificity at clinically relevant thresholds. Net benefit incorporates estimates of prevalence and misclassification costs, and it is clinically interpretable since it reflects changes in correct and incorrect diagnoses when a new diagnostic test is introduced.

CONCLUSIONS

ROC AUC is most useful in the early stages of test assessment whereas methods based on net benefit are more useful to assess radiological tests where the clinical context is known. Net benefit is more useful for assessing clinical impact.

KEY POINTS

• The area under the receiver operating characteristic curve (ROC AUC) measures diagnostic accuracy. • Confidence scores used to build ROC curves may be difficult to assign. • False-positive and false-negative diagnoses have different misclassification costs. • Excessive ROC curve extrapolation is undesirable. • Net benefit methods may provide more meaningful and clinically interpretable results than ROC AUC.

A longitudinal study on α-synuclein in blood plasma as a biomarker for Parkinson's disease

There have been no longitudinal studies on α-synuclein as a potential biomarker for the progression of Parkinson's disease (PD). Here, blood plasma ‘total α-synuclein’ and ‘Ser-129 phosphorylated α-synuclein’ were assayed at 4–6 monthly intervals from a cohort of 189 newly-diagnosed patients with PD. For log-transformed data, plasma total α-synuclein levels increased with time for up to 20 yrs after the appearance of initial symptoms (p = 0.012), whereas phosphorylated α-synuclein remained constant over this same period. The mean level of phosphorylated α-synuclein, but not of total α-synuclein, was higher in the PD plasma samples taken at first visit than in single samples taken from a group of 91 healthy controls (p = 0.012). Overall, we conclude that the plasma level of phosphorylated α-synuclein has potential value as a diagnostic tool, whereas the level of total α-synuclein could act as a surrogate marker for the progression of PD.

Using the probability-probability plot and index to augment interpretation of treatment effect for patient-reported outcome measures

The interpretation of treatment effect can pose challenges, especially for patient-reported outcomes. As subjective assessments, patient-reported outcomes frequently lack a historical record to support what their scores mean, making their interpretation of treatment differences challenging. We show how the probability-probability (p-p) plot a graph of the test-treatment distribution percentiles versus the control-treatment distribution percentiles, can complement and supplement interpretation of treatment effect. From this plot, we introduce the p-p index as a new measure of treatment effect, illustrating the method with two examples. The p-p index represents, across all percentiles, the average difference in percentile rank for any pair of subjects on two different treatments with the same outcome score. This measure, which complements other metrics of treatment effect, captures full information by integrating across all percentiles and thus accurately summarizes and augments the interpretation of treatment effect.

Heterogeneity and breadth of host antibody response to KSHV infection demonstrated by systematic analysis of the KSHV proteome

The Kaposi sarcoma associated herpesvirus (KSHV) genome encodes more than 85 open reading frames (ORFs). Serological evaluation of KSHV infection now generally relies on reactivity to just one latent and/or one lytic protein (commonly ORF73 and K8.1). Most of the other polypeptides encoded by the virus have unknown antigenic profiles. We have systematically expressed and purified products from 72 KSHV ORFs in recombinant systems and analyzed seroreactivity in US patients with KSHV-associated malignancies, and US blood donors (low KSHV seroprevalence population). We identified several KSHV proteins (ORF38, ORF61, ORF59 and K5) that elicited significant responses in individuals with KSHV-associated diseases. In these patients, patterns of reactivity were heterogeneous; however, HIV infection appeared to be associated with breadth and intensity of serological responses. Improved antigenic characterization of additional ORFs may increase the sensitivity of serologic assays, lead to more rapid progresses in understanding immune responses to KSHV, and allow for better comprehension of the natural history of KSHV infection. To this end, we have developed a bead-based multiplex assay detecting antibodies to six KSHV antigens.

Using the AUDIT-PC to predict alcohol withdrawal in hospitalized patients

BACKGROUND

Alcohol withdrawal syndrome (AWS) occurs when alcohol-dependent individuals abruptly reduce or stop drinking. Hospitalized alcohol-dependent patients are at risk. Hospitals need a validated screening tool to assess withdrawal risk, but no validated tools are currently available.

OBJECTIVE

To examine the admission Alcohol Use Disorders Identification Test-(Piccinelli) Consumption (AUDIT-PC) ability to predict the subsequent development of AWS among hospitalized medical-surgical patients admitted to a non-intensive care setting.

DESIGN

Retrospective case–control study of patients discharged from the hospital with a diagnosis of AWS. All patients with AWS were classified as presenting with AWS or developing AWS later during admission. Patients admitted to an intensive care setting and those missing AUDIT-PC scores were excluded from analysis. A hierarchical (by hospital unit) logistic regression was performed and receiver-operating characteristics were examined on those developing AWS after admission and randomly selected controls. Because those diagnosing AWS were not blinded to the AUDIT-PC scores, a sensitivity analysis was performed.

PARTICIPANTS

The study cohort included all patients age ≥18 years admitted to any medical or surgical units in a single health care system from 6 October 2009 to 7 October 2010.

KEY RESULTS

After exclusions, 414 patients were identified with AWS. The 223 (53.9 %) who developed AWS after admission were compared to 466 randomly selected controls without AWS. An AUDIT-PC score ≥4 at admission provides 91.0 % sensitivity and 89.7 % specificity (AUC=0.95; 95 % CI, 0.94–0.97) for AWS, and maximizes the correct classification while resulting in 17 false positives for every true positive identified. Performance remained excellent on sensitivity analysis (AUC=0.92; 95 % CI, 0.90–0.93). Increasing AUDIT-PC scores were associated with an increased risk of AWS (OR=1.68, 95 % CI 1.55–1.82, p<0.001).

CONCLUSIONS

The admission AUDIT-PC score is an excellent discriminator of AWS and could be an important component of future clinical prediction rules. Calibration and further validation on a large prospectivecohort is indicated.

Optimal thresholds by maximizing or minimizing various metrics via ROC-type analysis

RATIONALE AND OBJECTIVES

Based on imaging features, the optimal thresholds are typically determined as cutoff points to dichotomize the corresponding measurement scales.

MATERIALS AND METHODS

Five metrics (ie, the Youden index, Euclidian distance, percent of correct diagnosis, kappa statistic, and mutual information) are individually maximized or minimized to derive the corresponding optimal threshold. These optimal thresholds are estimated under the parametric binormal assumption. Monte Carlo simulation studies are conducted to compare the performances of these different methods. A published radiological example on the choice of treatment outcomes following ureteral stones is used to illustrate and compare the estimated thresholds both empirically and parametrically.

RESULTS

The optimal threshold can be a "moving target" because it would depend on modeling assumptions, metrics, and variability in the data. Even with large samples, disease prevalence has an impact on the robustness of the metrics.

CONCLUSIONS

It is recommended that researchers compare different optimal cutoff points using several metrics and select one that is most clinically relevant. The ultimate goal is to maximize diagnostic performances that are clinically meaningful to achieve improved global health.

An additive selection of markers to improve diagnostic accuracy based on a discriminatory measure

RATIONALE AND OBJECTIVES

The estimation of the area under the receiver operating characteristic (ROC) curve (AUC) often relies on the assumption that the truly positive population tends to have higher marker results than the truly negative population. The authors propose a discriminatory measure to relax such an assumption and apply the measure to identify the appropriate set of markers for combination.

MATERIALS AND METHODS

The proposed measure is based on the maximum of the AUC and 1-AUC. The existing methods are applied to estimate the measure. The subset of markers is selected using a combination method that maximizes a function of the proposed discriminatory score with the number of markers as a penalty in the function.

RESULTS

The properties of the estimators for the proposed measure were studied through large-scale simulation studies. The application was illustrated through a real example to identify the set of markers to combine.

CONCLUSION

Simulation results showed excellent small-sample performance of the estimators for the proposed measure. The application in the example yielded a reasonable subset of markers for combination.

Multivariate normally distributed biomarkers subject to limits of detection and receiver operating characteristic curve inference

RATIONALE AND OBJECTIVES

Biomarkers are of ever-increasing importance to clinical practice and epidemiologic research. Multiple biomarkers are often measured per patient. Measurement of true biomarker levels is limited by laboratory precision, specifically measuring relatively low, or high, biomarker levels resulting in undetectable levels below, or above, a limit of detection (LOD). Ignoring these missing observations or replacing them with a constant are methods commonly used although they have been shown to lead to biased estimates of several parameters of interest, including the area under the receiver operating characteristic (ROC) curve and regression coefficients.

MATERIALS AND METHODS

We developed asymptotically consistent, efficient estimators, via maximum likelihood techniques, for the mean vector and covariance matrix of multivariate normally distributed biomarkers affected by LOD. We also developed an approximation for the Fisher information and covariance matrix for our maximum likelihood estimations (MLEs). We apply these results to an ROC curve setting, generating an MLE for the area under the curve for the best linear combination of multiple biomarkers and accompanying confidence interval.

RESULTS

Point and confidence interval estimates are scrutinized by simulation study, with bias and root mean square error and coverage probability, respectively, displaying behavior consistent with MLEs. An example using three polychlorinated biphenyls to classify women with and without endometriosis illustrates how the underlying distribution of multiple biomarkers with LOD can be assessed and display increased discriminatory ability over naïve methods.

CONCLUSIONS

Properly addressing LODs can lead to optimal biomarker combinations with increased discriminatory ability that may have been ignored because of measurement obstacles.

Mixtures of receiver operating characteristic curves

RATIONALE AND OBJECTIVES

Receiver operating characteristic (ROC) curves are ubiquitous in the analysis of imaging metrics as markers of both diagnosis and prognosis. While empirical estimation of ROC curves remains the most popular method, there are several reasons to consider smooth estimates based on a parametric model.

MATERIALS AND METHODS

A mixture model is considered for modeling the distribution of the marker in the diseased population motivated by the biological observation that there is more heterogeneity in the diseased population than there is in the normal one. It is shown that this model results in an analytically tractable ROC curve which is itself a mixture of ROC curves.

RESULTS

The use of creatine kinase-BB isoenzyme in diagnosis of severe head trauma is used as an example. ROC curves are fit using the direct binormal method, ROCKIT software, and the Box-Cox transformation as well as the proposed mixture model. The mixture model generates an ROC curve that is much closer to the empirical one than the other methods considered.

CONCLUSIONS

Mixtures of ROC curves can be helpful in fitting smooth ROC curves in datasets where the diseased population has higher variability than can be explained by a single distribution.