Construction and Comparison of Two Receiver Operating Characteristic Curves Derived from the Same Samples

The clinical value of passive leg raising plus ultrasound to predict fluid responsiveness in children after cardiac surgery

Background

There are few non-invasive monitoring methods that can reliably predict fluid responsiveness (FR) in children. Here, we interrogate the value of doppler ultrasound evaluation of passive leg raising (PLR)-induced changes in stroke volume (SV) and cardiac output (CO) as a predictor of FR in children with mechanical ventilation after congenital cardiac surgery.

Methods

A total of 40 children with mechanical ventilation following congenital cardiac surgery, who required volume expansion (VE) were included in this study. Hemodynamic parameters such as heart rate (HR), mean arterial pressure (MAP), SV, and central venous pressure (CVP) were monitored before and after PLR and VE. Besides, we assessed changes in SV and CO by bedside ultrasound. Patients showing > 10 % increase in SV in response to VE were considered to be responders (26 patients), while the rest (14 patients) were defined as non-responders.

Results

Our data demonstrated that ΔSV-PLR and ΔCO- PLR were positively correlated with ΔSV-VE (r = 0.683, p < 0.001 and r = 0.374, p = 0.017, respectively), and the area under the ROC curve (AUC) of ΔSV-PLR was 0.879 (95 % CI [0.745 1.000], p < 0.001). The best cut-off value for ΔSV-PLR in predicting FR was 13 %, with its sensitivity and specificity were 81.8 and 86.3 %, respectively. ΔCVP, ΔHR, and ΔMAP were weak predictors of FR in the children.

Conclusions

Our study demonstrated that SV changes, as evaluated by noninvasive ultrasound combined with PLR, could effectively evaluate FR in children under mechanical ventilation after congenital cardiac surgery.

Discriminative Ability for Adverse Outcomes After Surgical Management of Hip Fractures: A Comparison of the Charlson Comorbidity Index, Elixhauser Comorbidity Measure, and Modified Frailty Index

OBJECTIVES

The Charlson comorbidity index (CCI), Elixhauser comorbidity measure (ECM), and modified frailty index (mFI) have been associated with mortality after hip fracture. The present study compares the clinically informative discriminative ability of CCI, ECM, and mFI, as well as demographic characteristics for predicting in-hospital adverse outcomes after surgical management of hip fractures.

METHODS

Patients undergoing hip fracture surgery were selected from the 2013 National Inpatient Sample. The discriminative ability of CCI, ECM, and mFI, as well as demographic factors for adverse outcomes were assessed using the area under the curve analysis from receiver operating characteristic curves. Outcomes included the occurrence of any adverse event, death, severe adverse events, minor adverse events, and extended hospital stay.

RESULTS

In total, 49,738 patients were included (mean age: 82 years). In comparison with CCI and mFI, ECM had the significantly largest discriminative ability for the occurrence of all outcomes. Among demographic factors, age had the sole or shared the significantly largest discriminative ability for all adverse outcomes except extended hospital stay. The best performing comorbidity index (ECM) outperformed the best performing demographic factor (age) for all outcomes.

CONCLUSION

Among both comorbidity indices and demographic factors, the ECM had the best overall discriminative ability for adverse outcomes after surgical management of hip fractures. The use of this index in correctly identifying patients at risk for postoperative complications may help set appropriate patient expectations, assist in optimizing prophylaxis regimens for medical management, and adjust reimbursements. More widespread use of this measure for hip fracture studies may be appropriately considered.

LEVEL OF EVIDENCE

Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Even Highly Correlated Measures Can Add Incrementally to Predicting Recidivism Among Sex Offenders

Criterion-referenced measures, such as those used in the assessment of crime and violence, prioritize predictive accuracy (discrimination) at the expense of construct validity. In this article, we compared the discrimination and incremental validity of three commonly used criterion-referenced measures for sex offenders (Rapid Risk Assessment for Sex Offence Recidivism [RRASOR], Static-99R, and Static-2002R). In a meta-analysis of 20 samples ( n = 7,491), Static-99R and Static-2002R provided similar discrimination but outperformed the RRASOR in the prediction of sexual, violent, and any recidivism. Remarkably, despite large correlations between them ( rs ranging from .70 to .92), these risk scales consistently added incremental validity to one another. The direction of the incremental effects, however, was not consistently positive. When controlling for the other measures, high scores on the RRASOR were associated with lower risk for violent and any recidivism. We also examined different methods of combining risk scales and found that the averaging approach produced better discrimination than choosing the highest score and produced better calibration than either choosing the lowest or highest risk score. The findings reinforce the importance of understanding the psychological content of criterion-referenced measures, even when the sole purpose is to predict a particular outcome and provide some direction concerning the best methods for combining risk scales.

Adding in Vivo Quantitative1H MR Spectroscopy to Improve Diagnostic Accuracy of Breast MR Imaging: Preliminary Results of Observer Performance Study at 4.0 T

PURPOSE

To determine whether the addition of in vivo quantitative hydrogen 1 (1H) magnetic resonance (MR) spectroscopy can improve the radiologist's diagnostic accuracy in interpreting breast MR images to distinguish benign from malignant lesions.

MATERIALS AND METHODS

The study was approved by the institutional review board and, where appropriate, was compliant with the Health Insurance Portability and Accountability Act. All patients provided written informed consent. Fifty-five breast MR imaging cases-one lesion each in 55 patients aged 24-66 years with biopsy-confirmed findings-were retrospectively evaluated by four radiologists. Patients were examined with contrast material-enhanced fat-suppressed T1-weighted 4.0-T MR imaging. The concentration of total choline-containing compounds (tCho) was quantified by using single-voxel 1H MR spectroscopy. For each case, the radiologists were asked to give the percentage probability of malignancy, the Breast Imaging and Reporting Data System category, and a recommendation for patient treatment. Two interpretations were performed for each case: The initial interpretation was based on the lesion's morphologic features and time-signal intensity curve, and the second interpretation was based on the lesion's morphologic features, time-signal intensity curve, and tCho concentration. Receiver operating characteristic (ROC), Wilcoxon signed rank, kappa statistic, and accuracy (based on the area under the ROC curve) analyses were performed.

RESULTS

Of the 55 lesions evaluated, 35 were invasive carcinomas and 20 were benign. The addition of 1H MR spectroscopy resulted in higher sensitivity, specificity, accuracy, and interobserver agreement for all four radiologists. More specifically, two of the four radiologists achieved a significant improvement in sensitivity (P=.03, P=.03), and all four radiologists achieved a significant improvement in accuracy (P = .01, P = .05, P = .009, P < .001).

CONCLUSION

Current study results suggest that the addition of quantitative 1H MR spectroscopy to the breast MR imaging examination may help to improve the radiologist's ability to distinguish benign from malignant breast lesions.

Evaluating collaborative filtering recommender systems

Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

Measuring health status in patients with skeletal metastases treated by surgery

Since the introduction of the Functional Living Index-Cancer (FLIC) > 10 years ago, a variety of general as well as disease-specific quality of life (QOL) instruments have been developed and used as a means of assessing the success of clinical interventions. Clinicians faced with selecting an appropriate QOL measurement tool will need guidance in both the selection of the instrument and its evaluation for applicability to the patient group under study. In our examination of this issue, we analyzed the results of administering the FLIC and the more general SF-36 questionnaire to a group of 17 patients who had undergone orthopaedic surgery to correct problems associated with bone metastatic cancer. Because the SF-36 has been widely adopted as a general Health Status evaluation tool and because of its extensive validation it may be considered a gold standard providing a means of assessing the validity of disease-specific instruments such as the FLIC. Using multivariate regression, we built a model in which five of the eight health status dimensions of the SF-36 account for 77% of the variation in the FLIC scores for this group of patients (R2 = 0.77). The implication of this result is that most of the information (i.e., 77%) provided by the FLIC could be equivalently obtained using the more general, more widely validated SF-36. Despite the small sample size, half of the regression coefficients in this model achieved a high level of statistical significance (p < 0.001). In addition, when applied to new patients, the model showed good predictive capability with relative error 2-20%. If these results can be replicated for other patient disease groups and in larger study populations, it may be possible to use the SF-36 as the primary QOL evaluation tool for cancer patients. The burden of this disease could then be more easily compared with that of other disease groups and the resulting standardization of QOL analysis would be simpler and more cost effective.