The three-class ideal observer for univariate normal data: Decision variable and ROC surface properties

Relying on pulse oximetry to avoid hypoxaemia and hyperoxia: A multicentre prospective cohort study in patients with circulatory failure

PURPOSE

The purpose of this study was to assess the predictive performance of pulse oximetry (SpO₂) to rule out hypoxaemia and hyperoxia in critically ill patients.

METHODS

SpO₂, arterial oxygenation (SaO₂), and arterial partial pressure of oxygen (PaO₂) were prospectively and simultaneously measured every 6 h during the first 24 h of intensive care unit admission in a multicentre cohort of critically ill patients suffering acute circulatory failure. Likelihood ratios associated with different cutoff values of SpO₂ to rule out hypoxaemia (SaO₂ < 90% or PaO₂ < 60 mmHg) or hyperoxia (SaO₂ > 95% or PaO₂ > 100 mmHg) and post-test probabilities were calculated. Mean bias between SpO₂ and SaO₂ and agreement interval were calculated. Area under the receiver operating characteristics associated with SpO₂ to predict different threshold values of SaO₂ and PaO₂ were calculated.

RESULTS

Five hundred seventy-one patients (mean [standard deviation] Simplified Acute Physiology Score II: 58.7 [20.1]; mechanically ventilated 75.6%) with 2643 available SaO₂ and PaO₂ samples and corresponding 2643 SpO₂ values were analysed. Mean bias between SpO₂ and SaO₂ was 1.1%, and its agreement interval ranged from -8.2 to +11.1%. SpO₂ cutoff values of 88%, 90%, and 92% left the possibility that 8%-13% of patients had hypoxaemia. SpO₂ < 95% left the possibility that 31% of patients had hyperoxia. All calculated areas under the receiver operating characteristics showed a lower limit of their 95% confidence interval below 0.85 CONCLUSION: In this cohort of patients with circulatory failure, SpO₂ had poor discriminative ability to rule out hypoxaemia and hyperoxia. Overconfidence upon SpO₂ monitoring may be dangerous.

Validation of Monte Carlo estimates of three-class ideal observer operating points for normal data

RATIONALE AND OBJECTIVES

Traditional two-class receiver operating characteristic (ROC) analysis is inadequate for the complete evaluation of observer performance in tasks with more than two classes.

MATERIALS AND METHODS

Here, a Monte Carlo estimation method for operating point coordinates on a three-class ROC surface is developed and compared with analytically calculated coordinates in two special cases: (1) univariate and (2) restricted bivariate trinormal underlying data.

RESULTS

In both cases, the statistical estimates were found to be good in the sense that the analytical values lay within the 95% confidence interval of the estimated values about 95% of the time.

CONCLUSIONS

The statistical estimation method should be key in the development of a pragmatic performance metric for evaluation of observers in classification tasks with three or more classes.