Bayesian reclassification statistics for assessing improvements in diagnostic accuracy

We propose a Bayesian approach to the estimation of the net reclassification improvement (NRI) and three versions of the integrated discrimination improvement (IDI) under the logistic regression model. Both NRI and IDI were proposed as numerical characterizations of accuracy improvement for diagnostic tests and were shown to retain certain practical advantage over analysis based on ROC curves and offer complementary information to the changes in area under the curve. Our development is a new contribution towards Bayesian solution for the estimation of NRI and IDI, which eases computational burden and increases flexibility. Our simulation results indicate that Bayesian estimation enjoys satisfactory performance comparable with frequentist estimation and achieves point estimation and credible interval construction simultaneously. We adopt the methodology to analyze a real data from the Singapore Malay Eye Study. Copyright © 2016 John Wiley & Sons, Ltd.

Inference for reclassification statistics under nested and non-nested models for biomarker evaluation

The Net Reclassification Improvement (NRI) and the Integrated Discrimination Improvement (IDI) are used to evaluate the diagnostic accuracy improvement for biomarkers in a wide range of applications. Most applications for these reclassification metrics are confined to nested model comparison. We emphasize the important extensions of these metrics to the non-nested comparison. Non-nested models are important in practice, in particular, in high-dimensional data analysis and in sophisticated semiparametric modeling. We demonstrate that the assessment of accuracy improvement may follow the familiar NRI and IDI evaluation. While the statistical properties of the estimators for NRI and IDI have been well studied in the nested setting, one cannot always rely on these asymptotic results to implement the inference procedure for practical data, especially for testing the null hypothesis of no improvement, and these properties have not been established for the non-nested setting. We propose a generic bootstrap re-sampling procedure for the construction of confidence intervals and hypothesis tests. Extensive simulations and real biomedical data examples illustrate the applicability of the proposed inference methods for both nested and non-nested models.