Statistical inference for decision curve analysis, with applications to cataract diagnosis

Quantifying Clinicians' Diagnostic Uncertainty When Making Initial Treatment Decisions for Microbial Keratitis

PURPOSE

There is a need to understand physicians' diagnostic uncertainty in the initial management of microbial keratitis (MK). This study aimed to understand corneal specialists' diagnostic uncertainty by establishing risk thresholds for treatment of MK that could be used to inform a decision curve analysis for prediction modeling.

METHODS

A cross-sectional survey of corneal specialists with at least 2 years clinical experience was conducted. Clinicians provided the percentage risk at which they would always or never treat MK types (bacterial, fungal, herpetic, and amoebic) based on initial ulcer sizes and locations (<2 mm 2 central, <2 mm 2 peripheral, and >8 mm 2 central).

RESULTS

Seventy-two of 99 ophthalmologists participated who were 50% female with an average of 14.7 (SD = 10.1) years of experience, 60% in academic practices, and 38% outside the United States. Clinicians reported they would "never" and "always" treat a <2 mm 2 central MK infection if the median risk was 0% and 20% for bacterial (interquartile range, IQR = 0-5 and 5-50), 4.5% and 27.5% for herpetic (IQR = 0-10 and 10-50), 5% and 50% for fungal (IQR = 0-10 and 20-75), and 5% and 50.5% for amoebic (IQR = 0-20 and 32-80), respectively. Mixed-effects models showed lower thresholds to treat larger and central infections ( P < 0.001, respectively), and thresholds to always treat differed between MK types for the United States ( P < 0.001) but not international clinicians.

CONCLUSIONS

Risk thresholds to treat differed by practice locations and MK types, location, and size. Researchers can use these thresholds to understand when a clinician is uncertain and to create decision support tools to guide clinicians' treatment decisions.

Potential of GJA8 gene variants in predicting age-related cataract: A comparison of supervised machine learning methods

Cataracts are the problems associated with the crystallins proteins of the eye lens. Any perturbation in the conformity of these proteins results in a cataract. Age-related cataract is the most common type among all cataracts as it accounts for almost 80% of cases of senile blindness worldwide. This research study was performed to predict the role of single nucleotide polymorphisms (SNPs) of the GJA8 gene with age-related cataracts in 718 subjects (400 age-related cataract patients and 318 healthy individuals). A comparison of supervised machine learning classification algorithm including logistic regression (LR), random forest (RF) and Artificial Neural Network (ANN) were presented to predict the age-related cataracts. The results indicated that LR is the best for predicting age-related cataracts. This successfully developed model after accounting different genetic and demographic factors to predict cataracts will help in effective disease management and decision-making medical practitioner and experts.

Decision curve analysis: confidence intervals and hypothesis testing for net benefit

BACKGROUND

A number of recent papers have proposed methods to calculate confidence intervals and p values for net benefit used in decision curve analysis. These papers are sparse on the rationale for doing so. We aim to assess the relation between sampling variability, inference, and decision-analytic concepts.

METHODS AND RESULTS

We review the underlying theory of decision analysis. When we are forced into a decision, we should choose the option with the highest expected utility, irrespective of p values or uncertainty. This is in some distinction to traditional hypothesis testing, where a decision such as whether to reject a given hypothesis can be postponed. Application of inference for net benefit would generally be harmful. In particular, insisting that differences in net benefit be statistically significant would dramatically change the criteria by which we consider a prediction model to be of value. We argue instead that uncertainty related to sampling variation for net benefit should be thought of in terms of the value of further research. Decision analysis tells us which decision to make for now, but we may also want to know how much confidence we should have in that decision. If we are insufficiently confident that we are right, further research is warranted.

CONCLUSION

Null hypothesis testing or simple consideration of confidence intervals are of questionable value for decision curve analysis, and methods such as value of information analysis or approaches to assess the probability of benefit should be considered instead.

Predicting cerebral white matter lesions based on the platelet-to-white blood cell ratio in hypertensive patients

Hypertension is a common chronic disease affecting many people. White matter lesions (WMLs) are one of the imaging features of cerebrovascular disease. Predicting the possibility of developing syncretic WMLs in patients with hypertension may contribute to the early identification of serious clinical conditions. This study aims to build a model to identify patients who suffered from moderate-to-severe WMLs by using recognized WMLs risk factors including age and history of diabetes and a new factor named platelet-to-white blood cell ratio (PWR). A total of 237 patients were included in this study. The Affiliated ZhongDa Hospital of Southeast University Research Ethics Committee approved this study (Ethics No. 2019ZDSYLL189-P01). We developed a nomogram to predict the risk of syncretic WMLs in patients with hypertension using the above factors. Higher total scores on the nomogram indicated a higher risk of syncretic WMLs. This means older age, smaller PWR, and patients suffering from diabetes contributed to a greater chance for the patient to suffer from syncretic WMLs. We used a decision analysis curve(DCA) to determine the net benefit of the prediction model. The DCA we constructed showed that using our model to decide whether patients suffered from syncretic WMLs or not was better than assuming they all suffered from syncretic WMLs or all WMLs-free. As a result, the area under the curve of our model was 0.787. By integrating PWR, history of diabetes, and age, we could estimate integrated WMLs in hypertensive patients. This study provides a potential tool to identify cerebrovascular disease in patients with hypertension.

External Validation of a Machine Learning Algorithm for Predicting Clinically Meaningful Functional Improvement After Arthroscopic Hip Preservation Surgery

BACKGROUND

Individualized risk prediction has become possible with machine learning (ML), which may have important implications in enhancing clinical decision making. We previously developed an ML algorithm to predict propensity for clinically meaningful outcome improvement after hip arthroscopy for femoroacetabular impingement syndrome. External validity of prognostic models is critical to determine generalizability, although it is rarely performed.

PURPOSE

To assess the external validity of an ML algorithm for predicting clinically meaningful improvement after hip arthroscopy.

STUDY DESIGN

Cohort study; Level of evidence, 3.

METHODS

An independent hip preservation registry at a tertiary academic medical center was queried for consecutive patients/athletes who underwent hip arthroscopy for femoroacetabular impingement syndrome between 2015 and 2017. By assuming a minimal clinically important difference (MCID) outcome/event proportion of 75% based on the original study, a minimum sample of 132 patients was required. In total, 154 patients were included. Age, body mass index, alpha angle on anteroposterior pelvic radiographs, Tönnis grade and angle, and preoperative Hip Outcome Score-Sports Subscale were used as model inputs to predict the MCID for the Hip Outcome Score-Sports Subscale 2 years postoperatively. Performance was assessed using identical metrics to the internal validation study and included discrimination, calibration, Brier score, and decision curve analysis.

RESULTS

The concordance statistic in the validation cohort was 0.80 (95% CI, 0.71 to 0.87), suggesting good to excellent discrimination. The calibration slope was 1.16 (95% CI, 0.74 to 1.61) and the calibration intercept 0.13 (95% CI, -0.26 to 0.53). The Brier score was 0.15 (95% CI, 0.12 to 0.18). The null model Brier score was 0.20. Decision curve analysis revealed favorable net treatment benefit for patients with use of the algorithm as compared with interventional changes made for all and no patients.

CONCLUSION

The performance of this algorithm in an independent patient population in the northeast region of the United States demonstrated superior discrimination and comparable calibration to that of the derivation cohort. The external validation of this algorithm suggests that it is a reliable method to predict propensity for clinically meaningful improvement after hip arthroscopy and is an essential step forward toward introducing initial use in clinical practice. Potential uses include integration into electronic medical records for automated prediction, enhanced shared decision making, and more informed allocation of resources to optimize patient outcomes.

An Efficient Nomogram for Discriminating Intrahepatic Cholangiocarcinoma From Hepatocellular Carcinoma: A Retrospective Study

Objective

This study aims to establish a nomogram and provide an effective method to distinguish between intrahepatic cholangiocarcinoma (ICC) and hepatocellular carcinoma (HCC).

Methods

A total of 1,591 patients with HCC or ICC hospitalized at Shandong Provincial Hospital between January 2016 and August 2021 were included and randomly divided into development and validation groups in a ratio of 3:1. Univariate and multivariate analyses were performed to determine the independent differential factors between HCC and ICC patients in the development cohort. By combining these independent differential factors, the nomogram was established for discriminating ICC from HCC. The accuracy of the nomogram was estimated by using receiver operating characteristic (ROC) curve and decision curve analysis (DCA). Furthermore, the predictive nomogram was assessed in the internal testing set.

Results

Through multivariate analysis, independent differential factors between HCC and ICC involved hepatitis B virus (HBV), logarithm of alpha-fetoprotein (Log AFP), logarithm of protein induced by vitamin K absence or antagonist-II (Log PIVKA-II), logarithm of carbohydrate antigen 199 (Log CA199), and logarithm of carbohydrate antigen 125 (Log CA125). A nomogram was finally established by incorporating these five independent differential factors. Comparing a model of conventional tumor biomarkers including AFP and CA199, the nomogram showed a better distinction between ICC and HCC. The area under the ROC curve (AUC) of ICC diagnosis was 0.951 (95% CI, 0.938–0.964) for the nomogram. The results were consistent in the validation cohort with an AUC of 0.958 (95% CI, 0.938–0.978). After integrating patient preferences into the analysis, the DCA showed that using this nomogram to distinguish ICC and HCC increased more benefit compared with the conventional model.

Conclusion

An efficient nomogram has been established for the differential diagnosis between ICC and HCC, which may facilitate the detection and diagnosis of ICC. Further use of the nomogram in multicenter investigations will confirm the practicality of the tool for future clinical application.

Correcting for partial verification bias in diagnostic accuracy studies: A tutorial using R

Diagnostic tests play a crucial role in medical care. Thus any new diagnostic tests must undergo a thorough evaluation. New diagnostic tests are evaluated in comparison with the respective gold standard tests. The performance of binary diagnostic tests is quantified by accuracy measures, with sensitivity and specificity being the most important measures. In any diagnostic accuracy study, the estimates of these measures are often biased owing to selective verification of the patients, which is referred to as partial verification bias. Several methods for correcting partial verification bias are available depending on the scale of the index test, target outcome, and missing data mechanism. However, these are not easily accessible to the researchers due to the complexity of the methods. This article aims to provide a brief overview of the methods available to correct for partial verification bias involving a binary diagnostic test and provide a practical tutorial on how to implement the methods using the statistical programming language R.

Supervised Learning Based Systemic Inflammatory Markers Enable Accurate Additional Surgery for pT1NxM0 Colorectal Cancer: A Comparative Analysis of Two Practical Prediction Models for Lymph Node Metastasis

Purpose

Predicting lymph node metastasis (LNM) after endoscopic resection is crucial in determining whether patients with pT1NxM0 colorectal cancer (CRC) should undergo additional surgery. This study was aimed to develop a predictive model that can be used to reduce the current likelihood of overtreatment.

Patients and Methods

We recruited a total of 1194 consecutive CRC patients with pT1NxM0 who underwent endoscopic or surgical resection at the Gezhouba Central Hospital of Sinopharm between January 1, 2006, and August 31, 2021. The random forest classifier (RFC) and generalized linear algorithm (GLM) were used to screen out the variables that greatly affected the LNM prediction, respectively. The area under the curve (AUC) and decision curve analysis (DCA) were applied to assess the accuracy of predictive models.

Results

Analysis identified the top 10 candidate factors including depth of submucosal invasion, neutrophil-lymphocyte ratio (NLR), platelet lymphocyte ratio (PLR), platelet-to-neutrophil ratio(PNR), venous invasion, poorly differentiated clusters, tumor budding, grade, lymphatic vascular invasion, and background adenoma. The performance of the GLM achieved the highest AUC of 0.79 (95% confidence interval [CI]: 0.30 to 1.28) in the training cohort and robust AUC of 0.80 (95% confidence interval [CI]: 0.36 to 1.24) in the validation cohort. Meanwhile, the RFC exhibited a robust AUC of 0.84 (95% confidence interval [CI]: 0.40 to 1.28) in the training cohort and a high AUC of 0.85 (95% CI: 0.41 to 1.29) in the validation cohort. DCAs also showed that the RFC had superior predictive ability.

Conclusion

Our supervised learning-based model incorporating histopathologic parameters and inflammatory markers showed a more accurate predictive performance compared to the GLM. This newly supervised learning-based predictive model can be used to determine an individually tailored treatment strategy.