A tutorial on the use of ROC analysis for computer-aided diagnostic systems

Can AI-based body composition assessment outperform body surface area in predicting dose-limiting toxicities for colonic cancer patients on chemotherapy?

PURPOSE

Gold standard chemotherapy dosage is based on body surface area (BSA); however many patients experience dose-limiting toxicities (DLT). We aimed to evaluate the effectiveness of BSA, two-dimensional (2D) and three-dimensional (3D) body composition (BC) measurements derived from Lumbar 3 vertebra (L3) computed tomography (CT) slices, in predicting DLT in colon cancer patients.

METHODS

203 patients (60.87 ± 12.42 years; 97 males, 47.8%) receiving adjuvant chemotherapy (Oxaliplatin and/or 5-Fluorouracil) were retrospectively evaluated. An artificial intelligence segmentation model was used to extract 2D and 3D body composition measurements from each patients' single mid-L3 CT slice as well as multiple-L3 CT scans to produce a 3D BC report. DLT was defined as any incidence of dose reduction or discontinuation due to chemotherapy toxicities. A receiver operating characteristic (ROC) analysis was performed on BSA and individual body composition measurements to demonstrate their predictive performance.

RESULTS

A total of 120 (59.1%) patients experienced DLT. Age and BSA did not vary significantly between DLT and non-DLT group. Females were significantly more likely to experience DLT (p = 4.9 × 10-3). In all patients, the predictive effectiveness of 2D body composition measurements (females: AUC = 0.50-0.54; males: AUC = 0.50-0.61) was equivalent to that of BSA (females: AUC = 0.49; males: AUC = 0.58). The L3 3D skeletal muscle volume was the most predictive indicator of DLT (AUC of 0.66 in females and 0.64 in males).

CONCLUSION

Compared to BSA and 2D body composition measurements, 3D L3 body composition measurements had greater potential to predict DLT in CRC patients receiving chemotherapy and this was sex dependent.

Efficacy of texture analysis of ultrasonographic images in the differentiation of metastatic and non-metastatic cervical lymph nodes in patients with squamous cell carcinoma of the tongue

OBJECTIVE

We investigated the efficacy of using texture analysis of ultrasonographic images of the cervical lymph nodes of patients with squamous cell carcinoma of the tongue to differentiate between metastatic and non-metastatic lymph nodes.

STUDY DESIGN

We analyzed 32 metastatic and 28 non-metastatic lymph nodes diagnosed by histopathologic examination on presurgical US images. Using the LIFEx texture analysis program, we extracted 36 texture features from the images and calculated the statistical significance of differences in texture features between metastatic and non-metastatic lymph nodes using the t test. To assess the diagnostic ability of the significantly different texture features to discriminate between metastatic and non-metastatic nodes, we performed receiver operating characteristic curve analysis and calculated the area under the curve. We set the cutoff points that maximized the sensitivity and specificity for each curve according to the Youden J statistic.

RESULTS

We found that 20 texture features significantly differed between metastatic and non-metastatic lymph nodes. Among them, only the gray-level run length matrix feature of run length non-uniformity and the gray-level zone length matrix features of gray-level non-uniformity and zone length non-uniformity showed an excellent ability to discriminate between metastatic and non-metastatic lymph nodes as indicated by the area under the curve and the sum of sensitivity and specificity.

CONCLUSIONS

Analysis of the texture features of run length non-uniformity, gray-level non-uniformity, and zone length non-uniformity values allows for differentiation between metastatic and non-metastatic lymph nodes, with the use of gray-level non-uniformity appearing to be the best means of predicting metastatic lymph nodes.

Validation of modified Mallampati test with addition of thyromental distance and sternomental distance to predict difficult endotracheal intubation in adults

Background and Aims:

Intubation is often a challenge for anaesthesiologists. Many parameters assist to predict difficult intubation. The present study was undertaken to assess the validity of different parameters in predicting difficult intubation for general anaesthesia (GA) in adults and effect of combining the parameters on the validity.

Methods:

The anaesthesiologist assessed oropharynx of 135 adult patients. Modified Mallampati test (MMT) was used and the thyromental distance (TMD) and sternomental distances (SMD) for each of the patients were also measured. The Cormack and Lehane laryngoscopic grading was assessed following laryngoscopy. The validity parameters such as sensitivity, specificity, false positive and negatives values, positive and negative predictive values were calculated. The effect of combining different measurements on the validity was also studied. Univariate analysis was performed using the parametric method.

Results:

The study group comprised of 135 patients. The sensitivity and specificity of MMT were 28.6% and 93%, respectively. The TMD (<6.5 CM) had sensitivity and specificity of 100% and 75.8%, respectively. The SMD (<12.5 CM) had sensitivity and specificity of 91% and 92.7%, respectively. Combination of MMT grading and TMD and SMD measurements increased the validity (sensitivity of 100% and specificity of 92.7%).

Conclusion:

MMT had high specificity. The validity of combination of MMT, SMD and TMD as compared to MMT alone was very high in predicting difficult intubation in adult patients. All parameters should be used in assessing an adult patient for surgery under GA.

Treating balance disorders by ultra-low-level laser stimulation of acupoints

BACKGROUND

Balance disorders, a widespread problem in the senior population, are very often related to painful musculoskeletal disabilities and/or to degenerative neurologic pathologies. In a previous work, we investigated the beneficial effects of acupuncture such as laser photobiostimulation, regardless of the etiological origin of the balance dysfunction.

AIMS

Attention was focused on two individuals showing balance deficits with different etiologies. One healthy control individual was receiving treatment, one healthy control individual did not receive any treatment. Balance performances were measured before and after ultra-low-level laser (ULLL) stimulation as a basis for the design of further investigations.

METHODS

Romberg's parameters were measured on a force platform. Of the four tested individuals, one showed dizziness symptoms, one was affected by a mild form of Parkinson's disease, and two showed no dysfunction and were used as controls.

RESULTS

It was found that a ULLL stimulation of peripheral Traditional Chinese Medicine (TCM) acupuncture points can be effective, regardless of the pathology.

DISCUSSION

A specific clinical analysis of the stability of the beneficial result as a function of homogeneous groups of dysfunctional individuals is needed.

Analysis of co-occurrence texture statistics as a function of gray-level quantization for classifying breast ultrasound

In this paper, we investigated the behavior of 22 co-occurrence statistics combined to six gray-scale quantization levels to classify breast lesions on ultrasound (BUS) images. The database of 436 BUS images used in this investigation was formed by 217 carcinoma and 219 benign lesions images. The region delimited by a minimum bounding rectangle around the lesion was employed to calculate the gray-level co-occurrence matrix (GLCM). Next, 22 co-occurrence statistics were computed regarding six quantization levels (8, 16, 32, 64, 128, and 256), four orientations (0° , 45° , 90° , and 135°), and ten distances (1, 2,...,10 pixels). Also, to reduce feature space dimensionality, texture descriptors of the same distance were averaged over all orientations, which is a common practice in the literature. Thereafter, the feature space was ranked using mutual information technique with minimal-redundancy-maximal-relevance (mRMR) criterion. Fisher linear discriminant analysis (FLDA) was applied to assess the discrimination power of texture features, by adding the first m-ranked features to the classification procedure iteratively until all of them were considered. The area under ROC curve (AUC) was used as figure of merit to measure the performance of the classifier. It was observed that averaging texture descriptors of a same distance impacts negatively the classification performance, since the best AUC of 0.81 was achieved with 32 gray levels and 109 features. On the other hand, regarding the single texture features (i.e., without averaging procedure), the quantization level does not impact the discrimination power, since AUC = 0.87 was obtained for the six quantization levels. Moreover, the number of features was reduced (between 17 and 24 features). The texture descriptors that contributed notably to distinguish breast lesions were contrast and correlation computed from GLCMs with orientation of 90° and distance more than five pixels.

Computer aided diagnosis of parotid gland lesions using ultrasonic multi-feature tissue characterization

In this article, an ultrasound based system for computer aided characterization of biologic tissue and its application to differential diagnosis of parotid gland lesions is proposed. Aiming at an automated differentiation between malignant and benign cases, the system is based on a supervised classification using tissue-describing features derived from ultrasound radio-frequency (RF) echo signals and image data. Standard diagnostic ultrasound equipment was employed to acquire ultrasound RF echo data from parotid glands of 138 patients. Lesions were manually demarcated as regions-of-interest (ROIs) in the B-mode images. Spectral ultrasound backscatter and attenuation parameters are estimated from diffraction corrected RF data, yielding spatially resolved parameter images. Histogram based statistical measures derived from the parameters distributions inside the ROI are used as tissue describing features. In addition, texture features and shape descriptors are extracted from demodulated ultrasound image data. The features are processed by a maximum likelihood classifier. An optimal set of 10 features was chosen by a sequential forward selection algorithm. The classifier's performance is evaluated using total cross validation and receiver operating characteristic (ROC) analysis. As a reference method, postoperative pathohistologic analysis was conducted and proved malignancy or prospective malignancy in 51 patients. The classification using the proposed system yielded an area under the ROC curve of 0.91, proving significant potential for differentiating between malignant and benign parotid gland lesions.

Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations

PURPOSE

To evaluate the performance of support vector machine, multi-layer perceptron and radial basis function neural network as auxiliary tools to identify keratoconus from Orbscan II maps.

METHODS

A total of 318 maps were selected and classified into four categories: normal (n = 172), astigmatism (n = 89), keratoconus (n = 46) and photorefractive keratectomy (n = 11). For each map, 11 attributes were obtained or calculated from data provided by the Orbscan II. Ten-fold cross-validation was used to train and test the classifiers. Besides accuracy, sensitivity and specificity, receiver operating characteristic (ROC) curves for each classifier were generated, and the areas under the curves were calculated.

RESULTS

The three selected classifiers provided a good performance, and there were no differences between their performances. The area under the ROC curve of the support vector machine, multi-layer perceptron and radial basis function neural network were significantly larger than those for all individual Orbscan II attributes evaluated (p < 0.05).

CONCLUSION

Overall, the results suggest that using a support vector machine, multi-layer perceptron classifiers and radial basis function neural network, these classifiers, trained on Orbscan II data, could represent useful techniques for keratoconus detection.

Evaluation of segmentation algorithms for coronary angiography

This paper presents an evaluation technique based on ROC analysis to compare four segmentation algorithms in coronary angiography images. We have implemented four algorithms and we have optimized their parameters; then, we have compared their performances. Both steps are done by means of a ROC analysis. Experimental results are presented on simulated data and real images.