The More Eyes, the Better to See? From Double to Quadruple Reading of Screening Mammograms

[Measurements errors in screening mammogram interpretation by radiologists]

The timely detection of breast cancer is achieved through mammography; however, the quality of the procedure should be addressed for proper performance and interpretation. Despite recent improvements in quality assurance in mammography, interpretation still depends on each reader; therefore, errors can be made when interpreting screening mammograms, leading to unnecessary biopsies and/or overdiagnosis, with sustained physical, economic and psychological consequences. Since interpretation is related to the perceptive and cognitive ability of the radiologist, it is necessary to have extensive knowledge about the possible errors that may occur during interpretation, as well as of the way how they can be reduced, prevented and/or corrected to provide the patient with the highest possible level of safety.

Analysis of underlying causes of inter-expert disagreement in retinopathy of prematurity diagnosis. Application of machine learning principles

OBJECTIVE

Inter-expert variability in image-based clinical diagnosis has been demonstrated in many diseases including retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and is a major cause of childhood blindness. In order to better understand the underlying causes of variability among experts, we propose a method to quantify the variability of expert decisions and analyze the relationship between expert diagnoses and features computed from the images. Identification of these features is relevant for development of computer-based decision support systems and educational systems in ROP, and these methods may be applicable to other diseases where inter-expert variability is observed.

METHODS

The experiments were carried out on a dataset of 34 retinal images, each with diagnoses provided independently by 22 experts. Analysis was performed using concepts of Mutual Information (MI) and Kernel Density Estimation. A large set of structural features (a total of 66) were extracted from retinal images. Feature selection was utilized to identify the most important features that correlated to actual clinical decisions by the 22 study experts. The best three features for each observer were selected by an exhaustive search on all possible feature subsets and considering joint MI as a relevance criterion. We also compared our results with the results of Cohen's Kappa [36] as an inter-rater reliability measure.

RESULTS

The results demonstrate that a group of observers (17 among 22) decide consistently with each other. Mean and second central moment of arteriolar tortuosity is among the reasons of disagreement between this group and the rest of the observers, meaning that the group of experts consider amount of tortuosity as well as the variation of tortuosity in the image.

CONCLUSION

Given a set of image-based features, the proposed analysis method can identify critical image-based features that lead to expert agreement and disagreement in diagnosis of ROP. Although tree-based features and various statistics such as central moment are not popular in the literature, our results suggest that they are important for diagnosis.