Investigation of optimal use of computer-aided detection systems: the role of the "machine" in decision making process

RATIONALE AND OBJECTIVES

The aim of this study was to explore different computerized models (the "machine") as a means to achieve optimal use of computer-aided detection (CAD) systems and to investigate whether these models can play a primary role in clinical decision making and possibly replace a clinician's subjective decision for combining his or her own assessment with that provided by a CAD system.

MATERIALS AND METHODS

Data previously collected from a fully crossed, multiple-reader, multiple-case observer study with and without CAD by seven observers asked to identify simulated small masses on two separate sets of 100 mammographic images (low-contrast and high-contrast sets; ie, low-contrast and high-contrast simulated masses added to random locations on normal mammograms) were used. This allowed testing two relative sensitivities between the observers and CAD. Seven models that combined detection assessments from CAD standalone, unaided read, and CAD-aided read (second read and concurrent read) were developed using the leave-one-out technique for training and testing. These models were personalized for each observer. Detection performance accuracies were analyzed using the area under a portion of the free-response receiver-operating characteristic curve (AUFC), sensitivity, and number of false-positives per image.

RESULTS

For the low-contrast set, the use of computerized models resulted in significantly higher AUFCs compared to the unaided read mode for all readers, whereas the increased AUFCs between CAD-aided (second and concurrent reads; ie, decisions made by the readers) and unaided read modes were statistically significant for a majority, but not all, of the readers (four and five of the seven readers, respectively). For the high-contrast set, there were no significant trends in the AUFCs whether or not a model was used to combine the original reading modes. Similar results were observed when using sensitivity as the figure of merit. However, the average number of false-positives per image resulting from the computerized models remained the same as that obtained from the unaided read modes.

CONCLUSIONS

Individual computerized models (the machine) that combine image assessments from CAD standalone, unaided read, and CAD-aided read can increase detection performance compared to the reading done by the observer. However, relative sensitivity (ie, the difference in sensitivity between CAD standalone and unaided read) was a critical factor that determined incremental improvement in decision making, whether made by the observer or using computerized models.