Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?

From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science

For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target 'T' if a different target 'T' was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science. Here we review the SSM literature in both radiology and cognitive science and discuss: (1) the current SSM theories (i.e., satisfaction, perceptual set, and resource depletion theories), (2) the eye movement errors that underlie the SSM effect, (3) the existing efforts tested to alleviate SSM errors, and (4) the evolution of methodologies and analyses used when calculating the SSM effect. Finally, we present the attentional template theory, a novel mechanistic explanation for SSM errors, which ties together our current understanding of SSM errors and the attentional template literature.

Mammography to tomosynthesis: examining the differences between two-dimensional and segmented-three-dimensional visual search

BACKGROUND

Radiological techniques for breast cancer detection are undergoing a massive technological shift-moving from mammography, a process that takes a two-dimensional (2D) image of breast tissue, to tomosynthesis, a technique that creates a segmented-three-dimensional (3D) image. There are distinct benefits of tomosynthesis over mammography with radiologists having fewer false positives and more accurate detections; yet there is a significant and meaningful disadvantage with tomosynthesis in that it takes longer to evaluate each patient. This added time can dramatically impact workflow and have negative attentional and cognitive impacts on interpretation of medical images. To better understand the nature of segmented-3D visual search and the implications for radiology, the current study looked to establish a new testing platform that could reliably examine differences between 2D and segmented-3D search.

RESULTS

In Experiment 1, both professionals (radiology residents and certified radiologists) and non-professionals (undergraduate students) were found to have fewer false positives and were more accurate in segmented-3D displays, but at the cost of taking significantly longer in search. Experiment 2 tested a second group of non-professional participants, using a background that more closely resembled a mammogram, and replicated the results of Experiment 1-search was more accurate and there were fewer false alarms in segmented 3D displays but took more time.

CONCLUSION

The results of Experiments 1 and 2 matched the performance patterns found in previous radiology studies and in the clinic, suggesting this novel experimental paradigm potentially provides a flexible and cost-effective tool that can be utilized with non-professional populations to inform relevant visual search performance. From an academic perspective, this paradigm holds promise for examining the nature of segmented-3D visual search.

Can computer-aided diagnosis assist in the identification of prostate cancer on prostate MRI? a multi-center, multi-reader investigation

For prostate cancer detection on prostate multiparametric MRI (mpMRI), the Prostate Imaging-Reporting and Data System version 2 (PI-RADSv2) and computer-aided diagnosis (CAD) systems aim to widely improve standardization across radiologists and centers. Our goal was to evaluate CAD assistance in prostate cancer detection compared with conventional mpMRI interpretation in a diverse dataset acquired from five institutions tested by nine readers of varying experience levels, in total representing 14 globally spread institutions. Index lesion sensitivities of mpMRI-alone were 79% (whole prostate (WP)), 84% (peripheral zone (PZ)), 71% (transition zone (TZ)), similar to CAD at 76% (WP, p=0.39), 77% (PZ, p=0.07), 79% (TZ, p=0.15). Greatest CAD benefit was in TZ for moderately-experienced readers at PI-RADSv2 <3 (84% vs mpMRI-alone 67%, p=0.055). Detection agreement was unchanged but CAD-assisted read times improved (4.6 vs 3.4 minutes, p<0.001). At PI-RADSv2 ≥ 3, CAD improved patient-level specificity (72%) compared to mpMRI-alone (45%, p<0.001). PI-RADSv2 and CAD-assisted mpMRI interpretations have similar sensitivities across multiple sites and readers while CAD has potential to improve specificity and moderately-experienced radiologists' detection of more difficult tumors in the center of the gland. The multi-institutional evidence provided is essential to future prostate MRI and CAD development.

How humans react to changing rewards during visual foraging

Much is known about the speed and accuracy of search in single-target search tasks, but less attention has been devoted to understanding search in multiple-target foraging tasks. These tasks raise and answer important questions about how individuals decide to terminate searches in cases in which the number of targets in each display is unknown. Even when asked to find every target, individuals quit before exhaustively searching a display. Because a failure to notice targets can have profound effects (e.g., missing a malignant tumor in an X-ray), it is important to develop strategies that could limit such errors. Here, we explored the impact of different reward patterns on these failures. In the Neutral condition, reward for finding a target was constant over time. In the Increasing condition, reward increased for each successive target in a display, penalizing early departure from a display. In the Decreasing condition, reward decreased for each successive target in a display. The experimental results demonstrate that observers will forage for longer (and find more targets) when the value of successive targets increases (and the opposite when value decreases). The data indicate that observers were learning to utilize knowledge of the reward pattern and to forage optimally over the course of the experiment. Simulation results further revealed that human behavior could be modeled with a variant of Charnov's Marginal Value Theorem (MVT) (Charnov, 1976) that includes roles for reward and learning.

An individual differences approach to multiple-target visual search errors: How search errors relate to different characteristics of attention

A persistent problem in visual search is that searchers are more likely to miss a target if they have already found another in the same display. This phenomenon, the Subsequent Search Miss (SSM) effect, has remained despite being a known issue for decades. Increasingly, evidence supports a resource depletion account of SSM errors-a previously detected target consumes attentional resources leaving fewer resources available for the processing of a second target. However, "attention" is broadly defined and is composed of many different characteristics, leaving considerable uncertainty about how attention affects second-target detection. The goal of the current study was to identify which attentional characteristics (i.e., selection, limited capacity, modulation, and vigilance) related to second-target misses. The current study compared second-target misses to an attentional blink task and a vigilance task, which both have established measures that were used to operationally define each of four attentional characteristics. Second-target misses in the multiple-target search were correlated with (1) a measure of the time it took for the second target to recovery from the blink in the attentional blink task (i.e., modulation), and (2) target sensitivity (d') in the vigilance task (i.e., vigilance). Participants with longer recovery and poorer vigilance had more second-target misses in the multiple-target visual search task. The results add further support to a resource depletion account of SSM errors and highlight that worse modulation and poor vigilance reflect a deficit in attentional resources that can account for SSM errors.

Satisfaction of search from detection of pulmonary nodules in computed tomography of the chest

RATIONALE AND OBJECTIVES

We tested whether satisfaction of search (SOS) effects that occur in computed tomography (CT) examination of the chest on detection of native abnormalities are produced by the addition of simulated pulmonary nodules.

MATERIALS AND METHODS

Two experiments were conducted. In the first experiment, 70 CT examinations, half that demonstrated diverse, subtle abnormalities and half that demonstrated no native lesions, were read by 18 radiology residents and fellows under two experimental conditions: presented with and without pulmonary nodules. In a second experiment, many of the examinations were replaced to include more salient native abnormalities. This set was read by 14 additional radiology residents and fellows. In both experiments, detection of the natural abnormalities was studied. Receiver operating characteristic (ROC) curve areas for each reader-treatment combination were estimated using empirical and proper ROC models. Additional analyses focused on decision thresholds and visual search time on abnormality-free CT slice ranges. Institutional review board approval and informed consent from 32 participants were obtained.

RESULTS

Observers more often missed diverse native abnormalities when pulmonary nodules were added, but also made fewer false-positive responses. There was no change in ROC area, but decision criteria grew more conservative. The SOS effect on decision thresholds was accompanied by a reduction in search time on abnormality-free CT slice ranges.

CONCLUSION

The SOS effect in CT examination of the chest is similar to that found in contrast examination of the abdomen, involving induced visual neglect.

Surgeons blinded by enhanced navigation: the effect of augmented reality on attention

BACKGROUND

Advanced image-guidance systems allowing presentation of three-dimensional navigational data in real time are being developed enthusiastically for many medical procedures. Other industries, including aviation and the military, have noted that shifting attention toward such compelling assistance has detrimental effects. Using the detection rate of unexpected findings, we assess whether inattentional blindness is significant in a surgical context and evaluate the impact of on-screen navigational cuing with augmented reality.

METHODS

Surgeons and trainees performed an endoscopic navigation exercise on a cadaveric specimen. The subjects were randomized to either a standard endoscopic view (control) or an AR view consisting of an endoscopic video fused with anatomic contours. Two unexpected findings were presented in close proximity to the target point: one critical complication and one foreign body (screw). Task completion time, accuracy, and recognition of findings were recorded.

RESULTS

Detection of the complication was 0/15 in the AR group versus 7/17 in the control group (p = 0.008). Detection of the screw was 1/15 (AR) and 7/17 (control) (p = 0.041). Recognition of either finding was 12/17 for the control group and 1/15 for the AR group (p < 0.001). Accuracy was greater for the AR group than for the control group, with the median distance from the target point measuring respectively 2.10 mm (interquartile range [IQR], 1.29-2.37) and 4.13 (IQR, 3.11-7.39) (p < 0.001).

CONCLUSION

Inattentional blindness was evident in both groups. Although more accurate, the AR group was less likely to identify significant unexpected findings clearly within view. Advanced navigational displays may increase precision, but strategies to mitigate attentional costs need further investigation to allow safe implementation.

Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day

The technology revolution in image acquisition, instrumentation, and methods has resulted in vast data sets that far outstrip the human observers' ability to view, digest, and interpret modern medical images by using traditional methods. This may require a paradigm shift in the radiologic interpretation process. As human observers, radiologists must search for, detect, and interpret targets. Potential interventions should be based on an understanding of human perceptual and attentional abilities and limitations. New technologies and tools already in use in other fields can be adapted to the health care environment to improve medical image analysis, visualization, and navigation through large data sets. This historical psychophysical and technical review touches on a broad range of disciplines but focuses mainly on the analysis, visualization, and navigation of image data performed during the interpretive process. Advanced postprocessing, including three-dimensional image display, multimodality image fusion, quantitative measures, and incorporation of innovative human-machine interfaces, will likely be the future. Successful new paradigms will integrate image and nonimage data, incorporate workflow considerations, and be informed by evidence-based practices. This overview is meant to heighten the awareness of the complexities and limitations of how radiologists interact with images, particularly the large image sets generated today. Also addressed is how human-machine interface and informatics technologies could combine to transform the interpretation process in the future to achieve safer and better quality care for patients and a more efficient and effective work environment for radiologists.

SUPPLEMENTAL MATERIAL

http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.11091276/-/DC1.

Diagnostic errors in pediatric radiology

BACKGROUND

Little information is known about the frequency, types and causes of diagnostic errors in imaging children.

OBJECTIVE

Our goals were to describe the patterns and potential etiologies of diagnostic error in our subspecialty.

MATERIALS AND METHODS

We reviewed 265 cases with clinically significant diagnostic errors identified during a 10-year period. Errors were defined as a diagnosis that was delayed, wrong or missed; they were classified as perceptual, cognitive, system-related or unavoidable; and they were evaluated by imaging modality and level of training of the physician involved.

RESULTS

We identified 484 specific errors in the 265 cases reviewed (mean:1.8 errors/case). Most discrepancies involved staff (45.5%). Two hundred fifty-eight individual cognitive errors were identified in 151 cases (mean = 1.7 errors/case). Of these, 83 cases (55%) had additional perceptual or system-related errors. One hundred sixty-five perceptual errors were identified in 165 cases. Of these, 68 cases (41%) also had cognitive or system-related errors. Fifty-four system-related errors were identified in 46 cases (mean = 1.2 errors/case) of which all were multi-factorial. Seven cases were unavoidable.

CONCLUSION

Our study defines a taxonomy of diagnostic errors in a large academic pediatric radiology practice and suggests that most are multi-factorial in etiology. Further study is needed to define effective strategies for improvement.

Biases in radiologic reasoning

OBJECTIVE

The purpose of this article is to outline common biases in medical reasoning that contribute to avoidable errors in diagnostic and therapeutic decision making.

CONCLUSION

By recognizing and understanding common biases in medical reasoning, we can more effectively counteract them.