Under-exploration of Three-Dimensional Images Leads to Search Errors for Small Salient Targets

Advances in 3D imaging technology are transforming how radiologists search for cancer1,2 and how security officers scrutinize baggage for dangerous objects.3 These new 3D technologies often improve search over 2D images4,5 but vastly increase the image data. Here, we investigate 3D search for targets of various sizes in filtered noise and digital breast phantoms. For a Bayesian ideal observer optimally processing the filtered noise and a convolutional neural network processing the digital breast phantoms, search with 3D image stacks increases target information and improves accuracy over search with 2D images. In contrast, 3D search by humans leads to high miss rates for small targets easily detected in 2D search, but not for larger targets more visible in the visual periphery. Analyses of human eye movements, perceptual judgments, and a computational model with a foveated visual system suggest that human errors can be explained by interaction among a target's peripheral visibility, eye movement under-exploration of the 3D images, and a perceived overestimation of the explored area. Instructing observers to extend the search reduces 75% of the small target misses without increasing false positives. Results with twelve radiologists confirm that even medical professionals reading realistic breast phantoms have high miss rates for small targets in 3D search. Thus, under-exploration represents a fundamental limitation to the efficacy with which humans search in 3D image stacks and miss targets with these prevalent image technologies.

Foveated Model Observers for Visual Search in 3D Medical Images

Model observers have a long history of success in predicting human observer performance in clinically-relevant detection tasks. New 3D image modalities provide more signal information but vastly increase the search space to be scrutinized. Here, we compared standard linear model observers (ideal observers, non-pre-whitening matched filter with eye filter, and various versions of Channelized Hotelling models) to human performance searching in 3D 1/f2.8 filtered noise images and assessed its relationship to the more traditional location known exactly detection tasks and 2D search. We investigated two different signal types that vary in their detectability away from the point of fixation (visual periphery). We show that the influence of 3D search on human performance interacts with the signal's detectability in the visual periphery. Detection performance for signals difficult to detect in the visual periphery deteriorates greatly in 3D search but not in 3D location known exactly and 2D search. Standard model observers do not predict the interaction between 3D search and signal type. A proposed extension of the Channelized Hotelling model (foveated search model) that processes the image with reduced spatial detail away from the point of fixation, explores the image through eye movements, and scrolls across slices can successfully predict the interaction observed in humans and also the types of errors in 3D search. Together, the findings highlight the need for foveated model observers for image quality evaluation with 3D search.

The anger superiority effect revisited: a visual crowding task

Visual search studies have shown that threatening facial expressions are more efficiently detected among a crowd of distractor faces than nonthreatening expressions, known as the anger superiority effect (ASE). However, the opposite finding has also been documented. The present study investigated the ASE in the visual periphery with a visual crowding task. In the study, the target face either appeared alone (uncrowded condition) or was crowded by four neutral or emotional faces (crowded condition). Participants were instructed to determine whether the target face was happy or angry. Experiment 1 showed an ASE when crowded by neutral faces. Intriguingly, this superiority vanished when the target face was crowded by emotional faces that had a different expression from the target as well as when the target face was presented alone. Experiment 2 replicated this result in an independent sample of East Asians (vs. Caucasians in Experiment 1) and thus demonstrated the robustness and cross-cultural consistency of our findings. Together, these results suggest that the ASE in the visual periphery is contingent on task demands induced by visual crowding.

The effects of shape crowding on grasping

Crowding refers to the deleterious effect of nearby objects on the identification of a target in the peripheral visual field. A recent study (Chen, Sperandio, & Goodale, 2015) showed that when a three-dimensional (3D) disk was crowded by disks of different sizes, participants could scale their grip aperture to the size of the target, even when they could not perceive its size. It is still unclear, however, whether or not grasping can also escape to some degree the crowding of other object features, such as shape. To test this, we presented 3D rectangular blocks in isolation or crowded by other blocks in the periphery. The target and flanking blocks had the same surface area but different dimensions. Participants were required either to grasp the target block across its width or to estimate its width. We found that, consistent with what we observed earlier with size, participants can also scale their grasp to the width of the target block even when they could not perceive its width. To further explore whether or not the effect of crowding on grasping depends on how proficient people are with their right hand, we had right-handed participants perform the same test but with their left hand. We found that left-hand grasping did not escape the crowding effect on shape perception at all. Taken together, our results suggest that people can also use invisible shape information to guide actions and that this ability depends on the proficiency of the action.

Feature- and Face-Exchange illusions: new insights and applications for the study of the binding problem

The binding problem is a longstanding issue in vision science: i.e., how are humans able to maintain a relatively stable representation of objects and features even though the visual system processes many aspects of the world separately and in parallel? We previously investigated this issue with a variant of the bounce-pass paradigm, which consists of two rectangular bars moving in opposite directions; if the bars are identical and never overlap, the motion could equally be interpreted as bouncing or passing. Although bars of different colors should be seen as passing each other (since the colors provide more information about the bars' paths), we found “Feature Exchange”: observers reported the paradoxical perception that the bars appear to bounce off of each other and exchange colors. Here we extend our previous findings with three demonstrations. “Peripheral Feature-Exchange” consists of two colored bars that physically bounce (they continually meet in the middle of the monitor and return to the sides). When viewed in the periphery, the bars appear to stream past each other even though this percept relies on the exchange of features and contradicts the information provided by the color of the bars. In “Face-Exchange” two different faces physically pass each other. When fixating centrally, observers typically report the perception of bouncing faces that swap features, indicating that the Feature Exchange effect can occur even with complex objects. In “Face-Go-Round,” one face repeatedly moves from left to right on the top of the monitor, and the other from right to left at the bottom of the monitor. Observers typically perceive the faces moving in a circle—a percept that contradicts information provided by the identity of the faces. We suggest that Feature Exchange and the paradigms used to elicit it can be useful for the investigation of the binding problem as well as other contemporary issues of interest to vision science.