An ideal observer with channels versus feature-independent processing of spatial frequency and orientation in visual search performance

Surround masking reveals binocular adding and differencing channels

Recent studies suggest that binocular adding S+ and differencing S- channels play an important role in binocular vision. To test for such a role in the context of binocular contrast detection and binocular summation, we employed a surround masking paradigm consisting of a central target disk surrounded by a mask annulus. All stimuli were horizontally oriented 0.5c/d sinusoidal gratings. Correlated stimuli were identical in interocular spatial phase while anticorrelated stimuli were opposite in interocular spatial phase. There were four target conditions: monocular left eye, monocular right eye, binocular correlated and binocular anticorrelated, and three surround mask conditions: no surround, binocularly correlated and binocularly anticorrelated. We observed consistent elevation of detection thresholds for monocular and binocular targets across the two binocular surround mask conditions. In addition, we found an interaction between the type of surround and the type of binocular target: both detection and summation were relatively enhanced by surround masks and targets with opposite interocular phase relationships and reduced by surround masks and targets with the same interocular phase relationships. The data were reasonably well accounted for by a model of binocular combination termed MAX (S+S-), in which the decision variable is the probability summation of modeled S+ and S- channel responses, with a free parameter determining the relative gains of the two channels. Our results support the existence of two channels involved in binocular combination, S+ and S-, whose relative gains are adjustable by surround context.

Under-exploration of Three-Dimensional Images Leads to Search Errors for Small Salient Targets

Advances in 3D imaging technology are transforming how radiologists search for cancer1,2 and how security officers scrutinize baggage for dangerous objects.3 These new 3D technologies often improve search over 2D images4,5 but vastly increase the image data. Here, we investigate 3D search for targets of various sizes in filtered noise and digital breast phantoms. For a Bayesian ideal observer optimally processing the filtered noise and a convolutional neural network processing the digital breast phantoms, search with 3D image stacks increases target information and improves accuracy over search with 2D images. In contrast, 3D search by humans leads to high miss rates for small targets easily detected in 2D search, but not for larger targets more visible in the visual periphery. Analyses of human eye movements, perceptual judgments, and a computational model with a foveated visual system suggest that human errors can be explained by interaction among a target's peripheral visibility, eye movement under-exploration of the 3D images, and a perceived overestimation of the explored area. Instructing observers to extend the search reduces 75% of the small target misses without increasing false positives. Results with twelve radiologists confirm that even medical professionals reading realistic breast phantoms have high miss rates for small targets in 3D search. Thus, under-exploration represents a fundamental limitation to the efficacy with which humans search in 3D image stacks and miss targets with these prevalent image technologies.

Binocular summation and efficient coding

Two eyes are better than one at detecting a pattern, an advantage termed binocular summation. It is widely believed that binocular summation is mediated by neurons that sum the two eyes' inputs. Here we suggest an alternative model based on a model of binocular interactions proposed by Cohn, Leong & Lasley (Vision Research, 1981, 21, 1017-1023) and further motivated by the efficient coding framework proposed by Li & Atick (Network: Computation in Neural Systems, 1994, 5, 157-174). In the model, termed MAX(S+S-), binocular summation is mediated by channels that compute the sum, S+, and difference, S-, of the two eyes' monocular signals. The S+ and S- signals are assumed to be perturbed by independent noise, have independent gains and contribute independently to detection via the MAX rule. To test the model we measured binocular summation for horizontally-oriented Gabor patches at a range of spatial-frequencies and bandwidths, at both contrast detection threshold and for increment thresholds on binocular pedestals at contrasts set to 10x detection threshold. The model's performance was compared to that of two conventional models of binocular summation, one in which the two eyes' signals remain separate at the decision stage, termed MAX(LR), the other in which the two eye's signals are summed by a single channel, termed B+, with both models incorporating interocular inhibition. The MAX(S+S-) model gave as good a performance as the other two models. Together with the evidence for the involvement of separately gain controlled S+ and S- signals underpinning a wide range of binocular behaviors, we conclude that the MAX(S+S-) model can and should be considered as a viable model for binocular summation.

Modeling of decision-making process for moving straight using inverse Bayesian inference

Humans sometimes make unreasonable decisions when viewed in objective terms. Even in the real world, we may lose sense of direction by turning around the corner several times or mistaking the estimation of travel distance. We experimented in virtual space how we lose sense of direction under what circumstances. In the experiment, subjects viewed a three-dimensional space displayed on a computer display in the first person's perspective and were instructed to go straight from the start to the goal position. Results showed that unreasonable selections that strayed from the centerline connecting the start and goal positions were frequently made. The change in the direction is more influential than the change in the distance, and the angle of turning also affects. Furthermore, the subject's decision - making process was modeled using Bayesian inference and inverse Bayesian inference. Comparing the two models, when the decision-making pattern suddenly changed, the model by inverse Bayesian inference could follow up faster than the Bayesian inference.

Statistical decision theory to relate neurons to behavior in the study of covert visual attention

Scrutiny of the numerous physiology and imaging studies of visual attention reveal that integration of results from neuroscience with the classic theories of visual attention based on behavioral work is not simple. The different subfields have pursued different questions, used distinct experimental paradigms and developed diverse models. The purpose of this review is to use statistical decision theory and computational modeling to relate classic theories of attention in psychological research to neural observables such as mean firing rate or functional imaging BOLD response, tuning functions, Fano factor, neuronal index of detectability and area under the receiver operating characteristic (ROC). We focus on cueing experiments and attempt to distinguish two major leading theories in the study of attention: limited resources model/increased sensitivity vs. selection/differential weighting. We use Bayesian ideal observer (BIO) modeling, in which predictive cues or prior knowledge change the differential weighting (prior) of sensory information to generate predictions of behavioral and neural observables based on Gaussian response variables and Poisson process neural based models. The ideal observer model can be modified to represent a number of classic psychological theories of visual attention by including hypothesized human attentional limited resources in the same way sequential ideal observer analysis has been used to include physiological processing components of human spatial vision (Geisler, W. S. (1989). Sequential ideal-observer analysis of visual discrimination. Psychological Review 96, 267-314.). In particular we compare new biologically plausible implementations of the BIO and variant models with limited resources. We find a close relationship between the behavioral effects of cues predicted by the models developed in the field of human psychophysics and their neuron-based analogs. Critically, we show that cue effects on experimental observables such as mean neural activity, variance, Fano factor and neuronal index of detectability can be consistent with the two major theoretical models of attention depending on whether the neuron is assumed to be computing likelihoods, log-likelihoods or a simple model operating directly on the Poisson variable. Change in neuronal tuning functions can also be consistent with both theories depending on whether the change in tuning is along the dimension being experimentally cued or a different dimension. We show that a neuron's sensitivity appropriately measured using the area under the Receive Operating Characteristic curve can be used to distinguish across both theories and is robust to the many transformations of the decision variable. We provide a summary table with the hope that it might provide some guidance in interpreting past results as well as planning future studies.

Detection of multidimensional targets in visual search

Search performance for targets defined along multiple dimensions was investigated with an accuracy visual search task. Initially, threshold was measured for targets that differed from homogeneous distractors along a single dimension (e.g., a reddish target among achromatic distractors, or a right-tilted target among vertically oriented distractors). Threshold was then measured for a multidimensional target (a redundant target) that differed from homogeneous distractors along two dimensions (e.g., a reddish AND right-tilted target among achromatic, vertically oriented distractors). Search performance for multidimensional target combinations of chromaticity and luminance, chromaticity and orientation, and chromaticity and spatial frequency was tested. Measurements were evaluated within several summation models, allowing for a test of the mechanisms mediating the detection of multidimensional targets in search. Measurements were generally consistent with probability summation suggesting the particular combinations of stimulus dimensions tested were coded along independent, noisy, neural mechanisms.

The Bayesian brain: the role of uncertainty in neural coding and computation

To use sensory information efficiently to make judgments and guide action in the world, the brain must represent and use information about uncertainty in its computations for perception and action. Bayesian methods have proven successful in building computational theories for perception and sensorimotor control, and psychophysics is providing a growing body of evidence that human perceptual computations are "Bayes' optimal". This leads to the "Bayesian coding hypothesis": that the brain represents sensory information probabilistically, in the form of probability distributions. Several computational schemes have recently been proposed for how this might be achieved in populations of neurons. Neurophysiological data on the hypothesis, however, is almost non-existent. A major challenge for neuroscientists is to test these ideas experimentally, and so determine whether and how neurons code information about sensory uncertainty.