Contrast statistics for foveated visual systems: fixation selection by minimizing contrast entropy

Probabilistic modeling of orthographic learning based on visuo-attentional dynamics

How is orthographic knowledge acquired? In line with the self-teaching hypothesis, most computational models assume that phonological recoding has a pivotal role in orthographic learning. However, these models make simplifying assumptions on the mechanisms involved in visuo-orthographic processing. Against evidence from eye movement data during orthographic learning, they assume that orthographic information on novel words is immediately available and accurately encoded after a single exposure. In this paper, we describe BRAID-Learn, a new computational model of orthographic learning. BRAID-Learn is a probabilistic and hierarchical model that incorporates the mechanisms of visual acuity, lateral interference, and visual attention involved in word recognition. Orthographic learning in the model rests on three main mechanisms: first, visual attention moves over the input string to optimize the gain of information on letter identity at each fixation; second, top-down lexical influence is modulated as a function of stimulus familiarity; third, after exploration, perceived information is used to create a new orthographic representation or stabilize a better-specified representation of the input word. BRAID-Learn was challenged on its capacity to simulate the eye movement patterns reported in humans during incidental orthographic learning. In line with the behavioral data, the model predicts a larger decline with exposures in number of fixations and processing time for novel words than for known words. For novel words, most changes occur between the first and second exposure, that is to say, after creation in memory of a new orthographic representation. Beyond phonological recoding, our results suggest that visuo-attentional exploration is an intrinsic portion of orthographic learning seldom taken into consideration by models or theoretical accounts.

Searching in clutter: Visual behavior and performance of expert action video game players

Searching for targets among distractors in visual scenes can be more difficult due to the presence of clutter. However, studies in various domains have shown differentiated effects according to the expertise of the searcher. The present study extended these findings to the domain of action video games expertise. 58 participants, split in 2 groups (action video game players and non-action video game players) searched for targets in visual scenes under two clutter conditions (uncluttered and high clutter). Reaction times and accuracy served as measures of performance, and the visual behavior was assessed using the number and duration of eye fixations. Our findings suggest that visual clutter has a negative influence on performance and alters the visual behavior during visual search in action video game scenes. Our results also suggest that expert action video game players might use different visual strategies to cope with clutter, leading however to no performance benefits.

The nature of correlation perception in scatterplots

For scatterplots with gaussian distributions of dots, the perception of Pearson correlation r can be described by two simple laws: a linear one for discrimination, and a logarithmic one for perceived magnitude (Rensink & Baldridge, 2010). The underlying perceptual mechanisms, however, remain poorly understood. To cast light on these, four different distributions of datapoints were examined. The first had 100 points with equal variance in both dimensions. Consistent with earlier results, just noticeable difference (JND) was a linear function of the distance away from r = 1, and the magnitude of perceived correlation a logarithmic function of this quantity. In addition, these laws were linked, with the intercept of the JND line being the inverse of the bias in perceived magnitude. Three other conditions were also examined: a dot cloud with 25 points, a horizontal compression of the cloud, and a cloud with a uniform distribution of dots. Performance was found to be similar in all conditions. The generality and form of these laws suggest that what underlies correlation perception is not a geometric structure such as the shape of the dot cloud, but the shape of the probability distribution of the dots, likely inferred via a form of ensemble coding. It is suggested that this reflects the ability of observers to perceive the information entropy in an image, with this quantity used as a proxy for Pearson correlation.

Efficient saccade planning requires time and clear choices

We use eye movements constantly to gather information. Saccades are efficient when they maximize the information required for the task, however there is controversy regarding the efficiency of eye movement planning. For example, saccades are efficient when searching for a single target (Nature, 434 (2005) 387-391), but are inefficient when searching for an unknown number of targets in noise, particularly under time pressure (Vision Research 74 (2012), 61-71). In this study, we used a multiple-target search paradigm and explored whether altering the noise level or increasing saccadic latency improved efficiency. Experiments used stimuli with two levels of discriminability such that saccades to the less discriminable stimuli provided more information. When these two noise levels corresponded to low and moderate visibility, most observers did not preferentially select informative locations, but looked at uncertain and probable target locations equally often. We then examined whether eye movements could be made more efficient by increasing the discriminability of the two stimulus levels and by delaying the first saccade so that there was more time for decision processes to influence the saccade choices. Some observers did indeed increase the proportion of their saccades to informative locations under these conditions. Others, however, made as many saccades as they could during the limited time and were unselective about the saccade goal. A clear trend that emerges across all experiments is that conditions with a greater proportion of efficient saccades are associated with a longer latency to initiate saccades, suggesting that the choice of informative locations requires deliberate planning.

Saccadic foraging: reduced reaction time to informative targets

The study of saccadic reaction times has revealed a great deal about the neural mechanisms underlying neural decision, in terms of Bayesian factors such as prior probability and information supply. In addition, recent work has shown that saccades are faster to visual targets associated with conventional monetary or food rewards. However, because the purpose of saccades is to acquire information, it could be argued that this is an unnatural situation: the most natural and fundamental reward is the amount of information supplied by a target. Here, we report the results of a study investigating the hypothesis that a saccade to a target whose colour provides information about the location of a subsequent target is faster than to one that does not. We show that the latencies of saccades to a location that provides reliable information about the location of a future target are indeed shorter, their distributions being shifted in a way that implies that the rate of rise of the underlying decision signal is increased. In a race between alternative targets, this means that expected information will be an important factor in deciding where to look, so that 'foraging' saccades are more likely to be made to useful targets.

Image free-viewing as intrinsically-motivated exploration: estimating the learnability of center-of-gaze image samples in infants and adults

We propose that free viewing of natural images in human infants can be understood and analyzed as the product of intrinsically-motivated visual exploration. We examined this idea by first generating five sets of center-of-gaze (COG) image samples, which were derived by presenting a series of natural images to groups of both real observers (i.e., 9-month-olds and adults) and artificial observers (i.e., an image-saliency model, an image-entropy model, and a random-gaze model). In order to assess the sequential learnability of the COG samples, we paired each group of samples with a simple recurrent network, which was trained to reproduce the corresponding sequence of COG samples. We then asked whether an intrinsically-motivated artificial agent would learn to identify the most successful network. In Simulation 1, the agent was rewarded for selecting the observer group and network with the lowest prediction errors, while in Simulation 2 the agent was rewarded for selecting the observer group and network with the largest rate of improvement. Our prediction was that if visual exploration in infants is intrinsically-motivated-and more specifically, the goal of exploration is to learn to produce sequentially-predictable gaze patterns-then the agent would show a preference for the COG samples produced by the infants over the other four observer groups. The results from both simulations supported our prediction. We conclude by highlighting the implications of our approach for understanding visual development in infants, and discussing how the model can be elaborated and improved.

Impact of the motion and visual complexity of the background on players' performance in video game-like displays

The visual interfaces of virtual environments such as video games often show scenes where objects are superimposed on a moving background. Three experiments were designed to better understand the impact of the complexity and/or overall motion of two types of visual backgrounds often used in video games on the detection and use of superimposed, stationary items. The impact of background complexity and motion was assessed during two typical video game tasks: a relatively complex visual search task and a classic, less demanding shooting task. Background motion impaired participants' performance only when they performed the shooting game task, and only when the simplest of the two backgrounds was used. In contrast, and independently of background motion, performance on both tasks was impaired when the complexity of the background increased. Eye movement recordings demonstrated that most of the findings reflected the impact of low-level features of the two backgrounds on gaze control.

Active search for multiple targets is inefficient

This study examines saccade strategy in a novel task where observers actively search a display to find multiple targets in a limited time. Theory predicts that the relative merit of different saccade strategies depends on the prior probability of the target at a location: when the target prior is low and multiple-target trials are rare, making a saccade to the most likely target location is close to the optimal strategy, but when the target prior is high and multiple-target trials are frequent, selecting uncertain locations is more informative. The prior probability of the target was varied from 0.17 to 0.67 to determine whether observers adjusted their saccades strategies to maximize information. Observers actively searched a noisy display with six potential target locations. Each location had an independent probability of a target, so the number of targets in a trial ranged from 0 to 6. For all target priors ranging from low to high, a trial-by-trial analysis of saccade strategy indicated that observers made saccades to the most likely target location more often than the most uncertain location. Fixating likely locations is efficient only when multiple targets are rare, as in the case of a low target prior, or in the case of the more standard single-target search task. Yet it is the preferred saccade strategy in all our conditions, even when multiple targets are frequent. These findings indicate that humans are far from ideal searchers in multiple-target search.

Reliability of cortical activity during natural stimulation

Response reliability is complementary to more conventional measurements of response amplitudes, and can reveal phenomena that response amplitudes do not. Here we review studies that measured reliability of cortical activity within or between human subjects in response to naturalistic stimulation (e.g. free viewing of movies). Despite the seemingly uncontrolled nature of the task, some of these complex stimuli evoke highly reliable, selective and time-locked activity in many brain areas, including some regions that show little response modulation in most conventional experimental protocols. This activity provides an opportunity to address novel questions concerning natural vision, temporal scale of processing, memory and the neural basis of inter-group differences.

Visuomotor characterization of eye movements in a drawing task

Understanding visuomotor coordination requires the study of tasks that engage mechanisms for the integration of visual and motor information; in this paper we choose a paradigmatic yet little studied example of such a task, namely realistic drawing. On the one hand, our data indicate that the motor task has little influence on which regions of the image are overall most likely to be fixated: salient features are fixated most often. Viceversa, the effect of motor constraints is revealed in the temporal aspect of the scanpaths: (1) subjects direct their gaze to an object mostly when they are acting upon (drawing) it; and (2) in support of graphically continuous hand movements, scanpaths resemble edge-following patterns along image contours. For a better understanding of such properties, a computational model is proposed in the form of a novel kind of Dynamic Bayesian Network, and simulation results are compared with human eye-hand data.

NIMBLE: a kernel density model of saccade-based visual memory

We present a Bayesian version of J. Lacroix, J. Murre, and E. Postma's (2006) Natural Input Memory (NIM) model of saccadic visual memory. Our model, which we call NIMBLE (NIM with Bayesian Likelihood Estimation), uses a cognitively plausible image sampling technique that provides a foveated representation of image patches. We conceive of these memorized image fragments as samples from image class distributions and model the memory of these fragments using kernel density estimation. Using these models, we derive class-conditional probabilities of new image fragments and combine individual fragment probabilities to classify images. Our Bayesian formulation of the model extends easily to handle multi-class problems. We validate our model by demonstrating human levels of performance on a face recognition memory task and high accuracy on multi-category face and object identification. We also use NIMBLE to examine the change in beliefs as more fixations are taken from an image. Using fixation data collected from human subjects, we directly compare the performance of NIMBLE's memory component to human performance, demonstrating that using human fixation locations allows NIMBLE to recognize familiar faces with only a single fixation.

Gaze motion clustering in scan-path estimation

Visual attention is considered nowadays a paramount ability both in Cognitive Sciences and in Cognitive Vision to bridge the gap between perception and higher level reasoning functions, such as scene interpretation and decision making. Bottom-up gaze shifting is the main mechanism used by humans when exploring a scene without a specific task. In this paper we investigated which criteria allow for the generation of plausible fixation clusters by analysing experimental data of human subjects. We suggest that fixations should be grouped in cliques whose saliency can be assessed through an innovation factor encompassing bottom-up cues, proximity, direction and memory components.

MICA: a multilinear ICA decomposition for natural scene modeling

We refine the classical independent component analysis (ICA) decomposition using a multilinear expansion of the probability density function of the source statistics. In particular, we introduce a specific nonlinear system that allows us to elegantly capture the statistical dependences between the responses of the multilinear ICA (MICA) filters. The resulting multilinear probability density is analytically tractable and does not require Monte Carlo simulations to estimate the model parameters. We demonstrate the MICA model on natural image textures and envision that the new model will prove useful for analyzing nonstationarity natural images using natural scene statistics models.

Visual Perception and the Statistical Properties of Natural Scenes

The environments in which we live and the tasks we must perform to survive and reproduce have shaped the design of our perceptual systems through evolution and experience. Therefore, direct measurement of the statistical regularities in natural environments (scenes) has great potential value for advancing our understanding of visual perception. This review begins with a general discussion of the natural scene statistics approach, of the different kinds of statistics that can be measured, and of some existing measurement techniques. This is followed by a summary of the natural scene statistics measured over the past 20 years. Finally, there is a summary of the hypotheses, models, and experiments that have emerged from the analysis of natural scene statistics.

Foveated analysis of image features at fixations

Analysis of the statistics of image features at observers' gaze can provide insights into the mechanisms of fixation selection in humans. Using a foveated analysis framework, in which image patches were analyzed at the resolution corresponding to their eccentricity from the prior fixation, we studied the statistics of four low-level local image features: luminance, RMS contrast, and bandpass outputs of both luminance and contrast, and discovered that the image patches around human fixations had, on average, higher values of each of these features at all eccentricities than the image patches selected at random. Bandpass contrast showed the greatest difference between human and random fixations, followed by bandpass luminance, RMS contrast, and luminance. An eccentricity-based analysis showed that shorter saccades were more likely to land on patches with higher values of these features. Compared to a full-resolution analysis, foveation produced an increased difference between human and random patch ensembles for contrast and its higher-order statistics.

Foveated visual search for corners

We cast the problem of corner detection as a corner search process. We develop principles of foveated visual search and automated fixation selection to accomplish the corner search, supplying a case study of both foveated search and foveated feature detection. The result is a new algorithm for finding corners, which is also a corner-based algorithm for aiming computed foveated visual fixations. In the algorithm, long saccades move the fovea to previously unexplored areas of the image, while short saccades improve the accuracy of putative corner locations. The system is tested on two natural scenes. As an interesting comparison study, we compare fixations generated by the algorithm with those of subjects viewing the same images, whose eye movements are being recorded by an eye tracker. The comparison of fixation patterns is made using an information-theoretic measure. Results show that the algorithm is a good locater of corners, but does not correlate particularly well with human visual fixations.

The long and the short of it: spatial statistics at fixation vary with saccade amplitude and task

We recorded over 90,000 saccades while observers viewed a diverse collection of natural images and measured low level visual features at fixation. The features that discriminated between where observers fixated and where they did not varied considerably with task, and the length of the preceding saccade. Short saccades (<8 degrees) are image feature dependent, long are less so. For free viewing, short saccades target high frequency information, long saccades are scale-invariant. When searching for luminance targets, saccades of all lengths are scale-invariant. We argue that models of saccade behaviour must account not only for task but also for saccade length and that long and short saccades are targeted differently.