Eye scanning of multi-element displays: I. Scanpath planning

Inhibition of saccade initiation improves saccade accuracy: The role of local and remote visual distractors in the control of saccadic eye movements

When a distractor appears close to the target location, saccades are less accurate. However, the presence of a further distractor, remote from those stimuli, increases the saccade response latency and improves accuracy. Explanations for this are either that the second, remote distractor impacts directly on target selection processes or that the remote distractor merely impairs the ability to initiate a saccade and changes the time at which unaffected target selection processes are accessed. In order to tease these two explanations apart, here we examine the relationship between latency and accuracy of saccades to a target and close distractor pair while a remote distractor appears at variable distance. Accuracy improvements are found to follow a similar pattern, regardless of the presence of the remote distractor, which suggests that the effect of the remote distractor is not the result of a direct impact on the target selection process. Our findings support the proposal that a remote distractor impairs the ability to initiate a saccade, meaning the competition between target and close distractor is accessed at a later time, thus resulting in more accurate saccades.

Scan path during change-detection visual search

When observing a particular image or object, one's perception depends upon prior expectations, memory, and cognitive abilities. It is hypothesized that cognitive processing in the form of top-down or bottom-up processing could be determined via analysis of the eye fixation scan path. To assess the variations in scan paths, 7 subjects underwent 5 change-detection trials. During each trial, they were presented with a specific set of images via a MATLAB program, in which the original image alternated with a modified image consisting of a single change. An open-source program called GazeRecorder was used to track the subject's eye movements and to record the eye fixations. The scan path was then analyzed through the use of a 4 by 4 grid pattern superimposed on the image to determine the subject's eye fixation distribution pattern in terms of Boxes Viewed and Concentration within a single area. It was determined that higher Concentration was positively correlated with faster Detection Speed (R = 0.84), while higher number of Boxes Viewed was negatively correlated with Detection Speed (R = -0.71). Among the subjects, the more optimal scan paths were found in those with a balance between Concentration and Boxes Viewed, as subjects with a more balanced approach had the greatest Accuracy (p = 0.02). This indicates an optimal scan path involves both top-down and bottom-up processing to more efficiently identify a change. Moreover, the methodology developed in this study could be used in the home or clinic for quantitative assessment of improvement following therapy in patients with neurological deficits.

Gain control of saccadic eye movements is probabilistic

Saccades are rapid eye movements that orient the visual axis toward objects of interest to allow their processing by the central, high-acuity retina. Our ability to collect visual information efficiently relies on saccadic accuracy, which is limited by a combination of uncertainty in the location of the target and motor noise. It has been observed that saccades have a systematic tendency to fall short of their intended targets, and it has been suggested that this bias originates from a cost function that overly penalizes hypermetric errors. Here, we tested this hypothesis by systematically manipulating the positional uncertainty of saccadic targets. We found that increasing uncertainty produced not only a larger spread of the saccadic endpoints but also more hypometric errors and a systematic bias toward the average of target locations in a given block, revealing that prior knowledge was integrated into saccadic planning. Moreover, by examining how variability and bias covaried across conditions, we estimated the asymmetry of the cost function and found that it was related to individual differences in the additional time needed to program secondary saccades for correcting hypermetric errors, relative to hypometric ones. Taken together, these findings reveal that the saccadic system uses a probabilistic-Bayesian control strategy to compensate for uncertainty in a statistically principled way and to minimize the expected cost of saccadic errors.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Refixation patterns reveal memory-encoding strategies in free viewing

We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation.

Dual Target Search is Neither Purely Simultaneous nor Purely Successive

Previous research shows that visual search for two different targets is less efficient than search for a single target. Stroud, Menneer, Cave and Donnelly (2012) concluded that two target colours are represented separately based on modeling the fixation patterns. Although those analyses provide evidence for two separate target representations, they do not show whether participants search simultaneously for both targets, or first search for one target and then the other. Some studies suggest that multiple target representations are simultaneously active, while others indicate that search can be voluntarily simultaneous, or switching, or a mixture of both. Stroud et al.'s participants were not explicitly instructed to use any particular strategy. These data were revisited to determine which strategy was employed. Each fixated item was categorised according to whether its colour was more similar to one target or the other. Once an item similar to one target is fixated, the next fixated item is more likely to be similar to that target than the other, showing that at a given moment during search, one target is generally favoured. However, the search for one target is not completed before search for the other begins. Instead, there are often short runs of one or two fixations to distractors similar to one target, with each run followed by a switch to the other target. Thus, the results suggest that one target is more highly weighted than the other at any given time, but not to the extent that search is purely successive.

Spatial ranking strategy and enhanced peripheral vision discrimination optimize performance and efficiency of visual sequential search

Visual sequential search might use a peripheral spatial ranking of the scene to put the next target of the sequence in the correct order. This strategy, indeed, might enhance the discriminative capacity of the human peripheral vision and spare neural resources associated with foveation. However, it is not known how exactly the peripheral vision sustains sequential search and whether the sparing of neural resources has a cost in terms of performance. To elucidate these issues, we compared strategy and performance during an alpha-numeric sequential task where peripheral vision was modulated in three different conditions: normal, blurred, or obscured. If spatial ranking is applied to increase the peripheral discrimination, its use as a strategy in visual sequencing should differ according to the degree of discriminative information that can be obtained from the periphery. Moreover, if this strategy spares neural resources without impairing the performance, its use should be associated with better performance. We found that spatial ranking was applied when peripheral vision was fully available, reducing the number and time of explorative fixations. When the periphery was obscured, explorative fixations were numerous and sparse; when the periphery was blurred, explorative fixations were longer and often located close to the items. Performance was significantly improved by this strategy. Our results demonstrated that spatial ranking is an efficient strategy adopted by the brain in visual sequencing to highlight peripheral detection and discrimination; it reduces the neural cost by avoiding unnecessary foveations, and promotes sequential search by facilitating the onset of a new saccade.

Spatial Position Influences Perception of Slope from Graphs

We routinely examine linear trends from bar graphs and scatterplots while taking a science class, attending a business presentation, or reading a magazine article. Graphs are placed in different positions on a page or a presentation slide for aesthetic considerations. However, because left and right positions tend to be associated with lower and higher values in the conventional depiction of numerical values, we hypothesized that the perception of positive and negative slopes may be influenced by the placement of a graph. Using a visual search task, with each display containing four bar graphs or scatterplots (one per quadrant), we have demonstrated that the detection of a negative slope is selectively slowed in the upper-right quadrant (for both bar graphs and scatterplots), whereas the detection of a positive slope is selectively slowed in the upper-left quadrant (for bar graphs only). These results suggest that an upper-right position is incompatible with perceiving negative slopes and an upper-left position is incompatible with perceiving positive slopes. Although the origin of these specific associations is unclear, our results have implications for where to place a graph depending on the slope it displays.

The spatial impact of visual distractors on saccade latency

Remote transient changes in the environment, such as the onset of visual distractors, impact on the execution of target directed saccadic eye movements. Studies that have examined the latency of the saccade response have shown conflicting results. When there was an element of target selection, saccade latency increased as the distance between distractor and target increased. In contrast, when target selection is minimized by restricting the target to appear on one axis position, latency has been found to be slowest when the distractor is shown at fixation and reduces as it moves away from this position, rather than from the target. Here we report four experiments examining saccade latency as target and distractor positions are varied. We find support for both a dependence of saccade latency on distractor distance from target and from fixation: saccade latency was longer when distractor is shown close to fixation and even longer still when shown in an opposite location (180°) to the target. We suggest that this is due to inhibitory interactions between the distractor, fixation and the target interfering with fixation disengagement and target selection.

Eye movements: the past 25 years

This article reviews the past 25 years of research on eye movements (1986-2011). Emphasis is on three oculomotor behaviors: gaze control, smooth pursuit and saccades, and on their interactions with vision. Focus over the past 25 years has remained on the fundamental and classical questions: What are the mechanisms that keep gaze stable with either stationary or moving targets? How does the motion of the image on the retina affect vision? Where do we look - and why - when performing a complex task? How can the world appear clear and stable despite continual movements of the eyes? The past 25 years of investigation of these questions has seen progress and transformations at all levels due to new approaches (behavioral, neural and theoretical) aimed at studying how eye movements cope with real-world visual and cognitive demands. The work has led to a better understanding of how prediction, learning and attention work with sensory signals to contribute to the effective operation of eye movements in visually rich environments.

Predictive activity in macaque frontal eye field neurons during natural scene searching

Generating sequences of multiple saccadic eye movements allows us to search our environment quickly and efficiently. Although the frontal eye field cortex (FEF) has been linked to target selection and making saccades, little is known about its role in the control and performance of the sequences of saccades made during self-guided visual search. We recorded from FEF cells while monkeys searched for a target embedded in natural scenes and examined the degree to which cells with visual and visuo-movement activity showed evidence of target selection for future saccades. We found that for about half of these cells, activity during the fixation period between saccades predicted the next saccade in a sequence at an early time that precluded selection based on current visual input to a cell's response field. In addition to predicting the next saccade, activity during the fixation prior to two successive saccades also predicted the direction and goal of the second saccade in the sequence. We refer to this as advanced predictive activity. Unlike activity indicating the upcoming saccade, advanced predictive activity occurred later in the fixation period, mirroring the order of the saccade sequence itself. The remaining cells without advanced predictive activity did not predict future saccades but reintroduced the signal for the upcoming saccade at an intermediate time in the fixation period. Together these findings suggest that during natural visual search the timing of FEF cell activity is consistent with a role in specifying targets for one or more future saccades in a search sequence.

Structural and semantic constraints on the resolution of pronouns and reflexives

We present four experiments on the interpretation of pronouns and reflexives in picture noun phrases with and without possessors (e.g. Andrew's picture of him/himself, the picture of him/himself). The experiments (two off-line studies and two visual-world eye-tracking experiments) investigate how syntactic and semantic factors guide the interpretation of pronouns and reflexives and how different kinds of information are integrated during real-time reference resolution. The results show that the interpretation of pronouns and reflexives in picture NP constructions is sensitive not only to purely structural information, as is commonly assumed in syntactically-oriented theories of anaphor resolution, but also to semantic information (see Kuno, S. (1987). Functional syntax: Anaphora, discourse and empathy. Chicago: University of Chicago Press; Tenny, C. (2003). Short distance pronouns in representational noun phrases and a grammar of sentience. Ms.). Moreover, the results show that pronouns and reflexives differ in the degree of sensitivity they exhibit to different kinds of information. This finding indicates that the form-specific multiple-constraints approach (see Brown-Schmidt, S., Byron, D. K., & Tanenhaus, M. K. (2005). Beyond salience: Interpretation of personal and demonstrative pronouns. Journal of Memory and Language, 53, 292-313; Kaiser, E. (2003). The quest for a referent: A crosslinguistic look at reference resolution. Ph.D. dissertation. University of Pennsylvania, Philadelphia, PA; Kaiser, E. (2005). When salience is not enough: Pronouns, demonstratives and the quest for an antecedent. In: Laury, R. (Ed.), Minimal reference in Finnic: The use and interpretation of pronouns and zero in Finnish and Estonian discourse (pp. 135-162). Helsinki, Finland: Suomalaisen Kirjallisuuden Seura; Kaiser, E., & Trueswell, J. (2008). Interpreting pronouns and demonstratives in Finnish: Evidence for a form-specific approach to reference resolution. Language and Cognitive Processes, 23(5), 709-748), which states that referential forms can exhibit asymmetrical sensitivities to the different constraints guiding reference resolution, also applies in the within-sentence domain.

The mechanism underlying inhibition of saccadic return

Human observers take longer to re-direct gaze to a previously fixated location. Although there has been some exploration of the characteristics of inhibition of saccadic return (ISR), the exact mechanisms by which ISR operates are currently unknown. In the framework of accumulation models of response times, in which evidence is integrated over time to a response threshold, ISR could reflect a reduction in the rate of accumulation for saccades to return locations or an increase in the effective criterion for response. In two experiments, participants generated sequences of three saccades, in response to a peripheral or a central cue. ISR occurred across these manipulations: saccade latency was consistently increased for movements to the immediately previously fixated location. Latency distributions from individual observers were fit with a Linear Ballistic Accumulator model. ISR was best accounted for as a change in the accumulation rate. We suggest this parameter represents the overall desirability of a particular course of action, the evidence for which may be derived from a variety of sensory and non-sensory sources.

A theory of eye movements during target acquisition

The gaze movements accompanying target localization were examined via human observers and a computational model (target acquisition model [TAM]). Search contexts ranged from fully realistic scenes to toys in a crib to Os and Qs, and manipulations included set size, target eccentricity, and target-distractor similarity. Observers and the model always previewed the same targets and searched identical displays. Behavioral and simulated eye movements were analyzed for acquisition accuracy, efficiency, and target guidance. TAM's behavior generally fell within the behavioral mean's 95% confidence interval for all measures in each experiment/condition. This agreement suggests that a fixed-parameter model using spatiochromatic filters and a simulated retina, when driven by the correct visual routines, can be a good general-purpose predictor of human target acquisition behavior.

Gaze motion clustering in scan-path estimation

Visual attention is considered nowadays a paramount ability both in Cognitive Sciences and in Cognitive Vision to bridge the gap between perception and higher level reasoning functions, such as scene interpretation and decision making. Bottom-up gaze shifting is the main mechanism used by humans when exploring a scene without a specific task. In this paper we investigated which criteria allow for the generation of plausible fixation clusters by analysing experimental data of human subjects. We suggest that fixations should be grouped in cliques whose saliency can be assessed through an innovation factor encompassing bottom-up cues, proximity, direction and memory components.

Training Eye Movements: Can Training People Where to Look Hinder the Processing of Fixated Objects?

An experiment designed to test the effects of different forms of training in a visual-search-like task is reported. Observers were presented with a series of displays containing a central letter and a ring of peripheral characters, one of which was a digit. Odd digits (catch trials) required a space-bar response; even digits required a different response contingent on the identity of the central letter. Two forms of training provided information either about the location of the peripheral digit, or about a quick way to classify the central letter. The aim was to relate training to Findlay and Walker's (1999, Behavioral and Brain Sciences22 661–721) model of saccadic eye-movement control by affecting the hypothesised Move and Fixate centres respectively. The results showed that training benefited search, but training of the Move centre alone generated significantly longer re-inspections of the central region (in a feedback condition). This highlights that the emphasis often placed upon broadening the range of visual search when training eye movements may be misplaced. More specifically, special attention should be given, not only to advising people how to move their eyes, but also to improving the ability to effectively process important visual stimuli when fixating.

Memory for the search path: evidence for a high-capacity representation of search history

Using a gaze-contingent paradigm, we directly measured observers' memory capacity for fixated distractor locations during search. After approximately half of the search objects had been fixated, they were masked and a spatial probe appeared at either a previously fixated location or a non-fixated location; observers then rated their confidence that the target had appeared at the probed location. Observers were able to differentiate the 12 most recently fixated distractor locations from non-fixated locations, but analyses revealed that these locations were represented fairly coarsely. We conclude that there exists a high-capacity, but low-resolution, memory for a search path.

Saccades and covert shifts of attention during active visual search: spatial distributions, memory, and items per fixation

Target detection during active visual search was examined. The chance corrected spatial distribution of target detection was found to be symmetrically distributed around the point of fixation and, unexpectedly, was independent of the proximity of fixations to the display boundaries. Memory was found to play a very limited role in target detection, but a significant role in the guidance of eye movements. A model of covert shifts was used to estimate the number and spatial distribution of shifts required to explain observed performance. An increase from one to five shifts per fixation across increasing array set size as estimated by two different methods was inconsistent with unchanging fixation durations, suggesting that multiple covert shifts are not occurring during the fixations in active search.