Changes in the functional visual field during search with and without eye movements

What is the basis of ensemble subset selection?

The visual system can rapidly calculate the ensemble statistics of a set of objects; for example, people can easily estimate an average size of apples on a tree. To accomplish this, it is not always useful to summarize all the visual information. If there are various types of objects, the visual system should select a relevant subset: only apples, not leaves and branches. Here, we ask what kind of visual information makes a "good" ensemble that can be selectively attended to provide an accurate summary estimate. We tested three candidate representations: basic features, preattentive object files, and full-fledged bound objects. In four experiments, we presented a target and several distractors' sets of differently colored objects. We found that conditions where a target ensemble had at least one unique color (basic feature) provided ensemble averaging performance comparable to the baseline displays without distractors. When the target subset was defined as a conjunction of two colors or color-shape partly shared with distractors (so that they could be differentiated only as preattentive object files), subset averaging was also possible but less accurate than in the baseline and feature conditions. Finally, performance was very poor when the target subset was defined by an exact feature relationship, such as in the spatial conjunction of two colors (spatially bound object). Overall, these results suggest that distinguishable features and, to a lesser degree, preattentive object files can serve as the representational basis of ensemble selection, while bound objects cannot.

Reduced spatial attentional distribution in older adults

Older adults show decline in visual search performance, but the underlying cause remains unclear. It has been suggested that older adults' altered performance may be related to reduced spatial attention to peripheral visual information compared with younger adults. In this study, 18 younger (M = 21.6 years) and 16 older (M = 69.1 years) participants performed pop-out and serial visual search tasks with variously sized gaze-contingent artificial central scotomas (3°, 5°, or 7° diameter). By occluding central vision, we measured how attention to the periphery was contributing to the search performance. We also tested the effect of target eccentricity on search times and eye movements. We hypothesized that, if attention is reduced primarily in the periphery in older adults, we would observe longer search times for more eccentric targets and with central occlusion. During the pop-out search, older adults showed a steeper decline in search performance with increasing eccentricity and central scotoma size compared with younger adults. In contrast, during the serial search, older adults had longer search times than younger adults overall, independent of target eccentricity and scotoma size. Longer search times were attributed to higher cost-per-item slopes, indicating increased difficulty in simultaneously processing complex symbols made up of separable features in aging, possibly stemming from challenges in spatially binding individual features. Altogether, our findings point to fewer attentional resources of simultaneous visual processing to distribute over space or separable features of objects, consistent with decreased dorsal visual stream functioning in aging.

Does crowding predict conjunction search? An individual differences approach

Searching for objects in the visual environment is an integral part of human behavior. Most of the information used during such visual search comes from the periphery of our vision, and understanding the basic mechanisms of search therefore requires taking into account the inherent limitations of peripheral vision. Our previous work using an individual differences approach has shown that one of the major factors limiting peripheral vision (crowding) is predictive of single feature search, as reflected in response time and eye movement measures. Here we extended this work, by testing the relationship between crowding and visual search in a conjunction-search paradigm. Given that conjunction search involves more fine-grained discrimination and more serial behavior, we predicted it would be strongly affected by crowding. We tested sixty participants with regard to their sensitivity to both orientation and color-based crowding (as measured by critical spacing) and their efficiency in searching for a color/orientation conjunction (as indicated by manual response times and eye movements). While the correlations between the different crowding tasks were high, the correlations between the different crowding measures and search performance were relatively modest, and no higher than those previously observed for single-feature search. Instead, observers showed very strong color selectivity during search. The results suggest that conjunction search behavior relies more on top-down guidance (here by color) and is therefore relatively less determined by individual differences in sensory limitations as caused by crowding.

The time course of visual foraging in the lifespan: Spatial scanning, organization search, and target processing

Visual foraging is a variant of visual search, consisting of searching for an undetermined number of targets among distractors (e.g., looking for various LEGO pieces in a box). Under non-exhaustive tasks, the observer scans the display, picking those targets needed, not necessarily all of them, before leaving the search. To understand how the organization of such natural foraging tasks works, several measures of spatial scanning and organization have been proposed in the exhaustive foraging literature: best-r, intertarget distances, PAO, and target intersections. In the present study, we apply these measures and new Bayesian indexes to determine how the time course of visual foraging is organized in a dynamic non-exhaustive paradigm. In a large sample of observers (279 participants, 4-25 years old), we compare feature and conjunction foraging and explore how factors like set size and time course, not previously tested in exhaustive foraging, might affect search organization in non-exhaustive dynamic tasks. The results replicate previous findings showing younger observers' searching being less organized, feature conditions being more organized than conjunction conditions, and organization leading to a more effective search. Interestingly, observers tend to be less organized as set size increases, and search is less organized within a patch as it advances in time: Search organization decreases when search termination is coming, suggesting organization measures as potential clues to understand quitting rules in search. Our results highlight the importance of studying search organization in foraging as a critical source of understanding complex cognitive processes in visual search.

Effects of overnight military training and acute battle stress on the cognitive performance of soldiers in simulated urban combat

Understanding the effect of stress, fatigue, and sleep deprivation on the ability to maintain an alert and attentive state in an ecologically valid setting is of importance as lapsing attention can, in many safety-critical professions, have devastating consequences. Here we studied the effect of close-quarters battle (CQ battle) exercise combined with overnight military training with sleep deprivation on cognitive performance, namely sustained attention and response inhibition. In addition, the effect of the CQ battle and overnight training on cardiac activity [heart rate and root mean square of the successive differences (RMSSD)] during the cognitive testing and the relationship between cardiac activity and cognitive performance were examined. Cognitive performance was measured with the psychomotor vigilance task (PVT) and the sustained attention to response task (SART). Altogether 45 conscripts participated in the study. The conscripts were divided into control (CON) and experimental (EXP) groups. The CON completed the training day after a night of sleep and the EXP after the overnight military training with no sleep. Results showed that the effect of the overnight training on cognitive performance and the between-group difference in heart rate (HR) and heart rate variability (HRV) depended on the cognitive test. Surprisingly, the cognitive performance was not largely affected by the CQ battle. However, as expected, the CQ battle resulted in a significant decrease in RMSSD and an increase in HR measured during the cognitive testing. Similarly, the HR parameters were related to cognitive performance, but the relationship was found only with the PVT. In conclusion, fatigue due to the overnight training impaired the ability to maintain sufficient alertness level. However, this impairment in arousal upregulation was counteracted by the arousing nature of the SART. Hence, the conscripts' cognitive performance was mainly preserved when performing a stimulating task, despite the fatigue from the sleep loss of the preceding night and physical activity.

Peripheral vision in real-world tasks: A systematic review

Peripheral vision is fundamental for many real-world tasks, including walking, driving, and aviation. Nonetheless, there has been no effort to connect these applied literatures to research in peripheral vision in basic vision science or sports science. To close this gap, we analyzed 60 relevant papers, chosen according to objective criteria. Applied research, with its real-world time constraints, complex stimuli, and performance measures, reveals new functions of peripheral vision. Peripheral vision is used to monitor the environment (e.g., road edges, traffic signs, or malfunctioning lights), in ways that differ from basic research. Applied research uncovers new actions that one can perform solely with peripheral vision (e.g., steering a car, climbing stairs). An important use of peripheral vision is that it helps compare the position of one’s body/vehicle to objects in the world. In addition, many real-world tasks require multitasking, and the fact that peripheral vision provides degraded but useful information means that tradeoffs are common in deciding whether to use peripheral vision or move one’s eyes. These tradeoffs are strongly influenced by factors like expertise, age, distraction, emotional state, task importance, and what the observer already knows. These tradeoffs make it hard to infer from eye movements alone what information is gathered from peripheral vision and what tasks we can do without it. Finally, we recommend three ways in which basic, sport, and applied science can benefit each other’s methodology, furthering our understanding of peripheral vision more generally.

The Functional Visual Field(s) in simple visual search

During a visual search for a target among distractors, observers do not fixate every location in the search array. Rather processing is thought to occur within a Functional Visual Field (FVF) surrounding each fixation. We argue that there are three questions that can be asked at each fixation and that these imply three different senses of the FVF. 1) Can I identify what is at location XY? This defines a resolution FVF. 2) To what shall I attend during this fixation? This defines an Attentional FVF. 3) Where should I fixate next? This defines an Exploratory FVF. We examine FVFs 2&3 using eye movements in visual search. In three Experiments, we collected eye movements during visual search for the target letter T among distractor letter Ls (Exps 1 and 3) or for a color X orientation conjunction (Exp 2). Saccades that do not go to the target can be used to define the Exploratory FVF. The saccade that goes to the target can be used to define the Attentional FVF since the target was probably covertly detected during the prior fixation. The Exploratory FVF is larger than the Attentional FVF for all three experiments. Interestingly, the probability that the next saccade would go to the target was always well below 1.0, even when the current fixation was close to the target and well within any reasonable estimate of the FVF. Measuring search-based Exploratory and Attentional FVFs sheds light on how we can miss clearly visible targets.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

Investigating Visual Crowding of Objects in Complex Real-World Scenes

Visual crowding, the impairment of object recognition in peripheral vision due to flanking objects, has generally been studied using simple stimuli on blank backgrounds. While crowding is widely assumed to occur in natural scenes, it has not been shown rigorously yet. Given that scene contexts can facilitate object recognition, crowding effects may be dampened in real-world scenes. Therefore, this study investigated crowding using objects in computer-generated real-world scenes. In two experiments, target objects were presented with four flanker objects placed uniformly around the target. Previous research indicates that crowding occurs when the distance between the target and flanker is approximately less than half the retinal eccentricity of the target. In each image, the spacing between the target and flanker objects was varied considerably above or below the standard (0.5) threshold to either suppress or facilitate the crowding effect. Experiment 1 cued the target location and then briefly flashed the scene image before participants could move their eyes. Participants then selected the target object’s category from a 15-alternative forced choice response set (including all objects shown in the scene). Experiment 2 used eye tracking to ensure participants were centrally fixating at the beginning of each trial and showed the image for the duration of the participant’s fixation. Both experiments found object recognition accuracy decreased with smaller spacing between targets and flanker objects. Thus, this study rigorously shows crowding of objects in semantically consistent real-world scenes.

Influence of scene structure and content on visual search strategies

When searching for a target within an image, our brain can adopt different strategies, but which one does it choose? This question can be answered by tracking the motion of the eye while it executes the task. Following many individuals performing various search tasks, we distinguish between two competing strategies. Motivated by these findings, we introduce a model that captures the interplay of the search strategies and allows us to create artificial eye-tracking trajectories, which could be compared with the experimental ones. Identifying the model parameters allows us to quantify the strategy employed in terms of ensemble averages, characterizing each experimental cohort. In this way, we can discern with high sensitivity the relation between the visual landscape and the average strategy, disclosing how small variations in the image induce changes in the strategy.

The Attentional Fields of Visual Search in Simultanagnosia and Healthy Individuals: How Object and Space Attention Interact

Simultanagnosia is a deficit in which patients are unable to perceive multiple objects simultaneously. To date, it remains disputed whether this deficit results from disrupted object or space perception. We asked both healthy participants as well as a patient with simultanagnosia to perform different visual search tasks of variable difficulty. We also modulated the number of objects (target and distracters) presented. For healthy participants, we found that each visual search task was performed with a specific "attentional field" depending on the difficulty of visual object processing but not on the number of objects falling within this "working space." This was demonstrated by measuring the cost in reaction times using different gaze-contingent visible window sizes. We found that bilateral damage to the superior parietal lobule impairs the spatial integration of separable features (within-object processing), shrinking the attentional field in which a target can be detected, but causing no deficit in processing multiple objects per se.

Evaluating eye fixation and color contrast effects in the measurement of visual lobe sizes: comment on Tsang, et al. (2013)

The visual lobe refers to the useful field of view, functional visual field, or conspicuity area in single eye fixation, and the visual lobe size is determined by the conspicuity of target presented amongst background characters. This comment discusses a study by Tsang, Chan, and Yu (2013) that investigated the effect of color contrast on the visual lobe. First, this comment suggests a way to improve the precision of their method for measuring a visual lobe. Measuring positions of the participants' eyes during visual search with an eye tracker would allow checking the eyes' fixation on the center of the search field, improving the precision of visual lobe measurement. This comment also suggests a more optimal experimental design to investigate the effect of color contrast on visual lobe size. Tsang, et al. used colors for target and background that had similar contrast (i.e., differences in chromaticity) for the experiments. If the color-contrast ratio in each experiment condition had differed enough, the color contrast effect on the visual lobe could have been investigated more effectively.

Letters in the forest: global precedence effect disappears for letters but not for non-letters under reading-like conditions

Normally skilled reading involves special processing strategies for letters, which are habitually funneled into an abstract letter code. On the basis of previous studies we argue that this habit leads to the preferred usage of an analytic strategy for the processing of letters, while non-letters are preferably processed via a holistic strategy. The well-known global precedence effect (GPE) seems to contradict to this assumption, since, with compound, hierarchical figures, including letter items, faster responses are observed to the global than to the local level of the figure, as well as an asymmetric interference effect from global to local level. We argue that with letters these effects depend on presentation conditions; only when they elicit the processing strategies automatized for reading, an analytic strategy for letters in contrast to non-letters is to be expected. We compared the GPE for letters and non-letters in central viewing, with the global stimulus size close to the functional visual field in whole word reading (6.5° of visual angle) and local stimuli close to the critical size for fluent reading of individual letters (0.5° of visual angle). Under these conditions, the GPE remained robust for non-letters. For letters, however, it disappeared: letters showed no overall response time advantage for the global level and symmetric congruence effects (local-to-global as well as global-to-local interference). We interpret these results as according to the view that reading is based on resident analytic visual processing strategies for letters.

Extrafoveal preview benefit during free-viewing visual search in the monkey

Previous studies have shown that subjects require less time to process a stimulus at the fovea after a saccade if they have viewed the same stimulus in the periphery immediately prior to the saccade. This extrafoveal preview benefit indicates that information about the visual form of an extrafoveally viewed stimulus can be transferred across a saccade. Here, we extend these findings by demonstrating and characterizing a similar extrafoveal preview benefit in monkeys during a free-viewing visual search task. We trained two monkeys to report the orientation of a target among distractors by releasing one of two bars with their hand; monkeys were free to move their eyes during the task. Both monkeys took less time to indicate the orientation of the target after foveating it, when the target lay closer to the fovea during the previous fixation. An extrafoveal preview benefit emerged even if there was more than one intervening saccade between the preview and the target fixation, indicating that information about target identity could be transferred across more than one saccade and could be obtained even if the search target was not the goal of the next saccade. An extrafoveal preview benefit was also found for distractor stimuli. These results aid future physiological investigations of the extrafoveal preview benefit.

Crowding during restricted and free viewing

Crowding impairs the perception of form in peripheral vision. It is likely to be a key limiting factor of form vision in patients without central vision. Crowding has been extensively studied in normally sighted individuals, typically with a stimulus duration of a few hundred milliseconds to avoid eye movements. These restricted testing conditions do not reflect the natural behavior of a patient with central field loss. Could unlimited stimulus duration and unrestricted eye movements change the properties of crowding in any fundamental way? We studied letter identification in the peripheral vision of normally sighted observers in three conditions: (i) a fixation condition with a brief stimulus presentation of 250 ms, (ii) another fixation condition but with an unlimited viewing time, and (iii) an unrestricted eye movement condition with an artificial central scotoma and an unlimited viewing time. In all conditions, contrast thresholds were measured as a function of target-to-flanker spacing, from which we estimated the spatial extent of crowding in terms of critical spacing. We found that presentation duration beyond 250 ms had little effect on critical spacing with stable gaze. With unrestricted eye movements and a simulated central scotoma, we found a large variability in critical spacing across observers, but more importantly, the variability in critical spacing was well correlated with the variability in target eccentricity. Our results assure that the large body of findings on crowding made with briefly presented stimuli remains relevant to conditions where viewing time is unconstrained. Our results further suggest that impaired oculomotor control associated with central vision loss can confound peripheral form vision beyond the limits imposed by crowding.

Effects of load on the time course of attentional engagement, disengagement, and orienting in reading

We examined how the frequency of the fixated word influences the spatiotemporal distribution of covert attention during reading. Participants discriminated gaze-contingent probes that occurred with different spatial and temporal offsets from randomly chosen fixation points during reading. We found that attention was initially focused at fixation and that subsequent defocusing was slower when the fixated word was lower in frequency. Later in a fixation, attention oriented more towards the next saccadic target for high- than for low-frequency words. These results constitute the first report of the time course of the effect of load on attentional engagement and orienting in reading. They are discussed in the context of serial and parallel models of reading.

Effects of Superposition on Oculomotor Guidance and Target Recognition

Searching for targets in complex scenes like X-ray baggage screening is demanding. One cause of impaired search performance is the superimposed objects of X-ray images. In this study, we compared searching for threat items in a superimposed display and a sparse display. Using both behavioral and eye movement measures, we explored the effects of superposition on search course. As expected, superposition affected accuracy rates and response times. Results showed searching in the superimposed display caused longer fixation durations, and the effect was pronounced throughout the search trial. We also found longer search initiation time and target verification time in superposition, suggesting that the initial planning of oculomotor guidance and target recognition was influenced.

Central V4 receptive fields are scaled by the V1 cortical magnification and correspond to a constant-sized sampling of the V1 surface

The mapping of the topographic representation of the visual field onto cortical areas changes throughout the hierarchy of cortical visual areas. The changes are believed to reflect the establishment of modules with different spatial processing emphasis. The receptive fields (RFs) of neurons within these modules, however, may not be governed by the same spatial topographic map parameters. Here it is shown that the RFs of area V4 neurons (centered 1-12 degrees in eccentricity) are based on a circularly symmetric sampling of the primary visual cortical retinotopic map. No eccentricity dependent magnification beyond that observed in V1 is apparent in the V4 neurons. The size and shape of V4 RFs can be explained by a simple, constant sized, two-dimensional Gaussian sample of visual input from the retinotopic map laid out across the surface of V1. Inferences about the spatial scale of interactions within the receptive fields of neurons cannot be based on a visual area's apparent cortical magnification derived from topographic mapping.