Crowding degrades saccadic search performance

Visuospatial, oculomotor, and executive reading skills evolve in elementary school, and errors are significant: a topological RAN study

We investigate the development of visuospatial and oculomotor reading skills in a cohort of elementary school children. Employing a longitudinal methodology, the study applies the Topological serial digit Rapid Automated Naming (Top-RAN) battery, which evaluates visuospatial reading skills leveraging metrics addressing crowding, distractors, and voluntary attention orientation. The participant pool comprises 142 students (66 males, 76 females), including 46 non-native speakers (21 males, 25 females), representing a diverse range of ethnic backgrounds. The Top-RAN dataset encompasses performance, error, and self-correction metrics for each subtest and student, underscoring the significance of these factors in the process of reading acquisition. Analytical methods include dimensionality reduction, clustering, and classification algorithms, consolidated into a Python package to facilitate reproducible results. Our results indicate that visuospatial reading abilities vary according to the task and demonstrate a marked evolution over time, as seen in the progressive decrease in execution times, errors, and self-corrections. This pattern supports the hypothesis that the growth of oculomotor, attentional, and executive skills is primarily fostered by educational experiences and maturation. This investigation provides valuable insights into the dynamic nature of these skills during pivotal educational stages.

Does crowding predict conjunction search? An individual differences approach

Searching for objects in the visual environment is an integral part of human behavior. Most of the information used during such visual search comes from the periphery of our vision, and understanding the basic mechanisms of search therefore requires taking into account the inherent limitations of peripheral vision. Our previous work using an individual differences approach has shown that one of the major factors limiting peripheral vision (crowding) is predictive of single feature search, as reflected in response time and eye movement measures. Here we extended this work, by testing the relationship between crowding and visual search in a conjunction-search paradigm. Given that conjunction search involves more fine-grained discrimination and more serial behavior, we predicted it would be strongly affected by crowding. We tested sixty participants with regard to their sensitivity to both orientation and color-based crowding (as measured by critical spacing) and their efficiency in searching for a color/orientation conjunction (as indicated by manual response times and eye movements). While the correlations between the different crowding tasks were high, the correlations between the different crowding measures and search performance were relatively modest, and no higher than those previously observed for single-feature search. Instead, observers showed very strong color selectivity during search. The results suggest that conjunction search behavior relies more on top-down guidance (here by color) and is therefore relatively less determined by individual differences in sensory limitations as caused by crowding.

Establishing gaze markers of perceptual load during multi-target visual search

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

Multidimensional feature interactions in visual crowding: When configural cues eliminate the polarity advantage

Crowding occurs when surrounding objects (flankers) impair target perception. A key property of crowding is the weaker interference when target and flankers strongly differ on a given dimension. For instance, identification of a target letter is usually superior with flankers of opposite versus the same contrast polarity as the target (the "polarity advantage"). High performance when target-flanker similarity is low has been attributed to the ungrouping of target and flankers. Here, we show that configural cues can override the usual advantage of low target-flanker similarity, and strong target-flanker grouping can reduce - instead of exacerbate - crowding. In Experiment 1, observers were presented with line triplets in the periphery and reported the tilt (left or right) of the central line. Target and flankers had the same (uniform condition) or opposite contrast polarity (alternating condition). Flanker configurations were either upright (||), unidirectionally tilted (\\ or //), or bidirectionally tilted (\/ or /\). Upright flankers yielded stronger crowding than unidirectional flankers, and weaker crowding than bidirectional flankers. Importantly, our results revealed a clear interaction between contrast polarity and flanker configuration. Triplets with upright and bidirectional flankers, but not unidirectional flankers, showed the polarity advantage. In Experiments 2 and 3, we showed that emergent features and redundancy masking (i.e. the reduction of the number of perceived items in repeating configurations) made it easier to discriminate between uniform triplets when flanker tilts were unidirectional (but not when bidirectional). We propose that the spatial configurations of uniform triplets with unidirectional flankers provided sufficient task-relevant information to enable a similar performance as with alternating triplets: strong-target flanker grouping alleviated crowding. We suggest that features which modulate crowding strength can interact non-additively, limiting the validity of typical crowding rules to contexts where only single, independent dimensions determine the effects of target-flanker similarity.

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

The Wealth State Awareness Effect on Attention Allocation in People From Impoverished and Affluent Groups

Previous studies have shown that poverty influences cognitive abilities and that those who have a negative living environment exhibit worse cognitive performance. In addition, eye measures vary following the manipulation of cognitive processing. We examined the distinctive changes in impoverished and affluent persons during tasks that require a high level of concentration using eye-tracking measures. Based on the poverty effect in impoverished people, this study explored how wealth state awareness (WSA) influences them. It was found that the pupillary state indexes of the impoverished participants significantly changed when their WSA was regarding poverty. The results suggest that awareness of poverty may cause impoverished individuals to engage in tasks with more attention allocation and more concentration in the more difficult tasks but that a WSA regarding wealth does not have such effect on them. WSA has no significant effects on their more affluent peers. The findings of this study can contribute to research on WSA effects on impoverished individuals from the perspective of eye measures.

Dwelling on simple stimuli in visual search

Research and theories on visual search often focus on visual guidance to explain differences in search. Guidance is the tuning of attention to target features and facilitates search because distractors that do not show target features can be more effectively ignored (skipping). As a general rule, the better the guidance is, the more efficient search is. Correspondingly, behavioral experiments often interpreted differences in efficiency as reflecting varying degrees of attentional guidance. But other factors such as the time spent on processing a distractor (dwelling) or multiple visits to the same stimulus in a search display (revisiting) are also involved in determining search efficiency. While there is some research showing that dwelling and revisiting modulate search times in addition to skipping, the corresponding studies used complex naturalistic and category-defined stimuli. The present study tests whether results from prior research can be generalized to more simple stimuli, where target-distractor similarity, a strong factor influencing search performance, can be manipulated in a detailed fashion. Thus, in the present study, simple stimuli with varying degrees of target-distractor similarity were used to deliver conclusive evidence for the contribution of dwelling and revisiting to search performance. The results have theoretical and methodological implications: They imply that visual search models should not treat dwelling and revisiting as constants across varying levels of search efficiency and that behavioral search experiments are equivocal with respect to the responsible processing mechanisms underlying more versus less efficient search. We also suggest that eye-tracking methods may be used to disentangle different search components such as skipping, dwelling, and revisiting.

Chromatic discriminations along two cardinal axes share a common attentional resource

Attentional modulation is specific to either luminance or chromatic contrast discrimination, implying separate attentional resources for processing luminance and chromatic information processing [e.g., Curr. Biol.12, 1134 (2002)CUBLE20960-982210.1016/S0960-9822(02)00921-1]. However, there are two distinct visual pathways that process chromatic information: the parvocellular (PC) and koniocellular (KC) pathways. It is unclear whether there are separate attentional resources modulating the chromatic processes in these pathways. Here, we examined the attentional modulation effects on chromatic contrast discrimination with chromaticities along the $l$l or $s$s cardinal axis on a cone chromaticity space for preferentially stimulating either the inferred PC or KC pathway, respectively. A dual-task interference paradigm was used, and chromatic contrast discrimination sensitivities under dual-task conditions were compared with that under a single-task condition. The results revealed that compared with the single-task condition, attending to a competing central task in the dual-task condition decreased the peripheral discrimination sensitivity in both chromatic cardinal axes, and sensitivity reduced regardless of whether the dual tasks were along the same or different chromatic cardinal axes. These findings indicate that attentional effects on chromatic processes are not specific to the cardinal axis, suggesting that the PC and KC pathways may share a common attention resource in modulating chromatic processing.

EEG & Eye Tracking User Experiments for Spatial Memory Task on Maps

The aim of this research is to evaluate the use of ET and EEG for studying the cognitive processes of expert and novice map users and to explore these processes by comparing two types of spatial memory experiments through cognitive load measurements. The first experiment consisted of single trials and participants were instructed to study a map stimulus without any time constraints in order to draw a sketch map afterwards. According to the ET metrics (i.e., average fixation duration and the number of fixations per second), no statistically significant differences emerged between experts and novices. A similar result was also obtained with EEG Frontal Alpha Asymmetry calculations. On the contrary, in terms of alpha power across all electrodes, novices exhibited significantly lower alpha power, indicating a higher cognitive load. In the second experiment, a larger number of stimuli were used to study the effect of task difficulty. The same ET metrics used in the first experiment indicated that the difference between these user groups was not statistically significant. The cognitive load was also extracted using EEG event-related spectral power changes at alpha and theta frequency bands. Preliminary data exploration mostly suggested an increase in theta power and a decrease in alpha power.

Observation of environments with different restorative potential results in differences in eye patron movements and pupillary size

•

Mean number of fixations, fixation durations and pupil dilation were calculated for high and low restorative potential scenes.

•

There were significantly more fixations for low restorative than for high restorative scenes.

•

Fixation times had the predicted inverse relationship, with low restorative settings having a significantly shorter time per fixation than high restorative scenes.

•

Higher measures of pupil dilatation were related to the view of high restorative potential vs. low restorative potential environments.

•

Eye movements related to photographs, low on fascination, were characterized by a greater exploration and a greater number of fixations compared to those which rated high on fascination.

The Environmental Psychological Restoration (EPR) is the result of the recovery of an antecedent deficit (e.g. stress or attentional fatigue) culminating in a restorative environment exposure. Recent findings suggest that visual contact with nature is important in triggering restorative responses. We measure the behavioral evidence of visual exposure to restorative environments in an eye-tracking study. Eye movement patterns (fixations and pupil dilatation) were evaluated while a sample of participants (n = 27; 15 females and 12 males) viewed photographs with High Restorative Potential (HRP) or Low Restorative Potential (LRP). The eye patterns during the observation of LRP were distinct to those of the HRP environments. Eye movements related to LRP photographs were characterized by a greater number of fixations compared to those related to HRP. Fixation times predicted an inverse relation, with LRP settings having a significantly shorter time per fixation than HRP pictures. Differences on pupil diameter were found. A higher pupil size was found during the view of HRP vs. LRP environments. Our eye tracking study suggest that restorative environment observation is associated with reduced eye movement activity relative to low restorative potential environment perception, which may reflect a lower cognitive effort in processing natural scenes. Likewise, pupillary dilatation variations suggest a possible link between the affective valences of the settings and its restorative quality. Data results are confronted according to attention restoration theory on restorative environments.

Processing speed in perceptual visual crowding

In this study, a perceptual visual crowding paradigm was designed to quantitatively assess the detection speed of (un)crowded meaningful visual targets using eye-movement responses. This paradigm was tested in individuals with dyslexia and age-matched controls. Trials were shown on a monitor with an integrated eye tracker to 25 control and 11 dyslexic subjects without any known ocular problems. Each trial started with fixation of a central target. Next, four peripheral targets were shown (left, right, top, bottom), one being a duplicate of the central target. The duplicate was either surrounded by flankers (crowding trials) or shown in isolation (reference trials). The timing of the primary saccades were obtained as a measure for detection speed. The performance of the reference trials was significantly higher compared to the crowding trials (p < 0.05) and a 54% increase in saccadic reaction time (SRT) was found for the crowding trials. The linear mixed model revealed a significant effect of critical spacing and chart type. For the reference trials, no significant differences in SRT were found between dyslexic and control subjects. However, for the crowding trials, a significant increase of ∼13% in SRT was found in the dyslexic subjects. A first application of this paradigm showed that dyslexic subjects perform equally well in identifying visual targets in crowded as well as uncrowded scenes compared to controls. However, they seem to need more time to identify targets in crowded scenes, which might be related to the reading difficulties that they experience in general.

Non-dominant hand use increases completion time on part B of the Trail Making Test but not on part A

The Trail Making Test (TMT) is used in neuropsychological clinical practice to assess aspects of attention and executive function. The test consists of two parts (A and B) and requires drawing a trail between elements. Many patients are assessed with their non-dominant hand because of motor dysfunction that prevents them from using their dominant hand. Since drawing with the non-dominant hand is not an automatic task for many people, we explored the effect of hand use on TMT performance. The TMT was administered digitally in order to analyze new outcome measures in addition to total completion time. In a sample of 82 healthy participants, we found that non-dominant hand use increased completion times on the TMT B but not on the TMT A. The average completion time increased by almost 5 seconds, which may be clinically relevant. A substantial number of participants who performed the TMT with their non-dominant hand had a B/A ratio score of 2.5 or higher. In clinical practice, an abnormally high B/A ratio score may be falsely attributed to cognitive dysfunction. With our digitized pen data, we further explored the causes of the reduced TMT B performance by using new outcome measures, including individual element completion times and interelement variability. These measures indicated selective interference between non-dominant hand use and executive functions. Both non-dominant hand use and performance of the TMT B seem to draw on the same, limited higher-order cognitive resources.

The crowding factor method applied to parafoveal vision

Crowding increases with eccentricity and is most readily observed in the periphery. During natural, active vision, however, central vision plays an important role. Measures of critical distance to estimate crowding are difficult in central vision, as these distances are small. Any overlap of flankers with the target may create an overlay masking confound. The crowding factor method avoids this issue by simultaneously modulating target size and flanker distance and using a ratio to compare crowded to uncrowded conditions. This method was developed and applied in the periphery (Petrov & Meleshkevich, 2011b). In this work, we apply the method to characterize crowding in parafoveal vision (<3.5 visual degrees) with spatial uncertainty. We find that eccentricity and hemifield have less impact on crowding than in the periphery, yet radial/tangential asymmetries are clearly preserved. There are considerable idiosyncratic differences observed between participants. The crowding factor method provides a powerful tool for examining crowding in central and peripheral vision, which will be useful in future studies that seek to understand visual processing under natural, active viewing conditions.

Competing Distractors Facilitate Visual Search in Heterogeneous Displays

In the present study, we examine how observers search among complex displays. Participants were asked to search for a big red horizontal line among 119 distractor lines of various sizes, orientations and colours, leading to 36 different feature combinations. To understand how people search in such a heterogeneous display, we evolved the search display by using a genetic algorithm (Experiment 1). The best displays (i.e., displays corresponding to the fastest reaction times) were selected and combined to create new, evolved displays. Search times declined over generations. Results show that items sharing the same colour and orientation as the target disappeared over generations, implying they interfered with search, but items sharing the same colour and were 12.5° different in orientation only interfered if they were also the same size. Furthermore, and inconsistent with most dominant visual search theories, we found that non-red horizontal distractors increased over generations, indicating that these distractors facilitated visual search while participants were searching for a big red horizontally oriented target. In Experiments 2 and 3, we replicated these results using conventional, factorial experiments. Interestingly, in Experiment 4, we found that this facilitation effect was only present when the displays were very heterogeneous. While current models of visual search are able to successfully describe search in homogeneous displays, our results challenge the ability of these models to describe visual search in heterogeneous environments.

Parametric modeling of visual search efficiency in real scenes

How should the efficiency of searching for real objects in real scenes be measured? Traditionally, when searching for artificial targets, e.g., letters or rectangles, among distractors, efficiency is measured by a reaction time (RT) × Set Size function. However, it is not clear whether the set size of real scenes is as effective a parameter for measuring search efficiency as the set size of artificial scenes. The present study investigated search efficiency in real scenes based on a combination of low-level features, e.g., visible size and target-flanker separation factors, and high-level features, e.g., category effect and target template. Visible size refers to the pixel number of visible parts of an object in a scene, whereas separation is defined as the sum of the flank distances from a target to the nearest distractors. During the experiment, observers searched for targets in various urban scenes, using pictures as the target templates. The results indicated that the effect of the set size in real scenes decreased according to the variances of other factors, e.g., visible size and separation. Increasing visible size and separation factors increased search efficiency. Based on these results, an RT × Visible Size × Separation function was proposed. These results suggest that the proposed function is a practicable predictor of search efficiency in real scenes.

ICAT: a computational model for the adaptive control of fixation durations

Eye movements depend on cognitive processes related to visual information processing. Much has been learned about the spatial selection of fixation locations, while the principles governing the temporal control (fixation durations) are less clear. Here, we review current theories for the control of fixation durations in tasks like visual search, scanning, scene perception, and reading and propose a new model for the control of fixation durations. We distinguish two local principles from one global principle of control. First, an autonomous saccade timer initiates saccades after random time intervals (local-I). Second, foveal inhibition permits immediate prolongation of fixation durations by ongoing processing (local-II). Third, saccade timing is adaptive, so that the mean timer value depends on task requirements and fixation history (Global). We demonstrate by numerical simulations that our model qualitatively reproduces patterns of mean fixation durations and fixation duration distributions observed in typical experiments. When combined with assumptions of saccade target selection and oculomotor control, the model accounts for both temporal and spatial aspects of eye movement control in two versions of a visual search task. We conclude that the model provides a promising framework for the control of fixation durations in saccadic tasks.

Display clutter: a review of definitions and measurement techniques

OBJECTIVE

We aimed to synthesize the literature on display clutter by reviewing definitions and measurement techniques and to develop a comprehensive, ergonomics-oriented perspective on clutter. We provide guidance for the selection of measurement approaches that can capture the various aspects and effects of clutter on visual search in particular.

BACKGROUND

There is agreement that clutter may represent a significant problem to operators. The challenge is to determine the ideal middle ground between excessive data and insufficient information. However, definitions of clutter vary widely, which explains the range of measurement approaches, including image-processing algorithms, performance evaluation, subjective evaluation, and eye tracking. It is important to understand the affordances of each technique and provide guidance for their use.

METHOD

We provide a systematic review of clutter definitions and develop a performance-oriented perspective for ergonomics research. Next, we present a critical overview of clutter measurement approaches. The benefits and limitations of each technique are detailed, and recommendations for best practice are provided.

RESULTS

From an ergonomics perspective, clutter matters to the extent that it affects performance, particularly, visual search. A combination of measurement techniques can be employed in order to assess the performance costs stemming from the multiple aspects of clutter.

CONCLUSION

Display clutter is a multifaceted construct that is a problem when it results in performance and attentional costs. The assessment of these costs is critical and requires the use of appropriate and complementary techniques.

APPLICATION

It is important that the performance costs of clutter are detected reliably to be able to better support attention management.

The prevalence effect in lateral masking and its relevance for visual search

In stimulus displays with or without a single target amid 1,644 identical distractors, target prevalence was varied between 20, 50 and 80 %. Maximum gaze deviation was measured to determine the strength of lateral masking in these arrays. The results show that lateral masking was strongest in the 20 % prevalence condition, which differed significantly from both the 50 and 80 % prevalence conditions. No difference was observed between the latter two. This pattern of results corresponds to that found in the literature on the prevalence effect in visual search (stronger lateral masking corresponding to longer search times). The data add to similar findings reported earlier (Wertheim et al. in Exp Brain Res, 170:387-402, 2006), according to which the effects of many well-known factors in visual search correspond to those on lateral masking. These were the effects of set size, disjunctions versus conjunctions, display area, distractor density, the asymmetry effect (Q vs. O's) and viewing distance. The present data, taken together with those earlier findings, may lend credit to a causal hypothesis that lateral masking could be a more important mechanism in visual search than usually assumed.

Substitution and pooling in visual crowding induced by similar and dissimilar distractors

Visual crowding refers to a phenomenon whereby objects that appear in the periphery of the visual field are more difficult to identify when embedded within clutter. Pooling models assert that crowding results from an obligatory averaging or other combination of target and distractor features that occurs prior to awareness. One well-known manifestation of pooling is feature averaging, with which the features of target and nontarget stimuli are combined at an early stage of visual processing. Conversely, substitution models assert that crowding results from binding a target and nearby distractors to incorrect spatial locations. Recent evidence suggests that substitution predominates when target-flanker feature similarity is low, but it is unclear whether averaging or substitution best explains crowding when similarity is high. Here, we examined participants' orientation report errors for targets crowded by similar or dissimilar flankers. In two experiments, we found evidence inconsistent with feature averaging regardless of target-flanker similarity. However, the observed data could be accommodated by a probabilistic substitution model in which participants occasionally "swap" a target for a distractor. Thus, we conclude that-at least for the displays used here-crowding likely results from a probabilistic substitution of targets and distractors, regardless of target-distractor feature similarity.

A SURVEY OF AUTOMATIC PERSON RECOGNITION USING EYE MOVEMENTS

The human eye is rich in physical and behavioral attributes that can be used for automatic person recognition. The physical attributes such as the iris attracted early attention and yielded significant recognition results, but like most physical biometrics, they have several disadvantages such as intrusive acquisition, vulnerability to spoofing attacks, etc. Consequently, during the last decade the behavioral attributes extracted from human eyes have steadily gained interest from the automatic person recognition research community. In this first of its kind survey, we present the studies utilizing the behavioral attributes of human eyes for automatic person recognition. We have proposed a unique classification based on the type of stimuli used to elicit behavioral attributes. In addition, for each approach we have carefully examined the common steps involved in automatic person recognition from database acquisition, feature extraction to classification. Lastly, we also present a comparison of the recognition results obtained by each approach.

Training shortens search times in children with visual impairment accompanied by nystagmus

Perceptual learning (PL) can improve near visual acuity (NVA) in 4–9 year old children with visual impairment (VI). However, the mechanisms underlying improved NVA are unknown. The present study compares feature search and oculomotor measures in 4–9 year old children with VI accompanied by nystagmus (VI+nys [n = 33]) and children with normal vision (NV [n = 29]). Children in the VI+nys group were divided into three training groups: an experimental PL group, a control PL group, and a magnifier group. They were seen before (baseline) and after 6 weeks of training. Children with NV were only seen at baseline. The feature search task entailed finding a target E among distractor E's (pointing right) with element spacing varied in four steps: 0.04°, 0.5°, 1°, and 2°. At baseline, children with VI+nys showed longer search times, shorter fixation durations, and larger saccade amplitudes than children with NV. After training, all training groups showed shorter search times. Only the experimental PL group showed prolonged fixation duration after training at 0.5° and 2° spacing, p's respectively 0.033 and 0.021. Prolonged fixation duration was associated with reduced crowding and improved crowded NVA. One of the mechanisms underlying improved crowded NVA after PL in children with VI+nys seems to be prolonged fixation duration.

Integrating retinotopic features in spatiotopic coordinates

The receptive fields of early visual neurons are anchored in retinotopic coordinates (Hubel and Wiesel, 1962). Eye movements shift these receptive fields and therefore require that different populations of neurons encode an object's constituent features across saccades. Whether feature groupings are preserved across successive fixations or processing starts anew with each fixation has been hotly debated (Melcher and Morrone, 2003; Melcher, 2005, 2010; Knapen et al., 2009; Cavanagh et al., 2010a,b; Morris et al., 2010). Here we show that feature integration initially occurs within retinotopic coordinates, but is then conserved within a spatiotopic coordinate frame independent of where the features fall on the retinas. With human observers, we first found that the relative timing of visual features plays a critical role in determining the spatial area over which features are grouped. We exploited this temporal dependence of feature integration to show that features co-occurring within 45 ms remain grouped across eye movements. Our results thus challenge purely feedforward models of feature integration (Pelli, 2008; Freeman and Simoncelli, 2011) that begin de novo after every eye movement, and implicate the involvement of brain areas beyond early visual cortex. The strong temporal dependence we quantify and its link with trans-saccadic object perception instead suggest that feature integration depends, at least in part, on feedback from higher brain areas (Mumford, 1992; Rao and Ballard, 1999; Di Lollo et al., 2000; Moore and Armstrong, 2003; Stanford et al., 2010).

Facilitating recognition of crowded faces with presaccadic attention

In daily life, we make several saccades per second to objects we cannot normally recognize in the periphery due to visual crowding. While we are aware of the presence of these objects, we cannot identify them and may, at best, only know that an object is present at a particular location. The process of planning a saccade involves a presaccadic attentional component known to be critical for saccadic accuracy, but whether this or other presaccadic processes facilitate object identification as opposed to object detection-especially with high level natural objects like faces-is less clear. In the following experiments, we show that presaccadic information about a crowded face reduces the deleterious effect of crowding, facilitating discrimination of two emotional faces, even when the target face is never foveated. While accurate identification of crowded objects is possible in the absence of a saccade, accurate identification of a crowded object is considerably facilitated by presaccadic attention. Our results provide converging evidence for a selective increase in available information about high level objects, such as faces, at a presaccadic stage.

Scan path entropy and arrow plots: capturing scanning behavior of multiple observers

Designers of visual communication material want their material to attract and retain attention. In marketing research, heat maps, dwell time, and time to AOI first hit are often used as evaluation parameters. Here we present two additional measures (1) "scan path entropy" to quantify gaze guidance and (2) the "arrow plot" to visualize the average scan path. Both are based on string representations of scan paths. The latter also incorporates transition matrices and time required for 50% of the observers to first hit AOIs (T50). The new measures were tested in an eye tracking study (48 observers, 39 advertisements). Scan path entropy is a sensible measure for gaze guidance and the new visualization method reveals aspects of the average scan path and gives a better indication in what order global scanning takes place.

It depends on how you look at it: scanpath comparison in multiple dimensions with MultiMatch, a vector-based approach

Eye movement sequences-or scanpaths-vary depending on the stimulus characteristics and the task (Foulsham & Underwood Journal of Vision, 8(2), 6:1-17, 2008; Land, Mennie, & Rusted, Perception, 28, 1311-1328, 1999). Common methods for comparing scanpaths, however, are limited in their ability to capture both the spatial and temporal properties of which a scanpath consists. Here, we validated a new method for scanpath comparison based on geometric vectors, which compares scanpaths over multiple dimensions while retaining positional and sequential information (Jarodzka, Holmqvist, & Nyström, Symposium on Eye-Tracking Research and Applications (pp. 211-218), 2010). "MultiMatch" was tested in two experiments and pitted against ScanMatch (Cristino, Mathôt, Theeuwes, & Gilchrist, Behavior Research Methods, 42, 692-700, 2010), the most comprehensive adaptation of the popular Levenshtein method. In Experiment 1, we used synthetic data, demonstrating the greater sensitivity of MultiMatch to variations in spatial position. In Experiment 2, real eye movement recordings were taken from participants viewing sequences of dots, designed to elicit scanpath pairs with commonalities known to be problematic for algorithms (e.g., when one scanpath is shifted in locus or when fixations fall on either side of an AOI boundary). The results illustrate the advantages of a multidimensional approach, revealing how two scanpaths differ. For instance, if one scanpath is the reverse copy of another, the difference is in the direction but not the positions of fixations; or if a scanpath is scaled down, the difference is in the length of the saccadic vectors but not in the overall shape. As well as having enormous potential for any task in which consistency in eye movements is important (e.g., learning), MultiMatch is particularly relevant for "eye movements to nothing" in mental imagery and embodiment-of-cognition research, where satisfactory scanpath comparison algorithms are lacking.

Visual search with image modification in age-related macular degeneration

PURPOSE

AMD results in loss of central vision and a dependence on low-resolution peripheral vision. While many image enhancement techniques have been proposed, there is a lack of quantitative comparison of the effectiveness of enhancement. We developed a natural visual search task that uses patients' eye movements as a quantitative and functional measure of the efficacy of image modification.

METHODS

Eye movements of 17 patients (mean age = 77 years) with AMD were recorded while they searched for target objects in natural images. Eight different image modification methods were implemented and included manipulations of local image or edge contrast, color, and crowding. In a subsequent task, patients ranked their preference of the image modifications.

RESULTS

Within individual participants, there was no significant difference in search duration or accuracy across eight different image manipulations. When data were collapsed across all image modifications, a multivariate model identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interactions among scotoma size, age, acuity, and contrast (P < 0.05). Additionally, an analysis of image statistics showed no correlation with search performance across all image modifications. Rank ordering of enhancement methods based on participants' preference revealed a trend that participants preferred the least modified images (P < 0.05).

CONCLUSIONS

There was no quantitative effect of image modification on search performance. A better understanding of low- and high-level components of visual search in natural scenes is necessary to improve future attempts at image enhancement for low vision patients. Different search tasks may require alternative image modifications to improve patient functioning and performance.

Eye Tracking Metrics: A Toolbox for Assessing the Effects of Clutter on Attention Allocation

Display clutter is a problem that affects operators in various data-rich environments. Clutter measurement techniques such as image processing and performance measures can provide an estimate of clutter but are largely not suited to tracing the effects of clutter on the dynamic allocation of attention. Eye tracking is a promising process-oriented tool that can help assess in real-time the attentional costs associated with the different aspects of clutter. In this experiment, we investigated which of a number of eye tracking metrics in the literature are sensitive to clutter. Twenty-two participants were asked to look for a target in static and dynamic images that were classified as either high or low in clutter. Response time and error rate were recorded, and an eye tracker was used to compute the identified eye tracking metrics. Results showed that, in both the static and dynamic conditions, a large number of eye tracking metrics were significantly affected by an increase in clutter. This suggests that eye tracking can be used to supplement other clutter measurement techniques by providing information about dynamic attention allocation.

Eye Movements in Patients with Glaucoma When Viewing Images of Everyday Scenes

This study tests the hypothesis that patients with bilateral glaucoma exhibit different eye movements compared to normally-sighted people when viewing computer displayed photographs of everyday scenes. Thirty glaucomatous patients and 30 age-related controls with normal vision viewed images on a computer monitor whilst eye movements were simultaneously recorded using an eye tracking system. The patients demonstrated a significant reduction in the average number of saccades compared to controls (; mean reduction of 7% (95% confidence interval (CI): 3–11%)). There was no difference in average saccade amplitude between groups but there was between-person variability in patients. The average elliptical region scanned by the patients by a bivariate contour ellipse area (BCEA) analysis, was more restricted compared to controls (; mean reduction of 23% (95% (CI): 11–35%)). A novel analysis mapping areas of interest in the images indicated a weak association between severity of functional deficit and a tendency to not view regions typically viewed by the controls. In conclusion, some eye movements in some patients with bilateral glaucomatous defects differ from normal-sighted people of a similar age when viewing images of everyday scenes, providing evidence for a potential new window for looking into the functional consequences of the disease.

When do I quit? The search termination problem in visual search

In visual search tasks, observers look for targets in displays or scenes containing distracting, non-target items. Most of the research on this topic has concerned the finding of those targets. Search termination is a less thoroughly studied topic. When is it time to abandon the current search? The answer is fairly straight forward when the one and only target has been found (There are my keys.). The problem is more vexed if nothing has been found (When is it time to stop looking for a weapon at the airport checkpoint?) or when the number of targets is unknown (Have we found all the tumors?). This chapter reviews the development of ideas about quitting time in visual search and offers an outline of our current theory.

Visual search for arbitrary objects in real scenes

How efficient is visual search in real scenes? In searches for targets among arrays of randomly placed distractors, efficiency is often indexed by the slope of the reaction time (RT) × Set Size function. However, it may be impossible to define set size for real scenes. As an approximation, we hand-labeled 100 indoor scenes and used the number of labeled regions as a surrogate for set size. In Experiment 1, observers searched for named objects (a chair, bowl, etc.). With set size defined as the number of labeled regions, search was very efficient (~5 ms/item). When we controlled for a possible guessing strategy in Experiment 2, slopes increased somewhat (~15 ms/item), but they were much shallower than search for a random object among other distinctive objects outside of a scene setting (Exp. 3: ~40 ms/item). In Experiments 4-6, observers searched repeatedly through the same scene for different objects. Increased familiarity with scenes had modest effects on RTs, while repetition of target items had large effects (>500 ms). We propose that visual search in scenes is efficient because scene-specific forms of attentional guidance can eliminate most regions from the "functional set size" of items that could possibly be the target.

Eye movements during action preparation

Looking at actions of others activates representations of similar own actions, that is, the action resonates. This may facilitate or interfere with the actions that one intends to make. We asked whether people promote or block those effects by making eye movements to or away from the actions of others. We investigated gaze behavior with a cup-clinking task: An actor shown on a video grabbed a cup and moved it toward the participant who next grabbed his own cup in the 'same' or in a different, 'complementary', way. In the 'same' condition, participants mostly looked at the place where the actor held the cup. In the 'complementary' condition, gaze behavior was similar at the start of the actor's action. To our surprise, as the action reached completion, participants started to look at the cup's site that corresponded to the grabbing instruction for their own action. A second experiment showed that this effect grew with delay of the go-signal. This indicates that a reason for the effect may be to support memorizing the instructed action. The bottom line of the study is that passively viewed scenes (passive in the sense that nothing in the observed scene is manipulated by the viewer) are scanned to support preparation of actions that one intends to make. We discuss how this finding relates to action resonance and how it relates to links between representations of actions and objects.

Reexamining the possible benefits of visual crowding: dissociating crowding from ensemble percepts

Peripheral objects and their features become indistinct when closely surrounding but nonoverlapping objects are present. Most models suggest that this phenomenon, called crowding, reflects limitations of visual processing, but an intriguing idea is that it may be, in part, adaptive. Specifically, the mechanism generating crowding may simultaneously facilitate ensemble representations of features, leaving meaningful information about clusters of objects. In two experiments, we tested whether visual crowding and the perception of ensemble features share a common mechanism. Observers judged the orientation of a crowded bar, or the ensemble orientation of all bars in the upper and lower visual fields. While crowding was predictably stronger in the upper relative to the lower visual field, the ensemble percept did not vary between the visual fields. Featural averaging within the crowded region does not always scale with the resolution limit defined by crowding, suggesting that dissociable processes contribute to visual crowding and ensemble percepts.

Saccadic selection and crowding in visual search: stronger lateral masking leads to shorter search times

We investigated the role of crowding in saccadic selection during visual search. To guide eye movements, often information from the visual periphery is used. Crowding is known to deteriorate the quality of peripheral information. In four search experiments, we studied the role of crowding, by accompanying individual search elements by flankers. Varying the difference between target and flankers allowed us to manipulate crowding strength throughout the stimulus. We found that eye movements are biased toward areas with little crowding for conditions where a target could be discriminated peripherally. Interestingly, for conditions in which the target could not be discriminated peripherally, this bias reversed to areas with strong crowding. This led to shorter search times for a target presented in areas with stronger crowding, compared to a target presented in areas with less crowding. These findings suggest a dual role for crowding in visual search. The presence of flankers similar to the target deteriorates the quality of the peripheral target signal but can also attract eye movements, as more potential targets are present over the area.

Visual crowding: a fundamental limit on conscious perception and object recognition

Crowding, the inability to recognize objects in clutter, sets a fundamental limit on conscious visual perception and object recognition throughout most of the visual field. Despite how widespread and essential it is to object recognition, reading and visually guided action, a solid operational definition of what crowding is has only recently become clear. The goal of this review is to provide a broad-based synthesis of the most recent findings in this area, to define what crowding is and is not, and to set the stage for future work that will extend our understanding of crowding well beyond low-level vision. Here we define six diagnostic criteria for what counts as crowding, and further describe factors that both escape and break crowding. All of these lead to the conclusion that crowding occurs at multiple stages in the visual hierarchy.

Visual conspicuity: a new simple standard, its reliability, validity and applicability

A general standard for quantifying conspicuity is described. It derives from a simple and easy method to quantitatively measure the visual conspicuity of an object. The method stems from the theoretical view that the conspicuity of an object is not a property of that object, but describes the degree to which the object is perceptually embedded in, i.e. laterally masked by, its visual environment. First, three variations of a simple method to measure the strength of such lateral masking are described and empirical evidence for its reliability and its validity is presented, as are several tests of predictions concerning the effects of viewing distance and ambient light. It is then shown how this method yields a conspicuity standard, expressed as a number, which can be made part of a rule of law, and which can be used to test whether or not, and to what extent, the conspicuity of a particular object, e.g. a traffic sign, meets a predetermined criterion. An additional feature is that, when used under different ambient light conditions, the method may also yield an index of the amount of visual clutter in the environment. Taken together the evidence illustrates the methods' applicability in both the laboratory and in real-life situations. STATEMENT OF RELEVANCE: This paper concerns a proposal for a new method to measure visual conspicuity, yielding a numerical index that can be used in a rule of law. It is of importance to ergonomists and human factor specialists who are asked to measure the conspicuity of an object, such as a traffic or rail-road sign, or any other object. The new method is simple and circumvents the need to perform elaborate (search) experiments and thus has great relevance as a simple tool for applied research.

The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search

Eye movements are now widely used to investigate cognitive processes during reading, scene perception, and visual search. In this article, research on the following topics is reviewed with respect to reading: (a) the perceptual span (or span of effective vision), (b) preview benefit, (c) eye movement control, and (d) models of eye movements. Related issues with respect to eye movements during scene perception and visual search are also reviewed. It is argued that research on eye movements during reading has been somewhat advanced over research on eye movements in scene perception and visual search and that some of the paradigms developed to study reading should be more widely adopted in the study of scene perception and visual search. Research dealing with “real-world” tasks and research utilizing the visual-world paradigm are also briefly discussed.

Exploring the visual world: The neural substrate of spatial orienting

Inspecting the visual environment, humans typically direct their attention across space by means of voluntary saccadic eye movements. Neuroimaging studies in healthy subjects have identified the superior parietal cortex and intraparietal sulcus as important structures involved in visual search. However, in apparent contrast, spatial disturbance of free exploration typically is observed after damage of brain structures located far more ventrally. Lesion studies in such patients disclosed the inferior parietal lobule (IPL) and temporo-parietal junction (TPJ), the superior temporal gyrus (STG) and insula, as well as the inferior frontal gyrus (IFG) of the right hemisphere. Here we used functional magnetic resonance imaging to investigate the involvement of these areas in active visual exploration in the intact brain. We conducted a region of interest analysis comparing free visual exploration of a dense stimulus array with the execution of stepwise horizontal and vertical saccades. The comparison of BOLD responses revealed significant signal increases during exploration in TPJ, STG, and IFG. This result calls for a reappraisal of the previous thinking on the function of these areas in visual search processes. In agreement with lesion studies, the data suggest that these areas are part of the network involved in human spatial orienting and exploration. The IPL dorsally of TPJ seem to be of minor importance for free visual exploration as these areas appear to be equally involved in the execution of spatially predetermined saccades.