Cognitive control of fixation duration in visual search: The role of extrafoveal processing

Establishing gaze markers of perceptual load during multi-target visual search

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

Lower grade students tend to give up early in multimedia learning

This study examined the relationship between grade and the ability of text-picture integration in terms of task completion and the pattern of using textual and pictorial information. Children (N = 144) from secondary schools were recruited from grade 5 and grade 8. Analyzing the time spent with the multimedia unit prior to an incorrect response, this study suggests that 5th graders tend to give up relatively early compared to 8th graders. Furthermore, early incorrect responses were more prevalent among students of the non-academic track and were predicted by a low proportion of fixations on the text in mental model construction and on the picture in adaptive mental model specification. An inappropriate approach of integrating text and picture can thus be the reason for giving up early. The work provides a basis for (1) automated feedback during the course of working on a multimedia unit. (2) Instructional design should be adapted to the characteristics of student groups to support text-picture integration and to avoid giving up early.

The Validity of Physiological Measures to Identify Differences in Intrinsic Cognitive Load

A sample of 33 experiments was extracted from the Web-of-Science database over a 5-year period (2016-2020) that used physiological measures to measure intrinsic cognitive load. Only studies that required participants to solve tasks of varying complexities using a within-subjects design were included. The sample identified a number of different physiological measures obtained by recording signals from four main body categories (heart and lungs, eyes, skin, and brain), as well as subjective measures. The overall validity of the measures was assessed by examining construct validity and sensitivity. It was found that the vast majority of physiological measures had some level of validity, but varied considerably in sensitivity to detect subtle changes in intrinsic cognitive load. Validity was also influenced by the type of task. Eye-measures were found to be the most sensitive followed by the heart and lungs, skin, and brain. However, subjective measures had the highest levels of validity. It is concluded that a combination of physiological and subjective measures is most effective in detecting changes in intrinsic cognitive load.

Extrafoveal Processing in Categorical Search for Geometric Shapes: General Tendencies and Individual Variations

The paper addresses the capabilities and limitations of extrafoveal processing during a categorical visual search. Previous research has established that a target could be identified from the very first or without any saccade, suggesting that extrafoveal perception is necessarily involved. However, the limits in complexity defining the processed information are still not clear. We performed four experiments with a gradual increase of stimuli complexity to determine the role of extrafoveal processing in searching for the categorically defined geometric shape. The series of experiments demonstrated a significant role of extrafoveal processing while searching for simple two-dimensional shapes and its gradual decrease in a condition with more complicated three-dimensional shapes. The factors of objects' spatial orientation and distractor homogeneity significantly influenced both reaction time and the number of saccades required to identify a categorically defined target. An analysis of the individual p-value distributions revealed pronounced individual differences in using extrafoveal analysis and allowed examination of the performance of each particular participant. The condition with the forced prohibition of eye movements enabled us to investigate the efficacy of covert attention in the condition with complicated shapes. Our results indicate that both foveal and extrafoveal processing are simultaneously involved during a categorical search, and the specificity of their interaction is determined by the spatial orientation of objects, type of distractors, the prohibition to use overt attention, and individual characteristics of the participants.

Dwelling on simple stimuli in visual search

Research and theories on visual search often focus on visual guidance to explain differences in search. Guidance is the tuning of attention to target features and facilitates search because distractors that do not show target features can be more effectively ignored (skipping). As a general rule, the better the guidance is, the more efficient search is. Correspondingly, behavioral experiments often interpreted differences in efficiency as reflecting varying degrees of attentional guidance. But other factors such as the time spent on processing a distractor (dwelling) or multiple visits to the same stimulus in a search display (revisiting) are also involved in determining search efficiency. While there is some research showing that dwelling and revisiting modulate search times in addition to skipping, the corresponding studies used complex naturalistic and category-defined stimuli. The present study tests whether results from prior research can be generalized to more simple stimuli, where target-distractor similarity, a strong factor influencing search performance, can be manipulated in a detailed fashion. Thus, in the present study, simple stimuli with varying degrees of target-distractor similarity were used to deliver conclusive evidence for the contribution of dwelling and revisiting to search performance. The results have theoretical and methodological implications: They imply that visual search models should not treat dwelling and revisiting as constants across varying levels of search efficiency and that behavioral search experiments are equivocal with respect to the responsible processing mechanisms underlying more versus less efficient search. We also suggest that eye-tracking methods may be used to disentangle different search components such as skipping, dwelling, and revisiting.

On the relative (un)importance of foveal vision during letter search in naturalistic scenes

The importance of high-acuity foveal vision to visual search can be assessed by denying foveal vision using the gaze-contingent Moving Mask technique. Foveal vision was necessary to attain normal performance when searching for a target letter in alphanumeric displays, Perception & Psychophysics, 62 (2000) 576-585. In contrast, foveal vision was not necessary to correctly locate and identify medium-sized target objects in natural scenes, Journal of Experimental Psychology: Human Perception and Performance, 40 (2014) 342-360. To explore these task differences, we used grayscale pictures of real-world scenes which included a target letter (Experiment 1: T, Experiment 2: T or L). To reduce between-scene variability with regard to target salience, we developed the Target Embedding Algorithm (T.E.A.) to place the letter in a location for which there was a median change in local contrast when inserting the letter into the scene. The presence or absence of foveal vision was crossed with four target sizes. In both experiments, search performance decreased for smaller targets, and was impaired when searching the scene without foveal vision. For correct trials, the process of target localization remained completely unimpaired by the foveal scotoma, but it took longer to accept the target. We reasoned that the size of the target may affect the importance of foveal vision to the task, but the present data remain ambiguous. In summary, the data highlight the importance of extrafoveal vision for target localization, and the importance of foveal vision for target verification during letter-in-scene search.

How neuroscience can inform the study of individual differences in cognitive abilities

Theories of human mental abilities should be consistent with what is known in neuroscience. Currently, tests of human mental abilities are modeled by cognitive constructs such as attention, working memory, and speed of information processing. These constructs are in turn related to a single general ability. However, brains are very complex systems and whether most of the variability between the operations of different brains can be ascribed to a single factor is questionable. Research in neuroscience suggests that psychological processes such as perception, attention, decision, and executive control are emergent properties of interacting distributed networks. The modules that make up these networks use similar computational processes that involve multiple forms of neural plasticity, each having different time constants. Accordingly, these networks might best be characterized in terms of the information they process rather than in terms of abstract psychological processes such as working memory and executive control.

The FVF framework and target prevalence effects

The Functional Visual Field (FVF) offers explanatory power. To us, it relates to existing literature on the flexibility of attentional focus in visual search and reading (Eriksen & St. James 1986; McConkie & Rayner 1975). The target article promotes reflection on existing findings. Here we consider the FVF as a mechanism in the Prevalence Effect (PE) in visual search.

The Utility of Mobile Eye Tracking Technology in the Forensic Analysis of Personal Injury

Early research on gaze patterns during human locomotion indicated that pedestrians tend to focus their gaze on the path 2 – 3 m ahead. More recent research in this area has utilized eye tracking technology to empirically assess eye movements during navigation in naturalistic environments. A similar approach to understanding personal injuries resulting in civil lawsuits has potential utility in the field of forensic human factors. The present study was conducted in the context of gathering empirical data to support expert opinions in a premises liability civil suit. Using a mobile head-mounted eye tracker, the gaze patterns of five individuals was assessed as they walked two paths covering the area where the plaintiff in the subject suit was injured. The results of this analysis indicated that, consistent with previous research, participants fixated the path ahead. Additionally, the duration of fixations on their final destination tended to be longer than fixations on other objects or areas in the forward view. The present study demonstrates the utility of empirical analysis of conditions surrounding a personal injury using eye tracking technology.

Modeling Lag-2 Revisits to Understand Trade-Offs in Mixed Control of Fixation Termination During Visual Search

An important question about eye-movement behavior is when the decision is made to terminate a fixation and program the following saccade. Different approaches have found converging evidence in favor of a mixed-control account, in which there is some overlap between processing information at fixation and planning the following saccade. We examined one interesting instance of mixed control in visual search: lag-2 revisits, during which observers fixate a stimulus, move to a different stimulus, and then revisit the first stimulus on the next fixation. Results show that the probability of lag-2 revisits occurring increased with the number of target-similar stimuli, and revisits were preceded by a brief fixation on the intervening distractor stimulus. We developed the Efficient Visual Sampling (EVS) computational model to simulate our findings (fixation durations and fixation locations) and to provide insight into mixed control of fixations and the perceptual, cognitive, and motor processes that produce lag-2 revisits.

Estimating the divergence point: a novel distributional analysis procedure for determining the onset of the influence of experimental variables

The divergence point analysis procedure is aimed at obtaining an estimate of the onset of the influence of an experimental variable on response latencies (e.g., fixation duration, reaction time). The procedure involves generating survival curves for two conditions, and using a bootstrapping technique to estimate the timing of the earliest discernible divergence between curves. In the present paper, several key extensions for this procedure were proposed and evaluated by conducting simulations and by reanalyzing data from previous studies. Our findings indicate that the modified versions of the procedure performed substantially better than the original procedure under conditions of low experimental power. Furthermore, unlike the original procedure, the modified procedures provided divergence point estimates for individual participants and permitted testing the significance of the difference between estimates across conditions. The advantages of the modified procedures are illustrated, the theoretical and methodological implications are discussed, and promising future directions are outlined.

Expert vs. novice differences in the detection of relevant information during a chess game: evidence from eye movements

The present study explored the ability of expert and novice chess players to rapidly distinguish between regions of a chessboard that were relevant to the best move on the board, and regions of the board that were irrelevant. Accordingly, we monitored the eye movements of expert and novice chess players, while they selected white's best move for a variety of chess problems. To manipulate relevancy, we constructed two different versions of each chess problem in the experiment, and we counterbalanced these versions across participants. These two versions of each problem were identical except that a single piece was changed from a bishop to a knight. This subtle change reversed the relevancy map of the board, such that regions that were relevant in one version of the board were now irrelevant (and vice versa). Using this paradigm, we demonstrated that both the experts and novices spent more time fixating the relevant relative to the irrelevant regions of the board. However, the experts were faster at detecting relevant information than the novices, as shown by the finding that experts (but not novices) were able to distinguish between relevant and irrelevant information during the early part of the trial. These findings further demonstrate the domain-related perceptual processing advantage of chess experts, using an experimental paradigm that allowed us to manipulate relevancy under tightly controlled conditions.