What controls attention in natural environments?

How do drivers mitigate the effects of naturalistic visual complexity? : On attentional strategies and their implications under a change blindness protocol

How do the limits of high-level visual processing affect human performance in naturalistic, dynamic settings of (multimodal) interaction where observers can draw on experience to strategically adapt attention to familiar forms of complexity? In this backdrop, we investigate change detection in a driving context to study attentional allocation aimed at overcoming environmental complexity and temporal load. Results indicate that visuospatial complexity substantially increases change blindness but also that participants effectively respond to this load by increasing their focus on safety-relevant events, by adjusting their driving, and by avoiding non-productive forms of attentional elaboration, thereby also controlling "looked-but-failed-to-see" errors. Furthermore, analyses of gaze patterns reveal that drivers occasionally, but effectively, limit attentional monitoring and lingering for irrelevant changes. Overall, the experimental outcomes reveal how drivers exhibit effective attentional compensation in highly complex situations. Our findings uncover implications for driving education and development of driving skill-testing methods, as well as for human-factors guided development of AI-based driving assistance systems.

Peripheral vision in real-world tasks: A systematic review

Peripheral vision is fundamental for many real-world tasks, including walking, driving, and aviation. Nonetheless, there has been no effort to connect these applied literatures to research in peripheral vision in basic vision science or sports science. To close this gap, we analyzed 60 relevant papers, chosen according to objective criteria. Applied research, with its real-world time constraints, complex stimuli, and performance measures, reveals new functions of peripheral vision. Peripheral vision is used to monitor the environment (e.g., road edges, traffic signs, or malfunctioning lights), in ways that differ from basic research. Applied research uncovers new actions that one can perform solely with peripheral vision (e.g., steering a car, climbing stairs). An important use of peripheral vision is that it helps compare the position of one’s body/vehicle to objects in the world. In addition, many real-world tasks require multitasking, and the fact that peripheral vision provides degraded but useful information means that tradeoffs are common in deciding whether to use peripheral vision or move one’s eyes. These tradeoffs are strongly influenced by factors like expertise, age, distraction, emotional state, task importance, and what the observer already knows. These tradeoffs make it hard to infer from eye movements alone what information is gathered from peripheral vision and what tasks we can do without it. Finally, we recommend three ways in which basic, sport, and applied science can benefit each other’s methodology, furthering our understanding of peripheral vision more generally.

Gaze Strategies in Driving-An Ecological Approach

Human performance in natural environments is deeply impressive, and still much beyond current AI. Experimental techniques, such as eye tracking, may be useful to understand the cognitive basis of this performance, and "the human advantage." Driving is domain where these techniques may deployed, in tasks ranging from rigorously controlled laboratory settings through high-fidelity simulations to naturalistic experiments in the wild. This research has revealed robust patterns that can be reliably identified and replicated in the field and reproduced in the lab. The purpose of this review is to cover the basics of what is known about these gaze behaviors, and some of their implications for understanding visually guided steering. The phenomena reviewed will be of interest to those working on any domain where visual guidance and control with similar task demands is involved (e.g., many sports). The paper is intended to be accessible to the non-specialist, without oversimplifying the complexity of real-world visual behavior. The literature reviewed will provide an information base useful for researchers working on oculomotor behaviors and physiology in the lab who wish to extend their research into more naturalistic locomotor tasks, or researchers in more applied fields (sports, transportation) who wish to bring aspects of the real-world ecology under experimental scrutiny. Part of a Research Topic on Gaze Strategies in Closed Self-paced tasks, this aspect of the driving task is discussed. It is in particular emphasized why it is important to carefully separate the visual strategies driving (quite closed and self-paced) from visual behaviors relevant to other forms of driver behavior (an open-ended menagerie of behaviors). There is always a balance to strike between ecological complexity and experimental control. One way to reconcile these demands is to look for natural, real-world tasks and behavior that are rich enough to be interesting yet sufficiently constrained and well-understood to be replicated in simulators and the lab. This ecological approach to driving as a model behavior and the way the connection between "lab" and "real world" can be spanned in this research is of interest to anyone keen to develop more ecologically representative designs for studying human gaze behavior.

Effects of temporal and spatiotemporal cues on detection of dynamic road hazards

While driving, dangerous situations can occur quickly, and giving drivers extra time to respond may make the road safer for everyone. Extensive research on attentional cueing in cognitive psychology has shown that targets are detected faster when preceded by a spatially valid cue, and slower when preceded by an invalid cue. However, it is unknown how these standard laboratory-based cueing effects may translate to dynamic, real-world situations like driving, where potential targets (i.e., hazardous events) are inherently more complex and variable. Observers in our study were required to correctly localize hazards in dynamic road scenes across three cue conditions (temporal, spatiotemporal valid and spatiotemporal invalid), and a no-cue baseline. All cues were presented at the first moment the hazardous situation began. Both types of valid cues reduced reaction time (by 58 and 60 ms, respectively, with no significant difference between them, a larger effect than in many classic studies). In addition, observers’ ability to accurately localize hazards dropped 11% in the spatiotemporal invalid condition, a result with dangerous implications on the road. This work demonstrates that, in spite of this added complexity, classic cueing effects persist—and may even be enhanced—for the detection of real-world hazards, and that valid cues have the potential to benefit drivers on the road.

Does task-irrelevant music affect gaze allocation during real-world scene viewing?

Gaze control manifests from a dynamic integration of visual and auditory information, with sound providing important cues for how a viewer should behave. Some past research suggests that music, even if entirely irrelevant to the current task demands, may also sway the timing and frequency of fixations. The current work sought to further assess this idea as well as investigate whether task-irrelevant music could also impact how gaze is spatially allocated. In preparation for a later memory test, participants studied pictures of urban scenes in silence or while simultaneously listening to one of two types of music. Eye tracking was recorded, and nine gaze behaviors were measured to characterize the temporal and spatial aspects of gaze control. Findings showed that while these gaze behaviors changed over the course of viewing, music had no impact. Participants in the music conditions, however, did show better memory performance than those who studied in silence. These findings are discussed within theories of multimodal gaze control.

Hazard Perception–Response: A Theoretical Framework to Explain Drivers’ Interactions with Roadway Hazards

Research suggests that novice drivers are most susceptible to errors when detecting and responding to hazards. If this were true, then hazard training should be effective in improving novice drivers’ performance. However, there is limited evidence to support this effectiveness. Much of this research has overlooked a fundamental aspect of psychological research: theory. Although four theoretical frameworks were developed to explain this process, none have been validated. We proposed a theoretical framework to more accurately explain drivers’ behavior when interacting with hazardous situations. This framework is novel in that it leverages support from visual attention and driving behavior research. Hazard-related constructs are defined and suitable metrics to evaluate the stages in hazard processing are suggested. Additionally, individual differences which affect hazard-related skills are also discussed. This new theoretical framework may explain why the conflicts in current hazard-related research fail to provide evidence that training such behaviors reduces crash risk. Future research is necessary to empirically test this framework.

Carry-over of attentional settings between distinct tasks: A transient effect independent of top-down contextual biases

Top-down attentional settings can persist between two unrelated tasks, influencing visual attention and performance. This study investigated whether top-down contextual information in a second task could moderate this "attentional inertia" effect. Forty participants searched through letter strings arranged horizontally, vertically, or randomly and then made a judgement about road, nature, or fractal images. Eye movements were recorded to the picture search and findings showed greater horizontal search in the pictures following horizontal letter strings and narrower horizontal search following vertical letter strings, but only in the first 1000 ms. This shows a brief persistence of attentional settings, consistent with past findings. Crucially, attentional inertia did not vary according to image type. This indicates that top-down contextual biases within a scene have limited impact on the persistence of previously relevant, but now irrelevant, attentional settings.

Visual search behavior and performance in luggage screening: effects of time pressure, automation aid, and target expectancy

Visual inspection of luggage using X-ray technology at airports is a time-sensitive task that is often supported by automated systems to increase performance and reduce workload. The present study evaluated how time pressure and automation support influence visual search behavior and performance in a simulated luggage screening task. Moreover, we also investigated how target expectancy (i.e., targets appearing in a target-often location or not) influenced performance and visual search behavior. We used a paradigm where participants used the mouse to uncover a portion of the screen which allowed us to track how much of the stimulus participants uncovered prior to their decision. Participants were randomly assigned to either a high (5-s time per trial) or a low (10-s time per trial) time-pressure condition. In half of the trials, participants were supported by an automated diagnostic aid (85% reliability) in deciding whether a threat item was present. Moreover, within each half, in target-present trials, targets appeared in a predictable location (i.e., 70% of targets appeared in the same quadrant of the image) to investigate effects of target expectancy. The results revealed better detection performance with low time pressure and faster response times with high time pressure. There was an overall negative effect of automation support because the automation was only moderately reliable. Participants also uncovered a smaller amount of the stimulus under high time pressure in target-absent trials. Target expectancy of target location improved accuracy, speed, and the amount of uncovered space needed for the search.Significance Statement Luggage screening is a safety-critical real-world visual search task which often has to be done under time pressure. The present research found that time pressure compromises performance and increases the risk to miss critical items even with automation support. Moreover, even highly reliable automated support may not improve performance if it does not exceed the manual capabilities of the human screener. Lastly, the present research also showed that heuristic search strategies (e.g., areas where targets appear more often) seem to guide attention also in luggage screening.

How does the number of targets affect visual search performance in visuospatial neglect?

INTRODUCTION

Impairments in visual search are a common symptom in visuospatial neglect (VSN). The severity of the lateralized attention bias in visual search tasks can vary depending on the number of distractors: the more distractors, the more targets are missed. However, little is known about how the number of targets affect search performance in VSN. The aim of the current study was to examine the effect of the number of targets on hit rate in VSN.

METHODS

We included 23 stroke patients with right-brain damage and VSN, 55 with right-brain damage without VSN, and 49 with left-brain damage without VSN, all admitted for inpatient rehabilitation. In a visual search task, patients had to find and tap targets, presented along with non-targets. The location and number of targets varied from trial to trial, allowing the evaluation of the effects of number and location of targets on hit rate.

RESULTS

VSN patients detected a lower percentage of targets when more targets were present. For patients with right-brain damage without VSN, adding targets only reduced the hit rate of the most contralesional target. No effect of number of targets on hit rate was seen in patients with left-brain damage. Additionally, VSN patients found less contralesional targets than ipsilesional targets, made more delayed revisits, and had an initial rightward bias when compared to the other groups. There were no differences in search time, search consistency, or immediate revisits between groups. There was a moderate positive relation between the hit rate asymmetry score in our search task and conventional paper-and-pencil VSN tasks, and neglect behavior in daily life.

CONCLUSIONS

In VSN patients, a higher number of targets reduces the hit rate. The reduced hit rate in visual search evoked by additional targets should be taken into account when assessing visual search in VSN.

Where the eyes wander: The relationship between mind wandering and fixation allocation to visually salient and semantically informative static scene content

Vision is crucial for many everyday activities, but the mind is not always focused on what the eyes see. Mind wandering occurs frequently and is associated with attenuated visual and cognitive processing of external information. Corresponding changes in gaze behavior—namely, fewer, longer, and more dispersed fixations—suggest a shift in how the visual system samples external information. Using three computational models of visual salience and two innovative approaches for measuring semantic informativeness, the current work assessed whether these changes reflect how the visual system prioritizes visually salient and semantically informative scene content, two major determinants in most theoretical frameworks and computational models of gaze control. Findings showed that, in a static scene viewing task, fixations were allocated to scene content that was more visually salient 10 seconds prior to probe-caught, self-reported mind wandering compared to self-reported attentive viewing. The relationship between mind wandering and semantic content was more equivocal, with weaker evidence that fixations are more likely to fall on locally informative scene regions. This indicates that the visual system is still able to discriminate visually salient and semantically informative scene content during mind wandering and may fixate on such information more frequently than during attentive viewing. Theoretical implications are discussed in light of these findings.

Digit eyes: Learning-related changes in information access in a computer game parallel those of oculomotor attention in laboratory studies

Active sensing theory is founded upon the dynamic relationship between information sampling and an observer's evolving goals. Oculomotor activity is a well studied method of sampling; a mouse or a keyboard can also be used to access information past the current screen. We examine information access patterns of StarCraft 2 players at multiple skill levels. The first measures are analogous to existing eye-movement studies: fixation frequency, fixation targets, and fixation duration all change as a function of skill, and are commensurate with known properties of eye movements in learning. Actions that require visual attention at moderate skill levels are eventually performed with little visual attention at all. This (a) confirms the generalizability of laboratory studies of attention and learning using eye movements to digital interface use, and (b) suggests that a wide variety of information access behaviors may be considered as a unified set of phenomena.

Children flexibly seek visual information to support signed and spoken language comprehension

During grounded language comprehension, listeners must link the incoming linguistic signal to the visual world despite uncertainty in the input. Information gathered through visual fixations can facilitate understanding. But do listeners flexibly seek supportive visual information? Here, we propose that even young children can adapt their gaze and actively gather information for the goal of language comprehension. We present 2 studies of eye movements during real-time language processing, where the value of fixating on a social partner varies across different contexts. First, compared with children learning spoken English (n = 80), young American Sign Language (ASL) learners (n = 30) delayed gaze shifts away from a language source and produced a higher proportion of language-consistent eye movements. This result provides evidence that ASL learners adapt their gaze to effectively divide attention between language and referents, which both compete for processing via the visual channel. Second, English-speaking preschoolers (n = 39) and adults (n = 31) fixated longer on a speaker's face while processing language in a noisy auditory environment. Critically, like the ASL learners in Experiment 1, this delay resulted in gathering more visual information and a higher proportion of language-consistent gaze shifts. Taken together, these studies suggest that young listeners can adapt their gaze to seek visual information from social partners to support real-time language comprehension. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A top-down saliency model with goal relevance

Most visual saliency models that integrate top-down factors process task and context information using machine learning techniques. Although these methods have been successful in improving prediction accuracy for human attention, they require significant training data and are unable to provide an understanding of what makes information relevant to a task such that it will attract gaze. This means that we still lack a general theory for the interaction between task and attention or eye movements. Recently, Tanner and Itti (2017) proposed the theory of goal relevance to explain what makes information relevant to goals. In this work, we record eye movements of 80 participants who each played one of four variants of a Mario video game and construct a combined saliency model using features from three sources: bottom-up, learned top-down, and goal relevance. We use this model to predict the eye behavior and find that the addition of goal relevance significantly improves the Normalized Scanpath Saliency score of the model from 4.35 to 5.82 (p < 1 × 10-100).

Natural visual behavior in individuals with peripheral visual-field loss

Retinitis pigmentosa (RP) is an inherited disease that causes progressive peripheral visual-field loss. In this study, we investigated how such loss affects visual exploration of natural images. Individuals with varying degrees of visual-field loss and healthy control participants freely observed images of different sizes while eye movements were recorded. We examined whether visual behavior differed when the scene content was shown in various extents of the visual field, and investigated the spatial bias, saccade amplitudes, and number and duration of fixations. We found that the healthy control group showed a central spatial bias during image viewing. The RP group showed similar biases on the group level, but with reproducible individual exploration patterns. For saccade amplitudes, the healthy control group and the RP group showed similar behavior throughout all image sizes. The RP group with severe loss of peripheral vision thus tended to target saccades toward blind areas of their visual field. The number of fixations did not change between the two groups, although fixation durations decreased in the RP group. In conclusion, the RP group scanned the images surprisingly similarly to the healthy control group; however, they showed individual idiosyncratic explorative strategies when the observed scene exceeded their visible field. Thus, although RP leads to a severe loss of the visual field, there is no general adaptive mechanism to change visual exploration. Instead, individuals rely on individual strategies, leading to high heterogeneity in the RP group.

Are longer advertising slogans more dangerous? The influence of the length of ad slogans on drivers’ attention and motor behavior

The purpose of this project was to verify whether slogans displayed on roadside advertisements created a distraction for drivers. In order to explain the mechanisms underlying this phenomenon, Study 1 examined the impact of slogan length on attentional processing efficiency. Study 2 investigated the relationship between the length of the slogan and the motor behavior of respondents driving a car simulator. We assumed that slogan length would decrease the drivers’ task performance in both studies. Study 1 was conducted on a group of 70 participants, who completed a modified version of the Attention Network Test (ANT; Fan et al. Journal of Cognitive Neuroscience, 14(3), 340–347, 2002). The task consisted of visual ads intended to distract respondents from the primary task. Reaction times were significantly longer when ads had longer slogans compared to shorter slogans. Study 2, involving a car simulator, was conducted on a group of 53 drivers performing a task of driving in a convoy. Participants were shown billboards with ads at the roadside in random order (two pairs of corresponding ads with short and long text on it). Participants’ driving performance decreased when longer slogans were presented in comparison to the short-slogan condition. In conclusion, we interpret the results of Study 1 to indicate that longer slogan leads to a greater load of attentional processing. This subsequently leads to a reduction of the processing efficiency within attentional systems and further increases the amount of time to resist the irrelevant stimulus. A consequence of this, as shown in Study 2 using a car simulator, is a decrease in cognitive resources necessary for safe driving and therefore worse performance on a driving task.

What Does It Take to Search Organized? The Cognitive Correlates of Search Organization During Cancellation After Stroke

OBJECTIVES

Stroke could lead to deficits in organization of visual search. Cancellation tests are frequently used in standard neuropsychological assessment and appear suitable to measure search organization. The current aim was to evaluate which cognitive functions are associated with cancellation organization measures after stroke.

METHODS

Stroke patients admitted to inpatient rehabilitation were included in this retrospective study. We performed exploratory factor analyses to explore cognitive domains. A digital shape cancellation test (SC) was administered, and measures of search organization (intersections rate and best r) were computed. The following cognitive functions were measured by neuropsychological testing: neglect (SC, line bisection; LB, Catherine Bergego Scale; CBS, and Balloons Test), visuospatial perception and construction (Rey Complex Figure Test, RCFT), psychomotor speed (Trail Making Test; TMT-A), executive functioning/working memory (TMT-B), spatial planning (Tower Test), rule learning (Brixton Test), short-term auditory memory (Digit Span Forward; DSF), and verbal working memory (Digit Span Backward; DSB).

RESULTS

In total, 439 stroke patients were included in our analyses. Four clusters were separated: "Executive functioning" (TMT-A, TMT-B, Brixton Test, and Tower Test), "Verbal memory" (DSF and DSB), "Search organization" (intersections rate and best r), and "Neglect" (CBS, RCFT copy, Balloons Test, SC, and LB).

CONCLUSIONS

Search organization during cancellation, as measured with intersections rate and best r, seems a distinct cognitive construct compared to existing cognitive domains that are tested during neuropsychological assessment. Administering cancellation tests and analyzing measures of search organization could provide useful additional insights into the visuospatial processes of stroke patients. (JINS, 2018, 24, 424-436).

Do cyclists make better drivers? Associations between cycling experience and change detection in road scenes

Efficient processing of visual information is crucial to safe driving. Previous research has demonstrated that driving experience strongly affects attentional allocation, with large differences between novice and experienced drivers. Expanding on this, we explored the influence of non-driving experiences on attentional allocation by comparing drivers with and without cycling experience. Based on situation awareness field studies, we predicted cyclist-drivers would demonstrate superior performance. Participants were 42 experienced drivers (17 female, 25 male) aged 30-50 years (M=39.8): 20 drivers and 22 cyclist-drivers. The experiment used a change detection flicker task, in which participants must determine whether two alternating images are identical (change-absent) or differ in a single detail (change-present). The changed object was either a road sign, car, pedestrian, or bicycle. Change target significantly affected both accuracy and response time: all participants were slower and less accurate at detecting changes to road signs, compared with when the change was a moving road user (i.e., car, pedestrian, bicycle). Accuracy did not differ significantly between groups, but cyclist-drivers were significantly faster than drivers at identifying changes, with the effect being largest for bicycle and sign changes. The results suggest that cycling experience is associated with more efficient attentional processing for road scenes.

Systematic Observation of an Expert Driver's Gaze Strategy-An On-Road Case Study

In this paper we present and qualitatively analyze an expert driver's gaze behavior in natural driving on a real road, with no specific experimental task or instruction. Previous eye tracking research on naturalistic tasks has revealed recurring patterns of gaze behavior that are surprisingly regular and repeatable. Lappi (2016) identified in the literature seven “qualitative laws of gaze behavior in the wild”: recurring patterns that tend to go together, the more so the more naturalistic the setting, all of them expected in extended sequences of fully naturalistic behavior. However, no study to date has observed all in a single experiment. Here, we wanted to do just that: present observations supporting all the “laws” in a single behavioral sequence by a single subject. We discuss the laws in terms of unresolved issues in driver modeling and open challenges for experimental and theoretical development.

Change detection in urban and rural driving scenes: Effects of target type and safety relevance on change blindness

The ability to detect changes is crucial for safe driving. Previous research has demonstrated that drivers often experience change blindness, which refers to failed or delayed change detection. The current study explored how susceptibility to change blindness varies as a function of the driving environment, type of object changed, and safety relevance of the change. Twenty-six fully-licenced drivers completed a driving-related change detection task. Changes occurred to seven target objects (road signs, cars, motorcycles, traffic lights, pedestrians, animals, or roadside trees) across two environments (urban or rural). The contextual safety relevance of the change was systematically manipulated within each object category, ranging from high safety relevance (i.e., requiring a response by the driver) to low safety relevance (i.e., requiring no response). When viewing rural scenes, compared with urban scenes, participants were significantly faster and more accurate at detecting changes, and were less susceptible to "looked-but-failed-to-see" errors. Interestingly, safety relevance of the change differentially affected performance in urban and rural environments. In urban scenes, participants were more efficient at detecting changes with higher safety relevance, whereas in rural scenes the effect of safety relevance has marginal to no effect on change detection. Finally, even after accounting for safety relevance, change blindness varied significantly between target types. Overall the results suggest that drivers are less susceptible to change blindness for objects that are likely to change or move (e.g., traffic lights vs. road signs), and for moving objects that pose greater danger (e.g., wild animals vs. pedestrians).

VREX: an open-source toolbox for creating 3D virtual reality experiments

Background

We present VREX, a free open-source Unity toolbox for virtual reality research in the fields of experimental psychology and neuroscience.

Results

Different study protocols about perception, attention, cognition and memory can be constructed using the toolbox. VREX provides a procedural generation of (interconnected) rooms that can be automatically furnished with a click of a button. VREX includes a menu system for creating and storing experiments with different stages. Researchers can combine different rooms and environments to perform end-to-end experiments including different testing situations and data collection. For fine-tuned control VREX also comes with an editor where all the objects in the virtual room can be manually placed and adjusted in the 3D world.

Conclusions

VREX simplifies the generation and setup of complicated VR scenes and experiments for researchers. VREX can be downloaded and easily installed from vrex.mozello.com

Electronic supplementary material

The online version of this article (doi:10.1186/s40359-017-0173-4) contains supplementary material, which is available to authorized users.

A novel computational model to probe visual search deficits during motor performance

Successful execution of many motor skills relies on well-organized visual search (voluntary eye movements that actively scan the environment for task-relevant information). Although impairments of visual search that result from brain injuries are linked to diminished motor performance, the neural processes that guide visual search within this context remain largely unknown. The first objective of this study was to examine how visual search in healthy adults and stroke survivors is used to guide hand movements during the Trail Making Test (TMT), a neuropsychological task that is a strong predictor of visuomotor and cognitive deficits. Our second objective was to develop a novel computational model to investigate combinatorial interactions between three underlying processes of visual search (spatial planning, working memory, and peripheral visual processing). We predicted that stroke survivors would exhibit deficits in integrating the three underlying processes, resulting in deteriorated overall task performance. We found that normal TMT performance is associated with patterns of visual search that primarily rely on spatial planning and/or working memory (but not peripheral visual processing). Our computational model suggested that abnormal TMT performance following stroke is associated with impairments of visual search that are characterized by deficits integrating spatial planning and working memory. This innovative methodology provides a novel framework for studying how the neural processes underlying visual search interact combinatorially to guide motor performance.

NEW & NOTEWORTHY

Visual search has traditionally been studied in cognitive and perceptual paradigms, but little is known about how it contributes to visuomotor performance. We have developed a novel computational model to examine how three underlying processes of visual search (spatial planning, working memory, and peripheral visual processing) contribute to visual search during a visuomotor task. We show that deficits integrating spatial planning and working memory underlie abnormal performance in stroke survivors with frontoparietal damage.

Attention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli

Attentional engagement is a major determinant of how effectively we gather information through our senses. Alongside the sheer growth in the amount and variety of information content that we are presented with through modern media, there is increased variability in the degree to which we "absorb" that information. Traditional research on attention has illuminated the basic principles of sensory selection to isolated features or locations, but it provides little insight into the neural underpinnings of our attentional engagement with modern naturalistic content. Here, we show in human subjects that the reliability of an individual's neural responses with respect to a larger group provides a highly robust index of the level of attentional engagement with a naturalistic narrative stimulus. Specifically, fast electroencephalographic evoked responses were more strongly correlated across subjects when naturally attending to auditory or audiovisual narratives than when attention was directed inward to a mental arithmetic task during stimulus presentation. This effect was strongest for audiovisual stimuli with a cohesive narrative and greatly reduced for speech stimuli lacking meaning. For compelling audiovisual narratives, the effect is remarkably strong, allowing perfect discrimination between attentional state across individuals. Control experiments rule out possible confounds related to altered eye movement trajectories or order of presentation. We conclude that reliability of evoked activity reproduced across subjects viewing the same movie is highly sensitive to the attentional state of the viewer and listener, which is aided by a cohesive narrative.

The transference of visual search between two unrelated tasks: Measuring the temporal characteristics of carry-over

Investigations into the persistence of top-down control settings do not accurately reflect the nature of dynamic tasks. They typically involve extended practice with an initial task, and this initial task usually shares similar stimuli with a second task. Recent work shows that visual attention and search can be affected by limited exposure to a preceding, unrelated task, and the current study explored the temporal characteristics of this “carry-over” effect. Thirty-four participants completed one, four, or eight simple letter searches and then searched a natural scene. The spatial layout of letters influenced spread of search in the pictures, and this was further impacted by the time spent in the initial task, yet the carry-over effect diminished quickly. The results have implications for theories of top-down control and models that attempt to predict search in natural scenes. They are also relevant to real-world tasks in which performance is closely related to visual attention and search.

The Potential Utility of Eye Movements in the Detection and Characterization of Everyday Functional Difficulties in Mild Cognitive Impairment

Mild cognitive impairment (MCI) refers to the intermediate period between the typical cognitive decline of normal aging and more severe decline associated with dementia, and it is associated with greater risk for progression to dementia. Research has suggested that functional abilities are compromised in MCI, but the degree of impairment and underlying mechanisms remain poorly understood. The development of sensitive measures to assess subtle functional decline poses a major challenge for characterizing functional limitations in MCI. Eye-tracking methodology has been used to describe visual processes in everyday, naturalistic action among healthy older adults as well as several case studies of severely impaired individuals, and it has successfully differentiated healthy older adults from those with MCI on specific visual tasks. These studies highlight the promise of eye-tracking technology as a method to characterize subtle functional decline in MCI. However, to date no studies have examined visual behaviors during completion of naturalistic tasks in MCI. This review describes the current understanding of functional ability in MCI, summarizes findings of eye-tracking studies in healthy individuals, severe impairment, and MCI, and presents future research directions to aid with early identification and prevention of functional decline in disorders of aging.

Linking attentional processes and conceptual problem solving: visual cues facilitate the automaticity of extracting relevant information from diagrams

This study investigated links between visual attention processes and conceptual problem solving. This was done by overlaying visual cues on conceptual physics problem diagrams to direct participants' attention to relevant areas to facilitate problem solving. Participants (N = 80) individually worked through four problem sets, each containing a diagram, while their eye movements were recorded. Each diagram contained regions that were relevant to solving the problem correctly and separate regions related to common incorrect responses. Problem sets contained an initial problem, six isomorphic training problems, and a transfer problem. The cued condition saw visual cues overlaid on the training problems. Participants' verbal responses were used to determine their accuracy. This study produced two major findings. First, short duration visual cues which draw attention to solution-relevant information and aid in the organizing and integrating of it, facilitate both immediate problem solving and generalization of that ability to new problems. Thus, visual cues can facilitate re-representing a problem and overcoming impasse, enabling a correct solution. Importantly, these cueing effects on problem solving did not involve the solvers' attention necessarily embodying the solution to the problem, but were instead caused by solvers attending to and integrating relevant information in the problems into a solution path. Second, this study demonstrates that when such cues are used across multiple problems, solvers can automatize the extraction of problem-relevant information extraction. These results suggest that low-level attentional selection processes provide a necessary gateway for relevant information to be used in problem solving, but are generally not sufficient for correct problem solving. Instead, factors that lead a solver to an impasse and to organize and integrate problem information also greatly facilitate arriving at correct solutions.

The influence of early aging on eye movements during motor simulation

Movement based interventions such as imagery and action observation are used increasingly to support physical rehabilitation of adults during early aging. The efficacy of these more covert approaches is based on an intuitively appealing assumption that movement execution, imagery and observation share neural substrate; alteration of one influences directly the function of the other two. Using eye movement metrics this paper reports findings that question the congruency of the three conditions. The data reveal that simulating movement through imagery and action observation may offer older adults movement practice conditions that are not constrained by the age-related decline observed in physical conditions. In addition, the findings provide support for action observation as a more effective technique for movement reproduction in comparison to imagery. This concern for imagery was also seen in the less congruent temporal relationship in movement time between imagery and movement execution suggesting imagery inaccuracy in early aging.

Over the hill at 24: persistent age-related cognitive-motor decline in reaction times in an ecologically valid video game task begins in early adulthood

Typically studies of the effects of aging on cognitive-motor performance emphasize changes in elderly populations. Although some research is directly concerned with when age-related decline actually begins, studies are often based on relatively simple reaction time tasks, making it impossible to gauge the impact of experience in compensating for this decline in a real world task. The present study investigates age-related changes in cognitive motor performance through adolescence and adulthood in a complex real world task, the real-time strategy video game StarCraft 2. In this paper we analyze the influence of age on performance using a dataset of 3,305 players, aged 16-44, collected by Thompson, Blair, Chen & Henrey [1]. Using a piecewise regression analysis, we find that age-related slowing of within-game, self-initiated response times begins at 24 years of age. We find no evidence for the common belief expertise should attenuate domain-specific cognitive decline. Domain-specific response time declines appear to persist regardless of skill level. A second analysis of dual-task performance finds no evidence of a corresponding age-related decline. Finally, an exploratory analyses of other age-related differences suggests that older participants may have been compensating for a loss in response speed through the use of game mechanics that reduce cognitive load.

Attention as foraging for information and value

What is the purpose of attention? One avenue of research has led to the proposal that attention might be crucial for gathering information about the environment, while other lines of study have demonstrated how attention may play a role in guiding behavior to rewarded options. Many experiments that study attention require participants to make a decision based on information acquired discretely at one point in time. In real-world situations, however, we are usually not presented with information about which option to select in such a manner. Rather we must initially search for information, weighing up reward values of options before we commit to a decision. Here, we propose that attention plays a role in both foraging for information and foraging for value. When foraging for information, attention is guided toward the unknown. When foraging for reward, attention is guided toward high reward values, allowing decision-making to proceed by accept-or-reject decisions on the currently attended option. According to this account, attention can be regarded as a low-cost alternative to moving around and physically interacting with the environment—“teleforaging”—before a decision is made to interact physically with the world. To track the timecourse of attention, we asked participants to seek out and acquire information about two gambles by directing their gaze, before choosing one of them. Participants often made multiple refixations on items before making a decision. Their eye movements revealed that early in the trial, attention was guided toward information, i.e., toward locations that reduced uncertainty about value. In contrast, late in the trial, attention was guided by expected value of the options. At the end of the decision period, participants were generally attending to the item they eventually chose. We suggest that attentional foraging shifts from an uncertainty-driven to a reward-driven mode during the evolution of a decision, permitting decisions to be made by an engage-or-search strategy.

Image free-viewing as intrinsically-motivated exploration: estimating the learnability of center-of-gaze image samples in infants and adults

We propose that free viewing of natural images in human infants can be understood and analyzed as the product of intrinsically-motivated visual exploration. We examined this idea by first generating five sets of center-of-gaze (COG) image samples, which were derived by presenting a series of natural images to groups of both real observers (i.e., 9-month-olds and adults) and artificial observers (i.e., an image-saliency model, an image-entropy model, and a random-gaze model). In order to assess the sequential learnability of the COG samples, we paired each group of samples with a simple recurrent network, which was trained to reproduce the corresponding sequence of COG samples. We then asked whether an intrinsically-motivated artificial agent would learn to identify the most successful network. In Simulation 1, the agent was rewarded for selecting the observer group and network with the lowest prediction errors, while in Simulation 2 the agent was rewarded for selecting the observer group and network with the largest rate of improvement. Our prediction was that if visual exploration in infants is intrinsically-motivated-and more specifically, the goal of exploration is to learn to produce sequentially-predictable gaze patterns-then the agent would show a preference for the COG samples produced by the infants over the other four observer groups. The results from both simulations supported our prediction. We conclude by highlighting the implications of our approach for understanding visual development in infants, and discussing how the model can be elaborated and improved.

Selection of events in time enhances activity throughout early visual cortex

Temporal selection poses unique challenges to the perceptual system. Selection is needed to protect goal-relevant stimuli from interference from new sensory input. In addition, contextual information that occurs at the same time as goal-relevant stimuli may be critical for learning. Using fMRI, we characterized how visual cortical regions respond to the temporal selection of auditory and visual stimuli. Critically, we focused on brain regions that are not involved in processing the target itself. Participants pressed a button when they heard a prespecified target tone and did not respond to other tones. Although more attention was directed to auditory input when the target tone was selected, activity in primary visual cortex increased more after target tones than after distractor tones. In contrast to spatial attention, this effect was larger in V1 than in V2 and V3. It was present in regions not typically involved in representing the target stimulus. Additional experiments demonstrated that these effects were not due to multimodal processing, rare targets, or motor responses to the targets. Thus temporal selection of behaviorally relevant stimuli enhances, rather than reduces, activity in perceptual regions involved in processing other information.

Looking to score: the dissociation of goal influence on eye movement and meta-attentional allocation in a complex dynamic natural scene

Several studies have reported that task instructions influence eye-movement behavior during static image observation. In contrast, during dynamic scene observation we show that while the specificity of the goal of a task influences observers' beliefs about where they look, the goal does not in turn influence eye-movement patterns. In our study observers watched short video clips of a single tennis match and were asked to make subjective judgments about the allocation of visual attention to the items presented in the clip (e.g., ball, players, court lines, and umpire). However, before attending to the clips, observers were either told to simply watch clips (non-specific goal), or they were told to watch the clips with a view to judging which of the two tennis players was awarded the point (specific goal). The results of subjective reports suggest that observers believed that they allocated their attention more to goal-related items (e.g. court lines) if they performed the goal-specific task. However, we did not find the effect of goal specificity on major eye-movement parameters (i.e., saccadic amplitudes, inter-saccadic intervals, and gaze coherence). We conclude that the specificity of a task goal can alter observer's beliefs about their attention allocation strategy, but such task-driven meta-attentional modulation does not necessarily correlate with eye-movement behavior.

Negotiating left-hand and right-hand bends: a motorcycle simulator study to investigate experiential and behaviour differences across rider groups

Why do motorcyclists crash on bends? To address this question we examined the riding styles of three groups of motorcyclists on a motorcycle simulator. Novice, experienced and advanced motorcyclists navigated a series of combined left and right bends while their speed and lane position were recorded. Each rider encountered an unexpected hazard on both a left- and right-hand bend section. Upon seeing the hazards, all riders decreased their speed before steering to avoid the hazard. Experienced riders tended to follow more of a racing line through the bends, which resulted in them having to make the most severe changes to their position to avoid a collision. Advanced riders adopted the safest road positions, choosing a position which offered greater visibility through the bends. As a result, they did not need to alter their road position in response to the hazard. Novice riders adopted similar road positions to experienced riders on the left-hand bends, but their road positions were more similar to advanced riders on right-hand bends, suggesting that they were more aware of the risks associated with right bends. Novice riders also adopted a safer position on post-hazard bends whilst the experienced riders failed to alter their behaviour even though they had performed the greatest evasive manoeuvre in response to the hazards. Advanced riders did not need to alter their position as their approach to the bends was already optimal. The results suggest that non-advanced riders were more likely to choose an inappropriate lane position than an inappropriate speed when entering a bend. Furthermore, the findings support the theory that expertise is achieved as a result of relearning, with advanced training overriding 'bad habits' gained through experience alone.

Overt attention and context factors: the impact of repeated presentations, image type, and individual motivation

The present study investigated the dynamic of the attention focus during observation of different categories of complex scenes and simultaneous consideration of individuals' memory and motivational state. We repeatedly presented four types of complex visual scenes in a pseudo-randomized order and recorded eye movements. Subjects were divided into groups according to their motivational disposition in terms of action orientation and individual rating of scene interest.

Statistical analysis of eye-tracking data revealed that the attention focus successively became locally expressed by increasing fixation duration; decreasing saccade length, saccade frequency, and single subject's fixation distribution over images; and increasing inter-subject variance of fixation distributions. The validity of these results was supported by verbal reports. This general tendency was weaker for the group of subjects who rated the image set as interesting as compared to the other group. Additionally, effects were partly mediated by subjects' motivational disposition. Finally, we found a generally strong impact of image type on eye movement parameters. We conclude that motivational tendencies linked to personality as well as individual preferences significantly affected viewing behaviour. Hence, it is important and fruitful to consider inter-individual differences on the level of motivation and personality traits within investigations of attention processes. We demonstrate that future studies on memory's impact on overt attention have to deal appropriately with several aspects that had been out of the research focus until now.

Eye guidance in natural vision: reinterpreting salience

Models of gaze allocation in complex scenes are derived mainly from studies of static picture viewing. The dominant framework to emerge has been image salience, where properties of the stimulus play a crucial role in guiding the eyes. However, salience-based schemes are poor at accounting for many aspects of picture viewing and can fail dramatically in the context of natural task performance. These failures have led to the development of new models of gaze allocation in scene viewing that address a number of these issues. However, models based on the picture-viewing paradigm are unlikely to generalize to a broader range of experimental contexts, because the stimulus context is limited, and the dynamic, task-driven nature of vision is not represented. We argue that there is a need to move away from this class of model and find the principles that govern gaze allocation in a broader range of settings. We outline the major limitations of salience-based selection schemes and highlight what we have learned from studies of gaze allocation in natural vision. Clear principles of selection are found across many instances of natural vision and these are not the principles that might be expected from picture-viewing studies. We discuss the emerging theoretical framework for gaze allocation on the basis of reward maximization and uncertainty reduction.

Change blindness and inattentional blindness

Change blindness and inattentional blindness are both failures of visual awareness. Change blindness is the failure to notice an obvious change. Inattentional blindness is the failure to notice the existence of an unexpected item. In each case, we fail to notice something that is clearly visible once we know to look for it. Despite similarities, each type of blindness has a unique background and distinct theoretical implications. Here, we discuss the central paradigms used to explore each phenomenon in a historical context. We also outline the central findings from each field and discuss their implications for visual perception and attention. In addition, we examine the impact of task and observer effects on both types of blindness as well as common pitfalls and confusions people make while studying these topics. WIREs Cogni Sci 2011 2 529-546 DOI: 10.1002/wcs.130 For further resources related to this article, please visit the WIREs website.

Adaptive allocation of vision under competing task demands

Human behavior in natural tasks consists of an intricately coordinated dance of cognitive, perceptual, and motor activities. Although much research has progressed in understanding the nature of cognitive, perceptual, or motor processing in isolation or in highly constrained settings, few studies have sought to examine how these systems are coordinated in the context of executing complex behavior. Previous research has suggested that, in the course of visually guided reaching movements, the eye and hand are yoked, or linked in a nonadaptive manner. In this work, we report an experiment that manipulated the demands that a task placed on the motor and visual systems, and then examined in detail the resulting changes in visuomotor coordination. We develop an ideal actor model that predicts the optimal coordination of vision and motor control in our task. On the basis of the predictions of our model, we demonstrate that human performance in our experiment reflects an adaptive response to the varying costs imposed by our experimental manipulations. Our results stand in contrast to previous theories that have assumed a fixed control mechanism for coordinating vision and motor control in reaching behavior.

Scanning Behaviour in Natural Scenes is Influenced by a Preceding Unrelated Visual Search Task

Three experiments explored the transference of visual scanning behaviour between two unrelated tasks. Participants first viewed letters presented horizontally, vertically, or as a random array. They then viewed still images (experiments 1 and 2) or video clips (experiment 3) of driving scenes, under varying task conditions. Despite having no relevance to the driving images, layout of stimuli in the letter task influenced scanning behaviour in this subsequent task. In the still images, a vertical letter search increased vertical scanning, and in the dynamic clips, a horizontal letter search decreased vertical scanning. This indicated that (i) models of scanning behaviour should account for the influence of a preceding unrelated task; (ii) carry-over is modulated by demand in the current task; and (iii) in situations where particular scanning strategies are important for primary task performance (eg driving safety), secondary task information should be displayed in a manner likely to produce a congruent scanning strategy.

How the Interpretation of Drivers' Behavior in Virtual Environment Can Become a Road Design Tool: A Case Study

Driving is the result of a psychological process that translates data, signals and direct/indirect messages into behavior, which is continuously adapted to the exchange of varying stimuli between man, environment and vehicle. These stimuli are at times not perceived and at others perceived but not understood by the driver, even if they derive from tools (vertical signs, horizontal marking) specifically conceived for his safety. The result is unsafe behavior of vehicle drivers. For this reason, the road environment needs to be radically redesigned. The paper describes a research, based on real and virtual environment surveys, aimed to better understand drivers' action-reaction mechanisms inside different scenarios, in order to gain informations useful for a correct organization (design) of the road space. The driving simulator can help in developing, from road to laboratory, the study of new road design tools (geometrical, compositional, constructive ones, street furniture, etc.), because it can be used to evaluate solutions before their usefulness is proved on the road.

Credit assignment in multiple goal embodied visuomotor behavior

The intrinsic complexity of the brain can lead one to set aside issues related to its relationships with the body, but the field of embodied cognition emphasizes that understanding brain function at the system level requires one to address the role of the brain-body interface. It has only recently been appreciated that this interface performs huge amounts of computation that does not have to be repeated by the brain, and thus affords the brain great simplifications in its representations. In effect the brain's abstract states can refer to coded representations of the world created by the body. But even if the brain can communicate with the world through abstractions, the severe speed limitations in its neural circuitry mean that vast amounts of indexing must be performed during development so that appropriate behavioral responses can be rapidly accessed. One way this could happen would be if the brain used a decomposition whereby behavioral primitives could be quickly accessed and combined. This realization motivates our study of independent sensorimotor task solvers, which we call modules, in directing behavior. The issue we focus on herein is how an embodied agent can learn to calibrate such individual visuomotor modules while pursuing multiple goals. The biologically plausible standard for module programming is that of reinforcement given during exploration of the environment. However this formulation contains a substantial issue when sensorimotor modules are used in combination: The credit for their overall performance must be divided amongst them. We show that this problem can be solved and that diverse task combinations are beneficial in learning and not a complication, as usually assumed. Our simulations show that fast algorithms are available that allot credit correctly and are insensitive to measurement noise.