1
|
Dillmann J, Den Hartigh RJR, Kurpiers CM, Raisch FK, Kadrileev N, Cox RFA, De Waard D. Repeated conditionally automated driving on the road: How do drivers leave the loop over time? ACCIDENT; ANALYSIS AND PREVENTION 2023; 181:106927. [PMID: 36584619 DOI: 10.1016/j.aap.2022.106927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 10/07/2022] [Accepted: 12/08/2022] [Indexed: 06/17/2023]
Abstract
The goal of this on the road driving study was to investigate how drivers adapt their behavior when driving with conditional vehicle automation (SAE L3) on different occasions. Specifically, we focused on changes in how fast drivers took over control from automation and how their gaze off the road changed over time. On each of three consecutive days, 21 participants drove for 50 min, in a conditionally automated vehicle (Wizard of Oz methodology), on a typical German commuting highway. Over these rides the take-over behavior and gaze behavior were analyzed. The data show that drivers' reactions to non-critical, system initiated, take-overs took about 5.62 s and did not change within individual rides, but on average became 0.72 s faster over the three rides. After these self-paced take-over requests a final urgent take-over request was issued at the end of the third ride. In this scenario participants took over rapidly with an average of 5.28 s. This urgent take-over time was not found to be different from the self-paced take-over requests in the same ride. Regarding gaze behavior, participants' overall longest glance off the road and the percentage of time looked off the road increased within each ride, but stayed stable over the three rides. Taken together, our results suggest that drivers regularly leave the loop by gazing off the road, but multiple exposures to take-over situations in automated driving allow drivers to come back into loop faster.
Collapse
Affiliation(s)
- J Dillmann
- Department of Psychology, University of Groningen, Groningen, the Netherlands; BMW Group Research and Development, Munich, Germany.
| | - R J R Den Hartigh
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| | - C M Kurpiers
- BMW Group Research and Development, Munich, Germany
| | - F K Raisch
- BMW Group Research and Development, Munich, Germany
| | | | - R F A Cox
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| | - D De Waard
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| |
Collapse
|
2
|
Weingarten E, Davidai S, Barasch A. Who's on first? People asymmetrically attend to higher-ranked (vs. lower-ranked) competitors. JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY 2023. [DOI: 10.1016/j.jesp.2022.104405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
3
|
Gonçalves RC, Louw TL, Madigan R, Quaresma M, Romano R, Merat N. The effect of information from dash-based human-machine interfaces on drivers' gaze patterns and lane-change manoeuvres after conditionally automated driving. ACCIDENT; ANALYSIS AND PREVENTION 2022; 174:106726. [PMID: 35716544 DOI: 10.1016/j.aap.2022.106726] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 04/13/2022] [Accepted: 05/28/2022] [Indexed: 06/15/2023]
Abstract
The goal of this paper was to measure the effect of Human-Machine Interface (HMI) information and guidance on drivers' gaze and takeover behaviour during transitions of control from automation. The motivation for this study came from a gap in the literature, where previous research reports improved performance of drivers' takeover based on HMI information, without considering its effect on drivers' visual attention distribution, and how drivers also use the information available in the environment to guide their response. This driving simulator study investigated drivers' lane-changing behaviour after resumption of control from automation. Different levels of information were provided on a dash-based HMI, prior to each lane change, to investigate how drivers distribute their attention between the surrounding environment and the HMI. The difficulty of the lane change was also manipulated by controlling the position of approaching vehicles in drivers' offside lane. Results indicated that drivers' decision-making time was sensitive to the presence of nearby vehicles in the offside lane, but not directly influenced by the information on the HMI. In terms of gaze behaviour, the closer the position of vehicles in the offside lane, the longer drivers looked in that direction. Drivers looked more at the HMI, and less towards the road centre, when the HMI presented information about automation status, and included an advisory message indicating it was safe to change lane. Machine learning techniques showed a strong relationship between drivers' gaze to the information presented on the HMI, and decision-making time (DMT). These results contribute to our understanding of HMI design for automated vehicles, by demonstrating the attentional costs of an overly-informative HMI, and that drivers still rely on environmental information to perform a lane-change, even when the same information can be acquired by the HMI of the vehicle.
Collapse
Affiliation(s)
| | - Tyron L Louw
- Pontifical Catholic University of Rio de Janeiro, Brazil
| | - Ruth Madigan
- Pontifical Catholic University of Rio de Janeiro, Brazil
| | - Manuela Quaresma
- University of Leeds, Institute for Transport Studies, United Kingdom
| | - Richard Romano
- Pontifical Catholic University of Rio de Janeiro, Brazil
| | - Natasha Merat
- Pontifical Catholic University of Rio de Janeiro, Brazil
| |
Collapse
|
4
|
Chakraborty S, Samaras D, Zelinsky GJ. Weighting the factors affecting attention guidance during free viewing and visual search: The unexpected role of object recognition uncertainty. J Vis 2022; 22:13. [PMID: 35323870 PMCID: PMC8963662 DOI: 10.1167/jov.22.4.13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
The factors determining how attention is allocated during visual tasks have been studied for decades, but few studies have attempted to model the weighting of several of these factors within and across tasks to better understand their relative contributions. Here we consider the roles of saliency, center bias, target features, and object recognition uncertainty in predicting the first nine changes in fixation made during free viewing and visual search tasks in the OSIE and COCO-Search18 datasets, respectively. We focus on the latter-most and least familiar of these factors by proposing a new method of quantifying uncertainty in an image, one based on object recognition. We hypothesize that the greater the number of object categories competing for an object proposal, the greater the uncertainty of how that object should be recognized and, hence, the greater the need for attention to resolve this uncertainty. As expected, we found that target features best predicted target-present search, with their dominance obscuring the use of other features. Unexpectedly, we found that target features were only weakly used during target-absent search. We also found that object recognition uncertainty outperformed an unsupervised saliency model in predicting free-viewing fixations, although saliency was slightly more predictive of search. We conclude that uncertainty in object recognition, a measure that is image computable and highly interpretable, is better than bottom–up saliency in predicting attention during free viewing.
Collapse
Affiliation(s)
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY, USA.,
| | - Gregory J Zelinsky
- Department of Psychology, Stony Brook University, Stony Brook, NY, USA.,Department of Computer Science, Stony Brook University, Stony Brook, NY, USA.,
| |
Collapse
|
5
|
Jokinen JPP, Kujala T, Oulasvirta A. Multitasking in Driving as Optimal Adaptation Under Uncertainty. HUMAN FACTORS 2021; 63:1324-1341. [PMID: 32731763 PMCID: PMC8593310 DOI: 10.1177/0018720820927687] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/12/2019] [Accepted: 04/21/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE The objective was to better understand how people adapt multitasking behavior when circumstances in driving change and how safe versus unsafe behaviors emerge. BACKGROUND Multitasking strategies in driving adapt to changes in the task environment, but the cognitive mechanisms of this adaptation are not well known. Missing is a unifying account to explain the joint contribution of task constraints, goals, cognitive capabilities, and beliefs about the driving environment. METHOD We model the driver's decision to deploy visual attention as a stochastic sequential decision-making problem and propose hierarchical reinforcement learning as a computationally tractable solution to it. The supervisory level deploys attention based on per-task value estimates, which incorporate beliefs about risk. Model simulations are compared against human data collected in a driving simulator. RESULTS Human data show adaptation to the attentional demands of ongoing tasks, as measured in lane deviation and in-car gaze deployment. The predictions of our model fit the human data on these metrics. CONCLUSION Multitasking strategies can be understood as optimal adaptation under uncertainty, wherein the driver adapts to cognitive constraints and the task environment's uncertainties, aiming to maximize the expected long-term utility. Safe and unsafe behaviors emerge as the driver has to arbitrate between conflicting goals and manage uncertainty about them. APPLICATION Simulations can inform studies of conditions that are likely to give rise to unsafe driving behavior.
Collapse
Affiliation(s)
| | | | - Antti Oulasvirta
- Aalto University, Finland
- Finnish Center for Artificial Intelligence FCAI, Finland
| |
Collapse
|
6
|
Dillmann J, den Hartigh RJR, Kurpiers CM, Pelzer J, Raisch FK, Cox RFA, de Waard D. Keeping the driver in the loop through semi-automated or manual lane changes in conditionally automated driving. ACCIDENT; ANALYSIS AND PREVENTION 2021; 162:106397. [PMID: 34563644 DOI: 10.1016/j.aap.2021.106397] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 08/30/2021] [Accepted: 09/09/2021] [Indexed: 06/13/2023]
Abstract
In the current study we investigated if drivers of conditionally automated vehicles can be kept in the loop through lane change maneuvers. More specifically, we examined whether involving drivers in lane-changes during a conditionally automated ride can influence critical take-over behavior and keep drivers' gaze on the road. In a repeated measures driving simulator study (n = 85), drivers drove the same route three times, each trial containing four lane changes that were all either (1) automated, (2) semi-automated or (3) manual. Each ride ended with a critical take-over situation that could be solved by braking and/or steering. Critical take-over reactions were analyzed with a linear mixed model and parametric accelerated failure time survival analysis. As expected, semi-automated and manual lane changes throughout the ride led to 13.5% and 17.0% faster maximum deceleration compared to automated lane changes. Additionally, semi-automated and manual lane changes improved the quality of the take-over by significantly decreasing standard deviation of the steering wheel angle. Unexpectedly, drivers in the semi-automated condition were slowest to start the braking maneuver. This may have been caused by the drivers' confusion as to how the semi-automated system would react. Additionally, the percentage gaze off-the-road was significantly decreased by the semi-automated (6.0%) and manual (6.6%) lane changes. Taken together, the results suggest that semi-automated and manual transitions may be an alarm-free instrument which developers could use to help maintain drivers' perception-action loop and improve automated driving safety.
Collapse
Affiliation(s)
- J Dillmann
- Department of Psychology, University of Groningen, Groningen, the Netherlands; BMW Group Research and Development, Munich, Germany.
| | - R J R den Hartigh
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| | - C M Kurpiers
- BMW Group Research and Development, Munich, Germany
| | - J Pelzer
- Institut für Psychologie, RWTH Aachen, Aachen, Germany
| | - F K Raisch
- BMW Group Research and Development, Munich, Germany
| | - R F A Cox
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| | - D de Waard
- Department of Psychology, University of Groningen, Groningen, the Netherlands
| |
Collapse
|
7
|
Nowakowska A, Clarke ADF, von Seth J, Hunt AR. Search strategies improve with practice, but not with time pressure or financial incentives. J Exp Psychol Hum Percept Perform 2021; 47:1009-1021. [PMID: 34424028 DOI: 10.1037/xhp0000912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
When searching for an object, do we minimize the number of eye movements we need to make? Under most circumstances, the cost of saccadic parsimony likely outweighs the benefit, given the cost is extensive computation and the benefit is a few hundred milliseconds of time saved. Previous research has measured the proportion of eye movements directed to locations where the target would have been visible in the periphery as a way of quantifying the proportion of superfluous fixations. A surprisingly large range of individual differences has emerged from these studies, suggesting some people are highly efficient and others much less so. Our question in the current study is whether these individual differences can be explained by differences in motivation. In two experiments, we demonstrate that neither time pressure nor financial incentive led to improvements of visual search strategies; the majority of participants continued to make many superfluous fixations in both experiments. The wide range of individual differences in efficiency observed previously was replicated here. We observed small but consistent improvements in strategy over the course of the experiment (regardless of reward or time pressure) suggesting practice, not motivation, makes participants more efficient. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
|
8
|
Kopiske K, Koska D, Baumann T, Maiwald C, Einhäuser W. Icy road ahead-rapid adjustments of gaze-gait interactions during perturbed naturalistic walking. J Vis 2021; 21:11. [PMID: 34351396 PMCID: PMC8354071 DOI: 10.1167/jov.21.8.11] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Most humans can walk effortlessly across uniform terrain even when they do not pay much attention to it. However, most natural terrain is far from uniform, and we need visual information to maintain stable gait. Recent advances in mobile eye-tracking technology have made it possible to study, in natural environments, how terrain affects gaze and thus the sampling of visual information. However, natural environments provide only limited experimental control, and some conditions cannot safely be tested. Typical laboratory setups, in contrast, are far from natural settings for walking. We used a setup consisting of a dual-belt treadmill, 240∘ projection screen, floor projection, three-dimensional optical motion tracking, and mobile eye tracking to investigate eye, head, and body movements during perturbed and unperturbed walking in a controlled yet naturalistic environment. In two experiments (N = 22 each), we simulated terrain difficulty by repeatedly inducing slipping through accelerating either of the two belts rapidly and unpredictably (Experiment 1) or sometimes following visual cues (Experiment 2). We quantified the distinct roles of eye and head movements for adjusting gaze on different time scales. While motor perturbations mainly influenced head movements, eye movements were primarily affected by the presence of visual cues. This was true both immediately following slips and—to a lesser extent—over the course of entire 5-min blocks. We find adapted gaze parameters already after the first perturbation in each block, with little transfer between blocks. In conclusion, gaze–gait interactions in experimentally perturbed yet naturalistic walking are adaptive, flexible, and effector specific.
Collapse
Affiliation(s)
- Karl Kopiske
- Cognitive Systems Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany.,
| | - Daniel Koska
- Group "Research Methodology and Data Analysis in Biomechanics," Institute of Human Movement Science and Health, Chemnitz University of Technology, Chemnitz, Germany.,
| | - Thomas Baumann
- Cognitive Systems Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany.,
| | - Christian Maiwald
- Group "Research Methodology and Data Analysis in Biomechanics," Institute of Human Movement Science and Health, Chemnitz University of Technology, Chemnitz, Germany.,
| | - Wolfgang Einhäuser
- Physics of Cognition Group, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany.,
| |
Collapse
|
9
|
Wang JZ, Kowler E. Micropursuit and the control of attention and eye movements in dynamic environments. J Vis 2021; 21:6. [PMID: 34347019 PMCID: PMC8340658 DOI: 10.1167/jov.21.8.6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
It is more challenging to plan eye movements during perceptual tasks performed in dynamic displays than in static displays. Decisions about the timing of saccades become more critical, and decisions must also involve smooth eye movements, as well as saccades. The present study examined eye movements when judging which of two moving discs would arrive first, or collide, at a common meeting point. Perceptual discrimination after training was precise (Weber fractions < 6%). Strategies reflected a combined contribution of saccades and smooth eye movements. The preferred strategy was to look near the meeting point when strategies were freely chosen. When strategies were assigned, looking near the meeting point produced better performance than switching between the discs. Smooth eye movements were engaged in two ways: (a) low-velocity smooth eye movements correlated with the motion of each disc (micropursuit) were found while the line of sight remained between the discs; and (b) spontaneous smooth pursuit of the pair of discs occurred after the perceptual report, when the discs moved as a pair along a common path. The results show clear preferences and advantages for those eye movement strategies during dynamic perceptual tasks that require minimal management or effort. In addition, smooth eye movements, whose involvement during perceptual tasks within dynamic displays may have previously escaped notice, provide useful indictors of the strategies used to select information and distribute attention during the performance of dynamic perceptual tasks.
Collapse
Affiliation(s)
- Jie Z Wang
- Department of Psychology, Rutgers University, Piscataway, NJ, USA.,http://orcid.org/0000-0002-8553-6706.,
| | - Eileen Kowler
- Department of Psychology, Rutgers University, Piscataway, NJ, USA.,http://orcid.org/0000-0001-7079-0376., https://ruccs.rutgers.edu/kowler
| |
Collapse
|
10
|
Wolf C, Lappe M. Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements. Cogn Neurodyn 2021; 15:547-568. [PMID: 34367360 PMCID: PMC8286912 DOI: 10.1007/s11571-020-09661-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 12/12/2020] [Accepted: 12/28/2020] [Indexed: 01/08/2023] Open
Abstract
Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets' luminance but also crucially on high-level factors like the expected reward or a targets' relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.
Collapse
Affiliation(s)
- Christian Wolf
- Institute for Psychology, University of Muenster, Fliednerstrasse 21, 48149 Münster, Germany
| | - Markus Lappe
- Institute for Psychology, University of Muenster, Fliednerstrasse 21, 48149 Münster, Germany
| |
Collapse
|
11
|
Saeedpour-Parizi MR, Hassan SE, Azad A, Baute KJ, Baniasadi T, Shea JB. Target position and avoidance margin effects on path planning in obstacle avoidance. Sci Rep 2021; 11:15285. [PMID: 34315936 PMCID: PMC8316463 DOI: 10.1038/s41598-021-94638-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 07/12/2021] [Indexed: 02/07/2023] Open
Abstract
This study examined how people choose their path to a target, and the visual information they use for path planning. Participants avoided stepping outside an avoidance margin between a stationary obstacle and the edge of a walkway as they walked to a bookcase and picked up a target from different locations on a shelf. We provided an integrated explanation for path selection by combining avoidance margin, deviation angle, and distance to the obstacle. We found that the combination of right and left avoidance margins accounted for 26%, deviation angle accounted for 39%, and distance to the obstacle accounted for 35% of the variability in decisions about the direction taken to circumvent an obstacle on the way to a target. Gaze analysis findings showed that participants directed their gaze to minimize the uncertainty involved in successful task performance and that gaze sequence changed with obstacle location. In some cases, participants chose to circumvent the obstacle on a side for which the gaze time was shorter, and the path was longer than for the opposite side. Our results of a path selection judgment test showed that the threshold for participants abandoning their preferred side for circumventing the obstacle was a target location of 15 cm to the left of the bookcase shelf center.
Collapse
Affiliation(s)
- Mohammad R. Saeedpour-Parizi
- grid.411377.70000 0001 0790 959XDepartment of Kinesiology, School of Public Health, Indiana University Bloomington, 1025 E 7th Street, Bloomington, IN 47405 USA ,grid.411377.70000 0001 0790 959XDepartment of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN USA
| | - Shirin E. Hassan
- grid.411377.70000 0001 0790 959XSchool of Optometry, Indiana University Bloomington, Bloomington, IN USA
| | - Ariful Azad
- grid.411377.70000 0001 0790 959XDepartment of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN USA
| | | | - Tayebeh Baniasadi
- grid.411377.70000 0001 0790 959XDepartment of Kinesiology, School of Public Health, Indiana University Bloomington, 1025 E 7th Street, Bloomington, IN 47405 USA
| | - John B. Shea
- grid.411377.70000 0001 0790 959XDepartment of Kinesiology, School of Public Health, Indiana University Bloomington, 1025 E 7th Street, Bloomington, IN 47405 USA
| |
Collapse
|
12
|
Drivers use active gaze to monitor waypoints during automated driving. Sci Rep 2021; 11:263. [PMID: 33420150 PMCID: PMC7794576 DOI: 10.1038/s41598-020-80126-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 12/14/2020] [Indexed: 11/08/2022] Open
Abstract
Automated vehicles (AVs) will change the role of the driver, from actively controlling the vehicle to primarily monitoring it. Removing the driver from the control loop could fundamentally change the way that drivers sample visual information from the scene, and in particular, alter the gaze patterns generated when under AV control. To better understand how automation affects gaze patterns this experiment used tightly controlled experimental conditions with a series of transitions from 'Manual' control to 'Automated' vehicle control. Automated trials were produced using either a 'Replay' of the driver's own steering trajectories or standard 'Stock' trials that were identical for all participants. Gaze patterns produced during Manual and Automated conditions were recorded and compared. Overall the gaze patterns across conditions were very similar, but detailed analysis shows that drivers looked slightly further ahead (increased gaze time headway) during Automation with only small differences between Stock and Replay trials. A novel mixture modelling method decomposed gaze patterns into two distinct categories and revealed that the gaze time headway increased during Automation. Further analyses revealed that while there was a general shift to look further ahead (and fixate the bend entry earlier) when under automated vehicle control, similar waypoint-tracking gaze patterns were produced during Manual driving and Automation. The consistency of gaze patterns across driving modes suggests that active-gaze models (developed for manual driving) might be useful for monitoring driver engagement during Automated driving, with deviations in gaze behaviour from what would be expected during manual control potentially indicating that a driver is not closely monitoring the automated system.
Collapse
|
13
|
Bruning AL, Lewis-Peacock JA. Long-term memory guides resource allocation in working memory. Sci Rep 2020; 10:22161. [PMID: 33335170 PMCID: PMC7747625 DOI: 10.1038/s41598-020-79108-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Accepted: 12/01/2020] [Indexed: 01/02/2023] Open
Abstract
Working memory capacity is incredibly limited and thus it is important to use this resource wisely. Prior knowledge in long-term memory can aid in efficient encoding of information by allowing for the prioritization of novel stimuli over familiar ones. Here we used a full-report procedure in a visual working memory paradigm, where participants reported the location of six colored circles in any order, to examine the influence of prior information on resource allocation in working memory. Participants learned that one of the items appeared in a restricted range of locations, whereas the remaining items could appear in any location. We found that participants' memory performance benefited from learning this prior information. Specifically, response precision increased for all items when prior information was available for one of the items. Responses for both familiar and novel items were systematically ordered from highest to lowest precision. Participants tended to report the familiar item in the second half of the six responses and did so with greater precision than for novel items. Moreover, novel items that appeared near the center of the prior location were reported with worse precision than novel items that appeared elsewhere. This shows that people strategically allocated working memory resources by ignoring information that appeared in predictable locations and prioritizing the encoding of information that appeared in unpredictable locations. Together these findings demonstrate that people rely on long-term memory not only for remembering familiar items, but also for the strategic allocation of their limited capacity working memory resources.
Collapse
Affiliation(s)
- Allison L Bruning
- Department of Psychology, Center for Learning and Memory, University of Texas at Austin, 108 E Dean Keeton St, Stop A8000, Austin, TX, 78712, USA.
| | - Jarrod A Lewis-Peacock
- Department of Psychology, Center for Learning and Memory, University of Texas at Austin, 108 E Dean Keeton St, Stop A8000, Austin, TX, 78712, USA
| |
Collapse
|
14
|
Goncalves RC, Louw TL, Quaresma M, Madigan R, Merat N. The effect of motor control requirements on drivers' eye-gaze pattern during automated driving. ACCIDENT; ANALYSIS AND PREVENTION 2020; 148:105788. [PMID: 33039820 DOI: 10.1016/j.aap.2020.105788] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2020] [Revised: 08/19/2020] [Accepted: 09/14/2020] [Indexed: 06/11/2023]
Abstract
This driving simulator study compared drivers' eye movements during a series of lane-changes, which required different levels of motor control for their execution. Participants completed 12 lane-changing manoeuvres in three drives, categorised by degree of manual engagement with the driving task: Fully Manual Drive, Manual Intervention Required, Fully Automated Drive (Manual drive, Partial automation, Full automation). For Partial automation, drivers resumed control from the automated system and changed lane manually. For Full automation, the automated system managed the lane change, but participants initiated the manoeuvre by pulling the indicator lever. Results were compared to the Manual drive condition, where drivers controlled the vehicle at all times. For each driving condition, lane changing was initiated by drivers, at their discretion, in response to a slow-moving lead vehicle, which entered their lane. Failure to change lane did not result in a collision. To understand how different motor control requirements affected driver visual attention, eye movements to the road centre, and drivers' vertical and horizontal gaze dispersion were compared during different stages of the lane change manoeuvre, for the three drives. Results showed that drivers' attention to the road centre was generally lower for drives with less motor control requirements, especially when they were not engaged in the lane change process. However, as drivers moved closer to the lead vehicle, and prepared to change lane, the pattern of eye movements to the road centre converged, regardless of whether drivers were responsible for the manual control of the lane change. While there were no significant differences in horizontal gaze dispersion between the three drives, vertical dispersion for the two levels of automation was quite different, with higher dispersion during Partial automation, which was due to a higher reliance on the HMI placed in the centre console.
Collapse
Affiliation(s)
- Rafael C Goncalves
- University of Leeds, Institute for Transport Studies, Pontifical Catholic University of Rio de Janeiro, LEUI, United Kingdom.
| | - Tyron L Louw
- University of Leeds, Institute for Transport Studies, Pontifical Catholic University of Rio de Janeiro, LEUI, United Kingdom
| | - Manuela Quaresma
- University of Leeds, Institute for Transport Studies, Pontifical Catholic University of Rio de Janeiro, LEUI, United Kingdom
| | - Ruth Madigan
- University of Leeds, Institute for Transport Studies, Pontifical Catholic University of Rio de Janeiro, LEUI, United Kingdom
| | - Natasha Merat
- University of Leeds, Institute for Transport Studies, Pontifical Catholic University of Rio de Janeiro, LEUI, United Kingdom
| |
Collapse
|
15
|
Stojić H, Orquin JL, Dayan P, Dolan RJ, Speekenbrink M. Uncertainty in learning, choice, and visual fixation. Proc Natl Acad Sci U S A 2020; 117:3291-3300. [PMID: 31980535 PMCID: PMC7022187 DOI: 10.1073/pnas.1911348117] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects' overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.
Collapse
Affiliation(s)
- Hrvoje Stojić
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom;
| | - Jacob L Orquin
- Department of Management/MAPP, Aarhus University, Aarhus 8210, Denmark
- Centre for Research in Marketing and Consumer Psychology, Reykjavik University, 101 Reykjavik, Iceland
| | - Peter Dayan
- Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Tübingen 72076, Germany
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom
| | - Maarten Speekenbrink
- Department of Experimental Psychology, University College London, London WC1H 0AP, United Kingdom
| |
Collapse
|
16
|
|
17
|
Domínguez-Zamora FJ, Gunn SM, Marigold DS. Adaptive Gaze Strategies to Reduce Environmental Uncertainty During a Sequential Visuomotor Behaviour. Sci Rep 2018; 8:14112. [PMID: 30237587 PMCID: PMC6148321 DOI: 10.1038/s41598-018-32504-0] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 09/10/2018] [Indexed: 11/17/2022] Open
Abstract
People must decide where, when, and for how long to allocate gaze to perform different motor behaviours. However, the factors guiding gaze during these ongoing, natural behaviours are poorly understood. Gaze shifts help acquire information, suggesting that people should direct gaze to locations where environmental details most relevant to the task are uncertain. To explore this, human subjects stepped on a series of targets as they walked. We used different levels of target uncertainty, and through instruction, altered the importance of (or subjective value assigned to) foot-placement accuracy. Gaze time on targets increased with greater target uncertainty when precise foot placement was more important, and these longer gaze times associated with reduced foot-placement error. Gaze times as well as the gaze shifts to and from targets relative to stepping differed depending on the target's position in the sequence and uncertainty level. Overall, we show that gaze is allocated to reduce uncertainty about target locations, and this depends on the value of this information gain for successful task performance. Furthermore, we show that the spatial-temporal pattern of gaze to resolve uncertainty changes with the evolution of the motor behaviour, indicating a flexible strategy to plan and control movement.
Collapse
Affiliation(s)
- F Javier Domínguez-Zamora
- Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, V5A 1S6, Canada
| | - Shaila M Gunn
- Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, V5A 1S6, Canada
| | - Daniel S Marigold
- Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, V5A 1S6, Canada.
- Behavioural and Cognitive Neuroscience Institute, Simon Fraser University, Burnaby, British Columbia, V5A 1S6, Canada.
| |
Collapse
|
18
|
Hayhoe MM, Matthis JS. Control of gaze in natural environments: effects of rewards and costs, uncertainty and memory in target selection. Interface Focus 2018; 8:20180009. [PMID: 29951189 DOI: 10.1098/rsfs.2018.0009] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/08/2018] [Indexed: 11/12/2022] Open
Abstract
The development of better eye and body tracking systems, and more flexible virtual environments have allowed more systematic exploration of natural vision and contributed a number of insights. In natural visually guided behaviour, humans make continuous sequences of sensory-motor decisions to satisfy current goals, and the role of vision is to provide the relevant information in order to achieve those goals. This paper reviews the factors that control gaze in natural visually guided actions such as locomotion, including the rewards and costs associated with the immediate behavioural goals, uncertainty about the state of the world and prior knowledge of the environment. These general features of human gaze control may inform the development of artificial systems.
Collapse
Affiliation(s)
- Mary M Hayhoe
- Center for Perceptual Systems, University of Texas Austin, Austin, TX, USA
| | | |
Collapse
|
19
|
Hayhoe MM. Davida Teller Award Lecture 2017: What can be learned from natural behavior? J Vis 2018; 18:10. [PMID: 29710300 PMCID: PMC5895074 DOI: 10.1167/18.4.10] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2017] [Accepted: 02/05/2018] [Indexed: 11/25/2022] Open
Abstract
The essentially active nature of vision has long been acknowledged but has been difficult to investigate because of limitations in the available instrumentation, both for measuring eye and body movements and for presenting realistic stimuli in the context of active behavior. These limitations have been substantially reduced in recent years, opening up a wider range of contexts where experimental control is possible. Given this, it is important to examine just what the benefits are for exploring natural vision, with its attendant disadvantages. Work over the last two decades provides insights into these benefits. Natural behavior turns out to be a rich domain for investigation, as it is remarkably stable and opens up new questions, and the behavioral context helps specify the momentary visual computations and their temporal evolution.
Collapse
Affiliation(s)
- Mary M Hayhoe
- Center for Perceptual Systems, University of Texas Austin, Austin, TX, USA
| |
Collapse
|
20
|
Affiliation(s)
- Miguel P. Eckstein
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, California 93106-9660
| |
Collapse
|
21
|
Gottlieb J. Understanding active sampling strategies: Empirical approaches and implications for attention and decision research. Cortex 2017; 102:150-160. [PMID: 28919222 DOI: 10.1016/j.cortex.2017.08.019] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2017] [Revised: 08/01/2017] [Accepted: 08/14/2017] [Indexed: 01/31/2023]
Abstract
In natural behavior we actively gather information using attention and active sensing behaviors (such as shifts of gaze) to sample relevant cues. However, while attention and decision making are naturally coordinated, in the laboratory they have been dissociated. Attention is studied independently of the actions it serves. Conversely, decision theories make the simplifying assumption that the relevant information is given, and do not attempt to describe how the decision maker may learn and implement active sampling policies. In this paper I review recent studies that address questions of attentional learning, cue validity and information seeking in humans and non-human primates. These studies suggest that learning a sampling policy involves large scale interactions between networks of attention and valuation, which implement these policies based on reward maximization, uncertainty reduction and the intrinsic utility of cognitive states. I discuss the importance of using such paradigms for formalizing the role of attention, as well as devising more realistic theories of decision making that capture a broader range of empirical observations.
Collapse
Affiliation(s)
- Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, USA.
| |
Collapse
|
22
|
Abstract
Investigation of natural behavior has contributed a number of insights to our understanding of visual guidance of actions by highlighting the importance of behavioral goals and focusing attention on how vision and action play out in time. In this context, humans make continuous sequences of sensory-motor decisions to satisfy current behavioral goals, and the role of vision is to provide the relevant information for making good decisions in order to achieve those goals. This conceptualization of visually guided actions as a sequence of sensory-motor decisions has been formalized within the framework of statistical decision theory, which structures the problem and provides the context for much recent progress in vision and action. Components of a good decision include the task, which defines the behavioral goals, the rewards and costs associated with those goals, uncertainty about the state of the world, and prior knowledge.
Collapse
Affiliation(s)
- Mary M Hayhoe
- Center for Perceptual Systems, University of Texas at Austin, Texas 78712;
| |
Collapse
|
23
|
Kircher K, Ahlstrom C. Minimum Required Attention: A Human-Centered Approach to Driver Inattention. HUMAN FACTORS 2017; 59:471-484. [PMID: 27738279 DOI: 10.1177/0018720816672756] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
OBJECTIVE To propose a driver attention theory based on the notion of driving as a satisficing and partially self-paced task and, within this framework, present a definition for driver inattention. BACKGROUND Many definitions of driver inattention and distraction have been proposed, but they are difficult to operationalize, and they are either unreasonably strict and inflexible or suffer from hindsight bias. METHOD Existing definitions of driver distraction are reviewed and their shortcomings identified. We then present the minimum required attention (MiRA) theory to overcome these shortcomings. Suggestions on how to operationalize MiRA are also presented. RESULTS MiRA describes which role the attention of the driver plays in the shared "situation awareness of the traffic system." A driver is considered attentive when sampling sufficient information to meet the demands of the system, namely, that he or she fulfills the preconditions to be able to form and maintain a good enough mental representation of the situation. A driver should only be considered inattentive when information sampling is not sufficient, regardless of whether the driver is concurrently executing an additional task or not. CONCLUSIONS The MiRA theory builds on well-established driver attention theories. It goes beyond available driver distraction definitions by first defining what a driver needs to be attentive to, being free from hindsight bias, and allowing the driver to adapt to the current demands of the traffic situation through satisficing and self-pacing. MiRA has the potential to provide the stepping stone for unbiased and operationalizable inattention detection and classification.
Collapse
Affiliation(s)
- Katja Kircher
- The Swedish National Road and Transport Research Institute (VTI), Linköping, Sweden
| | - Christer Ahlstrom
- The Swedish National Road and Transport Research Institute (VTI), Linköping, Sweden
| |
Collapse
|
24
|
Tong MH, Zohar O, Hayhoe MM. Control of gaze while walking: Task structure, reward, and uncertainty. J Vis 2017; 17:28. [PMID: 28114501 PMCID: PMC5256682 DOI: 10.1167/17.1.28] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2016] [Accepted: 11/20/2016] [Indexed: 11/24/2022] Open
Abstract
While it is universally acknowledged that both bottom up and top down factors contribute to allocation of gaze, we currently have limited understanding of how top-down factors determine gaze choices in the context of ongoing natural behavior. One purely top-down model by Sprague, Ballard, and Robinson (2007) suggests that natural behaviors can be understood in terms of simple component behaviors, or modules, that are executed according to their reward value, with gaze targets chosen in order to reduce uncertainty about the particular world state needed to execute those behaviors. We explore the plausibility of the central claims of this approach in the context of a task where subjects walk through a virtual environment performing interceptions, avoidance, and path following. Many aspects of both walking direction choices and gaze allocation are consistent with this approach. Subjects use gaze to reduce uncertainty for task-relevant information that is used to inform action choices. Notably the addition of motion to peripheral objects did not affect fixations when the objects were irrelevant to the task, suggesting that stimulus saliency was not a major factor in gaze allocation. The modular approach of independent component behaviors is consistent with the main aspects of performance, but there were a number of deviations suggesting that modules interact. Thus the model forms a useful, but incomplete, starting point for understanding top-down factors in active behavior.
Collapse
Affiliation(s)
- Matthew H Tong
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | - Oran Zohar
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | - Mary M Hayhoe
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| |
Collapse
|
25
|
How do walkers avoid a mobile robot crossing their way? Gait Posture 2017; 51:97-103. [PMID: 27744251 DOI: 10.1016/j.gaitpost.2016.09.022] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2015] [Revised: 09/19/2016] [Accepted: 09/24/2016] [Indexed: 02/02/2023]
Abstract
Robots and Humans have to share the same environment more and more often. In the aim of steering robots in a safe and convenient manner among humans it is required to understand how humans interact with them. This work focuses on collision avoidance between a human and a robot during locomotion. Having in mind previous results on human obstacle avoidance, as well as the description of the main principles which guide collision avoidance strategies, we observe how humans adapt a goal-directed locomotion task when they have to interfere with a mobile robot. Our results show differences in the strategy set by humans to avoid a robot in comparison with avoiding another human. Humans prefer to give the way to the robot even when they are likely to pass first at the beginning of the interaction.
Collapse
|
26
|
Wilming N, Kietzmann TC, Jutras M, Xue C, Treue S, Buffalo EA, König P. Differential Contribution of Low- and High-level Image Content to Eye Movements in Monkeys and Humans. Cereb Cortex 2017; 27:279-293. [PMID: 28077512 PMCID: PMC5942390 DOI: 10.1093/cercor/bhw399] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2016] [Accepted: 12/13/2016] [Indexed: 11/25/2022] Open
Abstract
Oculomotor selection exerts a fundamental impact on our experience of the environment. To better understand the underlying principles, researchers typically rely on behavioral data from humans, and electrophysiological recordings in macaque monkeys. This approach rests on the assumption that the same selection processes are at play in both species. To test this assumption, we compared the viewing behavior of 106 humans and 11 macaques in an unconstrained free-viewing task. Our data-driven clustering analyses revealed distinct human and macaque clusters, indicating species-specific selection strategies. Yet, cross-species predictions were found to be above chance, indicating some level of shared behavior. Analyses relying on computational models of visual saliency indicate that such cross-species commonalities in free viewing are largely due to similar low-level selection mechanisms, with only a small contribution by shared higher level selection mechanisms and with consistent viewing behavior of monkeys being a subset of the consistent viewing behavior of humans.
Collapse
Affiliation(s)
- Niklas Wilming
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany.,Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA.,Yerkes National Primate Research Center, Atlanta, GA 30329, USA.,Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.,Washington National Primate Research Center, Seattle, WA 09195, USA
| | - Tim C Kietzmann
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany.,Medical Research Council, Cognition and Brain Sciences Unit, Cambridge CB2 7EF, UK
| | - Megan Jutras
- Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA.,Yerkes National Primate Research Center, Atlanta, GA 30329, USA.,Washington National Primate Research Center, Seattle, WA 09195, USA
| | - Cheng Xue
- Cognitive Neuroscience Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Goettingen, Germany
| | - Stefan Treue
- Cognitive Neuroscience Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Goettingen, Germany.,Faculty of Biology and Psychology, Goettingen University, Goettingen, Germany.,Leibniz-ScienceCampus Primate Cognition, Goettingen, Germany
| | - Elizabeth A Buffalo
- Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA.,Yerkes National Primate Research Center, Atlanta, GA 30329, USA.,Washington National Primate Research Center, Seattle, WA 09195, USA
| | - Peter König
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany.,Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
27
|
Jiang M, Boix X, Roig G, Xu J, Van Gool L, Zhao Q. Learning to Predict Sequences of Human Visual Fixations. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2016; 27:1241-1252. [PMID: 26761903 DOI: 10.1109/tnnls.2015.2496306] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Most state-of-the-art visual attention models estimate the probability distribution of fixating the eyes in a location of the image, the so-called saliency maps. Yet, these models do not predict the temporal sequence of eye fixations, which may be valuable for better predicting the human eye fixations, as well as for understanding the role of the different cues during visual exploration. In this paper, we present a method for predicting the sequence of human eye fixations, which is learned from the recorded human eye-tracking data. We use least-squares policy iteration (LSPI) to learn a visual exploration policy that mimics the recorded eye-fixation examples. The model uses a different set of parameters for the different stages of visual exploration that capture the importance of the cues during the scanpath. In a series of experiments, we demonstrate the effectiveness of using LSPI for combining multiple cues at different stages of the scanpath. The learned parameters suggest that the low-level and high-level cues (semantics) are similarly important at the first eye fixation of the scanpath, and the contribution of high-level cues keeps increasing during the visual exploration. Results show that our approach obtains the state-of-the-art performances on two challenging data sets: 1) OSIE data set and 2) MIT data set.
Collapse
|
28
|
Eckstein MP, Schoonveld W, Zhang S, Mack SC, Akbas E. Optimal and human eye movements to clustered low value cues to increase decision rewards during search. Vision Res 2015; 113:137-54. [PMID: 26093154 PMCID: PMC6020051 DOI: 10.1016/j.visres.2015.05.016] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2014] [Revised: 05/25/2015] [Accepted: 05/28/2015] [Indexed: 11/16/2022]
Abstract
Rewards have important influences on the motor planning of primates and the firing of neurons coding visual information and action. When eye movements to a target are differentially rewarded across locations, primates execute saccades towards the possible target location with the highest expected value, a product of sensory evidence and potentially earned reward (saccade to maximum expected value model, sMEV). Yet, in the natural world eye movements are not directly rewarded. Their role is to gather information to support subsequent rewarded search decisions and actions. Less is known about the effects of decision rewards on saccades. We show that when varying the decision rewards across cued locations following visual search, humans can plan their eye movements to increase decision rewards. Critically, we report a scenario for which five of seven tested humans do not preferentially deploy saccades to the possible target location with the highest reward, a strategy which is optimal when rewarding eye movements. Instead, these humans make saccades towards lower value but clustered locations when this strategy optimizes decision rewards consistent with the preferences of an ideal Bayesian reward searcher that takes into account the visibility of the target across eccentricities. The ideal reward searcher can be approximated with a sMEV model with pooling of rewards from spatially clustered locations. We also find observers with systematic departures from the optimal strategy and inter-observer variability of eye movement plans. These deviations often reflect multiplicity of fixation strategies that lead to near optimal decision rewards but, for some observers, it relates to suboptimal choices in eye movement planning.
Collapse
Affiliation(s)
- Miguel P Eckstein
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106-9660, United States.
| | - Wade Schoonveld
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106-9660, United States
| | - Sheng Zhang
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106-9660, United States
| | - Stephen C Mack
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106-9660, United States
| | - Emre Akbas
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA 93106-9660, United States
| |
Collapse
|
29
|
Janssen CP, Brumby DP. Strategic Adaptation to Task Characteristics, Incentives, and Individual Differences in Dual-Tasking. PLoS One 2015; 10:e0130009. [PMID: 26161851 PMCID: PMC4498911 DOI: 10.1371/journal.pone.0130009] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2014] [Accepted: 05/15/2015] [Indexed: 11/19/2022] Open
Abstract
We investigate how good people are at multitasking by comparing behavior to a prediction of the optimal strategy for dividing attention between two concurrent tasks. In our experiment, 24 participants had to interleave entering digits on a keyboard with controlling a randomly moving cursor with a joystick. The difficulty of the tracking task was systematically varied as a within-subjects factor. Participants were also exposed to different explicit reward functions that varied the relative importance of the tracking task relative to the typing task (between-subjects). Results demonstrate that these changes in task characteristics and monetary incentives, together with individual differences in typing ability, influenced how participants choose to interleave tasks. This change in strategy then affected their performance on each task. A computational cognitive model was used to predict performance for a wide set of alternative strategies for how participants might have possibly interleaved tasks. This allowed for predictions of optimal performance to be derived, given the constraints placed on performance by the task and cognition. A comparison of human behavior with the predicted optimal strategy shows that participants behaved near optimally. Our findings have implications for the design and evaluation of technology for multitasking situations, as consideration should be given to the characteristics of the task, but also to how different users might use technology depending on their individual characteristics and their priorities.
Collapse
Affiliation(s)
- Christian P. Janssen
- UCL Interaction Centre, University College London, London, United Kingdom
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Duncan P. Brumby
- UCL Interaction Centre, University College London, London, United Kingdom
| |
Collapse
|
30
|
Gottlieb J, Hayhoe M, Hikosaka O, Rangel A. Attention, reward, and information seeking. J Neurosci 2014; 34:15497-504. [PMID: 25392517 PMCID: PMC4228145 DOI: 10.1523/jneurosci.3270-14.2014] [Citation(s) in RCA: 100] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2014] [Revised: 10/08/2014] [Accepted: 10/15/2014] [Indexed: 11/21/2022] Open
Abstract
Decision making is thought to be guided by the values of alternative options and involve the accumulation of evidence to an internal bound. However, in natural behavior, evidence accumulation is an active process whereby subjects decide when and which sensory stimulus to sample. These sampling decisions are naturally served by attention and rapid eye movements (saccades), but little is known about how saccades are controlled to guide future actions. Here we review evidence that was discussed at a recent symposium, which suggests that information selection involves basal ganglia and cortical mechanisms and that, across different contexts, it is guided by two central factors: the gains in reward and gains in information (uncertainty reduction) associated with sensory cues.
Collapse
Affiliation(s)
- Jacqueline Gottlieb
- Department of Neuroscience and Kavli Institute for Brain Science, Columbia University, New York, New York 10032,
| | - Mary Hayhoe
- University of Texas at Austin, Austin, Texas 78712
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892
| | - Antonio Rangel
- Division of Humanities and Social Sciences and Computational and Neural Systems, California Institute of Technology, Pasadena, California 91125, and
| |
Collapse
|
31
|
Abstract
In natural behavior, visual information is actively sampled from the environment by a sequence of gaze changes. The timing and choice of gaze targets, and the accompanying attentional shifts, are intimately linked with ongoing behavior. Nonetheless, modeling of the deployment of these fixations has been very difficult because they depend on characterizing the underlying task structure. Recently, advances in eye tracking during natural vision, together with the development of probabilistic modeling techniques, have provided insight into how the cognitive agenda might be included in the specification of fixations. These techniques take advantage of the decomposition of complex behaviors into modular components. A particular subset of these models casts the role of fixation as that of providing task-relevant information that is rewarding to the agent, with fixation being selected on the basis of expected reward and uncertainty about environmental state. We review this work here and describe how specific examples can reveal general principles in gaze control.
Collapse
Affiliation(s)
- Mary Hayhoe
- Department of Psychology, 108 E. Dean Keeton, Stop A8000, The University of Texas at Austin, Austin, TX 78712, USA.
| | - Dana Ballard
- Department of Computer Science, 2317 Speedway, Stop D9500, The University of Texas at Austin, Austin, TX 78712, USA.
| |
Collapse
|
32
|
Johnson L, Sullivan B, Hayhoe M, Ballard D. Predicting human visuomotor behaviour in a driving task. Philos Trans R Soc Lond B Biol Sci 2014; 369:20130044. [PMID: 24395971 PMCID: PMC3886332 DOI: 10.1098/rstb.2013.0044] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
The sequential deployment of gaze to regions of interest is an integral part of human visual function. Owing to its central importance, decades of research have focused on predicting gaze locations, but there has been relatively little formal attempt to predict the temporal aspects of gaze deployment in natural multi-tasking situations. We approach this problem by decomposing complex visual behaviour into individual task modules that require independent sources of visual information for control, in order to model human gaze deployment on different task-relevant objects. We introduce a softmax barrier model for gaze selection that uses two key elements: a priority parameter that represents task importance per module, and noise estimates that allow modules to represent uncertainty about the state of task-relevant visual information. Comparisons with human gaze data gathered in a virtual driving environment show that the model closely approximates human performance.
Collapse
Affiliation(s)
- Leif Johnson
- Department of Computer Science, University of Texas at Austin, , TX, USA
| | | | | | | |
Collapse
|