Toward a Theory of Visual Information Acquisition in Driving

Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood?

The present study investigated the effects of a driver monitoring system that triggers attention warnings in case distraction is detected. Based on the EuroNCAP protocol, distraction could either be long glances away from the forward roadway (≥3s) or visual attention time sharing (>10 cumulative seconds within a 30 s time interval). In a series of manual driving simulator drives, 30 participants completed both driving related tasks (e.g., changing multiple lanes in dense traffic) and non-driving related tasks (e.g., infotainment operations). Results of warning frequencies revealed that visual attention time sharing warnings occurred more frequently than long distraction warnings. Moreover, there was a large number of attention warnings during driving related tasks. Results also revealed that participants' mental models tended to be less accurate when it came to understanding of the visual attention time sharing warnings as compared to the long distraction warnings, which were understood more accurately. Based on these observations, the work discusses the applicability and design of driver monitoring warnings.

Hazard Perception in Visually Impaired Drivers Who Use Bioptic Telescopes

Purpose

Bioptic telescopic spectacles can allow individuals with central vision impairment to obtain or maintain driving privileges. The purpose of this study was to (1) compare hazard perception ability among bioptic drivers and traditionally licensed controls, (2) assess the impact of bioptic telescopic spectacles on hazard perception in drivers with vision impairment, and (3) analyze the relationships among vision and hazard detection in bioptic drivers.

Methods

Visual acuity, contrast sensitivity, and visual field were measured for each participant. All drivers completed the Driving Habits Questionnaire. Hazard perception testing was conducted using commercially available first-person video driving clips. Subjects signaled when they could first identify a traffic hazard requiring a change of speed or direction. Bioptic drivers were tested with and without their bioptic telescopes in alternating blocks. Hazard detection times for each clip were converted to z-scores, converted back to seconds using the average response time across all videos, and then compared among conditions.

Results

Twenty-one bioptic drivers and 21 normally sighted controls participated in the study. The hazard response time of bioptic drivers was improved when able to use the telescope (5.4 ± 1.4 seconds vs 6.3 ± 1.8 seconds without telescope); however, it remained significantly longer than for controls (4.0 ± 1.4 seconds). Poorer visual acuity, contrast sensitivity, and superior visual field sensitivity loss were related to longer hazard response times.

Conclusions

Drivers with central vision loss had improved hazard response times with the use of bioptic telescopic spectacles, although their responses were still slower than normally sighted control drivers.

Translational Relevance

The use of a bioptic telescope by licensed, visually impaired drivers improves their hazard detection speed on a video-based task, lending support to their use on the road.

Knowing me, knowing you-A study on top-down requirements for compensatory scanning in drivers with homonymous visual field loss

OBJECTIVE

It is currently still unknown why some drivers with visual field loss can compensate well for their visual impairment while others adopt ineffective strategies. This paper contributes to the methodological investigation of the associated top-down mechanisms and aims at validating a theoretical model on the requirements for successful compensation among drivers with homonymous visual field loss.

METHODS

A driving simulator study was conducted with eight participants with homonymous visual field loss and eight participants with normal vision. Participants drove through an urban surrounding and experienced a baseline scenario and scenarios with visual precursors indicating increased likelihoods of crossing hazards. Novel measures for the assessment of the mental model of their visual abilities, the mental model of the driving scene and the perceived attention demand were developed and used to investigate the top-down mechanisms behind attention allocation and hazard avoidance.

RESULTS

Participants with an overestimation of their visual field size tended to prioritize their seeing side over their blind side both in subjective and objective measures. The mental model of the driving scene showed close relations to the subjective and actual attention allocation. While participants with homonymous visual field loss were less anticipatory in their usage of the visual precursors and showed poorer performances compared to participants with normal vision, the results indicate a stronger reliance on top-down mechanism for drivers with visual impairments. A subjective focus on the seeing side or on near peripheries more frequently led to bad performances in terms of collisions with crossing cyclists.

CONCLUSION

The study yielded promising indicators for the potential of novel measures to elucidate top-down mechanisms in drivers with homonymous visual field loss. Furthermore, the results largely support the model of requirements for successful compensatory scanning. The findings highlight the importance of individualized interventions and driver assistance systems tailored to address these mechanisms.

An updating-based working memory load alters the dynamics of eye movements but not their spatial extent during free viewing of natural scenes

The relationship between spatial deployments of attention and working memory load is an important topic of study, with clear implications for real-world tasks such as driving. Previous research has generally shown that attentional breadth broadens under higher load, while exploratory eye-movement behaviour also appears to change with increasing load. However, relatively little research has compared the effects of working memory load on different kinds of spatial deployment, especially in conditions that require updating of the contents of working memory rather than simple retrieval. The present study undertook such a comparison by measuring participants' attentional breadth (via an undirected Navon task) and their exploratory eye-movement behaviour (a free-viewing recall task) under low and high updating working memory loads. While spatial aspects of task performance (attentional breadth, and peripheral extent of image exploration in the free-viewing task) were unaffected by the load manipulation, the exploratory dynamics of the free-viewing task (including fixation durations and scan-path lengths) changed under increasing load. These findings suggest that temporal dynamics, rather than the spatial extent of exploration, are the primary mechanism affected by working memory load during the spatial deployment of attention. Further, individual differences in exploratory behaviour were observed on the free-viewing task: all metrics were highly correlated across working memory load blocks. These findings suggest a need for further investigation of individual differences in eye-movement behaviour; potential factors associated with these individual differences, including working memory capacity and persistence versus flexibility orientations, are discussed.

High target prevalence may reduce the spread of attention during search tasks

Target prevalence influences many cognitive processes during visual search, including target detection, search efficiency, and item processing. The present research investigated whether target prevalence may also impact the spread of attention during search. Relative to low-prevalence searches, high-prevalence searches typically yield higher fixation counts, particularly during target-absent trials. This may emerge because the attention spread around each fixation may be smaller for high than low prevalence searches. To test this, observers searched for targets within object arrays in Experiments 1 (free-viewing) and 2 (gaze-contingent viewing). In Experiment 3, observers searched for targets in a Rapid Serial Visual Presentation (RSVP) stream at the center of the display while simultaneously processing occasional peripheral objects. Experiment 1 used fixation patterns to estimate attentional spread, and revealed that attention was narrowed during high, relative to low, prevalence searches. This effect was weakened during gaze-contingent search (Experiment 2) but emerged again when eye movements were unnecessary in RSVP search (Experiment 3). These results suggest that, although task demands impact how attention is allocated across displays, attention may also narrow when searching for frequent targets.

A novel collision warning system based on the visual road environment schema: An examination from vehicle and driver characteristics

Drivers pay unequal attention to different road environmental elements and visual fields, which greatly influences their driving behavior. However, existing collision warning systems ignore these visual characteristics of drivers, which limits the performance of collision warning systems. Therefore, this study proposes a novel collision warning system based on the visual road environment schema, in order to enhance the support for avoiding potential dangers in objects and areas that are easily overlooked by the drivers' vision. To capture the above visual characteristics of drivers, the visual road environment schema that consists of the semantic layer, the scene depth layer, the sensitive layer, and the visual field layer is established by using several different deep neural networks, which realizes the recognition, quantization, and analysis of the road environment from the drivers' visual perspective. The effectiveness of the novel collision warning system is verified by the driving simulation experiment from six indicators, including warning distance, maximum lateral acceleration, maximum longitudinal deceleration, minimum collision time, reaction time, and heart rate. Additionally, a grey target decision-making model is built to comprehensively evaluate the system. The results show that compared with the traditional collision warning system, the novel collision warning system proposed in this study performs significantly better and can discover potential dangers earlier, give timely warnings, enhance the vehicles' lateral stability and driving comfort, shorten reaction time, and relieve the drivers' nervousness. By integrating the drivers' visual characteristics into the collision warning system, this study could help to optimize the existing collision warning system and promote the mutual understanding between intelligent vehicles and human drivers.

On the relationship between occlusion times and in-car glance durations in simulated driving

Drivers have spare visual capacity in driving, and often this capacity is used for engaging in secondary in-car tasks. Previous research has suggested that the spare visual capacity could be estimated with the occlusion method. However, the relationship between drivers' occlusion times and in-car glance duration preferences has not been sufficiently investigated for granting occlusion times the role of an estimate of spare visual capacity. We conducted a driving simulator experiment (N = 30) and investigated if there is an association between drivers' occlusion times and in-car glance durations in a given driving scenario. Furthermore, we explored which factors and variables could explain the strength of the association. The findings suggest an association between occlusion time preferences and in-car glance durations in visually and cognitively low demanding unstructured tasks but that this association is lost if the in-car task is more demanding. The findings might be explained by the inability to utilize peripheral vision for lane-keeping when conducting in-car tasks and/or by in-car task structures that override drivers' preferences for the in-car glance durations. It seems that the occlusion technique could be utilized as an estimate of drivers' spare visual capacity in research - but with caution. It is strongly recommended to use occlusion times in combination with driving performance metrics. There is less spare visual capacity if this capacity is used for secondary tasks that interfere with the driver's ability to utilize peripheral vision for driving or preferences for the in-car glance durations. However, we suggest that the occlusion method can be a valid method to control for inter-individual differences in in-car glance duration preferences when investigating the visual distraction potential of, for instance, in-vehicle infotainment systems.

The perceived effects of augmented trail sensing and mood recognition abilities in a human-fish biohybrid system

The use of technologies to enhance human and animal perception has been explored in pioneering research about artificial life and biohybrid systems. These attempts have revealed that augmented sensing abilities can emerge from new interactions between individuals within or across species. Nevertheless, the diverse effects of different augmented capabilities have been less examined and compared. In this work, we built a human-fish biohybrid system that enhanced the vision of the ornamental fish by projecting human participants onto the arena background. In contrast, human participants were equipped with a mixed-reality device, which visualized individual fish trails (representing situation-oriented perceptions) and emotions (representing communication-oriented perceptions). We investigated the impacts of the two enhanced perceptions on the human side and documented the perceived effects from three aspects. First, both augmented perceptions considerably increase participants' attention toward ornamental fish, and the impact of emotion recognition is more potent than trail sense. Secondly, the frequency of human-fish interactions increases with the equipped perceptions. The mood recognition ability on the human side can indirectly promote the recorded positive mood of fish. Thirdly, most participants mentioned that they felt closer to those fish which had mood recognition ability, even if we added some mistakes in the accuracy of mood recognition. In contrast, the addition of trail sensing ability does not lead to a similar effect on the mental bond. These findings reveal several aspects of different perceived effects between the enhancements of communication-oriented and situation-oriented perceptions.

Effect of peripheral refractive errors on driving performance

The effect of peripheral refractive errors on driving while performing secondary tasks at 40° of eccentricity was studied in thirty-one young drivers. They drove a driving simulator under 7 different induced peripheral refractive errors (baseline (0D), spherical lenses of +/- 2D, +/- 4D and cylindrical lenses of +2D and +4D). Peripheral visual acuity and contrast sensitivity were also evaluated at 40°. Driving performance was significantly impaired by the addition of myopic defocus (4D) and astigmatism (4D). Worse driving significantly correlated with worse contrast sensitivity for the route in general, but also with worse visual acuity when participants interacted with the secondary task. Induced peripheral refractive errors may negatively impact driving when performing secondary tasks.

Normal blindness: when we Look But Fail To See

Humans routinely miss important information that is 'right in front of our eyes', from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these 'Looked But Failed To See' (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

Beyond gaze fixation: Modeling peripheral vision in relation to speed, Tesla Autopilot, cognitive load, and age in highway driving

OBJECTIVE

The study aims to model driver perception across the visual field in dynamic, real-world highway driving.

BACKGROUND

Peripheral vision acquires information across the visual field and guides a driver's information search. Studies in naturalistic settings are lacking however, with most research having been conducted in controlled simulation environments with limited eccentricities and driving dynamics.

METHODS

We analyzed data from 24 participants who drove a Tesla Model S with Autopilot on the highway. While driving, participants completed the peripheral detection task (PDT) using LEDs and the N-back task to generate cognitive load. The I-DT (identification by dispersion threshold) algorithm sampled naturalistic gaze fixations during PDTs to cover a broader and continuous spectrum of eccentricity. A generalized Bayesian regression model predicted LED detection probability during the PDT-as a surrogate for peripheral vision-in relation to eccentricity, vehicle speed, driving mode, cognitive load, and age.

RESULTS

The model predicted that LED detection probability was high and stable through near-peripheral vision but it declined rapidly beyond 20°-30° eccentricity, showing a narrower useful field over a broader visual field (maximum 70°) during highway driving. Reduced speed (while following another vehicle), cognitive load, and older age were the main factors that degraded the mid-peripheral vision (20°-50°), while using Autopilot had little effect.

CONCLUSIONS

Drivers can reliably detect objects through near-peripheral vision, but their peripheral detection degrades gradually due to further eccentricity, foveal demand during low-speed vehicle following, cognitive load, and age.

APPLICATIONS

The findings encourage the development of further multivariate computational models to estimate peripheral vision and assess driver situation awareness for crash prevention.

Peripheral vision in real-world tasks: A systematic review

Peripheral vision is fundamental for many real-world tasks, including walking, driving, and aviation. Nonetheless, there has been no effort to connect these applied literatures to research in peripheral vision in basic vision science or sports science. To close this gap, we analyzed 60 relevant papers, chosen according to objective criteria. Applied research, with its real-world time constraints, complex stimuli, and performance measures, reveals new functions of peripheral vision. Peripheral vision is used to monitor the environment (e.g., road edges, traffic signs, or malfunctioning lights), in ways that differ from basic research. Applied research uncovers new actions that one can perform solely with peripheral vision (e.g., steering a car, climbing stairs). An important use of peripheral vision is that it helps compare the position of one’s body/vehicle to objects in the world. In addition, many real-world tasks require multitasking, and the fact that peripheral vision provides degraded but useful information means that tradeoffs are common in deciding whether to use peripheral vision or move one’s eyes. These tradeoffs are strongly influenced by factors like expertise, age, distraction, emotional state, task importance, and what the observer already knows. These tradeoffs make it hard to infer from eye movements alone what information is gathered from peripheral vision and what tasks we can do without it. Finally, we recommend three ways in which basic, sport, and applied science can benefit each other’s methodology, furthering our understanding of peripheral vision more generally.

Visual search in naturalistic scenes from foveal to peripheral vision: A comparison between dynamic and static displays

How important foveal, parafoveal, and peripheral vision are depends on the task. For object search and letter search in static images of real-world scenes, peripheral vision is crucial for efficient search guidance, whereas foveal vision is relatively unimportant. Extending this research, we used gaze-contingent Blindspots and Spotlights to investigate visual search in complex dynamic and static naturalistic scenes. In Experiment 1, we used dynamic scenes only, whereas in Experiments 2 and 3, we directly compared dynamic and static scenes. Each scene contained a static, contextually irrelevant target (i.e., a gray annulus). Scene motion was not predictive of target location. For dynamic scenes, the search-time results from all three experiments converge on the novel finding that neither foveal nor central vision was necessary to attain normal search proficiency. Since motion is known to attract attention and gaze, we explored whether guidance to the target was equally efficient in dynamic as compared to static scenes. We found that the very first saccade was guided by motion in the scene. This was not the case for subsequent saccades made during the scanning epoch, representing the actual search process. Thus, effects of task-irrelevant motion were fast-acting and short-lived. Furthermore, when motion was potentially present (Spotlights) or absent (Blindspots) in foveal or central vision only, we observed differences in verification times for dynamic and static scenes (Experiment 2). When using scenes with greater visual complexity and more motion (Experiment 3), however, the differences between dynamic and static scenes were much reduced.

Eye Tracking in Driver Attention Research-How Gaze Data Interpretations Influence What We Learn

Eye tracking (ET) has been used extensively in driver attention research. Amongst other findings, ET data have increased our knowledge about what drivers look at in different traffic environments and how they distribute their glances when interacting with non-driving related tasks. Eye tracking is also the go-to method when determining driver distraction via glance target classification. At the same time, eye trackers are limited in the sense that they can only objectively measure the gaze direction. To learn more about why drivers look where they do, what information they acquire foveally and peripherally, how the road environment and traffic situation affect their behavior, and how their own expertise influences their actions, it is necessary to go beyond counting the targets that the driver foveates. In this perspective paper, we suggest a glance analysis approach that classifies glances based on their purpose. The main idea is to consider not only the intention behind each glance, but to also account for what is relevant in the surrounding scene, regardless of whether the driver has looked there or not. In essence, the old approaches, unaware as they are of the larger context or motivation behind eye movements, have taken us as far as they can. We propose this more integrative approach to gain a better understanding of the complexity of drivers' informational needs and how they satisfy them in the moment.

The role of eye movements in perceiving vehicle speed and time-to-arrival at the roadside

To avoid collisions, pedestrians depend on their ability to perceive and interpret the visual motion of other road users. Eye movements influence motion perception, yet pedestrians' gaze behavior has been little investigated. In the present study, we ask whether observers sample visual information differently when making two types of judgements based on the same virtual road-crossing scenario and to which extent spontaneous gaze behavior affects those judgements. Participants performed in succession a speed and a time-to-arrival two-interval discrimination task on the same simple traffic scenario-a car approaching at a constant speed (varying from 10 to 90 km/h) on a single-lane road. On average, observers were able to discriminate vehicle speeds of around 18 km/h and times-to-arrival of 0.7 s. In both tasks, observers placed their gaze closely towards the center of the vehicle's front plane while pursuing the vehicle. Other areas of the visual scene were sampled infrequently. No differences were found in the average gaze behavior between the two tasks and a pattern classifier (Support Vector Machine), trained on trial-level gaze patterns, failed to reliably classify the task from the spontaneous eye movements it elicited. Saccadic gaze behavior could predict time-to-arrival discrimination performance, demonstrating the relevance of gaze behavior for perceptual sensitivity in road-crossing.

Effects of temporal and spatiotemporal cues on detection of dynamic road hazards

While driving, dangerous situations can occur quickly, and giving drivers extra time to respond may make the road safer for everyone. Extensive research on attentional cueing in cognitive psychology has shown that targets are detected faster when preceded by a spatially valid cue, and slower when preceded by an invalid cue. However, it is unknown how these standard laboratory-based cueing effects may translate to dynamic, real-world situations like driving, where potential targets (i.e., hazardous events) are inherently more complex and variable. Observers in our study were required to correctly localize hazards in dynamic road scenes across three cue conditions (temporal, spatiotemporal valid and spatiotemporal invalid), and a no-cue baseline. All cues were presented at the first moment the hazardous situation began. Both types of valid cues reduced reaction time (by 58 and 60 ms, respectively, with no significant difference between them, a larger effect than in many classic studies). In addition, observers’ ability to accurately localize hazards dropped 11% in the spatiotemporal invalid condition, a result with dangerous implications on the road. This work demonstrates that, in spite of this added complexity, classic cueing effects persist—and may even be enhanced—for the detection of real-world hazards, and that valid cues have the potential to benefit drivers on the road.