Demonstrating Brain-Level Interactions Between Visuospatial Attentional Demands and Working Memory Load While Driving Using Functional Near-Infrared Spectroscopy

Multitasking While Driving: Central Bottleneck or Problem State Interference?

OBJECTIVE

The objective of this work was to investigate if visuospatial attention and working memory load interact at a central control resource or at a task-specific, information processing resource during driving.

BACKGROUND

In previous multitasking driving experiments, interactions between different cognitive concepts (e.g., attention and working memory) have been found. These interactions have been attributed to a central bottleneck or to the so-called problem-state bottleneck, related to working memory usage.

METHOD

We developed two different cognitive models in the cognitive architecture ACT-R, which implement the central vs. problem-state bottleneck. The models performed a driving task, during which we varied visuospatial attention and working memory load. We evaluated the model by conducting an experiment with human participants and compared the behavioral data to the model's behavior.

RESULTS

The problem-state-bottleneck model could account for decreased driving performance due to working memory load as well as increased visuospatial attentional demands as compared to the central-bottleneck model, which could not account for effects of increased working memory load.

CONCLUSION

The interaction between working memory and visuospatial attention in our dual tasking experiment can be best characterized by a bottleneck in the working memory. The model results suggest that as working memory load becomes higher, drivers manage to perform fewer control actions, which leads to decreasing driving performance.

APPLICATION

Predictions about the effect of different mental loads can be used to quantify the contribution of each subtask allowing for precise assessments of the current overall mental load, which automated driving systems may adapt to.

iVR-fNIRS: studying brain functions in a fully immersive virtual environment

Immersive virtual reality (iVR) employs head-mounted displays or cave-like environments to create a sensory-rich virtual experience that simulates the physical presence of a user in a digital space. The technology holds immense promise in neuroscience research and therapy. In particular, virtual reality (VR) technologies facilitate the development of diverse tasks and scenarios closely mirroring real-life situations to stimulate the brain within a controlled and secure setting. It also offers a cost-effective solution in providing a similar sense of interaction to users when conventional stimulation methods are limited or unfeasible. Although combining iVR with traditional brain imaging techniques may be difficult due to signal interference or instrumental issues, recent work has proposed the use of functional near infrared spectroscopy (fNIRS) in conjunction with iVR for versatile brain stimulation paradigms and flexible examination of brain responses. We present a comprehensive review of current research studies employing an iVR-fNIRS setup, covering device types, stimulation approaches, data analysis methods, and major scientific findings. The literature demonstrates a high potential for iVR-fNIRS to explore various types of cognitive, behavioral, and motor functions in a fully immersive VR (iVR) environment. Such studies should set a foundation for adaptive iVR programs for both training (e.g., in novel environments) and clinical therapeutics (e.g., pain, motor and sensory disorders and other psychiatric conditions).

Workload-dependent hemispheric asymmetries during the emotion-cognition interaction: a close-to-naturalistic fNIRS study

Introduction

We investigated brain activation patterns of interacting emotional distractions and cognitive processes in a close-to-naturalistic functional near-infrared spectroscopy (fNIRS) study.

Methods

Eighteen participants engaged in a monitoring-control task, mimicking common air traffic controller requirements. The scenario entailed experiencing both low and high workload, while concurrently being exposed to emotional speech distractions of positive, negative, and neutral valence.

Results

Our investigation identified hemispheric asymmetries in prefrontal cortex (PFC) activity during the presentation of negative and positive emotional speech distractions at different workload levels. Thereby, in particular, activation in the left inferior frontal gyrus (IFG) and orbitofrontal cortex (OFC) seems to play a crucial role. Brain activation patterns revealed a cross-over interaction indicating workload-dependent left hemispheric inhibition processes during negative distractions and high workload. For positive emotional distractions under low workload, we observed left-hemispheric PFC recruitment potentially associated with speech-related processes. Furthermore, we found a workload-independent negativity bias for neutral distractions, showing brain activation patterns similar to those of negative distractions.

Discussion

In conclusion, lateralized hemispheric processing, regulating emotional speech distractions and integrating emotional and cognitive processes, is influenced by workload levels and stimulus characteristics. These findings advance our understanding of the factors modulating hemispheric asymmetries during the processing and inhibition of emotional distractions, as well as the interplay between emotion and cognition. Moreover, they emphasize the significance of exploring emotion-cognition interactions in more naturalistic settings to gain a deeper understanding of their implications in real-world application scenarios (e.g., working and learning environments).

Driving risk cognition of passengers in highly automated driving based on the prefrontal cortex activity via fNIRS

For high-level automated vehicles, the human being acts as the passenger instead of the driver and does not need to operate vehicles, it makes the brain-computer interface system of high-level automated vehicles depend on the brain state of passengers rather than that of drivers. Particularly when confronting challenging driving situations, how to implement the mental states of passengers into safe driving is a vital choice in the future. Quantifying the cognition of the driving risk of the passenger is a basic step in achieving this goal. In this paper, the passengers' mental activities in low-risk episode and high-risk episode were compared, the influences on passengers' mental activities caused by driving scenario risk was first explored via fNIRS. The results showed that the mental activities of passengers caused by driving scenario risk in the Brodmann area 10 are very active, which was verified by examining the real-driving data collected in corresponding challenging experiments, and there is a positive correlation between the cerebral oxygen and the driving risk field. This initial finding provides a possible solution to design a human-centred intelligent system to promise safe driving for high-level automated vehicles using passengers' driving risk cognition.

A functional near-infrared spectroscopy study on the prefrontal correlates of cognitive offloading via a personal knowledge assistant

The saving of previously encoded information boosts both memory for subsequent information (saving-enhanced memory; SEM) as well as cognitive performance in general (saving-enhanced performance; SEP). These findings have been replicated in a setting that involves the assistance by an intelligent software that automatically structures and saves work content in an interactive sidebar. It is assumed that beneficial effects on cognitive performance due to (automatic) saving are caused by a reduction in current workload by means of cognitive offloading. We tested this assumption by measuring neural activity in the dorsolateral prefrontal cortex (DLPFC) via functional near infrared spectroscopy (fNIRS)-once after saving and once after deleting of previously collected information that had to be recalled later-on. On a behavioral level, there was a brief benefit of saving. However, cognitive offloading became most apparent on a neural level: after saving, participants showed significantly lower activation in the right DLPFC. Also, the more participants benefited from cognitive offloading, the more they were able to re-access previously collected, saved information. Thus, fNIRS results indicated reduced mental load after saving, confirming the assumption that saving triggers cognitive offloading.

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

Humans' performance varies due to the mental resources that are available to successfully pursue a task. To monitor users' current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications.

On the importance of working memory in the driving safety field: A systematic review

In recent years, many studies have used poor cognitive functions to explain risk safety differences among drivers. Working memory is a cognitive function with information storage and attentional control that plays a crucial role in driver information processing. Furthermore, it is inextricably linked to parameters such as driving performance, driving eye movements and driving neurophysiology, which have a significant impact on drivers' risky behavior and crash risk. In particular, crash risk is a serious risk to social safety and economic development. For this reason, it is necessary to understand how risk-related working memory affects driving so that pre-driving safety pre-training programs and in-vehicle safety assistance systems for driving can be developed accordingly, contributing to the development of semi-autonomous vehicles and even autonomous vehicles. In this paper, a systematic search of the literature over the past 23 years resulted in 78 articles that met the eligibility criteria and quality assessment. The results show that higher working memory capacity, as measured neuropsychologically, is associated with more consistent and safer driving-related parameters for drivers (e.g., lane keeping) and may be related to pupil dilation during risk perception while driving, which is associated with driving outcomes (tickets, pull-overs, penalty points and fines,and driving accidents) is closely related to the perceived usefulness of the human-machine interface, reaction time, standard deviation of steering wheel corners, etc. when the autonomous driving takes over. In addition, higher working memory load interference was associated with more inconsistent and unsafe driving-related parameters (including but not limited to eye movements, electrophysiology, etc.), with higher working memory load being associated with easier driver concentration on the road, faster heart rate, lower heart rate variability, and lower oxyhemoglobin (OxyHb) and deoxyhemoglobin (DeoxyHb). Only a limited number of studies have simultaneously investigated the relationship between working memory capacity, working memory load and driving, showing an interaction between working memory capacity and working memory load on lane change initiation and lane change correctness, with working memory capacity acting as a covariate that mediated the effect of working memory load on braking reaction time. In addition, working memory-related cognitive training had a transfer effect on improving driving ability. Overall, working memory capacity determines the upper limit of the number of working memory attention resources, while working memory load occupies part of the working memory attention resources, thus influencing information perception, decision judgment, operational response, and collision avoidance in driving. Future effective interventions for safe driving can be combined with capacity training and load alerting. These findings contribute to our understanding of the role of working memory in driving and provide new insights into the design of driver safety training programs and automated driving personalized in-vehicle safety systems and roadside devices such as signage.

Immediate effects of an acute bout of repeated soccer heading on cognitive performance

Purpose: There isgrowing concern surrounding the harmful effects of soccer heading on cognitive function. The present study aims to examine the immediate effects of heading.Methods: 30 recreational male soccer players were divided into three groups that undertook 20 consecutive headers with a soft (8.8 psi), hard (16.2 psi), or no (control) ball. A battery of neuropsychological tests was completed before and after heading: King-Devick, trail-making (TM) (A, B), digit span (DS) and spatial span (SS) (forward, backward).Results: Significant increase in the time (M = 4.44 s) and errors (M = 1.45) for the King-Devick test within the hard and soft groups, although there was no significant difference for TM-A and TM-B. Significant decline for SS forward within the hard and soft groups (M= -16%), although the declines for SS backward (M = -16%), DS forward (M= -23%) and DS backward (M = -25%) were present only for the hard group (ps <.05).Discussion: While outside of regular match-play, this study showed that heading negatively influenced one indicator of a suspected concussion (King-Devick), as well as working memory (DS, SS) that is essential for daily life. These findings contribute to the growing debate surrounding heading safety.

In search of a naturalistic neuroimaging approach: Exploration of general feasibility through the case of VR-fMRI and application in the domain of episodic memory

Virtual Reality (VR) is an increasingly widespread tool for research as it allows the creation of experiments taking place in multimodal and daily-life-like environments, while keeping a strong experimental control. Adding neuroimaging to VR leads to a better understanding of the underlying brain networks activated during a naturalistic task, whether for research purposes or rehabilitation. The present paper focuses on the specific use of concurrent VR and fMRI and its technical challenges and feasibility, with a brief examination of the general existing solutions. Following the PRISMA guidelines, the review investigates the particular case of how VR-fMRI has explored episodic memory so far, with a comparison of object- and place-based episodic memory. This review confirms the involvement of cerebral regions well-known to be implicated in episodic memory and unravels other regions devoted to bodily and narrative aspects of the self, promoting new avenues of research in the domain of naturalistic episodic memory. Future studies should develop more immersive and interactive virtual neuroimaging features to increase ecological and embodied neurocognition aspects.

A Methodological Review of fNIRS in Driving Research: Relevance to the Future of Autonomous Vehicles

As automobile manufacturers have begun to design, engineer, and test autonomous driving systems of the future, brain imaging with functional near-infrared spectroscopy (fNIRS) can provide unique insights about cognitive processes associated with evolving levels of autonomy implemented in the automobile. Modern fNIRS devices provide a portable, relatively affordable, and robust form of functional neuroimaging that allows researchers to investigate brain function in real-world environments. The trend toward "naturalistic neuroscience" is evident in the growing number of studies that leverage the methodological flexibility of fNIRS, and in doing so, significantly expand the scope of cognitive function that is accessible to observation via functional brain imaging (i.e., from the simulator to on-road scenarios). While more than a decade's worth of study in this field of fNIRS driving research has led to many interesting findings, the number of studies applying fNIRS during autonomous modes of operation is limited. To support future research that directly addresses this lack in autonomous driving research with fNIRS, we argue that a cogent distillation of the methods used to date will help facilitate and streamline this research of tomorrow. To that end, here we provide a methodological review of the existing fNIRS driving research, with the overarching goal of highlighting the current diversity in methodological approaches. We argue that standardization of these approaches will facilitate greater overlap of methods by researchers from all disciplines, which will, in-turn, allow for meta-analysis of future results. We conclude by providing recommendations for advancing the use of such fNIRS technology in furthering understanding the adoption of safe autonomous vehicle technology.

Animal presence modulates frontal brain activity of patients in a minimally conscious state: A pilot study

Integrating animals into therapy is applied increasingly in patients in a minimally conscious state (MCS). This pilot study investigates the effect of animal presence on frontal brain activity in MCS patients compared to healthy subjects. O₂HB, HHb and tHb of two MCS patients and two healthy adults was measured in frontal cortex using functional near-infrared spectroscopy during three sessions with a live animal and three sessions with a mechanical toy animal present. Each session had five phases: (1) baseline, (2) watching animal, (3) passive contact, (4) active contact, (5) neutral. Data were descriptively analysed. All participants showed the largest hemodynamic response during direct contact with the live or toy animal compared to "baseline" and "watching." During active contact, three of the four participants showed a stronger response when stroking the live compared to the toy animal. All participants showed an inverted signal with higher HHb than O₂Hb concentrations while stroking the live or toy animal. Animal contact leads to a neurovascular reaction in both MCS patients and healthy subjects, indicating elevated neural activity in the frontal cortex. We conclude that while a toy animal can elicit attention processes, active contact to a living animal is combined with emotional processes.

Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study

Optimal mental workload plays a key role in driving performance. Thus, driver-assisting systems that automatically adapt to a drivers current mental workload via brain-computer interfacing might greatly contribute to traffic safety. To design economic brain computer interfaces that do not compromise driver comfort, it is necessary to identify brain areas that are most sensitive to mental workload changes. In this study, we used functional near-infrared spectroscopy and subjective ratings to measure mental workload in two virtual driving environments with distinct demands. We found that demanding city environments induced both higher subjective workload ratings as well as higher bilateral middle frontal gyrus activation than less demanding country environments. A further analysis with higher spatial resolution revealed a center of activation in the right anterior dorsolateral prefrontal cortex. The area is highly involved in spatial working memory processing. Thus, a main component of drivers' mental workload in complex surroundings might stem from the fact that large amounts of spatial information about the course of the road as well as other road users has to constantly be upheld, processed and updated. We propose that the right middle frontal gyrus might be a suitable region for the application of powerful small-area brain computer interfaces.

Neurophysiological correlates of age differences in driving behavior during concurrent subtask performance

Driving is a complex cognitive-motor task that requires the continuous integration of multisensory information, cognitive processes, and motor actions. With higher age, driving becomes increasingly challenging as a result of naturally declining neurophysiological resources. Performing additional subtasks, such as conversations with passengers or interactions with in-vehicle devices (e.g., adjusting the radio), may further challenge neurocognitive resources that are required to maintain driving performance. Based on declining brain physiological resources and inferior neurocognitive functioning, older adults (OA) may show higher brain activation and larger performance decrements than younger adults (YA) when engaging in additional subtasks during driving. Age differences, however, may further vary for different neurocognitive task demands, such that driving performance of OA might be particularly affected by certain subtasks. In this study, we hence investigated the brain functional correlates of age differences in driving behavior during concurrent subtask performance in YA and OA. Our final sample consisted of thirty younger (21.80 ± 1.73y, 15 female) and thirty older (69.43 ± 3.30y, 12 female) regular drivers that drove along a typical rural road (25 - 30 min) in a driving simulator and performed three different concurrent subtasks that were presented auditorily or visually: typing a 3-digit number (TYPE), comparing traffic news and gas station prices (working memory, WM), and stating arguments (ARG). We measured variability in lateral car position, velocity, and following distance to a frontal lead car as the standard deviation from 0 to 15 s after subtask onset. Brain activity was continuously recorded using functional near-infrared spectroscopy over the dorsolateral prefrontal cortex. Both YA and OA particularly varied in their lateral position during TYPE with a more pronounced effect in OA. For YA, in contrast, ARG led to higher variability in velocity compared to TYPE and WM, whereas OA showed no task-specific differences. Substantiating our behavioral findings, OA revealed the largest brain functional response to TYPE, while YA demonstrated a very distinct activation during ARG and smaller hemodynamic responses to TYPE and WM. Brain activity in the DLPFC was, overall, not significantly, but small to moderately related to certain behavioral performance parameters (mainly lateral position). We conclude that both OA and YA are vulnerable to distractive subtasks while driving. Age differences, however, seem to largely depend on neurocognitive task demands. OA may be at higher risk for accidents when performing visuo-motor subtasks (e.g., interacting with navigational systems) during driving while YA may be more (cognitively) distracted when talking to passengers.

Vital Sign Monitoring in Car Seats Based on Electrocardiography, Ballistocardiography and Seismocardiography: A Review

This paper focuses on a thorough summary of vital function measuring methods in vehicles. The focus of this paper is to summarize and compare already existing methods integrated into car seats with the implementation of inter alia capacitive electrocardiogram (cECG), mechanical motion analysis Ballistocardiography (BCG) and Seismocardiography (SCG). In addition, a comprehensive overview of other methods of vital sign monitoring, such as camera-based systems or steering wheel sensors, is also presented in this article. Furthermore, this work contains a very thorough background study on advanced signal processing methods and their potential application for the purpose of vital sign monitoring in cars, which is prone to various disturbances and artifacts occurrence that have to be eliminated.