A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

Cardiac autonomic responses in relation to cognitive workload during simulated military flight

Understanding the operator's cognitive workload is crucial for efficiency and safety in human-machine systems. This study investigated how cognitive workload modulates cardiac autonomic regulation during a standardized military simulator flight. Military student pilots completed simulated flight tasks in a Hawk flight simulator. Continuous electrocardiography was recorded to analyze time and frequency domain heart rate variability (HRV). After the simulation, a flight instructor used a standardized method to evaluate student pilot's individual cognitive workload from video-recorded flight simulator data. Results indicated that HRV was able to differentiate flight phases that induced varying levels of cognitive workload; an increasing level of cognitive workload caused significant decreases in many HRV variables, mainly reflecting parasympathetic deactivation of cardiac autonomic regulation. In conclusion, autonomic physiological responses can be used to examine reactions to increased cognitive workload during simulated military flights. HRV could be beneficial in assessing individual responses to cognitive workload and pilot performance during simulator training.

Event-related driver stress detection with smartphones in an urban environment: a naturalistic driving study

Driving in urban areas can be challenging and encounter acute stress. To detect driver stress, collecting data on real roads without interfering the driver is preferred. A smartphone-based data collection protocol was developed to support a naturalistic driving study. Sixty-one participants drove on predetermined real road routes, and driving information as well as physiological, psychological, and facial data were collected. The algorithm identified potentially stressful events based on the collected data. Participants classified these events as low, medium, or highly stressful events by watching recorded videos after the experiment. These events were then used to train prediction models. The best model achieved an accuracy of 92.5% in classifying low/medium/highly stressful events. The contribution of physiological, psychological, and facial expression indices and individual profile information was evaluated. The method can be applied to visualise the geographical distribution of stressors, monitor driver behaviour, and help drivers regulate their driving habits.

Reliable but multi-dimensional cognitive demand in operating partially automated vehicles: implications for real-world automation research

The reliability of cognitive demand measures in controlled laboratory settings is well-documented; however, limited research has directly established their stability under real-life and high-stakes conditions, such as operating automated technology on actual highways. Partially automated vehicles have advanced to become an everyday mode of transportation, and research on driving these advanced vehicles requires reliable tools for evaluating the cognitive demand on motorists to sustain optimal engagement in the driving process. This study examined the reliability of five cognitive demand measures, while participants operated partially automated vehicles on real roads across four occasions. Seventy-one participants (aged 18-64 years) drove on actual highways while their heart rate, heart rate variability, electroencephalogram (EEG) alpha power, and behavioral performance on the Detection Response Task were measured simultaneously. Findings revealed that EEG alpha power had excellent test-retest reliability, heart rate and its variability were good, and Detection Response Task reaction time and hit-rate had moderate reliabilities. Thus, the current study addresses concerns regarding the reliability of these measures in assessing cognitive demand in real-world automation research, as acceptable test-retest reliabilities were found across all measures for drivers across occasions. Despite the high reliability of each measure, low intercorrelations among measures were observed, and internal consistency was better when cognitive demand was estimated as a multi-factorial construct. This suggests that they tap into different aspects of cognitive demand while operating automation in real life. The findings highlight that a combination of psychophysiological and behavioral methods can reliably capture multi-faceted cognitive demand in real-world automation research.

Drowsiness Mitigation Through Driver State Monitoring Systems: A Scoping Review

OBJECTIVE

To explore the scope of available research and to identify research gaps on in-vehicle interventions for drowsiness that utilize driver monitoring systems (DMS).

BACKGROUND

DMS are gaining popularity as a countermeasure against drowsiness. However, how these systems can be best utilized to guide driver attention is unclear.

METHODS

A scoping review was conducted in adherence to PRISMA guidelines. Five electronic databases (ACM Digital Library, Scopus, IEEE Xplore, TRID, and SAE Mobilus) were systematically searched in April 2022. Original studies examining in-vehicle drowsiness interventions that use DMS in a driving context (e.g., driving simulator and driver interviews) passed the screening. Data on study details, state detection methods, and interventions were extracted.

RESULTS

Twenty studies qualified for inclusion. Majority of interventions involved warnings (n = 16) with an auditory component (n = 14). Feedback displays (n = 4) and automation takeover (n = 4) were also investigated. Multistage interventions (n = 12) first cautioned the driver, then urged them to take an action, or initiated an automation takeover. Overall, interventions had a positive impact on sleepiness levels, driving performance, and user evaluations. Whether interventions effective for one type of sleepiness (e.g., passive vs. active fatigue) will perform well for another type is unclear.

CONCLUSION

Literature mainly focused on developing sensors and improving the accuracy of DMS, but not on the driver interactions with these technologies. More intervention studies are needed in general and for investigating their long-term effects.

APPLICATION

We list gaps and limitations in the DMS literature to guide researchers and practitioners in designing and evaluating effective safety systems for drowsy driving.

From the latin "re-cordis, passing through the heart": autonomic modulation differentiates migraineurs from controls when recounting a significant life event

OBJECTIVE

The literature on clinical psychophysiology highlights the possibility of using Heart Rate Variability (HRV) as an index of psychophysical balance and resilience to stress. This study investigates the differences in stress reactivity and subsequent recovery between a group of migraineurs and healthy controls.

METHODS

Socio-demographic (i.e., sex, age, profession, marital status, and level of education) and psychophysiological (HR and HRV) measures of a group of thirty subjects with migraine (26 migraineurs without aura (86.7%), 2 migraineurs with aura (6.7%), and 2 migraineurs with and without aura (6.7%)) and from thirty healthy control subjects were collected. In particular, HRV was analyzed through frequency-domain parameters, including Low-Frequency (LF; 0.04-0.15 Hz) and High-Frequency (HF; 0.15-0.4 Hz) bands as well as LF/HF ratio during a Psychophysiological Stress Profile (PSP) structured in seven phases: (1) Baseline, (2) Objective stressor 1 (Stroop Test), (3) Rest 1, (4) Objective stressor 2 (Mental Arithmetic Task), (5) Rest 2, (6) Subjective stressor (recount a significant life event), and (7) Rest 3. The LF, HF, and LF/HF ratio values were transformed into a logarithmic scale (i.e., log-LF, log-HF, and log LF/HF ratio). Additionally, LF and HF were converted into normalized units (0-100) (i.e., LF% and HF%) which, in turn, were used to obtain reactivity and recovery to stress through delta values (Δ) calculation.

RESULTS

Subjects with migraine reported greater ΔLF% levels of reactivity and recovery to subjective stressor, demonstrating a prevalence of sympathetic activity while recounting a personal life event. At the same time, a lowering of the same values was found in the subjects of the group control.

DISCUSSION

Our results underline the importance of conducting a psychophysiological assessment in patients with headaches because reduced stress management skills could influence the clinical manifestations of the disease, considering stress as one of the most common triggers for migraine patients.

This Is Your Brain on Autopilot 2.0: The Influence of Practice on Driver Workload and Engagement During On-Road, Partially Automated Driving

OBJECTIVE

This on-road study employed behavioral and neurophysiological measurement techniques to assess the influence of six weeks of practice driving a Level 2 partially automated vehicle on driver workload and engagement.

BACKGROUND

Level 2 partial automation requires a driver to maintain supervisory control of the vehicle to detect "edge cases" that the automation is not equipped to handle. There is mixed evidence regarding whether drivers can do so effectively. There is also an open question regarding how practice and familiarity with automation influence driver cognitive states over time.

METHOD

Behavioral and neurophysiological measures of driver workload and visual engagement were recorded from 30 participants at two testing sessions-with a six-week familiarization period in-between. At both testing sessions, participants drove a vehicle with partial automation engaged (Level 2) and not engaged (Level 0) on two interstate highways while reaction times to the detection response task (DRT) and neurophysiological (EEG) metrics of frontal theta and parietal alpha were recorded.

RESULTS

DRT results demonstrated that partially automated driving placed more cognitive load on drivers than manual driving and six weeks of practice decreased driver workload-though only when the driving environment was relatively simple. EEG metrics of frontal theta and parietal alpha showed null effects of partial automation.

CONCLUSION

Driver workload was influenced by level of automation, specific highway characteristics, and by practice over time, but only on a behavioral level and not on a neural level.

APPLICATION

These findings expand our understanding of the influence of practice on driver cognitive states under Level 2 partial automation.

A multifaceted approach to identifying and managing juvenile delinquency by integrating psycho-physiological indicators

To understand and treat juvenile delinquency, the study explores the relevance of psycho-physiological indicators. It also emphasizes the necessity for thorough research to minimize the gap existing between psycho-physiological measurements and conventional psychosocial components. The study focuses on the relevance of personality features, habituation, and autonomic arousal required to monitor the proper management of delinquent conduct. Through the integration of biological, psychological, and social elements into a multidimensional approach, researchers can uncover novel insights and create cutting-edge therapies for youths who are at risk of delinquent behavior. The study proposes to develop a comprehensive framework that considers biological antecedents in addition to conventional metrics to reach the root cause of delinquency; thereby drawing special attention to current literature and research that emphasizes the psycho-physiological correlates of delinquency. By examining the complex interactions between stress, physiology, emotions, behavior, and social structures, the study highlights the intricacy of delinquent conduct and the necessity for adopting a multifaceted strategy to fully address the problematic areas. Future research paths are emphasized, with a focus on the significance of longitudinal studies, moderating and mediating variables, and creative treatment techniques. By utilizing psycho-physiological markers and psychosocial traits, researchers can tailor intervention strategies to meet individual needs effectively. Early identification of psycho-physiological deficits in children is crucial for implementing successful behavior modification techniques and promoting the well-being of future generations. This is expected to help the government agencies to save time and public money.

Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review

Measuring pilot mental workload (MWL) is crucial for enhancing aviation safety. However, MWL is a multi-dimensional construct that could be affected by multiple factors. Particularly, in the context of a more automated cockpit setting, the traditional methods of assessing pilot MWL may face challenges. Heart rate variability (HRV) has emerged as a potential tool for detecting pilot MWL during real-flight operations. This review aims to investigate the relationship between HRV and pilot MWL and to assess the performance of machine-learning-based MWL detection systems using HRV parameters. A total of 29 relevant papers were extracted from three databases for review based on rigorous eligibility criteria. We observed significant variability across the reviewed studies, including study designs and measurement methods, as well as machine-learning techniques. Inconsistent results were observed regarding the differences in HRV measures between pilots under varying levels of MWL. Furthermore, for studies that developed HRV-based MWL detection systems, we examined the diverse model settings and discovered that several advanced techniques could be used to address specific challenges. This review serves as a practical guide for researchers and practitioners who are interested in employing HRV indicators for evaluating MWL and wish to incorporate cutting-edge techniques into their MWL measurement approaches.

A method to enhance drivers' hazard perception at night based on "knowledge-attitude-practice" theory

During nighttime driving, the inherent challenges of low-illuminance conditions often lead to an increased crash rate and higher fatalities by impairing drivers' ability to recognize imminent hazards. While the severity of this issue is widely recognized, a significant research void exists with regard to strategies to enhance hazard perception under such circumstances. To address this lacuna, our study examined the potential of an intervention grounded in the knowledge-attitude-practice (KAP) framework to bolster nighttime hazard detection among drivers. We engaged a cohort of sixty drivers split randomly into an intervention group (undergoing specialized training) and a control group and employed a holistic assessment that combined eye movement analytics, physiological response monitoring, and driving performance evaluations during simulated scenarios pre- and post-intervention. The data showed that the KAP-centric intervention honed drivers' visual search techniques during nighttime driving, allowing them to confront potential threats with reduced physiological tension and ensuring more adept vehicle handling. These compelling findings support the integration of this methodology in driver training curricula and present an innovative strategy to enhance road safety during nighttime journeys.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

We know what attention is!

Attention is one of the most thoroughly investigated psychological phenomena, yet skepticism about attention is widespread: we do not know what it is, it is too many things, there is no such thing. The deficiencies highlighted are not about experimental work but the adequacy of the scientific theory of attention. Combining common scientific claims about attention into a single theory leads to internal inconsistency. This paper demonstrates that a specific functional conception of attention is incorporated into the tasks used in standard experimental paradigms. In accepting these paradigms as valid probes of attention, we commit to this common conception. The conception unifies work at multiple levels of analysis into a coherent scientific explanation of attention. Thus, we all know what attention is.

Quantifying Workload and Stress in Intensive Care Unit Nurses: Preliminary Evaluation Using Continuous Eye-Tracking

OBJECTIVE

(1) To assess mental workloads of intensive care unit (ICU) nurses in 12-hour working shifts (days and nights) using eye movement data; (2) to explore the impact of stress on the ocular metrics of nurses performing patient care in the ICU.

BACKGROUND

Prior studies have employed workload scoring systems or accelerometer data to assess ICU nurses' workload. This is the first naturalistic attempt to explore nurses' mental workload using eye movement data.

METHODS

Tobii Pro Glasses 2 eye-tracking and Empatica E4 devices were used to collect eye movement and physiological data from 15 nurses during 12-hour shifts (252 observation hours). We used mixed-effect models and an ordinal regression model with a random effect to analyze the changes in eye movement metrics during high stress episodes.

RESULTS

While the cadence and characteristics of nurse workload can vary between day shift and night shift, no significant difference in eye movement values was detected. However, eye movement metrics showed that the initial handoff period of nursing shifts has a higher mental workload compared with other times. Analysis of ocular metrics showed that stress is positively associated with an increase in number of eye fixations and gaze entropy, but negatively correlated with the duration of saccades and pupil diameter.

CONCLUSION

Eye-tracking technology can be used to assess the temporal variation of stress and associated changes with mental workload in the ICU environment. A real-time system could be developed for monitoring stress and workload for intervention development.

Modulating Driver Alertness via Ambient Olfactory Stimulation: A Wearable Electroencephalography Study

Poor alertness levels and related changes in cognitive efficiency are common when performing monotonous tasks such as extended driving. Recent studies have investigated driver alertness decrement and possible strategies for modulating alertness with the goal of improving reaction times to safety critical events. However, most studies rely on subjective measures in assessing alertness changes, while the use of olfactory stimuli, which are known to be strong modulators of cognitive states, has not been commensurately explored in driving alertness settings. To address this gap, in the present study we investigated the effectiveness of olfactory stimuli in modulating the alertness state of drivers and explored the utility of electroencephalography (EEG) in developing objective brain-based tools for assessing the resulting changes in cortical activity. Olfactory stimulation induced a significant differential effect on braking reaction time. The corresponding effect to the cortical activity was characterized using EEG-derived metrics and the devised machine learning framework yielded a high discriminating accuracy (92.1%). Furthermore, neural activity in the alpha frequency band was found to be significantly associated with the observed drivers' behavioral changes. Overall, our results demonstrate the potential of olfactory stimuli to modulate the alertness state and the efficiency of EEG in objectively assessing the resulting cognitive changes.

Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios

Assessing drivers' mental workload is crucial for reducing road accidents. This study examined drivers' mental workload in a simulated auditory-based dual-task driving scenario, with driving tasks as the main task, and auditory-based N-back tasks as the secondary task. A total of three levels of mental workload (i.e., low, medium, high) were manipulated by varying the difficulty levels of the secondary task (i.e., no presence of secondary task, 1-back, 2-back). Multimodal measures, including a set of subjective measures, physiological measures, and behavioral performance measures, were collected during the experiment. The results showed that an increase in task difficulty led to increased subjective ratings of mental workload and a decrease in task performance for the secondary N-back tasks. Significant differences were observed across the different levels of mental workload in multimodal physiological measures, such as delta waves in EEG signals, fixation distance in eye movement signals, time- and frequency-domain measures in ECG signals, and skin conductance in EDA signals. In addition, four driving performance measures related to vehicle velocity and the deviation of pedal input and vehicle position also showed sensitivity to the changes in drivers' mental workload. The findings from this study can contribute to a comprehensive understanding of effective measures for mental workload assessment in driving scenarios and to the development of smart driving systems for the accurate recognition of drivers' mental states.

A Window Into the Tired Brain: Neurophysiological Dynamics of Visuospatial Working Memory Under Fatigue

OBJECTIVE

We examine the spatiotemporal dynamics of neural activity and its correlates in heart rate and its variability (HR/HRV) during a fatiguing visuospatial working memory task.

BACKGROUND

The neural and physiological drivers of fatigue are complex, coupled, and poorly understood. Investigations that combine the fidelity of neural indices and the field-readiness of physiological measures can facilitate measurements of fatigue states in operational settings.

METHOD

Sixteen healthy adults, balanced by sex, completed a 60-minute fatiguing visuospatial working memory task. Changes in task performance, subjective measures of effort and fatigue, cerebral hemodynamics, and HR/HRV were analyzed. Peak brain activation, functional and effective connections within relevant brain networks were contrasted against spectral and temporal features of HR/HRV.

RESULTS

Task performance elicited increased neural activation in regions responsible for maintaining working memory capacity. With the onset of time-on-task effects, resource utilization was seen to increase beyond task-relevant networks. Over time, functional connections in the prefrontal cortex were seen to weaken, with changes in the causal relationships between key regions known to drive working memory. HR/HRV indices were seen to closely follow activity in the prefrontal cortex.

CONCLUSION

This investigation provided a window into the neurophysiological underpinnings of working memory under the time-on-task effect. HR/HRV was largely shown to mirror changes in cortical networks responsible for working memory, therefore supporting the possibility of unobtrusive state recognition under ecologically valid conditions.

APPLICATIONS

Findings here can inform the development of a fieldable index for cognitive fatigue.

Get Ready for Take-Overs: Using Head-Up Display for Drivers to Engage in Non-Driving-Related Tasks in Automated Vehicles

OBJECTIVE

The study aims to investigate the potential of using HUD (head-up display) as an approach for drivers to engage in non-driving-related tasks (NDRTs) during automated driving, and examine the impacts on driver state and take-over performance in comparison to the traditional mobile phone.

BACKGROUND

Advances in automated vehicle technology have the potential to relieve drivers from driving tasks so that they can engage in NDRTs freely. However, drivers will still need to take-over control under certain circumstances.

METHOD

A driving simulation experiment was conducted using an Advanced Driving Simulator and real-world driving videos. Forty-six participants completed three drives in three display conditions, respectively (HUD, mobile phone and baseline without NDRT). The HUD was integrated with the vehicle in displaying NDRTs while the mobile phone was not. Drivers' visual (e.g. gaze, blink) and physiological (e.g. ECG, EDA) data were collected to measure driver state. Two take-over reaction times (hand and foot) were used to measure take-over performance.

RESULTS

The HUD significantly shortened the take-over reaction times compared to the mobile phone condition. Compared to the baseline condition, drivers in the HUD condition also experienced lower cognitive workload and physiological arousal. Drivers' take-over reaction times were significantly correlated with their visual and electrodermal activities during automated driving prior to the take-over request.

CONCLUSION

HUDs can improve driver performance and lower workload when used as an NDRT interface.

APPLICATION

The study sheds light on a promising approach for drivers to engage in NDRTs in future AVs.

A Bibliometric Analysis of Neuroscience Tools Use in Construction Health and Safety Management

Despite longstanding traditional construction health and safety management (CHSM) methods, the construction industry continues to face persistent challenges in this field. Neuroscience tools offer potential advantages in addressing these safety and health issues by providing objective data to indicate subjects' cognition and behavior. The application of neuroscience tools in the CHSM has received much attention in the construction research community, but comprehensive statistics on the application of neuroscience tools to CHSM is lacking to provide insights for the later scholars. Therefore, this study applied bibliometric analysis to examine the current state of neuroscience tools use in CHSM. The development phases; the most productive journals, regions, and institutions; influential scholars and articles; author collaboration; reference co-citation; and application domains of the tools were identified. It revealed four application domains: monitoring the safety status of construction workers, enhancing the construction hazard recognition ability, reducing work-related musculoskeletal disorders of construction workers, and integrating neuroscience tools with artificial intelligence techniques in enhancing occupational safety and health, where magnetoencephalography (EMG), electroencephalography (EEG), eye-tracking, and electrodermal activity (EDA) are four predominant neuroscience tools. It also shows a growing interest in integrating the neuroscience tools with artificial intelligence techniques to address the safety and health issues. In addition, future studies are suggested to facilitate the applications of these tools in construction workplaces by narrowing the gaps between experimental settings and real situations, enhancing the quality of data collected by neuroscience tools and performance of data processing algorithms, and overcoming user resistance in tools adoption.

Assessing the physiological effect of non-driving-related task performance and task modality in conditionally automated driving systems: A systematic review and meta-analysis

In conditionally automated driving, the driver is free to disengage from controlling the vehicle, but they are expected to resume driving in response to certain situations or events that the system is not equipped to respond to. As the level of vehicle automation increases, drivers often engage in non-driving-related tasks (NDRTs), defined as any secondary task unrelated to the primary task of driving. This engagement can have a detrimental effect on the driver's situation awareness and attentional resources. NDRTs with resource demands that overlap with the driving task, such as visual or manual tasks, may be particularly deleterious. Therefore, monitoring the driver's state is an important safety feature for conditionally automated vehicles, and physiological measures constitute a promising means of doing this. The present systematic review and meta-analysis synthesises findings from 32 studies concerning the effect of NDRTs on drivers' physiological responses, in addition to the effect of NDRTs with a visual or a manual modality. Evidence was found that NDRT engagement led to higher physiological arousal, indicated by increased heart rate, electrodermal activity and a decrease in heart rate variability. There was mixed evidence for an effect of both visual and manual NDRT modalities on all physiological measures. Understanding the relationship between task performance and arousal during automated driving is of critical importance to the development of driver monitoring systems and improving the safety of this technology.

This Is Your Brain on Autopilot: Neural Indices of Driver Workload and Engagement During Partial Vehicle Automation

OBJECTIVE

This research explores the effect of partial vehicle automation on neural indices of mental workload and visual engagement during on-road driving.

BACKGROUND

There is concern that the introduction of automated technology in vehicles may lead to low driver stimulation and subsequent disengagement from the driving environment. Simulator-based studies have examined the effect of automation on a driver's cognitive state, but it is unknown how the conclusions translate to on-road driving. Electroencephalographic (EEG) measures of frontal theta and parietal alpha can provide insight into a driver's mental workload and visual engagement while driving under various conditions.

METHOD

EEG was recorded from 71 participants while driving on the roadway. We examined two age cohorts, on two different highway configurations, in four different vehicles, with partial vehicle automation both engaged and disengaged.

RESULTS

Analysis of frontal theta and parietal alpha power revealed that there was no change in mental workload or visual engagement when driving manually compared with driving under partial vehicle automation.

CONCLUSION

Drivers new to the technology remained engaged with the driving environment when operating under partial vehicle automation. These findings suggest that the concern surrounding driver disengagement under vehicle automation may need to be tempered, at least for drivers new to the experience.

APPLICATION

These findings expand our understanding of the effects of partial vehicle automation on drivers' cognitive states.

Regulation of brain cognitive states through auditory, gustatory, and olfactory stimulation with wearable monitoring

Inspired by advances in wearable technologies, we design and perform human-subject experiments. We aim to investigate the effects of applying safe actuation (i.e., auditory, gustatory, and olfactory) for the purpose of regulating cognitive arousal and enhancing the performance states. In two proposed experiments, subjects are asked to perform a working memory experiment called n-back tasks. Next, we incorporate listening to different types of music, drinking coffee, and smelling perfume as safe actuators. We employ signal processing methods to seamlessly infer participants' brain cognitive states. The results demonstrate the effectiveness of the proposed safe actuation in regulating the arousal state and enhancing performance levels. Employing only wearable devices for human monitoring and using safe actuation intervention are the key components of the proposed experiments. Our dataset fills the existing gap of the lack of publicly available datasets for the self-management of internal brain states using wearable devices and safe everyday actuators. This dataset enables further machine learning and system identification investigations to facilitate future smart work environments. This would lead us to the ultimate idea of developing practical automated personalized closed-loop architectures for managing internal brain states and enhancing the quality of life.

Biometrics for Industry 4.0: a survey of recent applications

The Fourth Industrial Revolution, also known as Industry 4.0, represents the rise of digital industrial technology that is propagating at an exponential rate compared to the previous three revolutions. Interoperability is a basis of production, where there is a continuous exchange of information between machines and production units that act autonomously and intelligently. Workers play a central role in making autonomous decisions and using advanced technological tools. It may involve using measures that distinguish individuals, and their behaviours and reactions. Increasing the level of security, allowing only authorized personnel access to designated areas, and promoting worker welfare can have a positive impact on the entire assembly line. Thus, capturing biometric information, with or without individuals' knowledge, could allow identity verification and monitoring of of their emotional and cognitive states during the daily actions of work life. From the study of the literature, we outline three macro categories in which the principles of Industry 4.0 are merged and the functionalities of biometric systems are exploited: security, health monitoring, and quality work life analysis. In this review, we present an overview of all biometric features used in the context of Industry 4.0 with a focus on their advantages, limitations, and practical use. Attention is also paid to future research directions for which new answers are being explored.

EEG-Based Driver Fatigue Monitoring within a Human-Ship-Environment System: Implications for Ship Braking Safety

To address the uncontrollable risks associated with the overreliance on ship operators' driving in current ship safety braking methods, this study aims to reduce the impact of operator fatigue on navigation safety. Firstly, this study established a human-ship-environment monitoring system with functional and technical architecture, emphasizing the investigation of a ship braking model that integrates brain fatigue monitoring using electroencephalography (EEG) to reduce braking safety risks during navigation. Subsequently, the Stroop task experiment was employed to induce fatigue responses in drivers. By utilizing principal component analysis (PCA) to reduce dimensionality across multiple channels of the data acquisition device, this study extracted centroid frequency (CF) and power spectral entropy (PSE) features from channels 7 and 10. Additionally, a correlation analysis was conducted between these features and the Fatigue Severity Scale (FSS), a five-point scale for assessing fatigue severity in the subjects. This study established a model for scoring driver fatigue levels by selecting the three features with the highest correlation and utilizing ridge regression. The human-ship-environment monitoring system and fatigue prediction model proposed in this study, combined with the ship braking model, achieve a safer and more controllable ship braking process. By real-time monitoring and prediction of driver fatigue, appropriate measures can be taken in a timely manner to ensure navigation safety and driver health.

Rethinking infrastructure design: evaluating pedestrians and VRUs' psychophysiological and behavioral responses to different roadway designs

The integration of human-centric approaches has gained more attention recently due to more automated systems being introduced into our built environments (buildings, roads, vehicles, etc.), which requires a correct understanding of how humans perceive such systems and respond to them. This paper introduces an Immersive Virtual Environment-based method to evaluate the infrastructure design with psycho-physiological and behavioral responses from the vulnerable road users, especially for pedestrians. A case study of pedestrian mid-block crossings with three crossing infrastructure designs (painted crosswalk, crosswalk with flashing beacons, and a smartphone app for connected vehicles) are tested. Results from 51 participants indicate there are differences between the subjective and objective measurement. A higher subjective safety rating is reported for the flashing beacon design, while the psychophysiological and behavioral data indicate that the flashing beacon and smartphone app are similar in terms of crossing behaviors, eye tracking measurements, and heart rate. In addition, the smartphone app scenario appears to have a lower stress level as indicated by eye tracking data, although many participants do not have prior experience with it. Suggestions are made for the implementation of new technologies, which can increase public acceptance of new technologies and pedestrian safety in the future.

Wrinkles in subsecond time perception are synchronized to the heart

The role of the heart in the experience of time has been long theorized but empirical evidence is scarce. Here, we examined the interaction between fine-grained cardiac dynamics and the momentary experience of subsecond intervals. Participants performed a temporal bisection task for brief tones (80-188 ms) synchronized with the heart. We developed a cardiac Drift-Diffusion Model (cDDM) that embedded contemporaneous heart rate dynamics into the temporal decision model. Results revealed the existence of temporal wrinkles-dilation or contraction of short intervals-in synchrony with cardiac dynamics. A lower prestimulus heart rate was associated with an initial bias in encoding the millisecond-level stimulus duration as longer, consistent with facilitation of sensory intake. Concurrently, a higher prestimulus heart rate aided more consistent and faster temporal judgments through more efficient evidence accumulation. Additionally, a higher speed of poststimulus cardiac deceleration, a bodily marker of attention, was associated with a greater accumulation of sensory temporal evidence in the cDDM. These findings suggest a unique role of cardiac dynamics in the momentary experience of time. Our cDDM framework opens a new methodological avenue for investigating the role of the heart in time perception and perceptual judgment.

Drivers' self-regulatory behaviors in active and responsive scenarios

OBJECTIVE

Drivers usually appear to self-regulate their driving behaviors in situations considered to be challenging, such as mobile phone-distracted driving. It is important to clarify how drivers self-regulate their actual behaviors. In addition, few studies investigated driver distraction in active and responsive scenarios. Therefore, the present study aimed to gain a better understanding of drivers' actual self-regulation of driving behaviors and phone use behaviors while mobile phone-distracted driving in active and responsive scenarios. The contribution of compensatory beliefs to self-regulation was also explored.

METHODS

This study was conducted using a 2 (mobile phone use behaviors: phone calling vs. WeChat messaging) × 2 (scenarios: active vs. responsive) within-group design. A total of 34 participants completed a driving simulation experiment. The dependent variables of drivers' driving behaviors, phone use behaviors, and physiological data were collected. Participants' compensatory belief was also measured.

RESULTS

The results showed that the speed reduction in the stages with WeChat messaging was significantly greater than that in the stages with phone calls, and the speed reduction in the responsive scenario was significantly greater than that in the active scenario. Participants would adopt relatively equal phone-use-related self-regulatory behaviors in active and responsive scenarios. Participants with higher compensatory beliefs had relatively greater speed reduction in most scenarios, but fewer phone-use-related self-regulatory behaviors. In addition, the respiratory rate could contribute to evaluating the changes in drivers' physiological status during phone calling-distracted driving.

CONCLUSIONS

Participants would self-regulate driving behaviors and phone use behaviors according to different distracted driving tasks and scenarios. The driving-related self-regulation in WeChat messaging scenarios and responsive scenarios was greater. There was a trend in the effect of compensatory beliefs on actual self-regulatory behaviors, which needs to be further verified in the future. This study contributes to the verification of the different actual driving-related and phone-use-related self-regulatory behavior of drivers in active and responsive mobile phone distracted driving scenarios.

Neuroergonomics on the Go: An Evaluation of the Potential of Mobile EEG for Workplace Assessment and Design

OBJECTIVE

We demonstrate and discuss the use of mobile electroencephalogram (EEG) for neuroergonomics. Both technical state of the art as well as measures and cognitive concepts are systematically addressed.

BACKGROUND

Modern work is increasingly characterized by information processing. Therefore, the examination of mental states, mental load, or cognitive processing during work is becoming increasingly important for ergonomics.

RESULTS

Mobile EEG allows to measure mental states and processes under real live conditions. It can be used for various research questions in cognitive neuroergonomics. Besides measures in the frequency domain that have a long tradition in the investigation of mental fatigue, task load, and task engagement, new approaches-like blink-evoked potentials-render event-related analyses of the EEG possible also during unrestricted behavior.

CONCLUSION

Mobile EEG has become a valuable tool for evaluating mental states and mental processes on a highly objective level during work. The main advantage of this technique is that working environments don't have to be changed while systematically measuring brain functions at work. Moreover, the workflow is unaffected by such neuroergonomic approaches.

Detecting driver fatigue using heart rate variability: A systematic review

Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.

EEG-EMG coupling as a hybrid method for steering detection in car driving settings

Understanding mental processes in complex human behavior is a key issue in driving, representing a milestone for developing user-centered assistive driving devices. Here, we propose a hybrid method based on electroencephalographic (EEG) and electromyographic (EMG) signatures to distinguish left and right steering in driving scenarios. Twenty-four participants took part in the experiment consisting of recordings of 128-channel EEG and EMG activity from deltoids and forearm extensors in non-ecological and ecological steering tasks. Specifically, we identified the EEG mu rhythm modulation correlates with motor preparation of self-paced steering actions in the non-ecological task, while the concurrent EMG activity of the left (right) deltoids correlates with right (left) steering. Consequently, we exploited the mu rhythm de-synchronization resulting from the non-ecological task to detect the steering side using cross-correlation analysis with the ecological EMG signals. Results returned significant cross-correlation values showing the coupling between the non-ecological EEG feature and the muscular activity collected in ecological driving conditions. Moreover, such cross-correlation patterns discriminate the steering side earlier relative to the single EMG signal. This hybrid system overcomes the limitation of the EEG signals collected in ecological settings such as low reliability, accuracy, and adaptability, thus adding to the EMG the characteristic predictive power of the cerebral data. These results prove how it is possible to complement different physiological signals to control the level of assistance needed by the driver.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-021-09776-w.

Emotion Recognition: Photoplethysmography and Electrocardiography in Comparison

Automatically recognizing negative emotions, such as anger or stress, and also positive ones, such as euphoria, can contribute to improving well-being. In real-life, emotion recognition is a difficult task since many of the technologies used for this purpose in both laboratory and clinic environments, such as electroencephalography (EEG) and electrocardiography (ECG), cannot realistically be used. Photoplethysmography (PPG) is a non-invasive technology that can be easily integrated into wearable sensors. This paper focuses on the comparison between PPG and ECG concerning their efficacy in detecting the psychophysical and affective states of the subjects. It has been confirmed that the levels of accuracy in the recognition of affective variables obtained by PPG technology are comparable to those achievable with the more traditional ECG technology. Moreover, the affective psychological condition of the participants (anxiety and mood levels) may influence the psychophysiological responses recorded during the experimental tests.

Heart Rate Detection using a Piezoelectric Ceramic Sensor: Preliminary results

Real-time vital signs monitoring, particularly heart rate, is essential in today's medical practice and research. Heart rate detection allows the doctor to monitor the patient's health status to provide immediate action against possible cardiovascular diseases. We present a possible alternative to traditional heart rate signal monitoring systems, a cardiac pulse system using low-cost piezoelectric signal identification. This system could benefit health care and develop continuous pulse waveform monitoring systems. This paper introduces a heartbeat per minute (BPM) cardiac pulse detection system based on a low-cost piezoelectric ceramic sensor (PCS). The PCS is placed under the wrist and adjusted with a silicone wristband to measure the pressure exerted by the radial artery on the sensor and thus obtain the patient's BPM. We propose a signal conditioning stage to reduce the sensor's noise when acquiring the data and make it suitable for real-time BPM visualization. As a comparison, we performed a statistical test to compare the low-cost PCS with types of traditional sensors, along with the help of 21 volunteers. Experimental results show that the data collected by the PCS, when used for heart rate detection, is highly accurate and close to traditional sensor measurements. Therefore, we conclude that the system efficiently monitors the cardiac pulse signal in BPM. Keywords: Heart rate; Piezoelectric, BPM; Pulse Detection.

Ambient Light Conveying Reliability Improves Drivers’ Takeover Performance without Increasing Mental Workload

Drivers of L3 automated vehicles (AVs) are not required to continuously monitor the AV system. However, they must be prepared to take over when requested. Therefore, it is necessary to design an in-vehicle environment that allows drivers to adapt their levels of preparedness to the likelihood of control transition. This study evaluates ambient in-vehicle lighting that continuously communicates the current level of AV reliability, specifically on how it could influence drivers’ take-over performance and mental workload (MW). We conducted an experiment in a driving simulator with 42 participants who experienced 10 take-over requests (TORs). The experimental group experienced a four-stage ambient light display that communicated the current level of AV reliability, which was not provided to the control group. The experimental group demonstrated better take-over performance, based on lower vehicle jerks. Notably, perceived MW did not differ between the groups, and the EEG indices of MW (frontal theta power, parietal alpha power, Task–Load Index) did not differ between the groups. These findings suggest that communicating the current level of reliability using ambient light might help drivers be better prepared for TORs and perform better without increasing their MW.

Gender differences in professional drivers’ fatigue level measured with BAlert mobile app: A psychophysiological, time efficient, accessible, and innovative approach to fatigue management

Addressing fatigue is useful in a variety of scenarios and activities. Fatigue has recently been studied from a psychophysiological standpoint. As a result, the expression and impact of peripheral and central fatigue has been evaluated. Driving is one occupation where tiredness has disastrous consequences. BAlert is a smartphone app that approaches exhaustion with psychophysiological measures. More specifically, it evaluates the level of fatigue via heart rate variability (HRV) data and the cognitive compromise via Stroop effect. The goal of this study is to determine if there are gender differences in fatigue levels among professional drivers using the BAlert app. Statistically significant differences were found in the number of hours awake, in different parameters of HRV (AVNN, PNN50, RMSSD, and SDNN), in the level of stress, as well as in the cognitive response evaluated through the app. The results are discussed and their implications for the management of work fatigue are presented.

Cognitive performance and electroencephalographic variations in air traffic controllers under various mental workload and time of day

The aim of this study was to investigate the effects of mental workload (MWL) and time of day on cognitive performance and electroencephalographic (EEG) parameters of air traffic controllers. EEG signals recorded while 20 professional air traffic controllers performed cognitive tasks [A-X Continuous Performance Test (AX-CPT) and 3-back working memory task] after they were exposed to two levels of task difficulty (high and low MWL) in the morning and afternoon. Significant decreases in cognitive performance were found when the levels of task difficulty increased in both tasks. The results confirmed the sensitivity of the theta and beta activities to levels of task difficulty in the 3-back task, while they were not affected in the AX-CPT. Theta and beta activities were influenced by time of day in the AX-CPT. The findings provide guidance for application of changes in EEG parameters when MWL level is manipulated during the day that could be implemented in future for the development of real-time monitoring systems to improve aviation safety.

Predicting the Degree of Distracted Driving Based on fNIRS Functional Connectivity: A Pilot Study

Distracted driving is one of the main causes of traffic accidents. By predicting the attentional state of drivers, it is possible to prevent distractions and promote safe driving. In this study, we developed a model that could predict the degree of distracted driving based on brain activity. Changes in oxyhemoglobin concentrations were measured in drivers while driving a real car using functional near-infrared spectroscopy (fNIRS). A regression model was constructed for each participant using functional connectivity as an explanatory variable and brake reaction time to random beeps while driving as an objective variable. As a result, we were able to construct a prediction model with the mean absolute error of 5.58 × 102 ms for the BRT of the 12 participants. Furthermore, the regression model with the highest prediction accuracy for each participant was analyzed to gain a better understanding of the neural basis of distracted driving. The 11 of 12 models that showed significant accuracy were classified into five clusters by hierarchical clustering based on their functional connectivity edges used in each cluster. The results showed that the combinations of the dorsal attention network (DAN)-sensory-motor network (SMN) and DAN-ventral attention network (VAN) connections were common in all clusters and that these networks were essential to predict the degree of distraction in complex multitask driving. They also confirmed the existence of multiple types of prediction models with different within- and between-network connectivity patterns. These results indicate that it is possible to predict the degree of distracted driving based on the driver's brain activity during actual driving. These results are expected to contribute to the development of safe driving systems and elucidate the neural basis of distracted driving.

Representation Learning and Pattern Recognition in Cognitive Biometrics: A Survey

Cognitive biometrics is an emerging branch of biometric technology. Recent research has demonstrated great potential for using cognitive biometrics in versatile applications, including biometric recognition and cognitive and emotional state recognition. There is a major need to summarize the latest developments in this field. Existing surveys have mainly focused on a small subset of cognitive biometric modalities, such as EEG and ECG. This article provides a comprehensive review of cognitive biometrics, covering all the major biosignal modalities and applications. A taxonomy is designed to structure the corresponding knowledge and guide the survey from signal acquisition and pre-processing to representation learning and pattern recognition. We provide a unified view of the methodological advances in these four aspects across various biosignals and applications, facilitating interdisciplinary research and knowledge transfer across fields. Furthermore, this article discusses open research directions in cognitive biometrics and proposes future prospects for developing reliable and secure cognitive biometric systems.

The relationship between pupillary baseline manipulated by mental effort or luminance and subsequent pupillary responses

Measuring pupillary response is a prevalent technique to evaluate mental states. It is indispensable to conduct a correction procedure for the pupillary baseline to get a meaningful conclusion from the pupillary response. However, the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response varies among studies. In this study, we used the subtractive and proportional baseline corrections to analyze the results. Furthermore, we manipulated the pupillary baseline through mental effort or luminance in the baseline period and investigated whether the subsequent stimulus-evoked pupillary responses were affected. We found that the mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a higher mental effort, whereas it was unaffected with the baseline manipulated by luminance. Also, the luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a brighter disk, whereas it was unaffected with the baseline manipulated by mental effort. The results could be obtained from subtractive and proportional baseline corrections. Our results suggest that mental effort manipulated pupillary baseline interacts with the subsequent mental effort elicited pupillary response, but not with the luminance elicited pupillary response; the luminance manipulated pupillary baseline interacts with the subsequent luminance elicited pupillary response, but not with the mental effort elicited pupillary response. It is important to consider the ways of controlling the pupillary baseline and subsequent pupillary response simultaneously.

Effects of whole-body vibration on driver drowsiness: A review

INTRODUCTION

Whole-body vibration has direct impacts on driver vigilance by increasing physical and cognitive stress on the driver, which leads to drowsiness, fatigue and road traffic accidents. Although sleep deprivation, sleep apnoea and alcohol consumption can also lead to driver drowsiness, exposure to steady vibration is the factor most readily controlled by changes to vehicle design, yet it has received comparatively less attention.

METHODS

This review investigated interrelationships between the various components of whole-body vibration and the physiological and cognitive parameters that lead to driver drowsiness, as well as the effects of vibration parameters (frequency, amplitude, waveform and duration). Vibrations transmitted to the driver body from the vehicle floor and/or seat have been considered for this review, whereas hand-arm vibration, shocks, acute or transient vibration were excluded from consideration.

RESULTS

Drowsiness is affected by interactions between the frequency, amplitude, waveform and duration of the vibration. Under optimal conditions, whole-body vibration can induce significant drowsiness within 30 min. Low frequency whole-body vibrations, particularly vibrations of 4-10 Hz, are most effective at inducing drowsiness. This review notes some limitations of current studies and suggests directions for future research.

CONCLUSIONS

This review demonstrated a strong causal link exists between whole-body vibration and driver drowsiness. Since driver drowsiness has been established to be a significant contributor to motor vehicle accidents, research is needed to identify ways to minimise the components of whole-body vibration that contribute to drowsiness, as well as devising more effective ways to counteract drowsiness.

PRACTICAL APPLICATIONS

By raising awareness of the vibrational factors that contribute to drowsiness, manufacturers will be prompted to design vehicles that reduce the influence of these factors.

Multimodal driver state modeling through unsupervised learning

Naturalistic driving data (NDD) can help understand drivers' reactions to each driving scenario and provide personalized context to driving behavior. However, NDD requires a high amount of manual labor to label certain driver's state and behavioral patterns. Unsupervised analysis of NDD can be used to automatically detect different patterns from the driver and vehicle data. In this paper, we propose a methodology to understand changes in driver's physiological responses within different driving patterns. Our methodology first decomposes a driving scenario by using a Bayesian Change Point detection model. We then apply the Latent Dirichlet Allocation method on both driver state and behavior data to detect patterns. We present two case studies in which vehicles were equipped to collect exterior, interior, and driver behavioral data. Four patterns of driving behaviors (i.e., harsh brake, normal brake, curved driving, and highway driving), as well as two patterns of driver's heart rate (HR) (i.e., normal vs. abnormal high HR), and gaze entropy (i.e., low versus high), were detected in these two case studies. The findings of these case studies indicated that among our participants, the drivers' HR had a higher fraction of abnormal patterns during harsh brakes, accelerating and curved driving. Additionally, free-flow driving with close to zero accelerations on the highway was accompanied by more fraction of normal HR as well as a lower gaze entropy pattern. With the proposed methodology we can better understand variations in driver's psychophysiological states within different driving scenarios. The findings of this work, has the potential to guide future autonomous vehicles to take actions that are fit to each specific driver.

Trunk sway changes in professional bus drivers during actual shifts on long-distance routes

Although professional bus drivers are required to perform their task while adopting a prolonged constrained sitting posture, existence of possible effects in terms of postural strategies has been scarcely investigated under actual working conditions. This study aimed to characterise modifications of trunk sway in 14 professional bus drivers during regular shifts performed on non-urban routes using a pressure-sensitive mat placed on the seat. Centre-of-pressure (COP) time series were extracted from body-seat pressure data to calculate sway parameters (i.e. sway area, COP path length, COP displacements and velocities). Results show generalised increase in trunk sway as driving progresses, which becomes statistically significant after approximately 70-100 minutes of continuous driving. This may indicate the adoption of specific strategies to cope with discomfort onset or a fatigue-induced alteration of postural features. Trunk sway monitoring of bus drivers may be useful in detecting postural behaviours potentially associated with deteriorating performance and discomfort onset. Practitioner summary: Professional bus drivers operate in sitting position for prolonged time. Such constrained posture may induce discomfort and fatigue. We investigated trunk sway during actual shifts using pressure-sensitive mats. Significant increase of sway was detected after 70 min of continuous driving. Body-seat pressure data could be used as discomfort and fatigue markers. Abbreviations: ANOVA-RM: analysis of variance with repeated measures; AP: antero-posterior; COP: center of pressure; EC: ellipse's centroid; ML: medio-lateral; SA: sway area; SP: sway path.

Analysis of Physiological Signals for Stress Recognition with Different Car Handling Setups

When designing a car, the vehicle dynamics and handling are important aspects, as they can satisfy a purpose in professional racing, as well as contributing to driving pleasure and safety, real and perceived, in regular drivers. In this paper, we focus on the assessment of the emotional response in drivers while they are driving on a track with different car handling setups. The experiments were performed using a dynamic professional simulator prearranged with different car setups. We recorded various physiological signals, allowing us to analyze the response of the drivers and analyze which car setup is more influential in terms of stress arising in the subjects. We logged two skin potential responses (SPRs), the electrocardiogram (ECG) signal, and eye tracking information. In the experiments, three car setups were used (neutral, understeering, and oversteering). To evaluate how these affect the drivers, we analyzed their physiological signals using two statistical tests (t-test and Wilcoxon test) and various machine learning (ML) algorithms. The results of the Wilcoxon test show that SPR signals provide higher statistical significance when evaluating stress among different drivers, compared to the ECG and eye tracking signals. As for the ML classifiers, we count the number of positive or “stress” labels of 15 s SPR time intervals for each subject and each particular car setup. With the support vector machine classifier, the mean value of the number of positive labels for the four subjects is equal to 13.13% for the base setup, 44.16% for the oversteering setup, and 39.60% for the understeering setup. In the end, our findings show that the base car setup appears to be the least stressful, and that our system enables us to effectively recognize stress while the subjects are driving in the different car configurations.

Driver distraction and its effects on partially automated driving performance: A driving simulator study among young-experienced drivers

Drivers of partially automated vehicles (PAVs) are relieved from parts of the driving tasks allocated to the automated driver. Ironically, these drivers are obligated to continuously monitor the driving task at all times and keep their attention on the roadway. This reduction in the driving task's demands and cognitive workload may encourage drivers to engage with non-driving related tasks (NDRT), which may impair drivers' awareness of the road environment and, as a result, compromise safety. This study examined how engagement with a visual-manual NDRT affects the driving performance of PAV drivers. Thirty-seven participants were randomly assigned to one of two experimental conditions in a driving simulator. Each consisted of two simulated drives. The first experimental condition included one drive under manual driving conditions and another under partially automated driving conditions (i.e., L2). Both drives had no NDRT involved. The second experimental condition included one drive under L2 without an NDRT and one drive under L2, including engagement with an NDRT. Participants' eye movements and heart rate were recorded throughout the experiment. Across various measures, the findings showed that under L2 driving conditions, engagement with an NDRT impairs driving performance in two primary aspects: (1) drivers were less aware of road hazards, and (2) their mental workload was higher when they engaged with an NDRT. In addition, the findings reveal that for drivers engaged with an NDRT, the attentional time-sharing strategy between the NDRT and the roadway monitoring task affected the probability of identifying a hazard. This study shows the adverse effects of engagement with an NDRT under L2 driving conditions on driving performance. Future studies should examine different interventions to mitigate these effects, assuring that drivers are constantly aware of the roadway environment.

Validation of a Light EEG-Based Measure for Real-Time Stress Monitoring during Realistic Driving

Driver’s stress affects decision-making and the probability of risk occurrence, and it is therefore a key factor in road safety. This suggests the need for continuous stress monitoring. This work aims at validating a stress neurophysiological measure—a Neurometric—for out-of-the-lab use obtained from lightweight EEG relying on two wet sensors, in real-time, and without calibration. The Neurometric was tested during a multitasking experiment and validated with a realistic driving simulator. Twenty subjects participated in the experiment, and the resulting stress Neurometric was compared with the Random Forest (RF) model, calibrated by using EEG features and both intra-subject and cross-task approaches. The Neurometric was also compared with a measure based on skin conductance level (SCL), representing one of the physiological parameters investigated in the literature mostly correlated with stress variations. We found that during both multitasking and realistic driving experiments, the Neurometric was able to discriminate between low and high levels of stress with an average Area Under Curve (AUC) value higher than 0.9. Furthermore, the stress Neurometric showed higher AUC and stability than both the SCL measure and the RF calibrated with a cross-task approach. In conclusion, the Neurometric proposed in this work proved to be suitable for out-of-the-lab monitoring of stress levels.

Characterization of Indicators for Adaptive Human-Swarm Teaming

Swarm systems consist of large numbers of agents that collaborate autonomously. With an appropriate level of human control, swarm systems could be applied in a variety of contexts ranging from urban search and rescue situations to cyber defence. However, the successful deployment of the swarm in such applications is conditioned by the effective coupling between human and swarm. While adaptive autonomy promises to provide enhanced performance in human-machine interaction, distinct factors must be considered for its implementation within human-swarm interaction. This paper reviews the multidisciplinary literature on different aspects contributing to the facilitation of adaptive autonomy in human-swarm interaction. Specifically, five aspects that are necessary for an adaptive agent to operate properly are considered and discussed, including mission objectives, interaction, mission complexity, automation levels, and human states. We distill the corresponding indicators in each of the five aspects, and propose a framework, named MICAH (i.e., Mission-Interaction-Complexity-Automation-Human), which maps the primitive state indicators needed for adaptive human-swarm teaming.

Let Complexity Bring Clarity: A Multidimensional Assessment of Cognitive Load Using Physiological Measures

The effects of cognitive load on driver behavior and traffic safety are unclear and in need of further investigation. Reliable measures of cognitive load for use in research and, subsequently, in the development and implementation of driver monitoring systems are therefore sought. Physiological measures are of interest since they can provide continuous recordings of driver state. Currently, however, a few issues related to their use in this context are not usually taken into consideration, despite being well-known. First, cognitive load is a multidimensional construct consisting of many mental responses (cognitive load components) to added task demand. Yet, researchers treat it as unidimensional. Second, cognitive load does not occur in isolation; rather, it is part of a complex response to task demands in a specific operational setting. Third, physiological measures typically correlate with more than one mental state, limiting the inferences that can be made from them individually. We suggest that acknowledging these issues and studying multiple mental responses using multiple physiological measures and independent variables will lead to greatly improved measurability of cognitive load. To demonstrate the potential of this approach, we used data from a driving simulator study in which a number of physiological measures (heart rate, heart rate variability, breathing rate, skin conductance, pupil diameter, eye blink rate, eye blink duration, EEG alpha power, and EEG theta power) were analyzed. Participants performed a cognitively loading n-back task at two levels of difficulty while driving through three different traffic scenarios, each repeated four times. Cognitive load components and other coinciding mental responses were assessed by considering response patterns of multiple physiological measures in relation to multiple independent variables. With this approach, the construct validity of cognitive load is improved, which is important for interpreting results accurately. Also, the use of multiple measures and independent variables makes the measurements (when analyzed jointly) more diagnostic-that is, better able to distinguish between different cognitive load components. This in turn improves the overall external validity. With more detailed, diagnostic, and valid measures of cognitive load, the effects of cognitive load on traffic safety can be better understood, and hence possibly mitigated.

Is There a Difference in Brain Functional Connectivity between Chinese Coal Mine Workers Who Have Engaged in Unsafe Behavior and Those Who Have Not?

(1) Background: As a world-recognized high-risk occupation, coal mine workers need various cognitive functions to process the surrounding information to cope with a large number of perceived hazards or risks. Therefore, it is necessary to explore the connection between coal mine workers’ neural activity and unsafe behavior from the perspective of cognitive neuroscience. This study explored the functional brain connectivity of coal mine workers who have engaged in unsafe behaviors (EUB) and those who have not (NUB). (2) Methods: Based on functional near-infrared spectroscopy (fNIRS), a total of 106 workers from the Hongliulin coal mine of Shaanxi North Mining Group, one of the largest modern coal mines in China, completed the test. Pearson’s Correlation Coefficient (COR) analysis, brain network analysis, and two-sample t-test were used to investigate the difference in brain functional connectivity between the two groups. (3) Results: The results showed that there were significant differences in functional brain connectivity between EUB and NUB among the frontopolar area (p = 0.002325), orbitofrontal area (p = 0.02102), and pars triangularis Broca’s area (p = 0.02888). Small-world properties existed in the brain networks of both groups, and the dorsolateral prefrontal cortex had significant differences in clustering coefficient (p = 0.0004), nodal efficiency (p = 0.0384), and nodal local efficiency (p = 0.0004). (4) Conclusions: This study is the first application of fNIRS to the field of coal mine safety. The fNIRS brain functional connectivity analysis is a feasible method to investigate the neuropsychological mechanism of unsafe behavior in coal mine workers in the view of brain science.

The Origins of Passive, Active, and Sleep-Related Fatigue

Driving is a safety-critical task that requires an alert and vigilant driver. Most research on the topic of vigilance has focused on its proximate causes, namely low arousal and resource expenditure. The present article aims to build upon previous work by discussing the ultimate causes, or the processes that tend to precede low arousal and resource expenditure. The authors review different aspects of fatigue that contribute to a loss of vigilance and how they tend to occur; specifically, the neurochemistry of passive fatigue, the electrophysiology of active fatigue, and the chronobiology of sleep-related fatigue.

Sensitivity of Physiological Measures of Acute Driver Stress: A Meta-Analytic Review

Background: The link between driving performance impairment and driver stress is well-established. Identifying and understanding driver stress is therefore of major interest in terms of safety. Although many studies have examined various physiological measures to identify driver stress, none of these has as yet been definitively confirmed as offering definitive all-round validity in practice. Aims: Based on the data available in the literature, our main goal was to provide a quantitative assessment of the sensitivity of the physiological measures used to identify driver stress. The secondary goal was to assess the influence of individual factors (i.e., characteristics of the driver) and ambient factors (i.e., characteristics of the context) on driver stress. Age and gender were investigated as individual factors. Ambient factors were considered through the experimental apparatus (real-road vs. driving simulator), automation driving (manual driving vs. fully autonomous driving) and stressor exposure duration (short vs. long-term). Method: Nine meta-analyses were conducted to quantify the changes in each physiological measure during high-stress vs. low-stress driving. Meta-regressions and subgroup analyses were performed to assess the moderating effect of individual and ambient factors on driver stress. Results: Changes in stress responses suggest that several measures are sensitive to levels of driver stress, including heart rate, R-R intervals (RRI) and pupil diameter. No influence of individual and ambient factors was observed for heart rate. Applications and Perspective: These results provide an initial guide to researchers and practitioners when selecting physiological measures for quantifying driver stress. Based on the results, it is recommended that future research and practice use (i) multiple physiological measures, (ii) a triangulation-based methodology (combination of measurement modalities), and (iii) a multifactorial approach (analysis of the interaction of stressors and moderators).

Bicyclist-evoked arousal and greater attention to bicyclists independently promote safer driving

While attention has consistently been shown to be biased toward threatening objects in experimental settings, our understanding of how attention is modulated when the observer is in an anxious or aroused state and how this ultimately affects behavior is limited. In real-world environments, automobile drivers can sometimes carry negative perceptions toward bicyclists that share the road. It is unclear whether bicyclist encounters on a roadway lead to physiological changes and attentional biases that ultimately influence driving behavior. Here, we examined whether participants in a high-fidelity driving simulator exhibited an arousal response in the presence of a bicyclist and how this modulated eye movements and driving behavior. We hypothesized that bicyclists would evoke a robust arousal and orienting response, the strength of which would be associated with safer driving behavior. The results revealed that encountering a bicyclist evoked negative arousal by both self-report and physiological measures. Physiological and eye-tracking measures were themselves unrelated, however, being independently associated with safer driving behavior. Our findings offer a real-world demonstration of how arousal and attentional prioritization can lead to adaptive behavior.