Measuring Mental Workload with EEG+fNIRS

Adopting Stimulus Detection Tasks for Cognitive Workload Assessment: Some Considerations

OBJECTIVE

This article tackles the issue of correct data interpretation when using stimulus detection tasks for determining the operator's workload.

BACKGROUND

Stimulus detection tasks are a relative simple and inexpensive means of measuring the operator's state. While stimulus detection tasks may be better geared to measure conditions of high workload, adopting this approach for the assessment of low workload may be more problematic.

METHOD

This mini-review details the use of common stimulus detection tasks and their contributions to the Human Factors practice. It also borrows from the conceptual framework of the inverted-U shape model to discuss the issue of data interpretation.

RESULTS

The evidence being discussed here highlights a clear limitation of stimulus detection task paradigms.

CONCLUSION

There is an inherent risk in using a unidimensional tool like stimulus detection tasks as the primary source of information for determining the operator's psychophysiological state.

APPLICATION

Two recommendations are put forward to Human Factors researchers and practitioners dealing with the interpretation conundrum of dealing with stimulus detection tasks.

Strategic Integration: A Cross-Disciplinary Review of the fNIRS-EEG Dual-Modality Imaging System for Delivering Multimodal Neuroimaging to Applications

Background: Recent years have seen a surge of interest in dual-modality imaging systems that integrate functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) to probe brain function. This review aims to explore the advancements and clinical applications of this technology, emphasizing the synergistic integration of fNIRS and EEG. Methods: The review begins with a detailed examination of the fundamental principles and distinctive features of fNIRS and EEG techniques. It includes critical technical specifications, data-processing methodologies, and analysis techniques, alongside an exhaustive evaluation of 30 seminal studies that highlight the strengths and weaknesses of the fNIRS-EEG bimodal system. Results: The paper presents multiple case studies across various clinical domains-such as attention-deficit hyperactivity disorder, infantile spasms, depth of anesthesia, intelligence quotient estimation, and epilepsy-demonstrating the fNIRS-EEG system's potential in uncovering disease mechanisms, evaluating treatment efficacy, and providing precise diagnostic options. Noteworthy research findings and pivotal breakthroughs further reinforce the developmental trajectory of this interdisciplinary field. Conclusions: The review addresses challenges and anticipates future directions for the fNIRS-EEG dual-modal imaging system, including improvements in hardware and software, enhanced system performance, cost reduction, real-time monitoring capabilities, and broader clinical applications. It offers researchers a comprehensive understanding of the field, highlighting the potential applications of fNIRS-EEG systems in neuroscience and clinical medicine.

Investigating the relationship between CSAT scores and prefrontal fNIRS signals during cognitive tasks using a quantum annealing algorithm

Academic achievement is a critical measure of intellectual ability, prompting extensive research into cognitive tasks as potential predictors. Neuroimaging technologies, such as functional near-infrared spectroscopy (fNIRS), offer insights into brain hemodynamics, allowing understanding of the link between cognitive performance and academic achievement. Herein, we explored the association between cognitive tasks and academic achievement by analyzing prefrontal fNIRS signals. A novel quantum annealer (QA) feature selection algorithm was applied to fNIRS data to identify cognitive tasks correlated with CSAT scores. Twelve features (signal mean, median, variance, peak, number of peaks, sum of peaks, range, minimum, kurtosis, skewness, standard deviation, and root mean square) were extracted from fNIRS signals at two time windows (10- and 60-s) to compare results from various feature variable conditions. The feature selection results from the QA-based and XGBoost regressor algorithms were compared to validate the former's performance. In a two-step validation process using multiple linear regression models, model fitness (adjusted R2) and model prediction error (RMSE) values were calculated. The quantum annealer demonstrated comparable performance to classical machine learning models, and specific cognitive tasks, including verbal fluency, recognition, and the Corsi block tapping task, were correlated with academic achievement. Group analyses revealed stronger associations between Tower of London and N-back tasks with higher CSAT scores. Quantum annealing algorithms have significant potential in feature selection using fNIRS data, and represents a novel research approach. Future studies should explore predictors of academic achievement and cognitive ability.

Cognitive Workload Detection of Air Traffic Controllers Based on mRMR and Fewer EEG Channels

For air traffic controllers, the extent of their cognitive workload can significantly impact their cognitive function and response time, consequently influencing their operational efficiency or even resulting in safety incidents. In order to enhance the accuracy and efficiency in determining the cognitive workload of air traffic controllers, a cognitive workload detection method for air traffic controllers based on mRMR and fewer EEG channels was proposed in this study. First of all, a set of features related to gamma waves was initially proposed; subsequently, an EEG feature evaluation method based on the mRMR algorithm was employed to pinpoint the most relevant indicators for the detection of the cognitive workload. Consequently, a model for the detection of the cognitive workload of controllers was developed, and it was optimized by filtering out channel combinations that exhibited higher sensitivity to the workload using the mRMR algorithm. The results demonstrate that the enhanced model achieves the accuracy and stability required for practical applications. Notably, in this study, only three EEG channels were employed to achieve the highly precise detection of the cognitive workload of controllers. This approach markedly increases the practicality of employing EEG equipment for the detection of the cognitive workload and streamlines the detection process.

Cognitive workload estimation using physiological measures: a review

Estimating cognitive workload levels is an emerging research topic in the cognitive neuroscience domain, as participants' performance is highly influenced by cognitive overload or underload results. Different physiological measures such as Electroencephalography (EEG), Functional Magnetic Resonance Imaging, Functional near-infrared spectroscopy, respiratory activity, and eye activity are efficiently used to estimate workload levels with the help of machine learning or deep learning techniques. Some reviews focus only on EEG-based workload estimation using machine learning classifiers or multimodal fusion of different physiological measures for workload estimation. However, a detailed analysis of all physiological measures for estimating cognitive workload levels still needs to be discovered. Thus, this survey highlights the in-depth analysis of all the physiological measures for assessing cognitive workload. This survey emphasizes the basics of cognitive workload, open-access datasets, the experimental paradigm of cognitive tasks, and different measures for estimating workload levels. Lastly, we emphasize the significant findings from this review and identify the open challenges. In addition, we also specify future scopes for researchers to overcome those challenges.

Evaluation of physiological responses to mental workload in n-back and arithmetic tasks

Working memory tasks, such as n-back and arithmetic tasks, are frequently used in studying mental workload. The present study investigated and compared the sensitivity of several physiological measures at three levels of difficulty of n-back and arithmetic tasks. The results showed significant differences in fixation duration and pupil diameter among three task difficulty levels for both n-back and arithmetic tasks. Pupil diameters increase with increasing mental workload, whereas fixation duration decreases. Blink duration and heart rate (HR) were significantly increased as task difficulty increased in the n-back task, while root mean square of successive differences (RMSSD) and standard deviation of R-R intervals (SDNN) were significantly decreased in the arithmetic task. On the other hand, blink rate and Galvanic Skin Response (GSR) were not sensitive enough to assess the differences in task difficulty for both tasks. All significant physiological measures yielded significant differences between low and high task difficulty except for SDNN.Practitioner summary: This study aimed to assess the sensitivity levels of several physiological measures of mental workload in n-back and arithmetic tasks. It showed that pupil diameter was the most sensitive in both tasks. This study also found that most physiological indices are sensitive to an extreme change in task difficulty levels.

Effects of neurofeedback training on the alpha activity in quantitative electroencephalography, cognitive function, and speech perception in elderly with presbycusis: a quasi-experimental study

BACKGROUND

This study aimed to investigate the effects of neurofeedback training (NFT) on alpha activity in quantitative electroencephalography (QEEG), cognitive function, and speech perception in elderly with presbycusis.

METHODS

This study was conducted from June 15 to November 30, 2020. The experimental group (n = 28) underwent NFT, while the control group (n = 31) was instructed to continue with their routine daily life. The NFT conducted for 40 min, two times a week, for a total of 16 sessions and was performed using Neuroharmony S and BrainHealth 2.7. The alpha activity was measured as alpha waves using QEEG. The cognitive function was measured using the Korean version of Mini-Mental Status Examination, digit span forward and backward (DSF and DSB). The speech perception was measured using the word and sentence recognition score (WRS and SRS) using an audiometer with the Korean Standard Monosyllabic Word Lists for Adults.

RESULTS

The experimental group demonstrated improvement in the alpha wave of the left frontal lobe measured as alpha activity (t=-2.521, p = .018); MMSE-K (t=-3.467, p < .01), and DSF (t=-2.646, p < .05) measured as cognitive function; and WRS (t=-3.255, p = .003), and SRS (t=-2.851, p = .008) measured as speech perception compared to the control group.

CONCLUSIONS

This study suggests that NFT could be considered an effective cognitive and auditory rehabilitation method based on brain and cognitive science for improving alpha activity, cognitive function, and speech perception.

Rapid Mental Workload Detection of Air Traffic Controllers with Three EEG Sensors

Air traffic controllers' mental workload significantly impacts their operational efficiency and safety. Detecting their mental workload rapidly and accurately is crucial for preventing aviation accidents. This study introduces a mental workload detection model for controllers based on power spectrum features related to gamma waves. The model selects the feature with the highest classification accuracy, β + θ + α + γ, and utilizes the mRMR (Max-Relevance and Min-Redundancy) algorithm for channel selection. Furthermore, the channels that were less affected by ICA processing were identified, and the reliability of this result was demonstrated by artifact analysis brought about by EMG, ECG, etc. Finally, a model for rapid mental workload detection for controllers was developed and the detection rate for the 34 subjects reached 1, and the accuracy for the remaining subjects was as low as 0.986. In conclusion, we validated the usability of the mRMR algorithm in channel selection and proposed a rapid method for detecting mental workload in air traffic controllers using only three EEG channels. By reducing the number of EEG channels and shortening the data processing time, this approach simplifies equipment application and maintains detection accuracy, enhancing practical usability.

Evaluating the performance of the cognitive workload model with subjective endorsement in addition to EEG

The aptitude-oriented exercises from almost all domains impose cognitive load on their operators. Evaluating such load poses several challenges owing to many factors like measurement mode and complexity, nature of the load, overloading conditions, etc. Nevertheless, the physiological measurement of a specific genre of cognitive load and subjective measurement have not been reported along with each other. In this study, the electroencephalography (EEG)-driven machine learning (Support Vector Machine (SVM)) model is sought along with the support of NASA's Task Load Index (NASA-TLX) rating scale for a novel purpose in workload exploration of operators. The Cognitive Load Theory (CLT) was used as the foundation to design the intrinsic stimulus (Spot the Difference task), as most workloads operators are exposed to are notably intrinsic. The SVM-based three-level classification accuracy ranged from 85.4 to 97.4% (p < 0.05), and the NASA-TLX-based three-level classification accuracy ranged from 88.33 to 97.33%. The t-test results show that the neurometric indices contributing to the classification significantly differed (p < 0.05) for every level. The NASA-TLX scale was utilised for validation in its basic form after the validity (Pearson correlation coefficients 0.338 to 0.805 (p < 0.05)) and reliability (Cronbach's α = 0.753) test. This modeling is beneficial to phase out particular-level cognitive exercises from the curriculum during under or overload workload (critical) conditions.

Novel Feature Generation for Classification of Motor Activity from Functional Near-Infrared Spectroscopy Signals Using Machine Learning

Recent research in the field of cognitive motor action decoding focuses on data acquired from Functional Near-Infrared Spectroscopy (fNIRS) and its analysis. This research aims to classify two different motor activities, namely, mental drawing (MD) and spatial navigation (SN), using fNIRS data from non-motor baseline data and other motor activities. Accurate activity detection in non-stationary signals like fNIRS is challenging and requires complex feature descriptors. As a novel framework, a new feature generation by fusion of wavelet feature, Hilbert, symlet, and Hjorth parameters is proposed for improving the accuracy of the classification. This new fused feature has statistical descriptor elements, time-localization in the frequency domain, edge feature, texture features, and phase information to detect and locate the activity accurately. Three types of independent component analysis, including FastICA, Picard, and Infomax were implemented for preprocessing which removes noises and motion artifacts. Two independent binary classifiers are designed to handle the complexity of classification in which one is responsible for mental drawing (MD) detection and the other one is spatial navigation (SN). Four different types of algorithms including nearest neighbors (KNN), Linear Discriminant Analysis (LDA), light gradient-boosting machine (LGBM), and Extreme Gradient Boosting (XGBOOST) were implemented. It has been identified that the LGBM classifier gives high accuracies-98% for mental drawing and 97% for spatial navigation. Comparison with existing research proves that the proposed method gives the highest classification accuracies. Statistical validation of the proposed new feature generation by the Kruskal-Wallis H-test and Mann-Whitney U non-parametric test proves the reliability of the proposed mechanism.

Working memory load recognition with deep learning time series classification

Working memory load (WML) is one of the widely applied signals in the areas of human-machine interaction. The precise evaluation of the WML is crucial for this kind of application. This study aims to propose a deep learning (DL) time series classification (TSC) model for inter-subject WML decoding. We used fNIRS to record the hemodynamic signals of 27 participants during visual working memory tasks. Traditional machine learning and deep time series classification algorithms were respectively used for intra-subject and inter-subject WML decoding from the collected blood oxygen signals. The intra-subject classification accuracy of LDA and SVM were 94.6% and 79.1%. Our proposed TAResnet-BiLSTM model had the highest inter-subject WML decoding accuracy, reaching 92.4%. This study provides a new idea and method for the brain-computer interface application of fNIRS in real-time WML detection.

Trust in Shared-Space Collaborative Robots: Shedding Light on the Human Brain

BACKGROUND

Industry 4.0 is currently underway allowing for improved manufacturing processes that leverage the collective advantages of human and robot agents. Consideration of trust can improve the quality and safety in such shared-space human-robot collaboration environments.

OBJECTIVE

The use of physiological response to monitor and understand trust is currently limited due to a lack of knowledge on physiological indicators of trust. This study examines neural responses to trust within a shared-workcell human-robot collaboration task as well as discusses the use of granular and multimodal perspectives to study trust.

METHODS

Sixteen sex-balanced participants completed a surface finishing task in collaboration with a UR10 collaborative robot. All participants underwent robot reliability conditions and robot assistance level conditions. Brain activation and connectivity using functional near infrared spectroscopy, subjective responses, and performance were measured throughout the study.

RESULTS

Significantly, increased neural activation was observed in response to faulty robot behavior within the medial and right dorsolateral prefrontal cortex (PFC). A similar trend was observed for the anterior PFC, primary motor cortex, and primary visual cortex. Faulty robot behavior also resulted in reduced functional connectivity strengths throughout the brain.

DISCUSSION

These findings implicate regions in the prefrontal cortex along with specific connectivity patterns as signifiers of distrusting conditions. The neural response may be indicative of how trust is influenced, measured, and manifested for human-robot collaboration that requires active teaming.

APPLICATION

Neuroergonomic response metrics can reveal new perspectives on trust in automation that subjective responses alone are not able to provide.

Viewer Engagement in Response to Mixed and Uniform Emotional Content in Marketing Videos-An Electroencephalographic Study

This study presents the results of an experiment designed to investigate whether marketing videos containing mixed emotional content can sustain consumers interest longer compared to videos conveying a consistent emotional message. During the experiment, thirteen participants, wearing EEG (electroencephalographic) caps, were exposed to eight marketing videos with diverse emotional tones. Participant engagement was measured with an engagement index, a metric derived from the power of brain activity recorded over the frontal and parietal cortex and computed within three distinct frequency bands: theta (4-8 Hz), alpha (8-13 Hz), and beta (13-30 Hz). The outcomes indicated a statistically significant influence of emotional content type (mixed vs. consistent) on the duration of user engagement. Videos containing a mixed emotional message were notably more effective in sustaining user engagement, whereas the engagement level for videos with a consistent emotional message declined over time. The principal inference drawn from the study is that advertising materials conveying a consistent emotional message should be notably briefer than those featuring a mixed emotional message to achieve an equivalent level of message effectiveness, measured through engagement duration.

Neurophysiological measures and correlates of cognitive load in attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and dyslexia: A scoping review and research recommendations

Working memory is integral to a range of critical cognitive functions such as reasoning and decision-making. Although alterations in working memory have been observed in neurodivergent populations, there has been no review mapping how cognitive load is measured in common neurodevelopmental conditions such as attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and dyslexia. This scoping review explores the neurophysiological measures used to study cognitive load in these specific populations. Our findings highlight that electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are the most frequently used methods, with a limited number of studies employing functional near-infrared spectroscopy (fNIRs), magnetoencephalography (MEG) or eye-tracking. Notably, eye-related measures are less commonly used, despite their prominence in cognitive load research among neurotypical individuals. The review also highlights potential correlates of cognitive load, such as neural oscillations in the theta and alpha ranges for EEG studies, blood oxygenation level-dependent (BOLD) responses in lateral and medial frontal brain regions for fMRI and fNIRS studies and eye-related measures such as pupil dilation and blink rate. Finally, critical issues for future studies are discussed, including the technical challenges associated with multimodal approaches, the possible impact of atypical features on cognitive load measures and balancing data richness with participant well-being. These insights contribute to a more nuanced understanding of cognitive load measurement in neurodivergent populations and point to important methodological considerations for future neuroscientific research in this area.

Investigation of the neural correlation with task performance and its effect on cognitive load level classification

Electroencephalogram (EEG)-based cognitive load assessment is now an important assignment in psychological research. This type of research work is conducted by providing some mental task to the participants and their responses are counted through their EEG signal. In general assumption, it is considered that during different tasks, the cognitive workload is increased. This paper has investigated this specific idea and showed that the conventional hypothesis is not correct always. This paper showed that cognitive load can be varied according to the performance of the participants. In this paper, EEG data of 36 participants are taken against their resting and task (mental arithmetic) conditions. The features of the signal were extracted using the empirical mode decomposition (EMD) method and classified using the support vector machine (SVM) model. Based on the classification accuracy, some hypotheses are built upon the impact of subjects' performance on cognitive load. Based on some statistical consideration and graphical justification, it has been shown how the hypotheses are valid. This result will help to construct the machine learning-based model in predicting the cognitive load assessment more appropriately in a subject-independent approach.

Enhancing EEG-based cross-day mental workload classification using periodic component of power spectrum

Objective. The day-to-day variability of electroencephalogram (EEG) poses a significant challenge to decode human brain activity in EEG-based passive brain-computer interfaces (pBCIs). Conventionally, a time-consuming calibration process is required to collect data from users on a new day to ensure the performance of the machine learning-based decoding model, which hinders the application of pBCIs to monitor mental workload (MWL) states in real-world settings.Approach. This study investigated the day-to-day stability of the raw power spectral density (PSD) and their periodic and aperiodic components decomposed by the Fitting Oscillations and One-Over-F algorithm. In addition, we validated the feasibility of using periodic components to improve cross-day MWL classification performance.Main results. Compared to the raw PSD (69.9% ± 18.5%) and the aperiodic component (69.4% ± 19.2%), the periodic component had better day-to-day stability and significantly higher cross-day classification accuracy (84.2% ± 11.0%).Significance. These findings indicate that periodic components of EEG have the potential to be applied in decoding brain states for more robust pBCIs.

Adaptive Human-Robotic Interaction for Robotic-assisted Surgical Settings

INTRODUCTION

Increased complexity in robotic-assisted surgical system interfaces introduces problems with human-robot collaboration that result in excessive mental workload (MWL), adversely impacting a surgeon's task performance and increasing error probability. Real-time monitoring of the operator's MWL will aid in identifying when and how interventions can be best provided to moderate MWL. In this study, an MWL-based adaptive automation system is constructed and evaluated for its effectiveness during robotic-assisted surgery.

MATERIALS AND METHODS

This study recruited 10 participants first to perform surgical tasks under different cognitive workload levels. Physiological signals were obtained and employed to build a real-time system for cognitive workload monitoring. To evaluate the effectiveness of the proposed system, 15 participants were recruited to perform the surgical task with and without the proposed system. The participants' task performance and perceived workload were collected and compared.

RESULTS

The proposed neural network model achieved an accuracy of 77.9% in cognitive workload classification. In addition, better task performance and lower perceived workload were observed when participants completed the experimental task under the task condition supplemented with adaptive aiding using the proposed system.

CONCLUSIONS

The proposed MWL monitoring system successfully diminished the perceived workload of participants and increased their task performance under high-stress conditions via interventions by a semi-autonomous suction tool. The preliminary results from the comparative study show the potential impact of automated adaptive aiding systems in enhancing surgical task performance via cognitive workload-triggered interventions in robotic-assisted surgery.

Relaxed Alert Electroencephalography Screening for Mild Traumatic Brain Injury in Athletes

Due to the mildness of initial injury, many athletes with recurrent mild traumatic brain injury (mTBI) are misdiagnosed with other neuropsychiatric illnesses. This study was designed as a proof-of-principle feasibility trial for athletic trainers at a sports facility to generate electroencephalograms (EEGs) from student athletes for discriminating (mTBI) associated EEGs from uninjured ones. A total of 47 EEGs were generated, with 30 athletes recruited at baseline (BL) pre-season, after a concussive injury (IN), and post-season (PS). Outcomes included: 1) visual analyses of EEGs by a neurologist; 2) support vector machine (SVM) classification for inferences about whether particular groups belonged to the three subgroups of BL, IN, or PS; and 3) analyses of EEG synchronies including phase locking value (PLV) computed between pairs of distinct electrodes. All EEGs were visually interpreted as normal. SVM classification showed that BL and IN could be discriminated with 81% accuracy using features of EEG synchronies combined. Frontal inter-hemispheric phase synchronization measured by PLV was significantly lower in the IN group. It is feasible for athletic trainers to record high quality EEGs from student athletes. Also, spatially localized metrics of EEG synchrony can discriminate mTBI associated EEGs from control EEGs.

From brain to worksite: the role of fNIRS in cognitive studies and worker safety

Effective hazard recognition and decision-making are crucial factors in ensuring workplace safety in the construction industry. Workers' cognition closely relates to that hazard-handling behavior. Functional near-infrared spectroscopy (fNIRS) is a neurotechique tool that can evaluate the concentration vibration of oxygenated hemoglobin [ H b O 2 ] and deoxygenated hemoglobin [H b R ] to reflect the cognition process. It is essential to monitor workers' brain activity by fNIRS to analyze their cognitive status and reveal the mechanism in hazard recognition and decision-making process, providing guidance for capability evaluation and management enhancement. This review offers a systematic assessment of fNIRS, encompassing the basic theory, experiment analysis, data analysis, and discussion. A literature search and content analysis are conducted to identify the application of fNIRS in construction safety research, the limitations of selected studies, and the prospects of fNIRS in future research. This article serves as a guide for researchers keen on harnessing fNIRS to bolster construction safety standards and forwards insightful recommendations for subsequent studies.

Assessing the Impact of Electrosuit Therapy on Cerebral Palsy: A Study on the Users' Satisfaction and Potential Efficacy

The aim of this study is to evaluate the effectiveness of electrosuit therapy in the clinical treatment of children with Cerebral Palsy, focusing on the effect of the therapy on spasticity and trunk control. Moreover, the compliance of caregivers with respect to the use of the tool was investigated. During the period ranging from 2019 to 2022, a total of 26 children (18 M and 8 F), clinically stable and affected by CP and attending the Neurorehabilitation Unit of the "Padre Pio Foundation and Rehabilitation Centers", were enrolled in this study. A subset of 12 patients bought or rented the device; thus, they received the administration of the EMS-based therapy for one month, whereas the others received only one-hour training to evaluate the feasibility (by the caregivers) and short-term effects. The Gross Motor Function Classification System was utilized to evaluate gross motor functions and to classify the study sample, while the MAS and the LSS were employed to assess the outcomes of the EMS-based therapy. Moreover, between 80% and 90% of the study sample were satisfied with the safety, ease of use, comfort, adjustment, and after-sales service. Following a single session of electrical stimulation with EMS, patients exhibited a statistically significant enhancement in trunk control. For those who continued this study, the subscale of the QUEST with the best score was adaptability (0.74 ± 0.85), followed by competence (0.67 ± 0.70) and self-esteem (0.59 ± 0.60). This study investigates the impact of the employment of the EMS on CP children's ability to maintain trunk control. Specifically, after undergoing a single EMS session, LSS showed a discernible improvement in children's trunk control. In addition, the QUEST and the PIADS questionnaires demonstrated a good acceptability and satisfaction of the garment by the patients and the caregivers.

Characterisation of Cognitive Load Using Machine Learning Classifiers of Electroencephalogram Data

A high cognitive load can overload a person, potentially resulting in catastrophic accidents. It is therefore important to ensure the level of cognitive load associated with safety-critical tasks (such as driving a vehicle) remains manageable for drivers, enabling them to respond appropriately to changes in the driving environment. Although electroencephalography (EEG) has attracted significant interest in cognitive load research, few studies have used EEG to investigate cognitive load in the context of driving. This paper presents a feasibility study on the simulation of various levels of cognitive load through designing and implementing four driving tasks. We employ machine learning-based classification techniques using EEG recordings to differentiate driving conditions. An EEG dataset containing these four driving tasks from a group of 20 participants was collected to investigate whether EEG can be used as an indicator of changes in cognitive load. The collected dataset was used to train four Deep Neural Networks and four Support Vector Machine classification models. The results showed that the best model achieved a classification accuracy of 90.37%, utilising statistical features from multiple frequency bands in 24 EEG channels. Furthermore, the Gamma and Beta bands achieved higher classification accuracy than the Alpha and Theta bands during the analysis. The outcomes of this study have the potential to enhance the Human-Machine Interface of vehicles, contributing to improved safety.

Automatic depression diagnosis through hybrid EEG and near-infrared spectroscopy features using support vector machine

Depression is a common mental disorder that seriously affects patients' social function and daily life. Its accurate diagnosis remains a big challenge in depression treatment. In this study, we used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) and measured the whole brain EEG signals and forehead hemodynamic signals from 25 depression patients and 30 healthy subjects during the resting state. On one hand, we explored the EEG brain functional network properties, and found that the clustering coefficient and local efficiency of the delta and theta bands in patients were significantly higher than those in normal subjects. On the other hand, we extracted brain network properties, asymmetry, and brain oxygen entropy as alternative features, used a data-driven automated method to select features, and established a support vector machine model for automatic depression classification. The results showed the classification accuracy was 81.8% when using EEG features alone and increased to 92.7% when using hybrid EEG and fNIRS features. The brain network local efficiency in the delta band, hemispheric asymmetry in the theta band and brain oxygen sample entropy features differed significantly between the two groups (p < 0.05) and showed high depression distinguishing ability indicating that they may be effective biological markers for identifying depression. EEG, fNIRS and machine learning constitute an effective method for classifying depression at the individual level.

Hybrid Deep Learning Approach for Stress Detection Using Decomposed EEG Signals

Stress has an impact, not only on a person's physical health, but also on the ability to perform at the workplace in daily life. The well-established relation between psychological stress and its pathogeneses highlights the need for detecting psychological stress early, in order to prevent disease advancement and to save human lives. Electroencephalography (EEG) signal recording tools are widely used to collect these psychological signals/brain rhythms in the form of electric waves. The aim of the current research was to apply automatic feature extraction to decomposed multichannel EEG recordings, in order to efficiently detect psychological stress. The traditional deep learning techniques, namely the convolution neural network (CNN), long short-term memory (LSTM), bidirectional long short-term memory (BiLSTM), gated recurrent unit (GRU) and recurrent neural network (RNN) models, have been frequently used for stress detection. A hybrid combination of these techniques may provide improved performance, and can handle long-term dependencies in non-linear brain signals. Therefore, this study proposed an integration of deep learning models, called DWT-based CNN, BiLSTM, and two layers of a GRU network, to extract features and classify stress levels. Discrete wavelet transform (DWT) analysis was used to remove the non-linearity and non-stationarity from multi-channel (14 channel) EEG recordings, and to decompose them into different frequency bands. The decomposed signals were utilized for automatic feature extraction using the CNN, and the stress levels were classified using BiLSTM and two layers of GRU. This study compared five combinations of the CNN, LSTM, BiLSTM, GRU and RNN models with the proposed model. The proposed hybrid model performed better in classification accuracy compared to the other models. Therefore, hybrid combinations are appropriate for the clinical intervention and prevention of mental and physical problems.

Mental workload level assessment based on compounded hysteresis effect

In the domain of neuroergonomics, cognitive workload estimation has taken a significant concern among the researchers. This is because the knowledge gathered from its estimation is useful for distributing tasks among the operators, understanding human capability and intervening operators at times of havoc. Brain signals give a promising prospective for understanding cognitive workload. For this, electroencephalography (EEG) is by far the most efficient modality in interpreting the covert information arising in the brain. The present work explores the feasibility of EEG rhythms for monitoring continuous change occurring in a person's cognitive workload. This continuous monitoring is achieved by graphicallyinterpreting the cumulative effect of changes in EEG rhythms observed in the current instance and the former instance based on the hysteresis effect. In this work, classification is done to predict the data class label using an artificial neural network (ANN) architecture. The proposed model gives a classification accuracy of 98.66%.

Identification of autism spectrum disorder based on functional near-infrared spectroscopy using adaptive spatiotemporal graph convolution network

The accurate diagnosis of autism spectrum disorder (ASD) is of great practical significance in clinical practice. The spontaneous hemodynamic fluctuations were collected by functional near-infrared spectroscopy (fNIRS) from the bilateral frontal and temporal cortices of typically developing (TD) children and children with ASD. Since traditional machine learning and deep learning methods cannot make full use of the potential spatial dependence between variable pairs, and require a long time series to diagnose ASD. Therefore, we use adaptive spatiotemporal graph convolution network (ASGCN) and short time series to classify ASD and TD. To capture spatial and temporal features of fNIRS multivariable time series without the pre-defined graph, we combined the improved adaptive graph convolution network (GCN) and gated recurrent units (GRU). We conducted a series of experiments on the fNIRS dataset, and found that only using 2.1 s short time series could achieve high precision classification, with an accuracy of 95.4%. This suggests that our approach may have the potential to detect pathological signals in autism patients within 2.1 s. In different brain regions, the left frontal lobe has the best classification effect, and the abnormalities occur more frequently in left hemisphere and frontal lobe region. Moreover, we also found that there were correlations between multiple channels, which had different degrees of influence on the classification of ASD. From this, we can also generalize to a wider range, there may be potential correlations between different brain regions. This may help to better understand the cortical mechanism of ASD.

Benchmarking framework for machine learning classification from fNIRS data

Background

While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods

We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion

Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.

A Systematic Review of In-Vehicle Physiological Indices and Sensor Technology for Driver Mental Workload Monitoring

The concept of vehicle automation ceases to seem futuristic with the current advancement of the automotive industry. With the introduction of conditional automated vehicles, drivers are no longer expected to focus only on driving activities but are still required to stay alert to resume control. However, fluctuations in driving demands are known to alter the driver's mental workload (MWL), which might affect the driver's vehicle take-over capabilities. Driver mental workload can be specified as the driver's capacity for information processing for task performance. This paper summarizes the literature that relates to analysing driver mental workload through various in-vehicle physiological sensors focusing on cardiovascular and respiratory measures. The review highlights the type of study, hardware, method of analysis, test variable, and results of studies that have used physiological indices for MWL analysis in the automotive context.

Functional near-infrared spectroscopy measures of neural activity in children with and without developmental language disorder during a working memory task

INTRODUCTION

Children with developmental language disorder (DLD) exhibit cognitive deficits that interfere with their ability to learn language. Little is known about the functional neuroanatomical differences between children developing typically (TD) and children with DLD.

METHODS

Using functional near-infrared spectroscopy, we recorded oxygenated hemoglobin (O2 hb) concentration values associated with neural activity in children with and without DLD during an auditory N-back task that included 0-back, 1-back, and 2-back conditions. Analyses focused on the left dorsolateral prefrontal cortex (DLPFC) and left inferior parietal lobule (IPL). Multilevel models were constructed with accuracy, response time, and O2 hb as outcome measures, with 0-back outcomes as fixed effects to control for sustained attention.

RESULTS

Children with DLD were significantly less accurate than their TD peers at both the 1-back and 2-back tasks, and they demonstrated slower response times during 2-back. In addition, children in the TD group demonstrated significantly greater sensitivity to increased task difficulty, showing increased O2 hb to the IPL during 1-back and to the DLPFC during the 2-back, whereas the DLD group did not. A secondary analysis revealed that higher O2 hb in the DLPFC predicted better task accuracy across groups.

CONCLUSION

When task difficulty increased, children with DLD failed to recruit the DLPFC for monitoring information and the IPL for processing information. Reduced memory capacity and reduced engagement likely contribute to the language learning difficulties of children with DLD.

Cognitive Analyses for Interface Design Using Dual N-Back Tasks for Mental Workload (MWL) Evaluation

In the manufacturing environments of today, human-machine systems are constituted with complex and advanced technology, which demands workers' considerable mental workload. This work aims to design and evaluate a Graphical User Interface developed to induce mental workload based on Dual N-Back tasks for further analysis of human performance. This study's contribution lies in developing proper cognitive analyses of the graphical user interface, identifying human error when the Dual N-Back tasks are presented in an interface, and seeking better user-system interaction. Hierarchical task analysis and the Task Analysis Method for Error Identification were used for the cognitive analysis. Ten subjects participated voluntarily in the study, answering the NASA-TLX questionnaire at the end of the task. The NASA-TLX results determined the subjective participants' mental workload proving that the subjects were induced to different levels of mental workload (Low, Medium, and High) based on the ANOVA statistical results using the mean scores obtained and cognitive analysis identified redesign opportunities for graphical user interface improvement.

Impacts of complex electromagnetic radiation and low-frequency noise exposure conditions on the cognitive function of operators

Background

Both electromagnetic radiation (EMR) and low-frequency noise (LFN) are widespread and influential environmental factors, and operators are inevitably exposed to both EMR and LFN within a complex exposure environment. The potential adverse effects of such exposure on human health must be considered seriously. This study aimed to investigate the effects of EMR and LFN on cognitive function as well as their interaction effect, which remain unclear.

Methods

Sixty young male college students were randomly grouped and experiments were conducted with a 2 × 2 factorial design in a shielded chamber. Mental workload (MWL) levels of the study subjects were measured and assessed using the NASA-task load index (TLX) subjective scale, an n-back task paradigm, and the functional near-infrared spectroscopy (fNIRS) imaging technique.

Results

For the 3-back task, the NASA-TLX subjective scale revealed a statistically significant main effect of LFN intensity, which enhanced the subjects' MWL level (F = 8.716, p < 0.01). Behavioral performance revealed that EMR intensity (430.1357 MHz, 10.75 W/m²) and LFN intensity (0-200 Hz, 72.9 dB) had a synergistic interaction effect, and the correct response time was statistically significantly prolonged by the combined exposure (F = 4.343, p < 0.05). The fNIRS imaging technique revealed a synergistic interaction effect between operational EMR intensity and operational LFN intensity, with statistically significant effects on the activation levels in the left and right dorsolateral prefrontal cortex (DLPFC). The mean β values of DLPFC were significantly increased (L-DLPFC F = 5.391, p < 0.05, R-DLPFC F = 4.222, p < 0.05), and the relative concentrations of oxyhemoglobin in the DLPFC were also significantly increased (L-DLPFC F = 4.925, p < 0.05, R-DLPFC F = 9.715, p < 0.01).

Conclusion

We found a statistically significant interaction effect between EMR (430.1357 MHz, 10.75 W/m²) and LFN (0-200 Hz, 72.9 dB) when simultaneously exposing subjects to both for 30 min. We conclude that exposure to this complex environment can cause a statistically significant increase in the MWL level of operators, and even alterations in their cognitive function.

Screening for Alzheimer's disease using prefrontal resting-state functional near-infrared spectroscopy

INTRODUCTION

Alzheimer's disease (AD) is neurodegenerative dementia that causes neurovascular dysfunction and cognitive impairment. Currently, 50 million people live with dementia worldwide, and there are nearly 10 million new cases every year. There is a need for relatively less costly and more objective methods of screening and early diagnosis.

METHODS

Functional near-infrared spectroscopy (fNIRS) systems are a promising solution for the early Detection of AD. For a practical clinically relevant system, a smaller number of optimally placed channels are clearly preferable. In this study, we investigated the number and locations of the best-performing fNIRS channels measuring prefrontal cortex activations. Twenty-one subjects diagnosed with AD and eighteen healthy controls were recruited for the study.

RESULTS

We have shown that resting-state fNIRS recordings from a small number of prefrontal locations provide a promising methodology for detecting AD and monitoring its progression. A high-density continuous-wave fNIRS system was first used to verify the relatively lower hemodynamic activity in the prefrontal cortical areas observed in patients with AD. By using the episode averaged standard deviation of the oxyhemoglobin concentration changes as features that were fed into a Support Vector Machine; we then showed that the accuracy of subsets of optical channels in predicting the presence and severity of AD was significantly above chance. The results suggest that AD can be detected with a 0.76 sensitivity score and a 0.68 specificity score while the severity of AD could be detected with a 0.75 sensitivity score and a 0.72 specificity score with ≤5 channels.

DISCUSSION

These scores suggest that fNIRS is a viable technology for conveniently detecting and monitoring AD as well as investigating underlying mechanisms of disease progression.

EEG/fNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning

There is high demand for techniques to estimate human mental workload during some activities for productivity enhancement or accident prevention. Most studies focus on a single physiological sensing modality and use univariate methods to analyse multi-channel electroencephalography (EEG) data. This paper proposes a new framework that relies on the features of hybrid EEG-functional near-infrared spectroscopy (EEG-fNIRS), supported by machine-learning features to deal with multi-level mental workload classification. Furthermore, instead of the well-used univariate power spectral density (PSD) for EEG recording, we propose using bivariate functional brain connectivity (FBC) features in the time and frequency domains of three bands: delta (0.5-4 Hz), theta (4-7 Hz) and alpha (8-15 Hz). With the assistance of the fNIRS oxyhemoglobin and deoxyhemoglobin (HbO and HbR) indicators, the FBC technique significantly improved classification performance at a 77% accuracy for 0-back vs. 2-back and 83% for 0-back vs. 3-back using a public dataset. Moreover, topographic and heat-map visualisation indicated that the distinguishing regions for EEG and fNIRS showed a difference among the 0-back, 2-back and 3-back test results. It was determined that the best region to assist the discrimination of the mental workload for EEG and fNIRS is different. Specifically, the posterior area performed the best for the posterior midline occipital (POz) EEG in the alpha band and fNIRS had superiority in the right frontal region (AF8).

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Bimodal EEG-fNIRS in Neuroergonomics. Current Evidence and Prospects for Future Research

Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.

Information assistance systems as preventive mediators between increasing customization and mental workload

BACKGROUND: The future of work in Germany is shaped by megatrends like globalization, automatization, digitization, and the demographic change. Furthermore, mass customization and the increasing usage of AI even in manual assembly offers new opportunities as well as it creates new challenges. OBJECTIVE: The trend towards mass customization in turn leads to increased complexity in production, which results in additional mental workload. This effect will continue in the foreseeable future. METHOD: Especially for small and medium sized companies, the backbone of Germany’s economy, automatization and Human-Robot-Collaboration will take time to develop. Information assistance systems are and will be a bridging technology to help organizations to manage increasing complexity and the mental workload of their employees to not only boost productivity but also keep their workforce healthy. The ongoing demographic change further underlines the need to use information assistance systems to compensate possible age-associated deficits, but also keep older employees committed to their work and avoid effects of disengagement or disenfranchisement through participatory ergonomics. RESULTS: Information assistance systems can only develop their inherent potential if they are designed to support employees of varying age, competence levels, and affinity for technology. Participatory development and early engagement are key factors for an increased acceptance and usage of the systems as well as the individualization to make it suitable for each individual employee. CONCLUSION: Expanding the functionalities to an adaptive assistance system, using physiological correlates of mental workload as an input, is conceivable in the future.

Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS-EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS-EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS-EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS-EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS-EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS-EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS-EEG data analyses in future research.

2020 International brain-computer interface competition: A review

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

“We Will Let You Know”: An Assessment of Digital vs. Face-to-Face Job Interviews via EEG Connectivity Analysis

We focused on job interviews as critical examples of complex social interaction in organizational contexts. We aimed at investigating the effect of face-to-face vs. computer-mediated interaction, of role (candidate, recruiter), and of the interview phase (introductory, attitudinal, technical, conclusive) on intra-brain and inter-brain connectivity measures and autonomic synchronization. Twenty expert recruiters and potential candidates took part in a hyperscanning investigation. Namely, electroencephalography (delta, theta, alpha, beta bands) and autonomic (skin-conductance, heart-rate) data were collected in candidate-recruiter dyads during a simulated job interview and then concurrently analyzed. Analyses highlighted a link between face-to-face condition and greater intra-/inter-brain connectivity indices in delta and theta bands. Furthermore, intra-brain and inter-brain connectivity measures were higher for delta and theta bands in the final interview phases compared to the first ones. Consistently, autonomic synchronization was higher during the final interview phases, specifically in the face-to-face condition. Finally, recruiters showed higher intra-brain connectivity in the delta range over frontal and temporoparietal areas, while candidates showed higher intra-brain connectivity in the theta range over frontal areas. Findings highlight the value of hyperscanning investigations in exploring social attunement in professional contexts and hint at their potential to foster neuroscience-informed practices in human resource management processes.

Scale invariance in fNIRS as a measurement of cognitive load

Scale invariant neural dynamics are a relatively new but effective means of measuring changes in brain states as a result of varied cognitive load and task difficulty. This study tests whether scale invariance (as measured by the Hurst exponent, H) can be used with functional near-infrared spectroscopy (fNIRS) to quantify cognitive load, paving the way for scale-invariance to be measured in a variety of real-world settings. We analyzed H extracted from the fNIRS time series while participants completed an N-back working memory task. Consistent with what has been demonstrated in fMRI, the current results showed that scale-invariance analysis significantly differentiated between task and rest periods as calculated from both oxy- (HbO) and deoxy-hemoglobin (HbR) concentration changes. Results from both channel-averaged H and a multivariate partial least squares approach (Task PLS) demonstrated higher H during the 1-back task than the 2-back task. These results were stronger for H derived from HbR than from HbO. This suggests that scale-free brain states are a robust signature of cognitive load and not limited by the specific neuroimaging modality employed. Further, as fNIRS is relatively portable and robust to motion-related artifacts, these preliminary results shed light on the promising future of measuring cognitive load in real life settings.

The Effect of Think Aloud on Performance and Brain Oxygenation During Cycling – An Exploratory Study

In this study, we aimed to investigate the effect of Think Aloud (TA) on performance in trained and untrained participants, using functional Near Infrared Spectroscopy, during incrementally paced cycling. A mixed design was implemented with cycling expertise (10 untrained vs. 9 trained) as the between groups variable and trial stage (5 stages of increasing effort), and condition (silent vs. TA) as within groups independent variables (IVs). Dependent measures were changes in cortical oxygenation (O₂Hb) in 12 areas of the prefrontal cortex (PFC) and physiological indicators of percentage heart rate maximum (%HRmax), average power output (APO), peak power output (PPO), rate of perceived exertion (RPE) and blood lactate ([La]b) over time. Trained cyclists had higher APO and significantly higher PPO from stages 2–5, in addition to a greater increase in PPO over the duration of the test (range 168W–480 W vs. 133W–313 W). There were significant main effects of stage on %HRmax, Bla and RPE (p < .001), with effect sizes (ήp²) ranging from .31 to .97. On average, HRmax%, [La]b and RPE were significantly lower after stage 2 onwards within the TA trial than the silent trial, even though similar power outputs were obtained. Thus, the TA trial elicited a better pacing strategy. There was no main effect of group on changes in O₂Hb, though O₂Hb did change as a function of stage in four areas of the PFC, and as a function of condition in one area. In this first study to assess the effects of TA on performance during self-paced cycling, TA did not disrupt performance outcomes at low through to high levels of physical exertion for either untrained or trained participants.

"We will be in touch". A neuroscientific assessment of remote vs. face-to-face job interviews via EEG hyperscanning

In the last decades, improving remote communications in companies has been a compelling issue. With the outspread of SARS-CoV-2 pandemic, this phenomenon has undergone an acceleration. Despite this, little to no research, considering neurocognitive and emotional systems, was conducted on job interview, a critical organizational phase which significantly contributes to a company long-term success.In this study, we aimed at exploring the emotional and cognitive processes related to different phases of a job interview (introductory, attitudinal, technical and conclusion), when considering two conditions: face-to-face and remote, by simultaneously gathering EEG (frequency bands: alpha, beta, delta, and theta) and autonomic data (skin-conductance-level, SCL, skin-conductance-response, SCR, and heart rate, HR) in both candidates and recruiters. Data highlighted a generalized alpha desynchronization during the job interview interaction. Recruiters showed increased frontal theta activity, which is connected to socio-emotional situations and emotional processing. In addition, results showed how face-to-face condition is related to increased SCL and theta power in the central-brain area, associated with learning processes, via the mid-brain dopamine system and the anterior cingulate cortex. Furthermore, we found higher HR in the candidates. Present results call to re-examine the impact of information-technology in the organization, opening to translational opportunities.

Training Monitoring in Sports: It Is Time to Embrace Cognitive Demand

Appropriate training burden monitoring is still a challenge for the support staff, athletes, and coaches. Extensive research has been done in recent years that proposes several external and internal indicators. Among all measurements, the importance of cognitive factors has been indicated but has never been really considered in the training monitoring process. While there is strong evidence supporting the use of cognitive demand indicators in cognitive neuroscience, their importance in training monitoring for multiple sports settings must be better emphasized. The aims of this scoping review are to (1) provide an overview of the cognitive demand concept beside the physical demand in training; (2) highlight the current methods for assessing cognitive demand in an applied setting to sports in part through a neuroergonomics approach; (3) show how cognitive demand metrics can be exploited and applied to our better understanding of fatigue, sport injury, overtraining and individual performance capabilities. This review highlights also the potential new ways of brain imaging approaches for monitoring in situ. While assessment of cognitive demand is still in its infancy in sport, it may represent a very fruitful approach if applied with rigorous protocols and deep knowledge of both the neurobehavioral and cognitive aspects. It is time now to consider the cognitive demand to avoid underestimating the total training burden and its management.

Assessment of senior drivers' internal state in the event of simulated unexpected vehicle motion based on near-infrared spectroscopy

OBJECTIVE

A driver's internal state is a critical factor influencing driving performance, especially in cases of surprise or shock in response to unexpected incidents while driving. This study was designed to investigate the brain activity of a senior driver in response to simulated unexpected vehicle motion, compared with a relaxed state and normal driving.

METHODS

To accomplish this, we created a driving simulator paradigm wherein participants were involved in one of the following three scenarios: sitting down and relaxing, normal driving around the city with traffic signals and other vehicles, and the exiting of a parking area. In the scenario where the driver was to exit the parking area, the gear was switched automatically by the CarMaker software without the driver being notified, leading to an unexpected condition. The driver's internal states were classified by artificial intelligence, based on information obtained through four-channel near-infrared spectroscopy.

RESULTS

Significant differences were observed between the hemodynamic responses obtained in the three conditions. Ultimately, this method can be used to update advanced driver assistance systems, with a view to preventing future traffic accidents, by activating in-vehicle safety functions based on the driver's condition.

CONCLUSIONS

A driver's internal states in a panic situation while driving can be detected using near-infrared spectroscopy and artificial intelligence.

Optimized electroencephalogram and functional near-infrared spectroscopy-based mental workload detection method for practical applications

Background

Mental workload is a critical consideration in complex man–machine systems design. Among various mental workload detection techniques, multimodal detection techniques integrating electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals have attracted considerable attention. However, existing EEG–fNIRS-based mental workload detection methods have certain defects, such as complex signal acquisition channels and low detection accuracy, which restrict their practical application.

Methods

The signal acquisition configuration was optimized by analyzing the feature importance in mental workload recognition model and a more accurate and convenient EEG–fNIRS-based mental workload detection method was constructed. A classical Multi-Task Attribute Battery (MATB) task was conducted with 20 participating volunteers. Subjective scale data, 64-channel EEG data, and two-channel fNIRS data were collected.

Results

A higher number of EEG channels correspond to higher detection accuracy. However, there is no obvious improvement in accuracy once the number of EEG channels reaches 26, with a four-level mental workload detection accuracy of 76.25 ± 5.21%. Partial results of physiological analysis verify the results of previous studies, such as that the θ power of EEG and concentration of O₂Hb in the prefrontal region increase while the concentration of HHb decreases with task difficulty. It was further observed, for the first time, that the energy of each band of EEG signals was significantly different in the occipital lobe region, and the power of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta_{1}$$\end{document}β1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta_{2}$$\end{document}β2 bands in the occipital region increased significantly with task difficulty. The changing range and the mean amplitude of O₂Hb in high-difficulty tasks were significantly higher compared with those in low-difficulty tasks.

Conclusions

The channel configuration of EEG–fNIRS-based mental workload detection was optimized to 26 EEG channels and two frontal fNIRS channels. A four-level mental workload detection accuracy of 76.25 ± 5.21% was obtained, which is higher than previously reported results. The proposed configuration can promote the application of mental workload detection technology in military, driving, and other complex human–computer interaction systems.

Supervised Classification of Operator Functional State Based on Physiological Data: Application to Drones Swarm Piloting

To improve the safety and the performance of operators involved in risky and demanding missions (like drone operators), human-machine cooperation should be dynamically adapted, in terms of dialogue or function allocation. To support this reconfigurable cooperation, a crucial point is to assess online the operator’s ability to keep performing the mission. The article explores the concept of Operator Functional State (OFS), then it proposes to operationalize this concept (combining context and physiological indicators) on the specific activity of drone swarm monitoring, carried out by 22 participants on simulator SUSIE. With the aid of supervised learning methods (Support Vector Machine, k-Nearest Neighbors, and Random Forest), physiological and contextual are classified into three classes, corresponding to different levels of OFS. This classification would help for adapting the countermeasures to the situation faced by operators.

Inter-Brain Synchrony Levels According to Task Execution Modes and Difficulty Levels: an fNIRS/GSR Study

Hyperscanning is a brain imaging technique that measures brain synchrony caused by social interactions. Recent research on hyperscanning has revealed substantial inter-brain synchrony (IBS), but little is known about the link between IBS and mental workload. To study this link, we conducted an experiment consisting of button-pressing tasks of three different difficulty levels for the cooperation and competition modes with 56 participants aged 23.7±3.8 years (mean±standard deviation). We attempted to observe IBS using functional near-infrared spectroscopy (fNIRS) and galvanic skin response (GSR) to assess the activities of the human autonomic nervous system. We found that the IBS levels increased in a frequency band of 0.075-0.15 Hz, which was unrelated to the task repetition frequency in the cooperation mode according to the task difficulty level. Significant relative inter-brain synchrony (RIBS) increases were observed in three and 10 channels out of 15 for the hard tasks compared to the normal and easy tasks, respectively. We observed that the average GSR values increased with increasing task difficulty levels for the competition mode only. Thus, our results suggest that the IBS revealed by fNIRS and GSR is not related to the hemodynamic changes induced by mental workload, simple behavioral synchrony such as button-pressing timing, or autonomic nervous system activity. IBS is thus explicitly caused by social interactions such as cooperation.

Detecting Fear of Heights Response to a Virtual Reality Environment Using Functional Near-Infrared Spectroscopy

To enable virtual reality exposure therapy (VRET) that treats anxiety disorders by gradually exposing the patient to fear using virtual reality (VR), it is important to monitor the patient's fear levels during the exposure. Despite the evidence of a fear circuit in the brain as reflected by functional near-infrared spectroscopy (fNIRS), the measurement of fear response in highly immersive VR using fNIRS is limited, especially in combination with a head-mounted display (HMD). In particular, it is unclear to what extent fNIRS can differentiate users with and without anxiety disorders and detect fear response in a highly ecological setting using an HMD. In this study, we investigated fNIRS signals captured from participants with and without a fear of height response. To examine the extent to which fNIRS signals of both groups differ, we conducted an experiment during which participants with moderate fear of heights and participants without it were exposed to VR scenarios involving heights and no heights. The between-group statistical analysis shows that the fNIRS data of the control group and the experimental group are significantly different only in the channel located close to right frontotemporal lobe, where the grand average oxygenated hemoglobin Δ[HbO] contrast signal of the experimental group exceeds that of the control group. The within-group statistical analysis shows significant differences between the grand average Δ[HbO] contrast values during fear responses and those during no-fear responses, where the Δ[HbO] contrast values of the fear responses were significantly higher than those of the no-fear responses in the channels located towards the frontal part of the prefrontal cortex. Also, the channel located close to frontocentral lobe was found to show significant difference for the grand average deoxygenated hemoglobin contrast signals. Support vector machine-based classifier could detect fear responses at an accuracy up to 70% and 74% in subject-dependent and subject-independent classifications, respectively. The results demonstrate that cortical hemodynamic responses of a control group and an experimental group are different to a considerable extent, exhibiting the feasibility and ecological validity of the combination of VR-HMD and fNIRS to elicit and detect fear responses. This research thus paves a way toward the a brain-computer interface to effectively manipulate and control VRET.

Is There a Difference in Brain Functional Connectivity between Chinese Coal Mine Workers Who Have Engaged in Unsafe Behavior and Those Who Have Not?

(1) Background: As a world-recognized high-risk occupation, coal mine workers need various cognitive functions to process the surrounding information to cope with a large number of perceived hazards or risks. Therefore, it is necessary to explore the connection between coal mine workers’ neural activity and unsafe behavior from the perspective of cognitive neuroscience. This study explored the functional brain connectivity of coal mine workers who have engaged in unsafe behaviors (EUB) and those who have not (NUB). (2) Methods: Based on functional near-infrared spectroscopy (fNIRS), a total of 106 workers from the Hongliulin coal mine of Shaanxi North Mining Group, one of the largest modern coal mines in China, completed the test. Pearson’s Correlation Coefficient (COR) analysis, brain network analysis, and two-sample t-test were used to investigate the difference in brain functional connectivity between the two groups. (3) Results: The results showed that there were significant differences in functional brain connectivity between EUB and NUB among the frontopolar area (p = 0.002325), orbitofrontal area (p = 0.02102), and pars triangularis Broca’s area (p = 0.02888). Small-world properties existed in the brain networks of both groups, and the dorsolateral prefrontal cortex had significant differences in clustering coefficient (p = 0.0004), nodal efficiency (p = 0.0384), and nodal local efficiency (p = 0.0004). (4) Conclusions: This study is the first application of fNIRS to the field of coal mine safety. The fNIRS brain functional connectivity analysis is a feasible method to investigate the neuropsychological mechanism of unsafe behavior in coal mine workers in the view of brain science.