1
|
Pütz S, Mertens A, Chuang L, Nitsch V. Physiological measures of operators' mental state in supervisory process control tasks: a scoping review. ERGONOMICS 2024; 67:801-830. [PMID: 38031407 DOI: 10.1080/00140139.2023.2289858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 11/27/2023] [Indexed: 12/01/2023]
Abstract
Physiological measures are often used to assess the mental state of human operators in supervisory process control tasks. However, the diversity of research approaches creates a heterogeneous landscape of empirical evidence. To map existing evidence and provide guidance to researchers and practitioners, this paper systematically reviews 109 empirical studies that report relationships between peripheral nervous system measures and mental state dimensions (e.g. mental workload, mental fatigue, stress, and vigilance) of interest. Ocular and electrocardiac measures were the most prominent measures across application fields. Most studies sought to validate such measures for reliable assessments of cognitive task demands and time on task, with measures of pupil size receiving the most empirical support. In comparison, less research examined the utility of physiological measures in predicting human task performance. This approach is discussed as an opportunity to focus on operators' individual response to cognitive task demands and to advance the state of research.
Collapse
Affiliation(s)
- Sebastian Pütz
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
| | - Alexander Mertens
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
| | - Lewis Chuang
- Professorship for Humans and Technology, Chemnitz University of Technology, Chemnitz, Germany
| | - Verena Nitsch
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
- Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE, Aachen, Germany
| |
Collapse
|
2
|
Jin L, Ren Q, Mitchell V, May A. A video processing and machine learning based method for evaluating safety-critical operator engagement in a motorway control room. ERGONOMICS 2024; 67:356-376. [PMID: 37306555 DOI: 10.1080/00140139.2023.2223784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 06/04/2023] [Indexed: 06/13/2023]
Abstract
In safety-critical automatic systems, safety can be compromised if operators lack engagement. Effective detection of undesirable engagement states can inform the design of interventions for enhancing engagement. However, the existing engagement measurement methods suffer from several limitations which damage their effectiveness in the work environment. A novel engagement evaluation methodology, which adopts Artificial Intelligence (AI) technologies, has been proposed. It was developed using motorway control room operators as subjects. Openpose and Open Source Computer Vision Library (OpenCV) were used to estimate the body postures of operators, then a Support Vector Machine (SVM) was utilised to build the engagement evaluation model based on discrete states of operator engagement. The average accuracy of the evaluation results reached 0.89 and the weighted average precision, recall, and F1-score were all above 0.84. This study emphasises the importance of specific data labelling when measuring typical engagement states, forming the basis for potential control room improvements.Practitioner summary: This study demonstrates an automatic, real-time, objective, and relatively unobtrusive method for measuring dynamic operator engagement states. Computer vision technologies were used to estimate body posture, then machine learning (ML) was utilised to build the engagement evaluation model. The overall evaluation shows the effectiveness of this framework.Abbreviations: AI: Artificial Intelligence; OpenCV: Open Source Computer Vision Library; SVM: Support Vector Machine; UWES: Utrecht Work Engagement Scale; ISA Engagement Scale: Intellectual, Social, Affective Engagement Scale; DSSQ: Dundee Stress State Questionnaire; SSSQ: Short Stress State Questionnaire; EEG: electroencephalography; ECG: Electrocardiography; VMOE: Video-based Measurement for Operator Engagement; CMU: Carnegie Mellon University; CNN: Convolutional Neural Network; 2D: two dimensional; ML: Machine learning.
Collapse
Affiliation(s)
- Linyi Jin
- School of Design and Creative Arts, Loughborough University, Loughborough, UK
| | - Qingyu Ren
- Department of Industrial Design, School of Design, Southwest Jiaotong University, Chengdu, China
| | - Val Mitchell
- School of Design and Creative Arts, Loughborough University, Loughborough, UK
| | - Andrew May
- School of Design and Creative Arts, Loughborough University, Loughborough, UK
| |
Collapse
|
3
|
Liu C, Zhang C, Sun L, Liu K, Liu H, Zhu W, Jiang C. Detection of Pilot's Mental Workload Using a Wireless EEG Headset in Airfield Traffic Pattern Tasks. ENTROPY (BASEL, SWITZERLAND) 2023; 25:1035. [PMID: 37509982 PMCID: PMC10378707 DOI: 10.3390/e25071035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 06/25/2023] [Accepted: 07/05/2023] [Indexed: 07/30/2023]
Abstract
Elevated mental workload (MWL) experienced by pilots can result in increased reaction times or incorrect actions, potentially compromising flight safety. This study aims to develop a functional system to assist administrators in identifying and detecting pilots' real-time MWL and evaluate its effectiveness using designed airfield traffic pattern tasks within a realistic flight simulator. The perceived MWL in various situations was assessed and labeled using NASA Task Load Index (NASA-TLX) scores. Physiological features were then extracted using a fast Fourier transformation with 2-s sliding time windows. Feature selection was conducted by comparing the results of the Kruskal-Wallis (K-W) test and Sequential Forward Floating Selection (SFFS). The results proved that the optimal input was all PSD features. Moreover, the study analyzed the effects of electroencephalography (EEG) features from distinct brain regions and PSD changes across different MWL levels to further assess the proposed system's performance. A 10-fold cross-validation was performed on six classifiers, and the optimal accuracy of 87.57% was attained using a multi-class K-Nearest Neighbor (KNN) classifier for classifying different MWL levels. The findings indicate that the wireless headset-based system is reliable and feasible. Consequently, numerous wireless EEG device-based systems can be developed for application in diverse real-driving scenarios. Additionally, the current system contributes to future research on actual flight conditions.
Collapse
Affiliation(s)
- Chenglin Liu
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| | - Chenyang Zhang
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| | - Luohao Sun
- School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, China
| | - Kun Liu
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| | - Haiyue Liu
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| | - Wenbing Zhu
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| | - Chaozhe Jiang
- School of Transportation & Logistics, Southwest Jiaotong University, Chengdu 611756, China
| |
Collapse
|
4
|
Recognising situation awareness associated with different workloads using EEG and eye-tracking features in air traffic control tasks. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
5
|
Luong T, Lecuyer A, Martin N, Argelaguet F. A Survey on Affective and Cognitive VR. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2022; 28:5154-5171. [PMID: 34495833 DOI: 10.1109/tvcg.2021.3110459] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
In Virtual Reality (VR), users can be immersed in emotionally intense and cognitively engaging experiences. Yet, despite strong interest from scholars and a large amount of work associating VR and Affective and Cognitive States (ACS), there is a clear lack of structured and systematic form in which this research can be classified. We define "Affective and Cognitive VR" to relate to works which (1) induce ACS, (2) recognize ACS, or (3) exploit ACS by adapting virtual environments based on ACS measures. This survey clarifies the different models of ACS, presents the methods for measuring them with their respective advantages and drawbacks in VR, and showcases Affective and Cognitive VR studies done in an Immersive Virtual Environment (IVE) in a non-clinical context. Our article covers the main research lines in Affective and Cognitive VR. We provide a comprehensive list of references with the analysis of 63 research articles and summarize future works directions.
Collapse
|
6
|
Devlin SP, Brown NL, Drollinger S, Sibley C, Alami J, Riggs SL. Scan-based eye tracking measures are predictive of workload transition performance. APPLIED ERGONOMICS 2022; 105:103829. [PMID: 35930898 DOI: 10.1016/j.apergo.2022.103829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Revised: 06/06/2022] [Accepted: 06/08/2022] [Indexed: 06/15/2023]
Abstract
Given there is no unifying theory or design guidance for workload transitions, this work investigated how visual attention allocation patterns could inform both topics, by understanding if scan-based eye tracking metrics could predict workload transition performance trends in a context-relevant domain. The eye movements of sixty Naval flight students were tracked as workload transitioned at a slow, medium, and fast pace in an unmanned aerial vehicle testbed. Four scan-based metrics were significant predictors across the different growth curve models of response time and accuracy. Stationary gaze entropy (a measure of how dispersed visual attention transitions are across tasks) was predictive across all three transition rates. The other three predictive scan-based metrics captured different aspects of visual attention, including its spread, directness, and duration. The findings specify several missing details in both theory and design guidance, which is unprecedented, and serves as a basis of future workload transition research.
Collapse
Affiliation(s)
- Shannon P Devlin
- U.S. Naval Research Laboratory, Washington, D.C, USA; University of Virginia, Charlottesville, VA, USA.
| | | | | | - Ciara Sibley
- U.S. Naval Research Laboratory, Washington, D.C, USA
| | - Jawad Alami
- University of Virginia, Charlottesville, VA, USA
| | - Sara L Riggs
- University of Virginia, Charlottesville, VA, USA
| |
Collapse
|
7
|
Wang G, Yin Z, Zhao M, Tian Y, Sun Z. Identification of human mental workload levels in a language comprehension task with imbalance neurophysiological data. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 224:107011. [PMID: 35863122 DOI: 10.1016/j.cmpb.2022.107011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 05/23/2022] [Accepted: 07/06/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Operator's capability for accurately comprehending verbal commands is critically important to maintain the performance of human-machine interaction. It can be evaluated by human mental workload measured with electroencephalography (EEG). However, the time duration of different workload conditions within a task session is unequal due to varied psychophysiological processes across individuals. It leads to data imbalance of the EEG for training workload classifiers. METHODS In this study, we propose an EEG feature oversampling technique, Gaussian-SMOTE based feature ensemble (GSMOTE-FE), for workload recognition with imbalanced classes. First, artificial EEG instances are drawn from a Gaussian distribution in the margin between the minority and majority workload classes. Tomek links are detected as clues to remove redundant feature vectors. Then, we embed a feature selection module based on the GINI importance while an ensemble classifier committee with bootstrap aggregating is used to further enhance classification performance. RESULTS We validate the GSMOTE-FE framework based on an experiment that simulates operators to understand the correct meaning of the instructions in the Chinese language. Participants' EEG signals and reaction time data were both recorded to validate the proposed workload classifier. Workload classification accuracy and Macro-F1 values are 0.6553 and 0.5862, respectively. Corresponding G-mean and AUC achieve at 0.5757 and 0.5958, respectively. CONCLUSIONS The performance of the GSMOTE-FE is demonstrated to be comparable with the advanced oversampling techniques. The workload classifier has the capability to indicate low and high levels of the task demand of the Chinese language understanding task.
Collapse
Affiliation(s)
- Guangying Wang
- School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, PR China
| | - Zhong Yin
- Engineering Research Center of Optical Instrument and System, Ministry of Education, Shanghai Key Lab of Modern Optical System, University of Shanghai for Science and Technology, Shanghai, 200093, PR China; School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, PR China.
| | - Mengyuan Zhao
- College of Foreign Languages, University of Shanghai for Science and Technology, Shanghai, 200093, PR China
| | - Ying Tian
- School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, PR China
| | - Zhanquan Sun
- School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, PR China
| |
Collapse
|
8
|
Izadi Laybidi M, Rasoulzadeh Y, Dianat I, Samavati M, Asghari Jafarabadi M, Nazari MA. Cognitive performance and electroencephalographic variations in air traffic controllers under various mental workload and time of day. Physiol Behav 2022; 252:113842. [PMID: 35561808 DOI: 10.1016/j.physbeh.2022.113842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Revised: 03/12/2022] [Accepted: 05/09/2022] [Indexed: 11/19/2022]
Abstract
The aim of this study was to investigate the effects of mental workload (MWL) and time of day on cognitive performance and electroencephalographic (EEG) parameters of air traffic controllers. EEG signals recorded while 20 professional air traffic controllers performed cognitive tasks [A-X Continuous Performance Test (AX-CPT) and 3-back working memory task] after they were exposed to two levels of task difficulty (high and low MWL) in the morning and afternoon. Significant decreases in cognitive performance were found when the levels of task difficulty increased in both tasks. The results confirmed the sensitivity of the theta and beta activities to levels of task difficulty in the 3-back task, while they were not affected in the AX-CPT. Theta and beta activities were influenced by time of day in the AX-CPT. The findings provide guidance for application of changes in EEG parameters when MWL level is manipulated during the day that could be implemented in future for the development of real-time monitoring systems to improve aviation safety.
Collapse
Affiliation(s)
- Marzieh Izadi Laybidi
- Department of Occupational Health and Ergonomics, Faculty of Health, Tabriz University of Medical Sciences, Tabriz, Iran; Student Research Committee, Faculty of Health, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Yahya Rasoulzadeh
- Department of Occupational Health and Ergonomics, Faculty of Health, Tabriz University of Medical Sciences, Tabriz, Iran; Road Traffic Injury Research Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Iman Dianat
- Department of Occupational Health and Ergonomics, Faculty of Health, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Mehdi Samavati
- Research Center for Biomedical Technologies & Robotics, Tehran University of Medical Sciences, Tehran, Iran
| | - Mohammad Asghari Jafarabadi
- Center for the Development of Interdisciplinary Research in Islamic Sciences and Health Sciences, Tabriz University of Medical Sciences, Tabriz, Iran; Department of Statistics and Epidemiology, School of Medicine, Zanjan University of Medical Sciences, Zanjan, Iran
| | - Mohammad Ali Nazari
- Department of Neuroscience, Faculty of Advanced Technologies in Medicine, Iran University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
9
|
Pagnotta M, Jacobs DM, de Frutos PL, Rodríguez R, Ibáñez-Gijón J, Travieso D. Task difficulty and physiological measures of mental workload in air traffic control: a scoping review. ERGONOMICS 2022; 65:1095-1118. [PMID: 34904533 DOI: 10.1080/00140139.2021.2016998] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 12/05/2021] [Indexed: 06/14/2023]
Abstract
This study provides a systematic synthesis of empirical research on mental workload (MWL) in air traffic control (ATC). MWL is a key concept in research on innovative technologies, because the assessment of MWL is crucial to the evaluation of such technologies. Our specific focus was on physiological measures of MWL. The used search strategy identified 39 peer-reviewed publications that analysed ATC tasks, examined different levels of difficulty of the ATC task, and considered at least one physiological measure of MWL. Positive relations between measures of MWL and task difficulty were observed most frequently, indicating that the measures indeed allowed the assessment of MWL. The most commonly used physiological measures were brain measures (EEG and fNIR) and heart rate measures. The review revealed a need for more precise descriptions of crucial experimental parameters in order to permit a transition of the field towards more interactive and dynamic types of analysis. Practitioner summary: Research on innovative technology in air traffic control (ATC) depends on assessments of mental workload (MWL). We reviewed empirical research on MWL in ATC. Brain and heart measures often allow assessments of MWL. Better descriptions of experiments are needed to allow comparisons among studies and more dynamic and interactive analyses.
Collapse
Affiliation(s)
- Murillo Pagnotta
- Facultad de Psicología, Universidad Autónoma de Madrid, Madrid, Spain
| | - David M Jacobs
- Facultad de Psicología, Universidad Autónoma de Madrid, Madrid, Spain
| | | | - Ruben Rodríguez
- CRIDA A.I.E, ATM R&D + Innovation Reference Centre, Madrid, Spain
| | | | - David Travieso
- Facultad de Psicología, Universidad Autónoma de Madrid, Madrid, Spain
| |
Collapse
|
10
|
Pütz S, Rick V, Mertens A, Nitsch V. Using IoT devices for sensor-based monitoring of employees' mental workload: Investigating managers' expectations and concerns. APPLIED ERGONOMICS 2022; 102:103739. [PMID: 35279467 DOI: 10.1016/j.apergo.2022.103739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 02/24/2022] [Accepted: 03/01/2022] [Indexed: 06/14/2023]
Abstract
Although the objective assessment of mental workload has been a focus of human factors research, few studies have investigated stakeholders' attitudes towards its implementation in real workplaces. The present study addresses this research gap by surveying N = 702 managers in three European countries (Germany, United Kingdom, Spain) about their expectations and concerns regarding sensor-based monitoring of employee mental workload. The data confirm the relevance of expectations regarding improvements of workplace design and employee well-being, as well as concerns about restrictions of employees' privacy and sovereignty, for the implementation of workload monitoring. Furthermore, Bayesian regression models show that the examined expectations have a substantial positive association with managers' willingness to support workload monitoring in their company. Privacy concerns are identified as a significant barrier to the acceptance of workload monitoring, both in terms of their prevalence among managers and their strong negative relationship with monitoring support.
Collapse
Affiliation(s)
- Sebastian Pütz
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Eilfschornsteinstr. 18, 52062, Aachen, Germany.
| | - Vera Rick
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Eilfschornsteinstr. 18, 52062, Aachen, Germany
| | - Alexander Mertens
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Eilfschornsteinstr. 18, 52062, Aachen, Germany
| | - Verena Nitsch
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Eilfschornsteinstr. 18, 52062, Aachen, Germany; Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE, Campus-Boulevard 55-57, 52074, Aachen, Germany
| |
Collapse
|
11
|
Yu Y, Li J. Feature Fusion-Based Capsule Network for Cross-Subject Mental Workload Classification. Brain Inform 2022. [DOI: 10.1007/978-3-031-15037-1_14] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2022] Open
|
12
|
Vukovic M, Cavedon L, Thangarajah J, Rodriguez S. Performance degrades less under increased workload with the addition of speech control in a dynamic environment. APPLIED ERGONOMICS 2021; 96:103486. [PMID: 34139375 DOI: 10.1016/j.apergo.2021.103486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/18/2020] [Revised: 05/11/2021] [Accepted: 05/26/2021] [Indexed: 06/12/2023]
Abstract
This research empirically evaluates the introduction of speech to existing keyboard and mouse input modalities in an application used to control aircraft in a simulated, complex and dynamic environment. Task performance and task performance degradation are assessed for three levels of workload. Previous studies have evaluated task performance using these modalities however, only a couple have evaluated task performance under varying workload. Even though speech is a common addition to modern control interfaces, the effect of varying workload on this combination of control modalities has not yet been reported. Thirty-six participants commanded simulated aircraft through generated obstacle courses to reach a Combat Air Patrol (CAP) point while also responding to a secondary task. There were nine conditions that varied the control modality (Keyboard and Mouse (KM), Voice (V), and Keyboard, Mouse and Voice (KMV)), and workload by varying the number of aircraft being controlled (low, medium and high). Results showed that KM outperformed KMV and V for the low and medium workload levels. However, task performance with KMV was found to degrade the least as workload increased. KMV and KM were found to enable significantly more correct responses to the secondary task which was delivered aurally. Participants reported a preference for the combined modalities (KMV), self-assessing that KMV most reduced their workload. This research suggests that the addition of a speech interface to existing keyboard and mouse modalities, for control of aircraft in a simulation, may help manage cognitive load and may assist in controlling more aircraft under higher workloads.
Collapse
Affiliation(s)
- Maria Vukovic
- RMIT University, PO Box 2476, Melbourne, VIC, 3001, Australia; Defence Science and Technology Group, 506 Lorimer Street, Fishermans Bend, Melbourne, VIC, 3207, Australia.
| | | | | | | |
Collapse
|
13
|
Alyan E, Saad NM, Kamel N, Rahman MA. Workplace design-related stress effects on prefrontal cortex connectivity and neurovascular coupling. APPLIED ERGONOMICS 2021; 96:103497. [PMID: 34139374 DOI: 10.1016/j.apergo.2021.103497] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/17/2020] [Revised: 04/28/2021] [Accepted: 06/07/2021] [Indexed: 06/12/2023]
Abstract
This study aims to evaluate the effect of workstation type on the neural and vascular networks of the prefrontal cortex (PFC) underlying the cognitive activity involved during mental stress. Workstation design has been reported to affect the physical and mental health of employees. However, while the functional effects of ergonomic workstations have been documented, there is little research on the influence of workstation design on the executive function of the brain. In this study, 23 healthy volunteers in ergonomic and non-ergonomic workstations completed the Montreal imaging stress task, while their brain activity was recorded using the synchronized measurement of electroencephalography and functional near-infrared spectroscopy. The results revealed desynchronization in alpha rhythms and oxygenated hemoglobin, as well as decreased functional connectivity in the PFC networks at the non-ergonomic workstations. Additionally, a significant increase in salivary alpha-amylase activity was observed in all participants at the non-ergonomic workstations, confirming the presence of induced stress. These findings suggest that workstation design can significantly impact cognitive functioning and human capabilities at work. Therefore, the use of functional neuroimaging in workplace design can provide critical information on the causes of workplace-related stress.
Collapse
Affiliation(s)
- Emad Alyan
- Centre for Intelligent Signal and Imaging Research (CISIR), Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia.
| | - Naufal M Saad
- Centre for Intelligent Signal and Imaging Research (CISIR), Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
| | - Nidal Kamel
- Centre for Intelligent Signal and Imaging Research (CISIR), Department of Electrical and Electronic Engineering, Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
| | - Mohammad Abdul Rahman
- Faculty of Medicine, Universiti Kuala Lumpur Royal College of Medicine Perak, 30450, Perak, Malaysia
| |
Collapse
|
14
|
Leeuwis N, Pistone D, Flick N, van Bommel T. A Sound Prediction: EEG-Based Neural Synchrony Predicts Online Music Streams. Front Psychol 2021; 12:672980. [PMID: 34385953 PMCID: PMC8354316 DOI: 10.3389/fpsyg.2021.672980] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Accepted: 06/08/2021] [Indexed: 11/13/2022] Open
Abstract
Neuroforecasting predicts population-wide choices based on neural data of individuals and can be used, for example, in neuromarketing to estimate campaign successes. To deliver true value, the brain activity metrics should deliver predictive value above and beyond traditional stated preferences. Evidence from movie trailer research has proposed neural synchrony, which compares the similarity of brain responses across participants and has shown to be a promising tool in neuroforecasting for movie popularity. The music industry might also benefit from these increasingly accurate success predictors, but only one study has been forecasting music popularity, using functional magnetic resonance imaging measures. Current research validates the strength of neural synchrony as a predictive measure for popularity of music, making use of electroencephalogram to capture moment-to-moment neural similarity between respondents while they listen to music. Neural synchrony is demonstrated to be a significant predictor for public appreciation on Spotify 3 weeks and 10 months after the release of the albums, especially when combined with the release of a single. On an individual level, other brain measures were shown to relate to individual subjective likeability ratings, including Frontal Alpha Asymmetry and engagement when combined with the factors artist and single release. Our results show the predictive value of brain activity measures outperforms stated preferences. Especially, neural synchrony carries high predictive value for the popularity on Spotify, providing the music industry with an essential asset for efficient decision making and investments, in addition to other practical implications that include neuromarketing and advertising industries.
Collapse
Affiliation(s)
- Nikki Leeuwis
- Unravel Research, Utrecht, Netherlands
- Tilburg University, Tilburg, Netherlands
| | - Daniela Pistone
- Applied Cognitive Psychology, Utrecht University, Utrecht, Netherlands
| | | | | |
Collapse
|
15
|
Czerniak JN, Schierhorst N, Villani V, Sabattini L, Brandl C, Mertens A, Schwalm M, Nitsch V. The Index of Cognitive Activity - Eligibility for task-evoked informational strain and robustness towards visual influences. APPLIED ERGONOMICS 2021; 92:103342. [PMID: 33373789 DOI: 10.1016/j.apergo.2020.103342] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2020] [Revised: 11/24/2020] [Accepted: 12/10/2020] [Indexed: 06/12/2023]
Abstract
Various researchers have proposed pupillometric indicators to assess a person's cognitive strain. However, to distinguish the variation of pupil light response from psychosensory pupil response in experimental field conditions is a challenge. The Index of Cognitive Activity (ICA) addresses this problem by wavelet separation. This research investigates the ICA's sensitivity for multiple level task-evoked cognitive activity and visual influences concerning informational work tasks. Objective and subjective measures assessed cognitive strain of participants (N = 22) during various tasks. In a first experiment, mental arithmetic tasks were used to induce different levels of cognitive activity. In a second experiment, influences of screen polarity and presentation of information were investigated (N = 18). The results indicate that eye metrics are rarely sensitive to slight variations in task difficulty. Moreover, the ICA is likely to be sensitive towards constant screen illumination and shows tendencies regarding changes in displayed information. Possible ramifications for the objective assessment of cognitive strain are discussed.
Collapse
Affiliation(s)
- Julia N Czerniak
- Institute of Industrial Engineering and Ergonomics (IAW), RWTH Aachen University, Aachen, Germany.
| | - Nikolas Schierhorst
- Institute of Industrial Engineering and Ergonomics (IAW), RWTH Aachen University, Aachen, Germany.
| | - Valeria Villani
- Department of Sciences and Methods for Engineering (DISMI), University of Modena and Reggio Emilia, Reggio Emilia, Italy.
| | - Lorenzo Sabattini
- Department of Sciences and Methods for Engineering (DISMI), University of Modena and Reggio Emilia, Reggio Emilia, Italy.
| | - Christopher Brandl
- Institute of Industrial Engineering and Ergonomics (IAW), RWTH Aachen University, Aachen, Germany.
| | - Alexander Mertens
- Institute of Industrial Engineering and Ergonomics (IAW), RWTH Aachen University, Aachen, Germany.
| | - Maximilian Schwalm
- Institute of Highway Engineering (ISAC), RWTH Aachen University, Aachen, Germany.
| | - Verena Nitsch
- Institute of Industrial Engineering and Ergonomics (IAW), RWTH Aachen University, Aachen, Germany.
| |
Collapse
|
16
|
Belkhiria C, Peysakhovich V. Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020). FRONTIERS IN NEUROERGONOMICS 2020; 1:606719. [PMID: 38234309 PMCID: PMC10790927 DOI: 10.3389/fnrgo.2020.606719] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 11/17/2020] [Indexed: 01/19/2024]
Abstract
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
Collapse
|
17
|
Ruthig JC, Poltavski DP, Petros T. Examining Positivity Effect and Working Memory in Young-Old and Very Old Adults Using EEG-Derived Cognitive State Metrics. Res Aging 2019; 41:1014-1035. [PMID: 31409210 DOI: 10.1177/0164027519865310] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The positivity effect among older adults is a tendency to process more positive and/or less negative emotional stimuli compared to younger adults, with unknown upper age boundaries. Cognitive and emotional working memory were assessed in young-old adults (60-75) and very old adults (VOAs; 80+) to determine whether emotional working memory declines similar to the age-related decline of cognitive working memory. The moderating role of valence on the link between age and emotional working memory was examined to identify change in positivity effect with advanced age. Electroencephalography (EEG) markers of cognitive workload and engagement were obtained to test the theory of cognitive resource allocation in older adults' emotional stimuli processing. EEG recordings were collected during cognitive memory task and emotional working memory tasks that required rating emotional intensity of images pairs. Results indicate a positivity effect among VOAs that does not require additional cognitive effort and is not likely to diminish with age.
Collapse
|