1
|
Wang P, Houghton R, Majumdar A. Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:3723. [PMID: 38931507 PMCID: PMC11207491 DOI: 10.3390/s24123723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 05/30/2024] [Accepted: 06/04/2024] [Indexed: 06/28/2024]
Abstract
Measuring pilot mental workload (MWL) is crucial for enhancing aviation safety. However, MWL is a multi-dimensional construct that could be affected by multiple factors. Particularly, in the context of a more automated cockpit setting, the traditional methods of assessing pilot MWL may face challenges. Heart rate variability (HRV) has emerged as a potential tool for detecting pilot MWL during real-flight operations. This review aims to investigate the relationship between HRV and pilot MWL and to assess the performance of machine-learning-based MWL detection systems using HRV parameters. A total of 29 relevant papers were extracted from three databases for review based on rigorous eligibility criteria. We observed significant variability across the reviewed studies, including study designs and measurement methods, as well as machine-learning techniques. Inconsistent results were observed regarding the differences in HRV measures between pilots under varying levels of MWL. Furthermore, for studies that developed HRV-based MWL detection systems, we examined the diverse model settings and discovered that several advanced techniques could be used to address specific challenges. This review serves as a practical guide for researchers and practitioners who are interested in employing HRV indicators for evaluating MWL and wish to incorporate cutting-edge techniques into their MWL measurement approaches.
Collapse
Affiliation(s)
| | | | - Arnab Majumdar
- Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, London SW7 2AZ, UK; (P.W.); (R.H.)
| |
Collapse
|
2
|
Pütz S, Mertens A, Chuang L, Nitsch V. Physiological measures of operators' mental state in supervisory process control tasks: a scoping review. ERGONOMICS 2024; 67:801-830. [PMID: 38031407 DOI: 10.1080/00140139.2023.2289858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 11/27/2023] [Indexed: 12/01/2023]
Abstract
Physiological measures are often used to assess the mental state of human operators in supervisory process control tasks. However, the diversity of research approaches creates a heterogeneous landscape of empirical evidence. To map existing evidence and provide guidance to researchers and practitioners, this paper systematically reviews 109 empirical studies that report relationships between peripheral nervous system measures and mental state dimensions (e.g. mental workload, mental fatigue, stress, and vigilance) of interest. Ocular and electrocardiac measures were the most prominent measures across application fields. Most studies sought to validate such measures for reliable assessments of cognitive task demands and time on task, with measures of pupil size receiving the most empirical support. In comparison, less research examined the utility of physiological measures in predicting human task performance. This approach is discussed as an opportunity to focus on operators' individual response to cognitive task demands and to advance the state of research.
Collapse
Affiliation(s)
- Sebastian Pütz
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
| | - Alexander Mertens
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
| | - Lewis Chuang
- Professorship for Humans and Technology, Chemnitz University of Technology, Chemnitz, Germany
| | - Verena Nitsch
- Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, Germany
- Fraunhofer Institute for Communication, Information Processing and Ergonomics FKIE, Aachen, Germany
| |
Collapse
|
3
|
Ronca V, Martinez-Levy AC, Vozzi A, Giorgi A, Aricò P, Capotorto R, Borghini G, Babiloni F, Di Flumeri G. Wearable Technologies for Electrodermal and Cardiac Activity Measurements: A Comparison between Fitbit Sense, Empatica E4 and Shimmer GSR3. SENSORS (BASEL, SWITZERLAND) 2023; 23:5847. [PMID: 37447697 DOI: 10.3390/s23135847] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2023] [Revised: 06/16/2023] [Accepted: 06/22/2023] [Indexed: 07/15/2023]
Abstract
The capability of measuring specific neurophysiological and autonomic parameters plays a crucial role in the objective evaluation of a human's mental and emotional states. These human aspects are commonly known in the scientific literature to be involved in a wide range of processes, such as stress and arousal. These aspects represent a relevant factor especially in real and operational environments. Neurophysiological autonomic parameters, such as Electrodermal Activity (EDA) and Photoplethysmographic data (PPG), have been usually investigated through research-graded devices, therefore resulting in a high degree of invasiveness, which could negatively interfere with the monitored user's activity. For such a reason, in the last decade, recent consumer-grade wearable devices, usually designed for fitness-tracking purposes, are receiving increasing attention from the scientific community, and are characterized by a higher comfort, ease of use and, therefore, by a higher compatibility with daily-life environments. The present preliminary study was aimed at assessing the reliability of a consumer wearable device, i.e., the Fitbit Sense, with respect to a research-graded wearable, i.e., the Empatica E4 wristband, and a laboratory device, i.e., the Shimmer GSR3+. EDA and PPG data were collected among 12 participants while they performed multiple resting conditions. The results demonstrated that the EDA- and PPG-derived features computed through the wearable and research devices were positively and significantly correlated, while the reliability of the consumer device was significantly lower.
Collapse
Affiliation(s)
- Vincenzo Ronca
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, 00185 Rome, Italy
- BrainSigns Srl, 00198 Rome, Italy
| | - Ana C Martinez-Levy
- BrainSigns Srl, 00198 Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, 00185 Rome, Italy
| | - Alessia Vozzi
- BrainSigns Srl, 00198 Rome, Italy
- Department of Anatomical, Histological, Forensic and Orthopaedic Sciences, Sapienza University of Rome, 00185 Rome, Italy
| | - Andrea Giorgi
- BrainSigns Srl, 00198 Rome, Italy
- Department of Anatomical, Histological, Forensic and Orthopaedic Sciences, Sapienza University of Rome, 00185 Rome, Italy
| | - Pietro Aricò
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, 00185 Rome, Italy
- BrainSigns Srl, 00198 Rome, Italy
| | - Rossella Capotorto
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, 00185 Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, 00185 Rome, Italy
| | - Gianluca Borghini
- BrainSigns Srl, 00198 Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, 00185 Rome, Italy
| | - Fabio Babiloni
- BrainSigns Srl, 00198 Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, 00185 Rome, Italy
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310005, China
| | - Gianluca Di Flumeri
- BrainSigns Srl, 00198 Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, 00185 Rome, Italy
| |
Collapse
|
4
|
Wang JJ, Liu SH, Tsai CH, Manousakas I, Zhu X, Lee TL. Signal Quality Analysis of Single-Arm Electrocardiography. SENSORS (BASEL, SWITZERLAND) 2023; 23:5818. [PMID: 37447668 DOI: 10.3390/s23135818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/28/2023] [Revised: 06/15/2023] [Accepted: 06/21/2023] [Indexed: 07/15/2023]
Abstract
The number of people experiencing mental stress or emotional dysfunction has increased since the onset of the COVID-19 pandemic, as many individuals have had to adapt their daily lives. Numerous studies have demonstrated that mental health disorders can pose a risk for certain diseases, and they are also closely associated with the problem of mental workload. Now, wearable devices and mobile health applications are being utilized to monitor and assess individuals' mental health conditions on a daily basis using heart rate variability (HRV), typically measured by the R-to-R wave interval (RRI) of an electrocardiogram (ECG). However, portable or wearable ECG devices generally require two electrodes to perform bipolar limb leads, such as the Einthoven triangle. This study aims to develop a single-arm ECG measurement method, with lead I ECG serving as the gold standard. We conducted static and dynamic experiments to analyze the morphological performance and signal-to-noise ratio (SNR) of the single-arm ECG. Three morphological features were defined, RRI, the duration of the QRS complex wave, and the amplitude of the R wave. Thirty subjects participated in this study. The results indicated that RRI exhibited the highest cross-correlation (R = 0.9942) between the single-arm ECG and lead I ECG, while the duration of the QRS complex wave showed the weakest cross-correlation (R = 0.2201). The best SNR obtained was 26.1 ± 5.9 dB during the resting experiment, whereas the worst SNR was 12.5 ± 5.1 dB during the raising and lowering of the arm along the z-axis. This single-arm ECG measurement method offers easier operation compared to traditional ECG measurement techniques, making it applicable for HRV measurement and the detection of an irregular RRI.
Collapse
Affiliation(s)
- Jia-Jung Wang
- Department of Biomedical Engineering, I-Shou University, Kaohsiung 84001, Taiwan
| | - Shing-Hong Liu
- Department of Computer Science and Information Engineering, Chaoyang University of Technology, Taichung 413310, Taiwan
| | - Cheng-Hsien Tsai
- Department of Biomedical Engineering, I-Shou University, Kaohsiung 84001, Taiwan
| | - Ioannis Manousakas
- Department of Biomedical Engineering, I-Shou University, Kaohsiung 84001, Taiwan
| | - Xin Zhu
- Division of Information Systems, School of Computer Science and Engineering, The University of Aizu, Aizu-Wakamatsu City 965-8580, Japan
| | - Thung-Lip Lee
- Department of Cardiology, E-Da Hospital, Kaohsiung 84001, Taiwan
| |
Collapse
|
5
|
Campanella S, Altaleb A, Belli A, Pierleoni P, Palma L. A Method for Stress Detection Using Empatica E4 Bracelet and Machine-Learning Techniques. SENSORS (BASEL, SWITZERLAND) 2023; 23:3565. [PMID: 37050625 PMCID: PMC10098696 DOI: 10.3390/s23073565] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 03/18/2023] [Accepted: 03/28/2023] [Indexed: 06/19/2023]
Abstract
In response to challenging circumstances, the human body can experience marked levels of anxiety and distress. To prevent stress-related complications, timely identification of stress symptoms is crucial, necessitating the need for continuous stress monitoring. Wearable devices offer a means of real-time and ongoing data collection, facilitating personalized stress monitoring. Based on our protocol for data pre-processing, this study proposes to analyze signals obtained from the Empatica E4 bracelet using machine-learning algorithms (Random Forest, SVM, and Logistic Regression) to determine the efficacy of the abovementioned techniques in differentiating between stressful and non-stressful situations. Photoplethysmographic and electrodermal activity signals were collected from 29 subjects to extract 27 features which were then fed into three different machine-learning algorithms for binary classification. Using MATLAB after applying the chi-square test and Pearson's correlation coefficient on WEKA for features' importance ranking, the results demonstrated that the Random Forest model has the highest stability (accuracy of 76.5%) using all the features. Moreover, the Random Forest applying the chi-test for feature selection reached consistent results in terms of stress evaluation based on precision, recall, and F1-measure (71%, 60%, 65%, respectively).
Collapse
|
6
|
Fan Y, Liang J, Cao X, Pang L, Zhang J. Effects of Noise Exposure and Mental Workload on Physiological Responses during Task Execution. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph191912434. [PMID: 36231736 PMCID: PMC9566815 DOI: 10.3390/ijerph191912434] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 09/24/2022] [Accepted: 09/26/2022] [Indexed: 05/23/2023]
Abstract
Twelve healthy male students were recruited to investigate the physiological response to different noise exposure and mental workload (MW) conditions, while performing multi-attribute task battery (MATB) tasks. The experiments were conducted under three noise exposure conditions, with different sound pressure levels and sharpness. After adaptation to each noise condition, the participants were required to perform the resting test and the MATB task tests with low, medium, and high MW. The electroencephalogram (EEG), electrocardiogram (ECG), and eye movement data were obtained, during the periods when participants were in the resting and task taking state. The results showed that subjects' physiological responses at rest were unaffected by noise exposure conditions. However, during the execution of MATB tasks, the elevated sound pressure level and increased sharpness were significantly correlated with increased mean pupil diameter and heart rate variability (HRV). These responses suggested that the human body defends itself through physiological regulation when noise causes adverse effects. If the negative effects of noise were more severe, this could damage the body's health and result in a significant drop in task performance. The elevated mental demands led to increased stress on the subjects, which was reflected in a considerable increase in theta relative power. Either high or low MW was related with reduced saccade amplitude and a decrease in weighted task performance, indicating an inverted U-shaped relationship between workload level and work performance.
Collapse
Affiliation(s)
- Yurong Fan
- School of Aeronautic Science and Engineering, Beihang University, Beijing 100191, China
| | - Jin Liang
- Marine Human Factors Engineering Lab, China Institute of Marine Technology & Economy, Beijing 100081, China
| | - Xiaodong Cao
- School of Aeronautic Science and Engineering, Beihang University, Beijing 100191, China
| | - Liping Pang
- School of Aeronautic Science and Engineering, Beihang University, Beijing 100191, China
| | - Jie Zhang
- College of Aeronautics and Astronautics, Taiyuan University of Technology, Taiyuan 030024, China
| |
Collapse
|