1
|
Zeng C, Zhang J, Su Y, Li S, Wang Z, Li Q, Wang W. Driver Fatigue Detection Using Heart Rate Variability Features from 2-Minute Electrocardiogram Signals While Accounting for Sex Differences. SENSORS (BASEL, SWITZERLAND) 2024; 24:4316. [PMID: 39001095 PMCID: PMC11243895 DOI: 10.3390/s24134316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Revised: 06/22/2024] [Accepted: 07/01/2024] [Indexed: 07/16/2024]
Abstract
Traffic accidents due to fatigue account for a large proportion of road fatalities. Based on simulated driving experiments with drivers recruited from college students, this paper investigates the use of heart rate variability (HRV) features to detect driver fatigue while considering sex differences. Sex-independent and sex-specific differences in HRV features between alert and fatigued states derived from 2 min electrocardiogram (ECG) signals were determined. Then, decision trees were used for driver fatigue detection using the HRV features of either all subjects or those of only males or females. Nineteen, eighteen, and thirteen HRV features were significantly different (Mann-Whitney U test, p < 0.01) between the two mental states for all subjects, males, and females, respectively. The fatigue detection models for all subjects, males, and females achieved classification accuracies of 86.3%, 94.8%, and 92.0%, respectively. In conclusion, sex differences in HRV features between drivers' mental states were found according to both the statistical analysis and classification results. By considering sex differences, precise HRV feature-based driver fatigue detection systems can be developed. Moreover, in contrast to conventional methods using HRV features from 5 min ECG signals, our method uses HRV features from 2 min ECG signals, thus enabling more rapid driver fatigue detection.
Collapse
Affiliation(s)
- Chao Zeng
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China; (C.Z.)
- Hami Vocational and Technical College, Hami 839001, China
| | - Jiliang Zhang
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China; (C.Z.)
| | - Yizi Su
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| | - Shuguang Li
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Zhenyuan Wang
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| | - Qingkun Li
- Beijing Key Laboratory of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China
| | - Wenjun Wang
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| |
Collapse
|
2
|
Tsai CY, Cheong HI, Houghton R, Hsu WH, Lee KY, Kang JH, Kuan YC, Lee HC, Wu CJ, Li LYJ, Lin YT, Lin SY, Manole I, Majumdar A, Liu WT. Predicting Fatigue-Associated Aberrant Driving Behaviors Using a Dynamic Weighted Moving Average Model With a Long Short-Term Memory Network Based on Heart Rate Variability. HUMAN FACTORS 2024; 66:1681-1702. [PMID: 37387305 DOI: 10.1177/00187208231183874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
OBJECTIVE This study proposed a moving average (MA) approach to dynamically process heart rate variability (HRV) and developed aberrant driving behavior (ADB) prediction models by using long short-term memory (LSTM) networks. BACKGROUND Fatigue-associated ADBs have traffic safety implications. Numerous models to predict such acts based on physiological responses have been developed but are still in embryonic stages. METHOD This study recorded the data of 20 commercial bus drivers during their routine tasks on four consecutive days and subsequently asked them to complete questionnaires, including subjective sleep quality, driver behavior questionnaire and the Karolinska Sleepiness Scale. Driving behaviors and corresponding HRV were determined using a navigational mobile application and a wristwatch. The dynamic-weighted MA (DWMA) and exponential-weighted MA were used to process HRV in 5-min intervals. The data were independently separated for training and testing. Models were trained with 10-fold cross-validation strategy, their accuracies were evaluated, and Shapley additive explanation (SHAP) values were used to determine feature importance. RESULTS Significant increases in the standard deviation of NN intervals (SDNN), root mean square of successive heartbeat interval differences (RMSSD), and normalized spectrum of high frequency (nHF) were observed in the pre-event stage. The DWMA-based model exhibited the highest accuracy for both driver types (urban: 84.41%; highway: 80.56%). The SDNN, RMSSD, and nHF demonstrated relatively high SHAP values. CONCLUSION HRV metrics can serve as indicators of mental fatigue. DWMA-based LSTM could predict the occurrence of the level of fatigue associated with ADBs. APPLICATION The established models can be used in realistic driving scenarios.
Collapse
Affiliation(s)
- Cheng-Yu Tsai
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - He-In Cheong
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - Robert Houghton
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - Wen-Hua Hsu
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
| | - Kang-Yun Lee
- Division of Pulmonary Medicine, Department of Internal Medicine, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Jiunn-Horng Kang
- Department of Physical Medicine and Rehabilitation, Taipei Medical University Hospital, Taipei, Taiwan
- Research Center of Artificial Intelligence in Medicine, Taipei Medical University, Taipei, Taiwan
- Graduate Institute of Nanomedicine and Medical Engineering, College of Biomedical Engineering, Taipei Medical University, Taipei, Taiwan
| | - Yi-Chun Kuan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Department of Neurology, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Taipei Neuroscience Institute, Taipei Medical University, Taipei, Taiwan
- Department of Neurology, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Dementia Center, Taipei Medical University-Shuang Ho Hospital, New Taipei City, Taiwan
| | - Hsin-Chien Lee
- Department of Psychiatry, Taipei Medical University Hospital, Taipei, Taiwan
| | - Cheng-Jung Wu
- Department of Otolaryngology, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Lok-Yee Joyce Li
- Department of Medicine, Shin Kong Wu-Ho-Su Memorial Hospitall, Taipei, Taiwan
| | - Yin-Tzu Lin
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Shang-Yang Lin
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Iulia Manole
- Department of Civil and Environmental Engineering, Imperial College London, London, United Kingdom
| | - Arnab Majumdar
- Department of Civil and Environmental Engineering, Imperial College London, London, United Kingdom
| | - Wen-Te Liu
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Division of Pulmonary Medicine, Department of Internal Medicine, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Research Center of Artificial Intelligence in Medicine, Taipei Medical University, Taipei, Taiwan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| |
Collapse
|
3
|
Musicant O, Richmond-Hacham B, Botzer A. Cardiac indices of driver fatigue across in-lab and on-road studies. APPLIED ERGONOMICS 2024; 117:104202. [PMID: 38215606 DOI: 10.1016/j.apergo.2023.104202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 10/31/2023] [Accepted: 12/05/2023] [Indexed: 01/14/2024]
Abstract
Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.
Collapse
Affiliation(s)
- Oren Musicant
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| | - Bar Richmond-Hacham
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| | - Assaf Botzer
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| |
Collapse
|
4
|
Wang J, Stevens C, Bennett W, Yu D. Granular estimation of user cognitive workload using multi-modal physiological sensors. FRONTIERS IN NEUROERGONOMICS 2024; 5:1292627. [PMID: 38476759 PMCID: PMC10927958 DOI: 10.3389/fnrgo.2024.1292627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 02/02/2024] [Indexed: 03/14/2024]
Abstract
Mental workload (MWL) is a crucial area of study due to its significant influence on task performance and potential for significant operator error. However, measuring MWL presents challenges, as it is a multi-dimensional construct. Previous research on MWL models has focused on differentiating between two to three levels. Nonetheless, tasks can vary widely in their complexity, and little is known about how subtle variations in task difficulty influence workload indicators. To address this, we conducted an experiment inducing MWL in up to 5 levels, hypothesizing that our multi-modal metrics would be able to distinguish between each MWL stage. We measured the induced workload using task performance, subjective assessment, and physiological metrics. Our simulated task was designed to induce diverse MWL degrees, including five different math and three different verbal tiers. Our findings indicate that all investigated metrics successfully differentiated between various MWL levels induced by different tiers of math problems. Notably, performance metrics emerged as the most effective assessment, being the only metric capable of distinguishing all the levels. Some limitations were observed in the granularity of subjective and physiological metrics. Specifically, the subjective overall mental workload couldn't distinguish lower levels of workload, while all physiological metrics could detect a shift from lower to higher levels, but did not distinguish between workload tiers at the higher or lower ends of the scale (e.g., between the easy and the easy-medium tiers). Despite these limitations, each pair of levels was effectively differentiated by one or more metrics. This suggests a promising avenue for future research, exploring the integration or combination of multiple metrics. The findings suggest that subtle differences in workload levels may be distinguishable using combinations of subjective and physiological metrics.
Collapse
Affiliation(s)
- Jingkun Wang
- School of Industrial Engineering, Purdue University, West Lafayette, IN, United States
| | - Christopher Stevens
- Air Force Research Laboratory, Wright-Patterson AFB, Dayton, OH, United States
| | - Winston Bennett
- Air Force Research Laboratory, Wright-Patterson AFB, Dayton, OH, United States
| | - Denny Yu
- School of Industrial Engineering, Purdue University, West Lafayette, IN, United States
| |
Collapse
|
5
|
Wang F, Chen D, Zhang X. A transcutaneous acupoint electrical simulation glove for relieving the mental fatigue of crane drivers in real building environment. Comput Methods Biomech Biomed Engin 2024:1-14. [PMID: 38178699 DOI: 10.1080/10255842.2023.2301668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 12/29/2023] [Indexed: 01/06/2024]
Abstract
In the field of construction, the lifting environment of precast parts is more complex, which leads to the driver's fatigue. When the tower crane driver appears driving fatigue, it will appear slow operation response, hoisting precast parts appear abnormal swing, which will endanger the safety of on-site operators. Therefore, this study developed a kind of transcutaneous acupoint electrical stimulation gloves. When the crane driver wears this kind of glove, the good contraction of the glove can make the stimulation electrode closely fit with the three points, so as to perform electrical stimulation on the Neìguān point (PC6), Láogóng point (PC8) and Hégŭ point (L14) of the palm to relieve the driver's driving fatigue. In this study, non-periodic transcutaneous acupoint electrical stimulation (NPTAES) was used to stimulate human acupuncture points. This is different from the traditional periodic transcutaneous acupoint electrical stimulation (PTAES) method for relieving mental fatigue. In addition, this study used hilbert marginal spectral entropy (HMSE) to calculate the heart rate variability (HRV) characteristics of the subjects, so as to detect and analyze the driving fatigue of the drivers. At the same time, the drivers' blinking frequency and electroencephalogram (EEG) characteristics were analyzed comprehensively. The results show that: The NPTAES method used in this study is superior to the PTAES method in alleviating driving fatigue and greatly improves the efficiency of tower crane drivers. Compared to other methods, the HMSE method proposed in this study, when analyzing signals, stronger ability to characterize signal characteristics.
Collapse
Affiliation(s)
- Fuwang Wang
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| | - Daping Chen
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| | - Xiaolei Zhang
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| |
Collapse
|
6
|
Sriranga AK, Lu Q, Birrell S. A Systematic Review of In-Vehicle Physiological Indices and Sensor Technology for Driver Mental Workload Monitoring. SENSORS (BASEL, SWITZERLAND) 2023; 23:2214. [PMID: 36850812 PMCID: PMC9963326 DOI: 10.3390/s23042214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 02/06/2023] [Accepted: 02/13/2023] [Indexed: 06/18/2023]
Abstract
The concept of vehicle automation ceases to seem futuristic with the current advancement of the automotive industry. With the introduction of conditional automated vehicles, drivers are no longer expected to focus only on driving activities but are still required to stay alert to resume control. However, fluctuations in driving demands are known to alter the driver's mental workload (MWL), which might affect the driver's vehicle take-over capabilities. Driver mental workload can be specified as the driver's capacity for information processing for task performance. This paper summarizes the literature that relates to analysing driver mental workload through various in-vehicle physiological sensors focusing on cardiovascular and respiratory measures. The review highlights the type of study, hardware, method of analysis, test variable, and results of studies that have used physiological indices for MWL analysis in the automotive context.
Collapse
|
7
|
Lu K, Sjörs Dahlman A, Karlsson J, Candefjord S. Detecting driver fatigue using heart rate variability: A systematic review. ACCIDENT; ANALYSIS AND PREVENTION 2022; 178:106830. [PMID: 36155280 DOI: 10.1016/j.aap.2022.106830] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 07/05/2022] [Accepted: 09/01/2022] [Indexed: 06/16/2023]
Abstract
Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.
Collapse
Affiliation(s)
- Ke Lu
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden; SAFER Vehicle and Traffic Safety Centre, Chalmers University of Technology, Gothenburg, Sweden.
| | - Anna Sjörs Dahlman
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden; SAFER Vehicle and Traffic Safety Centre, Chalmers University of Technology, Gothenburg, Sweden; Swedish National Road and Transport Research Institute (VTI), Linköping, Sweden
| | - Johan Karlsson
- SAFER Vehicle and Traffic Safety Centre, Chalmers University of Technology, Gothenburg, Sweden; Autoliv Research, Autoliv Development AB, Vårgårda, Sweden
| | - Stefan Candefjord
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden; SAFER Vehicle and Traffic Safety Centre, Chalmers University of Technology, Gothenburg, Sweden
| |
Collapse
|
8
|
Wu Q, Fang G, Zhao J, Liu J. Effect of Transcranial Pulsed Current Stimulation on Fatigue Delay after Medium-Intensity Training. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19127042. [PMID: 35742289 PMCID: PMC9222574 DOI: 10.3390/ijerph19127042] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 06/02/2022] [Accepted: 06/07/2022] [Indexed: 02/04/2023]
Abstract
The purpose of this study was to investigate the effect of transcranial pulsed current stimulation (tPCS) on fatigue delay after medium-intensity training. Materials and Methods: Ninety healthy college athletes were randomly divided into an experimental group (n = 45) and control group (n = 45). The experimental group received medium-intensity training for a week. After each training, the experimental group received true stimulation of tPCS (continuous 15 min 1.5 mA current intensity stimulation). The control group received sham stimulation. The physiological and biochemical indicators of participants were tested before and after the experiment, and finally 30 participants in each group were included for data analysis. Results: In the experimental group, creatine kinase (CK), cortisol (C), time-domain heart rate variability indices root mean square of the successive differences (RMSSD), standard deviation of normal R-R intervals (SDNN), and frequency domain indicator low frequency (LF) all increased slowly after the intervention. Among these, CK, C, and SDNN values were significantly lower than those in the control group (p < 0.05). Testosterone (T), T/C, and heart rate variability frequency domain indicator high frequency (HF) in the experimental group decreased slowly after the intervention, and the HF value was significantly lower than that in the control group (p < 0.05). The changes in all of the indicators in the experimental group were smaller than those in the control group. Conclusion: The application of tPCS after medium-intensity training enhanced the adaptability to training and had a significant effect on the maintenance of physiological state. The application of tPCS can significantly promote the recovery of autonomic nervous system function, enhance the regulation of parasympathetic nerves, and delay the occurrence of fatigue.
Collapse
Affiliation(s)
- Qingchang Wu
- College of Sports Science, Nantong University, Nantong 226019, China;
| | - Guoliang Fang
- China Institute of Sport Science, Beijing 100061, China; (G.F.); (J.Z.)
| | - Jiexiu Zhao
- China Institute of Sport Science, Beijing 100061, China; (G.F.); (J.Z.)
| | - Jian Liu
- College of Sports Science, Nantong University, Nantong 226019, China;
- Correspondence:
| |
Collapse
|
9
|
Effects on Heart Rate Variability of Stress Level Responses to the Properties of Indoor Environmental Colors: A Preliminary Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18179136. [PMID: 34501724 PMCID: PMC8430831 DOI: 10.3390/ijerph18179136] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 08/26/2021] [Accepted: 08/27/2021] [Indexed: 12/24/2022]
Abstract
Color is the most potent stimulating factor affecting human vision, and the environmental color of an indoor space is a spatial component that affects the environmental stress level. As one of the methods of assessing the physiological response of the autonomic nervous system that influences stress, heart rate variability (HRV) has been utilized as a tool for measuring the user’s stress response in color environments. This study aims to identify the effects of the changes of hue, brightness, and saturation in environmental colors on the HRV of two groups with different stress levels—the stress potential group (n = 15) and the healthy group (n = 12)—based on their stress level indicated by the Psychosocial Well-being Index (PWI). The ln(LF), ln(HF), and RMSSD values collected during the subjects’ exposure to 12 environments colors of red and yellow with adjusted saturation and brightness, were statistically analyzed using t-test and two-way ANOVA. The results show that the HRV values in the two groups did not significantly vary in response to the changes in hue, brightness and saturation. The two groups’ stress factors distinguished according to the stress levels by the PWI scale affected the In(LF) parameter, which demonstrates that the PWI index can be utilized as a reliable scale for measuring stress levels. The ultra-short HRV measurement record and the use of a sole In(LF) parameter for stress assessment are regarded as the limitations of this study.
Collapse
|