1
|
Pan H, Payre W, Gao Z, Wang Y. Exploring driving anger-caused impairment of takeover performance among professional taxi drivers during partially automated driving. ACCIDENT; ANALYSIS AND PREVENTION 2024; 205:107686. [PMID: 38909484 DOI: 10.1016/j.aap.2024.107686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 05/24/2024] [Accepted: 06/16/2024] [Indexed: 06/25/2024]
Abstract
Partially automated systems are expected to reduce road crashes related to human error, even amongst professional drivers. Consequently, the applications of these systems into the taxi industry would potentially improve transportation safety. However, taxi drivers are prone to experiencing driving anger, which may subsequently affect their takeover performance. In this research, we explored how driving anger emotion affects taxi drivers' driving performance in various takeover scenarios, namely Mandatory Automation-Initiated transition (MAIT), Mandatory Driver-Initiated transition (MDIT), and Optional Driver-Initiated transition (ODIT). Forty-seven taxi drivers participated in this 2·3 mixed design simulator experiment (between-subjects: anger vs. calmness; within-subjects: MAIT vs. MDIT vs. ODIT). Compared to calmness, driving anger emotion led to a narrower field of attention (e.g., smaller standard deviations of horizontal fixation points position) and worse hazard perception (e.g., longer saccade latency, smaller amplitude of skin conductance responses), which resulted in longer takeover time and inferior vehicle control stability (e.g., higher standard deviations of lateral position) in MAIT and MDIT scenarios. Angry taxi drivers were more likely to deactivate vehicle automation and take over the vehicle in a more aggressive manner (e.g., higher maximal resulting acceleration, refusing to yield to other road users) in ODIT scenarios. The findings will contribute to addressing the safety concerns related to driving anger among professional taxi drivers and promote the widespread acceptance and integration of partially automated systems within the taxi industry.
Collapse
Affiliation(s)
- Hengyan Pan
- School of Transportation Engineering, Chang'an University, Xi'an 710018, China.
| | - William Payre
- National Transport Design Centre, Coventry University, Coventry CV1 2TT, UK.
| | - Zhixiang Gao
- School of Transportation Engineering, Chang'an University, Xi'an 710018, China
| | - Yonggang Wang
- School of Transportation Engineering, Chang'an University, Xi'an 710018, China.
| |
Collapse
|
2
|
Bögge L, Colás-Blanco I, Ferragu J, Gaston-Bellegarde A, La Corte V, Piolino P. Autonomic and cognitive control in memory: Investigating the psychophysiological link using heart rate variability biofeedback. Psychophysiology 2024; 61:e14588. [PMID: 38769698 DOI: 10.1111/psyp.14588] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 02/03/2024] [Accepted: 03/30/2024] [Indexed: 05/22/2024]
Abstract
Vagally mediated heart rate variability (vmHRV) at resting state has been associated to cognitive functions dependent on cognitive control, such as memory. However, little is known about the phasic interaction between cognitive and autonomic control. In a pre-registered within-between-subject designed experiment, the potential of vmHRV biofeedback to simultaneously stimulate vmHRV during memory processing and cognitive control over long-term memory was tested, along with investigating psychophysiological association. 71 young healthy adults completed (twice) a false memory task in virtual reality. Immediately before memory encoding and retrieval, participants practiced either vmHRV biofeedback or a control breathing exercise. Cognitive control over long-term memory was assessed as the confidence toward false memories and the capability to discriminate them from true memories. Resting-state vmHRV before each test and phasic vmHRV during memory encoding and retrieval were measured as the root mean square differences (RMSSD) in the heart period. vmHRV biofeedback had neither an immediate effect on measures of cognitive control over long-term memory nor on phasic RMSSD. Moreover, neither resting-state nor phasic vmHRV correlated to the cognitive scores. Consequently, the utility of HRV biofeedback as a psychophysiological stimulation tool and a link between vmHRV and cognitive control over long-term memory could not be verified. Exploratory analyses revealed that baseline shift in parasympathetic activity confounded the psychophysiological association. Future directions are provided that could shed light on the relationship between cognition and vmHRV.
Collapse
Affiliation(s)
- Lukas Bögge
- Institute of Psychology, Memory, Brain & Cognition Laboratory, Université Paris Cité, Paris, France
| | - Itsaso Colás-Blanco
- Institute of Psychology, Memory, Brain & Cognition Laboratory, Université Paris Cité, Paris, France
| | - Juliette Ferragu
- Institute of Psychology, Memory, Brain & Cognition Laboratory, Université Paris Cité, Paris, France
| | | | - Valentina La Corte
- Institute of Psychology, Memory, Brain & Cognition Laboratory, Université Paris Cité, Paris, France
- Institut Universitaire de France (IUF), Paris, France
| | - Pascale Piolino
- Institute of Psychology, Memory, Brain & Cognition Laboratory, Université Paris Cité, Paris, France
- Institut Universitaire de France (IUF), Paris, France
| |
Collapse
|
3
|
Zeng C, Zhang J, Su Y, Li S, Wang Z, Li Q, Wang W. Driver Fatigue Detection Using Heart Rate Variability Features from 2-Minute Electrocardiogram Signals While Accounting for Sex Differences. SENSORS (BASEL, SWITZERLAND) 2024; 24:4316. [PMID: 39001095 PMCID: PMC11243895 DOI: 10.3390/s24134316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Revised: 06/22/2024] [Accepted: 07/01/2024] [Indexed: 07/16/2024]
Abstract
Traffic accidents due to fatigue account for a large proportion of road fatalities. Based on simulated driving experiments with drivers recruited from college students, this paper investigates the use of heart rate variability (HRV) features to detect driver fatigue while considering sex differences. Sex-independent and sex-specific differences in HRV features between alert and fatigued states derived from 2 min electrocardiogram (ECG) signals were determined. Then, decision trees were used for driver fatigue detection using the HRV features of either all subjects or those of only males or females. Nineteen, eighteen, and thirteen HRV features were significantly different (Mann-Whitney U test, p < 0.01) between the two mental states for all subjects, males, and females, respectively. The fatigue detection models for all subjects, males, and females achieved classification accuracies of 86.3%, 94.8%, and 92.0%, respectively. In conclusion, sex differences in HRV features between drivers' mental states were found according to both the statistical analysis and classification results. By considering sex differences, precise HRV feature-based driver fatigue detection systems can be developed. Moreover, in contrast to conventional methods using HRV features from 5 min ECG signals, our method uses HRV features from 2 min ECG signals, thus enabling more rapid driver fatigue detection.
Collapse
Affiliation(s)
- Chao Zeng
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China; (C.Z.)
- Hami Vocational and Technical College, Hami 839001, China
| | - Jiliang Zhang
- College of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China; (C.Z.)
| | - Yizi Su
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| | - Shuguang Li
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Zhenyuan Wang
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| | - Qingkun Li
- Beijing Key Laboratory of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China
| | - Wenjun Wang
- School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
| |
Collapse
|
4
|
Wang P, Houghton R, Majumdar A. Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:3723. [PMID: 38931507 PMCID: PMC11207491 DOI: 10.3390/s24123723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 05/30/2024] [Accepted: 06/04/2024] [Indexed: 06/28/2024]
Abstract
Measuring pilot mental workload (MWL) is crucial for enhancing aviation safety. However, MWL is a multi-dimensional construct that could be affected by multiple factors. Particularly, in the context of a more automated cockpit setting, the traditional methods of assessing pilot MWL may face challenges. Heart rate variability (HRV) has emerged as a potential tool for detecting pilot MWL during real-flight operations. This review aims to investigate the relationship between HRV and pilot MWL and to assess the performance of machine-learning-based MWL detection systems using HRV parameters. A total of 29 relevant papers were extracted from three databases for review based on rigorous eligibility criteria. We observed significant variability across the reviewed studies, including study designs and measurement methods, as well as machine-learning techniques. Inconsistent results were observed regarding the differences in HRV measures between pilots under varying levels of MWL. Furthermore, for studies that developed HRV-based MWL detection systems, we examined the diverse model settings and discovered that several advanced techniques could be used to address specific challenges. This review serves as a practical guide for researchers and practitioners who are interested in employing HRV indicators for evaluating MWL and wish to incorporate cutting-edge techniques into their MWL measurement approaches.
Collapse
Affiliation(s)
| | | | - Arnab Majumdar
- Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, London SW7 2AZ, UK; (P.W.); (R.H.)
| |
Collapse
|
5
|
Ahlström C, Anund A. Development of sleepiness in professional truck drivers: Real-road testing for driver drowsiness and attention warning (DDAW) system evaluation. J Sleep Res 2024:e14259. [PMID: 38837467 DOI: 10.1111/jsr.14259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Revised: 05/16/2024] [Accepted: 05/22/2024] [Indexed: 06/07/2024]
Abstract
All new vehicle types within the European Union must now be equipped with a driver drowsiness and attention warning system starting from 2022. The specific requirements for the test procedure necessary for type approval are defined in the Annex of EU Regulation C/2021/2639. The objectives of this study were to: (i) investigate how sleepiness develops in professional truck drivers under real-road driving conditions; and (ii) assess the feasibility of a test procedure for validating driver drowsiness and attention warning systems according to the EU regulation. Twenty-four professional truck drivers participated in the test. They drove for 180 km on a dual-lane motorway, first during daytime after a normal night's sleep and then at nighttime after being awake since early morning. The results showed higher sleepiness levels during nighttime driving compared with daytime, with a faster increase in sleepiness with distance driven, especially during the night. Psychomotor vigilance task results corroborated these findings. From a driver drowsiness and attention warning testing perspective, the study design with sleep-deprived drivers at night was successful in inducing the targeted sleepiness level of a Karolinska Sleepiness Scale score of ≥ 8. Many drivers who reported a Karolinska Sleepiness Scale ≥ 8 during the drives also acknowledged feeling sleepy in the post-drive questionnaire. Reaching high levels of sleepiness on real roads during daytime is more problematic, not the least from legal and ethical perspectives as higher traffic densities during the daytime lead to increased risks.
Collapse
Affiliation(s)
- Christer Ahlström
- Swedish National Road and Transport Research Institute, Linköping, Sweden
- Department of Biomedical Engineering, Linköping University, Linköping, Sweden
| | - Anna Anund
- Swedish National Road and Transport Research Institute, Linköping, Sweden
- Rehabilitation Medicine, Linköping University, Linköping, Sweden
| |
Collapse
|
6
|
Tsai CY, Cheong HI, Houghton R, Hsu WH, Lee KY, Kang JH, Kuan YC, Lee HC, Wu CJ, Li LYJ, Lin YT, Lin SY, Manole I, Majumdar A, Liu WT. Predicting Fatigue-Associated Aberrant Driving Behaviors Using a Dynamic Weighted Moving Average Model With a Long Short-Term Memory Network Based on Heart Rate Variability. HUMAN FACTORS 2024; 66:1681-1702. [PMID: 37387305 DOI: 10.1177/00187208231183874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
OBJECTIVE This study proposed a moving average (MA) approach to dynamically process heart rate variability (HRV) and developed aberrant driving behavior (ADB) prediction models by using long short-term memory (LSTM) networks. BACKGROUND Fatigue-associated ADBs have traffic safety implications. Numerous models to predict such acts based on physiological responses have been developed but are still in embryonic stages. METHOD This study recorded the data of 20 commercial bus drivers during their routine tasks on four consecutive days and subsequently asked them to complete questionnaires, including subjective sleep quality, driver behavior questionnaire and the Karolinska Sleepiness Scale. Driving behaviors and corresponding HRV were determined using a navigational mobile application and a wristwatch. The dynamic-weighted MA (DWMA) and exponential-weighted MA were used to process HRV in 5-min intervals. The data were independently separated for training and testing. Models were trained with 10-fold cross-validation strategy, their accuracies were evaluated, and Shapley additive explanation (SHAP) values were used to determine feature importance. RESULTS Significant increases in the standard deviation of NN intervals (SDNN), root mean square of successive heartbeat interval differences (RMSSD), and normalized spectrum of high frequency (nHF) were observed in the pre-event stage. The DWMA-based model exhibited the highest accuracy for both driver types (urban: 84.41%; highway: 80.56%). The SDNN, RMSSD, and nHF demonstrated relatively high SHAP values. CONCLUSION HRV metrics can serve as indicators of mental fatigue. DWMA-based LSTM could predict the occurrence of the level of fatigue associated with ADBs. APPLICATION The established models can be used in realistic driving scenarios.
Collapse
Affiliation(s)
- Cheng-Yu Tsai
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - He-In Cheong
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - Robert Houghton
- Department of Civil and Environmental Engineering, Imperial College London, London, UK
| | - Wen-Hua Hsu
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
| | - Kang-Yun Lee
- Division of Pulmonary Medicine, Department of Internal Medicine, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Jiunn-Horng Kang
- Department of Physical Medicine and Rehabilitation, Taipei Medical University Hospital, Taipei, Taiwan
- Research Center of Artificial Intelligence in Medicine, Taipei Medical University, Taipei, Taiwan
- Graduate Institute of Nanomedicine and Medical Engineering, College of Biomedical Engineering, Taipei Medical University, Taipei, Taiwan
| | - Yi-Chun Kuan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Department of Neurology, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Taipei Neuroscience Institute, Taipei Medical University, Taipei, Taiwan
- Department of Neurology, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Dementia Center, Taipei Medical University-Shuang Ho Hospital, New Taipei City, Taiwan
| | - Hsin-Chien Lee
- Department of Psychiatry, Taipei Medical University Hospital, Taipei, Taiwan
| | - Cheng-Jung Wu
- Department of Otolaryngology, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Lok-Yee Joyce Li
- Department of Medicine, Shin Kong Wu-Ho-Su Memorial Hospitall, Taipei, Taiwan
| | - Yin-Tzu Lin
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Shang-Yang Lin
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Iulia Manole
- Department of Civil and Environmental Engineering, Imperial College London, London, United Kingdom
| | - Arnab Majumdar
- Department of Civil and Environmental Engineering, Imperial College London, London, United Kingdom
| | - Wen-Te Liu
- School of Respiratory Therapy, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Division of Pulmonary Medicine, Department of Internal Medicine, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
- Research Center of Artificial Intelligence in Medicine, Taipei Medical University, Taipei, Taiwan
- Sleep Center, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| |
Collapse
|
7
|
Dong M, Lee YY, Cha JS, Huang G. Drinking and driving: A systematic review of the impacts of alcohol consumption on manual and automated driving performance. JOURNAL OF SAFETY RESEARCH 2024; 89:1-12. [PMID: 38858032 DOI: 10.1016/j.jsr.2024.01.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 10/06/2023] [Accepted: 01/16/2024] [Indexed: 06/12/2024]
Abstract
INTRODUCTION Almost a third of car accidents involve driving after alcohol consumption. Autonomous vehicles (AVs) may offer accident-prevention benefits, but at current automation levels, drivers must still perform manual driving tasks when automated systems fail. Therefore, understanding how alcohol affects driving in both manual and automated contexts offers insight into the role of future vehicle design in mediating crash risks for alcohol-impaired driving. METHOD This study conducted a systematic review on alcohol effects on manual and automated (takeover) driving performance. Fifty-three articles from eight databases were analyzed, with findings structured based on the information processing model, which can be extended to the AV takeover model. RESULTS The literature indicates that different Blood Alcohol Concentration (BAC) levels affect driving skills essential for traffic safety at various information processing stages, such as delayed reacting time, impaired cognitive abilities, and hindered execution of driving tasks. Additionally, the driver's driving experience, drinking habits, and external driving environment play important roles in influencing driving performance. CONCLUSIONS Future work is needed to examine the effects of alcohol on driving performance, particularly in AVs and takeover situations, and to develop driver monitoring systems. PRACTICAL APPLICATIONS Findings from this review can inform future experiments, AV technology design, and the development of driver state monitoring systems.
Collapse
Affiliation(s)
- Miaomiao Dong
- Department of Industrial and Systems Engineering, San Jose State University One Washington Square, San Jose, CA 95192, USA
| | - Yuni Y Lee
- Department of Industrial and Systems Engineering, San Jose State University One Washington Square, San Jose, CA 95192, USA
| | - Jackie S Cha
- Department of Industrial Engineering, Clemson University 268 Freeman Hall, Clemson, SC 29634, USA
| | - Gaojian Huang
- Department of Industrial and Systems Engineering, San Jose State University One Washington Square, San Jose, CA 95192, USA.
| |
Collapse
|
8
|
Adam R, Lotankar Y, Sas C, Powell D, Martinez V, Green S, Cooper J, Bradbury K, Sive J, Hill DL. Understanding patterns of fatigue in health and disease: protocol for an ecological momentary assessment study using digital technologies. BMJ Open 2024; 14:e081416. [PMID: 38802273 PMCID: PMC11131111 DOI: 10.1136/bmjopen-2023-081416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 05/10/2024] [Indexed: 05/29/2024] Open
Abstract
INTRODUCTION Fatigue is prevalent across a wide range of medical conditions and can be debilitating and distressing. It is likely that fatigue is experienced differently according to the underlying aetiology, but this is poorly understood. Digital health technologies present a promising approach to give new insights into fatigue.The aim of this study is to use digital health technologies, real-time self-reports and qualitative interview data to investigate how fatigue is experienced over time in participants with myeloma, long COVID, heart failure and in controls without problematic fatigue. Objectives are to understand which sensed parameters add value to the characterisation of fatigue and to determine whether study processes are feasible, acceptable and scalable. METHODS AND ANALYSIS An ecological momentary assessment study will be carried out over 2 or 4 weeks (participant defined). Individuals with fatigue relating to myeloma (n=10), heart failure (n=10), long COVID (n=10) and controls without problematic fatigue or a study condition (n=10) will be recruited. ECG patches will measure heart rate variability, respiratory rate, body temperature, activity and posture. A wearable bracelet accompanied by environment beacons will measure physical activity, sleep and room location within the home. Self-reports of mental and physical fatigue will be collected via smartphone app four times daily and on-demand. Validated fatigue and affect questionnaires will be completed at baseline and at 2 weeks. End-of-study interviews will investigate experiences of fatigue and study participation. A feedback session will be offered to participants to discuss their data.Data will be analysed using multilevel modelling and machine learning. Interviews and feedback sessions will be analysed using content or thematic analyses. ETHICS AND DISSEMINATION This study was approved by the East of England-Cambridge East Research Ethics Committee (22/EE/0261). The results will be disseminated in peer-reviewed journals and at international conferences. TRIAL REGISTRATION NUMBER NCT05622669.
Collapse
Affiliation(s)
- Rosalind Adam
- Academic Primary Care, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Yojana Lotankar
- Psychology and Neuroscience, University of Westminster, London, UK
| | - Corina Sas
- School of Computing and Communications, Lancaster University, Lancaster, UK
| | - Daniel Powell
- Health Psychology, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | | | - Stephen Green
- Department of Engineering, University of Cambridge, Cambridge, UK
| | - Jonathan Cooper
- James Watt School of Engineering, University of Glasgow, Glasgow, UK
| | | | - Jonathan Sive
- University College London Hospitals NHS Foundation Trust, London, UK
| | - Derek L Hill
- Medical Physics & Biomedical engineering, University College London Faculty of Engineering Sciences, London, UK
- Panoramic Digital Health Ltd, Grenoble, France
| |
Collapse
|
9
|
Musicant O, Richmond-Hacham B, Botzer A. Cardiac indices of driver fatigue across in-lab and on-road studies. APPLIED ERGONOMICS 2024; 117:104202. [PMID: 38215606 DOI: 10.1016/j.apergo.2023.104202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 10/31/2023] [Accepted: 12/05/2023] [Indexed: 01/14/2024]
Abstract
Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.
Collapse
Affiliation(s)
- Oren Musicant
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| | - Bar Richmond-Hacham
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| | - Assaf Botzer
- Industrial Engineering & Management, Ariel University, Kiriat Hamada, Ariel, Israel.
| |
Collapse
|
10
|
He C, Xu P, Pei X, Wang Q, Yue Y, Han C. Fatigue at the wheel: A non-visual approach to truck driver fatigue detection by multi-feature fusion. ACCIDENT; ANALYSIS AND PREVENTION 2024; 199:107511. [PMID: 38387154 DOI: 10.1016/j.aap.2024.107511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 01/28/2024] [Accepted: 02/15/2024] [Indexed: 02/24/2024]
Abstract
BACKGROUND Monitoring of long-haul truck driver fatigue state has attracted considerable interest. Conventional fatigue driving detection methods based on the physiological and visual features are scarcely applicable, due to the intrusiveness, reliability, and cost-effectiveness concerns. METHODS We elaborately developed a fatigue driving detection method by fusion of non-visual features derived from the customized wristbands, vehicle-mounted equipment, and trip logs. To capture the spatiotemporal information within the sequential data, the bidirectional long short-term memory network with attention mechanism was proposed to determine whether the truck driver was fatigued within a fine-grained episode of one minute. The model was validated using a natural driving dataset with nine truck drivers on real-world roads in Guiyang, China during June and July 2021. RESULTS Our approach yielded 99.21 %, 84.44 %, 82.01 %, 99.63 %, and 83.21 % in accuracy, precision, recall, specificity, and F1-score, respectively. Compared with the mainstream visual-based methods, our approach outperformed particularly in terms of precision and recall. Photoplethysmogram stood out as the most important feature for truck driver fatigue state detection. Vehicle load, driving forward angle, cumulative driving time, midnight, and recent working hours were found to be positively associated with the probability of fatigue driving, while the galvanic skin response, vehicle acceleration, current time, and recent rest hours had a negative relationship. Specifically, truck drivers were more likely to fatigue when driving at 20-40 km/h, braking abruptly at 5-10 m/s2, with vehicle loads over 70 tons, and driving more than 100 min consecutively. CONCLUSIONS Our study is among the first to harness the natural driving dataset to delve into the real-life fatigue pattern of long-haul truck drivers without disruptions on routine driving tasks. The proposed method holds pragmatic prospects by providing a privacy-preserving, robust, real-time, and non-intrusive technical pathway for truck driver fatigue monitoring.
Collapse
Affiliation(s)
- Chen He
- Department of Automation, BNRIST, Tsinghua University, Beijing 100084, China
| | - Pengpeng Xu
- School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510641, Guangdong, China; Hunan Key Laboratory of Smart Roadway and Cooperative Vehicle-Infrastructure Systems, Changsha University of Science & Technology, Changsha 410114, Hunan, China
| | - Xin Pei
- Department of Automation, BNRIST, Tsinghua University, Beijing 100084, China.
| | - Qianfang Wang
- School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510641, Guangdong, China
| | - Yun Yue
- Department of Automation, BNRIST, Tsinghua University, Beijing 100084, China
| | - Chunyang Han
- Faculty of Transportation Engineering, Kunming University of Science and Technology, Kunming 650500, Yunnan, China
| |
Collapse
|
11
|
Kong L, Xie K, Niu K, He J, Zhang W. Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion. SENSORS (BASEL, SWITZERLAND) 2024; 24:455. [PMID: 38257546 PMCID: PMC11154312 DOI: 10.3390/s24020455] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2023] [Revised: 01/04/2024] [Accepted: 01/08/2024] [Indexed: 01/24/2024]
Abstract
Existing vision-based fatigue detection methods commonly utilize RGB cameras to extract facial and physiological features for monitoring driver fatigue. These features often include single indicators such as eyelid movement, yawning frequency, and heart rate. However, the accuracy of RGB cameras can be affected by factors like varying lighting conditions and motion. To address these challenges, we propose a non-invasive method for multi-modal fusion fatigue detection called RPPMT-CNN-BiLSTM. This method incorporates a feature extraction enhancement module based on the improved Pan-Tompkins algorithm and 1D-MTCNN. This enhances the accuracy of heart rate signal extraction and eyelid features. Furthermore, we use one-dimensional neural networks to construct two models based on heart rate and PERCLOS values, forming a fatigue detection model. To enhance the robustness and accuracy of fatigue detection, the trained model data results are input into the BiLSTM network. This generates a time-fitting relationship between the data extracted from the CNN, allowing for effective dynamic modeling and achieving multi-modal fusion fatigue detection. Numerous experiments validate the effectiveness of the proposed method, achieving an accuracy of 98.2% on the self-made MDAD (Multi-Modal Driver Alertness Dataset). This underscores the feasibility of the algorithm. In comparison with traditional methods, our approach demonstrates higher accuracy and positively contributes to maintaining traffic safety, thereby advancing the field of smart transportation.
Collapse
Affiliation(s)
- Lingjian Kong
- School of Electronic Information and Electrical Engineering, Yangtze University, Jingzhou 434023, China; (L.K.); (K.N.)
| | - Kai Xie
- School of Electronic Information and Electrical Engineering, Yangtze University, Jingzhou 434023, China; (L.K.); (K.N.)
| | - Kaixuan Niu
- School of Electronic Information and Electrical Engineering, Yangtze University, Jingzhou 434023, China; (L.K.); (K.N.)
| | - Jianbiao He
- School of Computer Science, Central South University, Changsha 410083, China;
| | - Wei Zhang
- School of Electronic Information, Central South University, Changsha 410083, China;
| |
Collapse
|
12
|
Csathó Á, Van der Linden D, Matuz A. Change in heart rate variability with increasing time-on-task as a marker for mental fatigue: A systematic review. Biol Psychol 2024; 185:108727. [PMID: 38056707 DOI: 10.1016/j.biopsycho.2023.108727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 11/30/2023] [Accepted: 12/01/2023] [Indexed: 12/08/2023]
Abstract
Fatigue-specific changes in the autonomic nervous system are often assumed to underlie the development of mental fatigue caused by prolonged cognitive tasks (i.e. Time-on-Task). Therefore, several previous studies have chosen to investigate the Time-on-Task related changes in heart rate variability (HRV). However, previous studies have used many different HRV indices, and their results often show inconsistencies. The present study, therefore, systematically reviewed previous empirical HRV studies with healthy individuals and in which mental fatigue is induced by prolonged cognitive tasks. Articles relevant to the objectives were systematically searched and selected by applying the PRISMA guidelines. We screened 360 records found on 4 databases and found that 19 studies were eligible for full review in accordance with the inclusion criteria. In general, all studies reviewed (with the exception of two studies) found significant changes in HRV with increasing Time-on-Task, suggesting that HRV is a reliable autonomic marker for Time-on-Task induced fatigue. The most conclusive HRV indices that showed a consistent Time-on-Task effect were the low frequency component of HRV and the time domain indices, particularly the root mean square of successive differences. Time-on-Task typically induced an increasing trend in both type of measures.
Collapse
Affiliation(s)
- Árpád Csathó
- Department of Behavioural Sciences, Medical School, University of Pécs, Pécs, Hungary; Szentágothai Research Centre, University of Pécs, Pécs, Hungary.
| | - Dimitri Van der Linden
- Department of Psychology, Education, and Child Studies, Erasmus University Rotterdam, Rotterdam, the Netherlands.
| | - András Matuz
- Department of Behavioural Sciences, Medical School, University of Pécs, Pécs, Hungary; Szentágothai Research Centre, University of Pécs, Pécs, Hungary.
| |
Collapse
|
13
|
Ayas S, Donmez B, Tang X. Drowsiness Mitigation Through Driver State Monitoring Systems: A Scoping Review. HUMAN FACTORS 2023:187208231208523. [PMID: 37982386 DOI: 10.1177/00187208231208523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/21/2023]
Abstract
OBJECTIVE To explore the scope of available research and to identify research gaps on in-vehicle interventions for drowsiness that utilize driver monitoring systems (DMS). BACKGROUND DMS are gaining popularity as a countermeasure against drowsiness. However, how these systems can be best utilized to guide driver attention is unclear. METHODS A scoping review was conducted in adherence to PRISMA guidelines. Five electronic databases (ACM Digital Library, Scopus, IEEE Xplore, TRID, and SAE Mobilus) were systematically searched in April 2022. Original studies examining in-vehicle drowsiness interventions that use DMS in a driving context (e.g., driving simulator and driver interviews) passed the screening. Data on study details, state detection methods, and interventions were extracted. RESULTS Twenty studies qualified for inclusion. Majority of interventions involved warnings (n = 16) with an auditory component (n = 14). Feedback displays (n = 4) and automation takeover (n = 4) were also investigated. Multistage interventions (n = 12) first cautioned the driver, then urged them to take an action, or initiated an automation takeover. Overall, interventions had a positive impact on sleepiness levels, driving performance, and user evaluations. Whether interventions effective for one type of sleepiness (e.g., passive vs. active fatigue) will perform well for another type is unclear. CONCLUSION Literature mainly focused on developing sensors and improving the accuracy of DMS, but not on the driver interactions with these technologies. More intervention studies are needed in general and for investigating their long-term effects. APPLICATION We list gaps and limitations in the DMS literature to guide researchers and practitioners in designing and evaluating effective safety systems for drowsy driving.
Collapse
Affiliation(s)
- Suzan Ayas
- University of Toronto, Toronto, ON, Canada
| | | | - Xing Tang
- University of Toronto, Toronto, ON, Canada
- Northwestern Polytechnical University, Xi'an, China
| |
Collapse
|
14
|
Coyne R, Ryan L, Moustafa M, Smeaton AF, Corcoran P, Walsh JC. Assessing the physiological effect of non-driving-related task performance and task modality in conditionally automated driving systems: A systematic review and meta-analysis. ACCIDENT; ANALYSIS AND PREVENTION 2023; 192:107243. [PMID: 37651857 DOI: 10.1016/j.aap.2023.107243] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 07/12/2023] [Accepted: 07/30/2023] [Indexed: 09/02/2023]
Abstract
In conditionally automated driving, the driver is free to disengage from controlling the vehicle, but they are expected to resume driving in response to certain situations or events that the system is not equipped to respond to. As the level of vehicle automation increases, drivers often engage in non-driving-related tasks (NDRTs), defined as any secondary task unrelated to the primary task of driving. This engagement can have a detrimental effect on the driver's situation awareness and attentional resources. NDRTs with resource demands that overlap with the driving task, such as visual or manual tasks, may be particularly deleterious. Therefore, monitoring the driver's state is an important safety feature for conditionally automated vehicles, and physiological measures constitute a promising means of doing this. The present systematic review and meta-analysis synthesises findings from 32 studies concerning the effect of NDRTs on drivers' physiological responses, in addition to the effect of NDRTs with a visual or a manual modality. Evidence was found that NDRT engagement led to higher physiological arousal, indicated by increased heart rate, electrodermal activity and a decrease in heart rate variability. There was mixed evidence for an effect of both visual and manual NDRT modalities on all physiological measures. Understanding the relationship between task performance and arousal during automated driving is of critical importance to the development of driver monitoring systems and improving the safety of this technology.
Collapse
Affiliation(s)
- Rory Coyne
- School of Psychology, University of Galway, Ireland.
| | - Leona Ryan
- School of Psychology, University of Galway, Ireland
| | | | - Alan F Smeaton
- School of Computing, Dublin City University, Dublin, Ireland
| | - Peter Corcoran
- Department of Electrical and Electronic Engineering, University of Galway, Ireland
| | - Jane C Walsh
- School of Psychology, University of Galway, Ireland
| |
Collapse
|
15
|
Wang L, Song F, Zhou TH, Hao J, Ryu KH. EEG and ECG-Based Multi-Sensor Fusion Computing for Real-Time Fatigue Driving Recognition Based on Feedback Mechanism. SENSORS (BASEL, SWITZERLAND) 2023; 23:8386. [PMID: 37896480 PMCID: PMC10611368 DOI: 10.3390/s23208386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 10/07/2023] [Accepted: 10/10/2023] [Indexed: 10/29/2023]
Abstract
A variety of technologies that could enhance driving safety are being actively explored, with the aim of reducing traffic accidents by accurately recognizing the driver's state. In this field, three mainstream detection methods have been widely applied, namely visual monitoring, physiological indicator monitoring and vehicle behavior analysis. In order to achieve more accurate driver state recognition, we adopted a multi-sensor fusion approach. We monitored driver physiological signals, electroencephalogram (EEG) signals and electrocardiogram (ECG) signals to determine fatigue state, while an in-vehicle camera observed driver behavior and provided more information for driver state assessment. In addition, an outside camera was used to monitor vehicle position to determine whether there were any driving deviations due to distraction or fatigue. After a series of experimental validations, our research results showed that our multi-sensor approach exhibited good performance for driver state recognition. This study could provide a solid foundation and development direction for future in-depth driver state recognition research, which is expected to further improve road safety.
Collapse
Affiliation(s)
- Ling Wang
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Fangjie Song
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Tie Hua Zhou
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Jiayu Hao
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Keun Ho Ryu
- Data Science Laboratory, Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam;
- Biomedical Engineering Institute, Chiang Mai University, Chiang Mai 50200, Thailand
- Department of Computer Science, College of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, Republic of Korea
| |
Collapse
|
16
|
Parlato S, Centracchio J, Esposito D, Bifulco P, Andreozzi E. ECG-Free Heartbeat Detection in Seismocardiography and Gyrocardiography Signals Provides Acceptable Heart Rate Variability Indices in Healthy and Pathological Subjects. SENSORS (BASEL, SWITZERLAND) 2023; 23:8114. [PMID: 37836942 PMCID: PMC10575135 DOI: 10.3390/s23198114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 09/12/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023]
Abstract
Cardio-mechanical monitoring techniques, such as Seismocardiography (SCG) and Gyrocardiography (GCG), have received an ever-growing interest in recent years as potential alternatives to Electrocardiography (ECG) for heart rate monitoring. Wearable SCG and GCG devices based on lightweight accelerometers and gyroscopes are particularly appealing for continuous, long-term monitoring of heart rate and its variability (HRV). Heartbeat detection in cardio-mechanical signals is usually performed with the support of a concurrent ECG lead, which, however, limits their applicability in standalone cardio-mechanical monitoring applications. The complex and variable morphology of SCG and GCG signals makes the ECG-free heartbeat detection task quite challenging; therefore, only a few methods have been proposed. Very recently, a template matching method based on normalized cross-correlation (NCC) has been demonstrated to provide very accurate detection of heartbeats and estimation of inter-beat intervals in SCG and GCG signals of pathological subjects. In this study, the accuracy of HRV indices obtained with this template matching method is evaluated by comparison with ECG. Tests were performed on two public datasets of SCG and GCG signals from healthy and pathological subjects. Linear regression, correlation, and Bland-Altman analyses were carried out to evaluate the agreement of 24 HRV indices obtained from SCG and GCG signals with those obtained from ECG signals, simultaneously acquired from the same subjects. The results of this study show that the NCC-based template matching method allowed estimating HRV indices from SCG and GCG signals of healthy subjects with acceptable accuracy. On healthy subjects, the relative errors on time-domain indices ranged from 0.25% to 15%, on frequency-domain indices ranged from 10% to 20%, and on non-linear indices were within 8%. The estimates obtained on signals from pathological subjects were affected by larger errors. Overall, GCG provided slightly better performances as compared to SCG, both on healthy and pathological subjects. These findings provide, for the first time, clear evidence that monitoring HRV via SCG and GCG sensors without concurrent ECG is feasible with the NCC-based template matching method for heartbeat detection.
Collapse
Affiliation(s)
| | - Jessica Centracchio
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy; (S.P.); (D.E.); (P.B.)
| | | | | | - Emilio Andreozzi
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy; (S.P.); (D.E.); (P.B.)
| |
Collapse
|
17
|
Centracchio J, Parlato S, Esposito D, Bifulco P, Andreozzi E. ECG-Free Heartbeat Detection in Seismocardiography Signals via Template Matching. SENSORS (BASEL, SWITZERLAND) 2023; 23:4684. [PMID: 37430606 DOI: 10.3390/s23104684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 05/08/2023] [Accepted: 05/10/2023] [Indexed: 07/12/2023]
Abstract
Cardiac monitoring can be performed by means of an accelerometer attached to a subject's chest, which produces the Seismocardiography (SCG) signal. Detection of SCG heartbeats is commonly carried out by taking advantage of a simultaneous electrocardiogram (ECG). SCG-based long-term monitoring would certainly be less obtrusive and easier to implement without an ECG. Few studies have addressed this issue using a variety of complex approaches. This study proposes a novel approach to ECG-free heartbeat detection in SCG signals via template matching, based on normalized cross-correlation as heartbeats similarity measure. The algorithm was tested on the SCG signals acquired from 77 patients with valvular heart diseases, available from a public database. The performance of the proposed approach was assessed in terms of sensitivity and positive predictive value (PPV) of the heartbeat detection and accuracy of inter-beat intervals measurement. Sensitivity and PPV of 96% and 97%, respectively, were obtained by considering templates that included both systolic and diastolic complexes. Regression, correlation, and Bland-Altman analyses carried out on inter-beat intervals reported slope and intercept of 0.997 and 2.8 ms (R2 > 0.999), as well as non-significant bias and limits of agreement of ±7.8 ms. The results are comparable or superior to those achieved by far more complex algorithms, also based on artificial intelligence. The low computational burden of the proposed approach makes it suitable for direct implementation in wearable devices.
Collapse
Affiliation(s)
- Jessica Centracchio
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy
| | - Salvatore Parlato
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy
| | - Daniele Esposito
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy
| | - Paolo Bifulco
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy
| | - Emilio Andreozzi
- Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy
| |
Collapse
|