1
|
Eniyandunmo D, Shin M, Lee C, Anwar A, Kim E, Kim K, Kim YH, Lee C. Utilising raw psycho-physiological data and functional data analysis for estimating mental workload in human drivers. ERGONOMICS 2024:1-17. [PMID: 39037945 DOI: 10.1080/00140139.2024.2379949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Accepted: 07/08/2024] [Indexed: 07/24/2024]
Abstract
Recent studies have focused on accurately estimating mental workload using machine learning algorithms and extracting features from physiological measures. However, feature extraction leads to the loss of valuable information and often results in binary classifications that lack specificity in the identification of optimum mental workload. This study investigates the feasibility of using raw physiological data (EEG, facial EMG, ECG, EDA, pupillometry) combined with Functional Data Analysis (FDA) to estimate the mental workload of human drivers. A driving scenario with five tasks was employed, and subjective ratings were collected. Results demonstrate that the FDA applied nine different combinations of raw physiological signals achieving a maximum 90% accuracy, outperforming extracted features by 73%. This study shows that the mental workload of human drivers can be accurately estimated without utilising burdensome feature extraction. The approach proposed in this study offers promise for mental workload assessment in real-world applications.
Collapse
Affiliation(s)
- David Eniyandunmo
- Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON, Canada
| | - MinJu Shin
- Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON, Canada
- Department of Statistics, Ewha Womans University, Seoul, Republic of Korea
| | - Chaeyoung Lee
- Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON, Canada
- Department of Statistics, Ewha Womans University, Seoul, Republic of Korea
| | - Alvee Anwar
- Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON, Canada
| | - Eunsik Kim
- Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON, Canada
| | - Kyongwon Kim
- Department of Statistics, Ewha Womans University, Seoul, Republic of Korea
| | - Yong Hoon Kim
- Civil and Environmental Engineering, University of Windsor, Windsor, ON, Canada
| | - Chris Lee
- Civil and Environmental Engineering, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
2
|
Beles H, Vesselenyi T, Rus A, Mitran T, Scurt FB, Tolea BA. Driver Drowsiness Multi-Method Detection for Vehicles with Autonomous Driving Functions. SENSORS (BASEL, SWITZERLAND) 2024; 24:1541. [PMID: 38475079 DOI: 10.3390/s24051541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 02/13/2024] [Accepted: 02/21/2024] [Indexed: 03/14/2024]
Abstract
The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands.
Collapse
Affiliation(s)
- Horia Beles
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| | - Tiberiu Vesselenyi
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| | - Alexandru Rus
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| | - Tudor Mitran
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| | - Florin Bogdan Scurt
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| | - Bogdan Adrian Tolea
- Department of Mechanical Engineering and Automotive, University of Oradea, Universitatii St. 1, 410087 Oradea, Romania
| |
Collapse
|
3
|
Huang J, Zhang Q, Zhang T, Wang T, Tao D. Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios. SENSORS (BASEL, SWITZERLAND) 2024; 24:1041. [PMID: 38339758 PMCID: PMC10857761 DOI: 10.3390/s24031041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/31/2023] [Revised: 01/18/2024] [Accepted: 02/02/2024] [Indexed: 02/12/2024]
Abstract
Assessing drivers' mental workload is crucial for reducing road accidents. This study examined drivers' mental workload in a simulated auditory-based dual-task driving scenario, with driving tasks as the main task, and auditory-based N-back tasks as the secondary task. A total of three levels of mental workload (i.e., low, medium, high) were manipulated by varying the difficulty levels of the secondary task (i.e., no presence of secondary task, 1-back, 2-back). Multimodal measures, including a set of subjective measures, physiological measures, and behavioral performance measures, were collected during the experiment. The results showed that an increase in task difficulty led to increased subjective ratings of mental workload and a decrease in task performance for the secondary N-back tasks. Significant differences were observed across the different levels of mental workload in multimodal physiological measures, such as delta waves in EEG signals, fixation distance in eye movement signals, time- and frequency-domain measures in ECG signals, and skin conductance in EDA signals. In addition, four driving performance measures related to vehicle velocity and the deviation of pedal input and vehicle position also showed sensitivity to the changes in drivers' mental workload. The findings from this study can contribute to a comprehensive understanding of effective measures for mental workload assessment in driving scenarios and to the development of smart driving systems for the accurate recognition of drivers' mental states.
Collapse
Affiliation(s)
- Jiaqi Huang
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen 518060, China; (J.H.)
| | - Qiliang Zhang
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen 518060, China; (J.H.)
- Physical Science and Technology College, Yichun University, Yichun 336000, China
| | - Tingru Zhang
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen 518060, China; (J.H.)
| | - Tieyan Wang
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen 518060, China; (J.H.)
- Xiamen Meiya Pico Information Co., Ltd., Xiamen 361008, China
| | - Da Tao
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen 518060, China; (J.H.)
| |
Collapse
|
4
|
Ma J, Wu Y, Rong J, Zhao X. A systematic review on the influence factors, measurement, and effect of driver workload. ACCIDENT; ANALYSIS AND PREVENTION 2023; 192:107289. [PMID: 37696063 DOI: 10.1016/j.aap.2023.107289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2023] [Revised: 08/21/2023] [Accepted: 09/05/2023] [Indexed: 09/13/2023]
Abstract
Driver workload (DWL) is an important factor that needs to be considered in the study of traffic safety. The research focus on DWL has undergone certain shifts with the rapid development of scientific and technological advancements in the field of transportation in recent years. This study aims to grasp the state of research on DWL by both bibliometric analysis and individual critical literature review. The knowledge structure and development trend are described using bibliometric analysis. The knowledge mapping method is applied to mine the available literature in depth. It is discovered that one of the current research focus on DWL has shifted towards investigating its application in the field of autonomous driving. Subjective questionnaires and experimental tests (including both simulation technology and field study) are the main approaches to analyze DWL. An individual critical literature review of the influencing factors, measurement, and performance of DWL is provided. Research findings have shown that DWL was highly impacted by both intrinsic (e.g., age, temperament, driving experience) and external factors (e.g., vehicles, roads, tasks, environments). Scholars are actively exploring the combined effects of various factors and the level of vehicle automation on DWL. In addition to assess DWL by using subjective measures or physiological parameter measures separately, studies have started to improve classification accuracy by combining multiple measurement methods. Safety thresholds of DWL are not sufficiently studied due to the various interference items corresponding to different scenarios, but it is expected to quantify the DWL and find the threshold by establishing assessment models considering these intrinsic and external multiple-factors simultaneously. Driver or vehicle performance indicators are controversial to measure DWL directly, but they were suitable to reflect the impact of DWL in different driving conditions.
Collapse
Affiliation(s)
- Jun Ma
- Beijing Key Laboratory of Traffic Engineering, Beijing University of Technology, Beijing, China
| | - Yiping Wu
- Beijing Key Laboratory of Traffic Engineering, Beijing University of Technology, Beijing, China.
| | - Jian Rong
- School of Civil Engineering, Guangzhou University, Guangzhou, China
| | - Xiaohua Zhao
- Beijing Key Laboratory of Traffic Engineering, Beijing University of Technology, Beijing, China
| |
Collapse
|
5
|
Russo S, Lorusso L, D’Onofrio G, Ciccone F, Tritto M, Nocco S, Cardone D, Perpetuini D, Lombardo M, Lombardo D, Sancarlo D, Greco A, Merla A, Giuliani F. Assessing Feasibility of Cognitive Impairment Testing Using Social Robotic Technology Augmented with Affective Computing and Emotional State Detection Systems. Biomimetics (Basel) 2023; 8:475. [PMID: 37887606 PMCID: PMC10604561 DOI: 10.3390/biomimetics8060475] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 09/26/2023] [Accepted: 09/27/2023] [Indexed: 10/28/2023] Open
Abstract
Social robots represent a valid opportunity to manage the diagnosis, treatment, care, and support of older people with dementia. The aim of this study is to validate the Mini-Mental State Examination (MMSE) test administered by the Pepper robot equipped with systems to detect psychophysical and emotional states in older patients. Our main result is that the Pepper robot is capable of administering the MMSE and that cognitive status is not a determinant in the effective use of a social robot. People with mild cognitive impairment appreciate the robot, as it interacts with them. Acceptability does not relate strictly to the user experience, but the willingness to interact with the robot is an important variable for engagement. We demonstrate the feasibility of a novel approach that, in the future, could lead to more natural human-machine interaction when delivering cognitive tests with the aid of a social robot and a Computational Psychophysiology Module (CPM).
Collapse
Affiliation(s)
- Sergio Russo
- Research & Innovation Unit, Foundation IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy;
| | - Letizia Lorusso
- Research & Innovation Unit, Foundation IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy;
- Interdisciplinary Department of Medicine, School of Medical Statistics and Biometry, University of Bari Aldo Moro, 70124 Bari, Italy
| | - Grazia D’Onofrio
- Clinical Psychology Service, Health Department, IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy; (G.D.); (F.C.)
| | - Filomena Ciccone
- Clinical Psychology Service, Health Department, IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy; (G.D.); (F.C.)
| | - Michele Tritto
- Next2U Srl, Via dei Peligni 137, 65127 Pescara, Italy; (M.T.); (S.N.)
| | - Sergio Nocco
- Next2U Srl, Via dei Peligni 137, 65127 Pescara, Italy; (M.T.); (S.N.)
| | - Daniela Cardone
- Department of Engineering and Geology, University G. D’Annunzio of Chieti-Pescara, 65127 Pescara, Italy; (D.C.); (D.P.); (A.M.)
| | - David Perpetuini
- Department of Engineering and Geology, University G. D’Annunzio of Chieti-Pescara, 65127 Pescara, Italy; (D.C.); (D.P.); (A.M.)
| | - Marco Lombardo
- Behaviour Labs S.r.l.s. Piazza Gen. di Brigata Luigi Sapienza 22, 95030 Sant’Agata Li Battiati, Italy (D.L.)
| | - Daniele Lombardo
- Behaviour Labs S.r.l.s. Piazza Gen. di Brigata Luigi Sapienza 22, 95030 Sant’Agata Li Battiati, Italy (D.L.)
| | - Daniele Sancarlo
- Geriatrics Unit, Foundation IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy; (D.S.); (A.G.)
| | - Antonio Greco
- Geriatrics Unit, Foundation IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy; (D.S.); (A.G.)
| | - Arcangelo Merla
- Department of Engineering and Geology, University G. D’Annunzio of Chieti-Pescara, 65127 Pescara, Italy; (D.C.); (D.P.); (A.M.)
| | - Francesco Giuliani
- Research & Innovation Unit, Foundation IRCCS Casa Sollievo della Sofferenza, 71013 San Giovanni Rotondo, Italy;
| |
Collapse
|
6
|
Liu X, Shi L, Ye C, Li Y, Wang J. Research on Mental Workload of Deep-Sea Oceanauts Driving Operation Tasks from EEG Data. Bioengineering (Basel) 2023; 10:1027. [PMID: 37760129 PMCID: PMC10525619 DOI: 10.3390/bioengineering10091027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Revised: 06/30/2023] [Accepted: 08/11/2023] [Indexed: 09/29/2023] Open
Abstract
A person's present mental state is closely associated with the frequency and temporal domain features of spontaneous electroencephalogram (EEG) impulses, which directly reflect neurophysiological signals of brain activity. EEG signals are employed in this study to measure the mental workload of drivers while they are operating a vehicle. A technique based on the quantum genetic algorithm (QGA) is suggested for improving the kernel function parameters of the multi-class support vector machine (MSVM). The performance of the algorithm based on the quantum genetic algorithm is found to be superior to that of other ways when other methods and the quantum genetic algorithm are evaluated for the parameter optimization of kernel function via simulation. A multi-classification support vector machine based on the quantum genetic algorithm (QGA-MSVM) is applied to identify the mental workload of oceanauts through the collection and feature extraction of EEG signals during driving simulation operation experiments in a sea basin area, a seamount area, and a hydrothermal area. Even with a limited data set, QGA-MSVM is able to accurately identify the cognitive burden experienced by ocean sailors, with an overall accuracy of 91.8%.
Collapse
Affiliation(s)
- Xiaoguang Liu
- School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, China;
- Shanghai Jiao Tong University and Chiba University International Cooperative Research Center (SJTC-CU-ICRC), Shanghai 200231, China
| | - Lu Shi
- Shanghai Jiao Tong University and Chiba University International Cooperative Research Center (SJTC-CU-ICRC), Shanghai 200231, China
- Institute of Underwater Technology, Shanghai Jiao Tong University, Shanghai 200231, China (J.W.)
| | - Cong Ye
- China Ship Scientific Research Center, Wuxi 214028, China;
| | - Yangyang Li
- Institute of Underwater Technology, Shanghai Jiao Tong University, Shanghai 200231, China (J.W.)
| | - Jing Wang
- Institute of Underwater Technology, Shanghai Jiao Tong University, Shanghai 200231, China (J.W.)
| |
Collapse
|
7
|
Perpetuini D, Formenti D, Priego-Quesada JI, Merla A. Editorial: Effect of neurophysiological conditions and mental workload on physical and cognitive performances: a multidimensional perspective. FRONTIERS IN NEUROERGONOMICS 2023; 4:1215376. [PMID: 38234485 PMCID: PMC10790942 DOI: 10.3389/fnrgo.2023.1215376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 05/05/2023] [Indexed: 01/19/2024]
Affiliation(s)
- David Perpetuini
- Department of Neuroscience, Imaging and Clinical Sciences, University of Studies G. d'Annunzio University of Chieti and Pescara, Chieti, Italy
| | - Damiano Formenti
- Department of Biotechnology and Life Sciences, University of Insubria, Varese, Italy
| | - Jose Ignacio Priego-Quesada
- Department of Physical Education and Sports, Faculty of Physical Activity and Sport Sciences, Universitat de València, Valencia, Spain
| | - Arcangelo Merla
- Department of Engineering and Geology, G. d'Annunzio University of Chieti and Pescara, Pescara, Italy
| |
Collapse
|
8
|
Ju H, Jaffrezic-Renault N. Special Issue "Feature Papers in Biosensors Section 2022". SENSORS (BASEL, SWITZERLAND) 2023; 23:3704. [PMID: 37050763 PMCID: PMC10099281 DOI: 10.3390/s23073704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 03/29/2023] [Indexed: 06/19/2023]
Abstract
Biosensors are devices composed of a biorecognition part and of a transduction part [...].
Collapse
Affiliation(s)
- Huangxian Ju
- State Key Laboratory of Analytical Chemistry for Life Science, School of Chemistry and Chemical Engineering, Nanjing University, Nanjing 210023, China
| | - Nicole Jaffrezic-Renault
- Institute of Analytical Sciences, University of Lyon, UMR CNRS 5280, 5 Rue de La Doua, 69100 Villeurbanne, France
| |
Collapse
|
9
|
Sriranga AK, Lu Q, Birrell S. A Systematic Review of In-Vehicle Physiological Indices and Sensor Technology for Driver Mental Workload Monitoring. SENSORS (BASEL, SWITZERLAND) 2023; 23:2214. [PMID: 36850812 PMCID: PMC9963326 DOI: 10.3390/s23042214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 02/06/2023] [Accepted: 02/13/2023] [Indexed: 06/18/2023]
Abstract
The concept of vehicle automation ceases to seem futuristic with the current advancement of the automotive industry. With the introduction of conditional automated vehicles, drivers are no longer expected to focus only on driving activities but are still required to stay alert to resume control. However, fluctuations in driving demands are known to alter the driver's mental workload (MWL), which might affect the driver's vehicle take-over capabilities. Driver mental workload can be specified as the driver's capacity for information processing for task performance. This paper summarizes the literature that relates to analysing driver mental workload through various in-vehicle physiological sensors focusing on cardiovascular and respiratory measures. The review highlights the type of study, hardware, method of analysis, test variable, and results of studies that have used physiological indices for MWL analysis in the automotive context.
Collapse
|
10
|
Perpetuini D, Russo EF, Cardone D, Palmieri R, Filippini C, Tritto M, Pellicano F, De Santis GP, Pellegrino R, Calabrò RS, Filoni S, Merla A. Psychophysiological Assessment of Children with Cerebral Palsy during Robotic-Assisted Gait Training through Infrared Imaging. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:15224. [PMID: 36429941 PMCID: PMC9690262 DOI: 10.3390/ijerph192215224] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 11/09/2022] [Accepted: 11/16/2022] [Indexed: 06/16/2023]
Abstract
Cerebral palsy (CP) is a non-progressive neurologic pathology representing a leading cause of spasticity and concerning gait impairments in children. Robotic-assisted gait training (RAGT) is widely employed to treat this pathology to improve children's gait pattern. Importantly, the effectiveness of the therapy is strictly related to the engagement of the patient in the rehabilitation process, which depends on his/her psychophysiological state. The aim of the study is to evaluate the psychophysiological condition of children with CP during RAGT through infrared thermography (IRT), which was acquired during three sessions in one month. A repeated measure ANOVA was performed (i.e., mean value, standard deviation, and sample entropy) extracted from the temperature time course collected over the nose and corrugator, which are known to be indicative of the psychophysiological state of the individual. Concerning the corrugator, significant differences were found for the sample entropy (F (1.477, 5.907) = 6.888; p = 0.033) and for the mean value (F (1.425, 5.7) = 5.88; p = 0.047). Regarding the nose tip, the sample entropy showed significant differences (F (1.134, 4.536) = 11.5; p = 0.041). The findings from this study suggests that this approach can be used to evaluate in a contactless manner the psychophysiological condition of the children with CP during RAGT, allowing to monitor their engagement to the therapy, increasing the benefits of the treatment.
Collapse
Affiliation(s)
- David Perpetuini
- Department of Neuroscience and Imaging, University G. D’Annunzio of Chieti-Pescara, 66100 Chieti, Italy
| | | | - Daniela Cardone
- Department of Engineering and Geology, University G. D’Annunzio of Chieti-Pescara, 65127 Pescara, Italy
| | - Roberta Palmieri
- Department of Basic Medical Sciences, Neurosciences and Sense Organs, Institute of Child Neuropsychiatry, University of Bari, 70121 Bari, Italy
| | - Chiara Filippini
- Department of Neuroscience and Imaging, University G. D’Annunzio of Chieti-Pescara, 66100 Chieti, Italy
| | | | - Federica Pellicano
- Padre Pio Foundation and Rehabilitation Centers, 71013 San Giovanni Rotondo, Italy
| | - Grazia Pia De Santis
- Padre Pio Foundation and Rehabilitation Centers, 71013 San Giovanni Rotondo, Italy
| | - Raffaello Pellegrino
- Department of Scientific Research, Campus Ludes, Off-Campus Semmelweis University, 6912 Lugano, Switzerland
| | | | - Serena Filoni
- Padre Pio Foundation and Rehabilitation Centers, 71013 San Giovanni Rotondo, Italy
| | - Arcangelo Merla
- Department of Engineering and Geology, University G. D’Annunzio of Chieti-Pescara, 65127 Pescara, Italy
- ITAB, Institute for Advanced Biomedical Technologies, 66100 Chieti, Italy
| |
Collapse
|