1
|
Zeng H, Zhao Y, Babiloni F, Tao M, Kong W, Dai G. A General DNA-Like Hybrid Symbiosis Framework: An EEG Cognitive Recognition Method. IEEE J Biomed Health Inform 2024; 28:6498-6511. [PMID: 39120983 DOI: 10.1109/jbhi.2024.3441332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/11/2024]
Abstract
In electroencephalogram (EEG) cognitive recognition research, the combined use of artificial neural networks (ANNs) and spiking neural networks (SNNs) plays an important role to realize different categories of recognition tasks. However, most of the existing studies focus on the unidirectional interaction between an ANN and a SNN, which may be overly dependent on the performance of ANNs or SNNs. Inspired by the symbiosis phenomenon in nature, in this study, we propose a general DNA-like Hybrid Symbiosis (DNA-HS) framework, which enables mutual learning between the ANN and the SNN generated by this ANN through parametric genetic algorithm and bidirectional interaction mechanism to enhance the optimization ability of the model parameters, resulting in a significant improvement of the performance of the DNA-HS framework in all aspects. By comparing with seven typical EEG cognitive recognition models, the performance of the seven hybrid network frameworks constructed using this method on different EEG-based cognitive recognition tasks are all improved to different degrees, verifying the effectiveness of the proposed method. This unified hybrid network framework similar to the DNA structure is expected to open up a new approach and form a new research paradigm for EEG-based cognitive recognition task.
Collapse
|
2
|
Borghini G, Ronca V, Giorgi A, Aricò P, Di Flumeri G, Capotorto R, Rooseleer F, Kirwan B, De Visscher I, Goman M, Pugh J, Abramov N, Granger G, Alarcon DPM, Humm E, Pozzi S, Babiloni F. Reducing flight upset risk and startle response: A study of the wake vortex alert with licensed commercial pilots. Brain Res Bull 2024; 215:111020. [PMID: 38909913 DOI: 10.1016/j.brainresbull.2024.111020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 05/24/2024] [Accepted: 06/20/2024] [Indexed: 06/25/2024]
Abstract
The study aimed at investigating the impact of an innovative Wake Vortex Alert (WVA) avionics on pilots' operation and mental states, intending to improve aviation safety by mitigating the risks associated with wake vortex encounters (WVEs). Wake vortices, generated by jet aircraft, pose a significant hazard to trailing or crossing aircrafts. Despite existing separation rules, incidents involving WVEs continue to occur, especially affecting smaller aircrafts like business jets, resulting in aircraft upsets and occasional cabin injuries. To address these challenges, the study focused on developing and validating an alert system that can be presented to air traffic controllers, enabling them to warn flight crews. This empowers the flight crews to either avoid the wake vortex or secure the cabin to prevent injuries. The research employed a multidimensional approach including an analysis of human performance and human factors (HF) issues to determine the potential impact of the alert on pilots' roles, tasks, and mental states. It also utilizes Human Assurance Levels (HALs) to evaluate the necessary human factors support based on the safety criticality of the new system. Realistic flight simulations were conducted to collect data of pilots' behavioural, subjective and neurophysiological responses during WVEs. The data allowed for an objective evaluation of the WVA impact on pilots' operation, behaviour and mental states (mental workload, stress levels and arousal). In particular, the results highlighted the effectiveness of the alert system in facilitating pilots' preparation, awareness and crew resource management (CRM). The results also highlighted the importance of avionics able to enhance aviation safety and reducing risks associated with wake vortex encounters. In particular, we demonstrated how providing timely information and improving situational awareness, the WVA will minimize the occurrence of WVEs and contribute to safer aviation operations.
Collapse
Affiliation(s)
- Gianluca Borghini
- Department of Molecular Medicine, Sapienza University of Rome, Italy; BrainSigns srl, Rome, Italy.
| | - Vincenzo Ronca
- BrainSigns srl, Rome, Italy; Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Italy
| | - Andrea Giorgi
- BrainSigns srl, Rome, Italy; Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Italy
| | - Pietro Aricò
- BrainSigns srl, Rome, Italy; Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Italy
| | - Gianluca Di Flumeri
- Department of Molecular Medicine, Sapienza University of Rome, Italy; BrainSigns srl, Rome, Italy
| | - Rossella Capotorto
- Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Italy
| | | | - Barry Kirwan
- EUROCONTROL, Centre du Bois des Bordes, Bretigny-sur-Orge, France
| | - Ivan De Visscher
- EUROCONTROL, Centre du Bois des Bordes, Bretigny-sur-Orge, France
| | - Mikhail Goman
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Jonathan Pugh
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Nikolay Abramov
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Géraud Granger
- Safety Management Research Program, École Nationale de l'Aviation Civile (ENAC), France
| | | | | | | | - Fabio Babiloni
- Department of Physiology and Pharmacology "Vittorio Erspamer", Sapienza University of Rome, Italy; School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| |
Collapse
|
3
|
Wang Q, Smythe D, Cao J, Hu Z, Proctor KJ, Owens AP, Zhao Y. Characterisation of Cognitive Load Using Machine Learning Classifiers of Electroencephalogram Data. SENSORS (BASEL, SWITZERLAND) 2023; 23:8528. [PMID: 37896621 PMCID: PMC10611194 DOI: 10.3390/s23208528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 10/09/2023] [Accepted: 10/16/2023] [Indexed: 10/29/2023]
Abstract
A high cognitive load can overload a person, potentially resulting in catastrophic accidents. It is therefore important to ensure the level of cognitive load associated with safety-critical tasks (such as driving a vehicle) remains manageable for drivers, enabling them to respond appropriately to changes in the driving environment. Although electroencephalography (EEG) has attracted significant interest in cognitive load research, few studies have used EEG to investigate cognitive load in the context of driving. This paper presents a feasibility study on the simulation of various levels of cognitive load through designing and implementing four driving tasks. We employ machine learning-based classification techniques using EEG recordings to differentiate driving conditions. An EEG dataset containing these four driving tasks from a group of 20 participants was collected to investigate whether EEG can be used as an indicator of changes in cognitive load. The collected dataset was used to train four Deep Neural Networks and four Support Vector Machine classification models. The results showed that the best model achieved a classification accuracy of 90.37%, utilising statistical features from multiple frequency bands in 24 EEG channels. Furthermore, the Gamma and Beta bands achieved higher classification accuracy than the Alpha and Theta bands during the analysis. The outcomes of this study have the potential to enhance the Human-Machine Interface of vehicles, contributing to improved safety.
Collapse
Affiliation(s)
- Qi Wang
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Daniel Smythe
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Jun Cao
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Zhilin Hu
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Karl J. Proctor
- Jaguar Land Rover Research, Coventry CV4 7AL, UK; (K.J.P.); (A.P.O.)
| | - Andrew P. Owens
- Jaguar Land Rover Research, Coventry CV4 7AL, UK; (K.J.P.); (A.P.O.)
| | - Yifan Zhao
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| |
Collapse
|
4
|
A new common spatial pattern-based unified channels algorithm for driver’s fatigue EEG signals classification. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-07833-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
5
|
Inguscio BMS, Mancini P, Greco A, Nicastri M, Giallini I, Leone CA, Grassia R, Di Nardo W, Di Cesare T, Rossi F, Canale A, Albera A, Giorgi A, Malerba P, Babiloni F, Cartocci G. ‘Musical effort’ and ‘musical pleasantness’: a pilot study on the neurophysiological correlates of classical music listening in adults normal hearing and unilateral cochlear implant users. HEARING, BALANCE AND COMMUNICATION 2022. [DOI: 10.1080/21695717.2022.2079325] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
| | - Patrizia Mancini
- Department of Sense Organs, Sapienza University of Rome, Rome, Italy
| | - Antonio Greco
- Department of Sense Organs, Sapienza University of Rome, Rome, Italy
| | - Maria Nicastri
- Department of Sense Organs, Sapienza University of Rome, Rome, Italy
| | - Ilaria Giallini
- Department of Sense Organs, Sapienza University of Rome, Rome, Italy
| | - Carlo Antonio Leone
- Department of Otolaryngology/Head and Neck Surgery, Monaldi Hospital, Naples, Italy
| | - Rosa Grassia
- Department of Otolaryngology/Head and Neck Surgery, Monaldi Hospital, Naples, Italy
| | - Walter Di Nardo
- Otorhinolaryngology and Physiology, Catholic University of Rome, Rome, Italy
| | - Tiziana Di Cesare
- Otorhinolaryngology and Physiology, Catholic University of Rome, Rome, Italy
| | - Federica Rossi
- Otorhinolaryngology and Physiology, Catholic University of Rome, Rome, Italy
| | - Andrea Canale
- Division of Otorhinolaryngology, Department of Surgical Sciences, University of Turin, Italy
| | - Andrea Albera
- Division of Otorhinolaryngology, Department of Surgical Sciences, University of Turin, Italy
| | | | | | - Fabio Babiloni
- BrainSigns Srl, Rome, Italy
- Department of Computer Science, Hangzhou Dianzi University, Hangzhou, China
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Giulia Cartocci
- BrainSigns Srl, Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
6
|
Li G, Chung WY. Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review. SENSORS 2022; 22:s22031100. [PMID: 35161844 PMCID: PMC8840041 DOI: 10.3390/s22031100] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Revised: 01/15/2022] [Accepted: 01/28/2022] [Indexed: 02/06/2023]
Abstract
Drowsiness is not only a core challenge to safe driving in traditional driving conditions but also a serious obstacle for the wide acceptance of added services of self-driving cars (because drowsiness is, in fact, one of the most representative early-stage symptoms of self-driving carsickness). In view of the importance of detecting drivers’ drowsiness, this paper reviews the algorithms of electroencephalogram (EEG)-based drivers’ drowsiness detection (DDD). To facilitate the review, the EEG-based DDD approaches are organized into a tree structure taxonomy, having two main categories, namely “detection only (open-loop)” and “management (closed-loop)”, both aimed at designing better DDD systems that ensure early detection, reliability and practical utility. To achieve this goal, we addressed seven questions, the answers of which helped in developing an EEG-based DDD system that is superior to the existing ones. A basic assumption in this review article is that although driver drowsiness and carsickness-induced drowsiness are caused by different factors, the brain network that regulates drowsiness is the same.
Collapse
Affiliation(s)
| | - Wan-Young Chung
- Correspondence: ; Tel.: +82-10-629-6223; Fax: +82-10-629-6210
| |
Collapse
|
7
|
Zhao Y, Dai G, Borghini G, Zhang J, Li X, Zhang Z, Aricò P, Di Flumeri G, Babiloni F, Zeng H. Label-Based Alignment Multi-Source Domain Adaptation for Cross-Subject EEG Fatigue Mental State Evaluation. Front Hum Neurosci 2021; 15:706270. [PMID: 34658814 PMCID: PMC8519604 DOI: 10.3389/fnhum.2021.706270] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Accepted: 08/27/2021] [Indexed: 11/13/2022] Open
Abstract
Accurate detection of driving fatigue is helpful in significantly reducing the rate of road traffic accidents. Electroencephalogram (EEG) based methods are proven to be efficient to evaluate mental fatigue. Due to its high non-linearity, as well as significant individual differences, how to perform EEG fatigue mental state evaluation across different subjects still keeps challenging. In this study, we propose a Label-based Alignment Multi-Source Domain Adaptation (LA-MSDA) for cross-subject EEG fatigue mental state evaluation. Specifically, LA-MSDA considers the local feature distributions of relevant labels between different domains, which efficiently eliminates the negative impact of significant individual differences by aligning label-based feature distributions. In addition, the strategy of global optimization is introduced to address the classifier confusion decision boundary issues and improve the generalization ability of LA-MSDA. Experimental results show LA-MSDA can achieve remarkable results on EEG-based fatigue mental state evaluation across subjects, which is expected to have wide application prospects in practical brain-computer interaction (BCI), such as online monitoring of driver fatigue, or assisting in the development of on-board safety systems.
Collapse
Affiliation(s)
- Yue Zhao
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Guojun Dai
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Gianluca Borghini
- Industrial NeuroScience Lab, University of Rome "La Sapienza", Rome, Italy
| | - Jiaming Zhang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Xiufeng Li
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Zhenyan Zhang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Pietro Aricò
- Industrial NeuroScience Lab, University of Rome "La Sapienza", Rome, Italy
| | | | - Fabio Babiloni
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China.,Industrial NeuroScience Lab, University of Rome "La Sapienza", Rome, Italy
| | - Hong Zeng
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China.,Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou, China
| |
Collapse
|
8
|
An EEG-Based Transfer Learning Method for Cross-Subject Fatigue Mental State Prediction. SENSORS 2021; 21:s21072369. [PMID: 33805522 PMCID: PMC8036954 DOI: 10.3390/s21072369] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Revised: 03/22/2021] [Accepted: 03/24/2021] [Indexed: 01/26/2023]
Abstract
Fatigued driving is one of the main causes of traffic accidents. The electroencephalogram (EEG)-based mental state analysis method is an effective and objective way of detecting fatigue. However, as EEG shows significant differences across subjects, effectively “transfering” the EEG analysis model of the existing subjects to the EEG signals of other subjects is still a challenge. Domain-Adversarial Neural Network (DANN) has excellent performance in transfer learning, especially in the fields of document analysis and image recognition, but has not been applied directly in EEG-based cross-subject fatigue detection. In this paper, we present a DANN-based model, Generative-DANN (GDANN), which combines Generative Adversarial Networks (GAN) to enhance the ability by addressing the issue of different distribution of EEG across subjects. The comparative results show that in the analysis of cross-subject tasks, GDANN has a higher average accuracy of 91.63% in fatigue detection across subjects than those of traditional classification models, which is expected to have much broader application prospects in practical brain–computer interaction (BCI).
Collapse
|
9
|
Zeng H, Zhang J, Zakaria W, Babiloni F, Gianluca B, Li X, Kong W. InstanceEasyTL: An Improved Transfer-Learning Method for EEG-Based Cross-Subject Fatigue Detection. SENSORS 2020; 20:s20247251. [PMID: 33348823 PMCID: PMC7766235 DOI: 10.3390/s20247251] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Revised: 12/14/2020] [Accepted: 12/15/2020] [Indexed: 11/22/2022]
Abstract
Electroencephalogram (EEG) is an effective indicator for the detection of driver fatigue. Due to the significant differences in EEG signals across subjects, and difficulty in collecting sufficient EEG samples for analysis during driving, detecting fatigue across subjects through using EEG signals remains a challenge. EasyTL is a kind of transfer-learning model, which has demonstrated better performance in the field of image recognition, but not yet been applied in cross-subject EEG-based applications. In this paper, we propose an improved EasyTL-based classifier, the InstanceEasyTL, to perform EEG-based analysis for cross-subject fatigue mental-state detection. Experimental results show that InstanceEasyTL not only requires less EEG data, but also obtains better performance in accuracy and robustness than EasyTL, as well as existing machine-learning models such as Support Vector Machine (SVM), Transfer Component Analysis (TCA), Geodesic Flow Kernel (GFK), and Domain-adversarial Neural Networks (DANN), etc.
Collapse
Affiliation(s)
- Hong Zeng
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.Z.); (J.Z.); (X.L.); (W.K.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Jiaming Zhang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.Z.); (J.Z.); (X.L.); (W.K.)
| | - Wael Zakaria
- Department of Mathematics-Computer Science, Faculty of Science, Ain Shams University, Abbassia, Cairo 11435, Egypt;
| | - Fabio Babiloni
- Industrial NeuroScience Lab, University of Rome “La Sapienza”, 00161 Rome, Italy;
- Correspondence:
| | - Borghini Gianluca
- Industrial NeuroScience Lab, University of Rome “La Sapienza”, 00161 Rome, Italy;
| | - Xiufeng Li
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.Z.); (J.Z.); (X.L.); (W.K.)
| | - Wanzeng Kong
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.Z.); (J.Z.); (X.L.); (W.K.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| |
Collapse
|
10
|
Kaduk SI, Roberts APJ, Stanton NA. The circadian effect on psychophysiological driver state monitoring. THEORETICAL ISSUES IN ERGONOMICS SCIENCE 2020. [DOI: 10.1080/1463922x.2020.1842548] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Sylwia I. Kaduk
- Human Factors Engineering, Transportation Research Group, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton, United Kingdom
| | - Aaron P. J. Roberts
- Human Factors Engineering, Transportation Research Group, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton, United Kingdom
| | - Neville A. Stanton
- Human Factors Engineering, Transportation Research Group, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton, United Kingdom
| |
Collapse
|
11
|
Monteiro TG, Li G, Skourup C, Zhang H. Investigating an Integrated Sensor Fusion System for Mental Fatigue Assessment for Demanding Maritime Operations. SENSORS (BASEL, SWITZERLAND) 2020; 20:E2588. [PMID: 32370110 PMCID: PMC7248988 DOI: 10.3390/s20092588] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/20/2020] [Revised: 04/24/2020] [Accepted: 04/30/2020] [Indexed: 11/17/2022]
Abstract
Human-related issues are currently the most significant factor in maritime causalities, especially in demanding operations that require coordination between two or more vessels and/or other maritime structures. Some of these human-related issues include incorrect, incomplete, or nonexistent following of procedures; lack of situational awareness; and physical or mental fatigue. Among these, mental fatigue is especially dangerous, due to its capacity to reduce reaction time, interfere in the decision-making process, and affect situational awareness. Mental fatigue is also especially hard to identify and quantify. Self-assessment of mental fatigue may not be reliable and few studies have assessed mental fatigue in maritime operations, especially in real time. In this work we propose an integrated sensor fusion system for mental fatigue assessment using physiological sensors and convolutional neural networks. We show, by using a simulated navigation experiment, how data from different sensors can be fused into a robust mental fatigue assessment tool, capable of achieving up to 100 % detection accuracy for single-subject classification. Additionally, the use of different sensors seems to favor the representation of the transition between mental fatigue states.
Collapse
Affiliation(s)
- Thiago Gabriel Monteiro
- Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, 6009 Ålesund, Norway; (T.G.M.); (G.L.)
| | - Guoyuan Li
- Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, 6009 Ålesund, Norway; (T.G.M.); (G.L.)
| | - Charlotte Skourup
- Products and Services R&D, Oil, Gas and Chemicals, ABB AS, 0666 Oslo, Norway;
| | - Houxiang Zhang
- Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology, 6009 Ålesund, Norway; (T.G.M.); (G.L.)
| |
Collapse
|
12
|
Zokaei M, Jafari MJ, Khosrowabadi R, Nahvi A, Khodakarim S, Pouyakian M. Tracing the physiological response and behavioral performance of drivers at different levels of mental workload using driving simulators. JOURNAL OF SAFETY RESEARCH 2020; 72:213-223. [PMID: 32199566 DOI: 10.1016/j.jsr.2019.12.022] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2018] [Revised: 08/10/2019] [Accepted: 12/26/2019] [Indexed: 05/27/2023]
Abstract
INTRODUCTION The use of mobile phones while driving is known to be a distraction factor and a cause of accidents. The way in which different kinds of conversations affect the behavioral performance of the driver as well as the persistence of the effects are not yet fully understood. METHOD In this study, in addition to comparing brain function and behavioral function in dual task conditions in three conversations types, the persistent effects of these types of conversations have also been traced. RESULTS The results show that the content of the mobile phone conversation while driving is the cause of the persistent changes in behavioral and brain functions. Increased time headway and lane departure was observed during and up to 5 min after the emotional conversation was finished. EEG bands also varied in different types of conversations. Cognitive conversations caused an increase in the activity of the alpha and beta bands while emotional conversations enhanced the rate of gamma and beta bands. A meaningful correlation was found between changes in the theta and alpha bands and changes in behavioral performance both during the dual task condition and after the conversation was finished, was also observed. CONCLUSIONS The content of the conversation is one of the most important factors that increase the risk of road accidents. This can also deteriorate the behavioral performance of the driver and can have persistent effects on behavioral performance and the brain. Practical applications: The findings of this study provide a basis to measure and tracing drivers' cognitive distractions induced by different levels of mental workload through physiological and behavioral performances.
Collapse
Affiliation(s)
- Mojtaba Zokaei
- Department of Occupational Health Engineering, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mohammad Javad Jafari
- Department of Occupational Health Engineering, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Reza Khosrowabadi
- Institute for Cognitive and Brain Science, Shahid Beheshti University GC, Tehran, Iran
| | - Ali Nahvi
- Department of Mechanical Engineering, K.N. Toosi University of Technology, Tehran, Iran
| | - Sohila Khodakarim
- Department of Epidemiology, School of Allied Medical Sciences, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mostafa Pouyakian
- Department of Occupational Health Engineering, School of Public Health and Safety, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
13
|
Cherubino P, Martinez-Levy AC, Caratù M, Cartocci G, Di Flumeri G, Modica E, Rossi D, Mancini M, Trettel A. Consumer Behaviour through the Eyes of Neurophysiological Measures: State-of-the-Art and Future Trends. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:1976847. [PMID: 31641346 PMCID: PMC6766676 DOI: 10.1155/2019/1976847] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/15/2019] [Accepted: 07/31/2019] [Indexed: 01/08/2023]
Abstract
The new technological advances achieved during the last decade allowed the scientific community to investigate and employ neurophysiological measures not only for research purposes but also for the study of human behaviour in real and daily life situations. The aim of this review is to understand how and whether neuroscientific technologies can be effectively employed to better understand the human behaviour in real decision-making contexts. To do so, firstly, we will describe the historical development of neuromarketing and its main applications in assessing the sensory perceptions of some marketing and advertising stimuli. Then, we will describe the main neuroscientific tools available for such kind of investigations (e.g., measuring the cerebral electrical or hemodynamic activity, the eye movements, and the psychometric responses). Also, this review will present different brain measurement techniques, along with their pros and cons, and the main cerebral indexes linked to the specific mental states of interest (used in most of the neuromarketing research). Such indexes have been supported by adequate validations from the scientific community and are largely employed in neuromarketing research. This review will also discuss a series of papers that present different neuromarketing applications, such us in-store choices and retail, services, pricing, brand perception, web usability, neuropolitics, evaluation of the food and wine taste, and aesthetic perception of artworks. Furthermore, this work will face the ethical issues arisen on the use of these tools for the evaluation of the human behaviour during decision-making tasks. In conclusion, the main challenges that neuromarketing is going to face, as well as future directions and possible scenarios that could be derived by the use of neuroscience in the marketing field, will be identified and discussed.
Collapse
Affiliation(s)
- Patrizia Cherubino
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
| | - Ana C. Martinez-Levy
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
- Department of Communication and Social Research, Sapienza University of Rome, Via Salaria, 113, 00198 Rome, Italy
| | - Myriam Caratù
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
- Department of Communication and Social Research, Sapienza University of Rome, Via Salaria, 113, 00198 Rome, Italy
| | - Giulia Cartocci
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
| | - Gianluca Di Flumeri
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
| | - Enrica Modica
- Department of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Dario Rossi
- Department of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Marco Mancini
- BrainSigns Srl, Via Sesto Celere 7/c, 00152 Rome, Italy
| | | |
Collapse
|
14
|
A LightGBM-Based EEG Analysis Method for Driver Mental States Classification. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:3761203. [PMID: 31611912 PMCID: PMC6755292 DOI: 10.1155/2019/3761203] [Citation(s) in RCA: 48] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/06/2019] [Revised: 06/29/2019] [Accepted: 07/22/2019] [Indexed: 01/19/2023]
Abstract
Fatigue driving can easily lead to road traffic accidents and bring great harm to individuals and families. Recently, electroencephalography- (EEG-) based physiological and brain activities for fatigue detection have been increasingly investigated. However, how to find an effective method or model to timely and efficiently detect the mental states of drivers still remains a challenge. In this paper, we combine common spatial pattern (CSP) and propose a light-weighted classifier, LightFD, which is based on gradient boosting framework for EEG mental states identification. The comparable results with traditional classifiers, such as support vector machine (SVM), convolutional neural network (CNN), gated recurrent unit (GRU), and large margin nearest neighbor (LMNN), show that the proposed model could achieve better classification performance, as well as the decision efficiency. Furthermore, we also test and validate that LightFD has better transfer learning performance in EEG classification of driver mental states. In summary, our proposed LightFD classifier has better performance in real-time EEG mental state prediction, and it is expected to have broad application prospects in practical brain-computer interaction (BCI).
Collapse
|
15
|
Collet C, Musicant O. Associating Vehicles Automation With Drivers Functional State Assessment Systems: A Challenge for Road Safety in the Future. Front Hum Neurosci 2019; 13:131. [PMID: 31114489 PMCID: PMC6503868 DOI: 10.3389/fnhum.2019.00131] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2018] [Accepted: 04/01/2019] [Indexed: 11/13/2022] Open
Abstract
In the near future, vehicles will gradually gain more autonomous functionalities. Drivers' activity will be less about driving than about monitoring intelligent systems to which driving action will be delegated. Road safety, therefore, remains dependent on the human factor and we should identify the limits beyond which driver's functional state (DFS) may no longer be able to ensure safety. Depending on the level of automation, estimating the DFS may have different targets, e.g., assessing driver's situation awareness in lower levels of automation and his ability to respond to emerging hazard or assessing driver's ability to monitor the vehicle performing operational tasks in higher levels of automation. Unfitted DFS (e.g., drowsiness) may impact the driver ability respond to taking over abilities. This paper reviews the most appropriate psychophysiological indices in naturalistic driving while considering the DFS through exogenous sensors, providing the more efficient trade-off between reliability and intrusiveness. The DFS also originates from kinematic data of the vehicle, thus providing information that indirectly relates to drivers behavior. The whole data should be synchronously processed, providing a diagnosis on the DFS, and bringing it to the attention of the decision maker in real time. Next, making the information available can be permanent or intermittent (or even undelivered), and may also depend on the automation level. Such interface can include recommendations for decision support or simply give neutral instruction. Mapping of relevant psychophysiological and behavioral indicators for DFS will enable practitioners and researchers provide reliable estimates, fitted to the level of automation.
Collapse
Affiliation(s)
- Christian Collet
- Inter-University Laboratory of Human Movement Biology (EA 7424), Univ Lyon, Université Claude Bernard Lyon 1, Villeurbanne, France
| | - Oren Musicant
- Department of Industrial Engineering and Management, Ariel University, Ariel, Israel
| |
Collapse
|
16
|
Antismoking Campaigns' Perception and Gender Differences: A Comparison among EEG Indices. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:7348795. [PMID: 31143204 PMCID: PMC6501276 DOI: 10.1155/2019/7348795] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Accepted: 03/24/2019] [Indexed: 12/22/2022]
Abstract
Human factors' aim is to understand and evaluate the interactions between people and tasks, technologies, and environment. Among human factors, it is possible then to include the subjective reaction to external stimuli, due to individual's characteristics and states of mind. These processes are also involved in the perception of antismoking public service announcements (PSAs), the main tool for governments to contrast the first cause of preventable deaths in the world: tobacco addiction. In the light of that, in the present article, it has been investigated through the comparison of different electroencephalographic (EEG) indices a typical item known to be able of influencing PSA perception, that is gender. In order to investigate the neurophysiological underpinnings of such different perception, we tested two PSAs: one with a female character and one with a male character. Furthermore, the experimental sample was divided into men and women, as well as smokers and nonsmokers. The employed EEG indices were the mental engagement (ME: the ratio between beta activity and the sum of alpha and theta activity); the approach/withdrawal (AW: the frontal alpha asymmetry in the alpha band); and the frontal theta activity and the spectral asymmetry index (SASI: the ratio between beta minus theta and beta plus theta). Results suggested that the ME and the AW presented an opposite trend, with smokers showing higher ME and lower AW than nonsmokers. The ME and the frontal theta also evidenced a statistically significant interaction between the kind of the PSA and the gender of the observers; specifically, women showed higher ME and frontal theta activity for the male character PSA. This study then supports the usefulness of the ME and frontal theta for purposes of PSAs targeting on the basis of gender issues and of the ME and the AW and for purposes of PSAs targeting on the basis of smoking habits.
Collapse
|
17
|
Di Flumeri G, Borghini G, Aricò P, Sciaraffa N, Lanzi P, Pozzi S, Vignali V, Lantieri C, Bichicchi A, Simone A, Babiloni F. EEG-Based Mental Workload Neurometric to Evaluate the Impact of Different Traffic and Road Conditions in Real Driving Settings. Front Hum Neurosci 2018; 12:509. [PMID: 30618686 PMCID: PMC6305466 DOI: 10.3389/fnhum.2018.00509] [Citation(s) in RCA: 60] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2018] [Accepted: 12/05/2018] [Indexed: 12/02/2022] Open
Abstract
Car driving is considered a very complex activity, consisting of different concomitant tasks and subtasks, thus it is crucial to understand the impact of different factors, such as road complexity, traffic, dashboard devices, and external events on the driver's behavior and performance. For this reason, in particular situations the cognitive demand experienced by the driver could be very high, inducing an excessive experienced mental workload and consequently an increasing of error commission probability. In this regard, it has been demonstrated that human error is the main cause of the 57% of road accidents and a contributing factor in most of them. In this study, 20 young subjects have been involved in a real driving experiment, performed under different traffic conditions (rush hour and not) and along different road types (main and secondary streets). Moreover, during the driving tasks different specific events, in particular a pedestrian crossing the road and a car entering the traffic flow just ahead of the experimental subject, have been acted. A Workload Index based on the Electroencephalographic (EEG), i.e., brain activity, of the drivers has been employed to investigate the impact of the different factors on the driver's workload. Eye-Tracking (ET) technology and subjective measures have also been employed in order to have a comprehensive overview of the driver's perceived workload and to investigate the different insights obtainable from the employed methodologies. The employment of such EEG-based Workload index confirmed the significant impact of both traffic and road types on the drivers' behavior (increasing their workload), with the advantage of being under real settings. Also, it allowed to highlight the increased workload related to external events while driving, in particular with a significant effect during those situations when the traffic was low. Finally, the comparison between methodologies revealed the higher sensitivity of neurophysiological measures with respect to ET and subjective ones. In conclusion, such an EEG-based Workload index would allow to assess objectively the mental workload experienced by the driver, standing out as a powerful tool for research aimed to investigate drivers' behavior and providing additional and complementary insights with respect to traditional methodologies employed within road safety research.
Collapse
Affiliation(s)
- Gianluca Di Flumeri
- BrainSigns srl, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Gianluca Borghini
- BrainSigns srl, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Pietro Aricò
- BrainSigns srl, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Nicolina Sciaraffa
- BrainSigns srl, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Rome, Italy
- Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Rome, Italy
| | | | | | - Valeria Vignali
- Department of Civil, Chemical, Environmental and Materials Engineering (DICAM), School of Engineering and Architecture, University of Bologna, Bologna, Italy
| | - Claudio Lantieri
- Department of Civil, Chemical, Environmental and Materials Engineering (DICAM), School of Engineering and Architecture, University of Bologna, Bologna, Italy
| | - Arianna Bichicchi
- Department of Civil, Chemical, Environmental and Materials Engineering (DICAM), School of Engineering and Architecture, University of Bologna, Bologna, Italy
| | - Andrea Simone
- Department of Civil, Chemical, Environmental and Materials Engineering (DICAM), School of Engineering and Architecture, University of Bologna, Bologna, Italy
| | - Fabio Babiloni
- BrainSigns srl, Rome, Italy
- Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
- Department of Computer Science, Hangzhou Dianzi University, Hangzhou, China
| |
Collapse
|
18
|
Neurophysiological Profile of Antismoking Campaigns. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2018; 2018:9721561. [PMID: 30327667 PMCID: PMC6169221 DOI: 10.1155/2018/9721561] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/09/2018] [Accepted: 08/15/2018] [Indexed: 12/18/2022]
Abstract
Over the past few decades, antismoking public service announcements (PSAs) have been used by governments to promote healthy behaviours in citizens, for instance, against drinking before the drive and against smoke. Effectiveness of such PSAs has been suggested especially for young persons. By now, PSAs efficacy is still mainly assessed through traditional methods (questionnaires and metrics) and could be performed only after the PSAs broadcasting, leading to waste of economic resources and time in the case of Ineffective PSAs. One possible countermeasure to such ineffective use of PSAs could be promoted by the evaluation of the cerebral reaction to the PSA of particular segments of population (e.g., old, young, and heavy smokers). In addition, it is crucial to gather such cerebral activity in front of PSAs that have been assessed to be effective against smoke (Effective PSAs), comparing results to the cerebral reactions to PSAs that have been certified to be not effective (Ineffective PSAs). The eventual differences between the cerebral responses toward the two PSA groups will provide crucial information about the possible outcome of new PSAs before to its broadcasting. This study focused on adult population, by investigating the cerebral reaction to the vision of different PSA images, which have already been shown to be Effective and Ineffective for the promotion of an antismoking behaviour. Results showed how variables as gender and smoking habits can influence the perception of PSA images, and how different communication styles of the antismoking campaigns could facilitate the comprehension of PSA's message and then enhance the related impact.
Collapse
|
19
|
Sciaraffa N, Borghini G, Arico P, Di Flumeri G, Toppi J, Colosimo A, Bezerianos A, Thakor NV, Babiloni F. How the workload impacts on cognitive cooperation: A pilot study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2017:3961-3964. [PMID: 29060764 DOI: 10.1109/embc.2017.8037723] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Cooperation degradation can be seen as one of the main causes of human errors. Poor cooperation could arise from aberrant mental processes, such as mental overload, that negatively affect the user's performance. Using different levels of difficulty in a cooperative task, we combined behavioural, subjective and neurophysiological data with the aim to i) quantify the mental workload under which the crew was operating, ii) evaluate the degree of their cooperation, and iii) assess the impact of the workload demands on the cooperation levels. The combination of such data showed that high workload demand impacted significantly on the performance, workload perception, and degree of cooperation.
Collapse
|
20
|
Aricò P, Borghini G, Di Flumeri G, Sciaraffa N, Babiloni F. Passive BCI beyond the lab: current trends and future directions. Physiol Meas 2018; 39:08TR02. [DOI: 10.1088/1361-6579/aad57e] [Citation(s) in RCA: 115] [Impact Index Per Article: 16.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
21
|
Hernández LG, Mozos OM, Ferrández JM, Antelis JM. EEG-Based Detection of Braking Intention Under Different Car Driving Conditions. Front Neuroinform 2018; 12:29. [PMID: 29910722 PMCID: PMC5992396 DOI: 10.3389/fninf.2018.00029] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2017] [Accepted: 05/08/2018] [Indexed: 11/13/2022] Open
Abstract
The anticipatory recognition of braking is essential to prevent traffic accidents. For instance, driving assistance systems can be useful to properly respond to emergency braking situations. Moreover, the response time to emergency braking situations can be affected and even increased by different driver's cognitive states caused by stress, fatigue, and extra workload. This work investigates the detection of emergency braking from driver's electroencephalographic (EEG) signals that precede the brake pedal actuation. Bioelectrical signals were recorded while participants were driving in a car simulator while avoiding potential collisions by performing emergency braking. In addition, participants were subjected to stress, workload, and fatigue. EEG signals were classified using support vector machines (SVM) and convolutional neural networks (CNN) in order to discriminate between braking intention and normal driving. Results showed significant recognition of emergency braking intention which was on average 71.1% for SVM and 71.8% CNN. In addition, the classification accuracy for the best participant was 80.1 and 88.1% for SVM and CNN, respectively. These results show the feasibility of incorporating recognizable driver's bioelectrical responses into advanced driver-assistance systems to carry out early detection of emergency braking situations which could be useful to reduce car accidents.
Collapse
Affiliation(s)
- Luis G Hernández
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Zapopan, Mexico
| | | | | | - Javier M Antelis
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Zapopan, Mexico
| |
Collapse
|
22
|
Arico P, Borghini G, Di Flumeri G, Bonelli S, Golfetti A, Graziani I, Pozzi S, Imbert JP, Granger G, Benhacene R, Schaefer D, Babiloni F. Human Factors and Neurophysiological Metrics in Air Traffic Control: A Critical Review. IEEE Rev Biomed Eng 2017; 10:250-263. [PMID: 28422665 DOI: 10.1109/rbme.2017.2694142] [Citation(s) in RCA: 36] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This paper provides a focused and organized review of the research progress on neurophysiological indicators, also called "neurometrics," to show how they can effectively address some of the most important human factors (HFs) needs in the air traffic management (ATM) field. In order to better understand and highlight available opportunities of such neuroscientific applications, state of the art on the most involved HFs and related cognitive processes (e.g., mental workload and cognitive training) are presented together with examples of possible applications in current and future ATM scenarios. Furthermore, this paper will discuss the potential enhancements that further research and development activities could bring to the efficiency and safety of the ATM service.
Collapse
|
23
|
Borghini G, Aricò P, Di Flumeri G, Cartocci G, Colosimo A, Bonelli S, Golfetti A, Imbert JP, Granger G, Benhacene R, Pozzi S, Babiloni F. EEG-Based Cognitive Control Behaviour Assessment: an Ecological study with Professional Air Traffic Controllers. Sci Rep 2017; 7:547. [PMID: 28373684 PMCID: PMC5428823 DOI: 10.1038/s41598-017-00633-7] [Citation(s) in RCA: 62] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2016] [Accepted: 02/28/2017] [Indexed: 01/13/2023] Open
Abstract
Several models defining different types of cognitive human behaviour are available. For this work, we have selected the Skill, Rule and Knowledge (SRK) model proposed by Rasmussen in 1983. This model is currently broadly used in safety critical domains, such as the aviation. Nowadays, there are no tools able to assess at which level of cognitive control the operator is dealing with the considered task, that is if he/she is performing the task as an automated routine (skill level), as procedures-based activity (rule level), or as a problem-solving process (knowledge level). Several studies tried to model the SRK behaviours from a Human Factor perspective. Despite such studies, there are no evidences in which such behaviours have been evaluated from a neurophysiological point of view, for example, by considering brain activity variations across the different SRK levels. Therefore, the proposed study aimed to investigate the use of neurophysiological signals to assess the cognitive control behaviours accordingly to the SRK taxonomy. The results of the study, performed on 37 professional Air Traffic Controllers, demonstrated that specific brain features could characterize and discriminate the different SRK levels, therefore enabling an objective assessment of the degree of cognitive control behaviours in realistic settings.
Collapse
Affiliation(s)
- Gianluca Borghini
- Dept. of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy.
- BrainSigns srl, via Sesto Celere, 00152, Rome, Italy.
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina, 306, 00179, Rome, Italy.
| | - Pietro Aricò
- Dept. of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy
- BrainSigns srl, via Sesto Celere, 00152, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina, 306, 00179, Rome, Italy
| | - Gianluca Di Flumeri
- BrainSigns srl, via Sesto Celere, 00152, Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina, 306, 00179, Rome, Italy
- Dept. of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy
| | - Giulia Cartocci
- Dept. of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy
- BrainSigns srl, via Sesto Celere, 00152, Rome, Italy
| | - Alfredo Colosimo
- Dept. of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy
| | | | | | - Jean Paul Imbert
- École Nationale de l'Aviation Civile, 7 Avenue Edouard Belin, 31000, Toulouse, France
| | - Géraud Granger
- École Nationale de l'Aviation Civile, 7 Avenue Edouard Belin, 31000, Toulouse, France
| | - Railane Benhacene
- École Nationale de l'Aviation Civile, 7 Avenue Edouard Belin, 31000, Toulouse, France
| | - Simone Pozzi
- DeepBlue srl, Piazza Buenos Aires 20, 00185, Rome, Italy
| | - Fabio Babiloni
- Dept. of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185, Rome, Italy
- BrainSigns srl, via Sesto Celere, 00152, Rome, Italy
| |
Collapse
|
24
|
Stone BT, Correa KA, Brown TL, Spurgin AL, Stikic M, Johnson RR, Berka C. Behavioral and Neurophysiological Signatures of Benzodiazepine-Related Driving Impairments. Front Psychol 2015; 6:1799. [PMID: 26635697 PMCID: PMC4659917 DOI: 10.3389/fpsyg.2015.01799] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2015] [Accepted: 11/09/2015] [Indexed: 02/03/2023] Open
Abstract
Impaired driving due to drug use is a growing problem worldwide; estimates show that 18-23.5% of fatal accidents, and up to 34% of injury accidents may be caused by drivers under the influence of drugs (Drummer et al., 2003; Walsh et al., 2004; NHTSA, 2010). Furthermore, at any given time, up to 16% of drivers may be using drugs that can impair one's driving abilities (NHTSA, 2009). Currently, drug recognition experts (DREs; law enforcement officers with specialized training to identify drugged driving), have the most difficult time with identifying drivers potentially impaired on central nervous system (CNS) depressants (Smith et al., 2002). The fact that the use of benzodiazepines, a type of CNS depressant, is also associated with the greatest likelihood of causing accidents (Dassanayake et al., 2011), further emphasizes the need to improve research tools in this area which can facilitate the refinement of, or additions to, current assessments of impaired driving. Our laboratories collaborated to evaluate both the behavioral and neurophysiological effects of a benzodiazepine, alprazolam, in a driving simulation (miniSim(TM)). This drive was combined with a neurocognitive assessment utilizing time synched neurophysiology (electroencephalography, ECG). While the behavioral effects of benzodiazepines are well characterized (Rapoport et al., 2009), we hypothesized that, with the addition of real-time neurophysiology and the utilization of simulation and neurocognitive assessment, we could find objective assessments of drug impairment that could improve the detection capabilities of DREs. Our analyses revealed that (1) specific driving conditions were significantly more difficult for benzodiazepine impaired drivers and (2) the neurocognitive tasks' metrics were able to classify "impaired" vs. "unimpaired" with up to 80% accuracy based on lane position deviation and lane departures. While this work requires replication in larger studies, our results not only identified criteria that could potentially improve the identification of benzodiazepine intoxication by DREs, but also demonstrated the promise for future studies using this approach to improve upon current, real-world assessments of impaired driving.
Collapse
Affiliation(s)
- Bradly T. Stone
- Advanced Brain Monitoring, Inc., Carlsbad, CAUSA,*Correspondence: Bradly T. Stone,
| | | | - Timothy L. Brown
- National Advanced Driving Simulator, Center for Computer Aided Design, The University of IowaIowa City, IA, USA
| | - Andrew L. Spurgin
- National Advanced Driving Simulator, Center for Computer Aided Design, The University of IowaIowa City, IA, USA,College of Pharmacy, The University of IowaIowa City, IA, USA
| | - Maja Stikic
- Advanced Brain Monitoring, Inc., Carlsbad, CAUSA
| | | | - Chris Berka
- Advanced Brain Monitoring, Inc., Carlsbad, CAUSA
| |
Collapse
|
25
|
A Context-Aware EEG Headset System for Early Detection of Driver Drowsiness. SENSORS 2015; 15:20873-93. [PMID: 26308002 PMCID: PMC4570452 DOI: 10.3390/s150820873] [Citation(s) in RCA: 62] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/06/2015] [Revised: 07/30/2015] [Accepted: 08/18/2015] [Indexed: 11/17/2022]
Abstract
Driver drowsiness is a major cause of mortality in traffic accidents worldwide. Electroencephalographic (EEG) signal, which reflects the brain activities, is more directly related to drowsiness. Thus, many Brain-Machine-Interface (BMI) systems have been proposed to detect driver drowsiness. However, detecting driver drowsiness at its early stage poses a major practical hurdle when using existing BMI systems. This study proposes a context-aware BMI system aimed to detect driver drowsiness at its early stage by enriching the EEG data with the intensity of head-movements. The proposed system is carefully designed for low-power consumption with on-chip feature extraction and low energy Bluetooth connection. Also, the proposed system is implemented using JAVA programming language as a mobile application for on-line analysis. In total, 266 datasets obtained from six subjects who participated in a one-hour monotonous driving simulation experiment were used to evaluate this system. According to a video-based reference, the proposed system obtained an overall detection accuracy of 82.71% for classifying alert and slightly drowsy events by using EEG data alone and 96.24% by using the hybrid data of head-movement and EEG. These results indicate that the combination of EEG data and head-movement contextual information constitutes a robust solution for the early detection of driver drowsiness.
Collapse
|
26
|
Cartocci G, Maglione AG, Vecchiato G, Di Flumeri G, Colosimo A, Scorpecci A, Marsella P, Giannantonio S, Malerba P, Borghini G, Arico P, Babiloni F. Mental workload estimations in unilateral deafened children. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2015; 2015:1654-1657. [PMID: 26736593 DOI: 10.1109/embc.2015.7318693] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Despite of technological innovations, noisy environments still constitute a challenging and stressful situation for words recognition by hearing impaired subjects. The evaluation of the mental workload imposed by the noisy environments for the recognition of the words in prelingually deaf children is then of paramount importance since it could affect the speed of the learning process during scholar period.The aim of the present study was to investigate different electroencephalographic (EEG) power spectral density (PSD) components (in theta 4-8 Hz - and alpha - 8-12 Hz - frequency bands) to estimate the mental workload index in different noise conditions during a word recognition task in prelingually deaf children, a population not yet investigated in relation to the workload index during auditory tasks. A pilot study involving a small group of prelingually deaf children was then subjected to EEG recordings during an auditory task composed by a listening and a successive recognition of words with different noise conditions. Results showed that in the pre-word listening phase frontal EEG PSD in theta band and the ratio of the frontal EEG PSD in theta band and the parietal EEG PSD in alpha band (workload index; IWL) reported highest values in the most demanding noise condition. In addition, in the phase preceding the word forced-choice task the highest parietal EEG PSD in alpha band and IWL values were reported at the presumably simplest condition (noise emitted in correspondence of the subject's deaf ear). These results could suggest the prominence of EEG PSD theta component activity in the pre-word listening phase. In addition, a more challenging noise situation in the pre-choice phase would be so "over-demanding" to fail to enhance both the alpha power and the IWL in comparison to the already demanding "simple" condition.
Collapse
|