1
|
Giannakopoulou O, Kakkos I, Dimitrakopoulos GN, Tarousi M, Sun Y, Bezerianos A, Koutsouris DD, Matsopoulos GK. Individual Variability in Brain Connectivity Patterns and Driving-Fatigue Dynamics. SENSORS (BASEL, SWITZERLAND) 2024; 24:3894. [PMID: 38931678 PMCID: PMC11207888 DOI: 10.3390/s24123894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2024] [Revised: 06/05/2024] [Accepted: 06/14/2024] [Indexed: 06/28/2024]
Abstract
Mental fatigue during driving poses significant risks to road safety, necessitating accurate assessment methods to mitigate potential hazards. This study explores the impact of individual variability in brain networks on driving fatigue assessment, hypothesizing that subject-specific connectivity patterns play a pivotal role in understanding fatigue dynamics. By conducting a linear regression analysis of subject-specific brain networks in different frequency bands, this research aims to elucidate the relationships between frequency-specific connectivity patterns and driving fatigue. As such, an EEG sustained driving simulation experiment was carried out, estimating individuals' brain networks using the Phase Lag Index (PLI) to capture shared connectivity patterns. The results unveiled notable variability in connectivity patterns across frequency bands, with the alpha band exhibiting heightened sensitivity to driving fatigue. Individualized connectivity analysis underscored the complexity of fatigue assessment and the potential for personalized approaches. These findings emphasize the importance of subject-specific brain networks in comprehending fatigue dynamics, while providing sensor space minimization, advocating for the development of efficient mobile sensor applications for real-time fatigue detection in driving scenarios.
Collapse
Affiliation(s)
- Olympia Giannakopoulou
- Biomedical Engineering Laboratory, National Technical University of Athens, 15772 Athens, Greece; (O.G.); (D.D.K.); (G.K.M.)
| | - Ioannis Kakkos
- Biomedical Engineering Laboratory, National Technical University of Athens, 15772 Athens, Greece; (O.G.); (D.D.K.); (G.K.M.)
- Department of Biomedical Engineering, University of West Attica, 12243 Athens, Greece
| | | | - Marilena Tarousi
- Biomedical Engineering Laboratory, National Technical University of Athens, 15772 Athens, Greece; (O.G.); (D.D.K.); (G.K.M.)
| | - Yu Sun
- Key Laboratory for Biomedical Engineering of Ministry of Education of China, Department of Biomedical Engineering, Zhejiang University, Hangzhou 310027, China
| | - Anastasios Bezerianos
- Brain Dynamics Laboratory, Barrow Neurological Institute (BNI), St. Joseph’s Hospital and Medical Center, Phoenix, AZ 85013, USA;
| | - Dimitrios D. Koutsouris
- Biomedical Engineering Laboratory, National Technical University of Athens, 15772 Athens, Greece; (O.G.); (D.D.K.); (G.K.M.)
| | - George K. Matsopoulos
- Biomedical Engineering Laboratory, National Technical University of Athens, 15772 Athens, Greece; (O.G.); (D.D.K.); (G.K.M.)
| |
Collapse
|
2
|
Minhas R, Peker NY, Hakkoz MA, Arbatli S, Celik Y, Erdem CE, Semiz B, Peker Y. Association of Visual-Based Signals with Electroencephalography Patterns in Enhancing the Drowsiness Detection in Drivers with Obstructive Sleep Apnea. SENSORS (BASEL, SWITZERLAND) 2024; 24:2625. [PMID: 38676243 PMCID: PMC11055081 DOI: 10.3390/s24082625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Revised: 04/08/2024] [Accepted: 04/15/2024] [Indexed: 04/28/2024]
Abstract
Individuals with obstructive sleep apnea (OSA) face increased accident risks due to excessive daytime sleepiness. PERCLOS, a recognized drowsiness detection method, encounters challenges from image quality, eyewear interference, and lighting variations, impacting its performance, and requiring validation through physiological signals. We propose visual-based scoring using adaptive thresholding for eye aspect ratio with OpenCV for face detection and Dlib for eye detection from video recordings. This technique identified 453 drowsiness (PERCLOS ≥ 0.3 || CLOSDUR ≥ 2 s) and 474 wakefulness episodes (PERCLOS < 0.3 and CLOSDUR < 2 s) among fifty OSA drivers in a 50 min driving simulation while wearing six-channel EEG electrodes. Applying discrete wavelet transform, we derived ten EEG features, correlated them with visual-based episodes using various criteria, and assessed the sensitivity of brain regions and individual EEG channels. Among these features, theta-alpha-ratio exhibited robust mapping (94.7%) with visual-based scoring, followed by delta-alpha-ratio (87.2%) and delta-theta-ratio (86.7%). Frontal area (86.4%) and channel F4 (75.4%) aligned most episodes with theta-alpha-ratio, while frontal, and occipital regions, particularly channels F4 and O2, displayed superior alignment across multiple features. Adding frontal or occipital channels could correlate all episodes with EEG patterns, reducing hardware needs. Our work could potentially enhance real-time drowsiness detection reliability and assess fitness to drive in OSA drivers.
Collapse
Affiliation(s)
- Riaz Minhas
- College of Engineering, Koc University, Istanbul 34450, Turkey; (R.M.); (B.S.)
| | - Nur Yasin Peker
- Department of Mechatronics Engineering, Sakarya University of Applied Sciences, Sakarya 54050, Turkey;
| | - Mustafa Abdullah Hakkoz
- Graduate School of Computer Engineering, Istanbul Technical University, Istanbul 34469, Turkey;
| | - Semih Arbatli
- Graduate School of Health Sciences, Koc University, Istanbul 34010, Turkey;
| | - Yeliz Celik
- Research Center for Translational Medicine (KUTTAM), Koc University, Istanbul 34010, Turkey;
| | - Cigdem Eroglu Erdem
- Department of Electrical and Electronics Engineering, Ozyegin University, Istanbul 34794, Turkey;
| | - Beren Semiz
- College of Engineering, Koc University, Istanbul 34450, Turkey; (R.M.); (B.S.)
| | - Yuksel Peker
- Research Center for Translational Medicine (KUTTAM), Koc University, Istanbul 34010, Turkey;
- Department of Pulmonary Medicine, School of Medicine, Koc University, Istanbul 34010, Turkey
- Sahlgrenska Academy, University of Gothenburg, 40530 Gothenburg, Sweden
- School of Medicine, Lund University, 22185 Lund, Sweden
- School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA
| |
Collapse
|
3
|
Giorgi A, Ronca V, Vozzi A, Aricò P, Borghini G, Capotorto R, Tamborra L, Simonetti I, Sportiello S, Petrelli M, Polidori C, Varga R, van Gasteren M, Barua A, Ahmed MU, Babiloni F, Di Flumeri G. Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving. Front Neurorobot 2023; 17:1240933. [PMID: 38107403 PMCID: PMC10721973 DOI: 10.3389/fnbot.2023.1240933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 10/18/2023] [Indexed: 12/19/2023] Open
Abstract
The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.
Collapse
Affiliation(s)
- Andrea Giorgi
- Department of Anatomical, Histological, Forensic and Orthopaedic Sciences, Sapienza University of Rome, Rome, Italy
- BrainSigns SRL, Rome, Italy
| | - Vincenzo Ronca
- BrainSigns SRL, Rome, Italy
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, Rome, Italy
| | - Alessia Vozzi
- Department of Anatomical, Histological, Forensic and Orthopaedic Sciences, Sapienza University of Rome, Rome, Italy
- BrainSigns SRL, Rome, Italy
| | - Pietro Aricò
- BrainSigns SRL, Rome, Italy
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, Rome, Italy
| | - Gianluca Borghini
- BrainSigns SRL, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Rossella Capotorto
- Department of Computer, Control, and Management Engineering, Sapienza University of Rome, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Luca Tamborra
- BrainSigns SRL, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Ilaria Simonetti
- BrainSigns SRL, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| | - Simone Sportiello
- Department of Civil Engineering, Computer Science and Aeronautical Technologies, Roma Tre University, Rome, Italy
- Department of Enterprise Engineering, University of Rome Tor Vergata, Rome, Italy
| | - Marco Petrelli
- Department of Civil Engineering, Computer Science and Aeronautical Technologies, Roma Tre University, Rome, Italy
| | - Carlo Polidori
- Italian Association of Road Safety Professionals (AIPSS), Rome, Italy
| | - Rodrigo Varga
- Instituto Tecnologico de Castilla y Leon, Burgos, Spain
| | | | - Arnab Barua
- Academy for Innovation, Design and Technology, Mälardalens University, Västerås, Sweden
| | - Mobyen Uddin Ahmed
- Academy for Innovation, Design and Technology, Mälardalens University, Västerås, Sweden
| | - Fabio Babiloni
- BrainSigns SRL, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Gianluca Di Flumeri
- BrainSigns SRL, Rome, Italy
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
4
|
Marois A, Kopf M, Fortin M, Huot-Lavoie M, Martel A, Boyd JG, Gagnon JF, Archambault PM. Psychophysiological models of hypovigilance detection: A scoping review. Psychophysiology 2023; 60:e14370. [PMID: 37350389 DOI: 10.1111/psyp.14370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 05/31/2023] [Accepted: 06/01/2023] [Indexed: 06/24/2023]
Abstract
Hypovigilance represents a major contributor to accidents. In operational contexts, the burden of monitoring/managing vigilance often rests on operators. Recent advances in sensing technologies allow for the development of psychophysiology-based (hypo)vigilance prediction models. Still, these models remain scarcely applied to operational situations and need better understanding. The current scoping review provides a state of knowledge regarding psychophysiological models of hypovigilance detection. Records evaluating vigilance measuring tools with gold standard comparisons and hypovigilance prediction performances were extracted from MEDLINE, PsychInfo, and Inspec. Exclusion criteria comprised aspects related to language, non-empirical papers, and sleep studies. The Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS) and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were used for bias evaluation. Twenty-one records were reviewed. They were mainly characterized by participant selection and analysis biases. Papers predominantly focused on driving and employed several common psychophysiological techniques. Yet, prediction methods and gold standards varied widely. Overall, we outline the main strategies used to assess hypovigilance, their principal limitations, and we discuss applications of these models.
Collapse
Affiliation(s)
- Alexandre Marois
- Thales Research and Technology Canada, Quebec City, Québec, Canada
- School of Psychology and Computer Science, University of Central Lancashire, Preston, Lancashire, United Kingdom
| | - Maëlle Kopf
- Thales Research and Technology Canada, Quebec City, Québec, Canada
| | - Michelle Fortin
- Faculty of Medicine, Université Laval, Quebec City, Québec, Canada
| | | | - Alexandre Martel
- Faculty of Medicine, Université Laval, Quebec City, Québec, Canada
| | - J Gordon Boyd
- Department of Medicine, Queen's University, Kingston, Ontario, Canada
- Kingston General Hospital, Kingston, Ontario, Canada
| | | | - Patrick M Archambault
- Faculty of Medicine, Université Laval, Quebec City, Québec, Canada
- Centre de recherche intégrée pour un système apprenant en santé et services sociaux, Centre intégré de santé et de services sociaux de Chaudière-Appalaches, Lévis, Québec, Canada
- VITAM - Centre de recherche en santé durable, Centre intégré universitaire de santé et de services sociaux de la Capitale-Nationale, Quebec City, Québec, Canada
| |
Collapse
|
5
|
Teicher MH, Bolger E, Garcia LCH, Hafezi P, Weiser LP, McGreenery CE, Khan A, Ohashi K. Bright light therapy and early morning attention, mathematical performance, electroencephalography and brain connectivity in adolescents with morning sleepiness. PLoS One 2023; 18:e0273269. [PMID: 37607203 PMCID: PMC10443881 DOI: 10.1371/journal.pone.0273269] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 07/18/2023] [Indexed: 08/24/2023] Open
Abstract
Adolescents typically sleep too little and feel drowsy during morning classes. We assessed whether morning use of an LED bright light device could increase alertness in school students. Twenty-six (8M/18F) healthy, unmedicated participants, ages 13-18 years, (mean 17.1±1.4) were recruited following screenings to exclude psychopathology. Baseline assessments were made of actigraph-assessed sleep, attention, math solving ability, electroencephalography and structural and functional MRI (N = 10-11, pre-post). Participants nonrandomly received 3-4 weeks of bright light therapy (BLT) for 30 minutes each morning and used blue light blocking glasses for 2 hours before bedtime. BLT devices were modified to surreptitiously record degree of use so that the hypothesis tested was whether there was a significant relationship between degree of use and outcome. They were used 57±18% (range 23%-90%) of recommended time. There was a significant association between degree of use and: (1) increased beta spectral power in frontal EEG leads (primary measure); (2) greater post-test improvement in math performance and reduction in errors of omission on attention test; (3) reduced day-to-day variability in bed times, sleep onset, and sleep duration during school days; (4) increased dentate gyrus volume and (5) enhanced frontal connectivity with temporal, occipital and cerebellar regions during Go/No-Go task performance. BLT was associated with improvement in sleep cycle consistency, arousal, attention and functional connectivity, but not sleep onset or duration (primary measures). Although this was an open study, it suggests that use of bright morning light and blue light blocking glasses before bed may benefit adolescents experiencing daytime sleepiness. Clinical trial registration: Clinicaltrials.gov ID-NCT05383690.
Collapse
Affiliation(s)
- Martin H. Teicher
- Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Elizabeth Bolger
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Laura C. Hernandez Garcia
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Poopak Hafezi
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Leslie P. Weiser
- Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Cynthia E. McGreenery
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Alaptagin Khan
- Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| | - Kyoko Ohashi
- Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America
- Developmental Biopsychiatry Research Program, McLean Hospital, Belmont, Massachusetts, United States of America
| |
Collapse
|
6
|
Minami S, Watanabe K, Saijo N, Kashino M. Neural oscillation amplitude in the frontal cortex predicts esport results. iScience 2023; 26:106845. [PMID: 37250772 PMCID: PMC10212977 DOI: 10.1016/j.isci.2023.106845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 03/28/2023] [Accepted: 05/04/2023] [Indexed: 05/31/2023] Open
Abstract
In competitive matches, strategic decisions and emotional control are important. Relevant cognitive functions and corresponding neural activities in simple and short-term laboratory tasks have been reported. Brain resources are intensively allocated in the frontal cortex during strategic decision-making. The suppression of the frontal cortex with alpha-synchronization optimizes emotional control. However, no studies have reported the contribution of neural activity to the outcome of a more complex and prolonged task. To clarify this issue, we focused on a fighting video game following a two-round first-pass system. Frontal high-gamma and alpha power in the first and third pre-round periods, respectively, were found to be increased in a winning match. Furthermore, inter-participant variations in the importance of strategic decisions and emotional control in the first and third pre-round periods were correlated with frontal high-gamma and alpha power, respectively. Therefore, the psychological and mental state, involving frontal neural fluctuations, is predictive of match outcome.
Collapse
Affiliation(s)
- Sorato Minami
- NTT Communication Science Laboratories, 3-1, Morinosato Wakamiya, Atsugi, Kanagawa 243-0198, Japan
| | - Ken Watanabe
- NTT Communication Science Laboratories, 3-1, Morinosato Wakamiya, Atsugi, Kanagawa 243-0198, Japan
- School of Fundamental Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku, Tokyo 169-8555, Japan
| | - Naoki Saijo
- NTT Communication Science Laboratories, 3-1, Morinosato Wakamiya, Atsugi, Kanagawa 243-0198, Japan
| | - Makio Kashino
- NTT Communication Science Laboratories, 3-1, Morinosato Wakamiya, Atsugi, Kanagawa 243-0198, Japan
| |
Collapse
|
7
|
Yadav H, Maini S. Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-45. [PMID: 37362726 PMCID: PMC10157593 DOI: 10.1007/s11042-023-15653-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 07/17/2022] [Accepted: 04/22/2023] [Indexed: 06/28/2023]
Abstract
Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.
Collapse
Affiliation(s)
- Hitesh Yadav
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| | - Surita Maini
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| |
Collapse
|
8
|
Hamann A, Carstengerdes N. Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement. Sci Rep 2023; 13:4738. [PMID: 36959334 PMCID: PMC10036528 DOI: 10.1038/s41598-023-31264-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 03/08/2023] [Indexed: 03/25/2023] Open
Abstract
Mental fatigue (MF) can impair pilots' performance and reactions to unforeseen events and is therefore an important concept within aviation. The physiological measurement of MF, especially with EEG and, in recent years, fNIRS, has gained much attention. However, a systematic investigation and comparison of the measurements is seldomly done. We induced MF via time on task during a 90-min simulated flight task and collected concurrent EEG-fNIRS, performance and self-report data from 31 participants. While their subjective MF increased linearly, the participants were able to keep their performance stable over the course of the experiment. EEG data showed an early increase and levelling in parietal alpha power and a slower, but steady increase in frontal theta power. No consistent trend could be observed in the fNIRS data. Thus, more research on fNIRS is needed to understand its possibilities and limits for MF assessment, and a combination with EEG is advisable to compare and validate results. Until then, EEG remains the better choice for continuous MF assessment in cockpit applications because of its high sensitivity to a transition from alert to fatigued, even before performance is impaired.
Collapse
Affiliation(s)
- Anneke Hamann
- Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Flugführung, Lilienthalplatz 7, 38108, Braunschweig, Germany.
| | - Nils Carstengerdes
- Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Flugführung, Lilienthalplatz 7, 38108, Braunschweig, Germany
| |
Collapse
|
9
|
Xu X, Tang J, Xu T, Lin M. Mental Fatigue Degree Recognition Based on Relative Band Power and Fuzzy Entropy of EEG. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:1447. [PMID: 36674202 PMCID: PMC9861020 DOI: 10.3390/ijerph20021447] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 01/04/2023] [Accepted: 01/11/2023] [Indexed: 06/17/2023]
Abstract
Mental fatigue is a common phenomenon in our daily lives. Long-term fatigue can lead to a decline in a person's operational functions and seriously affect work efficiency. In this paper, a method that recognizes the degree of mental fatigue based on relative band power and fuzzy entropy of Electroencephalogram (EEG) is proposed. The N-back experiment was used to induce mental fatigue in subjects, and the corresponding EEG signals were recorded during the experiment. A preprocessing method based on complementary ensemble empirical modal decomposition (CEEMD) and independent component analysis (ICA) was designed to remove noise from the raw EEG signal. The relative band power feature, which has been used extensively in fatigue recognition studies, was extracted from the EEG signals. Meanwhile, fuzzy entropy, a feature commonly used in attention recognition, was also extracted for fatigue recognition, based on previous findings that an increase in fatigue is accompanied by a decrease in attention. The two features were fed into an extreme gradient boosting (XGBoost) classifier to distinguish three different degrees of fatigue, which resulted in an average accuracy of 92.39% based on data from eight subjects. The promising results indicate the effectiveness of the proposed method in mental fatigue degree identification.
Collapse
|
10
|
Verma AK, Yu Y, Acosta-Lenis SF, Havel T, Sanabria DE, Molnar GF, MacKinnon CD, Howell MJ, Vitek JL, Johnson LA. Parkinsonian daytime sleep-wake classification using deep brain stimulation lead recordings. Neurobiol Dis 2023; 176:105963. [PMID: 36521781 PMCID: PMC9869648 DOI: 10.1016/j.nbd.2022.105963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 12/01/2022] [Accepted: 12/10/2022] [Indexed: 12/14/2022] Open
Abstract
Excessive daytime sleepiness is a recognized non-motor symptom that adversely impacts the quality of life of people with Parkinson's disease (PD), yet effective treatment options remain limited. Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective treatment for PD motor signs. Reliable daytime sleep-wake classification using local field potentials (LFPs) recorded from DBS leads implanted in STN can inform the development of closed-loop DBS approaches for prompt detection and disruption of sleep-related neural oscillations. We performed STN DBS lead recordings in three nonhuman primates rendered parkinsonian by administrating neurotoxin 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP). Reference sleep-wake states were determined on a second-by-second basis by video monitoring of eyes (eyes-open, wake and eyes-closed, sleep). The spectral power in delta (1-4 Hz), theta (4-8 Hz), low-beta (8-20 Hz), high-beta (20-35 Hz), gamma (35-90 Hz), and high-frequency (200-400 Hz) bands were extracted from each wake and sleep epochs for training (70% data) and testing (30% data) a support vector machines classifier for each subject independently. The spectral features yielded reasonable daytime sleep-wake classification (sensitivity: 90.68 ± 1.28; specificity: 88.16 ± 1.08; accuracy: 89.42 ± 0.68; positive predictive value; 88.70 ± 0.89, n = 3). Our findings support the plausibility of monitoring daytime sleep-wake states using DBS lead recordings. These results could have future clinical implications in informing the development of closed-loop DBS approaches for automatic detection and disruption of sleep-related neural oscillations in people with PD to promote wakefulness.
Collapse
Affiliation(s)
- Ajay K Verma
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Ying Yu
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Sergio F Acosta-Lenis
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Tyler Havel
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | | | - Gregory F Molnar
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Colum D MacKinnon
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Michael J Howell
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Jerrold L Vitek
- Department of Neurology, University of Minnesota, Minneapolis, United States of America
| | - Luke A Johnson
- Department of Neurology, University of Minnesota, Minneapolis, United States of America.
| |
Collapse
|
11
|
Salihu AT, Hill KD, Jaberzadeh S. Neural mechanisms underlying state mental fatigue: a systematic review and activation likelihood estimation meta-analysis. Rev Neurosci 2022; 33:889-917. [PMID: 35700454 DOI: 10.1515/revneuro-2022-0023] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Accepted: 04/26/2022] [Indexed: 12/14/2022]
Abstract
Sustained performance of cognitive tasks could lead to the development of state mental fatigue characterized by subjective sensation of mental weariness and decrease in cognitive performance. In addition to the occupational hazards associated with mental fatigue, it can also affect physical performance reducing endurance, balance, and sport-specific technical skills. Similarly, mental fatigue is a common symptom in certain chronic health conditions such as multiple sclerosis affecting quality of life of the patients. Despite its widely acknowledged negative impact, the neural mechanisms underlining this phenomenon are still not fully understood. We conducted a systematic review and activation likelihood estimation (ALE) meta-analysis of functional neuroimaging studies investigating the effect of mental fatigue due to time-on-task (TOT) on brain activity to elucidate the possible underlying mechanisms. Studies were included if they examined change in brain activity induced by experimental mental fatigue (TOT effect) or investigated the relationship between brain activity and subjective mental fatigue due to TOT. A total of 33 studies met the review's inclusion criteria, 13 of which were included in meta-analyses. Results of the meta-analyses revealed a decrease in activity with TOT in brain areas that constitute the cognitive control network. Additionally, an increased activity with TOT, as well as negative relationship with subjective mental fatigue was found in parts of the default mode network of the brain. The changes in cognitive control and the default mode networks of the brain due to state mental fatigue observed in this study were discussed in relation to the existing theories of mental fatigue.
Collapse
Affiliation(s)
- Abubakar Tijjani Salihu
- Monash Neuromodulation Research Unit, Department of Physiotherapy, School of Primary and Allied Health Care, Faculty of Medicine, Nursing and Health Science, Monash University, Melbourne, Australia
| | - Keith D Hill
- Rehabilitation, Ageing and Independent Living (RAIL) Research Centre, School of Primary and Allied Health Care, Monash University, Frankston, Australia
| | - Shapour Jaberzadeh
- Monash Neuromodulation Research Unit, Department of Physiotherapy, School of Primary and Allied Health Care, Faculty of Medicine, Nursing and Health Science, Monash University, Melbourne, Australia
| |
Collapse
|
12
|
Cao J, Garro EM, Zhao Y. EEG/fNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22197623. [PMID: 36236725 PMCID: PMC9571712 DOI: 10.3390/s22197623] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 10/03/2022] [Accepted: 10/06/2022] [Indexed: 05/07/2023]
Abstract
There is high demand for techniques to estimate human mental workload during some activities for productivity enhancement or accident prevention. Most studies focus on a single physiological sensing modality and use univariate methods to analyse multi-channel electroencephalography (EEG) data. This paper proposes a new framework that relies on the features of hybrid EEG-functional near-infrared spectroscopy (EEG-fNIRS), supported by machine-learning features to deal with multi-level mental workload classification. Furthermore, instead of the well-used univariate power spectral density (PSD) for EEG recording, we propose using bivariate functional brain connectivity (FBC) features in the time and frequency domains of three bands: delta (0.5-4 Hz), theta (4-7 Hz) and alpha (8-15 Hz). With the assistance of the fNIRS oxyhemoglobin and deoxyhemoglobin (HbO and HbR) indicators, the FBC technique significantly improved classification performance at a 77% accuracy for 0-back vs. 2-back and 83% for 0-back vs. 3-back using a public dataset. Moreover, topographic and heat-map visualisation indicated that the distinguishing regions for EEG and fNIRS showed a difference among the 0-back, 2-back and 3-back test results. It was determined that the best region to assist the discrimination of the mental workload for EEG and fNIRS is different. Specifically, the posterior area performed the best for the posterior midline occipital (POz) EEG in the alpha band and fNIRS had superiority in the right frontal region (AF8).
Collapse
|
13
|
Hosni SMI, Borgheai SB, McLinden J, Zhu S, Huang X, Ostadabbas S, Shahriari Y. A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI. Neuroinformatics 2022; 20:1169-1189. [PMID: 35907174 DOI: 10.1007/s12021-022-09595-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/05/2022] [Indexed: 12/31/2022]
Abstract
Decoding neural responses from multimodal information sources, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), has the transformative potential to advance hybrid brain-computer interfaces (hBCIs). However, existing modest performance improvement of hBCIs might be attributed to the lack of computational frameworks that exploit complementary synergistic properties in multimodal features. This study proposes a multimodal data fusion framework to represent and decode synergistic multimodal motor imagery (MI) neural responses. We hypothesize that exploiting EEG nonlinear dynamics adds a new informative dimension to the commonly combined EEG-fNIRS features and will ultimately increase the synergy between EEG and fNIRS features toward an enhanced hBCI. The EEG nonlinear dynamics were quantified by extracting graph-based recurrence quantification analysis (RQA) features to complement the commonly used spectral features for an enhanced multimodal configuration when combined with fNIRS. The high-dimensional multimodal features were further given to a feature selection algorithm relying on the least absolute shrinkage and selection operator (LASSO) for fused feature selection. Linear support vector machine (SVM) was then used to evaluate the framework. The mean hybrid classification performance improved by up to 15% and 4% compared to the unimodal EEG and fNIRS, respectively. The proposed graph-based framework substantially increased the contribution of EEG features for hBCI classification from 28.16% up to 52.9% when introduced the nonlinear dynamics and improved the performance by approximately 2%. These findings suggest that graph-based nonlinear dynamics can increase the synergy between EEG and fNIRS features for an enhanced MI response representation that is not dominated by a single modality.
Collapse
Affiliation(s)
- Sarah M I Hosni
- Department of Electrical, Computer & Biomedical Engineering, University of Rhode Island (URI), Kingston, RI, 02881, USA
| | - Seyyed B Borgheai
- Department of Electrical, Computer & Biomedical Engineering, University of Rhode Island (URI), Kingston, RI, 02881, USA
| | - John McLinden
- Department of Electrical, Computer & Biomedical Engineering, University of Rhode Island (URI), Kingston, RI, 02881, USA
| | - Shaotong Zhu
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115, USA
| | - Xiaofei Huang
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115, USA
| | - Sarah Ostadabbas
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115, USA
| | - Yalda Shahriari
- Department of Electrical, Computer & Biomedical Engineering, University of Rhode Island (URI), Kingston, RI, 02881, USA.
| |
Collapse
|
14
|
Bourguignon NJ, Bue SL, Guerrero-Mosquera C, Borragán G. Bimodal EEG-fNIRS in Neuroergonomics. Current Evidence and Prospects for Future Research. FRONTIERS IN NEUROERGONOMICS 2022; 3:934234. [PMID: 38235461 PMCID: PMC10790898 DOI: 10.3389/fnrgo.2022.934234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Accepted: 06/20/2022] [Indexed: 01/19/2024]
Abstract
Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.
Collapse
Affiliation(s)
| | - Salvatore Lo Bue
- Department of Life Sciences, Royal Military Academy of Belgium, Brussels, Belgium
| | | | - Guillermo Borragán
- Center for Research in Cognition and Neuroscience, Université Libre de Bruxelles, Brussels, Belgium
| |
Collapse
|
15
|
Bhuiyan MHU, Fard M, Robinson SR. Effects of whole-body vibration on driver drowsiness: A review. JOURNAL OF SAFETY RESEARCH 2022; 81:175-189. [PMID: 35589288 DOI: 10.1016/j.jsr.2022.02.009] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Revised: 09/29/2021] [Accepted: 02/14/2022] [Indexed: 05/19/2023]
Abstract
INTRODUCTION Whole-body vibration has direct impacts on driver vigilance by increasing physical and cognitive stress on the driver, which leads to drowsiness, fatigue and road traffic accidents. Although sleep deprivation, sleep apnoea and alcohol consumption can also lead to driver drowsiness, exposure to steady vibration is the factor most readily controlled by changes to vehicle design, yet it has received comparatively less attention. METHODS This review investigated interrelationships between the various components of whole-body vibration and the physiological and cognitive parameters that lead to driver drowsiness, as well as the effects of vibration parameters (frequency, amplitude, waveform and duration). Vibrations transmitted to the driver body from the vehicle floor and/or seat have been considered for this review, whereas hand-arm vibration, shocks, acute or transient vibration were excluded from consideration. RESULTS Drowsiness is affected by interactions between the frequency, amplitude, waveform and duration of the vibration. Under optimal conditions, whole-body vibration can induce significant drowsiness within 30 min. Low frequency whole-body vibrations, particularly vibrations of 4-10 Hz, are most effective at inducing drowsiness. This review notes some limitations of current studies and suggests directions for future research. CONCLUSIONS This review demonstrated a strong causal link exists between whole-body vibration and driver drowsiness. Since driver drowsiness has been established to be a significant contributor to motor vehicle accidents, research is needed to identify ways to minimise the components of whole-body vibration that contribute to drowsiness, as well as devising more effective ways to counteract drowsiness. PRACTICAL APPLICATIONS By raising awareness of the vibrational factors that contribute to drowsiness, manufacturers will be prompted to design vehicles that reduce the influence of these factors.
Collapse
Affiliation(s)
| | - Mohamad Fard
- School of Engineering, RMIT University, Melbourne, Australia
| | - Stephen R Robinson
- School of Health and Biomedical Sciences, RMIT University, Melbourne, Australia
| |
Collapse
|
16
|
Study on the Effect of Different Transcranial Pulse Current Stimulation Intervention Programs for Eliminating Physical Fatigue. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12115609] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Previous studies have reported the effect of transcranial pulsed current stimulation (tPCS) on eliminating cognitive fatigue, but there is little research on optimizing the intervention program of tPCS. The purpose of this study was to explore the effect of different tPCS intervention programs on the elimination of physical fatigue in college athletes. Accordingly, 40 healthy college athletes were randomly divided into two groups of 20, denoted as A and B. Both groups exercised on treadmills. There were 15 subjects in group A who met the criteria of moderate physical fatigue, and 15 subjects in group B who met the criteria of severe physical fatigue. The subjects in each group were intervened with five different intervention programs of tPCS (intervention programs I, II, III, IV and V). The heart rate variability (HRV) and concentrations of oxygenated hemoglobin (HbO2) were measured before and after each intervention to judge the elimination effects of different intervention programs on different degrees of physical fatigue; the measurement indicators of the HRV include RMSSD, SDNN, HF and LF. The results indicated that tPCS intervention can eliminate both moderate and severe physical fatigue. Programs II, III, and IV had a significant effect on eliminating the moderate physical fatigue of athletes (p < 0.05), among which program II, with a stimulation time of 30 min and a stimulation intensity of sensory intensity, had the best effect. Programs I, II, III, and IV all had significant effects on eliminating the severe physical fatigue of athletes (p < 0.05), among which program I, with a stimulation time of 30 min and a stimulation intensity of sensory intensity + 0.2 mA, had the best effect. We conclude that different tPCS intervention programs can have different effects on the elimination of physical fatigue. The effects of the five intervention programs on the elimination of physical fatigue in athletes are as follows: program II is most suitable for moderate physical fatigue, and program I is most suitable for severe physical fatigue.
Collapse
|
17
|
Hamann A, Carstengerdes N. Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Sci Rep 2022; 12:6449. [PMID: 35440733 PMCID: PMC9018717 DOI: 10.1038/s41598-022-10044-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 03/30/2022] [Indexed: 11/09/2022] Open
Abstract
Monitoring pilots' cognitive states becomes increasingly important in aviation. Physiological measurement can detect increased mental workload (MWL) even before performance declines. Yet, changes in MWL are rarely varied systematically and few studies control for confounding effects of other cognitive states. The present study targets these shortcomings by analysing the effects of stepwise increased MWL on cortical activation, while controlling for mental fatigue (MF). 35 participants conducted a simulated flight with an incorporated adapted n-back and monitoring task. We recorded cortical activation with concurrent EEG and fNIRS measurement, performance, self-reported MWL and MF. Our results show the successful manipulation of MWL without confounding effects of MF. Higher task difficulty elicited higher subjective MWL ratings, performance decline, higher frontal theta activity and reduced frontal deoxyhaemoglobin (Hbr) concentration. Using both EEG and fNIRS, we could discriminate all induced MWL levels. fNIRS was more sensitive to tasks with low difficulty, and EEG to tasks with high difficulty. Our findings further suggest a plateau effect for high MWL that could present an upper boundary to individual cognitive capacity. Our results highlight the benefits of physiological measurement in aviation, both for assessment of cognitive states and as a data source for adaptive assistance systems.
Collapse
Affiliation(s)
- Anneke Hamann
- Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Flugführung, Lilienthalplatz 7, 38108, Braunschweig, Germany.
| | - Nils Carstengerdes
- Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Flugführung, Lilienthalplatz 7, 38108, Braunschweig, Germany
| |
Collapse
|
18
|
Yeung MK, Chu VW. Viewing neurovascular coupling through the lens of combined EEG-fNIRS: A systematic review of current methods. Psychophysiology 2022; 59:e14054. [PMID: 35357703 DOI: 10.1111/psyp.14054] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Revised: 02/01/2022] [Accepted: 03/08/2022] [Indexed: 12/25/2022]
Abstract
Neurovascular coupling is a key physiological mechanism that occurs in the healthy human brain, and understanding this process has implications for understanding the aging and neuropsychiatric populations. Combined electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) has emerged as a promising, noninvasive tool for probing neurovascular interactions in humans. However, the utility of this approach critically depends on the methodological quality used for multimodal integration. Despite a growing number of combined EEG-fNIRS applications reported in recent years, the methodological rigor of past studies remains unclear, limiting the accurate interpretation of reported findings and hindering the translational application of this multimodal approach. To fill this knowledge gap, we critically evaluated various methodological aspects of previous combined EEG-fNIRS studies performed in healthy individuals. A literature search was conducted using PubMed and PsycINFO on June 28, 2021. Studies involving concurrent EEG and fNIRS measurements in awake and healthy individuals were selected. After screening and eligibility assessment, 96 studies were included in the methodological evaluation. Specifically, we critically reviewed various aspects of participant sampling, experimental design, signal acquisition, data preprocessing, outcome selection, data analysis, and results presentation reported in these studies. Altogether, we identified several notable strengths and limitations of the existing EEG-fNIRS literature. In light of these limitations and the features of combined EEG-fNIRS, recommendations are made to improve and standardize research practices to facilitate the use of combined EEG-fNIRS when studying healthy neurovascular coupling processes and alterations in neurovascular coupling among various populations.
Collapse
Affiliation(s)
- Michael K Yeung
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, China
| | - Vivian W Chu
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, China
| |
Collapse
|
19
|
Le AS, Xuan NH, Aoki H. Assessment of senior drivers' internal state in the event of simulated unexpected vehicle motion based on near-infrared spectroscopy. TRAFFIC INJURY PREVENTION 2022; 23:221-225. [PMID: 35333671 DOI: 10.1080/15389588.2022.2051019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 03/04/2022] [Accepted: 03/05/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE A driver's internal state is a critical factor influencing driving performance, especially in cases of surprise or shock in response to unexpected incidents while driving. This study was designed to investigate the brain activity of a senior driver in response to simulated unexpected vehicle motion, compared with a relaxed state and normal driving. METHODS To accomplish this, we created a driving simulator paradigm wherein participants were involved in one of the following three scenarios: sitting down and relaxing, normal driving around the city with traffic signals and other vehicles, and the exiting of a parking area. In the scenario where the driver was to exit the parking area, the gear was switched automatically by the CarMaker software without the driver being notified, leading to an unexpected condition. The driver's internal states were classified by artificial intelligence, based on information obtained through four-channel near-infrared spectroscopy. RESULTS Significant differences were observed between the hemodynamic responses obtained in the three conditions. Ultimately, this method can be used to update advanced driver assistance systems, with a view to preventing future traffic accidents, by activating in-vehicle safety functions based on the driver's condition. CONCLUSIONS A driver's internal states in a panic situation while driving can be detected using near-infrared spectroscopy and artificial intelligence.
Collapse
Affiliation(s)
- Anh Son Le
- Phenikaa Research and Technology Institute, Phenikaa Group, Hanoi, Vietnam
- Faculty of Vehicle and Energy Engineering, Phenikaa University, Hanoi, Vietnam
- Institute of Innovation for Future Society, Nagoya University, Nagoya, Japan
| | - Nang Ho Xuan
- Phenikaa Research and Technology Institute, Phenikaa Group, Hanoi, Vietnam
- Faculty of Vehicle and Energy Engineering, Phenikaa University, Hanoi, Vietnam
| | - Hirofumi Aoki
- Institute of Innovation for Future Society, Nagoya University, Nagoya, Japan
| |
Collapse
|
20
|
Research on the Identification of Pilots’ Fatigue Status Based on Functional Near-Infrared Spectroscopy. AEROSPACE 2022. [DOI: 10.3390/aerospace9030173] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/10/2022]
Abstract
Fatigue can lead to sluggish responses, misjudgments, flight illusions and other problems for pilots, which could easily bring about serious flight accidents. In this paper, a wearable functional near-infrared spectroscopy (fNIRS) device was used to record the changes of hemoglobin concentration of pilots during flight missions. The data was pre-processed, and 1080 valid samples were determined. Then, mean value, variance, standard deviation, kurtosis, skewness, coefficient of variation, peak value, and range of oxyhemoglobin (HbO2) in each channel were extracted. These indexes were regarded as the input of a stacked denoising autoencoder (SDAE) and were used to train the identification model of pilots’ fatigue state. The identification model of pilots’ fatigue status was established. The identification accuracy of the SDAE model was 91.32%, which was 23.26% and 15.97% higher than that of linear discriminant analysis (LDA) models and support vector machines (SVM) models, respectively. Results show that the SDAE model established in our study has high identification accuracy, which can accurately identify different fatigue states of pilots. Identification of pilots’ fatigue status based on fNIRS has important practical significance for reducing flight accidents caused by pilot fatigue.
Collapse
|
21
|
Albadawi Y, Takruri M, Awad M. A Review of Recent Developments in Driver Drowsiness Detection Systems. SENSORS 2022; 22:s22052069. [PMID: 35271215 PMCID: PMC8914892 DOI: 10.3390/s22052069] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Revised: 03/03/2022] [Accepted: 03/04/2022] [Indexed: 02/01/2023]
Abstract
Continuous advancements in computing technology and artificial intelligence in the past decade have led to improvements in driver monitoring systems. Numerous experimental studies have collected real driver drowsiness data and applied various artificial intelligence algorithms and feature combinations with the goal of significantly enhancing the performance of these systems in real-time. This paper presents an up-to-date review of the driver drowsiness detection systems implemented over the last decade. The paper illustrates and reviews recent systems using different measures to track and detect drowsiness. Each system falls under one of four possible categories, based on the information used. Each system presented in this paper is associated with a detailed description of the features, classification algorithms, and used datasets. In addition, an evaluation of these systems is presented, in terms of the final classification accuracy, sensitivity, and precision. Furthermore, the paper highlights the recent challenges in the area of driver drowsiness detection, discusses the practicality and reliability of each of the four system types, and presents some of the future trends in the field.
Collapse
Affiliation(s)
- Yaman Albadawi
- Department of Computer Science and Engineering, American University of Ras Al Khaimah, Ras Al Khaimah 72603, United Arab Emirates; (Y.A.); (M.A.)
| | - Maen Takruri
- Department of Electrical, Electronics and Communications Engineering, American University of Ras Al Khaimah, Ras Al Khaimah 72603, United Arab Emirates
- Correspondence:
| | - Mohammed Awad
- Department of Computer Science and Engineering, American University of Ras Al Khaimah, Ras Al Khaimah 72603, United Arab Emirates; (Y.A.); (M.A.)
| |
Collapse
|
22
|
Arefnezhad S, Hamet J, Eichberger A, Frühwirth M, Ischebeck A, Koglbauer IV, Moser M, Yousefi A. Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework. Sci Rep 2022; 12:2650. [PMID: 35173189 PMCID: PMC8850607 DOI: 10.1038/s41598-022-05810-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 01/14/2022] [Indexed: 01/22/2023] Open
Abstract
Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.
Collapse
Affiliation(s)
- Sadegh Arefnezhad
- Institute of Automotive Engineering, Graz University of Technology, 8010, Graz, Austria.
| | - James Hamet
- Neurable Company, Boston, MA, 02108, USA.,Vistim Labs Company, Salt Lake City, UT, 84103, USA
| | - Arno Eichberger
- Institute of Automotive Engineering, Graz University of Technology, 8010, Graz, Austria
| | | | - Anja Ischebeck
- Institute of Psychology, University of Graz, 8010, Graz, Austria
| | - Ioana Victoria Koglbauer
- Institute of Engineering and Business Informatics, Graz University of Technology, Graz, 8010, Austria
| | - Maximilian Moser
- Human Research Institute, Weiz, 8160, Austria.,Chair of Department of Physiology, Medical University of Graz, 8036, Graz, Austria
| | - Ali Yousefi
- Neurable Company, Boston, MA, 02108, USA.,Department of Computer Science Worcester Polytechnic Institute, 100 Institute Road, MA, 01609, Worcester, USA
| |
Collapse
|
23
|
Hasan MM, Watling CN, Larue GS. Physiological signal-based drowsiness detection using machine learning: Singular and hybrid signal approaches. JOURNAL OF SAFETY RESEARCH 2022; 80:215-225. [PMID: 35249601 DOI: 10.1016/j.jsr.2021.12.001] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 05/18/2021] [Accepted: 12/02/2021] [Indexed: 06/14/2023]
Abstract
INTRODUCTION Drowsiness is one of the main contributors to road-related crashes and fatalities worldwide. To address this pressing global issue, researchers are continuing to develop driver drowsiness detection systems that use a variety of measures. However, most research on drowsiness detection uses approaches based on a singular metric and, as a result, fail to attain satisfactory reliability and validity to be implemented in vehicles. METHOD This study examines the utility of drowsiness detection based on singular and a hybrid approach. This approach considered a range of metrics from three physiological signals - electroencephalography (EEG), electrooculography (EOG), and electrocardiography (ECG) - and used subjective sleepiness indices (assessed via the Karolinska Sleepiness Scale) as ground truth. The methodology consisted of signal recording with a psychomotor vigilance test (PVT), pre-processing, extracting, and determining the important features from the physiological signals for drowsiness detection. Finally, four supervised machine learning models were developed based on the subjective sleepiness responses using the extracted physiological features to detect drowsiness levels. RESULTS The results illustrate that the singular physiological measures show a specific performance metric pattern, with higher sensitivity and lower specificity or vice versa. In contrast, the hybrid biosignal-based models provide a better performance profile, reducing the disparity between the two metrics. CONCLUSIONS The outcome of the study indicates that the selected features provided higher performance in the hybrid approaches than the singular approaches, which could be useful for future research implications. Practical Applications: Use of a hybrid approach seems warranted to improve in-vehicle driver drowsiness detection system. Practical applications will need to consider factors such as intrusiveness, ergonomics, cost-effectiveness, and user-friendliness of any driver drowsiness detection system.
Collapse
Affiliation(s)
- Md Mahmudul Hasan
- Queensland University of Technology (QUT), Centre for Accident Research and Road Safety Queensland (CARRS-Q), Australia; Queensland University of Technology (QUT), Institute of Health and Biomedical Innovation (IHBI), Australia.
| | - Christopher N Watling
- Queensland University of Technology (QUT), Centre for Accident Research and Road Safety Queensland (CARRS-Q), Australia; Queensland University of Technology (QUT), Institute of Health and Biomedical Innovation (IHBI), Australia
| | - Grégoire S Larue
- Queensland University of Technology (QUT), Centre for Accident Research and Road Safety Queensland (CARRS-Q), Australia; Queensland University of Technology (QUT), Institute of Health and Biomedical Innovation (IHBI), Australia
| |
Collapse
|
24
|
Hao C, Wang R, Li M, Ma C, Cai Q, Gao Z. Convolutional neural network based on recurrence plot for EEG recognition. CHAOS (WOODBURY, N.Y.) 2021; 31:123120. [PMID: 34972327 DOI: 10.1063/5.0062242] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Accepted: 12/03/2021] [Indexed: 06/14/2023]
Abstract
Electroencephalogram (EEG) is a typical physiological signal. The classification of EEG signals is of great significance to human beings. Combining recurrence plot and convolutional neural network (CNN), we develop a novel method for classifying EEG signals. We select two typical EEG signals, namely, epileptic EEG and fatigue driving EEG, to verify the effectiveness of our method. We construct recurrence plots from EEG signals. Then, we build a CNN framework to classify the EEG signals under different brain states. For the classification of epileptic EEG signals, we design three different experiments to evaluate the performance of our method. The results suggest that the proposed framework can accurately distinguish the normal state and the seizure state of epilepsy. Similarly, for the classification of fatigue driving EEG signals, the method also has a good classification accuracy. In addition, we compare with the existing methods, and the results show that our method can significantly improve the detection results.
Collapse
Affiliation(s)
- Chongqing Hao
- School of Electrical Engineering, Hebei University of Science and Technology, Shijiazhuang, Hebei 050018, China
| | - Ruiqi Wang
- School of Electrical Engineering, Hebei University of Science and Technology, Shijiazhuang, Hebei 050018, China
| | - Mengyu Li
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Chao Ma
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Qing Cai
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| | - Zhongke Gao
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
| |
Collapse
|
25
|
Extended-Range Prediction Model Using NSGA-III Optimized RNN-GRU-LSTM for Driver Stress and Drowsiness. SENSORS 2021; 21:s21196412. [PMID: 34640732 PMCID: PMC8512694 DOI: 10.3390/s21196412] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 09/13/2021] [Accepted: 09/20/2021] [Indexed: 11/18/2022]
Abstract
Road traffic accidents have been listed in the top 10 global causes of death for many decades. Traditional measures such as education and legislation have contributed to limited improvements in terms of reducing accidents due to people driving in undesirable statuses, such as when suffering from stress or drowsiness. Attention is drawn to predicting drivers’ future status so that precautions can be taken in advance as effective preventative measures. Common prediction algorithms include recurrent neural networks (RNNs), gated recurrent units (GRUs), and long short-term memory (LSTM) networks. To benefit from the advantages of each algorithm, nondominated sorting genetic algorithm-III (NSGA-III) can be applied to merge the three algorithms. This is named NSGA-III-optimized RNN-GRU-LSTM. An analysis can be made to compare the proposed prediction algorithm with the individual RNN, GRU, and LSTM algorithms. Our proposed model improves the overall accuracy by 11.2–13.6% and 10.2–12.2% in driver stress prediction and driver drowsiness prediction, respectively. Likewise, it improves the overall accuracy by 6.9–12.7% and 6.9–8.9%, respectively, compared with boosting learning with multiple RNNs, multiple GRUs, and multiple LSTMs algorithms. Compared with existing works, this proposal offers to enhance performance by taking some key factors into account—namely, using a real-world driving dataset, a greater sample size, hybrid algorithms, and cross-validation. Future research directions have been suggested for further exploration and performance enhancement.
Collapse
|
26
|
Stancin I, Cifrek M, Jovic A. A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems. SENSORS 2021; 21:s21113786. [PMID: 34070732 PMCID: PMC8198610 DOI: 10.3390/s21113786] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Revised: 05/26/2021] [Accepted: 05/28/2021] [Indexed: 01/05/2023]
Abstract
Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.
Collapse
|
27
|
Arif S, Khan MJ, Naseer N, Hong KS, Sajid H, Ayaz Y. Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface. Front Hum Neurosci 2021; 15:658444. [PMID: 33994983 PMCID: PMC8121150 DOI: 10.3389/fnhum.2021.658444] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 03/09/2021] [Indexed: 11/13/2022] Open
Abstract
A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.
Collapse
Affiliation(s)
- Saad Arif
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Muhammad Jawad Khan
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| | - Noman Naseer
- Department of Mechatronics Engineering, Air University, Islamabad, Pakistan
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Hasan Sajid
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| | - Yasar Ayaz
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| |
Collapse
|
28
|
Balters S, Baker JM, Geeseman JW, Reiss AL. A Methodological Review of fNIRS in Driving Research: Relevance to the Future of Autonomous Vehicles. Front Hum Neurosci 2021; 15:637589. [PMID: 33967721 PMCID: PMC8100525 DOI: 10.3389/fnhum.2021.637589] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2020] [Accepted: 02/09/2021] [Indexed: 11/13/2022] Open
Abstract
As automobile manufacturers have begun to design, engineer, and test autonomous driving systems of the future, brain imaging with functional near-infrared spectroscopy (fNIRS) can provide unique insights about cognitive processes associated with evolving levels of autonomy implemented in the automobile. Modern fNIRS devices provide a portable, relatively affordable, and robust form of functional neuroimaging that allows researchers to investigate brain function in real-world environments. The trend toward "naturalistic neuroscience" is evident in the growing number of studies that leverage the methodological flexibility of fNIRS, and in doing so, significantly expand the scope of cognitive function that is accessible to observation via functional brain imaging (i.e., from the simulator to on-road scenarios). While more than a decade's worth of study in this field of fNIRS driving research has led to many interesting findings, the number of studies applying fNIRS during autonomous modes of operation is limited. To support future research that directly addresses this lack in autonomous driving research with fNIRS, we argue that a cogent distillation of the methods used to date will help facilitate and streamline this research of tomorrow. To that end, here we provide a methodological review of the existing fNIRS driving research, with the overarching goal of highlighting the current diversity in methodological approaches. We argue that standardization of these approaches will facilitate greater overlap of methods by researchers from all disciplines, which will, in-turn, allow for meta-analysis of future results. We conclude by providing recommendations for advancing the use of such fNIRS technology in furthering understanding the adoption of safe autonomous vehicle technology.
Collapse
Affiliation(s)
- Stephanie Balters
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, School of Medicine, Stanford University, Stanford, CA, United States
| | - Joseph M. Baker
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, School of Medicine, Stanford University, Stanford, CA, United States
| | | | - Allan L. Reiss
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, School of Medicine, Stanford University, Stanford, CA, United States
- Department of Radiology, School of Medicine, Stanford University, Stanford, CA, United States
- Department of Pediatrics, School of Medicine, Stanford University, Stanford, CA, United States
| |
Collapse
|
29
|
Liu Q, Liu Y, Chen K, Wang L, Li Z, Ai Q, Ma L. Research on Channel Selection and Multi-Feature Fusion of EEG Signals for Mental Fatigue Detection. ENTROPY (BASEL, SWITZERLAND) 2021; 23:457. [PMID: 33924528 PMCID: PMC8069717 DOI: 10.3390/e23040457] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 04/06/2021] [Accepted: 04/11/2021] [Indexed: 11/16/2022]
Abstract
With the rapid development of modern social science and technology, the pace of life is getting faster, and brain fatigue has become a sub-health state that seriously affects the normal life of people. Electroencephalogram (EEG) signals reflect changes in the central nervous system. Using EEG signals to assess mental fatigue is a research hotspot in related fields. Most existing fatigue detection methods are time-consuming or don't achieve satisfactory results due to insufficient features extracted from EEG signals. In this paper, a 2-back task is designed to induce fatigue. The weight value of each channel under a single feature is calculated by ReliefF algorithm. The classification accuracy of each channel under the corresponding features is analyzed. The classification accuracy of each single channel is combined to perform weighted summation to obtain the weight value of each channel. The first half channels sorted in descending order based on the weight value is chosen as the common channels. Multi-features in frequency and time domains are extracted from the common channel data, and the sparse representation method is used to perform feature fusion to obtain sparse fused features. Finally, the SRDA classifier is used to detect the fatigue state. Experimental results show that the proposed methods in our work effectively reduce the number of channels for computation and also improve the mental fatigue detection accuracy.
Collapse
Affiliation(s)
| | | | - Kun Chen
- School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; (Q.L.); (Y.L.); (L.W.); (Z.L.); (Q.A.); (L.M.)
| | | | | | | | | |
Collapse
|
30
|
An EEG-Based Transfer Learning Method for Cross-Subject Fatigue Mental State Prediction. SENSORS 2021; 21:s21072369. [PMID: 33805522 PMCID: PMC8036954 DOI: 10.3390/s21072369] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Revised: 03/22/2021] [Accepted: 03/24/2021] [Indexed: 01/26/2023]
Abstract
Fatigued driving is one of the main causes of traffic accidents. The electroencephalogram (EEG)-based mental state analysis method is an effective and objective way of detecting fatigue. However, as EEG shows significant differences across subjects, effectively “transfering” the EEG analysis model of the existing subjects to the EEG signals of other subjects is still a challenge. Domain-Adversarial Neural Network (DANN) has excellent performance in transfer learning, especially in the fields of document analysis and image recognition, but has not been applied directly in EEG-based cross-subject fatigue detection. In this paper, we present a DANN-based model, Generative-DANN (GDANN), which combines Generative Adversarial Networks (GAN) to enhance the ability by addressing the issue of different distribution of EEG across subjects. The comparative results show that in the analysis of cross-subject tasks, GDANN has a higher average accuracy of 91.63% in fatigue detection across subjects than those of traditional classification models, which is expected to have much broader application prospects in practical brain–computer interaction (BCI).
Collapse
|
31
|
Dang W, Gao Z, Lv D, Sun X, Cheng C. Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue. IEEE J Biomed Health Inform 2021; 25:693-700. [PMID: 32750954 DOI: 10.1109/jbhi.2020.3008229] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Fatigue driving has attracted a great deal of attention for its huge influence on automobile accidents. Recognizing driving fatigue provides a primary but significant way for addressing this problem. In this paper, we first conduct the simulated driving experiments to acquire the EEG signals in alert and fatigue states. Then, for multi-channel EEG signals without pre-processing, a novel rhythm-dependent multilayer brain network (RDMB network) is developed and analyzed for driving fatigue detection. We find that there exists a significant difference between alert and fatigue states from the view of network science. Further, key sub-RDMB network based on closeness centrality are extracted. We calculate six network measures from the key sub-RDMB network and construct feature vectors to classify the alert and fatigue states. The results show that our method can respectively achieve the average accuracy of 95.28% (with sample length of 5 s), 90.25% (2 s), and 87.69% (1 s), significantly higher than compared methods. All these validate the effectiveness of RDMB network for reliable driving fatigue detection via EEG.
Collapse
|
32
|
Chaabene S, Bouaziz B, Boudaya A, Hökelmann A, Ammar A, Chaari L. Convolutional Neural Network for Drowsiness Detection Using EEG Signals. SENSORS (BASEL, SWITZERLAND) 2021; 21:1734. [PMID: 33802357 PMCID: PMC7959292 DOI: 10.3390/s21051734] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Revised: 02/04/2021] [Accepted: 02/24/2021] [Indexed: 12/18/2022]
Abstract
Drowsiness detection (DD) has become a relevant area of active research in biomedical signal processing. Recently, various deep learning (DL) researches based on the EEG signals have been proposed to detect fatigue conditions. The research presented in this paper proposes an EEG classification system for DD based on DL networks. However, the proposed DD system is mainly realized into two procedures; (i) data acquisition and (ii) model analysis. For the data acquisition procedure, two key steps are considered, which are the signal collection using a wearable Emotiv EPOC+ headset to record 14 channels of EEG, and the signal annotation. Furthermore, a data augmentation (DA) step has been added to the proposed system to overcome the problem of over-fitting and to improve accuracy. As regards the model analysis, a comparative study is also introduced in this paper to argue the choice of DL architecture and frameworks used in our DD system. In this sense, The proposed DD protocol makes use of a convolutional neural network (CNN) architecture implemented using the Keras library. The results showed a high accuracy value (90.42%) in drowsy/awake discrimination and revealed the efficiency of the proposed DD system compared to other research works.
Collapse
Affiliation(s)
- Siwar Chaabene
- Multimedia InfoRmation Systems and Advanced Computing Laboratory (MIRACL), University of Sfax, Sfax 3021, Tunisia; (S.C.); (B.B.); (A.B.)
- Digital Research Center of Sfax, B.P. 275, Sakiet Ezzit, Sfax 3021, Tunisia
| | - Bassem Bouaziz
- Multimedia InfoRmation Systems and Advanced Computing Laboratory (MIRACL), University of Sfax, Sfax 3021, Tunisia; (S.C.); (B.B.); (A.B.)
- Digital Research Center of Sfax, B.P. 275, Sakiet Ezzit, Sfax 3021, Tunisia
| | - Amal Boudaya
- Multimedia InfoRmation Systems and Advanced Computing Laboratory (MIRACL), University of Sfax, Sfax 3021, Tunisia; (S.C.); (B.B.); (A.B.)
- Digital Research Center of Sfax, B.P. 275, Sakiet Ezzit, Sfax 3021, Tunisia
| | - Anita Hökelmann
- Institute of Sport Science, Otto-von-Guericke University Magdeburg, 39104 Magdeburg, Germany;
| | - Achraf Ammar
- Institute of Sport Science, Otto-von-Guericke University Magdeburg, 39104 Magdeburg, Germany;
- Interdisciplinary Laboratory in Neurosciences, Physiology and Psychology: Physical Activity, Health and Learning (LINP2), UFR STAPS, UPL, Paris Nanterre University, 92000 Nanterre, France
| | - Lotfi Chaari
- IRIT-ENSEEIHT, University of Toulouse, 31013 Toulouse, France;
| |
Collapse
|
33
|
Deligani RJ, Borgheai SB, McLinden J, Shahriari Y. Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework. BIOMEDICAL OPTICS EXPRESS 2021; 12:1635-1650. [PMID: 33796378 PMCID: PMC7984774 DOI: 10.1364/boe.413666] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Revised: 02/15/2021] [Accepted: 02/16/2021] [Indexed: 05/26/2023]
Abstract
Multimodal data fusion is one of the current primary neuroimaging research directions to overcome the fundamental limitations of individual modalities by exploiting complementary information from different modalities. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are especially compelling modalities due to their potentially complementary features reflecting the electro-hemodynamic characteristics of neural responses. However, the current multimodal studies lack a comprehensive systematic approach to properly merge the complementary features from their multimodal data. Identifying a systematic approach to properly fuse EEG-fNIRS data and exploit their complementary potential is crucial in improving performance. This paper proposes a framework for classifying fused EEG-fNIRS data at the feature level, relying on a mutual information-based feature selection approach with respect to the complementarity between features. The goal is to optimize the complementarity, redundancy and relevance between multimodal features with respect to the class labels as belonging to a pathological condition or healthy control. Nine amyotrophic lateral sclerosis (ALS) patients and nine controls underwent multimodal data recording during a visuo-mental task. Multiple spectral and temporal features were extracted and fed to a feature selection algorithm followed by a classifier, which selected the optimized subset of features through a cross-validation process. The results demonstrated considerably improved hybrid classification performance compared to the individual modalities and compared to conventional classification without feature selection, suggesting a potential efficacy of our proposed framework for wider neuro-clinical applications.
Collapse
Affiliation(s)
- Roohollah Jafari Deligani
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - Seyyed Bahram Borgheai
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - John McLinden
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
| | - Yalda Shahriari
- Department of Electrical, Computer and
Biomedical Engineering; University of Rhode
Island, Kingston, RI 02881, USA
- Interdisciplinary Neuroscience Program;
University of Rhode Island, Kingston, RI
02881, USA
| |
Collapse
|
34
|
Geissler CF, Schneider J, Frings C. Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study. Sci Rep 2021; 11:705. [PMID: 33436950 PMCID: PMC7804012 DOI: 10.1038/s41598-020-80477-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Accepted: 12/18/2020] [Indexed: 11/09/2022] Open
Abstract
Optimal mental workload plays a key role in driving performance. Thus, driver-assisting systems that automatically adapt to a drivers current mental workload via brain-computer interfacing might greatly contribute to traffic safety. To design economic brain computer interfaces that do not compromise driver comfort, it is necessary to identify brain areas that are most sensitive to mental workload changes. In this study, we used functional near-infrared spectroscopy and subjective ratings to measure mental workload in two virtual driving environments with distinct demands. We found that demanding city environments induced both higher subjective workload ratings as well as higher bilateral middle frontal gyrus activation than less demanding country environments. A further analysis with higher spatial resolution revealed a center of activation in the right anterior dorsolateral prefrontal cortex. The area is highly involved in spatial working memory processing. Thus, a main component of drivers' mental workload in complex surroundings might stem from the fact that large amounts of spatial information about the course of the road as well as other road users has to constantly be upheld, processed and updated. We propose that the right middle frontal gyrus might be a suitable region for the application of powerful small-area brain computer interfaces.
Collapse
Affiliation(s)
| | - Jörn Schneider
- Department of Computer Science, Trier University of Applied Sciences, Schneidershof, 54293, Trier, Germany
| | - Christian Frings
- Department of Cognitive Psychology, University of Trier, 54286, Trier, Germany
| |
Collapse
|
35
|
Belkhiria C, Peysakhovich V. Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020). FRONTIERS IN NEUROERGONOMICS 2020; 1:606719. [PMID: 38234309 PMCID: PMC10790927 DOI: 10.3389/fnrgo.2020.606719] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 11/17/2020] [Indexed: 01/19/2024]
Abstract
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
Collapse
|
36
|
Ismail LE, Karwowski W. Applications of EEG indices for the quantification of human cognitive performance: A systematic review and bibliometric analysis. PLoS One 2020; 15:e0242857. [PMID: 33275632 PMCID: PMC7717519 DOI: 10.1371/journal.pone.0242857] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Accepted: 11/10/2020] [Indexed: 12/02/2022] Open
Abstract
BACKGROUND Neuroergonomics combines neuroscience with ergonomics to study human performance using recorded brain signals. Such neural signatures of performance can be measured using a variety of neuroimaging techniques, including functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), and electroencephalography (EEG). EEG has an excellent temporal resolution, and EEG indices are highly sensitive to human brain activity fluctuations. OBJECTIVE The focus of this systematic review was to explore the applications of EEG indices for quantifying human performance in a variety of cognitive tasks at the macro and micro scales. To identify trends and the state of the field, we examined global patterns among selected articles, such as journal contributions, highly cited papers, affiliations, and high-frequency keywords. Moreover, we discussed the most frequently used EEG indices and synthesized current knowledge regarding the EEG signatures of associated human performance measurements. METHODS In this systematic review, we analyzed articles published in English (from peer-reviewed journals, proceedings, and conference papers), Ph.D. dissertations, textbooks, and reference books. All articles reviewed herein included exclusively EEG-based experimental studies in healthy participants. We searched Web-of-Science and Scopus databases using specific sets of keywords. RESULTS Out of 143 papers, a considerable number of cognitive studies focused on quantifying human performance with respect to mental fatigue, mental workload, mental effort, visual fatigue, emotion, and stress. An increasing trend for publication in this area was observed, with the highest number of publications in 2017. Most studies applied linear methods (e.g., EEG power spectral density and the amplitude of event-related potentials) to evaluate human cognitive performance. A few papers utilized nonlinear methods, such as fractal dimension, largest Lyapunov exponent, and signal entropy. More than 50% of the studies focused on evaluating an individual's mental states while operating a vehicle. Several different methods of artifact removal have also been noted. Based on the reviewed articles, research gaps, trends, and potential directions for future research were explored. CONCLUSION This systematic review synthesized current knowledge regarding the application of EEG indices for quantifying human performance in a wide variety of cognitive tasks. This knowledge is useful for understanding the global patterns of applications of EEG indices for the analysis and design of cognitive tasks.
Collapse
Affiliation(s)
- Lina Elsherif Ismail
- Department of Industrial Engineering and Management Systems, Computational Neuroergonomics Laboratory, University of Central Florida, Orlando, FL, United States of America
| | - Waldemar Karwowski
- Department of Industrial Engineering and Management Systems, Computational Neuroergonomics Laboratory, University of Central Florida, Orlando, FL, United States of America
| |
Collapse
|
37
|
Kaveh R, Doong J, Zhou A, Schwendeman C, Gopalan K, Burghardt FL, Arias AC, Maharbiz MM, Muller R. Wireless User-Generic Ear EEG. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2020; 14:727-737. [PMID: 32746342 DOI: 10.1109/tbcas.2020.3001265] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
In the past few years it has been demonstrated that electroencephalography (EEG) can be recorded from inside the ear (in-ear EEG). To open the door to low-profile earpieces as wearable brain-computer interfaces (BCIs), this work presents a practical in-ear EEG device based on multiple dry electrodes, a user-generic design, and a lightweight wireless interface for streaming data and device programming. The earpiece is designed for improved ear canal contact across a wide population of users and is fabricated in a low-cost and scalable manufacturing process based on standard techniques such as vacuum forming, plasma-treatment, and spray coating. A 2.5 × 2.5 cm2 wireless recording module is designed to record and stream data wirelessly to a host computer. Performance was evaluated on three human subjects over three months and compared with clinical-grade wet scalp EEG recordings. Recordings of spontaneous and evoked physiological signals, eye-blinks, alpha rhythm, and the auditory steady-state response (ASSR), are presented. This is the first wireless in-ear EEG to our knowledge to incorporate a dry multielectrode, user-generic design. The user-generic ear EEG recorded a mean alpha modulation of 2.17, outperforming the state-of-the-art in dry electrode in-ear EEG systems.
Collapse
|
38
|
Zhu Y, Rodriguez-Paras C, Rhee J, Mehta RK. Methodological Approaches and Recommendations for Functional Near-Infrared Spectroscopy Applications in HF/E Research. HUMAN FACTORS 2020; 62:613-642. [PMID: 31107601 DOI: 10.1177/0018720819845275] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
OBJECTIVE The objective of this study was to systematically document current methods and protocols employed when using functional near-infrared spectroscopy (fNIRS) techniques in human factors and ergonomics (HF/E) research and generate recommendations for conducting and reporting fNIRS findings in HF/E applications. METHOD A total of 1,687 articles were identified through Ovid-MEDLINE, PubMed, Web of Science, and Scopus databases, of which 37 articles were included in the review based on review inclusion/exclusion criteria. RESULTS A majority of the HF/E fNIRS investigations were found in transportation, both ground and aviation, and in assessing cognitive (e.g., workload, working memory) over physical constructs. There were large variations pertaining to data cleaning, processing, and analysis approaches across the studies that warrant standardization of methodological approaches. The review identified major challenges in transparency and reporting of important fNIRS data collection and analyses specifications that diminishes study replicability, introduces potential biases, and increases likelihood of inaccurate results. As such, results reported in existing fNIRS studies need to be cautiously approached. CONCLUSION To improve the quality of fNIRS investigations and/or to facilitate its adoption and integration in different HF/E applications, such as occupational ergonomics and rehabilitation, recommendations for fNIRS data collection, processing, analysis, and reporting are provided.
Collapse
Affiliation(s)
- Yibo Zhu
- 14736 Texas A&M University, College Station, USA
| | | | - Joohyun Rhee
- 14736 Texas A&M University, College Station, USA
| | | |
Collapse
|
39
|
Driving Drowsiness Detection with EEG Using a Modified Hierarchical Extreme Learning Machine Algorithm with Particle Swarm Optimization: A Pilot Study. ELECTRONICS 2020. [DOI: 10.3390/electronics9050775] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Driving fatigue accounts for a large number of traffic accidents in modern life nowadays. It is therefore of great importance to reduce this risky factor by detecting the driver’s drowsiness condition. This study aimed to detect drivers’ drowsiness using an advanced electroencephalography (EEG)-based classification technique. We first collected EEG data from six healthy adults under two different awareness conditions (wakefulness and drowsiness) in a virtual driving experiment. Five different machine learning techniques, including the K-nearest neighbor (KNN), support vector machine (SVM), extreme learning machine (ELM), hierarchical extreme learning machine (H-ELM), and the proposed modified hierarchical extreme learning machine algorithm with particle swarm optimization (PSO-H-ELM), were applied to classify the subject’s drowsiness based on the power spectral density (PSD) feature extracted from the EEG data. The mean accuracies of the five classifiers were 79.31%, 79.31%, 74.08%, 81.67%, and 83.12%, respectively, demonstrating the superior performance of our new PSO-H-ELM algorithm in detecting drivers’ drowsiness, compared to the other techniques.
Collapse
|
40
|
Guragain B, Rad AB, Wang C, Verma AK, Archer L, Wilson N, Tavakolian K. EEG-based Classification of Microsleep by Means of Feature Selection: An Application in Aviation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:4060-4063. [PMID: 31946764 DOI: 10.1109/embc.2019.8856429] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper presents a method for classification of microsleep (MS) from baseline utilizing linear and non-linear features derived from electroencephalography (EEG), which is recorded from five brain regions: frontal, central, parietal, occipital, and temporal. The EEG is acquired from sixteen commercially-rated pilots during the window of circadian low (2:00 am-6:00 am). MS events are annotated using the Driver Monitoring System and further verified using electrooculogram (EOG). A total of 55 features are extracted from EEG. A subset of these features is then selected using a wrapper-based method. The selected features are fed into a linear or quadratic discriminant analysis (LDA or QDA) classifier to automatically differentiate baseline from MS states. The overall classification performance of the best-proposed algorithm is 87.11% in terms of F1 score. This preliminary result highlights the potential of the proposed method towards automatic drowsiness detection which could assist mitigating aviation accidents in the future, pending hardware development to record such EEG signals from the confines of the aviation headset.
Collapse
|
41
|
Jeong JH, Yu BW, Lee DH, Lee SW. Classification of Drowsiness Levels Based on a Deep Spatio-Temporal Convolutional Bidirectional LSTM Network Using Electroencephalography Signals. Brain Sci 2019; 9:E348. [PMID: 31795445 PMCID: PMC6956039 DOI: 10.3390/brainsci9120348] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2019] [Revised: 11/22/2019] [Accepted: 11/26/2019] [Indexed: 11/16/2022] Open
Abstract
Non-invasive brain-computer interfaces (BCI) have been developed for recognizing human mental states with high accuracy and for decoding various types of mental conditions. In particular, accurately decoding a pilot's mental state is a critical issue as more than 70% of aviation accidents are caused by human factors, such as fatigue or drowsiness. In this study, we report the classification of not only two mental states (i.e., alert and drowsy states) but also five drowsiness levels from electroencephalogram (EEG) signals. To the best of our knowledge, this approach is the first to classify drowsiness levels in detail using only EEG signals. We acquired EEG data from ten pilots in a simulated night flight environment. For accurate detection, we proposed a deep spatio-temporal convolutional bidirectional long short-term memory network (DSTCLN) model. We evaluated the classification performance using Karolinska sleepiness scale (KSS) values for two mental states and five drowsiness levels. The grand-averaged classification accuracies were 0.87 (±0.01) and 0.69 (±0.02), respectively. Hence, we demonstrated the feasibility of classifying five drowsiness levels with high accuracy using deep learning.
Collapse
Affiliation(s)
- Ji-Hoon Jeong
- Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea; (J.-H.J.); (B.-W.Y.); (D.-H.L.)
| | - Baek-Woon Yu
- Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea; (J.-H.J.); (B.-W.Y.); (D.-H.L.)
| | - Dae-Hyeok Lee
- Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea; (J.-H.J.); (B.-W.Y.); (D.-H.L.)
| | - Seong-Whan Lee
- Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea; (J.-H.J.); (B.-W.Y.); (D.-H.L.)
- Department of Artificial Intelligence, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea
| |
Collapse
|
42
|
Applications of Functional Near-Infrared Spectroscopy in Fatigue, Sleep Deprivation, and Social Cognition. Brain Topogr 2019; 32:998-1012. [DOI: 10.1007/s10548-019-00740-w] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2019] [Accepted: 10/18/2019] [Indexed: 01/05/2023]
|
43
|
Nguyen T, Kim M, Gwak J, Lee JJ, Choi KY, Lee KH, Kim JG. Investigation of brain functional connectivity in patients with mild cognitive impairment: A functional near-infrared spectroscopy (fNIRS) study. JOURNAL OF BIOPHOTONICS 2019; 12:e201800298. [PMID: 30963713 DOI: 10.1002/jbio.201800298] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2018] [Revised: 03/17/2019] [Accepted: 04/04/2019] [Indexed: 06/09/2023]
Abstract
This study examines brain functional connectivity in both cognitively normal seniors and patients with mild cognitive impairment (MCI) to elucidate prospective markers of MCI. A homemade four-channel functional near-infrared spectroscopy (fNIRS) system was employed to measure hemodynamic responses in the subjects' prefrontal cortex during a resting state, an oddball task, a 1-back task, and a verbal fluency task. Brain functional connectivity was calculated as the Pearson correlation coefficients between fNIRS channels. The results show that during the verbal fluency task, while the healthy control (HC) group presents a significantly stronger inter-hemispheric connectivity compared to intra-hemispheric connectivity, there is no difference between the inter- and intra-hemispheric connectivity in the MCI group. In addition, a comparison between the MCI and HC connectivity reveals that the MCI group has a statistically higher right and inter-hemispheric connectivity during the resting state, but a significantly lower left and inter-hemispheric connectivity during the verbal fluency test. These findings demonstrate the potential of fNIRS to study brain functional connectivity in neurodegenerative diseases.
Collapse
Affiliation(s)
- Thien Nguyen
- Department of Biomedical Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Minhee Kim
- Department of Biomedical Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| | - Jeonghwan Gwak
- Department of Software, Korea National University of Transportation, Chungju, Republic of Korea
| | - Jang J Lee
- Department of Biomedical Science, Chosun University, Gwangju, Republic of Korea
| | - Kyu Y Choi
- Department of Biomedical Science, Chosun University, Gwangju, Republic of Korea
| | - Kun H Lee
- Department of Biomedical Science, Chosun University, Gwangju, Republic of Korea
- National Research Center for Dementia, Chosun University, Gwangju, Republic of Korea
| | - Jae G Kim
- Department of Biomedical Science and Engineering, Gwangju Institute of Science and Technology, Gwangju, Republic of Korea
| |
Collapse
|
44
|
Hertig-Godeschalk A, Skorucak J, Malafeev A, Achermann P, Mathis J, Schreier DR. Microsleep episodes in the borderland between wakefulness and sleep. Sleep 2019; 43:5536744. [DOI: 10.1093/sleep/zsz163] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2018] [Revised: 05/16/2019] [Indexed: 11/14/2022] Open
Abstract
AbstractStudy objectivesThe wake-sleep transition zone represents a poorly defined borderland, containing, for example, microsleep episodes (MSEs), which are of potential relevance for diagnosis and may have consequences while driving. Yet, the scoring guidelines of the American Academy of Sleep Medicine (AASM) completely neglect it. We aimed to explore the borderland between wakefulness and sleep by developing the Bern continuous and high-resolution wake-sleep (BERN) criteria for visual scoring, focusing on MSEs visible in the electroencephalography (EEG), as opposed to purely behavior- or performance-defined MSEs.MethodsMaintenance of Wakefulness Test (MWT) trials of 76 randomly selected patients were retrospectively scored according to both the AASM and the newly developed BERN scoring criteria. The visual scoring was compared with spectral analysis of the EEG. The quantitative EEG analysis enabled a reliable objectification of the visually scored MSEs. For less distinct episodes within the borderland, either ambiguous or no quantitative patterns were found.ResultsAs expected, the latency to the first MSE was significantly shorter in comparison to the sleep latency, defined according to the AASM criteria. In certain cases, a large difference between the two latencies was observed and a substantial number of MSEs occurred between the first MSE and sleep. Series of MSEs were more frequent in patients with shorter sleep latencies, while isolated MSEs were more frequent in patients who did not reach sleep.ConclusionThe BERN criteria extend the AASM criteria and represent a valuable tool for in-depth analysis of the wake-sleep transition zone, particularly important in the MWT.
Collapse
Affiliation(s)
- Anneke Hertig-Godeschalk
- Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Jelena Skorucak
- Institute of Pharmacology and Toxicology, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
- Sleep and Health Zurich, University of Zurich, Zurich, Switzerland
| | - Alexander Malafeev
- Institute of Pharmacology and Toxicology, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Peter Achermann
- Institute of Pharmacology and Toxicology, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
- Sleep and Health Zurich, University of Zurich, Zurich, Switzerland
- KEY Institute for Brain Mind Research, Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland
| | - Johannes Mathis
- Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - David R Schreier
- Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Medicine, Spital STS AG Thun, Thun, Switzerland
| |
Collapse
|
45
|
Ma Y, Chen B, Li R, Wang C, Wang J, She Q, Luo Z, Zhang Y. Driving Fatigue Detection from EEG Using a Modified PCANet Method. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:4721863. [PMID: 31396270 PMCID: PMC6664732 DOI: 10.1155/2019/4721863] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Revised: 04/28/2019] [Accepted: 06/19/2019] [Indexed: 11/29/2022]
Abstract
The rapid development of the automotive industry has brought great convenience to our life, which also leads to a dramatic increase in the amount of traffic accidents. A large proportion of traffic accidents were caused by driving fatigue. EEG is considered as a direct, effective, and promising modality to detect driving fatigue. In this study, we presented a novel feature extraction strategy based on a deep learning model to achieve high classification accuracy and efficiency in using EEG for driving fatigue detection. EEG signals were recorded from six healthy volunteers in a simulated driving experiment. The feature extraction strategy was developed by integrating the principal component analysis (PCA) and a deep learning model called PCA network (PCANet). In particular, the principal component analysis (PCA) was used to preprocess EEG data to reduce its dimension in order to overcome the limitation of dimension explosion caused by PCANet, making this approach feasible for EEG-based driving fatigue detection. Results demonstrated high and robust performance of the proposed modified PCANet method with classification accuracy up to 95%, which outperformed the conventional feature extraction strategies in the field. We also identified that the parietal and occipital lobes of the brain were strongly associated with driving fatigue. This is the first study, to the best of our knowledge, to practically apply the modified PCANet technique for EEG-based driving fatigue detection.
Collapse
Affiliation(s)
- Yuliang Ma
- Intelligent Control & Robotics Institute, College of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Bin Chen
- Intelligent Control & Robotics Institute, College of Automation, Hangzhou Dianzi University, Hangzhou, China
- Department of Biomedical Engineering, University of Houston, Houston, Texas, USA
| | - Rihui Li
- Department of Biomedical Engineering, University of Houston, Houston, Texas, USA
| | - Chushan Wang
- Guangdong Provincial Work Injury Rehabilitation Hospital, Guangzhou, China
| | - Jun Wang
- Guangdong Provincial Work Injury Rehabilitation Hospital, Guangzhou, China
| | - Qingshan She
- Intelligent Control & Robotics Institute, College of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Zhizeng Luo
- Intelligent Control & Robotics Institute, College of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, Texas, USA
| |
Collapse
|
46
|
Kumar S, Sharma A, Tsunoda T. Brain wave classification using long short-term memory network based OPTICAL predictor. Sci Rep 2019; 9:9153. [PMID: 31235800 PMCID: PMC6591300 DOI: 10.1038/s41598-019-45605-1] [Citation(s) in RCA: 44] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Accepted: 06/07/2019] [Indexed: 11/09/2022] Open
Abstract
Brain-computer interface (BCI) systems having the ability to classify brain waves with greater accuracy are highly desirable. To this end, a number of techniques have been proposed aiming to be able to classify brain waves with high accuracy. However, the ability to classify brain waves and its implementation in real-time is still limited. In this study, we introduce a novel scheme for classifying motor imagery (MI) tasks using electroencephalography (EEG) signal that can be implemented in real-time having high classification accuracy between different MI tasks. We propose a new predictor, OPTICAL, that uses a combination of common spatial pattern (CSP) and long short-term memory (LSTM) network for obtaining improved MI EEG signal classification. A sliding window approach is proposed to obtain the time-series input from the spatially filtered data, which becomes input to the LSTM network. Moreover, instead of using LSTM directly for classification, we use regression based output of the LSTM network as one of the features for classification. On the other hand, linear discriminant analysis (LDA) is used to reduce the dimensionality of the CSP variance based features. The features in the reduced dimensional plane after performing LDA are used as input to the support vector machine (SVM) classifier together with the regression based feature obtained from the LSTM network. The regression based feature further boosts the performance of the proposed OPTICAL predictor. OPTICAL showed significant improvement in the ability to accurately classify left and right-hand MI tasks on two publically available datasets. The improvements in the average misclassification rates are 3.09% and 2.07% for BCI Competition IV Dataset I and GigaDB dataset, respectively. The Matlab code is available at https://github.com/ShiuKumar/OPTICAL .
Collapse
Affiliation(s)
- Shiu Kumar
- The University of the South Pacific, Suva, Fiji. .,Fiji National University, Suva, Fiji.
| | - Alok Sharma
- Institute for Integrated and Intelligent Systems, Griffith University, Brisbane, QLD-4111, Australia. .,Department of Medical Science Mathematics, Medical Research Institute, Tokyo Medical and Dental University, Tokyo, Japan. .,Laboratory for Medical Science Mathematics, RIKEN Center for Integrative Medical Sciences, Yokohama, Kanagawa, Japan. .,The University of the South Pacific, Suva, Fiji.
| | - Tatsuhiko Tsunoda
- Department of Medical Science Mathematics, Medical Research Institute, Tokyo Medical and Dental University, Tokyo, Japan.,Laboratory for Medical Science Mathematics, RIKEN Center for Integrative Medical Sciences, Yokohama, Kanagawa, Japan.,CREST, JST, Tokyo, 102-8666, Japan
| |
Collapse
|
47
|
Lin CT, King JT, Chuang CH, Ding W, Chuang WY, Liao LD, Wang YK. Exploring the Brain Responses to Driving Fatigue Through Simultaneous EEG and fNIRS Measurements. Int J Neural Syst 2019; 30:1950018. [PMID: 31366249 DOI: 10.1142/s0129065719500187] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Fatigue is one problem with driving as it can lead to difficulties with sustaining attention, behavioral lapses, and a tendency to ignore vital information or operations. In this research, we explore multimodal physiological phenomena in response to driving fatigue through simultaneous functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) recordings with the aim of investigating the relationships between hemodynamic and electrical features and driving performance. Sixteen subjects participated in an event-related lane-deviation driving task while measuring their brain dynamics through fNIRS and EEGs. Three performance groups, classified as Optimal, Suboptimal, and Poor, were defined for comparison. From our analysis, we find that tonic variations occur before a deviation, and phasic variations occur afterward. The tonic results show an increased concentration of oxygenated hemoglobin (HbO2) and power changes in the EEG theta, alpha, and beta bands. Both dynamics are significantly correlated with deteriorated driving performance. The phasic EEG results demonstrate event-related desynchronization associated with the onset of steering vehicle in all power bands. The concentration of phasic HbO2 decreased as performance worsened. Further, the negative correlations between tonic EEG delta and alpha power and HbO2 oscillations suggest that activations in HbO2 are related to mental fatigue. In summary, combined hemodynamic and electrodynamic activities can provide complete knowledge of the brain's responses as evidence of state changes during fatigue driving.
Collapse
Affiliation(s)
- Chin-Teng Lin
- CIBCI, Centre for Artificial Intelligence, FEIT, University of Technology Sydney, Sydney, Broadway, 15, Ultimo NSW 2007, Australia
| | - Jung-Tai King
- Brain Research Center, National Chiao Tung University, Hsinchu 300, Taiwan
| | - Chun-Hsiang Chuang
- Department of Computer Science and Engineering, National Taiwan Ocean University, Keelung 202, Taiwan
| | - Weiping Ding
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Wei-Yu Chuang
- Brain Research Center, National Chiao Tung University, Hsinchu 300, Taiwan
| | - Lun-De Liao
- Institute of Biomedical Engineering and Nanomedicine, National Health Research Institutes, Zhunan, 350, Taiwan
| | - Yu-Kai Wang
- CIBCI, Centre for Artificial Intelligence, FEIT, University of Technology Sydney, Sydney, Broadway, 15, Ultimo NSW 2007, Australia
| |
Collapse
|
48
|
Lee S, Shin Y, Kumar A, Kim M, Lee HN. Dry Electrode-Based Fully Isolated EEG/fNIRS Hybrid Brain-Monitoring System. IEEE Trans Biomed Eng 2019; 66:1055-1068. [DOI: 10.1109/tbme.2018.2866550] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
49
|
Oken B, Memmott T, Eddy B, Wiedrick J, Fried-Oken M. Vigilance state fluctuations and performance using brain-computer interface for communication. BRAIN-COMPUTER INTERFACES 2019; 5:146-156. [PMID: 31236425 DOI: 10.1080/2326263x.2019.1571356] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
The effect of fatigue and drowsiness on brain-computer interface (BCI) performance was evaluated. 20 healthy participants performed a standardized 11-minute calibration of a Rapid Serial Visual Presentation BCI system five times over two hours. For each calibration, BCI performance was evaluated using area under the receiver operating characteristic curve (AUC). Self-rated measures were obtained following each calibration including the Karolinska Sleepiness Scale and a standardized boredom scale. Physiological measures were obtained during each calibration including P300 amplitude, theta power, alpha power, median power frequency and eye-blink rate. There was a significant decrease in AUC over the five sessions. This was paralleled by increases in self-rated sleepiness and boredom and decreases in P300 amplitude. Alpha power, median power frequency, and eye-blink rate also increased but more modestly. AUC changes were only partly explained by changes in P300 amplitude. There was a decrease in BCI performance over time that related to increases in sleepiness and boredom. This worsened performance was only partly explained by decreases in P300 amplitude. Thus, drowsiness and boredom have a negative impact on BCI performance. Increased BCI performance may be possible by developing physiological measures to provide feedback to the user or to adapt the classifier to state.
Collapse
Affiliation(s)
- Barry Oken
- Department of Neurology, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239 USA
| | - Tab Memmott
- Department of Neurology, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239 USA
| | - Brandon Eddy
- Institute on Development and Disability, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239 USA
| | - Jack Wiedrick
- Biostatistics and Design Program, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239 USA
| | - Melanie Fried-Oken
- Institute on Development and Disability, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239 USA
| |
Collapse
|
50
|
Fujiwara K, Abe E, Kamata K, Nakayama C, Suzuki Y, Yamakawa T, Hiraoka T, Kano M, Sumi Y, Masuda F, Matsuo M, Kadotani H. Heart Rate Variability-Based Driver Drowsiness Detection and Its Validation With EEG. IEEE Trans Biomed Eng 2018; 66:1769-1778. [PMID: 30403616 DOI: 10.1109/tbme.2018.2879346] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
OBJECTIVE Driver drowsiness detection is a key technology that can prevent fatal car accidents caused by drowsy driving. The present work proposes a driver drowsiness detection algorithm based on heart rate variability (HRV) analysis and validates the proposed method by comparing with electroencephalography (EEG)-based sleep scoring. METHODS Changes in sleep condition affect the autonomic nervous system and then HRV, which is defined as an RR interval (RRI) fluctuation on an electrocardiogram trace. Eight HRV features are monitored for detecting changes in HRV by using multivariate statistical process control, which is a well known anomaly detection method. RESULT The performance of the proposed algorithm was evaluated through an experiment using a driving simulator. In this experiment, RRI data were measured from 34 participants during driving, and their sleep onsets were determined based on the EEG data by a sleep specialist. The validation result of the experimental data with the EEG data showed that drowsiness was detected in 12 out of 13 pre-N1 episodes prior to the sleep onsets, and the false positive rate was 1.7 times per hour. CONCLUSION The present work also demonstrates the usefulness of the framework of HRV-based anomaly detection that was originally proposed for epileptic seizure prediction. SIGNIFICANCE The proposed method can contribute to preventing accidents caused by drowsy driving.
Collapse
|