1
|
Callan DE, Torre–Tresols JJ, Laguerta J, Ishii S. Shredding artifacts: extracting brain activity in EEG from extreme artifacts during skateboarding using ASR and ICA. FRONTIERS IN NEUROERGONOMICS 2024; 5:1358660. [PMID: 38989056 PMCID: PMC11233536 DOI: 10.3389/fnrgo.2024.1358660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Accepted: 05/30/2024] [Indexed: 07/12/2024]
Abstract
Introduction To understand brain function in natural real-world settings, it is crucial to acquire brain activity data in noisy environments with diverse artifacts. Electroencephalography (EEG), while susceptible to environmental and physiological artifacts, can be cleaned using advanced signal processing techniques like Artifact Subspace Reconstruction (ASR) and Independent Component Analysis (ICA). This study aims to demonstrate that ASR and ICA can effectively extract brain activity from the substantial artifacts occurring while skateboarding on a half-pipe ramp. Methods A dual-task paradigm was used, where subjects were presented with auditory stimuli during skateboarding and rest conditions. The effectiveness of ASR and ICA in cleaning artifacts was evaluated using a support vector machine to classify the presence or absence of a sound stimulus in single-trial EEG data. The study evaluated the effectiveness of ASR and ICA in artifact cleaning using five different pipelines: (1) Minimal cleaning (bandpass filtering), (2) ASR only, (3) ICA only, (4) ICA followed by ASR (ICAASR), and (5) ASR preceding ICA (ASRICA). Three skateboarders participated in the experiment. Results Results showed that all ICA-containing pipelines, especially ASRICA (69%, 68%, 63%), outperformed minimal cleaning (55%, 52%, 50%) in single-trial classification during skateboarding. The ASRICA pipeline performed significantly better than other pipelines containing ICA for two of the three subjects, with no other pipeline performing better than ASRICA. The superior performance of ASRICA likely results from ASR removing non-stationary artifacts, enhancing ICA decomposition. Evidenced by ASRICA identifying more brain components via ICLabel than ICA alone or ICAASR for all subjects. For the rest condition, with fewer artifacts, the ASRICA pipeline (71%, 82%, 75%) showed slight improvement over minimal cleaning (73%, 70%, 72%), performing significantly better for two subjects. Discussion This study demonstrates that ASRICA can effectively clean artifacts to extract single-trial brain activity during skateboarding. These findings affirm the feasibility of recording brain activity during physically demanding tasks involving substantial body movement, laying the groundwork for future research into the neural processes governing complex and coordinated body movements.
Collapse
Affiliation(s)
- Daniel E. Callan
- Brain Information Communication Research Laboratory, Advanced Telecommunications Research Institute International, Kyoto, Japan
- Institut Supérieur de l'Aéronautique et de l'Espace, University of Toulouse, Toulouse, France
| | - Juan Jesus Torre–Tresols
- Brain Information Communication Research Laboratory, Advanced Telecommunications Research Institute International, Kyoto, Japan
- Institut Supérieur de l'Aéronautique et de l'Espace, University of Toulouse, Toulouse, France
| | - Jamie Laguerta
- Brain Information Communication Research Laboratory, Advanced Telecommunications Research Institute International, Kyoto, Japan
- Department of Integrated Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Shin Ishii
- Brain Information Communication Research Laboratory, Advanced Telecommunications Research Institute International, Kyoto, Japan
- Graduate School of Informatics, Kyoto University, Kyoto, Japan
| |
Collapse
|
2
|
He C, Chen YY, Phang CR, Stevenson C, Chen IP, Jung TP, Ko LW. Diversity and Suitability of the State-of-the-Art Wearable and Wireless EEG Systems Review. IEEE J Biomed Health Inform 2023; 27:3830-3843. [PMID: 37022001 DOI: 10.1109/jbhi.2023.3239053] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Wireless electroencephalography (EEG) systems have been attracting increasing attention in recent times. Both the number of articles discussing wireless EEG and their proportion relative to general EEG publications have increased over years. These trends indicate that wireless EEG systems could be more accessible to researchers and the research community has recognized the potential of wireless EEG systems. To explore the development and diverse applications of wireless EEG systems, this review highlights the trends in wearable and wireless EEG systems over the past decade and compares the specifications and research applications of the major wireless systems marketed by 16 companies. For each product, five parameters (number of channels, sampling rate, cost, battery life, and resolution) were assessed for comparison. Currently, these wearable and portable wireless EEG systems have three main application areas: consumer, clinical, and research. To address this multitude of options, the article also discussed the thought process to find a suitable device that meets personalization and use cases specificities. These investigations suggest that low-price and convenience are key factors for consumer applications, wireless EEG systems with FDA or CE-certification may be more suitable for clinical settings, and devices that provide raw EEG data with high-density channels are important for laboratory research. This article presents an overview of the current state of the wireless EEG systems specifications and possible applications and serves as a guide point as it is expected that more influential and novel research will cyclically promote the development of such EEG systems.
Collapse
|
3
|
Russo C, Senese VP. Functional near-infrared spectroscopy is a useful tool for multi-perspective psychobiological study of neurophysiological correlates of parenting behaviour. Eur J Neurosci 2023; 57:258-284. [PMID: 36485015 DOI: 10.1111/ejn.15890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 12/02/2022] [Accepted: 12/08/2022] [Indexed: 12/13/2022]
Abstract
The quality of the relationship between caregiver and child has long-term effects on the cognitive and socio-emotional development of children. A process involved in human parenting is the bio-behavioural synchrony that occurs between the partners in the relationship during interaction. Through interaction, bio-behavioural synchronicity allows the adaptation of the physiological systems of the parent to those of the child and promotes the positive development and modelling of the child's social brain. The role of bio-behavioural synchrony in building social bonds could be investigated using functional near-infrared spectroscopy (fNIRS). In this paper we have (a) highlighted the importance of the quality of the caregiver-child relationship for the child's cognitive and socio-emotional development, as well as the relevance of infantile stimuli in the activation of parenting behaviour; (b) discussed the tools used in the study of the neurophysiological substrates of the parental response; (c) proposed fNIRS as a particularly suitable tool for the study of parental responses; and (d) underlined the need for a multi-systemic psychobiological approach to understand the mechanisms that regulate caregiver-child interactions and their bio-behavioural synchrony. We propose to adopt a multi-system psychobiological approach to the study of parental behaviour and social interaction.
Collapse
Affiliation(s)
- Carmela Russo
- Psychometric Laboratory, Department of Psychology, University of Campania "Luigi Vanvitelli", Caserta, Italy
| | - Vincenzo Paolo Senese
- Psychometric Laboratory, Department of Psychology, University of Campania "Luigi Vanvitelli", Caserta, Italy
| |
Collapse
|
4
|
van Weelden E, Alimardani M, Wiltshire TJ, Louwerse MM. Aviation and neurophysiology: A systematic review. APPLIED ERGONOMICS 2022; 105:103838. [PMID: 35939991 DOI: 10.1016/j.apergo.2022.103838] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/09/2021] [Revised: 06/22/2022] [Accepted: 06/23/2022] [Indexed: 05/24/2023]
Abstract
This paper systematically reviews 20 years of publications (N = 54) on aviation and neurophysiology. The main goal is to provide an account of neurophysiological changes associated with flight training with the aim of identifying neurometrics indicative of pilot's flight training level and task relevant mental states, as well as to capture the current state-of-art of (neuro)ergonomic design and practice in flight training. We identified multiple candidate neurometrics of training progress and workload, such as frontal theta power, the EEG Engagement Index and the Cognitive Stability Index. Furthermore, we discovered that several types of classifiers could be used to accurately detect mental states, such as the detection of drowsiness and mental fatigue. The paper advances practical guidelines on terminology usage, simulator fidelity, and multimodality, as well as future research ideas including the potential of Virtual Reality flight simulations for training, and a brain-computer interface for flight training.
Collapse
Affiliation(s)
- Evy van Weelden
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands.
| | - Maryam Alimardani
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| | - Travis J Wiltshire
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| | - Max M Louwerse
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| |
Collapse
|
5
|
Dadebayev D, Goh WW, Tan EX. EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2022. [DOI: 10.1016/j.jksuci.2021.03.009] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
6
|
Singh G, Chanel CPC, Roy RN. Mental Workload Estimation Based on Physiological Features for Pilot-UAV Teaming Applications. Front Hum Neurosci 2021; 15:692878. [PMID: 34489660 PMCID: PMC8417701 DOI: 10.3389/fnhum.2021.692878] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Accepted: 07/27/2021] [Indexed: 11/24/2022] Open
Abstract
Manned-Unmanned Teaming (MUM-T) can be defined as the teaming of aerial robots (artificial agents) along with a human pilot (natural agent), in which the human agent is not an authoritative controller but rather a cooperative team player. To our knowledge, no study has yet evaluated the impact of MUM-T scenarios on operators' mental workload (MW) using a neuroergonomic approach (i.e., using physiological measures), nor provided a MW estimation through classification applied on those measures. Moreover, the impact of the non-stationarity of the physiological signal is seldom taken into account in classification pipelines, particularly regarding the validation design. Therefore this study was designed with two goals: (i) to characterize and estimate MW in a MUM-T setting based on physiological signals; (ii) to assess the impact of the validation procedure on classification accuracy. In this context, a search and rescue (S&R) scenario was developed in which 14 participants played the role of a pilot cooperating with three UAVs (Unmanned Aerial Vehicles). Missions were designed to induce high and low MW levels, which were evaluated using self-reported, behavioral and physiological measures (i.e., cerebral, cardiac, and oculomotor features). Supervised classification pipelines based on various combinations of these physiological features were benchmarked, and two validation procedures were compared (i.e., a traditional one that does not take time into account vs. an ecological one that does). The main results are: (i) a significant impact of MW on all measures, (ii) a higher intra-subject classification accuracy (75%) reached using ECG features alone or in combination with EEG and ET ones with the Adaboost, Linear Discriminant Analysis or the Support Vector Machine classifiers. However this was only true with the traditional validation. There was a significant drop in classification accuracy using the ecological one. Interestingly, inter-subject classification with ecological validation (59.8%) surpassed both intra-subject with ecological and inter-subject with traditional validation. These results highlight the need for further developments to perform MW monitoring in such operational contexts.
Collapse
Affiliation(s)
| | - Caroline P C Chanel
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France.,Artificial and Natural Intelligence Toulouse Institute - ANITI, Toulouse, France
| | - Raphaëlle N Roy
- ISAE-SUPAERO, Université de Toulouse, Toulouse, France.,Artificial and Natural Intelligence Toulouse Institute - ANITI, Toulouse, France
| |
Collapse
|
7
|
Belkhiria C, Peysakhovich V. Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020). FRONTIERS IN NEUROERGONOMICS 2020; 1:606719. [PMID: 38234309 PMCID: PMC10790927 DOI: 10.3389/fnrgo.2020.606719] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 11/17/2020] [Indexed: 01/19/2024]
Abstract
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
Collapse
|
8
|
Sasaki M, Iversen J, Callan DE. Music Improvisation Is Characterized by Increase EEG Spectral Power in Prefrontal and Perceptual Motor Cortical Sources and Can be Reliably Classified From Non-improvisatory Performance. Front Hum Neurosci 2019; 13:435. [PMID: 31920594 PMCID: PMC6915035 DOI: 10.3389/fnhum.2019.00435] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2019] [Accepted: 11/27/2019] [Indexed: 01/31/2023] Open
Abstract
This study expores neural activity underlying creative processes through the investigation of music improvisation. Fourteen guitar players with a high level of improvisation skill participated in this experiment. The experimental task involved playing 32-s alternating blocks of improvisation and scales on guitar. electroencephalography (EEG) data was measured continuously throughout the experiment. In order to remove potential artifacts and extract brain-related activity the following signal processing techniques were employed: bandpass filtering, Artifact Subspace Reconstruction, and Independent Component Analysis (ICA). For each participant, artifact related independent components (ICs) were removed from the EEG data and only ICs found to be from brain activity were retained. Source localization using this brain-related activity was carried out using sLORETA. Greater activity for improvisation over scale was found in multiple frequency bands (theta, alpha, and beta) localized primarily in the medial frontal cortex (MFC), Middle frontal gyrus (MFG), anterior cingulate, polar medial prefrontal cortex (MPFC), premotor cortex (PMC), pre and postcentral gyrus (PreCG and PostCG), superior temporal gyrus (STG), inferior parietal lobule (IPL), and the temporal-parietal junction. Together this collection of brain regions suggests that improvisation was mediated by processes involved in coordinating planned sequences of movement that are modulated in response to ongoing environmental context through monitoring and feedback of sensory states in relation to internal plans and goals. Machine-learning using Common Spatial Patterns (CSP) for EEG feature extraction attained a mean of over 75% classification performance for improvisation vs. scale conditions across participants. These machine-learning results are a step towards the development of a brain-computer interface that could be used for neurofeedback training to improve creativity.
Collapse
Affiliation(s)
- Masaru Sasaki
- Graduate School of Frontier Biosciences, Osaka University, Osaka, Japan
| | - John Iversen
- Swartz Center for Computational Neuroscience, University of California, San Diego, San Diego, CA, United States
| | - Daniel E Callan
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan
| |
Collapse
|
9
|
Marini F, Lee C, Wagner J, Makeig S, Gola M. A comparative evaluation of signal quality between a research-grade and a wireless dry-electrode mobile EEG system. J Neural Eng 2019; 16:054001. [PMID: 31096191 DOI: 10.1088/1741-2552/ab21f2] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
OBJECTIVE Electroencephalography (EEG) is widely used by clinicians, scientists, engineers and other professionals worldwide, with an increasing number of low-cost, commercially-oriented EEG systems that have become available in recent years. One such system is the Cognionics Quick-20 (Cognionics Inc., San Diego, USA), which uses dry electrodes and offers the convenience of portability thanks to its built-in amplifier and wireless connection. Because of such characteristics, this system has been used in several applications for both clinical and basic research studies. However, an investigation of the quality of the signals that are recorded using this system has not yet been reported. APPROACH To bridge this gap, here we conducted a systematic comparison of signal quality between the Cognionics Quick-20 system and the Brain Products actiCAP/actiCHamp (Brain Products GmbH, Munich, Germany), a state-of-the-art, wet-electrode, research-oriented EEG system. Resting-state EEG data were recorded from twelve human participants at rest in eyes open and eyes closed conditions. For both systems we evaluated the similarity of mean recorded power spectral density, and detection of alpha suppression associated with eyes open relative to eyes closed. MAIN RESULTS Power spectral densities were highly correlated across systems, with only minor topographical variability across the scalp. Both systems recorded alpha suppression during eyes open relative to eyes closed conditions. SIGNIFICANCE These results attest to the robustness and reliability of the dry-electrode Cognionics system relatively to the widely used Brain Products laboratory EEG system, and thus validate its utility for clinical and basic research purposes, at least in studies in which participants do not move.
Collapse
Affiliation(s)
- Francesco Marini
- Swartz Center for Computational Neuroscience, University of California San Diego, La Jolla, CA, United States of America. Center for Neuromodulation, University of California San Diego, La Jolla, CA, United States of America
| | | | | | | | | |
Collapse
|
10
|
Dehais F, Duprès A, Blum S, Drougard N, Scannella S, Roy RN, Lotte F. Monitoring Pilot's Mental Workload Using ERPs and Spectral Power with a Six-Dry-Electrode EEG System in Real Flight Conditions. SENSORS (BASEL, SWITZERLAND) 2019; 19:E1324. [PMID: 30884825 PMCID: PMC6471557 DOI: 10.3390/s19061324] [Citation(s) in RCA: 54] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/12/2019] [Revised: 02/24/2019] [Accepted: 03/12/2019] [Indexed: 11/29/2022]
Abstract
Recent technological progress has allowed the development of low-cost and highly portable brain sensors such as pre-amplified dry-electrodes to measure cognitive activity out of the laboratory. This technology opens promising perspectives to monitor the "brain at work" in complex real-life situations such as while operating aircraft. However, there is a need to benchmark these sensors in real operational conditions. We therefore designed a scenario in which twenty-two pilots equipped with a six-dry-electrode EEG system had to perform one low load and one high load traffic pattern along with a passive auditory oddball. In the low load condition, the participants were monitoring the flight handled by a flight instructor, whereas they were flying the aircraft in the high load condition. At the group level, statistical analyses disclosed higher P300 amplitude for the auditory target (Pz, P4 and Oz electrodes) along with higher alpha band power (Pz electrode), and higher theta band power (Oz electrode) in the low load condition as compared to the high load one. Single trial classification accuracy using both event-related potentials and event-related frequency features at the same time did not exceed chance level to discriminate the two load conditions. However, when considering only the frequency features computed over the continuous signal, classification accuracy reached around 70% on average. This study demonstrates the potential of dry-EEG to monitor cognition in a highly ecological and noisy environment, but also reveals that hardware improvement is still needed before it can be used for everyday flight operations.
Collapse
Affiliation(s)
- Frédéric Dehais
- ISAE-SUPAERO, Université de Toulouse, 31055 Toulouse, France.
| | - Alban Duprès
- ISAE-SUPAERO, Université de Toulouse, 31055 Toulouse, France.
| | - Sarah Blum
- Department of Psychology, University of Oldenburg, 26122 Oldenburg, Germany.
| | | | | | - Raphaëlle N Roy
- ISAE-SUPAERO, Université de Toulouse, 31055 Toulouse, France.
| | - Fabien Lotte
- Inria Bordeaux Sud Ouest, LaBRI, University of Bordeaux, Potioc Team, 33400 Talence, France.
| |
Collapse
|
11
|
Lau-Zhu A, Lau MPH, McLoughlin G. Mobile EEG in research on neurodevelopmental disorders: Opportunities and challenges. Dev Cogn Neurosci 2019; 36:100635. [PMID: 30877927 PMCID: PMC6534774 DOI: 10.1016/j.dcn.2019.100635] [Citation(s) in RCA: 89] [Impact Index Per Article: 17.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2018] [Revised: 03/06/2019] [Accepted: 03/06/2019] [Indexed: 11/23/2022] Open
Abstract
Mobile electroencephalography (mobile EEG) represents a next-generation neuroscientific technology – to study real-time brain activity – that is relatively inexpensive, non-invasive and portable. Mobile EEG leverages state-of-the-art hardware alongside established advantages of traditional EEG and recent advances in signal processing. In this review, we propose that mobile EEG could open unprecedented possibilities for studying neurodevelopmental disorders. We first present a brief overview of recent developments in mobile EEG technologies, emphasising the proliferation of studies in several neuroscientific domains. As these developments have yet to be exploited by neurodevelopmentalists, we then identify three research opportunities: 1) increase in the ease and flexibility of brain data acquisition in neurodevelopmental populations; 2) integration into powerful developmentally-informative research designs; 3) development of innovative non-stationary EEG-based paradigms. Critically, we address key challenges that should be considered to fully realise the potential of mobile EEG for neurodevelopmental research and for understanding developmental psychopathology more broadly, and suggest future research directions.
Collapse
Affiliation(s)
- Alex Lau-Zhu
- Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom.
| | - Michael P H Lau
- Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
| | - Gráinne McLoughlin
- Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
| |
Collapse
|
12
|
Inattentional deafness to auditory alarms: Inter-individual differences, electrophysiological signature and single trial classification. Behav Brain Res 2018; 360:51-59. [PMID: 30508609 DOI: 10.1016/j.bbr.2018.11.045] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2018] [Revised: 11/22/2018] [Accepted: 11/29/2018] [Indexed: 02/03/2023]
Abstract
Inattentional deafness can have deleterious consequences in complex real-life situations (e.g. healthcare, aviation) leading to miss critical auditory signals. Such failure of auditory attention is thought to rely on top-down biasing mechanisms at the central executive level. A complementary approach to account for this phenomenon is to consider the existence of visual dominance over hearing that could be implemented via direct visual-to-auditory pathways. To investigate this phenomenon, thirteen aircraft pilots, equipped with a 32-channel EEG system, faced a low and high workload scenarii along with an auditory oddball task in a motion flight simulator. Prior to the flying task, the pilots were screened to assess their working memory span and visual dominance susceptibility. The behavioral results disclosed that the volunteers missed 57.7% of the auditory alarms in the difficult condition. Among all evaluated capabilities, only the visual dominance index was predictive of the miss rate in the difficult scenario. These findings provide behavioral evidences that other early cross-modal competitive process than top down modulation process could account for inattentional deafness. The electrophysiological analyses showed that the miss over the hit alarms led to a significant amplitude reduction of early perceptual (N100) and late attentional (P3a and P3b) event-related potentials components. Eventually, we implemented an EEG-based processing pipeline to perform single-trial classification of inattentional deafness. The results indicate that this processing chain could be used in an ecological setting as it led to 72.2% mean accuracy to discriminate missed from hit auditory alarms.
Collapse
|
13
|
Scannella S, Peysakhovich V, Ehrig F, Lepron E, Dehais F. Assessment of Ocular and Physiological Metrics to Discriminate Flight Phases in Real Light Aircraft. HUMAN FACTORS 2018; 60:922-935. [PMID: 30044142 DOI: 10.1177/0018720818787135] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
OBJECTIVE The purpose of the present study was to find psychophysiological proxies that are straightforward to use and could be implemented in actual flight conditions to accurately discriminate pilots' workload levels. BACKGROUND Piloting an aircraft is a complex activity where cognitive limitations may jeopardize flight safety. There is a need to implement solutions to monitor pilots' workload level to improve flight safety. There has been recent interest in combining psychophysiological measurements. Most of these studies were conducted in flight simulators at the group level, limiting the interpretation of the results. METHODS We conducted an experiment with 11 pilots performing two standard traffic patterns in a light aircraft. Five metrics were derived from their ocular and cardiac activities and were evaluated through three flight phases: takeoff, downwind, and landing. RESULTS Statistical analyses showed that the saccadic rate was the most efficient metric to distinguish between the three flight phases. In addition, a classifier trained on the ocular data collected from the first run predicted the flight phase within a second run with an accuracy of 75%. No gain in the classifier accuracy has been found by combining cardiac and ocular metrics. CONCLUSIONS Ocular-based metrics may be more suitable than cardiac ones to provide relevant information on pilots' flying activity in operational settings. APPLICATIONS Electrocardiographic and eye-tracking devices could be implemented in future cockpits as additional flight data for accident analysis, an objective pilot's state evaluation for training, and proxies for human-machine interactions to improve flight safety.
Collapse
|
14
|
Soto V, Tyson-Carr J, Kokmotou K, Roberts H, Cook S, Fallon N, Giesbrecht T, Stancak A. Brain Responses to Emotional Faces in Natural Settings: A Wireless Mobile EEG Recording Study. Front Psychol 2018; 9:2003. [PMID: 30410458 PMCID: PMC6209651 DOI: 10.3389/fpsyg.2018.02003] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2018] [Accepted: 09/28/2018] [Indexed: 11/25/2022] Open
Abstract
The detection of a human face in a visual field and correct reading of emotional expression of faces are important elements in everyday social interactions, decision making and emotional responses. Although brain correlates of face processing have been established in previous fMRI and electroencephalography (EEG)/MEG studies, little is known about how the brain representation of faces and emotional expressions of faces in freely moving humans. The present study aimed to detect brain electrical potentials that occur during the viewing of human faces in natural settings. 64-channel wireless EEG and eye-tracking data were recorded in 19 participants while they moved in a mock art gallery and stopped at times to evaluate pictures hung on the walls. Positive, negative and neutral valence pictures of objects and human faces were displayed. The time instants in which pictures first occurred in the visual field were identified in eye-tracking data and used to reconstruct the triggers in continuous EEG data after synchronizing the time axes of the EEG and eye-tracking device. EEG data showed a clear face-related event-related potential (ERP) in the latency interval ranging from 165 to 210 ms (N170); this component was not seen whilst participants were viewing non-living objects. The face ERP component was stronger during viewing disgusted compared to neutral faces. Source dipole analysis revealed an equivalent current dipole in the right fusiform gyrus (BA37) accounting for N170 potential. Our study demonstrates for the first time the possibility of recording brain responses to human faces and emotional expressions in natural settings. This finding opens new possibilities for clinical, developmental, social, forensic, or marketing research in which information about face processing is of importance.
Collapse
Affiliation(s)
- Vicente Soto
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
| | - John Tyson-Carr
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
| | - Katerina Kokmotou
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
- Institute for Risk and Uncertainty, University of Liverpool, Liverpool, United Kingdom
| | - Hannah Roberts
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
| | - Stephanie Cook
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
| | - Nicholas Fallon
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
| | - Timo Giesbrecht
- Unilever Research & Development Port Sunlight Laboratory, Merseyside, United Kingdom
| | - Andrej Stancak
- Department of Psychological Sciences, University of Liverpool, Liverpool, United Kingdom
- Institute for Risk and Uncertainty, University of Liverpool, Liverpool, United Kingdom
| |
Collapse
|
15
|
Gateau T, Ayaz H, Dehais F. In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI. Front Hum Neurosci 2018; 12:187. [PMID: 29867411 PMCID: PMC5966564 DOI: 10.3389/fnhum.2018.00187] [Citation(s) in RCA: 53] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2017] [Accepted: 04/17/2018] [Indexed: 11/13/2022] Open
Abstract
There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations.
Collapse
Affiliation(s)
- Thibault Gateau
- ISAE-SUPAERO, Institut Supérieur de l'Aéronautique et de l'Espace, Université Fédérale de Midi-Pyrénées, Toulouse, France
| | - Hasan Ayaz
- School of Biomedical Engineering, Science Health Systems, Drexel University, Philadelphia, PA, United States
| | - Frédéric Dehais
- ISAE-SUPAERO, Institut Supérieur de l'Aéronautique et de l'Espace, Université Fédérale de Midi-Pyrénées, Toulouse, France
| |
Collapse
|
16
|
Callan DE, Gateau T, Durantin G, Gonthier N, Dehais F. Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting. Hum Brain Mapp 2018; 39:2596-2608. [PMID: 29484760 DOI: 10.1002/hbm.24026] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2017] [Revised: 02/19/2018] [Accepted: 02/20/2018] [Indexed: 11/10/2022] Open
Abstract
Individuals often have reduced ability to hear alarms in real world situations (e.g., anesthesia monitoring, flying airplanes) when attention is focused on another task, sometimes with devastating consequences. This phenomenon is called inattentional deafness and usually occurs under critical high workload conditions. It is difficult to simulate the critical nature of these tasks in the laboratory. In this study, dry electroencephalography is used to investigate inattentional deafness in real flight while piloting an airplane. The pilots participating in the experiment responded to audio alarms while experiencing critical high workload situations. It was found that missed relative to detected alarms were marked by reduced stimulus evoked phase synchrony in theta and alpha frequencies (6-14 Hz) from 120 to 230 ms poststimulus onset. Correlation of alarm detection performance with intertrial coherence measures of neural phase synchrony showed different frequency and time ranges for detected and missed alarms. These results are consistent with selective attentional processes actively disrupting oscillatory coherence in sensory networks not involved with the primary task (piloting in this case) under critical high load conditions. This hypothesis is corroborated by analyses of flight parameters showing greater maneuvering associated with difficult phases of flight occurring during missed alarms. Our results suggest modulation of neural oscillation is a general mechanism of attention utilizing enhancement of phase synchrony to sharpen alarm perception during successful divided attention, and disruption of phase synchrony in brain networks when attentional demands of the primary task are great, such as in the case of inattentional deafness.
Collapse
Affiliation(s)
- Daniel E Callan
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan.,Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| | - Thibault Gateau
- Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| | - Gautier Durantin
- School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia
| | - Nicolas Gonthier
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan.,Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| | - Frédéric Dehais
- Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| |
Collapse
|
17
|
Radüntz T. Signal Quality Evaluation of Emerging EEG Devices. Front Physiol 2018; 9:98. [PMID: 29491841 PMCID: PMC5817086 DOI: 10.3389/fphys.2018.00098] [Citation(s) in RCA: 49] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2017] [Accepted: 01/29/2018] [Indexed: 11/15/2022] Open
Abstract
Electroencephalogram (EEG) registration as a direct measure of brain activity has unique potentials. It is one of the most reliable and predicative indicators when studying human cognition, evaluating a subject's health condition, or monitoring their mental state. Unfortunately, standard signal acquisition procedures limit the usability of EEG devices and narrow their application outside the lab. Emerging sensor technology allows gel-free EEG registration and wireless signal transmission. Thus, it enables quick and easy application of EEG devices by users themselves. Although a main requirement for the interpretation of an EEG is good signal quality, there is a lack of research on this topic in relation to new devices. In our work, we compared the signal quality of six very different EEG devices. On six consecutive days, 24 subjects wore each device for 60 min and completed tasks and games on the computer. The registered signals were evaluated in the time and frequency domains. In the time domain, we examined the percentage of artifact-contaminated EEG segments and the signal-to-noise ratios. In the frequency domain, we focused on the band power variation in relation to task demands. The results indicated that the signal quality of a mobile, gel-based EEG system could not be surpassed by that of a gel-free system. However, some of the mobile dry-electrode devices offered signals that were almost comparable and were very promising. This study provided a differentiated view of the signal quality of emerging mobile and gel-free EEG recording technology and allowed an assessment of the functionality of the new devices. Hence, it provided a crucial prerequisite for their general application, while simultaneously supporting their further development.
Collapse
Affiliation(s)
- Thea Radüntz
- Mental Health and Cognitive Capacity, Work and Health, Federal Institute for Occupational Safety and Health, Berlin, Germany
| |
Collapse
|
18
|
Dehais F, Roy RN, Durantin G, Gateau T, Callan D. EEG-Engagement Index and Auditory Alarm Misperception: An Inattentional Deafness Study in Actual Flight Condition. ADVANCES IN NEUROERGONOMICS AND COGNITIVE ENGINEERING 2018. [DOI: 10.1007/978-3-319-60642-2_21] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
19
|
Durantin G, Dehais F, Gonthier N, Terzibas C, Callan DE. Neural signature of inattentional deafness. Hum Brain Mapp 2017; 38:5440-5455. [PMID: 28744950 DOI: 10.1002/hbm.23735] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2017] [Revised: 07/10/2017] [Accepted: 07/11/2017] [Indexed: 11/10/2022] Open
Abstract
Inattentional deafness is the failure to hear otherwise audible sounds (usually alarms) that may occur under high workload conditions. One potential cause for its occurrence could be an attentional bottleneck that occurs when task demands are high, resulting in lack of resources for processing of additional tasks. In this fMRI experiment, we explore the brain regions active during the occurrence of inattentional deafness using a difficult perceptual-motor task in which the participants fly through a simulated Red Bull air race course and at the same time push a button on the joystick to the presence of audio alarms. Participants were instructed to focus on the difficult piloting task and to press the button on the joystick quickly when they noticed an audio alarm. The fMRI results revealed that audio misses relative to hits had significantly greater activity in the right inferior frontal gyrus IFG and the superior medial frontal cortex. Consistent with an attentional bottleneck, activity in these regions was also present for poor flying performance (contrast of gates missed versus gates passed for the flying task). A psychophysiological interaction analysis from the IFG identified reduced effective connectivity to auditory processing regions in the right superior temporal gyrus for missed audio alarms relative to audio alarms that were heard. This study identifies a neural signature of inattentional deafness in an ecologically valid situation by directly measuring differences in brain activity and effective connectivity between audio alarms that were not heard compared to those that were heard. Hum Brain Mapp 38:5440-5455, 2017. © 2017 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Gautier Durantin
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan.,Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France.,School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia
| | - Frederic Dehais
- Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| | - Nicolas Gonthier
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan.,Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France
| | - Cengiz Terzibas
- Multisensory Cognition and Computation Laboratory, Universal Communication Research Institute, National Institute of Information and Communications Technology, Kyoto, Japan
| | - Daniel E Callan
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka University, Osaka, Japan.,Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), Université Fédérale Toulouse Midi-Pyrénées, Toulouse, France.,Multisensory Cognition and Computation Laboratory, Universal Communication Research Institute, National Institute of Information and Communications Technology, Kyoto, Japan
| |
Collapse
|
20
|
Bodranghien FCAA, Langlois Mahe M, Clément S, Manto MU. A Pilot Study on the Effects of Transcranial Direct Current Stimulation on Brain Rhythms and Entropy during Self-Paced Finger Movement using the Epoc Helmet. Front Hum Neurosci 2017; 11:201. [PMID: 28503139 PMCID: PMC5408787 DOI: 10.3389/fnhum.2017.00201] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2017] [Accepted: 04/06/2017] [Indexed: 11/13/2022] Open
Abstract
Transcranial direct current stimulation (tDCS) of the cerebellum is emerging as a novel non-invasive tool to modulate the activity of the cerebellar circuitry. In a single blinded study, we applied anodal tDCS (atDCS) of the cerebellum to assess its effects on brain entropy and brain rhythms during self-paced sequential finger movements in a group of healthy volunteers. Although wearable electroencephalogram (EEG) systems cannot compete with traditional clinical/laboratory set-ups in terms of accuracy and channel density, they have now reached a sufficient maturity to envision daily life applications. Therefore, the EEG was recorded with a comfortable and easy to wear 14 channels wireless helmet (Epoc headset; electrode location was based on the 10-20 system). Cerebellar neurostimulation modified brain rhythmicity with a decrease in the delta band (electrode F3 and T8, p < 0.05). By contrast, our study did not show any significant change in entropy ratios and laterality coefficients (LC) after atDCS of the cerebellum in the 14 channels. The cerebellum is heavily connected with the cerebral cortex including the frontal lobes and parietal lobes via the cerebello-thalamo-cortical pathway. We propose that the effects of anodal stimulation of the cerebellar cortex upon cerebral cortical rhythms are mediated by this key-pathway. Additional studies using high-density EEG recordings and behavioral correlates are now required to confirm our findings, especially given the limited coverage of Epoc headset.
Collapse
Affiliation(s)
- Florian C. A. A. Bodranghien
- Unité d’Etude du Mouvement (UEM-GRIM), Fonds de la Recherche Scientifique, Université Libre De BruxellesBruxelles, Belgium
| | | | - Serge Clément
- Haute Ecole Libre de Bruxelles Ilya Prigogine (HELB)Bruxelles, Belgium
| | - Mario U. Manto
- Unité d’Etude du Mouvement (UEM-GRIM), Fonds de la Recherche Scientifique, Université Libre De BruxellesBruxelles, Belgium
- Haute Ecole Libre de Bruxelles Ilya Prigogine (HELB)Bruxelles, Belgium
| |
Collapse
|
21
|
Minguillon J, Lopez-Gordo MA, Pelayo F. Trends in EEG-BCI for daily-life: Requirements for artifact removal. Biomed Signal Process Control 2017. [DOI: 10.1016/j.bspc.2016.09.005] [Citation(s) in RCA: 61] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
|
22
|
Callan DE, Terzibas C, Cassel DB, Sato MA, Parasuraman R. The Brain Is Faster than the Hand in Split-Second Intentions to Respond to an Impending Hazard: A Simulation of Neuroadaptive Automation to Speed Recovery to Perturbation in Flight Attitude. Front Hum Neurosci 2016; 10:187. [PMID: 27199710 PMCID: PMC4846799 DOI: 10.3389/fnhum.2016.00187] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2015] [Accepted: 04/12/2016] [Indexed: 11/13/2022] Open
Abstract
The goal of this research is to test the potential for neuroadaptive automation to improve response speed to a hazardous event by using a brain-computer interface (BCI) to decode perceptual-motor intention. Seven participants underwent four experimental sessions while measuring brain activity with magnetoencephalograpy. The first three sessions were of a simple constrained task in which the participant was to pull back on the control stick to recover from a perturbation in attitude in one condition and to passively observe the perturbation in the other condition. The fourth session consisted of having to recover from a perturbation in attitude while piloting the plane through the Grand Canyon constantly maneuvering to track over the river below. Independent component analysis was used on the first two sessions to extract artifacts and find an event related component associated with the onset of the perturbation. These two sessions were used to train a decoder to classify trials in which the participant recovered from the perturbation (motor intention) vs. just passively viewing the perturbation. The BCI-decoder was tested on the third session of the same simple task and found to be able to significantly distinguish motor intention trials from passive viewing trials (mean = 69.8%). The same BCI-decoder was then used to test the fourth session on the complex task. The BCI-decoder significantly classified perturbation from no perturbation trials (73.3%) with a significant time savings of 72.3 ms (Original response time of 425.0-352.7 ms for BCI-decoder). The BCI-decoder model of the best subject was shown to generalize for both performance and time savings to the other subjects. The results of our off-line open loop simulation demonstrate that BCI based neuroadaptive automation has the potential to decode motor intention faster than manual control in response to a hazardous perturbation in flight attitude while ignoring ongoing motor and visual induced activity related to piloting the airplane.
Collapse
Affiliation(s)
- Daniel E Callan
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Osaka UniversityOsaka, Japan; Multisensory Cognition and Computation Laboratory, Universal Communication Research Institute, National Institute of Information and Communications TechnologyKyoto, Japan
| | - Cengiz Terzibas
- Multisensory Cognition and Computation Laboratory, Universal Communication Research Institute, National Institute of Information and Communications Technology Kyoto, Japan
| | | | - Masa-Aki Sato
- Neural Information Analysis Laboratories, Advanced Telecommunications Research Institute Kyoto, Japan
| | - Raja Parasuraman
- Center of Excellence in Neuroergonomics, Technology, and Cognition, George Mason University Fairfax, VA, USA
| |
Collapse
|
23
|
Durantin G, Scannella S, Gateau T, Delorme A, Dehais F. Processing Functional Near Infrared Spectroscopy Signal with a Kalman Filter to Assess Working Memory during Simulated Flight. Front Hum Neurosci 2016; 9:707. [PMID: 26834607 PMCID: PMC4719469 DOI: 10.3389/fnhum.2015.00707] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2015] [Accepted: 12/17/2015] [Indexed: 11/13/2022] Open
Abstract
Working memory (WM) is a key executive function for operating aircraft, especially when pilots have to recall series of air traffic control instructions. There is a need to implement tools to monitor WM as its limitation may jeopardize flight safety. An innovative way to address this issue is to adopt a Neuroergonomics approach that merges knowledge and methods from Human Factors, System Engineering, and Neuroscience. A challenge of great importance for Neuroergonomics is to implement efficient brain imaging techniques to measure the brain at work and to design Brain Computer Interfaces (BCI). We used functional near infrared spectroscopy as it has been already successfully tested to measure WM capacity in complex environment with air traffic controllers (ATC), pilots, or unmanned vehicle operators. However, the extraction of relevant features from the raw signal in ecological environment is still a critical issue due to the complexity of implementing real-time signal processing techniques without a priori knowledge. We proposed to implement the Kalman filtering approach, a signal processing technique that is efficient when the dynamics of the signal can be modeled. We based our approach on the Boynton model of hemodynamic response. We conducted a first experiment with nine participants involving a basic WM task to estimate the noise covariances of the Kalman filter. We then conducted a more ecological experiment in our flight simulator with 18 pilots who interacted with ATC instructions (two levels of difficulty). The data was processed with the same Kalman filter settings implemented in the first experiment. This filter was benchmarked with a classical pass-band IIR filter and a Moving Average Convergence Divergence (MACD) filter. Statistical analysis revealed that the Kalman filter was the most efficient to separate the two levels of load, by increasing the observed effect size in prefrontal areas involved in WM. In addition, the use of a Kalman filter increased the performance of the classification of WM levels based on brain signal. The results suggest that Kalman filter is a suitable approach for real-time improvement of near infrared spectroscopy signal in ecological situations and the development of BCI.
Collapse
Affiliation(s)
- Gautier Durantin
- Département Conception et Conduite des Véhicules Aéronautiques et Spatiaux, Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-Supaéro)Toulouse, France; Centre de Recherche Cerveau et Cognition, Université Toulouse III - Paul SabatierToulouse, France; Centre National de la Recherche Scientifique, Centre de Recherche Cerveau et CognitionToulouse, France
| | - Sébastien Scannella
- Département Conception et Conduite des Véhicules Aéronautiques et Spatiaux, Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-Supaéro) Toulouse, France
| | - Thibault Gateau
- Département Conception et Conduite des Véhicules Aéronautiques et Spatiaux, Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-Supaéro) Toulouse, France
| | - Arnaud Delorme
- Centre de Recherche Cerveau et Cognition, Université Toulouse III - Paul SabatierToulouse, France; Centre National de la Recherche Scientifique, Centre de Recherche Cerveau et CognitionToulouse, France
| | - Frédéric Dehais
- Département Conception et Conduite des Véhicules Aéronautiques et Spatiaux, Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-Supaéro) Toulouse, France
| |
Collapse
|