Brain at Work and in Everyday Life as the Next Frontier: Grand Field Challenges for Neuroergonomics

Shredding artifacts: extracting brain activity in EEG from extreme artifacts during skateboarding using ASR and ICA

Introduction

To understand brain function in natural real-world settings, it is crucial to acquire brain activity data in noisy environments with diverse artifacts. Electroencephalography (EEG), while susceptible to environmental and physiological artifacts, can be cleaned using advanced signal processing techniques like Artifact Subspace Reconstruction (ASR) and Independent Component Analysis (ICA). This study aims to demonstrate that ASR and ICA can effectively extract brain activity from the substantial artifacts occurring while skateboarding on a half-pipe ramp.

Methods

A dual-task paradigm was used, where subjects were presented with auditory stimuli during skateboarding and rest conditions. The effectiveness of ASR and ICA in cleaning artifacts was evaluated using a support vector machine to classify the presence or absence of a sound stimulus in single-trial EEG data. The study evaluated the effectiveness of ASR and ICA in artifact cleaning using five different pipelines: (1) Minimal cleaning (bandpass filtering), (2) ASR only, (3) ICA only, (4) ICA followed by ASR (ICAASR), and (5) ASR preceding ICA (ASRICA). Three skateboarders participated in the experiment.

Results

Results showed that all ICA-containing pipelines, especially ASRICA (69%, 68%, 63%), outperformed minimal cleaning (55%, 52%, 50%) in single-trial classification during skateboarding. The ASRICA pipeline performed significantly better than other pipelines containing ICA for two of the three subjects, with no other pipeline performing better than ASRICA. The superior performance of ASRICA likely results from ASR removing non-stationary artifacts, enhancing ICA decomposition. Evidenced by ASRICA identifying more brain components via ICLabel than ICA alone or ICAASR for all subjects. For the rest condition, with fewer artifacts, the ASRICA pipeline (71%, 82%, 75%) showed slight improvement over minimal cleaning (73%, 70%, 72%), performing significantly better for two subjects.

Discussion

This study demonstrates that ASRICA can effectively clean artifacts to extract single-trial brain activity during skateboarding. These findings affirm the feasibility of recording brain activity during physically demanding tasks involving substantial body movement, laying the groundwork for future research into the neural processes governing complex and coordinated body movements.

Bringing together multimodal and multilevel approaches to study the emergence of social bonds between children and improve social AI

This protocol paper outlines an innovative multimodal and multilevel approach to studying the emergence and evolution of how children build social bonds with their peers, and its potential application to improving social artificial intelligence (AI). We detail a unique hyperscanning experimental framework utilizing functional near-infrared spectroscopy (fNIRS) to observe inter-brain synchrony in child dyads during collaborative tasks and social interactions. Our proposed longitudinal study spans middle childhood, aiming to capture the dynamic development of social connections and cognitive engagement in naturalistic settings. To do so we bring together four kinds of data: the multimodal conversational behaviors that dyads of children engage in, evidence of their state of interpersonal rapport, collaborative performance on educational tasks, and inter-brain synchrony. Preliminary pilot data provide foundational support for our approach, indicating promising directions for identifying neural patterns associated with productive social interactions. The planned research will explore the neural correlates of social bond formation, informing the creation of a virtual peer learning partner in the field of Social Neuroergonomics. This protocol promises significant contributions to understanding the neural basis of social connectivity in children, while also offering a blueprint for designing empathetic and effective social AI tools, particularly for educational contexts.

Prefrontal Cortex Responses to Social Video Stimuli in Young Children with and without Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a neurodevelopmental disorder affecting individuals worldwide and characterized by deficits in social interaction along with the presence of restricted interest and repetitive behaviors. Despite decades of behavioral research, little is known about the brain mechanisms that influence social behaviors among children with ASD. This, in part, is due to limitations of traditional imaging techniques specifically targeting pediatric populations. As a portable and scalable optical brain monitoring technology, functional near infrared spectroscopy (fNIRS) provides a measure of cerebral hemodynamics related to sensory, motor, or cognitive function. Here, we utilized fNIRS to investigate the prefrontal cortex (PFC) activity of young children with ASD and with typical development while they watched social and nonsocial video clips. The PFC activity of ASD children was significantly higher for social stimuli at medial PFC, which is implicated in social cognition/processing. Moreover, this activity was also consistently correlated with clinical measures, and higher activation of the same brain area only during social video viewing was associated with more ASD symptoms. This is the first study to implement a neuroergonomics approach to investigate cognitive load in response to realistic, complex, and dynamic audiovisual social stimuli for young children with and without autism. Our results further confirm that new generation of portable fNIRS neuroimaging can be used for ecologically valid measurements of the brain function of toddlers and preschool children with ASD.

Evaluation of a headphones-fitted EEG system for the recording of auditory evoked potentials and mental workload assessment

Advancements in portable neuroimaging technologies open up new opportunities to gain insight into the neural dynamics and cognitive processes underlying day-to-day behaviors. In this study, we evaluated the relevance of a headphone- mounted electroencephalogram (EEG) system for monitoring mental workload. The participants (N = 12) were instructed to pay attention to auditory alarms presented sporadically while performing the Multi-Attribute Task Battery (MATB) whose difficulty was staged across three conditions to manipulate mental workload. The P300 Event-Related Potentials (ERP) elicited by the presentation of auditory alarms were used as probes of attentional resources available. The amplitude and latency of P300 ERPs were compared across experimental conditions. Our findings indicate that the P300 ERP component can be captured using a headphone-mounted EEG system. Moreover, neural responses to alarm could be used to classify mental workload with high accuracy (over 80%) at a single-trial level. Our analyses indicated that the signal-to-noise ratio acquired by the sponge-based sensors remained stable throughout the recordings. These results highlight the potential of portable neuroimaging technology for the development of neuroassistive applications while underscoring the current limitations and challenges associated with the integration of EEG sensors in everyday-life wearable technologies. Overall, our study contributes to the growing body of research exploring the feasibility and validity of wearable neuroimaging technologies for the study of human cognition and behavior in real-world settings.

Blink-Related Oscillations Provide Naturalistic Assessments of Brain Function and Cognitive Workload within Complex Real-World Multitasking Environments

BACKGROUND

There is a significant need to monitor human cognitive performance in complex environments, with one example being pilot performance. However, existing assessments largely focus on subjective experiences (e.g., questionnaires) and the evaluation of behavior (e.g., aircraft handling) as surrogates for cognition or utilize brainwave measures which require artificial setups (e.g., simultaneous auditory stimuli) that intrude on the primary tasks. Blink-related oscillations (BROs) are a recently discovered neural phenomenon associated with spontaneous blinking that can be captured without artificial setups and are also modulated by cognitive loading and the external sensory environment-making them ideal for brain function assessment within complex operational settings.

METHODS

Electroencephalography (EEG) data were recorded from eight adult participants (five F, M = 21.1 years) while they completed the Multi-Attribute Task Battery under three different cognitive loading conditions. BRO responses in time and frequency domains were derived from the EEG data, and comparisons of BRO responses across cognitive loading conditions were undertaken. Simultaneously, assessments of blink behavior were also undertaken.

RESULTS

Blink behavior assessments revealed decreasing blink rate with increasing cognitive load (p < 0.001). Prototypical BRO responses were successfully captured in all participants (p < 0.001). BRO responses reflected differences in task-induced cognitive loading in both time and frequency domains (p < 0.05). Additionally, reduced pre-blink theta band desynchronization with increasing cognitive load was also observed (p < 0.05).

CONCLUSION

This study confirms the ability of BRO responses to capture cognitive loading effects as well as preparatory pre-blink cognitive processes in anticipation of the upcoming blink during a complex multitasking situation. These successful results suggest that blink-related neural processing could be a potential avenue for cognitive state evaluation in operational settings-both specialized environments such as cockpits, space exploration, military units, etc. and everyday situations such as driving, athletics, human-machine interactions, etc.-where human cognition needs to be seamlessly monitored and optimized.

Cognitive Effort during Visuospatial Problem Solving in Physical Real World, on Computer Screen, and in Virtual Reality

Spatial cognition plays a crucial role in academic achievement, particularly in science, technology, engineering, and mathematics (STEM) domains. Immersive virtual environments (VRs) have the growing potential to reduce cognitive load and improve spatial reasoning. However, traditional methods struggle to assess the mental effort required for visuospatial processes due to the difficulty in verbalizing actions and other limitations in self-reported evaluations. In this neuroergonomics study, we aimed to capture the neural activity associated with cognitive workload during visuospatial tasks and evaluate the impact of the visualization medium on visuospatial task performance. We utilized functional near-infrared spectroscopy (fNIRS) wearable neuroimaging to assess cognitive effort during spatial-reasoning-based problem-solving and compared a VR, a computer screen, and a physical real-world task presentation. Our results reveal a higher neural efficiency in the prefrontal cortex (PFC) during 3D geometry puzzles in VR settings compared to the settings in the physical world and on the computer screen. VR appears to reduce the visuospatial task load by facilitating spatial visualization and providing visual cues. This makes it a valuable tool for spatial cognition training, especially for beginners. Additionally, our multimodal approach allows for progressively increasing task complexity, maintaining a challenge throughout training. This study underscores the potential of VR in developing spatial skills and highlights the value of comparing brain data and human interaction across different training settings.

Modeling Complex EEG Data Distribution on the Riemannian Manifold Toward Outlier Detection and Multimodal Classification

OBJECTIVE

The usage of Riemannian geometry for Brain-computer interfaces (BCIs) has gained momentum in recent years. Most of the machine learning techniques proposed for Riemannian BCIs consider the data distribution on a manifold to be unimodal. However, the distribution is likely to be multimodal rather than unimodal since high-data variability is a crucial limitation of electroencephalography (EEG). In this paper, we propose a novel data modeling method for considering complex data distributions on a Riemannian manifold of EEG covariance matrices, aiming to improve BCI reliability.

METHODS

Our method, Riemannian spectral clustering (RiSC), represents EEG covariance matrix distribution on a manifold using a graph with proposed similarity measurement based on geodesic distances, then clusters the graph nodes through spectral clustering. This allows flexibility to model both a unimodal and a multimodal distribution on a manifold. RiSC can be used as a basis to design an outlier detector named outlier detection Riemannian spectral clustering (odenRiSC) and a multimodal classifier named multimodal classifier Riemannian spectral clustering (mcRiSC). All required parameters of odenRiSC/mcRiSC are selected in data-driven manner. Moreover, there is no need to pre-set a threshold for outlier detection and the number of modes for multimodal classification.

RESULTS

The experimental evaluation revealed odenRiSC can detect EEG outliers more accurately than existing methods and mcRiSC outperformed the standard unimodal classifier, especially on high-variability datasets.

CONCLUSION

odenRiSC/mcRiSC are anticipated to contribute to making real-life BCIs outside labs and neuroergonomics applications more robust.

SIGNIFICANCE

RiSC can work as a robust EEG outlier detector and multimodal classifier.

Isometric Arm Forces Exerted by Females at Different Levels of Physical Comfort and Their EEG Signatures

A variety of subjective measures have traditionally been used to assess the perception of physical exertion at work and related body responses. However, the current understanding of physical comfort experienced at work is very limited. The main objective of this study was first to investigate the magnitude of isometric arm forces exerted by females at different levels of physical comfort measured on a new comfort scale and, second, to assess their corresponding neural signatures expressed in terms of power spectral density (PSD). The study assessed PSDs of four major electroencephalography (EEG) frequency bands, focusing on the brain regions controlling motor and perceptual processing. The results showed statistically significant differences in exerted arm forces and the rate of perceived exertion at the various levels of comfort. Significant differences in power spectrum density at different physical comfort levels were found for the beta EEG band. Such knowledge can be useful in incorporating female users' force requirements in the design of consumer products, including tablets, laptops, and other hand-held information technology devices, as well as various industrial processes and work systems.

Simultaneous fMRI and tDCS for Enhancing Training of Flight Tasks

There is a gap in our understanding of how best to apply transcranial direct-current stimulation (tDCS) to enhance learning in complex, realistic, and multifocus tasks such as aviation. Our goal is to assess the effects of tDCS and feedback training on task performance, brain activity, and connectivity using functional magnetic resonance imaging (fMRI). Experienced glider pilots were recruited to perform a one-day, three-run flight-simulator task involving varying difficulty conditions and a secondary auditory task, mimicking real flight requirements. The stimulation group (versus sham) received 1.5 mA high-definition HD-tDCS to the right dorsolateral prefrontal cortex (DLPFC) for 30 min during the training. Whole-brain fMRI was collected before, during, and after stimulation. Active stimulation improved piloting performance both during and post-training, particularly in novice pilots. The fMRI revealed a number of tDCS-induced effects on brain activation, including an increase in the left cerebellum and bilateral basal ganglia for the most difficult conditions, an increase in DLPFC activation and connectivity to the cerebellum during stimulation, and an inhibition in the secondary task-related auditory cortex and Broca's area. Here, we show that stimulation increases activity and connectivity in flight-related brain areas, particularly in novices, and increases the brain's ability to focus on flying and ignore distractors. These findings can guide applied neurostimulation in real pilot training to enhance skill acquisition and can be applied widely in other complex perceptual-motor real-world tasks.

Altered Inhibitory Control Mechanism of Internet Addiction: An Electroencephalogram Study of Brain Oscillations and Connectivity

The development of the Internet has changed people's lives and has resulted in a new type of addictive behavior. In the past decade, Internet game addiction has been identified as a mental illness. Considering internet game addiction as the only cause of mental illness is limited in its view, as internet games, social platforms and other internet multimedia are also widely used. Thus, other internet-related behaviors, that maybe addictive, should also be included. Previous neuroimaging studies have reported a role of alteration in brain's inhibitory control mechanism in addiction. However, the results are still diverse with inconsistent findings. In this study, we used an Internet-related stop signal task with EEG signals recorded to study the relationship between internet addiction through brain oscillations and functional connectivity. We also compared the differences in the brain connectivity between addicted and non-addicted participants using phase lag index. We found that the brain connectivity in participants addicted to the internet is significantly greater than that of nonaddicted users.Clinical Relevance- In this study, we assessed brain functional networks of participants with Internet Gaming Disorder and internet addiction.

Physiological synchrony in electrodermal activity predicts decreased vigilant attention induced by sleep deprivation

Introduction

When multiple individuals are presented with narrative movie or audio clips, their electrodermal activity (EDA) and heart rate show significant similarities. Higher levels of such inter-subject physiological synchrony are related with higher levels of attention toward the narrative, as for instance expressed by more correctly answered questions about the narrative. We here investigate whether physiological synchrony in EDA and heart rate during watching of movie clips predicts performance on a subsequent vigilant attention task among participants exposed to a night of total sleep deprivation.

Methods

We recorded EDA and heart rate of 54 participants during a night of total sleep deprivation. Every hour from 22:00 to 07:00 participants watched a 10-min movie clip during which we computed inter-subject physiological synchrony. Afterwards, they answered questions about the movie and performed the psychomotor vigilance task (PVT) to capture attentional performance.

Results

We replicated findings that inter-subject correlations in EDA and heart rate predicted the number of correct answers on questions about the movie clips. Furthermore, we found that inter-subject correlations in EDA, but not in heart rate, predicted PVT performance. Individuals' mean EDA and heart rate also predicted their PVT performance. For EDA, inter-subject correlations explained more variance of PVT performance than individuals' mean EDA.

Discussion

Together, these findings confirm the association between physiological synchrony and attention. Physiological synchrony in EDA does not only capture the attentional processing during the time that it is determined, but also proves valuable for capturing more general changes in the attentional state of monitored individuals.

The role of brain-localized gamma and alpha oscillations in inattentional deafness: implications for understanding human attention

Introduction

The processes involved in how the attention system selectively focuses on perceptual and motor aspects related to a specific task, while suppressing features of other tasks and/or objects in the environment, are of considerable interest for cognitive neuroscience. The goal of this experiment was to investigate neural processes involved in selective attention and performance under multi-task situations. Several studies have suggested that attention-related gamma-band activity facilitates processing in task-specific modalities, while alpha-band activity inhibits processing in non-task-related modalities. However, investigations into the phenomenon of inattentional deafness/blindness (inability to observe stimuli in non-dominant task when primary task is demanding) have yet to observe gamma-band activity.

Methods

This EEG experiment utilizes an engaging whole-body perceptual motor task while carrying out a secondary auditory detection task to investigate neural correlates of inattentional deafness in natural immersive high workload conditions. Differences between hits and misses on the auditory detection task in the gamma (30-50 Hz) and alpha frequency (8-12 Hz) range were carried out at the cortical source level using LORETA.

Results

Participant auditory task performance correlated with an increase in gamma-band activity for hits over misses pre- and post-stimulus in left auditory processing regions. Alpha-band activity was greater for misses relative to hits in right auditory processing regions pre- and post-stimulus onset. These results are consistent with the facilitatory/inhibitory role of gamma/alpha-band activity for neural processing. Additional gamma- and alpha-band activity was found in frontal and parietal brain regions which are thought to reflect various attentional monitoring, selection, and switching processes.

Discussion

The results of this study help to elucidate the role of gamma and alpha frequency bands in frontal and modality-specific regions involved with selective attention in multi-task immersive situations.

Information assistance systems as preventive mediators between increasing customization and mental workload

BACKGROUND: The future of work in Germany is shaped by megatrends like globalization, automatization, digitization, and the demographic change. Furthermore, mass customization and the increasing usage of AI even in manual assembly offers new opportunities as well as it creates new challenges. OBJECTIVE: The trend towards mass customization in turn leads to increased complexity in production, which results in additional mental workload. This effect will continue in the foreseeable future. METHOD: Especially for small and medium sized companies, the backbone of Germany’s economy, automatization and Human-Robot-Collaboration will take time to develop. Information assistance systems are and will be a bridging technology to help organizations to manage increasing complexity and the mental workload of their employees to not only boost productivity but also keep their workforce healthy. The ongoing demographic change further underlines the need to use information assistance systems to compensate possible age-associated deficits, but also keep older employees committed to their work and avoid effects of disengagement or disenfranchisement through participatory ergonomics. RESULTS: Information assistance systems can only develop their inherent potential if they are designed to support employees of varying age, competence levels, and affinity for technology. Participatory development and early engagement are key factors for an increased acceptance and usage of the systems as well as the individualization to make it suitable for each individual employee. CONCLUSION: Expanding the functionalities to an adaptive assistance system, using physiological correlates of mental workload as an input, is conceivable in the future.

Neuroergonomic assessment of developmental coordination disorder

Until recently, neural assessments of gross motor coordination could not reliably handle active tasks, particularly in realistic environments, and offered a narrow understanding of motor-cognition. By applying a comprehensive neuroergonomic approach using optical mobile neuroimaging, we probed the neural correlates of motor functioning in young people with Developmental Coordination Disorder (DCD), a motor-learning deficit affecting 5-6% of children with lifelong complications. Neural recordings using fNIRS were collected during active ambulatory behavioral task execution from 37 Typically Developed and 48 DCD Children who performed cognitive and physical tasks in both single and dual conditions. This is the first of its kind study targeting regions of prefrontal cortical dysfunction for identification of neuropathophysiology for DCD during realistic motor tasks and is one of the largest neuroimaging study (across all modalities) involving DCD. We demonstrated that DCD is a motor-cognitive disability, as gross motor /complex tasks revealed neuro-hemodynamic deficits and dysfunction within the right middle and superior frontal gyri of the prefrontal cortex through functional near infrared spectroscopy. Furthermore, by incorporating behavioral performance, decreased neural efficiency in these regions were revealed in children with DCD, specifically during motor tasks. Lastly, we provide a framework, evaluating disorder impact in ecologically valid contexts to identify when and for whom interventional approaches are most needed and open the door for precision therapies.

Dual Passive Reactive Brain-Computer Interface: A Novel Approach to Human-Machine Symbiosis

The present study proposes a novel concept of neuroadaptive technology, namely a dual passive-reactive Brain-Computer Interface (BCI), that enables bi-directional interaction between humans and machines. We have implemented such a system in a realistic flight simulator using the NextMind classification algorithms and framework to decode pilots' intention (reactive BCI) and to infer their level of attention (passive BCI). Twelve pilots used the reactive BCI to perform checklists along with an anti-collision radar monitoring task that was supervised by the passive BCI. The latter simulated an automatic avoidance maneuver when it detected that pilots missed an incoming collision. The reactive BCI reached 100% classification accuracy with a mean reaction time of 1.6 s when exclusively performing the checklist task. Accuracy was up to 98.5% with a mean reaction time of 2.5 s when pilots also had to fly the aircraft and monitor the anti-collision radar. The passive BCI achieved a F1-score of 0.94. This first demonstration shows the potential of a dual BCI to improve human-machine teaming which could be applied to a variety of applications.

Training Monitoring in Sports: It Is Time to Embrace Cognitive Demand

Appropriate training burden monitoring is still a challenge for the support staff, athletes, and coaches. Extensive research has been done in recent years that proposes several external and internal indicators. Among all measurements, the importance of cognitive factors has been indicated but has never been really considered in the training monitoring process. While there is strong evidence supporting the use of cognitive demand indicators in cognitive neuroscience, their importance in training monitoring for multiple sports settings must be better emphasized. The aims of this scoping review are to (1) provide an overview of the cognitive demand concept beside the physical demand in training; (2) highlight the current methods for assessing cognitive demand in an applied setting to sports in part through a neuroergonomics approach; (3) show how cognitive demand metrics can be exploited and applied to our better understanding of fatigue, sport injury, overtraining and individual performance capabilities. This review highlights also the potential new ways of brain imaging approaches for monitoring in situ. While assessment of cognitive demand is still in its infancy in sport, it may represent a very fruitful approach if applied with rigorous protocols and deep knowledge of both the neurobehavioral and cognitive aspects. It is time now to consider the cognitive demand to avoid underestimating the total training burden and its management.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

Analyzing Classification Performance of fNIRS-BCI for Gait Rehabilitation Using Deep Neural Networks

This research presents a brain-computer interface (BCI) framework for brain signal classification using deep learning (DL) and machine learning (ML) approaches on functional near-infrared spectroscopy (fNIRS) signals. fNIRS signals of motor execution for walking and rest tasks are acquired from the primary motor cortex in the brain's left hemisphere for nine subjects. DL algorithms, including convolutional neural networks (CNNs), long short-term memory (LSTM), and bidirectional LSTM (Bi-LSTM) are used to achieve average classification accuracies of 88.50%, 84.24%, and 85.13%, respectively. For comparison purposes, three conventional ML algorithms, support vector machine (SVM), k-nearest neighbor (k-NN), and linear discriminant analysis (LDA) are also used for classification, resulting in average classification accuracies of 73.91%, 74.24%, and 65.85%, respectively. This study successfully demonstrates that the enhanced performance of fNIRS-BCI can be achieved in terms of classification accuracy using DL approaches compared to conventional ML approaches. Furthermore, the control commands generated by these classifiers can be used to initiate and stop the gait cycle of the lower limb exoskeleton for gait rehabilitation.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Pocketable Labs for Everyone: Synchronized Multi-Sensor Data Streaming and Recording on Smartphones with the Lab Streaming Layer

The streaming and recording of smartphone sensor signals is desirable for mHealth, telemedicine, environmental monitoring and other applications. Time series data gathered in these fields typically benefit from the time-synchronized integration of different sensor signals. However, solutions required for this synchronization are mostly available for stationary setups. We hope to contribute to the important emerging field of portable data acquisition by presenting open-source Android applications both for the synchronized streaming (Send-a) and recording (Record-a) of multiple sensor data streams. We validate the applications in terms of functionality, flexibility and precision in fully mobile setups and in hybrid setups combining mobile and desktop hardware. Our results show that the fully mobile solution is equivalent to well-established desktop versions. With the streaming application Send-a and the recording application Record-a, purely smartphone-based setups for mobile research and personal health settings can be realized on off-the-shelf Android devices.

Hybrid Systems to Boost EEG-Based Real-Time Action Decoding in Car Driving Scenarios

The complexity of concurrent cerebral processes underlying driving makes such human behavior one of the most studied real-world activities in neuroergonomics. Several attempts have been made to decode, both offline and online, cerebral activity during car driving with the ultimate goal to develop brain-based systems for assistive devices. Electroencephalography (EEG) is the cornerstone of these studies providing the highest temporal resolution to track those cerebral processes underlying overt behavior. Particularly when investigating real-world scenarios as driving, EEG is constrained by factors such as robustness, comfortability, and high data variability affecting the decoding performance. Hence, additional peripheral signals can be combined with EEG for increasing replicability and the overall performance of the brain-based action decoder. In this regard, hybrid systems have been proposed for the detection of braking and steering actions in driving scenarios to improve the predictive power of the single neurophysiological measurement. These recent results represent a proof of concept of the level of technological maturity. They may pave the way for increasing the predictive power of peripheral signals, such as electroculogram (EOG) and electromyography (EMG), collected in real-world scenarios when informed by EEG measurements, even if collected only offline in standard laboratory settings. The promising usability of such hybrid systems should be further investigated in other domains of neuroergonomics.

Projections and the Potential Societal Impact of the Future of Neurotechnologies

Traditionally, recording from and stimulating the brain with high spatial and temporal resolution required invasive means. However, recently, the technical capabilities of less invasive and non-invasive neuro-interfacing technology have been dramatically improving, and laboratories and funders aim to further improve these capabilities. These technologies can facilitate functions such as multi-person communication, mood regulation and memory recall. We consider a potential future where the less invasive technology is in high demand. Will this demand match that the current-day demand for a smartphone? Here, we draw upon existing research to project which particular neuroethics issues may arise in this potential future and what preparatory steps may be taken to address these issues.

Exploring the Embodiment of a Virtual Hand in a Spatially Augmented Respiratory Biofeedback Setting

Enhancing the embodiment of artificial limbs—the individuals' feeling that a virtual or robotic limb is integrated in their own body scheme—is an impactful strategy for improving prosthetic technology acceptance and human-machine interaction. Most studies so far focused on visuo-tactile strategies to empower the embodiment processes. However, novel approaches could emerge from self-regulation techniques able to change the psychophysiological conditions of an individual. Accordingly, this pilot study investigates the effects of a self-regulated breathing exercise on the processes of body ownership underlying the embodiment of a virtual right hand within a Spatially Augmented Respiratory Biofeedback (SARB) setting. This investigation also aims at evaluating the feasibility of the breathing exercise enabled by a low-cost SARB implementation designed for upcoming remote studies (a need emerged during the COVID-19 pandemic). Twenty-two subjects without impairments, and two transradial prosthesis users for a preparatory test, were asked (in each condition of a within-group design) to maintain a normal (about 14 breaths/min) or slow (about 6 breaths/min) respiratory rate to keep a static virtual right hand “visible” on a screen. Meanwhile, a computer-generated sphere moved from left to right toward the virtual hand during each trial (1 min) of 16. If the participant's breathing rate was within the target (slow or normal) range, a visuo-tactile event was triggered by the sphere passing under the virtual hand (the subjects observed it shaking while they perceived a vibratory feedback generated by a smartphone). Our results—mainly based on questionnaire scores and proprioceptive drift—highlight that the slow breathing condition induced higher embodiment than the normal one. This preliminary study reveals the feasibility and potential of a novel psychophysiological training strategy to enhance the embodiment of artificial limbs. Future studies are needed to further investigate mechanisms, efficacy and generalizability of the SARB techniques in training a bionic limb embodiment.

Prefrontal Inter-brain Synchronization Reflects Convergence and Divergence of Flow Dynamics in Collaborative Learning: A Pilot Study

Flow is a highly motivated and affectively positive state in which a person is deeply engaged in an activity and feeling enjoyment from it. In collaborative activities, it would be optimal if all participants were in a state of flow. However, flow states fluctuate amongst individuals due to differences in the dynamics of motivation and cognition. To explore the possibility that inter-brain synchronization can provide a quantitative measure of the convergence and divergence of collective motivational dynamics, we conducted a pilot study to investigate the relationship between inter-brain synchronization and the interpersonal similarity of flow state dynamics during the collaborative learning process. In two English as a Foreign Language (EFL) classes, students were divided into groups of three-four and seated at desks facing each other while conducting a 60-min group work. In both classes, two groups with four members were randomly selected, and their medial prefrontal neural activities were measured simultaneously using wireless functional near-infrared spectroscopy (fNIRS) devices. Later the participants observed their own activities on recorded videos and retrospectively rated their subjective degree of flow state on a seven-point scale for each 2-min period. For the pairs of students whose neural activities were measured, the similarity of their flow experience dynamics was evaluated by the temporal correlation between their flow ratings. Prefrontal inter-brain synchronization of the same student pairs during group work was evaluated using wavelet transform coherence. Statistical analyses revealed that: (1) flow dynamics were significantly more similar for the student pairs within the same group compared to the pairs of students assigned across different groups; (2) prefrontal inter-brain synchronization in the relatively short time scale (9.3-13.9 s) was significantly higher for the within-group pairs than for the cross-group pairs; and (3) the prefrontal inter-brain synchronization at the same short time scale was significantly and positively correlated with the similarity of flow dynamics, even after controlling for the effects of within- vs. cross-group pair types from the two variables. These suggest that inter-brain synchronization can indeed provide a quantitative measure for converging and diverging collective motivational dynamics during collaborative learning, with higher inter-brain synchronization corresponding to a more convergent flow experience.

Brain Activation Changes While Walking in Adults with and without Neurological Disease: Systematic Review and Meta-Analysis of Functional Near-Infrared Spectroscopy Studies

(1) Functional near-infrared spectroscopy (fNIRS) provides a useful tool for monitoring brain activation changes while walking in adults with neurological disorders. When combined with dual task walking paradigms, fNIRS allows for changes in brain activation to be monitored when individuals concurrently attend to multiple tasks. However, differences in dual task paradigms, baseline, and coverage of cortical areas, presents uncertainty in the interpretation of the overarching findings. (2) Methods: By conducting a systematic review of 35 studies and meta-analysis of 75 effect sizes from 17 studies on adults with or without neurological disorders, we show that the performance of obstacle walking, serial subtraction and letter generation tasks while walking result in significant increases in brain activation in the prefrontal cortex relative to standing or walking baselines. (3) Results: Overall, we find that letter generation tasks have the largest brain activation effect sizes relative to walking, and that significant differences between dual task and single task gait are seen in persons with multiple sclerosis and stroke. (4) Conclusions: Older adults with neurological disease generally showed increased brain activation suggesting use of more attentional resources during dual task walking, which could lead to increased fall risk and mobility impairments. PROSPERO ID: 235228.

Time in Brain: How Biological Rhythms Impact on EEG Signals and on EEG-Derived Brain Networks

Electroencephalography (EEG) is a widely employed tool for exploring brain dynamics and is used extensively in various domains, ranging from clinical diagnosis via neuroscience, cognitive science, cognitive psychology, psychophysiology, neuromarketing, neurolinguistics, and pharmacology to research on brain computer interfaces. EEG is the only technique that enables the continuous recording of brain dynamics over periods of time that range from a few seconds to hours and days and beyond. When taking long-term recordings, various endogenous and exogenous biological rhythms may impinge on characteristics of EEG signals. While the impact of the circadian rhythm and of ultradian rhythms on spectral characteristics of EEG signals has been investigated for more than half a century, only little is known on how biological rhythms influence characteristics of brain dynamics assessed with modern EEG analysis techniques. At the example of multiday, multichannel non-invasive and invasive EEG recordings, we here discuss the impact of biological rhythms on temporal changes of various characteristics of human brain dynamics: higher-order statistical moments and interaction properties of multichannel EEG signals as well as local and global characteristics of EEG-derived evolving functional brain networks. Our findings emphasize the need to take into account the impact of biological rhythms in order to avoid erroneous statements about brain dynamics and about evolving functional brain networks.