Motor imagery and mental fatigue: inter-relationship and EEG based estimation

Chrono-EEG dynamics influencing hand gesture decoding: a 10-hour study

Long-term electroencephalography (EEG) recordings have primarily been used to study resting-state fluctuations. These recordings provide valuable insights into various phenomena such as sleep stages, cognitive processes, and neurological disorders. However, this study explores a new angle, focusing for the first time on the evolving nature of EEG dynamics over time within the context of movement. Twenty-two healthy individuals were measured six times from 2 p.m. to 12 a.m. with intervals of 2 h while performing four right-hand gestures. Analysis of movement-related cortical potentials (MRCPs) revealed a reduction in amplitude for the motor and post-motor potential during later hours of the day. Evaluation in source space displayed an increase in the activity of M1 of the contralateral hemisphere and the SMA of both hemispheres until 8 p.m. followed by a decline until midnight. Furthermore, we investigated how changes over time in MRCP dynamics affect the ability to decode motor information. This was achieved by developing classification schemes to assess performance across different scenarios. The observed variations in classification accuracies over time strongly indicate the need for adaptive decoders. Such adaptive decoders would be instrumental in delivering robust results, essential for the practical application of BCIs during day and nighttime usage.

User Evaluation of a Shared Robot Control System Combining BCI and Eye Tracking in a Portable Augmented Reality User Interface

This study evaluates an innovative control approach to assistive robotics by integrating brain-computer interface (BCI) technology and eye tracking into a shared control system for a mobile augmented reality user interface. Aimed at enhancing the autonomy of individuals with physical disabilities, particularly those with impaired motor function due to conditions such as stroke, the system utilizes BCI to interpret user intentions from electroencephalography signals and eye tracking to identify the object of focus, thus refining control commands. This integration seeks to create a more intuitive and responsive assistive robot control strategy. The real-world usability was evaluated, demonstrating significant potential to improve autonomy for individuals with severe motor impairments. The control system was compared with an eye-tracking-based alternative to identify areas needing improvement. Although BCI achieved an acceptable success rate of 0.83 in the final phase, eye tracking was more effective with a perfect success rate and consistently lower completion times (p<0.001). The user experience responses favored eye tracking in 11 out of 26 questions, with no significant differences in the remaining questions, and subjective fatigue was higher with BCI use (p=0.04). While BCI performance lagged behind eye tracking, the user evaluation supports the validity of our control strategy, showing that it could be deployed in real-world conditions and suggesting a pathway for further advancements.

Research on mental fatigue during long-term motor imagery: a pilot study

Mental fatigue during long-term motor imagery (MI) may affect intention recognition in MI applications. However, the current research lacks the monitoring of mental fatigue during MI and the definition of robust biomarkers. The present study aims to reveal the effects of mental fatigue on motor imagery recognition at the brain region level and explore biomarkers of mental fatigue. To achieve this, we recruited 10 healthy participants and asked them to complete a long-term motor imagery task involving both right- and left-handed movements. During the experiment, we recorded 32-channel EEG data and carried out a fatigue questionnaire for each participant. As a result, we found that mental fatigue significantly decreased the subjects' motor imagery recognition rate during MI. Additionally the theta power of frontal, central, parietal, and occipital clusters significantly increased after the presence of mental fatigue. Furthermore, the phase synchronization between the central cluster and the frontal and occipital lobes was significantly weakened. To summarize, the theta bands of frontal, central, and parieto-occipital clusters may serve as powerful biomarkers for monitoring mental fatigue during motor imagery. Additionally, changes in functional connectivity between the central cluster and the prefrontal and occipital lobes during motor imagery could be investigated as potential biomarkers.

BCI-controlled wheelchairs: end-users' perceptions, needs, and expectations, an interview-based study

PURPOSE

Brain-computer interface (BCI)-controlled wheelchairs have the potential to improve the independence of people with mobility impairments. The low uptake of BCI devices has been linked to a lack of knowledge among researchers of the needs of end-users that should influence BCI development.

MATERIALS AND METHODS

This study used semi-structured interviews to learn about the perceptions, needs, and expectations of spinal cord injury (SCI) patients with regards to a BCI-controlled wheelchair. Topics discussed in the interview include: paradigms, shared control, safety, robustness, channel selection, hardware, and experimental design. The interviews were recorded and then transcribed. Analysis was carried out using coding based on grounded theory principles.

RESULTS

The majority of participants had a positive view of BCI-controlled wheelchair technology and were willing to use the technology. Core issues were raised regarding safety, cost and aesthetics. Interview discussions were linked to state-of-the-art BCI technology. The results challenge the current reliance of researchers on the motor-imagery paradigm by suggesting end-users expect highly intuitive paradigms. There also needs to be a stronger focus on obstacle avoidance and safety features in BCI wheelchairs. Finally, the development of control approaches that can be personalized for individual users may be instrumental for widespread adoption of these devices.

CONCLUSIONS

This study, based on interviews with SCI patients, indicates that BCI-controlled wheelchairs are a promising assistive technology that would be well received by end-users. Recommendations for a more person-centered design of BCI controlled wheelchairs are made and clear avenues for future research are identified.

Fatigue in children using motor imagery and P300 brain-computer interfaces

BACKGROUND

Brain-computer interface (BCI) technology offers children with quadriplegic cerebral palsy unique opportunities for communication, environmental exploration, learning, and game play. Research in adults demonstrates a negative impact of fatigue on BCI enjoyment, while effects on BCI performance are variable. To date, there have been no pediatric studies of BCI fatigue. The purpose of this study was to assess the effects of two different BCI paradigms, motor imagery and visual P300, on the development of self-reported fatigue and an electroencephalography (EEG) biomarker of fatigue in typically developing children.

METHODS

Thirty-seven typically-developing school-aged children were recruited to a prospective, crossover study. Participants attended three sessions: (A) motor imagery-BCI, (B) visual P300-BCI, and (C) video viewing (control). The motor imagery task involved an imagined left- or right-hand squeeze. The P300 task involved attending to one square on a 3 × 3 grid during a random single flash sequence. Each paradigm had respective calibration periods and a similar visual counting game. Primary outcomes were self-reported fatigue and the power of the EEG alpha band both collected during resting-state periods pre- and post-task. Self-reported fatigue was measured using a 10-point visual analog scale. EEG alpha band power was calculated as the integrated power spectral density from 8 to 12 Hz of the EEG spectrum.

RESULTS

Thirty-two children completed the protocol (age range 7-16, 63% female). Self-reported fatigue and EEG alpha band power increased across all sessions (F(1,155) = 33.9, p < 0.001; F = 5.0(1,149), p = 0.027 respectively). No differences in fatigue development were observed between session types. There was no correlation between self-reported fatigue and EEG alpha band power change. BCI performance varied between participants and paradigms as expected but was not associated with self-reported fatigue or EEG alpha band power.

CONCLUSION

Short periods (30-mintues) of BCI use can increase self-reported fatigue and EEG alpha band power to a similar degree in children performing motor imagery and P300 BCI paradigms. Performance was not associated with our measures of fatigue; the impact of fatigue on useability and enjoyment is unclear. Our results reflect the variability of fatigue and the BCI experience more broadly in children and warrant further investigation.

Intermittent hypoxia training effectively protects against cognitive decline caused by acute hypoxia exposure

Intermittent hypoxia training (IHT) is a promising approach that has been used to induce acclimatization to hypoxia and subsequently lower the risk of developing acute mountain sickness (AMS). However, the effects of IHT on cognitive and cerebrovascular function after acute hypoxia exposure have not been characterized. In the present study, we first confirmed that the simplified IHT paradigm was effective at relieving AMS at 4300 m. Second, we found that IHT improved participants' cognitive and neural alterations when they were exposed to hypoxia. Specifically, impaired working memory performance, decreased conflict control function, impaired cognitive control, and aggravated mental fatigue induced by acute hypoxia exposure were significantly alleviated in the IHT group. Furthermore, a reversal of brain swelling induced by acute hypoxia exposure was visualized in the IHT group using magnetic resonance imaging. An increase in cerebral blood flow (CBF) was observed in multiple brain regions of the IHT group after hypoxia exposure as compared with the control group. Based on these findings, the simplified IHT paradigm might facilitate hypoxia acclimatization, alleviate AMS symptoms, and increase CBF in multiple brain regions, thus ameliorating brain swelling and cognitive dysfunction.

Transferring a deep learning model from healthy subjects to stroke patients in a motor imagery brain-computer interface

Objective.Motor imagery (MI) brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have been developed primarily for stroke rehabilitation, however, due to limited stroke data, current deep learning methods for cross-subject classification rely on healthy data. This study aims to assess the feasibility of applying MI-BCI models pre-trained using data from healthy individuals to detect MI in stroke patients.Approach.We introduce a new transfer learning approach where features from two-class MI data of healthy individuals are used to detect MI in stroke patients. We compare the results of the proposed method with those obtained from analyses within stroke data. Experiments were conducted using Deep ConvNet and state-of-the-art subject-specific machine learning MI classifiers, evaluated on OpenBMI two-class MI-EEG data from healthy subjects and two-class MI versus rest data from stroke patients.Main results.Results of our study indicate that through domain adaptation of a model pre-trained using healthy subjects' data, an average MI detection accuracy of 71.15% (±12.46%) can be achieved across 71 stroke patients. We demonstrate that the accuracy of the pre-trained model increased by 18.15% after transfer learning (p<0.001). Additionally, the proposed transfer learning method outperforms the subject-specific results achieved by Deep ConvNet and FBCSP, with significant enhancements of 7.64% (p<0.001) and 5.55% (p<0.001) in performance, respectively. Notably, the healthy-to-stroke transfer learning approach achieved similar performance to stroke-to-stroke transfer learning, with no significant difference (p>0.05). Explainable AI analyses using transfer models determined channel relevance patterns that indicate contributions from the bilateral motor, frontal, and parietal regions of the cortex towards MI detection in stroke patients.Significance.Transfer learning from healthy to stroke can enhance the clinical use of BCI algorithms by overcoming the challenge of insufficient clinical data for optimal training.

Differences in Motor Imagery Ability between People with Parkinson's Disease and Healthy Controls, and Its Relationship with Functionality, Independence and Quality of Life

Motor imagery (MI) has been shown to be effective for the acquisition of motor skills; however, it is still unknown whether similar benefits can be achieved in neurological patients. Previous findings of differences in MI ability between people with Parkinson's disease (PwPD) and healthy controls (HCs) are mixed. This study examined differences in the ability to both create and maintain MI as well as investigating the relationship between the ability to create and maintain MI and motor function, independence and quality of life (QoL). A case-control study was conducted (31 PwPD and 31 HCs), collecting gender, age, dominance, socio-demographic data, duration and impact of the disease. MI intensity (MIQ-RS and KVIQ-34) and temporal accuracy of MI (imagined box and block test [iBBT], imagined timed stand and walk test [iTUG]) were assessed. Functional and clinical assessments included upper limb motor function, balance, gait, independence in activities of daily living and quality of life measures. Statistically significant differences in temporal accuracy were observed and partial and weak relationships were revealed between MI measures and functioning, independence and QoL. PwPD retain the ability to create MI, indicating the suitability of MI in this population. Temporal accuracy might be altered as a reflection of bradykinesia on the mentally simulated actions.

Online semi-supervised learning for motor imagery EEG classification

OBJECTIVE

Time-consuming data labeling in brain-computer interfaces (BCIs) raises many problems such as mental fatigue and is one key factor that hinders the real-world adoption of motor imagery (MI)-based BCIs. An alternative approach is to integrate readily available, as well as informative, unlabeled data online, whereas this approach is less investigated.

APPROACH

We proposed an online semi-supervised learning scheme to improve the classification performance of MI-based BCI. This scheme uses regularized weighted online sequential extreme learning machine (RWOS-ELM) as the base classifier and updates its model parameters with incoming balanced data chunk-by-chunk. In the initial stage, we designed a technique that combines the synthetic minority oversampling with the edited nearest neighbor rule for data augmentation to construct more discriminative initial classifiers. When used online, the incoming chunk of data is first pseudo-labeled by RWOS-ELM as well as an auxiliary classifier, and then balanced again by the above-mentioned technique. Initial classifiers are further updated based on these class-balanced data.

MAIN RESULTS

Offline experimental results on two publicly available MI datasets demonstrate the superiority of the proposed scheme over its counterparts. Further online experiments on six subjects show that their BCI performance gradually improved by learning from incoming unlabeled data.

SIGNIFICANCE

Our proposed online semi-supervised learning scheme has higher computation and memory usage efficiency, which is promising for online MI-based BCIs, especially in the case of insufficient labeled training data.

Online Ternary Classification of Covert Speech by Leveraging the Passive Perception of Speech

Brain-computer interfaces (BCIs) provide communicative alternatives to those without functional speech. Covert speech (CS)-based BCIs enable communication simply by thinking of words and thus have intuitive appeal. However, an elusive barrier to their clinical translation is the collection of voluminous examples of high-quality CS signals, as iteratively rehearsing words for long durations is mentally fatiguing. Research on CS and speech perception (SP) identifies common spatiotemporal patterns in their respective electroencephalographic (EEG) signals, pointing towards shared encoding mechanisms. The goal of this study was to investigate whether a model that leverages the signal similarities between SP and CS can differentiate speech-related EEG signals online. Ten participants completed a dyadic protocol where in each trial, they listened to a randomly selected word and then subsequently mentally rehearsed the word. In the offline sessions, eight words were presented to participants. For the subsequent online sessions, the two most distinct words (most separable in terms of their EEG signals) were chosen to form a ternary classification problem (two words and rest). The model comprised a functional mapping derived from SP and CS signals of the same speech token (features are extracted via a Riemannian approach). An average ternary online accuracy of 75.3% (60% chance level) was achieved across participants, with individual accuracies as high as 93%. Moreover, we observed that the signal-to-noise ratio (SNR) of CS signals was enhanced by perception-covert modeling according to the level of high-frequency ([Formula: see text]-band) correspondence between CS and SP. These findings may lead to less burdensome data collection for training speech BCIs, which could eventually enhance the rate at which the vocabulary can grow.

A Quantitative Investigation of Mental Fatigue Elicited during Motor Imagery Practice: Selective Effects on Maximal Force Performance and Imagery Ability

In the present study, we examined the development of mental fatigue during the kinesthetic motor imagery (MI) of isometric force contractions performed with the dominant upper limb. Participants (n = 24) underwent four blocks of 20 MI trials of isometric contractions at 20% of the maximal voluntary contraction threshold (20% MVC_MI) and 20 MI trials of maximal isometric contractions (100% MVC_MI). Mental fatigue was assessed after each block using a visual analogue scale (VAS). We assessed maximal isometric force before, during and after MI sessions. We also assessed MI ability from self-report ratings and skin conductance recordings. Results showed a logarithmic pattern of increase in mental fatigue over the course of MI, which was superior during 100% MVC_MI. Unexpectedly, maximal force improved during 100% MVC_MI between the 1st and 2nd evaluations but remained unchanged during 20% MVC_MI. MI ease and vividness improved during 100% MVC_MI, with a positive association between phasic skin conductance and VAS mental fatigue scores. Conversely, subjective measures revealed decreased MI ability during 20% MVC_MI. Mental fatigue did not hamper the priming effects of MI on maximal force performance, nor MI's ability for tasks involving high physical demands. By contrast, mental fatigue impaired MI vividness and elicited boredom effects in the case of motor tasks with low physical demands.

Source-Based EEG Neurofeedback for Sustained Motor Imagery of a Single Leg

The aim of the study was to test the feasibility of visual-neurofeedback-guided motor imagery (MI) of the dominant leg, based on source analysis with real-time sLORETA derived from 44 EEG channels. Ten able-bodied participants took part in two sessions: session 1 sustained MI without feedback and session 2 sustained MI of a single leg with neurofeedback. MI was performed in 20 s on and 20 s off intervals to mimic functional magnetic resonance imaging. Neurofeedback in the form of a cortical slice presenting the motor cortex was provided from a frequency band with the strongest activity during real movements. The sLORETA processing delay was 250 ms. Session 1 resulted in bilateral/contralateral activity in the 8-15 Hz band dominantly over the prefrontal cortex while session 2 resulted in ipsi/bilateral activity over the primary motor cortex, covering similar areas as during motor execution. Different frequency bands and spatial distributions in sessions with and without neurofeedback may reflect different motor strategies, most notably a larger proprioception in session 1 and operant conditioning in session 2. Single-leg MI might be used in the early phases of rehabilitation of stroke patients. Simpler visual feedback and motor cueing rather than sustained MI might further increase the intensity of cortical activation.

Enhancing motor imagery detection efficacy using multisensory virtual reality priming

Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent directly from electroencephalography (EEG) signals. For some users, however, it is difficult to elicit patterns in the EEG signal that can be detected with existing features and classifiers. As such, new user control strategies and training paradigms have been highly sought-after to help improve motor imagery performance. Virtual reality (VR) has emerged as one potential tool where improvements in user engagement and level of immersion have shown to improve BCI accuracy. Motor priming in VR, in turn, has shown to further enhance BCI accuracy. In this pilot study, we take the first steps to explore if multisensory VR motor priming, where haptic and olfactory stimuli are present, can improve motor imagery detection efficacy in terms of both improved accuracy and faster detection. Experiments with 10 participants equipped with a biosensor-embedded VR headset, an off-the-shelf scent diffusion device, and a haptic glove with force feedback showed that significant improvements in motor imagery detection could be achieved. Increased activity in the six common spatial pattern filters used were also observed and peak accuracy could be achieved with analysis windows that were 2 s shorter. Combined, the results suggest that multisensory motor priming prior to motor imagery could improve detection efficacy.

Review of public motor imagery and execution datasets in brain-computer interfaces

The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.

BCI-activated electrical stimulation in children with perinatal stroke and hemiparesis: A pilot study

Background

Perinatal stroke (PS) causes most hemiparetic cerebral palsy (CP) and results in lifelong disability. Children with severe hemiparesis have limited rehabilitation options. Brain computer interface- activated functional electrical stimulation (BCI-FES) of target muscles may enhance upper extremity function in hemiparetic adults. We conducted a pilot clinical trial to assess the safety and feasibility of BCI-FES in children with hemiparetic CP.

Methods

Thirteen participants (mean age = 12.2 years, 31% female) were recruited from a population-based cohort. Inclusion criteria were: (1) MRI-confirmed PS, (2) disabling hemiparetic CP, (3) age 6–18 years, (4) informed consent/assent. Those with neurological comorbidities or unstable epilepsy were excluded. Participants attended two BCI sessions: training and rehabilitation. They wore an EEG-BCI headset and two forearm extensor stimulation electrodes. Participants’ imagination of wrist extension was classified on EEG, after which muscle stimulation and visual feedback were provided when the correct visualization was detected.

Results

No serious adverse events or dropouts occurred. The most common complaints were mild headache, headset discomfort and muscle fatigue. Children ranked the experience as comparable to a long car ride and none reported as unpleasant. Sessions lasted a mean of 87 min with 33 min of stimulation delivered. Mean classification accuracies were (M = 78.78%, SD = 9.97) for training and (M = 73.48, SD = 12.41) for rehabilitation. Mean Cohen’s Kappa across rehabilitation trials was M = 0.43, SD = 0.29, range = 0.019–1.00, suggesting BCI competency.

Conclusion

Brain computer interface-FES was well -tolerated and feasible in children with hemiparesis. This paves the way for clinical trials to optimize approaches and test efficacy.

Immediate effects of short-term meditation on sensorimotor rhythm-based brain-computer interface performance

Introduction

Meditation has been shown to enhance a user's ability to control a sensorimotor rhythm (SMR)-based brain-computer interface (BCI). For example, prior work have demonstrated that long-term meditation practices and an 8-week mindfulness-based stress reduction (MBSR) training have positive behavioral and neurophysiological effects on SMR-based BCI. However, the effects of short-term meditation practice on SMR-based BCI control are still unknown.

Methods

In this study, we investigated the immediate effects of a short, 20-minute meditation on SMR-based BCI control. Thirty-seven subjects performed several runs of one-dimensional cursor control tasks before and after two types of 20-minute interventions: a guided mindfulness meditation exercise and a recording of a narrator reading a journal article.

Results

We found that there is no significant change in BCI performance and Electroencephalography (EEG) BCI control signal following either 20-minute intervention. Moreover, the change in BCI performance between the meditation group and the control group was found to be not significant.

Discussion

The present results suggest that a longer period of meditation is needed to improve SMR-based BCI control.

Continuous Repetition Motor Imagery Training and Physical Practice Training Exert the Growth of Fatigue and Its Effect on Performance

Continuous repetition of motor imagery leads to mental fatigue. This study aimed to examine whether fatigue caused by motor imagery training affects improvement in performance and the change in corticospinal excitability. The participants were divided into “physical practice training” and “motor imagery training” groups, and a visuomotor task (set at 50% of maximal voluntary contraction in participants) was performed to assess the training effect on fatigue. The measurements were recorded before and after training. Corticospinal excitability at rest was measured by transcranial magnetic stimulation according to the Neurophysiological Index. Subjective mental fatigue and muscle fatigue were assessed by using the visual analog scale and by measuring the pinch force, respectively. Additionally, the error area was evaluated and calculated at pre-, mid-, and post-terms after training, using a visuomotor task. After training, muscle fatigue, subjective mental fatigue, and decreased corticospinal excitability were noted in both of the groups. Moreover, the visuomotor task decreased the error area by training; however, there was no difference in the error area between the mid- and post-terms. In conclusion, motor imagery training resulted in central fatigue by continuous repetition, which influenced the improvement in performance in the same manner as physical practice training.

Influence of the number of trials on evoked motor cortical activity in EEG recordings

OBJECTIVE

Improvements in electroencephalography enable the study of the localization of active brain regions during motor tasks. Movement-related cortical potentials (MRCPs), and event-related desynchronization (ERD) and synchronization (ERS) are the main motor-related cortical phenomena/neural correlates observed when a movement is elicited. When assessing neurological diseases, averaging techniques are commonly applied to characterize motor related processes better. In this case, a large number of trials is required to obtain a motor potential that is representative enough of the subject's condition. This study aimed to assess the effect of a limited number of trials on motor-related activity corresponding to different upper limb movements (elbow flexion/extension, pronation/supination and hand open/close).

APPROACH

An open dataset consisting on 15 healthy subjects was used for the analysis. A Monte Carlo simulation approach was applied to analyse, in a robust way, different typical time- and frequency-domain features, topography, and low-resolution tomography (LORETA).

MAIN RESULTS

Grand average potentials, and topographic and tomographic maps showed few differences when using fewer trials, but shifts in the localization of motor-related activity were found for several individuals. MRCP and beta ERD features were more robust to a limited number of trials, yielding differences lower than 20% for cases with 50 trials or more. Strong correlations between features were obtained for subsets above 50 trials. However, the inter-subject variability increased as the number of trials decreased. The elbow flexion/extension movement showed a more robust performance for a limited number of trials, both in population and in individual-based analysis.

SIGNIFICANCE

Our findings suggested that 50 trials can be an appropriate number to obtain stable motor-related features in terms of differences in the averaged motor features, correlation, and changes in topography and tomography.

Analysis of the Relationship Between Motor Imagery and Age-Related Fatigue for CNN Classification of the EEG Data

Background

The aging of the world population poses a major health challenge, and brain–computer interface (BCI) technology has the potential to provide assistance and rehabilitation for the elderly.

Objectives

This study aimed to investigate the electroencephalogram (EEG) characteristics during motor imagery by comparing young and elderly, and study Convolutional Neural Networks (CNNs) classification for the elderly population in terms of fatigue analysis in both frontal and parietal regions.

Methods

A total of 20 healthy individuals participated in the study, including 10 young and 10 older adults. All participants completed the left- and right-hand motor imagery experiment. The energy changes in the motor imagery process were analyzed using time–frequency graphs and quantified event-related desynchronization (ERD) values. The fatigue level of the motor imagery was assessed by two indicators: (θ + α)/β and θ/β, and fatigue-sensitive channels were distinguished from the parietal region of the brain. Then, rhythm entropy was introduced to analyze the complexity of the cognitive activity. The phase-lock values related to the parietal and frontal lobes were calculated, and their temporal synchronization was discussed. Finally, the motor imagery EEG data was classified by CNNs, and the accuracy was discussed based on the analysis results.

Result

For the young and elderly, ERD was observed in C3 and C4 channels, and their fatigue-sensitive channels in the parietal region were slightly different. During the experiment, the rhythm entropy of the frontal lobe showed a decreasing trend with time for most of the young subjects, while there was an increasing trend for most of the older ones. Using the CNN classification method, the elderly achieved around 70% of the average classification accuracy, which is almost the same for the young adults.

Conclusion

Compared with the young adults, the elderly are less affected by the level of cognitive fatigue during motor imagery, but the classification accuracy of motor imagery data in the elderly may be slightly lower than that in young persons. At the same time, the deep learning method also provides a potentially feasible option for the application of motor-imagery BCI (MI-BCI) in the elderly by considering the ERD and fatigue phenomenon together.

Neural Correlates of Motor Imagery during Action Observation in Affordance-based Actions: Preliminary Results

Object affordance, a characterization of the different functionalities of an object, refers to an object's numerous possibilities of interaction. It has a significant part to play in priming motoric actions which depends on the actor's spontaneous neurological behaviour. Action Observation (AO) and Motor Imagery (MI) also lead to the stimulation of motor system. In fact, AO and MI result in activation of overlapping brain areas as the actual motor task. AO combined with MI (referred to as AO+MI) initiates higher cortical activity in comparison with either MI or AO alone. In this paper, we investigate the influence of affordance driven motor priming during AO, MI and AO + MI. Source current density as an EEG parameter is estimated by Low Resolution Electromagnetic Tomography (LORETA). Our results demonstrate that affordance driven motor priming during AO+MI indicates stronger electrophysiological and behavioural changes. This is evident from the N2 ERP component. Further, the current source density (in brain regions associated with motor planning) during affordance driven AO+MI is found to be maximum.

Embodiment Comfort Levels During Motor Imagery Training Combined With Immersive Virtual Reality in a Spinal Cord Injury Patient

Brain-machine interfaces combining visual, auditory, and tactile feedback have been previously used to generate embodiment experiences during spinal cord injury (SCI) rehabilitation. It is not known if adding temperature to these modalities can result in discomfort with embodiment experiences. Here, comfort levels with the embodiment experiences were investigated in an intervention that required a chronic pain SCI patient to generate lower limb motor imagery commands in an immersive environment combining visual (virtual reality -VR), auditory, tactile, and thermal feedback. Assessments were made pre-/ post-, throughout the intervention (Weeks 0-5), and at 7 weeks follow up. Overall, high levels of embodiment in the adapted three-domain scale of embodiment were found throughout the sessions. No significant adverse effects of VR were reported. Although sessions induced only a modest reduction in pain levels, an overall reduction occurred in all pain scales (Faces, Intensity, and Verbal) at follow up. A high degree of comfort in the comfort scale for the thermal-tactile sleeve, in both the thermal and tactile feedback components of the sleeve was reported. This study supports the feasibility of combining multimodal stimulation involving visual (VR), auditory, tactile, and thermal feedback to generate embodiment experiences in neurorehabilitation programs.

Applying Implementation Science to Support Active Collaboration in Noninvasive Brain-Computer Interface Development and Translation for Augmentative and Alternative Communication

PURPOSE

The purpose of this article is to consider how, alongside engineering advancements, noninvasive brain-computer interface (BCI) for augmentative and alternative communication (AAC; BCI-AAC) developments can leverage implementation science to increase the clinical impact of this technology. We offer the Consolidated Framework for Implementation Research (CFIR) as a structure to help guide future BCI-AAC research. Specifically, we discuss CFIR primary domains that include intervention characteristics, the outer and inner settings, the individuals involved in the intervention, and the process of implementation, alongside pertinent subdomains including adaptability, cost, patient needs and recourses, implementation climate, other personal attributes, and the process of engaging. The authors support their view with current citations from both the AAC and BCI-AAC fields.

CONCLUSIONS

The article aimed to provide thoughtful considerations for how future research may leverage the CFIR to support meaningful BCI-AAC translation for those with severe physical impairments. We believe that, although significant barriers to BCI-AAC development still exist, incorporating implementation research may be timely for the field of BCI-AAC and help account for diversity in end users, navigate implementation obstacles, and support a smooth and efficient translation of BCI-AAC technology. Moreover, the sooner clinicians, individuals who use AAC, their support networks, and engineers collectively improve BCI-AAC outcomes and the efficiency of translation, the sooner BCI-AAC may become an everyday tool in the AAC arsenal.

A comparison and classification of oscillatory characteristics in speech perception and covert speech

Covert speech, the mental imagery of speaking, has been studied increasingly to understand and decode thoughts in the context of brain-computer interfaces. In studies of speech comprehension, neural oscillations are thought to play a key role in the temporal encoding of speech. However, little is known about the role of oscillations in covert speech. In this study, we investigated the oscillatory involvements in covert speech and speech perception. Data were collected from 10 participants with 64 channel EEG. Participants heard the words, 'blue' and 'orange', and subsequently mentally rehearsed them. First, continuous wavelet transform was performed on epoched signals and subsequently two-tailed t-tests between two classes were conducted to determine statistical differences in frequency and time (t-CWT). Features were also extracted using t-CWT and subsequently classified using a support vector machine. θ and γ phase amplitude coupling (PAC) was also assessed within and between tasks. All binary classifications produced accuracies significantly greater (80-90%) than chance level, supporting the use of t-CWT in determining relative oscillatory involvements. While the perception task dynamically invoked all frequencies with more prominent θ and α activity, the covert task favoured higher frequencies with significantly higher γ activity than perception. Moreover, the perception condition produced significant θ-γ PAC, corroborating a reported linkage between syllabic and phonemic sampling. Although this coupling was found to be suppressed in the covert condition, we found significant cross-task coupling between perception θ and covert speech γ. Covert speech processing appears to be largely associated with higher frequencies of EEG. Importantly, the significant cross-task coupling between speech perception and covert speech, in the absence of within-task covert speech PAC, supports the notion that the γ- and θ-bands subserve, respectively, shared and unique encoding processes across tasks.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject's real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject's physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

Exploring Fatigue Effects on Performance Variation of Intensive Brain-Computer Interface Practice

Motor imagery (MI) is an endogenous mental process and is commonly used as an electroencephalogram (EEG)-based brain-computer interface (BCI) strategy. Previous studies of P300 and MI-based (without online feedback) BCI have shown that mental states like fatigue can negatively affect participants' EEG signatures. However, exogenous stimuli cause visual fatigue, which might have a different mechanism than endogenous tasks do. Furthermore, subjects could adjust themselves if online feedback is provided. In this sense, it is still unclear how fatigue affects online MI-based BCI performance. With this question, 12 healthy subjects are recruited to investigate this issue, and an MI-based online BCI experiment is performed for four sessions on different days. The first session is for training, and the other three sessions differ in rest condition and duration-no rest, 16-min eyes-open rest, and 16-min eyes-closed rest-arranged in a pseudo-random order. Multidimensional fatigue inventory (MFI) and short stress state questionnaire (SSSQ) reveal that general fatigue, mental fatigue, and distress have increased, while engagement has decreased significantly within certain sessions. However, the BCI performances, including percent valid correct (PVC) and information transfer rate (ITR), show no significant change across 400 trials. The results suggest that although the repetitive MI task has affected subjects' mental states, their BCI performances and feature separability within a session are not affected by the task significantly. Further electrophysiological analysis reveals that the alpha-band power in the sensorimotor area has an increasing tendency, while event-related desynchronization (ERD) modulation level has a decreasing trend. During the rest time, no physiological difference has been found in the eyes-open rest condition; on the contrary, the alpha-band power increase and subsequent decrease appear in the eyes-closed rest condition. In summary, this experiment shows evidence that mental states can change dramatically in the intensive MI-BCI practice, but BCI performances could be maintained.

Toward EEG-Based BCI Applications for Industry 4.0: Challenges and Possible Applications

In the last few decades, Brain-Computer Interface (BCI) research has focused predominantly on clinical applications, notably to enable severely disabled people to interact with the environment. However, recent studies rely mostly on the use of non-invasive electroencephalographic (EEG) devices, suggesting that BCI might be ready to be used outside laboratories. In particular, Industry 4.0 is a rapidly evolving sector that aims to restructure traditional methods by deploying digital tools and cyber-physical systems. BCI-based solutions are attracting increasing attention in this field to support industrial performance by optimizing the cognitive load of industrial operators, facilitating human-robot interactions, and make operations in critical conditions more secure. Although these advancements seem promising, numerous aspects must be considered before developing any operational solutions. Indeed, the development of novel applications outside optimal laboratory conditions raises many challenges. In the current study, we carried out a detailed literature review to investigate the main challenges and present criteria relevant to the future deployment of BCI applications for Industry 4.0.

Revisiting the acute effects of resistance exercise on motor imagery ability

Motor imagery (MI) shares psychological and physiological similarities with the physical practice of the same action. Yet, it remains unclear whether fatigue elicited by exercise impairs MI ability. Fourteen participants performed MI of a self-paced walking sequence of 22 m before and after a resistance exercise eliciting muscle fatigue from upper and lower limbs, selectively. We indexed MI ability using psychometric and behavioral methods. Electromyography of the quadriceps was also recorded during physical practice trials of the walking sequence. For both experimental conditions, we recorded improved temporal congruence between MI and physical practice of the walking sequence (9.89 %, 95 % CI [7.03, 12.75], p < 0.01). Vividness decreased immediately after the fatiguing exercise (6.35 %, 95 % CI [5.18, 7.51], p < 0.05), before rapidly returning to pre-fatigue values during recovery trials. The results challenge the hypothesis of an effect of acute fatigue elicited by a resistance exercise on MI ability, i.e. restricted to MI tasks focusing fatigued effectors. The beneficial effects of fatigue conditions on the psychometric and behavioral indexes of MI ability are discussed in the broader context of psychobiological fatigue models linking perceived exertion with the reallocation of attentional resources. The general perception of fatigue, rather than local muscle fatigue, appeared linked to the acute effects of resistance exercise on MI ability.

Neurophysiological mechanisms and features of autonomic support of productive cognitive activity of intuitive type in young adults

Is intuition a conscious or subconscious process, a “sixth sense” or a product of learning? This article contains an answer to this question which is based on the disclosure of neurophysiological mechanisms of one of the least understandable types of human cognitive activity. For the first time with the use of cerebral cortex electrical activity mapping, a comprehensive study of the organization of cortical neural networks and the functional state of the autonomic nervous system of female biology students engaged in intuitive thinking has been conducted. The productivity of intuitive thinking is associated with increasing the spectral power of beta2-, delta-, theta-components of the electroencephalogram. The synchronization of the activity of most frequency bands is manifested in the frontal and motor areas of the cortex, which have close connections with the basal ganglia, which are responsible for the formation of skills. In the beta1-band there are probable intrahemispheric long coherences between the anterior and posterior parts of the right hemisphere, at the frequency of beta1,2- and theta-bands of the electroencephalogram they are combined into large cellular ensembles that cover the central-parietal-temporal loci of the cortex. Synchronization of biopotentials in the delta range covers large areas of the cerebral cortex. It has been established that in intuitive thinking the female students with a low standard of efficiency had a high level of central rhythm regulation, and the female students with a high standard of efficiency had a high level of autonomous regulation. Higher performance under intuitive thinking negatively correlated with low frequency findings and positively correlated with high frequency and the value of the square root of the mean squares of the intervals between heartbeats (rMSSD) in the structure of cardiorhythm. Correlation analysis found that productive mental activity is conditioned by the specific integration of cortico-visceral processes: productive intuitive thinking is associated with the activation of autonomic regulation of heart rate variability and coherence in the evolutionarily older delta and delta-theta systems of the brain. Thus, intuition is a scientific set of skills and knowledge, and the topographic signs of synchronization of electrical processes of the cerebral cortex can serve as objective criteria for successful intuitive thinking, which allow one to predict both individual abilities and the state that contributes to their realization.

Evaluating person-centered factors associated with brain-computer interface access to a commercial augmentative and alternative communication paradigm

Current BCI-AAC systems largely utilize custom-made software and displays that may be unfamiliar to AAC stakeholders. Further, there is limited information available exploring the heterogenous profiles of individuals who may use BCI-AAC. Therefore, in this study, we aimed to evaluate how individuals with amyotrophic lateral sclerosis (ALS) learned to control a motor-based BCI switch in a row-column AAC scanning pattern, and person-centered factors associated with BCI-AAC performance. Four individuals with ALS completed 12 BCI-AAC training sessions, and three individuals without neurological impairment completed 3 BCI-AAC training sessions. To assess person-centered factors associated with BCI-AAC performance, participants completed both initial and recurring assessment measures including levels of cognition, motor ability, fatigue, and motivation. Three of four participants demonstrated either BCI-AAC performance in the range of neurotypical peers, or an improving BCI-AAC learning trajectory. However, BCI-AAC learning trajectories were variable. Assessment measures revealed that two participants presented with a suspicion for cognitive impairment yet achieved the highest levels of BCI-AAC accuracy with their increased levels of performance being possibly supported by largely unimpaired motor skills. Motor-based BCI switch access to a commercial AAC row-column scanning may be feasible for individuals with ALS and possibly supported by timely intervention.

Developing a Motor Imagery-Based Real-Time Asynchronous Hybrid BCI Controller for a Lower-Limb Exoskeleton

This study aimed to develop an intuitive gait-related motor imagery (MI)-based hybrid brain-computer interface (BCI) controller for a lower-limb exoskeleton and investigate the feasibility of the controller under a practical scenario including stand-up, gait-forward, and sit-down. A filter bank common spatial pattern (FBCSP) and mutual information-based best individual feature (MIBIF) selection were used in the study to decode MI electroencephalogram (EEG) signals and extract a feature matrix as an input to the support vector machine (SVM) classifier. A successive eye-blink switch was sequentially combined with the EEG decoder in operating the lower-limb exoskeleton. Ten subjects demonstrated more than 80% accuracy in both offline (training) and online. All subjects successfully completed a gait task by wearing the lower-limb exoskeleton through the developed real-time BCI controller. The BCI controller achieved a time ratio of 1.45 compared with a manual smartwatch controller. The developed system can potentially be benefit people with neurological disorders who may have difficulties operating manual control.

Spatial constraints and cognitive fatigue affect motor imagery of walking in people with multiple sclerosis

Motor imagery (MI) is the mental simulation of an action without any overt motor execution. Interestingly, a temporal coupling between durations of real and imagined movements, i.e., the so-called isochrony principle, has been demonstrated in healthy adults. On the contrary, anisochrony has frequently been reported in elderly subjects or those with neurological disease such as Parkinson disease or multiple sclerosis (MS). Here, we tested whether people with MS (PwMS) may have impaired MI when they imagined themselves walking on paths with different widths. When required to mentally simulate a walking movement along a constrained pathway, PwMS tended to overestimate mental movement duration with respect to actual movement duration. Interestingly, in line with previous evidence, cognitive fatigue was found to play a role in the MI of PwMS. These results suggest that investigating the relationship between cognitive fatigue and MI performances could be key to shedding new light on the motor representation of PwMS and providing critical insights into effective and tailored rehabilitative treatments.

Multiclass Classification Based on Combined Motor Imageries

Motor imagery (MI) allows the design of self-paced brain–computer interfaces (BCIs), which can potentially afford an intuitive and continuous interaction. However, the implementation of non-invasive MI-based BCIs with more than three commands is still a difficult task. First, the number of MIs for decoding different actions is limited by the constraint of maintaining an adequate spacing among the corresponding sources, since the electroencephalography (EEG) activity from near regions may add up. Second, EEG generates a rather noisy image of brain activity, which results in a poor classification performance. Here, we propose a solution to address the limitation of identifiable motor activities by using combined MIs (i.e., MIs involving 2 or more body parts at the same time). And we propose two new multilabel uses of the Common Spatial Pattern (CSP) algorithm to optimize the signal-to-noise ratio, namely MC2CMI and MC2SMI approaches. We recorded EEG signals from seven healthy subjects during an 8-class EEG experiment including the rest condition and all possible combinations using the left hand, right hand, and feet. The proposed multilabel approaches convert the original 8-class problem into a set of three binary problems to facilitate the use of the CSP algorithm. In the case of the MC2CMI method, each binary problem groups together in one class all the MIs engaging one of the three selected body parts, while the rest of MIs that do not engage the same body part are grouped together in the second class. In this way, for each binary problem, the CSP algorithm produces features to determine if the specific body part is engaged in the task or not. Finally, three sets of features are merged together to predict the user intention by applying an 8-class linear discriminant analysis. The MC2SMI method is quite similar, the only difference is that any of the combined MIs is considered during the training phase, which drastically accelerates the calibration time. For all subjects, both the MC2CMI and the MC2SMI approaches reached a higher accuracy than the classic pair-wise (PW) and one-vs.-all (OVA) methods. Our results show that, when brain activity is properly modulated, multilabel approaches represent a very interesting solution to increase the number of commands, and thus to provide a better interaction.

A review of user training methods in brain computer interfaces based on mental tasks

Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.

Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue

OBJECTIVE

Electroencephalogram (EEG) signals are non-stationary. This could be due to internal fluctuation of brain states such as fatigue, frustration, etc. This necessitates the development of adaptive brain-computer interfaces (BCI) whose performance does not deteriorate significantly with the adversary change in the cognitive state. In this paper, we put forward an unsupervised adaptive scheme to adapt the feature extractor of motor imagery (MI) BCIs by tracking the fatigue level of the user.

APPROACH

Eleven subjects participated in the study during which they accomplished MI tasks while self-reporting their perceived levels of mental fatigue. Out of the 11 subjects, only six completed the whole experiment, while the others quit in the middle because of experiencing high fatigue. The adaptive feature extractor is attained through the adaptation of the common spatial patterns (CSP), one of the most popular feature extraction algorithms in EEG-based BCIs. The proposed method was analyzed in two ways: offline and in near real-time. The separability of the MI EEG features extracted by the proposed adaptive CSP (ADCSP) has been compared with that by the conventional CSP (C-CSP) and another CSP based adaptive method (ACSP) in terms of: Davies Bouldin index (DBI), Fisher score (FS) and Dunn's index (DI).

MAIN RESULTS

Experimental results show significant improvement in the separability of MI EEG features extracted by ADCSP as compared to that by C-CSP and ACSP.

SIGNIFICANCE

Collectively, the results of the experiments in this study suggest that adapting CSP based on mental fatigue can improve the class separability of MI EEG features.

Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment

This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.