Signal alignment for cross-datasets in P300 brain-computer interfaces

Objective.Transfer learning has become an important issue in the brain-computer interface (BCI) field, and studies on subject-to-subject transfer within the same dataset have been performed. However, few studies have been performed on dataset-to-dataset transfer, including paradigm-to-paradigm transfer. In this study, we propose a signal alignment (SA) for P300 event-related potential (ERP) signals that is intuitive, simple, computationally less expensive, and can be used for cross-dataset transfer learning.Approach.We proposed a linear SA that uses the P300's latency, amplitude scale, and reverse factor to transform signals. For evaluation, four datasets were introduced (two from conventional P300 Speller BCIs, one from a P300 Speller with face stimuli, and the last from a standard auditory oddball paradigm).Results.Although the standard approach without SA had an average precision (AP) score of 25.5%, the approach demonstrated a 35.8% AP score, and we observed that the number of subjects showing improvement was 36.0% on average. Particularly, we confirmed that the Speller dataset with face stimuli was more comparable with other datasets.Significance.We proposed a simple and intuitive way to align ERP signals that uses the characteristics of ERP signals. The results demonstrated the feasibility of cross-dataset transfer learning even between datasets with different paradigms.

Comparison of recognition methods for an asynchronous (un-cued) BCI system: an investigation with 40-class SSVEP dataset

Steady-state visual evoked potential (SSVEP)-based brain-computer Interface (BCI) has demonstrated the potential to manage multi-command targets to achieve high-speed communication. Recent studies on multi-class SSVEP-based BCI have focused on synchronous systems, which rely on predefined time and task indicators; thus, these systems that use passive approaches may be less suitable for practical applications. Asynchronous systems recognize the user's intention (whether or not the user is willing to use systems) from brain activity; then, after recognizing the user's willingness, they begin to operate by switching swiftly for real-time control. Consequently, various methodologies have been proposed to capture the user's intention. However, in-depth investigation of recognition methods in asynchronous BCI system is lacking. Thus, in this work, three recognition methods (power spectral density analysis, canonical correlation analysis (CCA), and support vector machine (SVM)) used widely in asynchronous SSVEP BCI systems were explored to compare their performance. Further, we categorized asynchronous systems into two approaches (1-stage and 2-stage) based upon the recognition process's design, and compared their performance. To do so, a 40-class SSVEP dataset collected from 40 subjects was introduced. Finally, we found that the CCA-based method in the 2-stage approach demonstrated statistically significantly higher performance with a sensitivity of 97.62 ± 02.06%, specificity of 76.50 ± 23.50%, and accuracy of 75.59 ± 10.09%. Thus, it is expected that the 2-stage approach together with CCA-based recognition and FB-CCA classification have good potential to be implemented in practical asynchronous SSVEP BCI systems.

A comprehensive research setup for monitoring Alzheimer's disease using EEG, fNIRS, and Gait analysis

Alzheimer's disease (AD) has a detrimental impact on brain function, affecting various aspects such as cognition, memory, language, and motor skills. Previous research has dominantly used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) to individually measure brain signals or combine the two methods to target specific brain functions. However, comprehending Alzheimer's disease requires monitoring various brain functions rather than focusing on a single function. This paper presents a comprehensive research setup for a monitoring platform for AD. The platform incorporates a 32-channel dry electrode EEG, a custom-built four-channel fNIRS, and gait monitoring using a depth camera and pressure sensor. Various tasks are employed to target multiple brain functions. The paper introduced the detailed instrumentation of the fNIRS system, which measures the prefrontal cortex, outlines the experimental design targeting various brain functioning programmed in BCI2000 for visualizing EEG signals synchronized with experimental stimulation, and describes the gait monitoring hardware and software and protocol design. The ultimate goal of this platform is to develop an easy-to-perform brain and gait monitoring method for elderly individuals and patients with Alzheimer's disease.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-023-00306-7.

TSANet: multibranch attention deep neural network for classifying tactile selective attention in brain-computer interfaces

Brain-computer interfaces (BCIs) enable communication between the brain and a computer and electroencephalography (EEG) has been widely used to implement BCIs because of its high temporal resolution and noninvasiveness. Recently, a tactile-based EEG task was introduced to overcome the current limitations of visual-based tasks, such as visual fatigue from sustained attention. However, the classification performance of tactile-based BCIs as control signals is unsatisfactory. Therefore, a novel classification approach is required for this purpose. Here, we propose TSANet, a deep neural network, that uses multibranch convolutional neural networks and a feature-attention mechanism to classify tactile selective attention (TSA) in a tactile-based BCI system. We tested TSANet under three evaluation conditions, namely, within-subject, leave-one-out, and cross-subject. We found that TSANet achieved the highest classification performance compared with conventional deep neural network models under all evaluation conditions. Additionally, we show that TSANet extracts reasonable features for TSA by investigating the weights of spatial filters. Our results demonstrate that TSANet has the potential to be used as an efficient end-to-end learning approach in tactile-based BCIs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-023-00309-4.

Pain Classification using Evoked EEG Induced by Thermal Grill Illusion - Deep Neural Network Approach

As the quantification of pain has emerged in biomedical engineering today, studies have been developing biomarkers associated with pain actively by measuring bio-signals such as electroencephalogram (EEG). Recently, some EEG studies of cold and hot pain have been reported. However, they used one type of stimulus condition for each trial and a relatively long stimulation time to collect EEG features. In this study, EEG signals during Cool (20 °C), Warm (40 °C), and Thermal Grill Illusion (TGI, 20-40 °C) stimuli were collected from 43 subjects, and were classified by a deep convolutional neural network referred to as EEGNet. Three binary classifications for the three conditions (TGI, Cool, Warm) were conducted for each subject individually. Classification accuracies for TGI-Cool, TGI-Warm, and Warm-Cool were 0.74±0.01, 0.71±0.01, and 0.74±0.01, respectively. For subjects who rated the TGI significantly hotter than the Warm stimulus, the classification accuracy for TGI-Cool (0.74±0.01) was significantly higher than for TGI-Warm (0.71±0.01). In contrast, the classification accuracy for TGI-Cool (0.72±0.03) did not differ statistically from TGI-Warm (0.73±0.01) in subjects without illusion. We found that the TGI and Cool stimuli were classified better than the TGI and Warm stimuli, implying that objective EEG features are consistent with subjective behavioral results. Further, we observed that most discriminative features between the TGI and the Cool or Warm conditions appeared in the parietal area for subjects who perceived the illusion. We postulate that the somato-sensory cortex may be activated when TGI is perceived to be hot pain.

Corrigendum: Review of public motor imagery and execution datasets in brain-computer interfaces

[This corrects the article DOI: 10.3389/fnhum.2023.1134869.].

Can vibrotactile stimulation and tDCS help inefficient BCI users?

Brain-computer interface (BCI) has helped people by allowing them to control a computer or machine through brain activity without actual body movement. Despite this advantage, BCI cannot be used widely because some people cannot achieve controllable performance. To solve this problem, researchers have proposed stimulation methods to modulate relevant brain activity to improve BCI performance. However, multiple studies have reported mixed results following stimulation, and the comparative study of different stimulation modalities has been overlooked. Accordingly, this study was designed to compare vibrotactile stimulation and transcranial direct current stimulation's (tDCS) effects on brain activity modulation and motor imagery BCI performance among inefficient BCI users. We recruited 44 subjects and divided them into sham, vibrotactile stimulation, and tDCS groups, and low performers were selected from each stimulation group. We found that the latter's BCI performance in the vibrotactile stimulation group increased significantly by 9.13% (p < 0.01), and while the tDCS group subjects' performance increased by 5.13%, it was not significant. In contrast, sham group subjects showed no increased performance. In addition to BCI performance, pre-stimulus alpha band power and the phase locking values (PLVs) averaged over sensory motor areas showed significant increases in low performers following stimulation in the vibrotactile stimulation and tDCS groups, while sham stimulation group subjects and high performers showed no significant stimulation effects across all groups. Our findings suggest that stimulation effects may differ depending upon BCI efficiency, and inefficient BCI users have greater plasticity than efficient BCI users.

Review of public motor imagery and execution datasets in brain-computer interfaces

The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.

Are invasive cortical stimulations effective in brain atrophy?

Electrical brain stimulation is a treatment method for brain disorder patients. The majority of patients with a severe brain disorder have brain atrophy. However, it is not clearly understood if electrical brain stimulation is effective even to brain atrophy. In this work, we developed anatomical head models with varying degrees of brain atrophy, so that we could investigate the effects of subdural/epidural cortical stimulations. The correlation between brain atrophy and cortical stimulation was quantified by calculating the effective volume that cortical stimulation influenced in this brain atrophy simulation study. The results showed that the effective volumes in both cortical stimulations decreased significantly with brain atrophy. There was also a strong correlation (0.9989) between the cerebrospinal fluid (CSF) and brain atrophy. The increase in CSF volume following brain atrophy reinforced the shunting effect between the brain and CSF and appeared to be the cause of a decrease in the stimulation effect on the brain. Overall, the epidural cortical stimulation was more sensitive (up to 57%) to the severity of the brain atrophy than the subdural cortical stimulation.

EEG Dataset for RSVP and P300 Speller Brain-Computer Interfaces

As attention to deep learning techniques has grown, many researchers have attempted to develop ready-to-go brain-computer interfaces (BCIs) that include automatic processing pipelines. However, to do so, a large and clear dataset is essential to increase the model's reliability and performance. Accordingly, our electroencephalogram (EEG) dataset for rapid serial visual representation (RSVP) and P300 speller may contribute to increasing such BCI research. We validated our dataset with respect to features and accuracy. For the RSVP, the participants (N = 50) achieved about 92% mean target detection accuracy. At the feature level, we observed notable ERPs (at 315 ms in the RSVP; at 262 ms in the P300 speller) during target events compared to non-target events. Regarding P300 speller performance, the participants (N = 55) achieved about 92% mean accuracy. In addition, P300 speller performance over trial repetitions up to 15 was explored. The presented dataset could potentially improve P300 speller applications. Further, it may be used to evaluate feature extraction and classification algorithm effectively, such as for cross-subjects/cross-datasets, and even for the cross-paradigm BCI model.

Spindle-targeted acoustic stimulation may stabilize an ongoing nap

There have been numerous attempts over the decades to introduce closed-loop feedback to induce sleep oscillations. Recently, our group also introduced closed-loop acoustic feedback to the sleep spindle and reported improved procedural memory consolidation during a nap with spindle-targeted pink noise stimulation. In this study, we replicated our previous work with a control condition in an attempt to investigate the effect of closed-loop feedback on procedural memory. The results demonstrated a significant improvement in the subjects' procedural learning and reduced wake time during the nap with closed-loop acoustic stimulation compared with the control condition. Further, we found that randomized acoustic stimuli lead to more frequent spindle activity and a faster decrement in slow oscillation power compared with the sham condition. There were strong correlations between slow oscillation and measures related to sleep efficiency as well. Interestingly, we found a marginal enhancement in procedural learning during the nap with the closed-loop acoustic stimulation compared with the sham nap. We also found a marginal decrement in theta power during the nap with closed-loop feedback compared with the sham nap, and a negative correlation between slow oscillation and theta power. We speculate that the marginal improvement in procedural learning may be related to closed-loop acoustic feedback's stabilization of non-rapid eye movement sleep. Taken together, this study shows that the closed-loop feedback method has the potential to stabilize sleep and improve procedural memory.

Is electric field strength deterministic in cortical neurons' response to transcranial electrical stimulation?

Transcranial electrical stimulation (tES), which modulates cortical excitability via electric currents, has attracted increasing attention because of its application in treating neurologic and psychiatric disorders. To obtain a better understanding of the brain areas affected and stimulation's cellular effects, a multi-scale model was proposed that combines multi-compartmental neuronal models and a head model. While one multi-scale model of tES that used straight axons reported that the direction of electric field (EF) is a determining factor in a neuronal response, another model of transcranial magnetic stimulation (TMS) that used arborized axons reported that EF magnitude is more crucial than EF direction because of arborized axons' reduced sensitivity to the latter. Our goal was to investigate whether EF magnitude remains a crucial factor in the neuronal response in a multi-scale model of tES into which an arborized axon is integrated. To achieve this goal, we constructed a multi-scale model that integrated three types of neurons and a realistic head model, and then simulated the neuronal response to realistic EF. We found that EF magnitude was correlated with excitation threshold, and thus, it may be one of the determining factors in cortical neurons' response to tES.Clinical Relevance-This multi-scale model based on biophysical and morphological properties and realistic brain geometry may help elucidate tES's neural mechanisms. Moreover, given its clinical applications, this model may help predict a patient's neuronal response to brain stimulation effectively.

Selective Subject Pooling Strategy to Improve Model Generalization for a Motor Imagery BCI

Brain–computer interfaces (BCIs) facilitate communication for people who cannot move their own body. A BCI system requires a lengthy calibration phase to produce a reasonable classifier. To reduce the duration of the calibration phase, it is natural to attempt to create a subject-independent classifier with all subject datasets that are available; however, electroencephalogram (EEG) data have notable inter-subject variability. Thus, it is very challenging to achieve subject-independent BCI performance comparable to subject-specific BCI performance. In this study, we investigate the potential for achieving better subject-independent motor imagery BCI performance by conducting comparative performance tests with several selective subject pooling strategies (i.e., choosing subjects who yield reasonable performance selectively and using them for training) rather than using all subjects available. We observed that the selective subject pooling strategy worked reasonably well with public MI BCI datasets. Finally, based upon the findings, criteria to select subjects for subject-independent BCIs are proposed here.

Computational exploration of epidural cortical stimulation using a realistic head model

Motor cortex stimulation, either non-invasively or with implanted electrodes, has been applied worldwide as a treatment for intractable neuropathic pain syndromes. Although computer simulations of non-invasive brain stimulation have been investigated largely to optimize protocols and improve our understanding of underlying mechanisms using a realistic head model, computational studies of invasive cortical stimulation are rare and limited to very simplified cortical models. In this paper, we present an anatomically realistic head model for epidural cortical stimulation that includes the most sophisticated epidural electrodes with an insulating paddle. The head model predicted the stimulus-induced field strengths according to two different stimulation techniques, bipolar and monopolar stimulations. We found that the stimulus-induced field focused on the precentral and postcentral gyri because of the epidural lead's invasiveness. Different stimulation configurations influenced the shape of the field markedly, and complex patterns of inward and outward directions of the radial field were observed in bipolar stimulation compared to those in monopolar stimulation. The spatial distributions of field strength showed that the optimal stimulation varied according to the target areas. In conclusion, we proposed an anatomically realistic head model and a sophisticated epidural lead to simulate epidural cortical stimulation-induced field strengths and identified the importance of such detailed modeling for epidural cortical stimulation because of the current's shunting through the cerebrospinal fluid.

Morphological Influence and Electric Field Direction's Influence on Activation of Cortical Neurons in Electrical Brain Stimulation: a Computational Study

Electrical brain stimulation (EBS) has been actively researched because of its clinical application and usefulness in brain research. However, its effect on individual neurons remains uncertain, as each neuron's response to EBS is highly variable and dependent on its morphology and the axis in which a neuron lies. Hence, our goal was to investigate the way that neuronal morphology affects the cellular response to extracellular stimulation from multiple directions. In this computational study, we observed that the varying neuronal morphology and direction of applied electrical field (EF) had some influence on the excitation threshold, which generates an action potential. Further, change of the excitation threshold depending on EF directions was observed.Clinical Relevance- These findings would help us to understand the variability in the modulatory effects of EBS at the cellular level and would be the basis for understanding the packed fibers' responses to EBS. Ultimately, considering EBS' clinical application, it may also help to predict patient's results from EBS treatment.

CANet: A Channel Attention Network to Determine Informative Multi-channel for Image Classification from Brain Signals

Changes in brain state that depend on various visual image stimulations have been investigated recently; however, it is difficult to decode visual image information from brain signal information. Recently, deep learning techniques have been applied to classify brain signals in various experiments, such as motor imagery and steady state visual evoked potential. However, although the deep learning model seems powerful, it is understood poorly, and thus, can be considered a black box. Accordingly, when multi-channel brain signals are trained, which channels include important information is not understood clearly. In this paper, we proposed a channel attention network (CANet) and investigated the way the deep learning network may determine which channels contain more important information that represents brainwaves' characteristics and the way it may visualize that information. Using such spatial channel information, we found that our proposed deep learning architecture outperforms basic approaches (spatial channel information is not considered) to classifying categorized images from visual evoked magnetoencephalographic (MEG) brain signals.

Simultaneous EEG Acquisition System for Multiple Users: Development and Related Issues

Social interaction is one of humans' most important activities and many efforts have been made to understand the phenomenon. Recently, some investigators have attempted to apply advanced brain signal acquisition systems that allow dynamic brain activities to be measured simultaneously during social interactions. Most studies to date have investigated dyadic interactions, although multilateral interactions are more common in reality. However, it is believed that most studies have focused on such interactions because of methodological limitations, in that it is very difficult to design a well-controlled experiment for multiple users at a reasonable cost. Accordingly, there are few simultaneous acquisition systems for multiple users. In this study, we propose a design framework for an acquisition system that measures EEG data simultaneously in an environment with 10 or more people. Our proposed framework allowed us to acquire EEG data at up to 1 kHz frequency from up to 20 people simultaneously. Details of our acquisition system are described from hardware and software perspectives. In addition, various related issues that arose in the system's development-such as synchronization techniques, system loads, electrodes, and applications-are discussed. In addition, simultaneous visual ERP experiments were conducted with a group of nine people to validate the EEG acquisition framework proposed. We found that our framework worked reasonably well with respect to less than 4 ms delay and average loss rates of 1%. It is expected that this system can be used in various hyperscanning studies, such as those on crowd psychology, large-scale human interactions, and collaborative brain-computer interface, among others.

Stimulation Effect of Inter-subject Variability in tDCS-Multi-scale Modeling Study

Transcranial direct current stimulation (tDCS) is an emerging non-invasive neuromodulation method that is convenient and popular in clinical use. However, there is a practical issue in applying tDCS; it is difficult to optimize the montage for each individual because of inherent inter-subject variability. Thus, the stimulation effect of such individual anatomical head variation has been investigated using anatomically realistic models. In this work, we developed a multi-scale computational model, which combined head models based on magnetic resonance imaging (MRI) and multi-compartmental neuronal models of pyramidal neurons (PNs), to investigate both the macroscopic and microscopic effects oftDCS. We constructed three different head models and compared the stimulation effects of tDCS in the primary cortex area (Brodmann area 4) with respect to the electric fields induced and steady-state membrane polarizations. We observed that the electric field behavior and induced somatic polarizations varied across subjects in accordance with the thicknesses of cerebrospinal fluid (CSF) and skull measured in each model. Thus, we concluded that variations in the CSF and skull might be correlated with the effects of tDCS.

The Neurophysiological Effect of Acoustic Stimulation with Real-time Sleep Spindle Detection

Sleep spindle is a salient brain activity found in the sigma frequency range (11-16 Hz) during sleep stage 2. It has been demonstrated that sleep spindle is related to memory consolidation, neurodegenerative disease, and mental disorders. Slow wave activity (0.5-4 Hz) is the most prominent EEG activity during sleep and appears as a large, spontaneous synchronization of cortical neurons. The role of slow wave activity has been proposed to regulate synaptic strength and memory consolidation. Many studies have investigated the effect of acoustic stimuli during the sleep slow wave. However, there have been few studies which investigated an effect of acoustic stimulation during sleep spindle activity. In this study, we examined the neurophysiological effect of acoustic stimulation during sleep spindle activity. We delivered pink noise after the detection of sleep spindle, and surmised that acoustic stimulation after sleep spindle detection may preserve delta activity during ongoing sleep. Further, we observed suppression of the sleep spindle activity around the times of acoustic stimulation and evoked slow wave activity and theta band activity immediately after tone onset.

P300 Speller Performance Predictor Based on RSVP Multi-feature

Brain-computer interface (BCI) systems were developed so that people can control computers or machines through their brain activity without moving their limbs. The P300 speller is one of the BCI applications used most commonly, as is very simple and reliable and can achieve satisfactory performance. However, like other BCIs, the P300 speller still has room for improvements in terms of its practical use, for example, selecting the best compromise between spelling accuracy and information transfer rate (ITR; speed) so that the P300 speller can maintain high accuracy while increasing spelling speed. Therefore, seeking correlates of, and predicting, the P300 speller's performance is necessary to understand and improve the technique. In this work, we investigated the correlations between rapid serial visual presentation (RSVP) task features and the P300 speller's performance. Fifty-five subjects participated in the RSVP and conventional matrix P300 speller tasks and RSVP behavioral and electroencephalography (EEG) features were compared in the P300's speller performance. We found that several of the RSVP's event-related potential (ERP) and behavioral features were correlated with the P300 speller's offline binary classification accuracy. Using these features, we propose a simple multi-feature performance predictor (r = 0.53, p = 0.0001) that outperforms any single feature performance predictor, including that of the conventional RSVP T1% predictor (r = 0.28, p = 0.06). This result demonstrates that selective multi-features can predict BCI performance better than a single feature alone.

EEG datasets for motor imagery brain-computer interface

Background:

Most investigators of brain–computer interface (BCI) research believe that BCI can be achieved through induced neuronal activity from the cortex, but not by evoked neuronal activity. Motor imagery (MI)–based BCI is one of the standard concepts of BCI, in that the user can generate induced activity by imagining motor movements. However, variations in performance over sessions and subjects are too severe to overcome easily; therefore, a basic understanding and investigation of BCI performance variation is necessary to find critical evidence of performance variation. Here we present not only EEG datasets for MI BCI from 52 subjects, but also the results of a psychological and physiological questionnaire, EMG datasets, the locations of 3D EEG electrodes, and EEGs for non-task-related states.

Findings:

We validated our EEG datasets by using the percentage of bad trials, event-related desynchronization/synchronization (ERD/ERS) analysis, and classification analysis. After conventional rejection of bad trials, we showed contralateral ERD and ipsilateral ERS in the somatosensory area, which are well-known patterns of MI. Finally, we showed that 73.08% of datasets (38 subjects) included reasonably discriminative information.

Conclusions:

Our EEG datasets included the information necessary to determine statistical significance; they consisted of well-discriminated datasets (38 subjects) and less-discriminative datasets. These may provide researchers with opportunities to investigate human factors related to MI BCI performance variation, and may also achieve subject-to-subject transfer by using metadata, including a questionnaire, EEG coordinates, and EEGs for non-task-related states.

Interbrain phase synchronization during turn-taking verbal interaction-a hyperscanning study using simultaneous EEG/MEG

Recently, neurophysiological findings about social interaction have been investigated widely, and hardware has been developed that can measure multiple subjects' brain activities simultaneously. These hyperscanning studies have enabled us to discover new and important evidences of interbrain interactions. Yet, very little is known about verbal interaction without any visual input. Therefore, we conducted a new hyperscanning study based on verbal, interbrain turn-taking interaction using simultaneous EEG/MEG, which measures rapidly changing brain activities. To establish turn-taking verbal interactions between a pair of subjects, we set up two EEG/MEG systems (19 and 146 channels of EEG and MEG, respectively) located ∼100 miles apart. Subjects engaged in verbal communication via condenser microphones and magnetic-compatible earphones, and a network time protocol synchronized the two systems. Ten subjects participated in this experiment and performed verbal interaction and noninteraction tasks separately. We found significant oscillations in EEG alpha and MEG alpha/gamma bands in several brain regions for all subjects. Furthermore, we estimated phase synchronization between two brains using the weighted phase lag index and found statistically significant synchronization in EEG and MEG data. Our novel paradigm and neurophysiological findings may foster a basic understanding of the functional mechanisms involved in human social interactions. Hum Brain Mapp 39:171-188, 2018. © 2017 Wiley Periodicals, Inc.

Cognitive responses and cortical oscillatory processing at various stereoscopic depths - a simultaneous EEG/MEG study

Due to the recent explosion in various forms of 3D content, the evaluation of such content from a neuroscience perspective is quite interesting. However, existing investigations of cortical oscillatory responses in stereoscopic depth perception are quite rare. Therefore, we investigated spatiotemporal and spatio-temporo-spectral features at four different stereoscopic depths within the comfort zone. We adopted a simultaneous EEG/MEG acquisition technique to collect the oscillatory responses of eight participants. We defined subject-specific retinal disparities and designed a single trial-based stereoscopic viewing experimental paradigm. In the group analysis, we observed that, as the depth increased from Level 1 to Level 3, there was a time-locked increase in the N200 component in MEG and the P300 component in EEG in the occipital and parietal areas, respectively. In addition, initial alpha and beta event-related desynchronizations (ERD) were observed at approximately 500 to 1000 msec, while theta, alpha, and beta event-related synchronizations (ERS) appeared at approximately 1000 to 2000 ms. Interestingly, there was a saturation point in the increase in cognitive responses, including N200, P300, and alpha ERD, even when the depth increased only within the comfort zone. Meanwhile, the magnitude of low beta ERD decreased in the dorsal pathway as depth increased. From these findings, we concluded that cognitive responses are likely to become saturated in the visual comfort zone, while perceptual load may increase with depth.

A wellness platform for stereoscopic 3D video systems using EEG-based visual discomfort evaluation technology

Recent advances in three-dimensional (3D) video technology have extended the range of our experience while providing various 3D applications to our everyday life. Nevertheless, the so-called visual discomfort (VD) problem inevitably degrades the quality of experience in stereoscopic 3D (S3D) displays. Meanwhile, electroencephalography (EEG) has been regarded as one of the most promising brain imaging modalities in the field of cognitive neuroscience. In an effort to facilitate comfort with S3D displays, we propose a new wellness platform using EEG. We first reveal features in EEG signals that are applicable to practical S3D video systems as an index for VD perception. We then develop a framework that can automatically determine severe perception of VD based on the EEG features during S3D video viewing by capitalizing on machine-learning-based braincomputer interface technology. The proposed platform can cooperate with advanced S3D video systems whose stereo baseline is adjustable. Thus, the optimal S3D content can be reconstructed according to a viewer's sensation of VD. Applications of the proposed platform to various S3D industries are suggested, and further technical challenges are discussed for follow-up research.

A computational study on effect of a transcranial channel as a skull/brain interface in the conventional rectangular patch-type transcranial direct current stimulation

A conceptual model of a transcranial channel was recently proposed for conveying external currents of the skull to the brain; the effects of the transcranial channel combined with high-definition transcranial direct current stimulation (HD-tDCS) was investigated, resulting in discovery of increased stimulus intensity and focality. In this work, rather than HD-tDCS using smaller disc-type electrodes, we proposed the use of the transcranial channel with conventional tDCS using larger patch electrodes. We used multi-scale computational models that couple an anatomically realistic head model with multi-compartmental models of cortical neurons. We then predicted the excitability in the hand knob (target area) based on stimulus-induced electric fields and steady-state membrane polarizations. Conventional tDCS without the transcranial channel resulted in diffuse distributions of electric fields that covered the frontal cortex, while the spatial focality and intensity of the excitability increased significantly at the target area in the presence of the transcranial channel. Thus, it is expected that conventional tDCS with the transcranial channel allows a better targeting neuromodulation with higher intensity and may be promising for applying prolonged and stabilized tDCS.

The Effect of a Transcranial Channel as a Skull/Brain Interface in High-Definition Transcranial Direct Current Stimulation-A Computational Study

A transcranial channel is an interface between the skull and brain; it consists of a biocompatible and highly conductive material that helps convey the current induced by transcranial direct current stimulation (tDCS) to the target area. However, it has been proposed only conceptually, and there has been no concrete study of its efficacy. In this work, we conducted a computational investigation of this conceptual transcranial model with high-definition tDCS, inducing focalized neuromodulation to determine whether inclusion of a transcranial channel performs effectively. To do so, we constructed an anatomically realistic head model and compartmental pyramidal neuronal models. We analyzed membrane polarization by extracellular stimulation and found that the inclusion of a transcranial channel induced polarization at the target area 11 times greater than conventional HD-tDCS without the transcranial channel. Furthermore, the stimulation effect of the transcranial channel persisted up to approximately 80%, even when the stimulus electrodes were displaced approximately 5 mm from the target area. We investigated the efficacy of the transcranial channel and found that greatly improved stimulation intensity and focality may be achieved. Thus, the use of these channels may be promising for clinical treatment.

Cortical Responses and Shape Complexity of Stereoscopic Image - A Simultaneous EEG/MEG Study

BACKGROUND/AIMS

In exploring human factors, stereoscopic 3D images have been used to investigate the neural responses associated with excessive depth, texture complexity, and other factors. However, the cortical oscillation associated with the complexity of stereoscopic images has been studied rarely. Here, we demonstrated that the oscillatory responses to three differently shaped 3D images (circle, star, and bat) increase as the complexity of the image increases.

METHODS

We recorded simultaneous EEG/MEG for three different stimuli. Spatio-temporal and spatio-spectro-temporal features were investigated by non-parametric permutation test.

RESULTS

The results showed that N300 and alpha inhibition increased in the ventral area as the shape complexity of the stereoscopic image increased.

CONCLUSION

It seems that the relative disparity in complex stereoscopic images may increase cognitive processing (N300) and cortical load (alpha inhibition) in the ventral area.

A multi-scale computational model of the effects of TMS on motor cortex

The detailed biophysical mechanisms through which transcranial magnetic stimulation (TMS) activates cortical circuits are still not fully understood. Here we present a multi-scale computational model to describe and explain the activation of different pyramidal cell types in motor cortex due to TMS. Our model determines precise electric fields based on an individual head model derived from magnetic resonance imaging and calculates how these electric fields activate morphologically detailed models of different neuron types. We predict neural activation patterns for different coil orientations consistent with experimental findings. Beyond this, our model allows us to calculate activation thresholds for individual neurons and precise initiation sites of individual action potentials on the neurons’ complex morphologies. Specifically, our model predicts that cortical layer 3 pyramidal neurons are generally easier to stimulate than layer 5 pyramidal neurons, thereby explaining the lower stimulation thresholds observed for I-waves compared to D-waves. It also shows differences in the regions of activated cortical layer 5 and layer 3 pyramidal cells depending on coil orientation. Finally, it predicts that under standard stimulation conditions, action potentials are mostly generated at the axon initial segment of cortical pyramidal cells, with a much less important activation site being the part of a layer 5 pyramidal cell axon where it crosses the boundary between grey matter and white matter. In conclusion, our computational model offers a detailed account of the mechanisms through which TMS activates different cortical pyramidal cell types, paving the way for more targeted application of TMS based on individual brain morphology in clinical and basic research settings.

Effects of electrode displacement in high-definition transcranial direct current stimulation: A computational study

In order to understand better the ways in which cortical excitability is linked to target brain areas, this study describes the effects of focalized high-definition transcranial direct current stimulation (HD-tDCS), and investigates the way in which these effects persisted after the stimulus electrodes were displaced from the target area. We constructed a 3D volume conduction model of an anatomically realistic head that is ideal for HD-tDCS, as well as compartmental models of layer 3 and layer 5 pyramidal neurons. Using extracellular approaches, we observed stimulus-induced electric fields and simulated neuronal responses by combining stimulus-induced potential fields with pyramidal neuronal models coupled with the head model. We found that the stimulus-induced electric fields were focused on the hand-knob when the electrodes were placed directly above the target region; further, the neuronal responses varied, such that the upper parts of the dendrites were hyperpolarized, while the soma and axons were depolarized. The magnitude of the electric fields, as well as the maximum polarizations at each compartment decreased according to the displacement of the electrodes from the target area. Considerable excitability at the target area within the range of 5 mm displacement between electrodes and the target area was shown by means of stimulus-induced electric fields and membrane polarization. In conclusion, using detailed computational approaches, we discovered the ways in which excitability in the target area persisted even with increased distance from the active electrode.

Steady-State Somatosensory Evoked Potential for Brain-Computer Interface-Present and Future

Brain-computer interface (BCI) performance has achieved continued improvement over recent decades, and sensorimotor rhythm-based BCIs that use motor function have been popular subjects of investigation. However, it remains problematic to introduce them to the public market because of their low reliability. As an alternative resolution to this issue, visual-based BCIs that use P300 or steady-state visually evoked potentials (SSVEPs) seem promising; however, the inherent visual fatigue that occurs with these BCIs may be unavoidable. For these reasons, steady-state somatosensory evoked potential (SSSEP) BCIs, which are based on tactile selective attention, have gained increasing attention recently. These may reduce the fatigue induced by visual attention and overcome the low reliability of motor activity. In this literature survey, recent findings on SSSEP and its methodological uses in BCI are reviewed. Further, existing limitations of SSSEP BCI and potential future directions for the technique are discussed.

Increasing session-to-session transfer in a brain-computer interface with on-site background noise acquisition

OBJECTIVE

A brain-computer interface (BCI) usually requires a time-consuming training phase during which data are collected and used to generate a classifier. Because brain signals vary dynamically over time (and even over sessions), this training phase may be necessary each time the BCI system is used, which is impractical. However, the variability in background noise, which is less dependent on a control signal, may dominate the dynamics of brain signals. Therefore, we hypothesized that an understanding of variations in background noise may allow existing data to be reused by incorporating the noise characteristics into the feature extraction framework; in this way, new session data are not required each time and this increases the feasibility of the BCI systems.

APPROACH

In this work, we collected background noise during a single, brief on-site acquisition session (approximately 3 min) immediately before a new session, and we found that variations in background noise were predictable to some extent. Then we implemented this simple session-to-session transfer strategy with a regularized spatiotemporal filter (RSTF), and we tested it with a total of 20 cross-session datasets collected over multiple days from 12 subjects. We also proposed and tested a bias correction (BC) in the RSTF.

MAIN RESULTS

We found that our proposed session-to-session strategies yielded a slightly less or comparable performance to the conventional paradigm (each session training phase is needed with an on-site training dataset). Furthermore, using an RSTF only and an RSTF with a BC outperformed existing approaches in session-to-session transfers.

SIGNIFICANCE

We inferred from our results that, with an on-site background noise suppression feature extractor and pre-existing training data, further training time may be unnecessary.

Validation of Computational Studies for Electrical Brain Stimulation With Phantom Head Experiments

BACKGROUND

Although computational studies of electrical brain stimulation (EBS) have received attention as a cost-effective tool, few studies have validated the technique, particularly in invasive cortical stimulation.

OBJECTIVE

In order to validate such studies, we used EBS to compare electric potential distributions generated by both numerical simulations and empirical measurements in three phantom head models (one-/three-layered spherical heads and MRI-based head).

METHODS

We constructed spherical phantom heads that consisted of one or three layers, and an anatomical, MRI-based phantom that consisted of three layers and represented the brain or brain/skull/scalp in order to perform both numerical simulations using the finite element method (FEM) and experimental measurements. Two stimulation electrodes (cathode and anode) were implanted in the phantoms to inject regulated input voltage, and the electric potential distributions induced were measured at various points located either on the surface or deep within the phantoms.

RESULTS

We observed that both the electric potential distributions from the numerical simulations and experiments behaved similarly and resulted in average relative differences of 5.4% (spherical phantom) and 10.3% (MRI-based phantom).

CONCLUSIONS

This study demonstrated that numerical simulation is reasonably consistent with actual experimental measurements; thus, because of its cost-effectiveness, EBS computational studies may be an attractive approach for necessary intensive/extensive studies.

Computational Study of Subdural Cortical Stimulation: Effects of Simulating Anisotropic Conductivity on Activation of Cortical Neurons

Subdural cortical stimulation (SuCS) is an appealing method in the treatment of neurological disorders, and computational modeling studies of SuCS have been applied to determine the optimal design for electrotherapy. To achieve a better understanding of computational modeling on the stimulation effects of SuCS, the influence of anisotropic white matter conductivity on the activation of cortical neurons was investigated in a realistic head model. In this paper, we constructed pyramidal neuronal models (layers 3 and 5) that showed primary excitation of the corticospinal tract, and an anatomically realistic head model reflecting complex brain geometry. The anisotropic information was acquired from diffusion tensor magnetic resonance imaging (DT-MRI) and then applied to the white matter at various ratios of anisotropic conductivity. First, we compared the isotropic and anisotropic models; compared to the isotropic model, the anisotropic model showed that neurons were activated in the deeper bank during cathodal stimulation and in the wider crown during anodal stimulation. Second, several popular anisotropic principles were adapted to investigate the effects of variations in anisotropic information. We observed that excitation thresholds varied with anisotropic principles, especially with anodal stimulation. Overall, incorporating anisotropic conductivity into the anatomically realistic head model is critical for accurate estimation of neuronal responses; however, caution should be used in the selection of anisotropic information.

Achieving a hybrid brain-computer interface with tactile selective attention and motor imagery

OBJECTIVE

We propose a new hybrid brain-computer interface (BCI) system that integrates two different EEG tasks: tactile selective attention (TSA) using a vibro-tactile stimulator on the left/right finger and motor imagery (MI) of left/right hand movement. Event-related desynchronization (ERD) from the MI task and steady-state somatosensory evoked potential (SSSEP) from the TSA task are retrieved and combined into two hybrid senses.

APPROACH

One hybrid approach is to measure two tasks simultaneously; the features of each task are combined for testing. Another hybrid approach is to measure two tasks consecutively (TSA first and MI next) using only MI features. For comparison with the hybrid approaches, the TSA and MI tasks are measured independently.

MAIN RESULTS

Using a total of 16 subject datasets, we analyzed the BCI classification performance for MI, TSA and two hybrid approaches in a comparative manner; we found that the consecutive hybrid approach outperformed the others, yielding about a 10% improvement in classification accuracy relative to MI alone. It is understood that TSA may play a crucial role as a prestimulus in that it helps to generate earlier ERD prior to MI and thus sustains ERD longer and to a stronger degree; this ERD may give more discriminative information than ERD in MI alone.

SIGNIFICANCE

Overall, our proposed consecutive hybrid approach is very promising for the development of advanced BCI systems.

Computational study on subdural cortical stimulation - the influence of the head geometry, anisotropic conductivity, and electrode configuration

Subdural cortical stimulation (SuCS) is a method used to inject electrical current through electrodes beneath the dura mater, and is known to be useful in treating brain disorders. However, precisely how SuCS must be applied to yield the most effective results has rarely been investigated. For this purpose, we developed a three-dimensional computational model that represents an anatomically realistic brain model including an upper chest. With this computational model, we investigated the influence of stimulation amplitudes, electrode configurations (single or paddle-array), and white matter conductivities (isotropy or anisotropy). Further, the effects of stimulation were compared with two other computational models, including an anatomically realistic brain-only model and the simplified extruded slab model representing the precentral gyrus area. The results of voltage stimulation suggested that there was a synergistic effect with the paddle-array due to the use of multiple electrodes; however, a single electrode was more efficient with current stimulation. The conventional model (simplified extruded slab) far overestimated the effects of stimulation with both voltage and current by comparison to our proposed realistic upper body model. However, the realistic upper body and full brain-only models demonstrated similar stimulation effects. In our investigation of the influence of anisotropic conductivity, model with a fixed ratio (1∶10) anisotropic conductivity yielded deeper penetration depths and larger extents of stimulation than others. However, isotropic and anisotropic models with fixed ratios (1∶2, 1∶5) yielded similar stimulation effects. Lastly, whether the reference electrode was located on the right or left chest had no substantial effects on stimulation.

A review of brain-computer interface games and an opinion survey from researchers, developers and users

In recent years, research on Brain-Computer Interface (BCI) technology for healthy users has attracted considerable interest, and BCI games are especially popular. This study reviews the current status of, and describes future directions, in the field of BCI games. To this end, we conducted a literature search and found that BCI control paradigms using electroencephalographic signals (motor imagery, P300, steady state visual evoked potential and passive approach reading mental state) have been the primary focus of research. We also conducted a survey of nearly three hundred participants that included researchers, game developers and users around the world. From this survey, we found that all three groups (researchers, developers and users) agreed on the significant influence and applicability of BCI and BCI games, and they all selected prostheses, rehabilitation and games as the most promising BCI applications. User and developer groups tended to give low priority to passive BCI and the whole head sensor array. Developers gave higher priorities to “the easiness of playing” and the “development platform” as important elements for BCI games and the market. Based on our assessment, we discuss the critical point at which BCI games will be able to progress from their current stage to widespread marketing to consumers. In conclusion, we propose three critical elements important for expansion of the BCI game market: standards, gameplay and appropriate integration.

High theta and low alpha powers may be indicative of BCI-illiteracy in motor imagery

In most brain computer interface (BCI) systems, some target users have significant difficulty in using BCI systems. Such target users are called ‘BCI-illiterate’. This phenomenon has been poorly investigated, and a clear understanding of the BCI-illiteracy mechanism or a solution to this problem has not been reported to date. In this study, we sought to demonstrate the neurophysiological differences between two groups (literate, illiterate) with a total of 52 subjects. We investigated recordings under non-task related state (NTS) which is collected during subject is relaxed with eyes open. We found that high theta and low alpha waves were noticeable in the BCI-illiterate relative to the BCI-literate people. Furthermore, these high theta and low alpha wave patterns were preserved across different mental states, such as NTS, resting before motor imagery (MI), and MI states, even though the spatial distribution of both BCI-illiterate and BCI-literate groups did not differ. From these findings, an effective strategy for pre-screening subjects for BCI illiteracy has been determined, and a performance factor that reflects potential user performance has been proposed using a simple combination of band powers. Our proposed performance factor gave an r = 0.59 (r² = 0.34) in a correlation analysis with BCI performance and yielded as much as r = 0.70 (r² = 0.50) when seven outliers were rejected during the evaluation of whole data (N = 61), including BCI competition datasets (N = 9). These findings may be directly applicable to online BCI systems.

Comparison of neuronal excitation between extruded slab partial head model and full head model in subdural cortical stimulation

Cortical stimulation (CS) is an appealing and emerging treatment for neurological disorders. CS is known to promote functional recovery effectively; however, its underlying mechanism and the optimal parameters for the effective treatment are not clearly understood. In this work, we developed a realistic three-dimensional full head and chest model for subdural CS. Our proposed model was compared at the neuron level with an existing simplified extruded slab partial head model depicting around precentral gyral cortex only. Each model was coupled with the pyramidal neuronal model in order to investigate an extent of neuronal excitation. We found that the crown of the cortex was the most excitable area in the unipolar stimulation, while in the bipolar stimulation, the lip and bank were excited more easily than other areas. Finally, it was evident that our proposed model was substantially different in excitation threshold from the existing simplified model, which is compelling to do computational CS study on more realistic head models.