Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG

EEG response to game-craving according to personal preference for games

Recently, the World Health Organization included ‘gaming disorder’ in its latest revision of the international classification of diseases (ICD-11). Despite extensive research on internet gaming disorder (IGD), few studies have addressed game-related stimuli eliciting craving, which plays an important role in addiction. Particularly, most previous studies did not consider personal preferences in games presented to subjects as stimuli. In this study, we compared neurophysiological responses elicited for favorite game (FG) videos and non-favorite game (NFG) videos. We aimed to demonstrate neurophysiological characteristics according to the game preference in the IGD group. We measured participants’ electroencephalogram (EEG) while they watched FG, NFG and neutral videos. For FG videos, the parieto-occipital theta power (TP_PO) were significantly increased compared with those for NFG videos (P < 0.05, paired t-test). TP_PO also differed significantly between the healthy control and IGD groups only on FG videos controlling covariate (TP_PO on neutral videos) (P < 0.05, analysis of covariance [ANCOVA]). And TP_PO was significantly correlated to self-reported craving score only on FG videos (r = 0.334, P < 0.05). In the present study, we demonstrate that FG videos induce higher TP_PO than that induced by NFG videos in the IGD group and TP_PO is a reliable EEG feature associated with craving for gaming.

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

Fronto-lateral alpha power asymmetry in panic disorder

Resting state alpha power asymmetry in frontal and temporal regions has been reported in various clinical populations, possibly indicating deficits in prefrontal control. In panic disorder (PD), results regarding alpha asymmetric activity to date have been mixed. This study compared 55 PD patients and 42 healthy controls (HC) with regards to resting state alpha power asymmetry. Our results show more right-than-left fronto-lateral alpha power in PD, whereas at other sites and in HC no significant differences were detected. These results support the notion of altered neurobiological processes in PD that possibly represent a vulnerability to the experience of panic attacks. Further studies are needed to clarify potential causal implications of this finding in the genesis of PD, as well as to specify the functional significance of fronto-lateral alpha power asymmetry in PD.

Causal role of cross-frequency coupling in distinct components of cognitive control

Cognitive control is the capacity to guide motor and perceptual systems towards abstract goals. High-frequency neural oscillations related to motor activity in the beta band (13-30 Hz) and to visual processing in the gamma band (>30 Hz) are known to be modulated by cognitive control signals. One proposed mechanism for cognitive control is via cross-frequency coupling whereby low frequency network oscillations in prefrontal cortex (delta from 2-3 Hz and theta from 4-8 Hz) guide the expression of motor-related activity in action planning and guide perception-related activity in memory access. However, there is no causal evidence for cross-frequency coupling in these dissociable components of cognitive control. To address this important gap in knowledge, we delivered cross-frequency transcranial alternating current stimulation (CF-tACS) during performance of a task that manipulated cognitive control demands along two dimensions: the abstraction of the rules of the task (nested levels of action selection) that increased delta-beta coupling and the number of rules (set-size held in memory) that increased theta-gamma coupling. As hypothesized, we found that CF-tACS increased the targeted phase-amplitude coupling and modulated task performance of the associated cognitive control component. These findings provide causal evidence that prefrontal cortex orchestrates different components of cognitive control via two different cross-frequency coupling modalities.

Closed-Loop Neurofeedback of α Synchrony during Goal-Directed Attention

α Oscillations in sensory cortex, under frontal control, desynchronize during attentive preparation. Here, in a selective attention study with simultaneous EEG in humans of either sex, we first demonstrate that diminished anticipatory α synchrony between the mid-frontal region of the dorsal attention network and ventral visual sensory cortex [frontal-sensory synchrony (FSS)] significantly correlates with greater task performance. Then, in a double-blind, randomized controlled study in healthy adults, we implement closed-loop neurofeedback (NF) of the anticipatory α FSS signal over 10 d of training. We refer to this closed-loop experimental approach of rapid NF integrated within a cognitive task as cognitive NF (cNF). We show that cNF results in significant trial-by-trial modulation of the anticipatory α FSS measure during training, concomitant plasticity of stimulus-evoked α/θ responses, as well as transfer of benefits to response time (RT) improvements on a standard test of sustained attention. In a third study, we implement cNF training in children with attention deficit hyperactivity disorder (ADHD), replicating trial-by-trial modulation of the anticipatory α FSS signal as well as significant improvement of sustained attention RTs. These first findings demonstrate the basic mechanisms and translational utility of rapid cognitive-task-integrated NF.SIGNIFICANCE STATEMENT When humans prepare to attend to incoming sensory information, neural oscillations in the α band (8-14 Hz) undergo desynchronization under the control of prefrontal cortex. Here, in an attention study with electroencephalography, we first show that frontal-sensory synchrony (FSS) of α oscillations during attentive preparation significantly correlates with task performance. Then, in a randomized controlled study in healthy adults, we show that neurofeedback (NF) training of this α FSS signal within the attention task is feasible. We show that this rapid cognitive NF (cNF) approach engenders plasticity of stimulus-evoked neural responses, and improves performance on a standard test of sustained attention. In a final study, we implement cNF in children with attention deficit hyperactivity disorder (ADHD), replicating the improvement of sustained attention found in adults.

RT-NET: real-time reconstruction of neural activity using high-density electroencephalography

High-density electroencephalography (hdEEG) has been successfully used for large-scale investigations of neural activity in the healthy and diseased human brain. Because of their high computational demand, analyses of source-projected hdEEG data are typically performed offline. Here, we present a real-time noninvasive electrophysiology toolbox, RT-NET, which has been specifically developed for online reconstruction of neural activity using hdEEG. RT-NET relies on the Lab Streaming Layer for acquiring raw data from a large number of EEG amplifiers and for streaming the processed data to external applications. RT-NET estimates a spatial filter for artifact removal and source activity reconstruction using a calibration dataset. This spatial filter is then applied to the hdEEG data as they are acquired, thereby ensuring low latencies and computation times. Overall, our analyses show that RT-NET can estimate real-time neural activity with performance comparable to offline analysis methods. It may therefore enable the development of novel brain–computer interface applications such as source-based neurofeedback.

Neuroergonomic Stress Assessment with Two Different Methodologies, in a Manual Repetitive Task-Product Assembly

Emotions are a fundamental part of mental health and human behavior. In the workplace, optimal performance of employees is necessary for productivity enhancements and its relation to the quality of a manufacturing product, therefore leading a company to advantages and competitiveness. This means that the workplace staff must remain in a neutral or a calm emotional state, for an adequate job performance. When an operation is not pleasant or the same task is carried out for a long period of time (repetitive), it can cause negative emotions such as stress, and this will have repercussions in poor work performance. The purpose of this research is, by means of an electroencephalogram (EEG), to identify the stress in the repetitive assembly of a manufacturing product. To measure brain waves, the Emotiv Epoc equipment was used and a manufacturing line was designed, divided into three workstations, where the assembly of product comprising a LEGO car was carried out within a manual repetitive approach. The appearance of stress was determined by employing two different methodologies, the prefrontal relative gamma marker (RG) and the valence, arousal, and dominance (VAD) emotional categories. The results obtained from the first methodology, corresponding to the RG marker, displayed a significant more change between the relaxation state and the product assembly carried out at 70% of the standard time (ST). A less significant change was observed between the relaxation state and the product assembly carried out at 100% ST, thus signaling the presence of stress. Additionally, the results from the VAD methodology resulted in moderate and low levels of stress, when the product assembly was carried out at 70% and 100% standard time, respectively.

Improved cortical activity and reduced gait asymmetry during poststroke self-paced walking rehabilitation

Background

For patients with gait impairment due to neurological disorders, body weight-supported treadmill training (BWSTT) has been widely used for gait rehabilitation. On a conventional (passive) treadmill that runs at a constant speed, however, the level of patient engagement and cortical activity decreased compared with gait training on the ground. To increase the level of cognitive engagement and brain activity during gait rehabilitation, a self-paced (active) treadmill is introduced to allow patients to actively control walking speed, as with overground walking.

Methods

To validate the effects of self-paced treadmill walking on cortical activities, this paper presents a clinical test with stroke survivors. We hypothesized that cortical activities on the affected side of the brain would also increase during active walking because patients have to match the target walking speed with the affected lower limbs. Thus, asymmetric gait patterns such as limping or hobbling might also decrease during active walking.

Results

Although the clinical test was conducted in a short period, the patients showed higher cognitive engagement, improved brain activities assessed by electroencephalography (EEG), and decreased gait asymmetry with the self-paced treadmill. As expected, increases in the spectral power of the low γ and β bands in the prefrontal cortex (PFC), premotor cortex (PMC), and supramarginal gyrus (SG) were found, which are possibly related to processing sensory data and planning voluntary movements. In addition, these changes in cortical activities were also found with the affected lower limbs during the swing phase. Since our treadmill controller tracked the swing speed of the leg to control walking speed, such results imply that subjects made substantial effort to control their affected legs in the swing phase to match the target walking speed.

Conclusions

The patients also showed reduced gait asymmetry patterns. Based on the results, the self-paced gait training system has the potential to train the symmetric gait and to promote the related cortical activities after stroke.

Trial registration Not applicable

Disinhibition of right inferior frontal gyrus underlies alpha asymmetry in women with low testosterone

Asymmetrical expression of alpha oscillations in the frontal cortex, increased left relative to right, is a phenotype associated with increased behavioral inhibition and mood-related psychiatric illnesses. However, investigations of frontal alpha asymmetry in mood-disorders have yielded inconsistent findings. A better understanding of factors that contribute to individual differences is required to establish a useful biomarker for the diagnosis and treatment of mood and stress related disorders. A novel factor is hormone concentration, as steroid hormones play a prominent role in regulating mood and stress. To investigate this question, concentrations of testosterone and estradiol were sampled. Multiple linear regression revealed that low levels of testosterone correlated with greater frontal alpha asymmetry in women. Source localization found that frontal asymmetry was driven by decreased alpha power in right inferior frontal gyrus that correlated with increased behavioral inhibition in women. Together, these findings might explain inconsistencies in previous investigation on frontal alpha asymmetry.

Psychophysiological responses to manual lifting of unknown loads

BACKGROUND

The handling of unknown weights, which is common in daily routines either at work or during leisure time, is suspected to be highly associated with the incidence of low back pain (LBP).

OBJECTIVES

To investigate the effects of knowledge and magnitude of a load (to be lifted) on brain responses, autonomic nervous activity, and trapezius and erector spinae muscle activity.

METHODS

A randomized, within-subjects experiment involving manual lifting was conducted, wherein 10 participants lifted three different weights (1.1, 5, and 15 kg) under two conditions: either having or not having prior knowledge of the weight to be lifted.

RESULTS

The results revealed that the lifting of unknown weights caused increased average heart rate and percentage of maximum voluntary contraction (%MVC) but decreased average inter-beat interval, very-low-frequency power, low-frequency power, and low-frequency/high-frequency ratio. Regardless of the weight magnitude, lifting of unknown weights was associated with smaller theta activities in the power spectrum density (PSD) of the central region, smaller alpha activities in the PSD of the frontal region, and smaller beta activities in the PSDs of both the frontal and central regions. Moreover, smaller alpha and beta activities in the PSD of the parietal region were associated only with lifting of unknown lightweights.

CONCLUSIONS

Uncertainty regarding the weight to be lifted could be considered as a stress-adding variable that may increase the required physical demand to be sustained during manual lifting tasks. The findings of this study stress the importance of eliminating uncertainty associated with handling unknown weights, such as in the cases of handling patients and dispatching luggage. This can be achieved through preliminary self-sensing of the load to be lifted, or the cautious disclosure of the actual weight of manually lifted objects, for example, through clear labeling and/or a coding system.

OPTICAL+: a frequency-based deep learning scheme for recognizing brain wave signals

A human-computer interaction (HCI) system can be used to detect different categories of the brain wave signals that can be beneficial for neurorehabilitation, seizure detection and sleep stage classification. Research on developing HCI systems using brain wave signals has progressed a lot over the years. However, real-time implementation, computational complexity and accuracy are still a concern. In this work, we address the problem of selecting the appropriate filtering frequency band while also achieving a good system performance by proposing a frequency-based approach using long short-term memory network (LSTM) for recognizing different brain wave signals. Adaptive filtering using genetic algorithm is incorporated for a hybrid system utilizing common spatial pattern and LSTM network. The proposed method (OPTICAL+) achieved an overall average classification error rate of 30.41% and a kappa coefficient value of 0.398, outperforming the state-of-the-art methods. The proposed OPTICAL+ predictor can be used to develop improved HCI systems that will aid in neurorehabilitation and may also be beneficial for sleep stage classification and seizure detection.

The influence of psychological and cognitive states on error-related negativity evoked during post-stroke rehabilitation movements

BACKGROUND

Recently, error-related negativity (ERN) signals are proposed to develop an assist-as-needed robotic stroke rehabilitation program. Stroke patients' state-of-mind, such as motivation to participate and active involvement in the rehabilitation program, affects their rate of recovery from motor disability. If the characteristics of the robotic stroke rehabilitation program can be altered based on the state-of-mind of the patients, such that the patients remain engaged in the program, the rate of recovery from their motor disability can be improved. However, before that, it is imperative to understand how the states-of-mind of a participant affect their ERN signal.

METHODS

This study aimed to determine the association between the ERN signal and the psychological and cognitive states of the participants. Experiments were conducted on stroke patients, which involved performing a physical rehabilitation exercise and a questionnaire to measure participants' subjective experience on four factors: motivation in participating in the experiment, perceived effort, perceived pressure, awareness of uncompleted exercise trials while performing the rehabilitation exercise. Statistical correlation analysis, EEG time-series and topographical analysis were used to assess the association between the ERN signals and the psychological and cognitive states of the participants.

RESULTS

A strong correlation between the amplitude of the ERN signal and the psychological and cognitive states of the participants was observed, which indicate the possibility of estimating the said states using the amplitudes of the novel ERN signal.

CONCLUSIONS

The findings pave the way for the development of an ERN based dynamically adaptive assist-as-needed robotic stroke rehabilitation program of which characteristics can be altered to keep the participants' motivation, effort, engagement in the rehabilitation program high. In future, the single-trial prediction ability of the novel ERN signals to predict the state-of-mind of stroke patients will be evaluated.

A Real-Time Movement Artifact Removal Method for Ambulatory Brain-Computer Interfaces

Recently, practical brain-computer interfaces (BCIs) have been widely investigated for detecting human intentions in real world. However, performance differences still exist between the laboratory and the real world environments. One of the main reasons for such differences comes from the user's unstable physical states (e.g., human movements are not strictly controlled), which produce unexpected signal artifacts. Hence, to minimize the performance degradation of electroencephalography (EEG)-based BCIs, we present a novel artifact removal method named constrained independent component analysis with online learning (cIOL). The cIOL can find and reject the noise-like components related to human body movements (i.e., movement artifacts) in the EEG signals. To obtain movement information, isolated electrodes are used to block electrical signals from the brain using high-resistance materials. We estimate artifacts with movement information using constrained independent component analysis from EEG signals and then extract artifact-free signals using online learning in each sample. In addition, the cIOL is evaluated by signal processing under 16 different experimental conditions (two types of EEG devices × two BCI paradigms × four different walking speeds). The experimental results show that the cIOL has the highest accuracy in both scalp- and ear-EEG, and has the highest signal-to-noise ratio in scalp-EEG among the state-of-the-art methods, except for the case of steady-state visual evoked potential at 2.0 m/s with superposition problem.

Pinging the brain with transcranial magnetic stimulation reveals cortical reactivity in time and space

BACKGROUND

Single-pulse transcranial magnetic stimulation (TMS) elicits an evoked electroencephalography (EEG) potential (TMS-evoked potential, TEP), which is interpreted as direct evidence of cortical reactivity to TMS. Thus, combining TMS with EEG can be used to investigate the mechanism underlying brain network engagement in TMS treatment paradigms. However, controversy remains regarding whether TEP is a genuine marker of TMS-induced cortical reactivity or if it is confounded by responses to peripheral somatosensory and auditory inputs. Resolving this controversy is of great significance for the field and will validate TMS as a tool to probe networks of interest in cognitive and clinical neuroscience.

OBJECTIVE

Here, we delineated the cortical origin of TEP by spatially and temporally localizing successive TEP components, and modulating them with transcranial direct current stimulation (tDCS) to investigate cortical reactivity elicited by single-pulse TMS and its causal relationship with cortical excitability.

METHODS

We recruited 18 healthy participants in a double-blind, cross-over, sham-controlled design. We collected motor-evoked potentials (MEPs) and TEPs elicited by suprathreshold single-pulse TMS targeting the left primary motor cortex (M1). To causally test cortical and corticospinal excitability, we applied tDCS to the left M1.

RESULTS

We found that the earliest TEP component (P25) was localized to the left M1. The following TEP components (N45 and P60) were largely localized to the primary somatosensory cortex, which may reflect afferent input by hand-muscle twitches. The later TEP components (N100, P180, and N280) were largely localized to the auditory cortex. As hypothesized, tDCS selectively modulated cortical and corticospinal excitability by modulating the pre-stimulus mu-rhythm oscillatory power.

CONCLUSION

Together, our findings provide causal evidence that the early TEP components reflect cortical reactivity to TMS.

Characterization of the Stages of Creative Writing With Mobile EEG Using Generalized Partial Directed Coherence

Two stages of the creative writing process were characterized through mobile scalp electroencephalography (EEG) in a 16-week creative writing workshop. Portable dry EEG systems (four channels: TP09, AF07, AF08, TP10) with synchronized head acceleration, video recordings, and journal entries, recorded mobile brain-body activity of Spanish heritage students. Each student's brain-body activity was recorded as they experienced spaces in Houston, Texas (“Preparation” stage), and while they worked on their creative texts (“Generation” stage). We used Generalized Partial Directed Coherence (gPDC) to compare the functional connectivity among both stages. There was a trend of higher gPDC in the Preparation stage from right temporo-parietal (TP10) to left anterior-frontal (AF07) brain scalp areas within 1–50 Hz, not reaching statistical significance. The opposite directionality was found for the Generation stage, with statistical significant differences (p < 0.05) restricted to the delta band (1–4 Hz). There was statistically higher gPDC observed for the inter-hemispheric connections AF07–AF08 in the delta and theta bands (1–8 Hz), and AF08 to TP09 in the alpha and beta (8–30 Hz) bands. The left anterior-frontal (AF07) recordings showed higher power localized to the gamma band (32–50 Hz) for the Generation stage. An ancillary analysis of Sample Entropy did not show significant difference. The information transfer from anterior-frontal to temporal-parietal areas of the scalp may reflect multisensory interpretation during the Preparation stage, while brain signals originating at temporal-parietal toward frontal locations during the Generation stage may reflect the final decision making process to translate the multisensory experience into a creative text.

What Can Neuromarketing Tell Us about Food Packaging?

Packaging is a powerful tool for brands, which can not only catch consumers' attention but also influence their purchase decisions. The application of neuromarketing techniques to the study of food packaging has recently gained considerable popularity both in academia and practice, but there are still some concerns about the methods and metrics commercially offered and the interpretation of their findings. This represents the motivation of this investigation, whose objective is twofold: (1) to analyze the methodologies and measurements commonly used in neuromarketing commercial research on packaging, and (2) to examine the extent to which the results of food packaging studies applying neuromarketing techniques can be reproduced under similar methodologies. Obtained results shed light on the application of neuromarketing techniques in the evaluation of food packaging and reveal that neuromarketing and declarative methodologies are complementary, and its combination may strengthen the studies' results. Additionally, this study highlights the importance of having a framework that improves the validity and reliability of neuromarketing studies to eradicate mistrust toward the discipline and provide brands with valuable insights into food packing design.

A Novel Method to Understand Neural Oscillations During Full-Body Reaching: A Combined EEG and 3D Virtual Reality Study

Virtual reality (VR) can be used to create environments that are not possible in the real-world. Producing movements in VR holds enormous promise for rehabilitation and offers a platform from which to understand the neural control of movement. However, no study has examined the impact of a 3D fully immersive head-mounted display (HMD) VR system on the integrity of neural data. We assessed the quality of 64-channel EEG data with and without HMD VR during rest and during a full-body reaching task. We compared resting EEG while subjects completed three conditions: No HMD (EEG-only), HMD powered off (VR-off), and HMD powered on (VR-on). Within the same session, EEG were collected while subjects completed full-body reaching movements in two conditions (EEG-only, VR-on). During rest, no significant differences in data quality and power spectrum were observed between EEG-only, VR-off, and VR-on conditions. During reaching movements, the proportion of components attributed to the brain was greater in the EEG-only condition compared to the VR-on condition. Despite this difference, neural oscillations in source space were not significantly different between conditions, with both conditions associated with decreases in alpha and beta power in sensorimotor cortex during movements. Our findings demonstrate that the integrity of EEG data can be maintained while individuals execute full-body reaching movements within an immersive 3D VR environment. Clinical impact: Integrating VR and EEG is a viable approach to understanding the cortical processes of movement. Simultaneously recording movement and brain activity in combination with VR provides the foundation for neurobiologically informed rehabilitation therapies.

Predicting Sex From EEG: Validity and Generalizability of Deep-Learning-Based Interpretable Classifier

Explainable artificial intelligence holds a great promise for neuroscience and plays an important role in the hypothesis generation process. We follow-up a recent machine learning-oriented study that constructed a deep convolutional neural network to automatically identify biological sex from EEG recordings in healthy individuals and highlighted the discriminative role of beta-band power. If generalizing, this finding would be relevant not only theoretically by pointing to some specific neurobiological sexual dimorphisms, but potentially also as a relevant confound in quantitative EEG diagnostic practice. To put this finding to test, we assess whether the automatic identification of biological sex generalizes to another dataset, particularly in the presence of a psychiatric disease, by testing the hypothesis of higher beta power in women compared to men on 134 patients suffering from Major Depressive Disorder. Moreover, we construct ROC curves and compare the performance of the classifiers in determining sex both before and after the antidepressant treatment. We replicate the observation of a significant difference in beta-band power between men and women, providing classification accuracy of nearly 77%. The difference was consistent across the majority of electrodes, however multivariate classification models did not generally improve the performance. Similar results were observed also after the antidepressant treatment (classification accuracy above 70%), further supporting the robustness of the initial finding.

Prior cortical activity differences during an action observation plus motor imagery task related to motor adaptation performance of a coordinated multi-limb complex task

Motor adaptation is the ability to develop new motor skills that makes performing a consolidated motor task under different psychophysical conditions possible. There exists a proven relationship between prior brain activity at rest and motor adaptation. However, the brain activity at rest is highly variable both between and within subjects. Here we hypothesize that the cortical activity during the original task to be later adapted is a more reliable and stronger determinant of motor adaptation. Consequently, we present a study to find cortical areas whose activity, both at rest and during first-person virtual reality simulation of bicycle riding, characterizes the subjects who did and did not adapt to ride a reverse steering bicycle, a complex motor adaptation task involving all limbs and balance. The results showed that cortical activity differences during the simulated task were higher, more significant, spatially larger, and spectrally wider than at rest for good performers. In this sense, the activity of the left anterior insula, left dorsolateral and ventrolateral inferior prefrontal areas, and left inferior premotor cortex (action understanding hub of the mirror neuron circuit) during simulated bicycle riding are the areas with the most descriptive power for the ability of adapting the motor task. Trials registration Trial was registered with the NIH Clinical Trials Registry (clinicaltrials.gov), with the registration number NCT02999516 (21/12/2016).

Years of Blindness Lead to "Visualize" Space Through Time

Spatial representation has been widely studied in early blindness, whereas research about late blindness is still limited. We recently demonstrated that the early (50-90 ms) event-related potential (ERP) response observed in sighted people during a spatial bisection task, is altered in early blind people and is influenced by the amount of time spent without vision in late blind individuals. Specifically, in late blind people a shorter period of blindness is associated with strong contralateral activation in occipital cortex and good performance during the spatial task-similar to that of sighted people. In contrast, non-lateralized occipital activation and lower performance characterize late blind individuals who have experienced a longer period of blindness-similar to that of early blind people. However, the same early occipital response activated in sighted individuals by spatial cues has been found to be activated by temporal cues in early blind individuals. Here, we investigate whether a similar temporal attraction can explain the neural and behavioral changes observed after many years of blindness in late blind people. An EEG recording was taken during a spatial bisection task where coherent and conflicting spatio-temporal information was presented. In participants with long blindness duration, the early recruitment of both visual and auditory areas is sensitive to temporal instead of spatial coordinates. These findings highlight some limits of neuroplasticity. Perceptual advantages from cross-sensory calibration during development seem to be subsequently lost following years of visual deprivation. This result has important implications for clinical outcomes following late blindness, highlighting the importance of timing in intervention and rehabilitation programs that activate compensatory strategies soon after sensory loss.

Motion and Muscle Artifact Removal Validation Using an Electrical Head Phantom, Robotic Motion Platform, and Dual Layer Mobile EEG

Motion and muscle artifacts can undermine signal quality in electroencephalography (EEG) recordings during locomotion. We evaluated approaches for recovering ground-truth artificial brain signals from noisy EEG recordings. We built an electrical head phantom that broadcast four brain and four muscle sources. Head movements were generated by a robotic motion platform. We recorded 128-channel dual layer EEG and 8-channel neck electromyography (EMG) from the head phantom during motion. We evaluated ground-truth electrocortical source signal recovery from artifact contaminated data using Independent Component Analysis (ICA) to determine: (1) the number of isolated noise sensor recordings needed to capture and remove motion artifacts, (2) the ability of Artifact Subspace Reconstruction to remove motion and muscle artifacts at contrasting artifact detection thresholds, (3) the number of neck EMG sensor recordings needed to capture and remove muscle artifacts, and (4) the ability of Canonical Correlation Analysis to remove muscle artifacts. We also evaluated source signal recovery by combining the best practices identified in aims 1-4. By including isolated noise and EMG recordings in the ICA decomposition, we more effectively recovered ground-truth artificial brain signals. A reduced subset of 32-noise and 6-EMG channels showed equivalent performance compared to including the complete arrays. Artifact Subspace Reconstruction improved source separation, but this was contingent on muscle activity amplitude. Canonical Correlation Analysis also improved source separation. Merging noise and EMG recordings into the ICA decomposition, with Artifact Subspace Reconstruction and Canonical Correlation Analysis preprocessing, improved source signal recovery. This study expands on previous head phantom experiments by including neck muscle source activity and evaluating artificial electrocortical spectral power fluctuations synchronized with gait events.

Personalizing neuromodulation

In the era of "big data", we are gaining rich person-specific information about neuroanatomy, neural function, and cognitive functions. However, the optimal ways to create precise approaches to optimize individuals' mental functions in health and disease are unclear. Multimodal analysis and modeling approaches can guide neuromodulation by combining anatomical networks, functional signal analysis, and cognitive neuroscience paradigms in single subjects. Our progress could be improved by progressing from statistical fits to mechanistic models. Using transcranial magnetic stimulation as an example, we discuss how integrating methods with a focus on mechanisms could improve our predictions TMS effects within individuals, refine our models of health and disease, and improve our treatments.

Progesterone modulates theta oscillations in the frontal-parietal network

The neuroactive metabolites of the steroid hormones progesterone (P4) and testosterone (T) are GABAergic modulators that influence cognition, yet, the specific effect of P4 and T on brain network activity remains poorly understood. Here, we investigated if a fundamental oscillatory network activity pattern, often related to cognitive control, frontal midline theta (FMT) oscillations, are modulated by steroids hormones, P4 and T. We measured the concentration of P4 and T using salivary enzyme immunoassay and FMT oscillations using high-density electroencephalography (EEG) during eyes-open resting-state in 55 healthy women and men. Electrical brain activity was analyzed using Fourier analysis, aperiodic signal fitting, and beamformer source localization. Steroid hormone concentrations and biological sex were used as predictors for scalp and source-estimated amplitude of theta oscillations. Elevated concentrations of P4 predicted increased amplitude of FMT oscillations across both sexes, and no relationship was found with T. The positive correlation with P4 was specific to the frontal midline electrodes and survived correction for the background aperiodic signal of the brain. Using source localization, FMT oscillations were localized to the frontal-parietal network (FPN). Additionally, theta amplitude within the FPN, but not the default mode network, positively correlated with P4 concentration. Our results suggest that P4 concentration modulates brain activity via upregulation of theta oscillations in the FPN.

Closed-Loop Frontal Midlineθ Neurofeedback: A Novel Approach for Training Focused-Attention Meditation

Cortical oscillations serve as an index of both sensory and cognitive processes and represent one of the most promising candidates for training and targeting the top-down mechanisms underlying executive functions. Research findings suggest that theta (θ) oscillations (3-7 Hz) recorded over frontal-midline electrodes are broadly associated with a number of higher-order cognitive processes and may serve as the mechanistic backbone for cognitive control. Frontal-midline theta (FMθ) oscillations have also been shown to inversely correlate with activity in the default mode network (DMN), a network in the brain linked to spontaneous thought processes such as mind-wandering and rumination. In line with these findings, we previously observed increased FMθ oscillations in expert meditation practitioners during reported periods of focused-attention meditation practice when compared to periods of mind-wandering. In an effort to narrow the explanatory gap by directly connecting observed neurophysiological activity in the brain to the phenomenological nature of reported experience, we designed a methodologically novel and adaptive neurofeedback protocol with the aim of modulating FMθ while having meditation novice participants implement breath-focus strategies derived from focused-attention mediation practices. Participants who received eight sessions of the adaptive FMθ-meditation neurofeedback protocol were able to significantly modulate FMθ over frontal electrodes using focused-attention meditation strategies relative to their baseline by the end of the training and demonstrated significantly faster reaction times on correct trials during the n-back working memory task assessed before and after the FMθ-meditation neurofeedback protocol. No significant differences in frontal theta activity or behavior were observed in the active control participants who received age and gender matched sham neurofeedback. These findings help lay the groundwork for the development of brain training protocols and neurofeedback applications that aim to train features of the mental states and traits associated with focused-attention meditation.

Decoding the Attended Speaker From EEG Using Adaptive Evaluation Intervals Captures Fluctuations in Attentional Listening

Listeners differ in their ability to attend to a speech stream in the presence of a competing sound. Differences in speech intelligibility in noise cannot be fully explained by the hearing ability which suggests the involvement of additional cognitive factors. A better understanding of the temporal fluctuations in the ability to pay selective auditory attention to a desired speech stream may help in explaining these variabilities. In order to better understand the temporal dynamics of selective auditory attention, we developed an online auditory attention decoding (AAD) processing pipeline based on speech envelope tracking in the electroencephalogram (EEG). Participants had to attend to one audiobook story while a second one had to be ignored. Online AAD was applied to track the attention toward the target speech signal. Individual temporal attention profiles were computed by combining an established AAD method with an adaptive staircase procedure. The individual decoding performance over time was analyzed and linked to behavioral performance as well as subjective ratings of listening effort, motivation, and fatigue. The grand average attended speaker decoding profile derived in the online experiment indicated performance above chance level. Parameters describing the individual AAD performance in each testing block indicated significant differences in decoding performance over time to be closely related to the behavioral performance in the selective listening task. Further, an exploratory analysis indicated that subjects with poor decoding performance reported higher listening effort and fatigue compared to good performers. Taken together our results show that online EEG based AAD in a complex listening situation is feasible. Adaptive attended speaker decoding profiles over time could be used as an objective measure of behavioral performance and listening effort. The developed online processing pipeline could also serve as a basis for future EEG based near real-time auditory neurofeedback systems.

Non-invasive Neurophysiology in Learning and Training: Mechanisms and a SWOT Analysis

Although many scholars deem non-invasive measures of neurophysiology to have promise in assessing learning, these measures are currently not widely applied, neither in educational settings nor in training. How can non-invasive neurophysiology provide insight into learning and how should research on this topic move forward to ensure valid applications? The current article addresses these questions by discussing the mechanisms underlying neurophysiological changes during learning followed by a SWOT (strengths, weaknesses, opportunities, and threats) analysis of non-invasive neurophysiology in learning and training. This type of analysis can provide a structured examination of factors relevant to the current state and future of a field. The findings of the SWOT analysis indicate that the field of neurophysiology in learning and training is developing rapidly. By leveraging the opportunities of neurophysiology in learning and training (while bearing in mind weaknesses, threats, and strengths) the field can move forward in promising directions. Suggestions for opportunities for future work are provided to ensure valid and effective application of non-invasive neurophysiology in a wide range of learning and training settings.

Temporal cues trick the visual and auditory cortices mimicking spatial cues in blind individuals

In the absence of vision, spatial representation may be altered. When asked to compare the relative distances between three sounds (i.e., auditory spatial bisection task), blind individuals demonstrate significant deficits and do not show an event-related potential response mimicking the visual C1 reported in sighted people. However, we have recently demonstrated that the spatial deficit disappears if coherent time and space cues are presented to blind people, suggesting that they may use time information to infer spatial maps. In this study, we examined whether the modification of temporal cues during space evaluation altered the recruitment of the visual and auditory cortices in blind individuals. We demonstrated that the early (50-90 ms) occipital response, mimicking the visual C1, is not elicited by the physical position of the sound, but by its virtual position suggested by its temporal delay. Even more impressively, in the same time window, the auditory cortex also showed this pattern and responded to temporal instead of spatial coordinates.

Does Reducing Sedentarity With Standing Desks Hinder Cognitive Performance?

OBJECTIVE

The goal of this study was to determine if using a standing desk would affect the productivity of workers, based on the type of work they perform.

BACKGROUND

Standing desks are a promising new health intervention in the workplace, but users and employers often require more specific recommendations related to productivity, such as the type of work that is more suited for the standing desk.

METHOD

Thirty-seven young and healthy adults performed eight cognitive tasks in a 2 × 2 × 2 within-subject design of the following independent variables: posture (sitting/standing), task difficulty (easy/hard), and input device (computer mouse/tactile screen) in a counterbalanced order.

RESULTS

Our results revealed that using a standing desk had no negative effect on performance or perception, but it did lead to increased brain activity in the alpha band for the parietal region (β = 0.186, p = .001).

CONCLUSION

We conclude that users of standing desks can freely stand for any level of task difficulty for work that involves working memory. However, more research is needed to generalize these results to other types of cognitive abilities and prolonged use of standing desks.

APPLICATION

Our results simplify recommendations for workers as they do not need to worry about the type of work they are performing when using a standing desk.

Neuroergonomic Assessment of Hot Beverage Preparation and Consumption: An EEG and EDA Study

Neuroergonomics is an emerging field that investigates the human brain about behavioral performance in natural environments and everyday settings. This study investigated the body and brain activity correlates of a typical daily activity, hot beverage preparation, and consumption in a realistic office environment where participants performed natural daily tasks. Using wearable, battery operated and wireless Electroencephalogram (EEG) and Electrodermal activity (EDA) sensors, neural and physiological responses were measured in untethered, freely moving participants who prepared hot beverages using two different machines (a market leader and follower as determined by annual US sales). They later consumed the drinks they had prepared in three blocks. Emotional valence was estimated using frontal asymmetry in EEG alpha band power and emotional arousal was estimated from EDA tonic and phasic activity. Results from 26 participants showed that the market-leading coffee machine was more efficient to use based on self-reports, behavioral performance measures, and there were significant within-subject differences in valence between the two machine use. Moreover, the market leader user interface led to greater self-reported product preference, which was further supported by significant differences in measured arousal and valence (EDA and EEG, respectively) during coffee production and consumption. This is the first study that uses a multimodal and comprehensive assessment of coffee machine use and beverage consumption in a naturalistic work environment. Approaches described in this study can be adapted in the future to other task-specific machine usability and consumer neuroscience studies.

Revealing Unconscious Consumer Reactions to Advertisements That Include Visual Metaphors. A Neurophysiological Experiment

The main challenge of advertising is to catch consumers' attention and evoke in them positive attitudes to consequently achieve product preference and higher purchase intentions. In modern advertising, visual metaphors are widely used due to their effects such as improving advertising recall, enhancing persuasiveness, and generating consumers' positive attitudes. Previous research has pointed out the existence of an "inverted U-curve" that describes a positive relationship between the conceptual complexity of metaphors and consumers' positive reactions to them, which ends where complexity outweighs comprehension. Despite the dominance of visual metaphors in modern advertising, academic research on this topic has been relatively sparse. The inverted U-curve pattern has been validated regarding ad appreciation, ad liking, and purchase intention by using declarative methods. However, at present, there is no evidence of consumers' neurophysiological responses to visual metaphors included in advertising. Given this gap, the aim of this research is to assess consumer neurophysiological responses to print advertisements that include visual metaphors, using neuroscience-based techniques. Forty-three participants (22W-21M) were exposed to 28 stimuli according to three levels of visual complexity, while their reactions were recorded with an electroencephalogram (EEG), eye tracking (ET), and galvanic skin response (GSR). The results indicated that, regardless of metaphor type, ads with metaphors evoke more positive reactions than non-metaphor ads. EEG results revealed a positive relationship between cognitive load and conceptual complexity that is not mediated by comprehension. This suggests that the cognitive load index could be a suitable indicator of complexity, as it reflects the amount of cognitive resources needed to process stimuli. ET results showed significant differences in the time dedicated to exploring the ads; however, comprehension doesn't mediate this relationship. Moreover, no cognitive load was detected from GSR. ET and GSR results suggest that neither methodology is a suitable measure of cognitive load in the case of visual metaphors. Instead, it seems that they are more related to the attention and/or emotion devoted to the stimuli. Our empirical analysis reveals the importance of using neurophysiological measures to analyze the appropriate use of visual metaphors and to find out how to maximize their impact on advertising effectiveness.

Prediction of gait intention from pre-movement EEG signals: a feasibility study

BACKGROUND

Prediction of Gait intention from pre-movement Electroencephalography (EEG) signals is a vital step in developing a real-time Brain-computer Interface (BCI) for a proper neuro-rehabilitation system. In that respect, this paper investigates the feasibility of a fully predictive methodology to detect the intention to start and stop a gait cycle by utilizing EEG signals obtained before the event occurrence.

METHODS

An eight-channel, custom-made, EEG system with electrodes placed around the sensorimotor cortex was used to acquire EEG data from six healthy subjects and two amputees. A discrete wavelet transform-based method was employed to capture event related information in alpha and beta bands in the time-frequency domain. The Hjorth parameters, namely activity, mobility, and complexity, were extracted as features while a two-sample unpaired Wilcoxon test was used to get rid of redundant features for better classification accuracy. The feature set thus obtained was then used to classify between 'walk vs. stop' and 'rest vs. start' classes using support vector machine (SVM) classifier with RBF kernel in a ten-fold cross-validation scheme.

RESULTS

Using a fully predictive intention detection system, 76.41±4.47% accuracy, 72.85±7.48% sensitivity, and 79.93±5.50% specificity were achieved for 'rest vs. start' classification. While for 'walk vs. stop' classification, the obtained mean accuracy, sensitivity, and specificity were 74.12±4.12%, 70.24±6.45%, and 77.78±7.01% respectively. Overall average True Positive Rate achieved by this methodology was 72.06±8.27% with 1.45 False Positives/min.

CONCLUSION

Extensive simulations and resulting classification results show that it is possible to achieve statistically similar intention detection accuracy using either only pre-movement EEG features or trans-movement EEG features. The classifier performance shows the potential of the proposed methodology to predict human movement intention exclusively from the pre-movement EEG signal to be applied in real-life prosthetic and neuro-rehabilitation systems.

Studying brain activity during word-by-word interactions using wireless EEG

We introduce here the word-by-word paradigm, a dynamic setting, in which two people take turns in producing a single sentence. This task requires a high degree of coordination between the partners and the simplicity of the task allows us to study with sufficient experimental control behavioral and neural processes that underlie this controlled interaction. For this study, 13 pairs of individuals engaged in a scripted word-by-word interaction, while we recorded the neural activity of both participants simultaneously using wireless EEG. To study expectation building, different semantic contexts were primed for each participant. Semantically unexpected continuations were introduced in 25% of all sentences. In line with the hypothesis, we observed amplitude differences for the P200-N400-P600 ERPs for unexpected compared to expected words. Moreover, we could successfully assess speech and reaction times. Our results show that it is possible to measure ERPs and RTs to semantically unexpected words in a dyadic interactive scenario.

Effect of Cycling on a Stationary Bike While Performing Assembly Tasks on Human Physiology and Performance Parameters

Objective: This study evaluated participants’ ability to assemble a computer keyboard while at a cycling workstation. Depending on task completion time, error percentage, and workload based on subjective workload ratings, subjective body discomfort, electroencephalography (EEG) and electrocardiographic (ECG) signals, human performances were compared at four different cycling conditions: no cycling, low level cycling (15 km/h), preferred level cycling, and high level cycling (25 km/h). Method: The experiment consisted of 16 participants. Each participant performed the test four times (each cycling condition) on different days. Results: The repeated measure test showed that the alpha and beta EEG signals were high during session times (post) when compared with session times (pre). Moreover, the mean interbeat (R-R) interval decreased after the participants performed the assembly while pedaling, possibly due to the physical effort of cycling. Conclusions: Pedaling had no significant effect on body discomfort ratings, task errors, or completion time.

Faster Gait Speeds Reduce Alpha and Beta EEG Spectral Power From Human Sensorimotor Cortex

OBJECTIVE

Our aim was to determine if walking speed affected human sensorimotor electrocortical dynamics using mobile high-density electroencephalography (EEG).

METHODS

To overcome limitations associated with motion and muscle artifact contamination in EEG recordings, we compared solutions for artifact removal using novel dual-layer EEG electrodes and alternative signal processing methods. Dual-layer EEG simultaneously recorded human electrocortical signals and isolated motion artifacts using pairs of mechanically coupled and electrically independent electrodes. For electrical muscle activity removal, we incorporated electromyographic (EMG) recordings from the neck into our mobile EEG data processing pipeline. We compared artifact removal methods during treadmill walking at four speeds (0.5, 1.0, 1.5, and 2.0 m/s).

RESULTS

Left and right sensorimotor alpha and beta spectral power increased in contralateral limb single support and push off, and decreased during contralateral limb swing at each speed. At faster walking speeds, sensorimotor spectral power fluctuations were less pronounced across the gait cycle with reduced alpha and beta power (p < 0.05) compared to slower speeds. Isolated noise recordings and neck EMG spectral power fluctuations matched gait events and showed broadband spectral power increases at faster speeds.

CONCLUSION AND SIGNIFICANCE

Dual-layer EEG enabled us to isolate changes in human sensorimotor electrocortical dynamics across walking speeds. A comparison of signal processing approaches revealed similar intrastride cortical fluctuations when applying common (e.g., artifact subspace reconstruction) and novel artifact rejection methods. Dual-layer EEG, however, allowed us to document and rule out residual artifacts, which exposed sensorimotor spectral power changes across gait speeds.

Prediction of Pilot's Reaction Time Based on EEG Signals

The main hypothesis of this work is that the time of delay in reaction to an unexpected event can be predicted on the basis of the brain activity recorded prior to that event. Such mental activity can be represented by electroencephalographic data. To test this hypothesis, we conducted a novel experiment involving 19 participants that took part in a 2-h long session of simulated aircraft flights. An EEG signal processing pipeline is proposed that consists of signal preprocessing, extracting bandpass features, and using regression to predict the reaction times. The prediction algorithms that are used in this study are the Least Absolute Shrinkage Operator and its Least Angle Regression modification, as well as Kernel Ridge and Radial Basis Support Vector Machine regression. The average Mean Absolute Error obtained across the 19 subjects was 114 ms. The present study demonstrates, for the first time, that it is possible to predict reaction times on the basis of EEG data. The presented solution can serve as a foundation for a system that can, in the future, increase the safety of air traffic.

Children With Unilateral Cerebral Palsy Utilize More Cortical Resources for Similar Motor Output During Treadmill Gait

Children with unilateral cerebral palsy (CP) walk independently although with an asymmetrical, more poorly coordinated pattern compared to their peers. While gait biomechanics in unilateral CP and their alteration from those without CP have been well documented, cortical mechanisms underlying gait remain inadequately understood. To the best of our knowledge, this is the first study utilizing electroencephalography (EEG) during treadmill gait in older children with and without CP. Lower limb surface electromyographic (EMG) data were collected and muscle synergy analyses performed to quantify motor output. Our primary goal was to evaluate the relationships between cortical and muscle activation within and across groups and hemispheres to provide novel insights into neural control of gait and how it may be disrupted by an early unilateral brain injury. Participants included 9 children with unilateral CP, mean age 16.0 ± 2.7 years, and 12 with typical development (TD), mean age 14.8 ± 3.0 years. EEG data were collected during a standing baseline and treadmill walking at self-selected speed. EMG of 16 lower limb muscles were also collected bilaterally and synchronized with EEG. No significant group differences were found in synergy number or structure across groups. Six cortical clusters were identified as having gait-related activation and all contained participants from both CP and TD groups; however, the percent of individuals per group appearing in different clusters varied. Notably, the cluster least represented in CP was the non-dominant motor region. Both groups showed mu-band ERD in the motor clusters during gait although sustained beta-band ERD was not evident in TD. The CP group showed greater cortical activation than TD during walking as measured by mu- and beta-ERD in the dominant and non-dominant motor and parietal regions and elevated low gamma-activity in the frontal and parietal areas, a unique finding in CP. CP showed greater bilateral motor EEG-EMG coherence in the gamma-band with the hallucis longus compared to TD. In summary, individuals with CP display increased cortical activation during gait possibly relating to differences in distal motor control of the more affected side. Strategies that iteratively reduce cortical activation while improving selective motor control are needed in CP.

The effects of handedness on sensorimotor rhythm desynchronization and motor-imagery BCI control

Brain–computer interfaces (BCIs) allow control of various applications or external devices solely by brain activity, e.g., measured by electroencephalography during motor imagery. Many users are unable to modulate their brain activity sufficiently in order to control a BCI. Most of the studies have been focusing on improving the accuracy of BCI control through advances in signal processing and BCI protocol modification. However, some research suggests that motor skills and physiological factors may affect BCI performance as well. Previous studies have indicated that there is differential lateralization of hand movements’ neural representation in right- and left-handed individuals. However, the effects of handedness on sensorimotor rhythm (SMR) distribution and BCI control have not been investigated in detail yet. Our study aims to fill this gap, by comparing the SMR patterns during motor imagery and real-feedback BCI control in right- (N = 20) and left-handers (N = 20). The results of our study show that the lateralization of SMR during a motor imagery task differs according to handedness. Left-handers present lower accuracy during BCI performance (single session) and weaker SMR suppression in the alpha band (8–13 Hz) during mental simulation of left-hand movements. Consequently, to improve BCI control, the user’s training should take into account individual differences in hand dominance.

Surprise response as a probe for compressed memory states

The limited capacity of recent memory inevitably leads to partial memory of past stimuli. There is also evidence that behavioral and neural responses to novel or rare stimuli are dependent on one’s memory of past stimuli. Thus, these responses may serve as a probe of different individuals’ remembering and forgetting characteristics. Here, we utilize two lossy compression models of stimulus sequences that inherently involve forgetting, which in addition to being a necessity under many conditions, also has theoretical and behavioral advantages. One model is based on a simple stimulus counter and the other on the Information Bottleneck (IB) framework which suggests a more general, theoretically justifiable principle for biological and cognitive phenomena. These models are applied to analyze a novelty-detection event-related potential commonly known as the P300. The trial-by-trial variations of the P300 response, recorded in an auditory oddball paradigm, were subjected to each model to extract two stimulus-compression parameters for each subject: memory length and representation accuracy. These parameters were then utilized to estimate the subjects’ recent memory capacity limit under the task conditions. The results, along with recently published findings on single neurons and the IB model, underscore how a lossy compression framework can be utilized to account for trial-by-trial variability of neural responses at different spatial scales and in different individuals, while at the same time providing estimates of individual memory characteristics at different levels of representation using a theoretically-based parsimonious model.

Surprise responses reflect expectations based on preceding stimuli representations, and hence can be used to infer the characteristics of memory utilized for a task. We suggest a quantitative method for extracting an individual estimate of effective memory capacity dedicated for a task based on the correspondence between a theoretical surprise model and electrophysiological single-trial surprise responses. We demonstrate this method on EEG responses recorded while participants were performing a simple auditory task; we show the correspondence between the theoretical and physiological surprise, and calculate an estimate of the utilized memory. The generality of this framework allows it to be applied to different EEG features that reflect different modes and levels of the processing hierarchy, as well as other physiological measures of surprise responses. Future studies may use this framework to construct a handy diagnostic tool for a quantitative, individualized characterization of memory-related disorders.

Neurophysiological substrates of configural face perception in schizotypy

Face perception is a highly developed function of the human visual system. Previous studies of event-related potentials (ERPs) have identified a face-selective ERP component (negative peak at about 170 ms after stimulus onset, N170) in healthy participants. In contrast, patients with schizophrenia exhibit reduced amplitude of the N170, which may represent a pathological deficit in the neurophysiology of face perception. Interestingly, healthy humans with schizophrenia-like experiences (schizotypy) also exhibit abnormal processing of face perception. Yet, it has remained unknown how schizotypy in healthy humans is associated with the neurophysiological substrates of face perception. Here, we recruited 35 healthy participants and assessed their schizotypy by the magical ideation rating scale. We used high-density electroencephalography to obtain ERPs elicited by a set of Mooney faces (face and non-face visual stimuli). We investigated median and mean reaction times and visual ERP components in response to the stimuli. We observed a significant difference in N170 amplitude between the two face-stimulus conditions and found that the measured schizotypy scores were significantly correlated with both reaction times and N170 amplitude in response to the face stimuli across all participants. Our results thus support the model of schizotypy as a manifestation of a continuum between healthy individuals and patients with schizophrenia, where the N170 impairment serves as a biomarker for the degree of pathology along this continuum.

Expressure: Detect Expressions Related to Emotional and Cognitive Activities Using Forehead Textile Pressure Mechanomyography

We investigate how pressure-sensitive smart textiles, in the form of a headband, can detect changes in facial expressions that are indicative of emotions and cognitive activities. Specifically, we present the Expressure system that performs surface pressure mechanomyography on the forehead using an array of textile pressure sensors that is not dependent on specific placement or attachment to the skin. Our approach is evaluated in systematic psychological experiments. First, through a mimicking expression experiment with 20 participants, we demonstrate the system’s ability to detect well-defined facial expressions. We achieved accuracies of 0.824 to classify among three eyebrow movements (0.333 chance-level) and 0.381 among seven full-face expressions (0.143 chance-level). A second experiment was conducted with 20 participants to induce cognitive loads with N-back tasks. Statistical analysis has shown significant correlations between the Expressure features on a fine time granularity and the cognitive activity. The results have also shown significant correlations between the Expressure features and the N-back score. From the 10 most facially expressive participants, our approach can predict whether the N-back score is above or below the average with 0.767 accuracy.

Electrocortical activity changes in response to unpredictable trip perturbations induced by a split-belt treadmill

This study explored the contributions of cortical activity in the primary sensorimotor cortex (SMC) and the posterior parietal cortex (PPC) to recovery responses following unpredictable trip perturbations. A technology platform equipped with a programmable split-belt treadmill induced unpredictable trip perturbations while walking. 128-channel non-invasive electroencephalography (EEG) signals were collected. Power spectral analysis was performed to quantify the electrocortical activity of two clusters in the SMC and PPC during quiet standing, steady state walking, and recovery periods. Alpha (8-13 Hz) power of the SMC and PPC was significantly suppressed during the recovery period compared to the standing and walking periods. The main finding of this study could inform the future development gait perturbation paradigms that facilitate the recovery responses in different populations, based on motor learning by repetition.

On Assessing Driver Awareness of Situational Criticalities: Multi-modal Bio-Sensing and Vision-Based Analysis, Evaluations, and Insights

Automobiles for our roadways are increasingly using advanced driver assistance systems. The adoption of such new technologies requires us to develop novel perception systems not only for accurately understanding the situational context of these vehicles, but also to infer the driver's awareness in differentiating between safe and critical situations. This manuscript focuses on the specific problem of inferring driver awareness in the context of attention analysis and hazardous incident activity. Even after the development of wearable and compact multi-modal bio-sensing systems in recent years, their application in driver awareness context has been scarcely explored. The capability of simultaneously recording different kinds of bio-sensing data in addition to traditionally employed computer vision systems provides exciting opportunities to explore the limitations of these sensor modalities. In this work, we explore the applications of three different bio-sensing modalities namely electroencephalogram (EEG), photoplethysmogram (PPG) and galvanic skin response (GSR) along with a camera-based vision system in driver awareness context. We assess the information from these sensors independently and together using both signal processing- and deep learning-based tools. We show that our methods outperform previously reported studies to classify driver attention and detecting hazardous/non-hazardous situations for short time scales of two seconds. We use EEG and vision data for high resolution temporal classification (two seconds) while additionally also employing PPG and GSR over longer time periods. We evaluate our methods by collecting user data on twelve subjects for two real-world driving datasets among which one is publicly available (KITTI dataset) while the other was collected by us (LISA dataset) with the vehicle being driven in an autonomous mode. This work presents an exhaustive evaluation of multiple sensor modalities on two different datasets for attention monitoring and hazardous events classification.

Comparison of Signal Processing Methods for Reducing Motion Artifacts in High-Density Electromyography During Human Locomotion

Objective: High-density electromyography (EMG) is useful for studying changes in myoelectric activity within a muscle during human movement, but it is prone to motion artifacts during locomotion. We compared canonical correlation analysis and principal component analysis methods for signal decomposition and component filtering with a traditional EMG high-pass filtering approach to quantify their relative performance at removing motion artifacts from high-density EMG of the gastrocnemius and tibialis anterior muscles during human walking and running. Results: Canonical correlation analysis filtering provided a greater reduction in signal content at frequency bands associated with motion artifacts than either traditional high-pass filtering or principal component analysis filtering. Canonical correlation analysis filtering also minimized signal reduction at frequency bands expected to consist of true myoelectric signal. Conclusions: Canonical correlation analysis filtering appears to outperform a standard high-pass filter and principal component analysis filter in cleaning high-density EMG collected during fast walking or running.

Error-Related Neural Responses Recorded by Electroencephalography During Post-stroke Rehabilitation Movements

Error-related potential (ErrP) based assist-as-needed robot-therapy can be an effective rehabilitation method. To date, several studies have shown the presence of ErrP under various task situations. However, in the context of assist-as-needed methods, the existence of ErrP is unexplored. Therefore, the principal objective of this study is to determine if an ErrP can be evoked when a subject is unable to complete a physical exercise in a given time. Fifteen stroke patients participated in an experiment that involved performing a physical rehabilitation exercise. Results showed that the electroencephalographic (EEG) response of the trials, where patients failed to complete the exercise, against the trials, where patients successfully completed the exercise, significantly differ from each other, and the resulting difference of event-related potentials resembles the previously reported ErrP signals as well as has some unique features. Along with the highly statistically significant difference, the trials differ in time-frequency patterns and scalp distribution maps. In summary, the results of the study provide a novel basis for the detection of the failure against the success events while executing rehabilitation exercises that can be used to improve the state-of-the-art robot-assisted rehabilitation methods.

Music Improvisation Is Characterized by Increase EEG Spectral Power in Prefrontal and Perceptual Motor Cortical Sources and Can be Reliably Classified From Non-improvisatory Performance

This study expores neural activity underlying creative processes through the investigation of music improvisation. Fourteen guitar players with a high level of improvisation skill participated in this experiment. The experimental task involved playing 32-s alternating blocks of improvisation and scales on guitar. electroencephalography (EEG) data was measured continuously throughout the experiment. In order to remove potential artifacts and extract brain-related activity the following signal processing techniques were employed: bandpass filtering, Artifact Subspace Reconstruction, and Independent Component Analysis (ICA). For each participant, artifact related independent components (ICs) were removed from the EEG data and only ICs found to be from brain activity were retained. Source localization using this brain-related activity was carried out using sLORETA. Greater activity for improvisation over scale was found in multiple frequency bands (theta, alpha, and beta) localized primarily in the medial frontal cortex (MFC), Middle frontal gyrus (MFG), anterior cingulate, polar medial prefrontal cortex (MPFC), premotor cortex (PMC), pre and postcentral gyrus (PreCG and PostCG), superior temporal gyrus (STG), inferior parietal lobule (IPL), and the temporal-parietal junction. Together this collection of brain regions suggests that improvisation was mediated by processes involved in coordinating planned sequences of movement that are modulated in response to ongoing environmental context through monitoring and feedback of sensory states in relation to internal plans and goals. Machine-learning using Common Spatial Patterns (CSP) for EEG feature extraction attained a mean of over 75% classification performance for improvisation vs. scale conditions across participants. These machine-learning results are a step towards the development of a brain-computer interface that could be used for neurofeedback training to improve creativity.

A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification

OBJECTIVE

Electroencephalogram (EEG) based brain-computer interfaces (BCI) in motor imagery (MI) have developed rapidly in recent years. A reliable feature extraction method is essential because of a low signal-to-noise ratio (SNR) and time-dependent covariates of EEG signals. Because of efficient application in various fields, deep learning has been adopted in EEG signal processing and has obtained competitive results compared with the traditional methods. However, designing and training an end-to-end network to fully extract potential features from EEG signals remains a challenge in MI.

APPROACH

In this study, we propose a parallel multiscale filter bank convolutional neural network (MSFBCNN) for MI classification. We introduce a layered end-to-end network structure, in which a feature-extraction network is used to extract temporal and spatial features. To enhance the transfer learning ability, we propose a network initialization and fine-tuning strategy to train an individual model for inter-subject classification on small datasets. We compare our MSFBCNN with the state-of-the-art approaches on open datasets.

RESULTS

The proposed method has a higher accuracy than the baselines in intra-subject classification. In addition, the transfer learning experiments indicate that our network can build an individual model and obtain acceptable results in inter-subject classification. The results suggest that the proposed network has superior performance, robustness, and transfer learning ability.