A neurophysiologically interpretable deep neural network predicts complex movement components from brain activity

FetchEEG: a hybrid approach combining feature extraction and temporal-channel joint attention for EEG-based emotion classification

Objective. Electroencephalogram (EEG) analysis has always been an important tool in neural engineering, and the recognition and classification of human emotions are one of the important tasks in neural engineering. EEG data, obtained from electrodes placed on the scalp, represent a valuable resource of information for brain activity analysis and emotion recognition. Feature extraction methods have shown promising results, but recent trends have shifted toward end-to-end methods based on deep learning. However, these approaches often overlook channel representations, and their complex structures pose certain challenges to model fitting.Approach. To address these challenges, this paper proposes a hybrid approach named FetchEEG that combines feature extraction and temporal-channel joint attention. Leveraging the advantages of both traditional feature extraction and deep learning, the FetchEEG adopts a multi-head self-attention mechanism to extract representations between different time moments and channels simultaneously. The joint representations are then concatenated and classified using fully-connected layers for emotion recognition. The performance of the FetchEEG is verified by comparison experiments on a self-developed dataset and two public datasets.Main results. In both subject-dependent and subject-independent experiments, the FetchEEG demonstrates better performance and stronger generalization ability than the state-of-the-art methods on all datasets. Moreover, the performance of the FetchEEG is analyzed for different sliding window sizes and overlap rates in the feature extraction module. The sensitivity of emotion recognition is investigated for three- and five-frequency-band scenarios.Significance. FetchEEG is a novel hybrid method based on EEG for emotion classification, which combines EEG feature extraction with Transformer neural networks. It has achieved state-of-the-art performance on both self-developed datasets and multiple public datasets, with significantly higher training efficiency compared to end-to-end methods, demonstrating its effectiveness and feasibility.

Distinguishing Laparoscopic Surgery Experts from Novices Using EEG Topographic Features

The study aimed to differentiate experts from novices in laparoscopic surgery tasks using electroencephalogram (EEG) topographic features. A microstate-based common spatial pattern (CSP) analysis with linear discriminant analysis (LDA) was compared to a topography-preserving convolutional neural network (CNN) approach. Expert surgeons (N = 10) and novice medical residents (N = 13) performed laparoscopic suturing tasks, and EEG data from 8 experts and 13 novices were analysed. Microstate-based CSP with LDA revealed distinct spatial patterns in the frontal and parietal cortices for experts, while novices showed frontal cortex involvement. The 3D CNN model (ESNet) demonstrated a superior classification performance (accuracy > 98%, sensitivity 99.30%, specificity 99.70%, F1 score 98.51%, MCC 97.56%) compared to the microstate based CSP analysis with LDA (accuracy ~90%). Combining spatial and temporal information in the 3D CNN model enhanced classifier accuracy and highlighted the importance of the parietal-temporal-occipital association region in differentiating experts and novices.

Multimodal intelligent logistics robot combining 3D CNN, LSTM, and visual SLAM for path planning and control

Introduction

In today's dynamic logistics landscape, the role of intelligent robots is paramount for enhancing efficiency, reducing costs, and ensuring safety. Traditional path planning methods often struggle to adapt to changing environments, resulting in issues like collisions and conflicts. This research addresses the challenge of path planning and control for logistics robots operating in complex environments. The proposed method aims to integrate information from various perception sources to enhance path planning and obstacle avoidance, thereby increasing the autonomy and reliability of logistics robots.

Methods

The method presented in this paper begins by employing a 3D Convolutional Neural Network (CNN) to learn feature representations of objects within the environment, enabling object recognition. Subsequently, Long Short-Term Memory (LSTM) models are utilized to capture spatio-temporal features and predict the behavior and trajectories of dynamic obstacles. This predictive capability empowers robots to more accurately anticipate the future positions of obstacles in intricate settings, thereby mitigating potential collision risks. Finally, the Dijkstra algorithm is employed for path planning and control decisions to ensure the selection of optimal paths across diverse scenarios.

Results

In a series of rigorous experiments, the proposed method outperforms traditional approaches in terms of both path planning accuracy and obstacle avoidance performance. These substantial improvements underscore the efficacy of the intelligent path planning and control scheme.

Discussion

This research contributes to enhancing the practicality of logistics robots in complex environments, thereby fostering increased efficiency and safety within the logistics industry. By combining object recognition, spatio-temporal modeling, and optimized path planning, the proposed method enables logistics robots to navigate intricate scenarios with higher precision and reliability, ultimately advancing the capabilities of autonomous logistics operations.

Posthoc Interpretability of Neural Responses by Grouping Subject Motor Imagery Skills Using CNN-Based Connectivity

Motor Imagery (MI) refers to imagining the mental representation of motor movements without overt motor activity, enhancing physical action execution and neural plasticity with potential applications in medical and professional fields like rehabilitation and education. Currently, the most promising approach for implementing the MI paradigm is the Brain-Computer Interface (BCI), which uses Electroencephalogram (EEG) sensors to detect brain activity. However, MI-BCI control depends on a synergy between user skills and EEG signal analysis. Thus, decoding brain neural responses recorded by scalp electrodes poses still challenging due to substantial limitations, such as non-stationarity and poor spatial resolution. Also, an estimated third of people need more skills to accurately perform MI tasks, leading to underperforming MI-BCI systems. As a strategy to deal with BCI-Inefficiency, this study identifies subjects with poor motor performance at the early stages of BCI training by assessing and interpreting the neural responses elicited by MI across the evaluated subject set. Using connectivity features extracted from class activation maps, we propose a Convolutional Neural Network-based framework for learning relevant information from high-dimensional dynamical data to distinguish between MI tasks while preserving the post-hoc interpretability of neural responses. Two approaches deal with inter/intra-subject variability of MI EEG data: (a) Extracting functional connectivity from spatiotemporal class activation maps through a novel kernel-based cross-spectral distribution estimator, (b) Clustering the subjects according to their achieved classifier accuracy, aiming to find common and discriminative patterns of motor skills. According to the validation results obtained on a bi-class database, an average accuracy enhancement of 10% is achieved compared to the baseline EEGNet approach, reducing the number of "poor skill" subjects from 40% to 20%. Overall, the proposed method can be used to help explain brain neural responses even in subjects with deficient MI skills, who have neural responses with high variability and poor EEG-BCI performance.

Experiment protocols for brain-body imaging of locomotion: A systematic review

Introduction

Human locomotion is affected by several factors, such as growth and aging, health conditions, and physical activity levels for maintaining overall health and well-being. Notably, impaired locomotion is a prevalent cause of disability, significantly impacting the quality of life of individuals. The uniqueness and high prevalence of human locomotion have led to a surge of research to develop experimental protocols for studying the brain substrates, muscle responses, and motion signatures associated with locomotion. However, from a technical perspective, reproducing locomotion experiments has been challenging due to the lack of standardized protocols and benchmarking tools, which impairs the evaluation of research quality and the validation of previous findings.

Methods

This paper addresses the challenges by conducting a systematic review of existing neuroimaging studies on human locomotion, focusing on the settings of experimental protocols, such as locomotion intensity, duration, distance, adopted brain imaging technologies, and corresponding brain activation patterns. Also, this study provides practical recommendations for future experiment protocols.

Results

The findings indicate that EEG is the preferred neuroimaging sensor for detecting brain activity patterns, compared to fMRI, fNIRS, and PET. Walking is the most studied human locomotion task, likely due to its fundamental nature and status as a reference task. In contrast, running has received little attention in research. Additionally, cycling on an ergometer at a speed of 60 rpm using fNIRS has provided some research basis. Dual-task walking tasks are typically used to observe changes in cognitive function. Moreover, research on locomotion has primarily focused on healthy individuals, as this is the scenario most closely resembling free-living activity in real-world environments.

Discussion

Finally, the paper outlines the standards and recommendations for setting up future experiment protocols based on the review findings. It discusses the impact of neurological and musculoskeletal factors, as well as the cognitive and locomotive demands, on the experiment design. It also considers the limitations imposed by the sensing techniques used, including the acceptable level of motion artifacts in brain-body imaging experiments and the effects of spatial and temporal resolutions on brain sensor performance. Additionally, various experiment protocol constraints that need to be addressed and analyzed are explained.

Decoding trajectories of imagined hand movement using electrocorticograms for brain-machine interface

OBJECTIVE

Reaching hand movement is an important motor skill actively examined in brain-computer interface (BCI). Among various components of movement analyzed is the hand's trajectory, which describes the hand's continuous positions in three-dimensional space. While a large body of studies have investigated the decoding of real movements and the reconstruction of real hand movement trajectories from neural signals, fewer studies have attempted to decode the trajectory of imagined hand movement. To develop BCI systems for patients with hand motor dysfunctions, the systems essentially require to achieve movement-free control of external devices, which is only possible through successful decoding of purely imagined hand movement.

APPROACH

To achieve this goal, this study used a machine learning technique (i.e., the variational Bayesian least square) to analyze the electrocorticogram (ECoG) of eighteen epilepsy patients obtained from when they performed movement execution (ME) and kinesthetic movement imagination (KMI) of the reach-and-grasp hand action.

MAIN RESULTS

The variational Bayesian decoding model was able to successfully predict the imagined trajectories of hand movement significantly above chance level. The Pearson's correlation coefficient between imagined and predicted trajectories was 0.3393 and 0.4936 for the KMI (KMI trials only) and MEKMI paradigm (alternating trials of ME and KMI) respectively.

SIGNIFICANCE

This study demonstrated a high accuracy of prediction for trajectories of imagined hand movement, and more importantly, higher decoding accuracy of imagined trajectories in the MEKMI paradigm than in the KMI paradigm solely.