Single-Trial Kernel-Based Functional Connectivity for Enhanced Feature Extraction in Motor-Related Tasks

Affective Neural Responses Sonified through Labeled Correlation Alignment

Sound synthesis refers to the creation of original acoustic signals with broad applications in artistic innovation, such as music creation for games and videos. Nonetheless, machine learning architectures face numerous challenges when learning musical structures from arbitrary corpora. This issue involves adapting patterns borrowed from other contexts to a concrete composition objective. Using Labeled Correlation Alignment (LCA), we propose an approach to sonify neural responses to affective music-listening data, identifying the brain features that are most congruent with the simultaneously extracted auditory features. For dealing with inter/intra-subject variability, a combination of Phase Locking Value and Gaussian Functional Connectivity is employed. The proposed two-step LCA approach embraces a separate coupling stage of input features to a set of emotion label sets using Centered Kernel Alignment. This step is followed by canonical correlation analysis to select multimodal representations with higher relationships. LCA enables physiological explanation by adding a backward transformation to estimate the matching contribution of each extracted brain neural feature set. Correlation estimates and partition quality represent performance measures. The evaluation uses a Vector Quantized Variational AutoEncoder to create an acoustic envelope from the tested Affective Music-Listening database. Validation results demonstrate the ability of the developed LCA approach to generate low-level music based on neural activity elicited by emotions while maintaining the ability to distinguish between the acoustic outputs.

A Novel OpenBCI Framework for EEG-Based Neurophysiological Experiments

An Open Brain-Computer Interface (OpenBCI) provides unparalleled freedom and flexibility through open-source hardware and firmware at a low-cost implementation. It exploits robust hardware platforms and powerful software development kits to create customized drivers with advanced capabilities. Still, several restrictions may significantly reduce the performance of OpenBCI. These limitations include the need for more effective communication between computers and peripheral devices and more flexibility for fast settings under specific protocols for neurophysiological data. This paper describes a flexible and scalable OpenBCI framework for electroencephalographic (EEG) data experiments using the Cyton acquisition board with updated drivers to maximize the hardware benefits of ADS1299 platforms. The framework handles distributed computing tasks and supports multiple sampling rates, communication protocols, free electrode placement, and single marker synchronization. As a result, the OpenBCI system delivers real-time feedback and controlled execution of EEG-based clinical protocols for implementing the steps of neural recording, decoding, stimulation, and real-time analysis. In addition, the system incorporates automatic background configuration and user-friendly widgets for stimuli delivery. Motor imagery tests the closed-loop BCI designed to enable real-time streaming within the required latency and jitter ranges. Therefore, the presented framework offers a promising solution for tailored neurophysiological data processing.

Posthoc Interpretability of Neural Responses by Grouping Subject Motor Imagery Skills Using CNN-Based Connectivity

Motor Imagery (MI) refers to imagining the mental representation of motor movements without overt motor activity, enhancing physical action execution and neural plasticity with potential applications in medical and professional fields like rehabilitation and education. Currently, the most promising approach for implementing the MI paradigm is the Brain-Computer Interface (BCI), which uses Electroencephalogram (EEG) sensors to detect brain activity. However, MI-BCI control depends on a synergy between user skills and EEG signal analysis. Thus, decoding brain neural responses recorded by scalp electrodes poses still challenging due to substantial limitations, such as non-stationarity and poor spatial resolution. Also, an estimated third of people need more skills to accurately perform MI tasks, leading to underperforming MI-BCI systems. As a strategy to deal with BCI-Inefficiency, this study identifies subjects with poor motor performance at the early stages of BCI training by assessing and interpreting the neural responses elicited by MI across the evaluated subject set. Using connectivity features extracted from class activation maps, we propose a Convolutional Neural Network-based framework for learning relevant information from high-dimensional dynamical data to distinguish between MI tasks while preserving the post-hoc interpretability of neural responses. Two approaches deal with inter/intra-subject variability of MI EEG data: (a) Extracting functional connectivity from spatiotemporal class activation maps through a novel kernel-based cross-spectral distribution estimator, (b) Clustering the subjects according to their achieved classifier accuracy, aiming to find common and discriminative patterns of motor skills. According to the validation results obtained on a bi-class database, an average accuracy enhancement of 10% is achieved compared to the baseline EEGNet approach, reducing the number of "poor skill" subjects from 40% to 20%. Overall, the proposed method can be used to help explain brain neural responses even in subjects with deficient MI skills, who have neural responses with high variability and poor EEG-BCI performance.

IMF-Based MF and HS Energy Feature Information of F5, and F6 Movement and Motor Imagery EEG Signals in Delta Rhythms Using HHT

This study aims to extract the energy feature distributions in the form of marginal frequency (MF) and Hilbert spectrum (HS) in the intrinsic mode functions (IMF) domain for actual movement (AM)-based and motor imagery (MI)-based electroencephalogram (EEG) signals using the Hilbert-Huang transformation (HHT) time frequency (TF) analysis method. Accordingly, F5 and F6 EEG signal TF energy feature distributions in delta (0.5-4 Hz) rhythm are explored. We propose IMF-based and residue function (RF)-based MF and HS feature information extraction methods with IMFRFERDD (IMFRF energy refereed distribution density), IMFRFMFERDD (IMFRF MF energy refereed distribution density), and IMFRFHSERDD (IMFRF HS energy refereed distribution density) parameters using HHT with application to AM, MI EEG F5, and F6 signals in delta rhythm. The AM and MI tasks involve simultaneously opening fists and feet, as well as simultaneously closing fists and feet. Eight samples (32 in total) with a time duration of 1000 ms are extracted for analyzing F5AM, F5MI, F6AM, and F6MI EEG signals, which are decomposed into five IMFs and one RF. The maximum average IMFRFERDD values of IMF4 are 3.70, 3.43, 3.65, and 3.69 for F5AM, F5MI, F6 AM, and F6MI, respectively. The maximum average IMFRFMFERDD values of IMF4 in the delta rhythm are 21.50, 20.15, 21.02, and 17.30, for F5AM, F5MI, F6AM, and F6MI, respectively. Additionally, the maximum average IMFRFHSERDD values of IMF4 in delta rhythm are 39,21, 39.14, 36.29, and 33.06 with time intervals of 500-600, 800-900, 800-900, and 500-600 ms, for F5AM, F5MI, F6AM, and F6MI, respectively. The results of this study, advance our understanding of meaningful feature information of F5MM, F5MI, F6MM, and F6MI, enabling the design of MI-based brain-computer interface assistive devices for disabled persons.

Subject-Dependent Artifact Removal for Enhancing Motor Imagery Classifier Performance under Poor Skills

The Electroencephalography (EEG)-based motor imagery (MI) paradigm is one of the most studied technologies for Brain-Computer Interface (BCI) development. Still, the low Signal-to-Noise Ratio (SNR) poses a challenge when constructing EEG-based BCI systems. Moreover, the non-stationary and nonlinear signal issues, the low-spatial data resolution, and the inter- and intra-subject variability hamper the extraction of discriminant features. Indeed, subjects with poor motor skills have difficulties in practicing MI tasks against low SNR scenarios. Here, we propose a subject-dependent preprocessing approach that includes the well-known Surface Laplacian Filtering and Independent Component Analysis algorithms to remove signal artifacts based on the MI performance. In addition, power- and phase-based functional connectivity measures are studied to extract relevant and interpretable patterns and identify subjects of inefficency. As a result, our proposal, Subject-dependent Artifact Removal (SD-AR), improves the MI classification performance in subjects with poor motor skills. Consequently, electrooculography and volume-conduction EEG artifacts are mitigated within a functional connectivity feature-extraction strategy, which favors the classification performance of a straightforward linear classifier.

Complex Pearson Correlation Coefficient for EEG Connectivity Analysis

In the background of all human thinking—acting and reacting are sets of connections between different neurons or groups of neurons. We studied and evaluated these connections using electroencephalography (EEG) brain signals. In this paper, we propose the use of the complex Pearson correlation coefficient (CPCC), which provides information on connectivity with and without consideration of the volume conduction effect. Although the Pearson correlation coefficient is a widely accepted measure of the statistical relationships between random variables and the relationships between signals, it is not being used for EEG data analysis. Its meaning for EEG is not straightforward and rarely well understood. In this work, we compare it to the most commonly used undirected connectivity analysis methods, which are phase locking value (PLV) and weighted phase lag index (wPLI). First, the relationship between the measures is shown analytically. Then, it is illustrated by a practical comparison using synthetic and real EEG data. The relationships between the observed connectivity measures are described in terms of the correlation values between them, which are, for the absolute values of CPCC and PLV, not lower that 0.97, and for the imaginary component of CPCC and wPLI—not lower than 0.92, for all observed frequency bands. Results show that the CPCC includes information of both other measures balanced in a single complex-numbered index.

An Enhanced Joint Hilbert Embedding-Based Metric to Support Mocap Data Classification with Preserved Interpretability

Motion capture (Mocap) data are widely used as time series to study human movement. Indeed, animation movies, video games, and biomechanical systems for rehabilitation are significant applications related to Mocap data. However, classifying multi-channel time series from Mocap requires coding the intrinsic dependencies (even nonlinear relationships) between human body joints. Furthermore, the same human action may have variations because the individual alters their movement and therefore the inter/intraclass variability. Here, we introduce an enhanced Hilbert embedding-based approach from a cross-covariance operator, termed EHECCO, to map the input Mocap time series to a tensor space built from both 3D skeletal joints and a principal component analysis-based projection. Obtained results demonstrate how EHECCO represents and discriminates joint probability distributions as kernel-based evaluation of input time series within a tensor reproducing kernel Hilbert space (RKHS). Our approach achieves competitive classification results for style/subject and action recognition tasks on well-known publicly available databases. Moreover, EHECCO favors the interpretation of relevant anthropometric variables correlated with players’ expertise and acted movement on a Tennis-Mocap database (also publicly available with this work). Thereby, our EHECCO-based framework provides a unified representation (through the tensor RKHS) of the Mocap time series to compute linear correlations between a coded metric from joint distributions and player properties, i.e., age, body measurements, and sport movement (action class).