A deep neural network approach for P300 detection-based BCI using single-channel EEG scalogram images

An Intrinsically Explainable Method to Decode P300 Waveforms from EEG Signal Plots Based on Convolutional Neural Networks

This work proposes an intrinsically explainable, straightforward method to decode P300 waveforms from electroencephalography (EEG) signals, overcoming the black box nature of deep learning techniques. The proposed method allows convolutional neural networks to decode information from images, an area where they have achieved astonishing performance. By plotting the EEG signal as an image, it can be both visually interpreted by physicians and technicians and detected by the network, offering a straightforward way of explaining the decision. The identification of this pattern is used to implement a P300-based speller device, which can serve as an alternative communication channel for persons affected by amyotrophic lateral sclerosis (ALS). This method is validated by identifying this signal by performing a brain-computer interface simulation on a public dataset from ALS patients. Letter identification rates from the speller on the dataset show that this method can identify the P300 signature on the set of 8 patients. The proposed approach achieves similar performance to other state-of-the-art proposals while providing clinically relevant explainability (XAI).

Eye-tracking control of an adjustable electric bed: construction and validation by immobile patients with multiple sclerosis

BACKGROUND

In severe conditions of limited motor abilities, frequent position changes for work or passive and active rest are essential bedside activities to prevent further health complications. We aimed to develop a system using eye movements for bed positioning and to verify its functionality in a control group and a group of patients with significant motor limitation caused by multiple sclerosis.

METHODS

The eye-tracking system utilized an innovative digital-to-analog converter module to control the positioning bed via a novel graphical user interface. We verified the ergonomics and usability of the system by performing a fixed sequence of positioning tasks, in which the leg and head support was repeatedly raised and then lowered. Fifteen women and eleven men aged 42.7 ± 15.9 years in the control group and nine women and eight men aged 60.3 ± 9.14 years in the patient group participated in the experiment. The degree of disability, according to the Expanded Disability Status Scale (EDSS), ranged from 7 to 9.5 points in the patients. We assessed the speed and efficiency of the bed control and the improvement during testing. In a questionnaire, we evaluated satisfaction with the system.

RESULTS

The control group mastered the task in 40.2 s (median) with an interquartile interval from 34.5 to 45.5 s, and patients mastered the task in in 56.5 (median) with an interquartile interval from 46.5 to 64.9 s. The efficiency of solving the task (100% corresponds to an optimal performance) was 86.3 (81.6; 91.0) % for the control group and 72.1 (63.0; 75.2) % for the patient group. Throughout testing, the patients learned to communicate with the system, and their efficiency and task time improved. A correlation analysis showed a negative relationship (rho = - 0.587) between efficiency improvement and the degree of impairment (EDSS). In the control group, the learning was not significant. On the questionnaire survey, sixteen patients reported gaining confidence in bed control. Seven patients preferred the offered form of bed control, and in six cases, they would choose another form of interface.

CONCLUSIONS

The proposed system and communication through eye movements are reliable for positioning the bed in people affected by advanced multiple sclerosis. Seven of 17 patients indicated that they would choose this system for bed control and wished to extend it for another application.

EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM

The constantly evolving human–machine interaction and advancement in sociotechnical systems have made it essential to analyze vital human factors such as mental workload, vigilance, fatigue, and stress by monitoring brain states for optimum performance and human safety. Similarly, brain signals have become paramount for rehabilitation and assistive purposes in fields such as brain–computer interface (BCI) and closed-loop neuromodulation for neurological disorders and motor disabilities. The complexity, non-stationary nature, and low signal-to-noise ratio of brain signals pose significant challenges for researchers to design robust and reliable BCI systems to accurately detect meaningful changes in brain states outside the laboratory environment. Different neuroimaging modalities are used in hybrid settings to enhance accuracy, increase control commands, and decrease the time required for brain activity detection. Functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) measure the hemodynamic and electrical activity of the brain with a good spatial and temporal resolution, respectively. However, in hybrid settings, where both modalities enhance the output performance of BCI, their data compatibility due to the huge discrepancy between their sampling rate and the number of channels remains a challenge for real-time BCI applications. Traditional methods, such as downsampling and channel selection, result in important information loss while making both modalities compatible. In this study, we present a novel recurrence plot (RP)-based time-distributed convolutional neural network and long short-term memory (CNN-LSTM) algorithm for the integrated classification of fNIRS EEG for hybrid BCI applications. The acquired brain signals are first projected into a non-linear dimension with RPs and fed into the CNN to extract essential features without performing any downsampling. Then, LSTM is used to learn the chronological features and time-dependence relation to detect brain activity. The average accuracies achieved with the proposed model were 78.44% for fNIRS, 86.24% for EEG, and 88.41% for hybrid EEG-fNIRS BCI. Moreover, the maximum accuracies achieved were 85.9, 88.1, and 92.4%, respectively. The results confirm the viability of the RP-based deep-learning algorithm for successful BCI systems.