1
|
Iwama S, Tsuchimoto S, Mizuguchi N, Ushiba J. EEG decoding with spatiotemporal convolutional neural network for visualization and closed-loop control of sensorimotor activities: A simultaneous EEG-fMRI study. Hum Brain Mapp 2024; 45:e26767. [PMID: 38923184 PMCID: PMC11199199 DOI: 10.1002/hbm.26767] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 06/06/2024] [Accepted: 06/10/2024] [Indexed: 06/28/2024] Open
Abstract
Closed-loop neurofeedback training utilizes neural signals such as scalp electroencephalograms (EEG) to manipulate specific neural activities and the associated behavioral performance. A spatiotemporal filter for high-density whole-head scalp EEG using a convolutional neural network can overcome the ambiguity of the signaling source because each EEG signal includes information on the remote regions. We simultaneously acquired EEG and functional magnetic resonance images in humans during the brain-computer interface (BCI) based neurofeedback training and compared the reconstructed and modeled hemodynamic responses of the sensorimotor network. Filters constructed with a convolutional neural network captured activities in the targeted network with spatial precision and specificity superior to those of the EEG signals preprocessed with standard pipelines used in BCI-based neurofeedback paradigms. The middle layers of the trained model were examined to characterize the neuronal oscillatory features that contributed to the reconstruction. Analysis of the layers for spatial convolution revealed the contribution of distributed cortical circuitries to reconstruction, including the frontoparietal and sensorimotor areas, and those of temporal convolution layers that successfully reconstructed the hemodynamic response function. Employing a spatiotemporal filter and leveraging the electrophysiological signatures of the sensorimotor excitability identified in our middle layer analysis would contribute to the development of a further effective neurofeedback intervention.
Collapse
Affiliation(s)
- Seitaro Iwama
- Department of Biosciences and Informatics, Faculty of Science and TechnologyKeio UniversityYokohamaJapan
| | - Shohei Tsuchimoto
- School of Fundamental Science and TechnologyGraduate School of Keio UniversityYokohamaJapan
- Department of System NeuroscienceNational Institute for Physiological SciencesOkazakiJapan
| | - Nobuaki Mizuguchi
- Research Organization of Science and TechnologyRitsumeikan UniversityKusatsuJapan
- Institute of Advanced Research for Sport and Health ScienceRitsumeikan UniversityKusatsuJapan
| | - Junichi Ushiba
- Department of Biosciences and Informatics, Faculty of Science and TechnologyKeio UniversityYokohamaJapan
| |
Collapse
|
2
|
Iwama S, Takemi M, Eguchi R, Hirose R, Morishige M, Ushiba J. Two common issues in synchronized multimodal recordings with EEG: Jitter and latency. Neurosci Res 2024; 203:1-7. [PMID: 38141782 DOI: 10.1016/j.neures.2023.12.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 11/19/2023] [Accepted: 12/13/2023] [Indexed: 12/25/2023]
Abstract
Multimodal recording using electroencephalogram (EEG) and other biological signals (e.g., muscle activities, eye movement, pupil diameters, or body kinematics data) is ubiquitous in human neuroscience research. However, the precise time alignment of multiple data from heterogeneous sources (i.e., devices) is often arduous due to variable recording parameters of commercially available research devices and complex experimental setups. In this review, we introduced the versatility of a Lab Streaming Layer (LSL)-based application that can overcome two common issues in measuring multimodal data: jitter and latency. We discussed the issues of jitter and latency in multimodal recordings and the benefits of time-synchronization when recording with multiple devices. In addition, a computer simulation was performed to highlight how the millisecond-order jitter readily affects the signal-to-noise ratio of the electrophysiological outcome. Together, we argue that the LSL-based system can be used for research requiring precise time-alignment of datasets. Studies that detect stimulus-induced transient neural responses or test hypotheses regarding temporal relationships of different functional aspects with multimodal data would benefit most from LSL-based systems.
Collapse
Affiliation(s)
- Seitaro Iwama
- Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Japan
| | - Mitsuaki Takemi
- Graduate School of Science and Technology, Keio University, Japan; Japan Science and Technology Agency PRESTO, Japan
| | - Ryo Eguchi
- Graduate School of Science and Technology, Keio University, Japan
| | - Ryotaro Hirose
- Graduate School of Science and Technology, Keio University, Japan
| | - Masumi Morishige
- Graduate School of Science and Technology, Keio University, Japan
| | - Junichi Ushiba
- Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Japan.
| |
Collapse
|
3
|
Hao Z, Zhai X, Peng B, Cheng D, Zhang Y, Pan Y, Dou W. CAMBA framework: Unveiling the brain asymmetry alterations and longitudinal changes after stroke using resting-state EEG. Neuroimage 2023; 282:120405. [PMID: 37820859 DOI: 10.1016/j.neuroimage.2023.120405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 09/19/2023] [Accepted: 10/08/2023] [Indexed: 10/13/2023] Open
Abstract
Hemispheric asymmetry or lateralization is a fundamental principle of brain organization. However, it is poorly understood to what extent the brain asymmetries across different levels of functional organizations are evident in health or altered in brain diseases. Here, we propose a framework that integrates three degrees of brain interactions (isolated nodes, node-node, and edge-edge) into a unified analysis pipeline to capture the sliding window-based asymmetry dynamics at both the node and hemisphere levels. We apply this framework to resting-state EEG in healthy and stroke populations and investigate the stroke-induced abnormal alterations in brain asymmetries and longitudinal asymmetry changes during poststroke rehabilitation. We observe that the mean asymmetry in patients was abnormally enhanced across different frequency bands and levels of brain interactions, with these abnormal patterns strongly associated with the side of the stroke lesion. Compared to healthy controls, patients displayed significant alterations in asymmetry fluctuations, disrupting and reconfiguring the balance of inter-hemispheric integration and segregation. Additionally, analyses reveal that specific abnormal asymmetry metrics in patients tend to move towards those observed in healthy controls after short-term brain-computer interface rehabilitation. Furthermore, preliminary evidence suggests that baseline clinical and asymmetry features can predict poststroke improvements in the Fugl-Meyer assessment of the lower extremity (mean absolute error of about 2). Overall, these findings advance our understanding of hemispheric asymmetry. Our framework offers new insights into the mechanisms underlying brain alterations and recovery after a brain lesion, may help identify prognostic biomarkers, and can be easily extended to different functional modalities.
Collapse
Affiliation(s)
- Zexuan Hao
- Department of Electronic Engineering, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
| | - Xiaoxue Zhai
- Department of Rehabilitation Medicine, School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 102218, China
| | - Bo Peng
- Department of Rehabilitation Medicine, School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 102218, China
| | - Dandan Cheng
- Department of Rehabilitation Medicine, School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 102218, China
| | - Yanlin Zhang
- Department of Rehabilitation Medicine, School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 102218, China
| | - Yu Pan
- Department of Rehabilitation Medicine, School of Clinical Medicine, Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 102218, China.
| | - Weibei Dou
- Department of Electronic Engineering, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China.
| |
Collapse
|
4
|
Fadli RA, Yamanouchi Y, Jovanovic LI, Popovic MR, Marquez-Chin C, Nomura T, Milosevic M. Effectiveness of motor and prefrontal cortical areas for brain-controlled functional electrical stimulation neuromodulation. J Neural Eng 2023; 20:056022. [PMID: 37714143 DOI: 10.1088/1741-2552/acfa22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 09/15/2023] [Indexed: 09/17/2023]
Abstract
Objective. Brain-computer interface (BCI)-controlled functional electrical stimulation (FES) could excite the central nervous system to enhance upper limb motor recovery. Our current study assessed the effectiveness of motor and prefrontal cortical activity-based BCI-FES to help elucidate the underlying neuromodulation mechanisms of this neurorehabilitation approach.Approach. The primary motor cortex (M1) and prefrontal cortex (PFC) BCI-FES interventions were performed for 25 min on separate days with twelve non-disabled participants. During the interventions, a single electrode from the contralateral M1 or PFC was used to detect event-related desynchronization (ERD) in the calibrated frequency range. If the BCI system detected ERD within 15 s of motor imagery, FES activated wrist extensor muscles. Otherwise, if the BCI system did not detect ERD within 15 s, a subsequent trial was initiated without FES. To evaluate neuromodulation effects, corticospinal excitability was assessed using single-pulse transcranial magnetic stimulation, and cortical excitability was assessed by motor imagery ERD and resting-state functional connectivity before, immediately, 30 min, and 60 min after each intervention.Main results. M1 and PFC BCI-FES interventions had similar success rates of approximately 80%, while the M1 intervention was faster in detecting ERD activity. Consequently, only the M1 intervention effectively elicited corticospinal excitability changes for at least 60 min around the targeted cortical area in the M1, suggesting a degree of spatial localization. However, cortical excitability measures did not indicate changes after either M1 or PFC BCI-FES.Significance. Neural mechanisms underlying the effectiveness of BCI-FES neuromodulation may be attributed to the M1 direct corticospinal projections and/or the closer timing between ERD detection and FES, which likely enhanced Hebbian-like plasticity by synchronizing cortical activation detected by the BCI system with the sensory nerve activation and movement related reafference elicited by FES.
Collapse
Affiliation(s)
- Rizaldi A Fadli
- Graduate School of Engineering Science, Department of Mechanical Science and Bioengineering, Osaka University, 1-3 Machikaneyama, Toyonaka 560-8531, Japan
- Department of Biomedical Engineering, University of Miami College of Engineering, 1251 Memorial Drive, Coral Gables, FL 33146, United States of America
- The Miami Project to Cure Paralysis, Department of Neurological Surgery, University of Miami Miller School of Medicine, 1095 NW 14th Terrace, Miami, FL 33136, United States of America
| | - Yuki Yamanouchi
- Graduate School of Engineering Science, Department of Mechanical Science and Bioengineering, Osaka University, 1-3 Machikaneyama, Toyonaka 560-8531, Japan
| | - Lazar I Jovanovic
- Institute of Biomedical Engineering, University of Toronto, 164 College Street, Toronto, Ontario M5S 3G9, Canada
- KITE Research Institute, Toronto Rehabilitation Institute-University Health Network, 520 Sutherland Drive, Toronto, Ontario M4G 3V9, Canada
| | - Milos R Popovic
- Institute of Biomedical Engineering, University of Toronto, 164 College Street, Toronto, Ontario M5S 3G9, Canada
- KITE Research Institute, Toronto Rehabilitation Institute-University Health Network, 520 Sutherland Drive, Toronto, Ontario M4G 3V9, Canada
- CRANIA, University Health Network & University of Toronto. 550 University Avenue, Toronto, Ontario M5G 2A2, Canada
| | - Cesar Marquez-Chin
- Institute of Biomedical Engineering, University of Toronto, 164 College Street, Toronto, Ontario M5S 3G9, Canada
- KITE Research Institute, Toronto Rehabilitation Institute-University Health Network, 520 Sutherland Drive, Toronto, Ontario M4G 3V9, Canada
- CRANIA, University Health Network & University of Toronto. 550 University Avenue, Toronto, Ontario M5G 2A2, Canada
| | - Taishin Nomura
- Graduate School of Engineering Science, Department of Mechanical Science and Bioengineering, Osaka University, 1-3 Machikaneyama, Toyonaka 560-8531, Japan
| | - Matija Milosevic
- Graduate School of Engineering Science, Department of Mechanical Science and Bioengineering, Osaka University, 1-3 Machikaneyama, Toyonaka 560-8531, Japan
- Department of Biomedical Engineering, University of Miami College of Engineering, 1251 Memorial Drive, Coral Gables, FL 33146, United States of America
- The Miami Project to Cure Paralysis, Department of Neurological Surgery, University of Miami Miller School of Medicine, 1095 NW 14th Terrace, Miami, FL 33136, United States of America
| |
Collapse
|
5
|
Kodama M, Iwama S, Morishige M, Ushiba J. Thirty-minute motor imagery exercise aided by EEG sensorimotor rhythm neurofeedback enhances morphing of sensorimotor cortices: a double-blind sham-controlled study. Cereb Cortex 2023:6967448. [PMID: 36600612 DOI: 10.1093/cercor/bhac525] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 12/15/2022] [Accepted: 12/16/2022] [Indexed: 01/06/2023] Open
Abstract
Neurofeedback training using electroencephalogram (EEG)-based brain-computer interfaces (BCIs) combined with mental rehearsals of motor behavior has demonstrated successful self-regulation of motor cortical excitability. However, it remains unclear whether the acquisition of skills to voluntarily control neural excitability is accompanied by structural plasticity boosted by neurofeedback. Here, we sought short-term changes in cortical structures induced by 30 min of BCI-based neurofeedback training, which aimed at the regulation of sensorimotor rhythm (SMR) in scalp EEG. When participants performed kinesthetic motor imagery of right finger movement with online feedback of either event-related desynchronisation (ERD) of SMR magnitude from the contralateral sensorimotor cortex (SM1) or those from other participants (i.e. placebo), the learning rate of SMR-ERD control was significantly different. Although overlapped structural changes in gray matter volumes were found in both groups, significant differences revealed by group-by-group comparison were spatially different; whereas the veritable neurofeedback group exhibited sensorimotor area-specific changes, the placebo exhibited spatially distributed changes. The white matter change indicated a significant decrease in the corpus callosum in the verum group. Furthermore, the learning rate of SMR regulation was correlated with the volume changes in the ipsilateral SM1, suggesting the involvement of interhemispheric motor control circuitries in BCI control tasks.
Collapse
Affiliation(s)
- Midori Kodama
- Graduate School of Science and Technology, Keio University, Kanagawa 108-0073, Japan
| | - Seitaro Iwama
- Graduate School of Science and Technology, Keio University, Kanagawa 108-0073, Japan.,Japan Society for the Promotion of Science, Tokyo 102-0082, Japan
| | - Masumi Morishige
- Graduate School of Science and Technology, Keio University, Kanagawa 108-0073, Japan
| | - Junichi Ushiba
- Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Kanagawa 108-0073, Japan
| |
Collapse
|
6
|
Belkacem AN, Jamil N, Khalid S, Alnajjar F. On closed-loop brain stimulation systems for improving the quality of life of patients with neurological disorders. Front Hum Neurosci 2023; 17:1085173. [PMID: 37033911 PMCID: PMC10076878 DOI: 10.3389/fnhum.2023.1085173] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 03/06/2023] [Indexed: 04/11/2023] Open
Abstract
Emerging brain technologies have significantly transformed human life in recent decades. For instance, the closed-loop brain-computer interface (BCI) is an advanced software-hardware system that interprets electrical signals from neurons, allowing communication with and control of the environment. The system then transmits these signals as controlled commands and provides feedback to the brain to execute specific tasks. This paper analyzes and presents the latest research on closed-loop BCI that utilizes electric/magnetic stimulation, optogenetic, and sonogenetic techniques. These techniques have demonstrated great potential in improving the quality of life for patients suffering from neurodegenerative or psychiatric diseases. We provide a comprehensive and systematic review of research on the modalities of closed-loop BCI in recent decades. To achieve this, the authors used a set of defined criteria to shortlist studies from well-known research databases into categories of brain stimulation techniques. These categories include deep brain stimulation, transcranial magnetic stimulation, transcranial direct-current stimulation, transcranial alternating-current stimulation, and optogenetics. These techniques have been useful in treating a wide range of disorders, such as Alzheimer's and Parkinson's disease, dementia, and depression. In total, 76 studies were shortlisted and analyzed to illustrate how closed-loop BCI can considerably improve, enhance, and restore specific brain functions. The analysis revealed that literature in the area has not adequately covered closed-loop BCI in the context of cognitive neural prosthetics and implanted neural devices. However, the authors demonstrate that the applications of closed-loop BCI are highly beneficial, and the technology is continually evolving to improve the lives of individuals with various ailments, including those with sensory-motor issues or cognitive deficiencies. By utilizing emerging techniques of stimulation, closed-loop BCI can safely improve patients' cognitive and affective skills, resulting in better healthcare outcomes.
Collapse
Affiliation(s)
- Abdelkader Nasreddine Belkacem
- Department of Computer and Network Engineering, College of Information Technology, UAE University, Al-Ain, United Arab Emirates
- *Correspondence: Abdelkader Nasreddine Belkacem
| | - Nuraini Jamil
- Department of Computer Science and Software Engineering, College of Information Technology, UAE University, Al-Ain, United Arab Emirates
| | - Sumayya Khalid
- Department of Computer Science and Software Engineering, College of Information Technology, UAE University, Al-Ain, United Arab Emirates
| | - Fady Alnajjar
- Department of Computer Science and Software Engineering, College of Information Technology, UAE University, Al-Ain, United Arab Emirates
- Center for Brain Science, RIKEN, Saitama, Japan
- Fady Alnajjar
| |
Collapse
|