1
|
Ottenhoff MC, Verwoert M, Goulis S, Wagner L, van Dijk JP, Kubben PL, Herff C. Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings. J Neural Eng 2024; 21:056034. [PMID: 39383883 DOI: 10.1088/1741-2552/ad851c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Accepted: 10/09/2024] [Indexed: 10/11/2024]
Abstract
Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.
Collapse
Affiliation(s)
- Maarten C Ottenhoff
- Department of Neurosurgery, Mental Health and Neuroscience Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Maxime Verwoert
- Department of Neurosurgery, Mental Health and Neuroscience Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Sophocles Goulis
- Department of Neurosurgery, Mental Health and Neuroscience Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Louis Wagner
- Academic Center of Epileptology Kempenhaeghe/Maastricht University Medical Center, Maastricht, The Netherlands
- Academic Center of Epileptology Kempenhaeghe/Maastricht University Medical Center, Heeze, The Netherlands
| | - Johannes P van Dijk
- Academic Center of Epileptology Kempenhaeghe/Maastricht University Medical Center, Heeze, The Netherlands
- Department of Orthodontics, Ulm University, Ulm, Germany
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Pieter L Kubben
- Department of Neurosurgery, Mental Health and Neuroscience Research Institute, Maastricht University, Maastricht, The Netherlands
- Academic Center of Epileptology Kempenhaeghe/Maastricht University Medical Center, Maastricht, The Netherlands
| | - Christian Herff
- Department of Neurosurgery, Mental Health and Neuroscience Research Institute, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
2
|
Kılınç Bülbül D, Güçlü B. Predicting lever press in a vibrotactile yes/no detection task from S1 cortex of freely behaving rats by µECoG arrays. Somatosens Mot Res 2024:1-8. [PMID: 38812257 DOI: 10.1080/08990220.2024.2358522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Accepted: 05/18/2024] [Indexed: 05/31/2024]
Abstract
AIM OF THE STUDY Brain-computer interfaces (BCIs) may help patients with severe neurological deficits communicate with the external world. Based on microelectrocorticography (µECoG) data recorded from the primary somatosensory cortex (S1) of unrestrained behaving rats, this study attempts to decode lever presses in a psychophysical detection task by using machine learning algorithms. MATERIALS AND METHODS 16-channel Pt-Ir microelectrode arrays were implanted on the S1 of two rats, and µECoG was recorded during a vibrotactile yes/no detection task. For this task, the rats were trained to press the right lever when they detected the vibrotactile stimulus and the left lever when they did not. The multichannel µECoG data was analysed offline by time-frequency methods and its features were used for binary classification of the lever press at each trial. Several machine learning algorithms were tested as such. RESULTS The psychophysical sensitivities (A') were similar and low for both rats (0.58). Rat 2 (B'': -0.11) had higher bias for the right lever than Rat 1 (B'': - 0.01). The lever presses could be predicted with accuracies over 66% with all the tested algorithms, and the highest average accuracy (78%) was with the support vector machine. CONCLUSION According to the recent studies, sensory feedback increases the benefit of the BCIs. The current proof-of-concept study shows that lever presses can be decoded from the S1; therefore, this area may be utilised for a bidirectional BCI in the future.
Collapse
Affiliation(s)
| | - Burak Güçlü
- Institute of Biomedical Engineering, Boğaziçi University, İstanbul, Turkey
| |
Collapse
|
3
|
Combrisson E, Di Rienzo F, Saive AL, Perrone-Bertolotti M, Soto JLP, Kahane P, Lachaux JP, Guillot A, Jerbi K. Human local field potentials in motor and non-motor brain areas encode upcoming movement direction. Commun Biol 2024; 7:506. [PMID: 38678058 PMCID: PMC11055917 DOI: 10.1038/s42003-024-06151-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 04/05/2024] [Indexed: 04/29/2024] Open
Abstract
Limb movement direction can be inferred from local field potentials in motor cortex during movement execution. Yet, it remains unclear to what extent intended hand movements can be predicted from brain activity recorded during movement planning. Here, we set out to probe the directional-tuning of oscillatory features during motor planning and execution, using a machine learning framework on multi-site local field potentials (LFPs) in humans. We recorded intracranial EEG data from implanted epilepsy patients as they performed a four-direction delayed center-out motor task. Fronto-parietal LFP low-frequency power predicted hand-movement direction during planning while execution was largely mediated by higher frequency power and low-frequency phase in motor areas. By contrast, Phase-Amplitude Coupling showed uniform modulations across directions. Finally, multivariate classification led to an increase in overall decoding accuracy (>80%). The novel insights revealed here extend our understanding of the role of neural oscillations in encoding motor plans.
Collapse
Affiliation(s)
- Etienne Combrisson
- Psychology Department, University of Montreal, Montreal, QC, Canada.
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France.
- Institut de Neurosciences de la Timone, Aix Marseille Université, UMR 7289 CNRS, 13005, Marseille, France.
| | - Franck Di Rienzo
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France
| | - Anne-Lise Saive
- Psychology Department, University of Montreal, Montreal, QC, Canada
- Cognitive Science Department, Lyfe Research and Innovation Center, Ecully, France
| | | | - Juan L P Soto
- Telecommunications and Control Engineering Department, University of Sao Paulo, Sao Paulo, Brazil
| | - Philippe Kahane
- Université Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, GIN, Grenoble, France
| | - Jean-Philippe Lachaux
- Lyon Neuroscience Research Center, EDUWELL team, INSERM UMRS 1028, CNRS UMR 5292, Université Claude Bernard Lyon 1, Université de Lyon, F-69000, Lyon, France
| | - Aymeric Guillot
- University of Lyon, UCBL-Lyon 1, Laboratoire Interuniversitaire de Biologie de la Motricité UR 7424, F-69622, Villeurbanne, France
| | - Karim Jerbi
- Psychology Department, University of Montreal, Montreal, QC, Canada.
- Mila (Quebec AI Institute), montreal, QC, Canada.
- UNIQUE Centre (Quebec Neuro-AI research Center), Montreal, QC, Canada.
| |
Collapse
|
4
|
Merrick CM, Doyle ON, Gallegos NE, Irwin ZT, Olson JW, Gonzalez CL, Knight RT, Ivry RB, Walker HC. Differential contribution of sensorimotor cortex and subthalamic nucleus to unimanual and bimanual hand movements. Cereb Cortex 2024; 34:bhad492. [PMID: 38124548 PMCID: PMC10793582 DOI: 10.1093/cercor/bhad492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 11/18/2023] [Accepted: 11/19/2023] [Indexed: 12/23/2023] Open
Abstract
Why does unilateral deep brain stimulation improve motor function bilaterally? To address this clinical observation, we collected parallel neural recordings from sensorimotor cortex (SMC) and the subthalamic nucleus (STN) during repetitive ipsilateral, contralateral, and bilateral hand movements in patients with Parkinson's disease. We used a cross-validated electrode-wise encoding model to map electromyography data to the neural signals. Electrodes in the STN encoded movement at a comparable level for both hands, whereas SMC electrodes displayed a strong contralateral bias. To examine representational overlap across the two hands, we trained the model with data from one condition (contralateral hand) and used the trained weights to predict neural activity for movements produced with the other hand (ipsilateral hand). Overall, between-hand generalization was poor, and this limitation was evident in both regions. A similar method was used to probe representational overlap across different task contexts (unimanual vs. bimanual). Task context was more important for the STN compared to the SMC indicating that neural activity in the STN showed greater divergence between the unimanual and bimanual conditions. These results indicate that SMC activity is strongly lateralized and relatively context-free, whereas the STN integrates contextual information with the ongoing behavior.
Collapse
Affiliation(s)
- Christina M Merrick
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
| | - Owen N Doyle
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, United States
| | - Natali E Gallegos
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, United States
| | - Zachary T Irwin
- Department of Neurosurgery, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Joseph W Olson
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Christopher L Gonzalez
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| | - Robert T Knight
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720, United States
| | - Richard B Ivry
- Department of Psychology, University of California Berkeley, Berkeley, CA 94720, United States
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720, United States
| | - Harrison C Walker
- Department of Neurosurgery, University of Alabama at Birmingham, Birmingham, AL 35294, United States
- Department of Neurology, University of Alabama at Birmingham, Birmingham, AL 35294, United States
- Department of Biomedical Engineering, University of Alabama at Birmingham, Birmingham, AL 35294, United States
| |
Collapse
|
5
|
Tatti E, Cacciola A, Carrara F, Luciani A, Quartarone A, Ghilardi MF. Movement-related ERS and connectivity in the gamma frequency decrease with practice. Neuroimage 2023; 284:120444. [PMID: 37926216 PMCID: PMC10758293 DOI: 10.1016/j.neuroimage.2023.120444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 10/30/2023] [Accepted: 11/03/2023] [Indexed: 11/07/2023] Open
Abstract
Previous work showed that movements are accompanied by modulation of electroencephalographic (EEG) activity in both beta (13-30 Hz) and gamma (>30 Hz) ranges. The amplitude of beta event-related synchronization (ERS) is not linked to movement characteristics, but progressively increases with motor practice, returning to baseline after a period of rest. Conversely, movement-related gamma ERS amplitude is proportional to movement distance and velocity. Here, high-density EEG was recorded in 51 healthy subjects to investigate whether i) three-hour practice in two learning tasks, one with a motor component and one without, affects gamma ERS amplitude and connectivity during a motor reaching test, and ii) 90-minutes of either sleep or quiet rest have an effect on gamma oscillatory activity. We found that, while gamma ERS was appropriately scaled to the target extent at all testing points, its amplitude decreased after practice, independently of the type of interposed learning, and after both quiet wake and nap, with partial correlations with subjective fatigue scores. During movement execution, connectivity patterns within fronto-parieto-occipital electrodes, over areas associated with attentional networks, decreased after both practice and after 90-minute rest. While confirming the prokinetic nature of movement-related gamma ERS, these findings demonstrated the preservation of gamma ERS scaling to movement velocity with practice, despite constant amplitude reduction. We thus speculate that such decreases, differently from the practice-related increases of beta ERS, are related to reduced attention or working memory mechanisms due to fatigue or a switch of strategy toward automatization of movement execution and do not specifically reflect plasticity phenomena.
Collapse
Affiliation(s)
- Elisa Tatti
- Department of Molecular, Cellular & Biomedical Sciences, CUNY, School of Medicine, New York, NY 10031, United States.
| | - Alberto Cacciola
- Brain Mapping Lab, Department of Biomedical, Dental Sciences and Morphological and Functional Images, University of Messina, Messina, Italy; Center for Complex Network Intelligence (CCNI), Tsinghua Laboratory of Brain and Intelligence (THBI), Tsinghua University, Beijing, China; Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Federico Carrara
- Department of Mathematics, Polytechnic University of Milan, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Adalgisa Luciani
- Department of Molecular, Cellular & Biomedical Sciences, CUNY, School of Medicine, New York, NY 10031, United States; Section of Psychiatry, Department of Neuroscience, School of Medicine, University of Naples "Federico II", Naples, Italy
| | - Angelo Quartarone
- IRCCS-Centro Neurolesi Bonino-Pulejo, S.S. 113, Via Palermo, C. da Casazza, 98124 Messina, Italy.
| | - M Felice Ghilardi
- Department of Molecular, Cellular & Biomedical Sciences, CUNY, School of Medicine, New York, NY 10031, United States.
| |
Collapse
|
6
|
Merino EC, Faes A, Van Hulle MM. The role of distinct ECoG frequency features in decoding finger movement. J Neural Eng 2023; 20:066014. [PMID: 37963397 DOI: 10.1088/1741-2552/ad0c5e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 11/14/2023] [Indexed: 11/16/2023]
Abstract
Objective.To identify the electrocorticography (ECoG) frequency features that encode distinct finger movement states during repeated finger flexions.Approach.We used the publicly available Stanford ECoG dataset of cue-based, repeated single finger flexions. Using linear regression, we identified the spectral features that contributed most to the encoding of movement dynamics and discriminating movement events from rest, and combined them to predict finger movement trajectories. Furthermore, we also looked into the effect of the used frequency range and the spatial distribution of the identified features.Main results.Two frequency features generate superior performance, each one for a different movement aspect: high gamma band activity distinguishes movement events from rest, whereas the local motor potential (LMP) codes for movement dynamics. Combining these two features in a finger movement decoder outperformed comparable prior work where the entire spectrum was used as the average correlation coefficient with the true trajectories increased from 0.45 to 0.5, both applied to the Stanford dataset, and erroneous predictions during rest were demoted. In addition, for the first time, our results show the influence of the upper cut-off frequency used to extract LMP, yielding a higher performance when this range is adjusted to the finger movement rate.Significance.This study shows the benefit of a detailed feature analysis prior to designing the finger movement decoder.
Collapse
Affiliation(s)
- Eva Calvo Merino
- Laboratory for Neuro- and Psychophysiology, KU Leuven, Leuven, Belgium
| | - A Faes
- Laboratory for Neuro- and Psychophysiology, KU Leuven, Leuven, Belgium
| | - M M Van Hulle
- Laboratory for Neuro- and Psychophysiology, KU Leuven, Leuven, Belgium
| |
Collapse
|
7
|
Wang HL, Kuo YT, Lo YC, Kuo CH, Chen BW, Wang CF, Wu ZY, Lee CE, Yang SH, Lin SH, Chen PC, Chen YY. Enhancing Prediction of Forelimb Movement Trajectory through a Calibrating-Feedback Paradigm Incorporating RAT Primary Motor and Agranular Cortical Ensemble Activity in the Goal-Directed Reaching Task. Int J Neural Syst 2023; 33:2350051. [PMID: 37632142 DOI: 10.1142/s012906572350051x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/27/2023]
Abstract
Complete reaching movements involve target sensing, motor planning, and arm movement execution, and this process requires the integration and communication of various brain regions. Previously, reaching movements have been decoded successfully from the motor cortex (M1) and applied to prosthetic control. However, most studies attempted to decode neural activities from a single brain region, resulting in reduced decoding accuracy during visually guided reaching motions. To enhance the decoding accuracy of visually guided forelimb reaching movements, we propose a parallel computing neural network using both M1 and medial agranular cortex (AGm) neural activities of rats to predict forelimb-reaching movements. The proposed network decodes M1 neural activities into the primary components of the forelimb movement and decodes AGm neural activities into internal feedforward information to calibrate the forelimb movement in a goal-reaching movement. We demonstrate that using AGm neural activity to calibrate M1 predicted forelimb movement can improve decoding performance significantly compared to neural decoders without calibration. We also show that the M1 and AGm neural activities contribute to controlling forelimb movement during goal-reaching movements, and we report an increase in the power of the local field potential (LFP) in beta and gamma bands over AGm in response to a change in the target distance, which may involve sensorimotor transformation and communication between the visual cortex and AGm when preparing for an upcoming reaching movement. The proposed parallel computing neural network with the internal feedback model improves prediction accuracy for goal-reaching movements.
Collapse
Affiliation(s)
- Han-Lin Wang
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Yun-Ting Kuo
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Yu-Chun Lo
- The Ph.D. Program in Medical Neuroscience, College of Medical Science and Technology, Taipei Medical University, 12F., Education & Research Building, Shuang-Ho Campus, No. 301, Yuantong Rd., New Taipei City 235235, Taiwan
| | - Chao-Hung Kuo
- Department of Neurosurgery, Neurological Institute Taipei Veterans General Hospital, No. 201, Sec. 2 Shipai Rd., Taipei 11217, Taiwan
| | - Bo-Wei Chen
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Ching-Fu Wang
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
- Biomedical Engineering Research and Development Center, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Zu-Yu Wu
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Chi-En Lee
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
| | - Shih-Hung Yang
- Department of Mechanical Engineering, National Cheng Kung University, No. 1, University Rd., Tainan 70101, Taiwan
| | - Sheng-Huang Lin
- Department of Neurology, Hualien Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 707, Sec. 3 Zhongyang Rd., Hualien 97002, Taiwan
- Department of Neurology, School of Medicine, Tzu Chi University, No. 701, Sec. 3, Zhongyang Rd., Hualien 97004, Taiwan
| | - Po-Chuan Chen
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
| | - You-Yin Chen
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, No. 155, Sec. 2 Linong St., Taipei 112304, Taiwan
- The Ph.D. Program in Medical Neuroscience, College of Medical Science and Technology, Taipei Medical University, 12F., Education & Research Building, Shuang-Ho Campus, No. 301, Yuantong Rd., New Taipei City 235235, Taiwan
| |
Collapse
|
8
|
Sutter C, Moinon A, Felicetti L, Massi F, Blouin J, Mouchnino L. Cortical facilitation of tactile afferents during the preparation of a body weight transfer when standing on a biomimetic surface. Front Neurol 2023; 14:1175667. [PMID: 37404946 PMCID: PMC10315651 DOI: 10.3389/fneur.2023.1175667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 05/30/2023] [Indexed: 07/06/2023] Open
Abstract
Self-generated movement shapes tactile perception, but few studies have investigated the brain mechanisms involved in the processing of the mechanical signals related to the static and transient skin deformations generated by forces and pressures exerted between the foot skin and the standing surface. We recently found that standing on a biomimetic surface (i.e., inspired by the characteristics of mechanoreceptors and skin dermatoglyphics), that magnified skin-surface interaction, increased the sensory flow to the somatosensory cortex and improved balance control compared to standing on control (e.g., smooth) surfaces. In this study, we tested whether the well-known sensory suppression that occurs during movements is alleviated when the tactile afferent signal becomes relevant with the use of a biomimetic surface. Eyes-closed participants (n = 25) self-stimulated their foot cutaneous receptors by shifting their body weight toward one of their legs while standing on either a biomimetic or a control (smooth) surface. In a control task, similar forces were exerted on the surfaces (i.e., similar skin-surface interaction) by passive translations of the surfaces. Sensory gating was assessed by measuring the amplitude of the somatosensory-evoked potential over the vertex (SEP, recorded by EEG). Significantly larger and shorter SEPs were found when participants stood on the biomimetic surface. This was observed whether the forces exerted on the surface were self-generated or passively generated. Contrary to our prediction, we found that the sensory attenuation related to the self-generated movement did not significantly differ between the biomimetic and control surfaces. However, we observed an increase in gamma activity (30-50 Hz) over centroparietal regions during the preparation phase of the weight shift only when participants stood on the biomimetic surface. This result might suggest that gamma-band oscillations play an important functional role in processing behaviorally relevant stimuli during the early stages of body weight transfer.
Collapse
Affiliation(s)
- Chloé Sutter
- Laboratoire de Neurosciences Cognitives, FR 3C, CNRS, Aix Marseille Université, Marseille, France
| | - Alix Moinon
- Laboratoire de Neurosciences Cognitives, FR 3C, CNRS, Aix Marseille Université, Marseille, France
| | - Livia Felicetti
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Rome, Italy
- LAMCOS, INSA Lyon, CNRS, UMR5259, Université Lyon, Villeurbanne, France
| | - Francesco Massi
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Rome, Italy
| | - Jean Blouin
- Laboratoire de Neurosciences Cognitives, FR 3C, CNRS, Aix Marseille Université, Marseille, France
| | - Laurence Mouchnino
- Laboratoire de Neurosciences Cognitives, FR 3C, CNRS, Aix Marseille Université, Marseille, France
- Institut Universitaire de France, Paris, France
| |
Collapse
|
9
|
Lin J, Lai D, Wan Z, Feng L, Zhu J, Zhang J, Wang Y, Xu K. Representation and decoding of bilateral arm motor imagery using unilateral cerebral LFP signals. Front Hum Neurosci 2023; 17:1168017. [PMID: 37388414 PMCID: PMC10304012 DOI: 10.3389/fnhum.2023.1168017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Accepted: 05/24/2023] [Indexed: 07/01/2023] Open
Abstract
Introduction In the field of upper limb brain computer interfaces (BCIs), the research focusing on bilateral decoding mostly based on the neural signals from two cerebral hemispheres. In addition, most studies used spikes for decoding. Here we examined the representation and decoding of different laterality and regions arm motor imagery in unilateral motor cortex based on local field potentials (LFPs). Methods The LFP signals were recorded from a 96-channel Utah microelectrode array implanted in the left primary motor cortex of a paralyzed participant. There were 7 kinds of tasks: rest, left, right and bilateral elbow and wrist flexion. We performed time-frequency analysis on the LFP signals and analyzed the representation and decoding of different tasks using the power and energy of different frequency bands. Results The frequency range of <8 Hz and >38 Hz showed power enhancement, whereas 8-38 Hz showed power suppression in spectrograms while performing motor imagery. There were significant differences in average energy between tasks. What's more, the movement region and laterality were represented in two dimensions by demixed principal component analysis. The 135-300 Hz band signal had the highest decoding accuracy among all frequency bands and the contralateral and bilateral signals had more similar single-channel power activation patterns and larger signal correlation than contralateral and ipsilateral signals, bilateral and ipsilateral signals. Discussion The results showed that unilateral LFP signals had different representations for bilateral motor imagery on the average energy of the full array and single-channel power levels, and different tasks could be decoded. These proved the feasibility of multilateral BCI based on the unilateral LFP signal to broaden the application of BCI technology. Clinical trial registration https://www.chictr.org.cn/showproj.aspx?proj=130829, identifier ChiCTR2100050705.
Collapse
Affiliation(s)
- Jiafan Lin
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory for Biomedical Engineering of Ministry of Education, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
| | - Dongrong Lai
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory for Biomedical Engineering of Ministry of Education, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
| | - Zijun Wan
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory for Biomedical Engineering of Ministry of Education, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
| | | | - Junming Zhu
- Department of Neurosurgery, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
- State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
| | - Jianmin Zhang
- Department of Neurosurgery, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
- State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
| | - Yueming Wang
- State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
| | - Kedi Xu
- Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
- Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Key Laboratory for Biomedical Engineering of Ministry of Education, School of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
- State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou, China
| |
Collapse
|
10
|
Merk T, Peterson V, Lipski WJ, Blankertz B, Turner RS, Li N, Horn A, Richardson RM, Neumann WJ. Electrocorticography is superior to subthalamic local field potentials for movement decoding in Parkinson's disease. eLife 2022; 11:e75126. [PMID: 35621994 PMCID: PMC9142148 DOI: 10.7554/elife.75126] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 05/15/2022] [Indexed: 01/07/2023] Open
Abstract
Brain signal decoding promises significant advances in the development of clinical brain computer interfaces (BCI). In Parkinson's disease (PD), first bidirectional BCI implants for adaptive deep brain stimulation (DBS) are now available. Brain signal decoding can extend the clinical utility of adaptive DBS but the impact of neural source, computational methods and PD pathophysiology on decoding performance are unknown. This represents an unmet need for the development of future neurotechnology. To address this, we developed an invasive brain-signal decoding approach based on intraoperative sensorimotor electrocorticography (ECoG) and subthalamic LFP to predict grip-force, a representative movement decoding application, in 11 PD patients undergoing DBS. We demonstrate that ECoG is superior to subthalamic LFP for accurate grip-force decoding. Gradient boosted decision trees (XGBOOST) outperformed other model architectures. ECoG based decoding performance negatively correlated with motor impairment, which could be attributed to subthalamic beta bursts in the motor preparation and movement period. This highlights the impact of PD pathophysiology on the neural capacity to encode movement vigor. Finally, we developed a connectomic analysis that could predict grip-force decoding performance of individual ECoG channels across patients by using their connectomic fingerprints. Our study provides a neurophysiological and computational framework for invasive brain signal decoding to aid the development of an individualized precision-medicine approach to intelligent adaptive DBS.
Collapse
Affiliation(s)
- Timon Merk
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu BerlinBerlinGermany
| | - Victoria Peterson
- Brain Modulation Lab, Department of Neurosurgery, Massachusetts General HospitalBostonUnited States
- Harvard Medical SchoolBostonUnited States
| | - Witold J Lipski
- Department of Neurobiology, University of PittsburghPittsburghUnited States
| | - Benjamin Blankertz
- Department of Computer Science, Technische Universität BerlnBerlinGermany
| | - Robert S Turner
- Department of Neurobiology, University of PittsburghPittsburghUnited States
| | - Ningfei Li
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu BerlinBerlinGermany
| | - Andreas Horn
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu BerlinBerlinGermany
| | - Robert Mark Richardson
- Brain Modulation Lab, Department of Neurosurgery, Massachusetts General HospitalBostonUnited States
- Harvard Medical SchoolBostonUnited States
| | - Wolf-Julian Neumann
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität zu BerlinBerlinGermany
| |
Collapse
|
11
|
Śliwowski M, Martin M, Souloumiac A, Blanchart P, Aksenova T. Decoding ECoG signal into 3D hand translation using deep learning. J Neural Eng 2022; 19. [PMID: 35287119 DOI: 10.1088/1741-2552/ac5d69] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 03/14/2022] [Indexed: 12/29/2022]
Abstract
Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.
Collapse
Affiliation(s)
- Maciej Śliwowski
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France.,Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France
| | - Matthieu Martin
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| | | | | | - Tetiana Aksenova
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| |
Collapse
|
12
|
Tatti E, Ferraioli F, Cacciola A, Chan C, Quartarone A, Ghilardi MF. Modulation of Gamma Spectral Amplitude and Connectivity During Reaching Predicts Peak Velocity and Movement Duration. Front Neurosci 2022; 16:836703. [PMID: 35281507 PMCID: PMC8908429 DOI: 10.3389/fnins.2022.836703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 02/02/2022] [Indexed: 11/21/2022] Open
Abstract
Modulation of gamma oscillations recorded from the human motor cortex and basal ganglia appears to play a key role in movement execution. However, there are still major questions to be answered about the specific role of cortical gamma activity in both the planning and execution of movement features such as the scaling of peak velocity and movement time. In this study, we characterized movement-related gamma oscillatory dynamics and its relationship with kinematic parameters based on 256-channels EEG recordings in 64 healthy subjects while performing fast and uncorrected reaching movements to targets located at three distances. In keeping with previous studies, we found that movement-related gamma synchronization occurred during movement execution. As a new finding, we showed that gamma synchronization occurred also before movement onset, with planning and execution phases involving different gamma peak frequencies and topographies. Importantly, the amplitude of gamma synchronization in both planning and execution increased with target distance and predicted peak velocity and movement time. Additional analysis of phase coherence revealed a gamma-coordinated long-range network involving occipital, frontal and central regions during movement execution that was positively related to kinematic features. This is the first evidence in humans supporting the notion that gamma synchronization amplitude and phase coherence pattern can reliably predict peak velocity amplitude and movement time. Therefore, these findings suggest that cortical gamma oscillations have a crucial role for the selection, implementation and control of the appropriate kinematic parameters of goal-directed reaching movements.
Collapse
Affiliation(s)
- Elisa Tatti
- Department of Molecular, Cellular and Biomedical Sciences, City University of New York (CUNY), School of Medicine, New York, NY, United States
- *Correspondence: Elisa Tatti,
| | - Francesca Ferraioli
- Department of Molecular, Cellular and Biomedical Sciences, City University of New York (CUNY), School of Medicine, New York, NY, United States
| | - Alberto Cacciola
- Department of Biomedical, Dental Sciences and Morphological and Functional Images, University of Messina, Messina, Italy
| | - Cameron Chan
- Department of Molecular, Cellular and Biomedical Sciences, City University of New York (CUNY), School of Medicine, New York, NY, United States
| | - Angelo Quartarone
- Department of Biomedical, Dental Sciences and Morphological and Functional Images, University of Messina, Messina, Italy
| | - Maria Felice Ghilardi
- Department of Molecular, Cellular and Biomedical Sciences, City University of New York (CUNY), School of Medicine, New York, NY, United States
- Maria Felice Ghilardi,
| |
Collapse
|
13
|
Inferring entire spiking activity from local field potentials. Sci Rep 2021; 11:19045. [PMID: 34561480 PMCID: PMC8463692 DOI: 10.1038/s41598-021-98021-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2020] [Accepted: 09/01/2021] [Indexed: 11/29/2022] Open
Abstract
Extracellular recordings are typically analysed by separating them into two distinct signals: local field potentials (LFPs) and spikes. Previous studies have shown that spikes, in the form of single-unit activity (SUA) or multiunit activity (MUA), can be inferred solely from LFPs with moderately good accuracy. SUA and MUA are typically extracted via threshold-based technique which may not be reliable when the recordings exhibit a low signal-to-noise ratio (SNR). Another type of spiking activity, referred to as entire spiking activity (ESA), can be extracted by a threshold-less, fast, and automated technique and has led to better performance in several tasks. However, its relationship with the LFPs has not been investigated. In this study, we aim to address this issue by inferring ESA from LFPs intracortically recorded from the motor cortex area of three monkeys performing different tasks. Results from long-term recording sessions and across subjects revealed that ESA can be inferred from LFPs with good accuracy. On average, the inference performance of ESA was consistently and significantly higher than those of SUA and MUA. In addition, local motor potential (LMP) was found to be the most predictive feature. The overall results indicate that LFPs contain substantial information about spiking activity, particularly ESA. This could be useful for understanding LFP-spike relationship and for the development of LFP-based BMIs.
Collapse
|
14
|
Gatti R, Atum Y, Schiaffino L, Jochumsen M, Biurrun Manresa J. Decoding kinetic features of hand motor preparation from single-trial EEG using convolutional neural networks. Eur J Neurosci 2020; 53:556-570. [PMID: 32781497 DOI: 10.1111/ejn.14936] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Revised: 07/17/2020] [Accepted: 08/03/2020] [Indexed: 11/28/2022]
Abstract
Building accurate movement decoding models from brain signals is crucial for many biomedical applications. Predicting specific movement features, such as speed and force, before movement execution may provide additional useful information at the expense of increasing the complexity of the decoding problem. Recent attempts to predict movement speed and force from the electroencephalogram (EEG) achieved classification accuracies at or slightly above chance levels, highlighting the need for more accurate prediction strategies. Thus, the aims of this study were to accurately predict hand movement speed and force from single-trial EEG signals and to decode neurophysiological information of motor preparation from the prediction strategies. To these ends, a decoding model based on convolutional neural networks (ConvNets) was implemented and compared against other state-of-the-art prediction strategies, such as support vector machines and decision trees. ConvNets outperformed the other prediction strategies, achieving an overall accuracy of 84% in the classification of two different levels of speed and force (four-class classification) from pre-movement single-trial EEG (100 ms and up to 1,600 ms prior to movement execution). Furthermore, an analysis of the ConvNet architectures suggests that the network performs a complex spatiotemporal integration of EEG data to optimize classification accuracy. These results show that movement speed and force can be accurately predicted from single-trial EEG, and that the prediction strategies may provide useful neurophysiological information about motor preparation.
Collapse
Affiliation(s)
- Ramiro Gatti
- Institute for Research and Development in Bioengineering and Bioinformatics (IBB), CONICET-UNER, Oro Verde, Argentina.,Laboratory for Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina
| | - Yanina Atum
- Laboratory for Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina
| | - Luciano Schiaffino
- Laboratory for Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina
| | - Mads Jochumsen
- Center for Sensory-Motor Interaction (SMI®), Aalborg University, Aalborg, Denmark
| | - José Biurrun Manresa
- Institute for Research and Development in Bioengineering and Bioinformatics (IBB), CONICET-UNER, Oro Verde, Argentina.,Laboratory for Rehabilitation Engineering and Neuromuscular and Sensory Research (LIRINS), Faculty of Engineering, National University of Entre Ríos, Oro Verde, Argentina.,Center for Neuroplasticity and Pain (CNAP), Aalborg University, Aalborg, Denmark
| |
Collapse
|
15
|
Madhavan R, Bansal AK, Madsen JR, Golby AJ, Tierney TS, Eskandar EN, Anderson WS, Kreiman G. Neural Interactions Underlying Visuomotor Associations in the Human Brain. Cereb Cortex 2019; 29:4551-4567. [PMID: 30590542 DOI: 10.1093/cercor/bhy333] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2018] [Revised: 10/30/2018] [Accepted: 12/03/2018] [Indexed: 11/13/2022] Open
Abstract
Rapid and flexible learning during behavioral choices is critical to our daily endeavors and constitutes a hallmark of dynamic reasoning. An important paradigm to examine flexible behavior involves learning new arbitrary associations mapping visual inputs to motor outputs. We conjectured that visuomotor rules are instantiated by translating visual signals into actions through dynamic interactions between visual, frontal and motor cortex. We evaluated the neural representation of such visuomotor rules by performing intracranial field potential recordings in epilepsy subjects during a rule-learning delayed match-to-behavior task. Learning new visuomotor mappings led to the emergence of specific responses associating visual signals with motor outputs in 3 anatomical clusters in frontal, anteroventral temporal and posterior parietal cortex. After learning, mapping selective signals during the delay period showed interactions with visual and motor signals. These observations provide initial steps towards elucidating the dynamic circuits underlying flexible behavior and how communication between subregions of frontal, temporal, and parietal cortex leads to rapid learning of task-relevant choices.
Collapse
Affiliation(s)
- Radhika Madhavan
- Departments of Ophthalmology and Neurosurgery, Children's Hospital, Harvard Medical School, 300 Longwood Avenue, Boston, MA, US
| | - Arjun K Bansal
- Departments of Ophthalmology and Neurosurgery, Children's Hospital, Harvard Medical School, 300 Longwood Avenue, Boston, MA, US.,Current affiliation: Nervana Systems, Inc., 12220 Scripps Summit Dr, San Diego, CA, US
| | - Joseph R Madsen
- Departments of Ophthalmology and Neurosurgery, Children's Hospital, Harvard Medical School, 300 Longwood Avenue, Boston, MA, US
| | - Alexandra J Golby
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, Boston, MA, US
| | - Travis S Tierney
- Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, Boston, MA, US
| | - Emad N Eskandar
- Department of Neurosurgery, Massachusetts General Hospital, 55 Fruit St, Boston, MA, US
| | - William S Anderson
- Department of Neurosurgery, Johns Hopkins Medical School, 733 N Broadway, Baltimore, MD, US
| | - Gabriel Kreiman
- Departments of Ophthalmology and Neurosurgery, Children's Hospital, Harvard Medical School, 300 Longwood Avenue, Boston, MA, US.,Center for Brain Science, Harvard University, 52 Oxford St, Cambridge, MA, US
| |
Collapse
|
16
|
Volkova K, Lebedev MA, Kaplan A, Ossadtchi A. Decoding Movement From Electrocorticographic Activity: A Review. Front Neuroinform 2019; 13:74. [PMID: 31849632 PMCID: PMC6901702 DOI: 10.3389/fninf.2019.00074] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Accepted: 11/14/2019] [Indexed: 01/08/2023] Open
Abstract
Electrocorticography (ECoG) holds promise to provide efficient neuroprosthetic solutions for people suffering from neurological disabilities. This recording technique combines adequate temporal and spatial resolution with the lower risks of medical complications compared to the other invasive methods. ECoG is routinely used in clinical practice for preoperative cortical mapping in epileptic patients. During the last two decades, research utilizing ECoG has considerably grown, including the paradigms where behaviorally relevant information is extracted from ECoG activity with decoding algorithms of different complexity. Several research groups have advanced toward the development of assistive devices driven by brain-computer interfaces (BCIs) that decode motor commands from multichannel ECoG recordings. Here we review the evolution of this field and its recent tendencies, and discuss the potential areas for future development.
Collapse
Affiliation(s)
- Ksenia Volkova
- Center for Bioelectric Interfaces, Higher School of Economics, National Research University, Moscow, Russia
| | - Mikhail A. Lebedev
- Center for Bioelectric Interfaces, Higher School of Economics, National Research University, Moscow, Russia
| | - Alexander Kaplan
- Center for Bioelectric Interfaces, Higher School of Economics, National Research University, Moscow, Russia
- Center for Biotechnology Development, National Research Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Laboratory for Neurophysiology and Neuro-Computer Interfaces, Faculty of Biology, Lomonosov Moscow State University, Moscow, Russia
| | - Alexei Ossadtchi
- Center for Bioelectric Interfaces, Higher School of Economics, National Research University, Moscow, Russia
| |
Collapse
|
17
|
Loza CA, Reddy CG, Akella S, Príncipe JC. Discrimination of Movement-Related Cortical Potentials Exploiting Unsupervised Learned Representations From ECoGs. Front Neurosci 2019; 13:1248. [PMID: 31824249 PMCID: PMC6882771 DOI: 10.3389/fnins.2019.01248] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2019] [Accepted: 11/05/2019] [Indexed: 11/13/2022] Open
Abstract
Brain–Computer Interfaces (BCI) aim to bypass the peripheral nervous system to link the brain to external devices via successful modeling of decoding mechanisms. BCI based on electrocorticogram or ECoG represent a viable compromise between clinical practicality, spatial resolution, and signal quality when it comes to extracellular electrical potentials from local neuronal assemblies. Classic analysis of ECoG traces usually falls under the umbrella of Time-Frequency decompositions with adaptations from Fourier analysis and wavelets as its most prominent variants. However, analyzing such high-dimensional, multivariate time series demands for specialized signal processing and neurophysiological principles. We propose a generative model for single-channel ECoGs that is able to fully characterize reoccurring rhythm–specific neuromodulations as weighted activations of prototypical templates over time. The set of timings, weights and indexes comprise a temporal marked point process (TMPP) that accesses a set of bases from vector spaces of different dimensions—a dictionary. The shallow nature of the model admits the equivalence between latent variables and representations. In this way, learning the model parameters is a case of unsupervised representation learning. We exploit principles of Minimum Description Length (MDL) encoding to effectively yield a data-driven framework where prototypical neuromodulations (not restricted to a particular duration) can be estimated alongside the timings and features of the TMPP. We validate the proposed methodology on discrimination of movement-related tasks utilizing 32-electrode grids implanted in the frontal cortex of six epileptic subjects. We show that the learned representations from the high-gamma band (85–145 Hz) are not only interpretable, but also discriminant in a lower dimensional space. The results also underscore the practicality of our algorithm, i.e., 2 main hyperparameters that can be readily set via neurophysiology, and emphasize the need of principled and interpretable representation learning in order to model encoding mechanisms in the brain.
Collapse
Affiliation(s)
- Carlos A. Loza
- Department of Mathematics, Universidad San Francisco de Quito, Quito, Ecuador
- Instituto de Neurociencias, Universidad San Francisco de Quito, Quito, Ecuador
- *Correspondence: Carlos A. Loza
| | - Chandan G. Reddy
- Department of Neurosurgery, University of Iowa, Iowa City, IA, United States
- Department of Neurosurgery, University of Florida, Gainesville, FL, United States
- Computational NeuroEngineering Lab, Electrical and Computer Engineering Department, University of Florida, Gainesville, FL, United States
| | - Shailaja Akella
- Computational NeuroEngineering Lab, Electrical and Computer Engineering Department, University of Florida, Gainesville, FL, United States
| | - José C. Príncipe
- Computational NeuroEngineering Lab, Electrical and Computer Engineering Department, University of Florida, Gainesville, FL, United States
| |
Collapse
|
18
|
Yang Y, Sani OG, Chang EF, Shanechi MM. Dynamic network modeling and dimensionality reduction for human ECoG activity. J Neural Eng 2019; 16:056014. [DOI: 10.1088/1741-2552/ab2214] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|
19
|
Kapeller C, Ogawa H, Schalk G, Kunii N, Coon WG, Scharinger J, Guger C, Kamada K. Real-time detection and discrimination of visual perception using electrocorticographic signals. J Neural Eng 2018; 15:036001. [PMID: 29359711 DOI: 10.1088/1741-2552/aaa9f6] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
OBJECTIVE Several neuroimaging studies have demonstrated that the ventral temporal cortex contains specialized regions that process visual stimuli. This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images. APPROACH ECoG signals from the participants were recorded while they were shown colored and greyscale versions of seven types of visual stimuli (images of faces, objects, bodies, line drawings, digits, and kanji and hiragana characters), resulting in 14 classes for discrimination (experiment I). Additionally, a real-time system asynchronously classified ECoG responses to faces, kanji and black screens presented via a monitor (experiment II), or to natural scenes (i.e. the face of an experimenter, natural images of faces and kanji, and a mirror) (experiment III). Outcome measures in all experiments included the discrimination performance across types based on broadband γ activity. MAIN RESULTS Experiment I demonstrated an offline classification accuracy of 72.9% when discriminating among the seven types (without color separation). Further discrimination of grey versus colored images reached an accuracy of 67.1%. Discriminating all colors and types (14 classes) yielded an accuracy of 52.1%. In experiment II and III, the real-time decoder correctly detected 73.7% responses to face, kanji and black computer stimuli and 74.8% responses to presented natural scenes. SIGNIFICANCE Seven different types and their color information (either grey or color) could be detected and discriminated using broadband γ activity. Discrimination performance maximized for combined spatial-temporal information. The discrimination of stimulus color information provided the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans. Stimulus categories can be detected by their ECoG responses in real time within 500 ms with respect to stimulus onset.
Collapse
Affiliation(s)
- C Kapeller
- Guger Technologies OG, Graz, Austria. Department of Computational Perception, Johannes Kepler University, Linz, Austria
| | | | | | | | | | | | | | | |
Collapse
|
20
|
From intentions to actions: Neural oscillations encode motor processes through phase, amplitude and phase-amplitude coupling. Neuroimage 2016; 147:473-487. [PMID: 27915117 DOI: 10.1016/j.neuroimage.2016.11.042] [Citation(s) in RCA: 49] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2016] [Revised: 09/19/2016] [Accepted: 11/16/2016] [Indexed: 12/24/2022] Open
Abstract
Goal-directed motor behavior is associated with changes in patterns of rhythmic neuronal activity across widely distributed brain areas. In particular, movement initiation and execution are mediated by patterns of synchronization and desynchronization that occur concurrently across distinct frequency bands and across multiple motor cortical areas. To date, motor-related local oscillatory modulations have been predominantly examined by quantifying increases or suppressions in spectral power. However, beyond signal power, spectral properties such as phase and phase-amplitude coupling (PAC) have also been shown to carry information with regards to the oscillatory dynamics underlying motor processes. Yet, the distinct functional roles of phase, amplitude and PAC across the planning and execution of goal-directed motor behavior remain largely elusive. Here, we address this question with unprecedented resolution thanks to multi-site intracerebral EEG recordings in human subjects while they performed a delayed motor task. To compare the roles of phase, amplitude and PAC, we monitored intracranial brain signals from 748 sites across six medically intractable epilepsy patients at movement execution, and during the delay period where motor intention is present but execution is withheld. In particular, we used a machine-learning framework to identify the key contributions of various neuronal responses. We found a high degree of overlap between brain network patterns observed during planning and those present during execution. Prominent amplitude increases in the delta (2-4Hz) and high gamma (60-200Hz) bands were observed during both planning and execution. In contrast, motor alpha (8-13Hz) and beta (13-30Hz) power were suppressed during execution, but enhanced during the delay period. Interestingly, single-trial classification revealed that low-frequency phase information, rather than spectral power change, was the most discriminant feature in dissociating action from intention. Additionally, despite providing weaker decoding, PAC features led to statistically significant classification of motor states, particularly in anterior cingulate cortex and premotor brain areas. These results advance our understanding of the distinct and partly overlapping involvement of phase, amplitude and the coupling between them, in the neuronal mechanisms underlying motor intentions and executions.
Collapse
|
21
|
Brumberg JS, Krusienski DJ, Chakrabarti S, Gunduz A, Brunner P, Ritaccio AL, Schalk G. Spatio-Temporal Progression of Cortical Activity Related to Continuous Overt and Covert Speech Production in a Reading Task. PLoS One 2016; 11:e0166872. [PMID: 27875590 PMCID: PMC5119784 DOI: 10.1371/journal.pone.0166872] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2016] [Accepted: 11/04/2016] [Indexed: 11/18/2022] Open
Abstract
How the human brain plans, executes, and monitors continuous and fluent speech has remained largely elusive. For example, previous research has defined the cortical locations most important for different aspects of speech function, but has not yet yielded a definition of the temporal progression of involvement of those locations as speech progresses either overtly or covertly. In this paper, we uncovered the spatio-temporal evolution of neuronal population-level activity related to continuous overt speech, and identified those locations that shared activity characteristics across overt and covert speech. Specifically, we asked subjects to repeat continuous sentences aloud or silently while we recorded electrical signals directly from the surface of the brain (electrocorticography (ECoG)). We then determined the relationship between cortical activity and speech output across different areas of cortex and at sub-second timescales. The results highlight a spatio-temporal progression of cortical involvement in the continuous speech process that initiates utterances in frontal-motor areas and ends with the monitoring of auditory feedback in superior temporal gyrus. Direct comparison of cortical activity related to overt versus covert conditions revealed a common network of brain regions involved in speech that may implement orthographic and phonological processing. Our results provide one of the first characterizations of the spatiotemporal electrophysiological representations of the continuous speech process, and also highlight the common neural substrate of overt and covert speech. These results thereby contribute to a refined understanding of speech functions in the human brain.
Collapse
Affiliation(s)
- Jonathan S. Brumberg
- Department of Speech-Language-Hearing: Sciences & Disorders, University of Kansas, Lawrence, KS, United States of America
- * E-mail:
| | - Dean J. Krusienski
- Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA, United States of America
| | - Shreya Chakrabarti
- Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA, United States of America
| | - Aysegul Gunduz
- J. Crayton Pruitt Family Dept. of Biomedical Engineering, University of Florida, Gainesville, FL, United States of America
| | - Peter Brunner
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| | - Anthony L. Ritaccio
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| |
Collapse
|