1
|
Rao N, Paek A, Contreras-Vidal JL, Parikh PJ. Entropy in Electroencephalographic Signals Modulates with Force Magnitude During Grasping - A Preliminary Report. J Mot Behav 2024:1-13. [PMID: 39056321 DOI: 10.1080/00222895.2024.2373241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Revised: 03/22/2024] [Accepted: 06/15/2024] [Indexed: 07/28/2024]
Abstract
The ability to hold objects relies on neural processes underlying grip force control during grasping. Brain activity lateralized to contralateral hemisphere averaged over trials is associated with grip force applied on an object. However, the involvement of neural variability within-trial during grip force control remains unclear. We examined dependence of neural variability over frontal, central, and parietal regions of interest (ROI) on grip force magnitude using noninvasive electroencephalography (EEG). We utilized our existing EEG dataset comprised of healthy young adults performing an isometric force control task, cued to exert 5, 10, or 15% of their maximum voluntary contraction (MVC) across trials and received visual feedback of their grip force. We quantified variability in EEG signal via sample entropy (sequence-dependent) and standard deviation (sequence-independent measure) over ROI. We found lateralized modulation in EEG sample entropy with force magnitude over central electrodes but not over frontal or parietal electrodes. However, modulation was not observed for standard deviation in the EEG activity. These findings highlight lateralized and spatially constrained modulation in sequence-dependent, but not sequence-independent component of EEG variability. We contextualize these findings in applications requiring finer precision (e.g., prosthesis), and propose directions for future studies investigating role of neural entropy in behavior.
Collapse
Affiliation(s)
- Nishant Rao
- Center for Neuromotor and Biomechanics Research, Department of Health and Human Performance, University of Houston, Houston, TX, USA
- Yale Child Study Center, Yale University, New Haven, CT, USA
- Department of Psychology, McGill University, Montreal, QC, Canada
| | - Andrew Paek
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA
| | - Jose L Contreras-Vidal
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, USA
| | - Pranav J Parikh
- Center for Neuromotor and Biomechanics Research, Department of Health and Human Performance, University of Houston, Houston, TX, USA
| |
Collapse
|
2
|
Palidis DJ, Fellows LK. Dorsomedial frontal cortex damage impairs error-based, but not reinforcement-based motor learning in humans. Cereb Cortex 2024; 34:bhad424. [PMID: 37955674 DOI: 10.1093/cercor/bhad424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/10/2023] [Accepted: 10/24/2023] [Indexed: 11/14/2023] Open
Abstract
We adapt our movements to new and changing environments through multiple processes. Sensory error-based learning counteracts environmental perturbations that affect the sensory consequences of movements. Sensory errors also cause the upregulation of reflexes and muscle co-contraction. Reinforcement-based learning enhances the selection of movements that produce rewarding outcomes. Although some findings have identified dissociable neural substrates of sensory error- and reinforcement-based learning, correlative methods have implicated dorsomedial frontal cortex in both. Here, we tested the causal contributions of dorsomedial frontal to adaptive motor control, studying people with chronic damage to this region. Seven human participants with focal brain lesions affecting the dorsomedial frontal and 20 controls performed a battery of arm movement tasks. Three experiments tested: (i) the upregulation of visuomotor reflexes and muscle co-contraction in response to unpredictable mechanical perturbations, (ii) sensory error-based learning in which participants learned to compensate predictively for mechanical force-field perturbations, and (iii) reinforcement-based motor learning based on binary feedback in the absence of sensory error feedback. Participants with dorsomedial frontal damage were impaired in the early stages of force field adaptation, but performed similarly to controls in all other measures. These results provide evidence for a specific and selective causal role for the dorsomedial frontal in sensory error-based learning.
Collapse
Affiliation(s)
- Dimitrios J Palidis
- Montreal Neurological Institute, Department of Neurology & Neurosurgery, McGill University, Montreal, QC H3A 2B4, Canada
| | - Lesley K Fellows
- Montreal Neurological Institute, Department of Neurology & Neurosurgery, McGill University, Montreal, QC H3A 2B4, Canada
| |
Collapse
|
3
|
Shih WY, Yu HY, Lee CC, Chou CC, Chen C, Glimcher PW, Wu SW. Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex. Nat Commun 2023; 14:7821. [PMID: 38016973 PMCID: PMC10684521 DOI: 10.1038/s41467-023-42092-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 09/28/2023] [Indexed: 11/30/2023] Open
Abstract
Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.
Collapse
Affiliation(s)
- Wan-Yu Shih
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| | - Hsiang-Yu Yu
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Cheng-Chia Lee
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
| | - Chien-Chen Chou
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Chien Chen
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Paul W Glimcher
- Neuroscience Institute, NYU Grossman School of Medicine, New York, NY, USA.
- Department of Neuroscience and Physiology, NYU Grossman School of Medicine, New York, NY, USA.
| | - Shih-Wei Wu
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| |
Collapse
|
4
|
Zhang L, Liu C, Zhou X, Zhou H, Luo S, Wang Q, Yao Z, Chen JF. Neural representation and modulation of volitional motivation in response to escalating efforts. J Physiol 2023; 601:631-645. [PMID: 36534700 PMCID: PMC10108165 DOI: 10.1113/jp283915] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 12/07/2022] [Indexed: 12/24/2022] Open
Abstract
Task-dependent volitional control of the selected neural activity in the cortex is critical to neuroprosthetic learning to achieve reliable and robust control of the external device. The volitional control of neural activity is driven by a motivational factor (volitional motivation), which directly reinforces the target neurons via real-time biofeedback. However, in the absence of motor behaviour, how do we evaluate volitional motivation? Here, we defined the criterion (ΔF/F) of the calcium fluorescence signal in a volitionally controlled neural task, then escalated the efforts by progressively increasing the number of reaching the criterion or holding time after reaching the criterion. We devised calcium-based progressive threshold-crossing events (termed 'Calcium PTE') and calcium-based progressive threshold-crossing holding-time (termed 'Calcium PTH') for quantitative assessment of volitional motivation in response to progressively escalating efforts. Furthermore, we used this novel neural representation of volitional motivation to explore the neural circuit and neuromodulator bases for volitional motivation. As with behavioural motivation, chemogenetic activation and pharmacological blockade of the striatopallidal pathway decreased and increased, respectively, the breakpoints of the 'Calcium PTE' and 'Calcium PTH' in response to escalating efforts. Furthermore, volitional and behavioural motivation shared similar dopamine dynamics in the nucleus accumbens in response to trial-by-trial escalating efforts. In general, the development of a neural representation of volitional motivation may open a new avenue for smooth and effective control of brain-machine interface tasks. KEY POINTS: Volitional motivation is quantitatively evaluated by M1 neural activity in response to progressively escalating volitional efforts. The striatopallidal pathway and adenosine A2A receptor modulate volitional motivation in response to escalating efforts. Dopamine dynamics encode prediction signal for reward in response to repeated escalating efforts during motor and volitional conditioning. Mice learn to modulate neural activity to compensate for repeated escalating efforts in volitional control.
Collapse
Affiliation(s)
- Liping Zhang
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Chengwei Liu
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Xiaopeng Zhou
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Hui Zhou
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Shengtao Luo
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Qin Wang
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Zhimo Yao
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Jiang-Fan Chen
- The Molecular Neuropharmacology Laboratory and the Eye-Brain Research Center, The State Key Laboratory of Ophthalmology, Optometry and Vision Science, School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China.,Oujiang Laboratory (Zhejiang Laboratory for Regenerative Medicine, Vision and Brain Health), School of Ophthalmology & Optometry and Eye Hospital, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
5
|
Adaikkan C, Wang J, Abdelaal K, Middleton SJ, Bozzelli PL, Wickersham IR, McHugh TJ, Tsai LH. Alterations in a cross-hemispheric circuit associates with novelty discrimination deficits in mouse models of neurodegeneration. Neuron 2022; 110:3091-3105.e9. [PMID: 35987206 PMCID: PMC9547933 DOI: 10.1016/j.neuron.2022.07.023] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 02/23/2022] [Accepted: 07/22/2022] [Indexed: 10/15/2022]
Abstract
A major pathological hallmark of neurodegenerative diseases, including Alzheimer's, is a significant reduction in the white matter connecting the two cerebral hemispheres, as well as in the correlated activity between anatomically corresponding bilateral brain areas. However, the underlying circuit mechanisms and the cognitive relevance of cross-hemispheric (CH) communication remain poorly understood. Here, we show that novelty discrimination behavior activates CH neurons and enhances homotopic synchronized neural oscillations in the visual cortex. CH neurons provide excitatory drive required for synchronous neural oscillations between hemispheres, and unilateral inhibition of the CH circuit is sufficient to impair synchronous oscillations and novelty discrimination behavior. In the 5XFAD and Tau P301S mouse models, CH communication is altered, and novelty discrimination is impaired. These data reveal a hitherto uncharacterized CH circuit in the visual cortex, establishing a causal link between this circuit and novelty discrimination behavior and highlighting its impairment in mouse models of neurodegeneration.
Collapse
Affiliation(s)
- Chinnakkaruppan Adaikkan
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Jun Wang
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Karim Abdelaal
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Steven J Middleton
- Laboratory for Circuit and Behavioral Physiology, RIKEN Center for Brain Science, Wakoshi, Saitama 351-0198, Japan
| | - P Lorenzo Bozzelli
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Ian R Wickersham
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Thomas J McHugh
- Laboratory for Circuit and Behavioral Physiology, RIKEN Center for Brain Science, Wakoshi, Saitama 351-0198, Japan; Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan.
| | - Li-Huei Tsai
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Broad Institute of Harvard and Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| |
Collapse
|
6
|
Girdler B, Caldbeck W, Bae J. Neural Decoders Using Reinforcement Learning in Brain Machine Interfaces: A Technical Review. Front Syst Neurosci 2022; 16:836778. [PMID: 36090185 PMCID: PMC9459159 DOI: 10.3389/fnsys.2022.836778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 06/21/2022] [Indexed: 11/18/2022] Open
Abstract
Creating flexible and robust brain machine interfaces (BMIs) is currently a popular topic of research that has been explored for decades in medicine, engineering, commercial, and machine-learning communities. In particular, the use of techniques using reinforcement learning (RL) has demonstrated impressive results but is under-represented in the BMI community. To shine more light on this promising relationship, this article aims to provide an exhaustive review of RL's applications to BMIs. Our primary focus in this review is to provide a technical summary of various algorithms used in RL-based BMIs to decode neural intention, without emphasizing preprocessing techniques on the neural signals and reward modeling for RL. We first organize the literature based on the type of RL methods used for neural decoding, and then each algorithm's learning strategy is explained along with its application in BMIs. A comparative analysis highlighting the similarities and uniqueness among neural decoders is provided. Finally, we end this review with a discussion about the current stage of RLBMIs including their limitations and promising directions for future research.
Collapse
Affiliation(s)
| | | | - Jihye Bae
- Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, United States
| |
Collapse
|
7
|
Kumar JNA, Francis JT. Improved Grip Force Prediction Using a Loss Function that Penalizes Reward Related Neural Information. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:2336-2339. [PMID: 36085700 DOI: 10.1109/embc48229.2022.9871920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Neural activity in the sensorimotor cortices has been previously shown to correlate with kinematics, kinetics, and non-sensorimotor variables, such as reward. In this work, we compare the grip force offline Brain Machine Interface (BMI) prediction performance, of a simple artificial neural network (ANN), under two loss functions: the standard mean squared error (MSE) and a modified reward penalized mean squared error (RP_MSE), which penalizes for correlation between reward and grip force. Our results show that the ANN performs significantly better under the RP_MSE loss function in three brain regions: dorsal premotor cortex (PMd), primary motor cortex (M1) and the primary somatosensory cortex (S1) by approximately 6%.
Collapse
|
8
|
Yadav T, Tellez OM, Francis JT. Reward-dependent Graded Suppression of Sensorimotor Beta-band Local Field Potentials During an Arm Reaching Task in NHP. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3123-3126. [PMID: 36086028 DOI: 10.1109/embc48229.2022.9871212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
A better understanding of reward signaling in the sensorimotor cortices can aid in developing Reinforcement Learning-based Brain-Computer Interfaces (RLBCI) for restoration of movement functions with fewer implants. Brain-computer interfaces (BCIs) using local field potentials (LFPs) have recently achieved performance comparable to spike-BCIs [1]. With superior stability over time, LFPs may be the preferred signal for BCIs. We show that sensorimotor LFPs can provide reward level information (R1 - R3) like spikes[2]. We used a cued reward-level reaching task in which reward information was temporally dissociated from movement information. This allowed the study of reward- and movement-related modulations in LFPs. We recorded simultaneously from contralateral primary -somatosensory (S1), -motor (M1), and the dorsal premotor (PMd) cortices in a female Macaca Mulatta. We found that all three cortices' average beta band (14-30 Hz) amplitude showed robust modulation with reward levels during the cue presentation period. Such modulation was consistently observed after controlling for cue color, differences in behavioral variables, and electromyogram (EMG) activity. Statistical amplitude analysis showed that reward level could be extracted from the simple LFP feature of beta band amplitude, even before a reaching target appeared, and no specific reach plan could be developed. Clinical Relevance - The availability of reward-related signals in the sensorimotor cortical (S1, M1,and PMd) LFPs' prior to movement planning opens new avenues to build RLBCIs with fewer implants recording fewer sites among different cortices Reward and motivational representations derived from LFPs compared to spikes allow the development of long-term clinical applications given LFP's stability and ease of recording over long periods.
Collapse
|
9
|
Suzuki M, Inoue KI, Nakagawa H, Ishida H, Kobayashi K, Isa T, Takada M, Nishimura Y. A multisynaptic pathway from the ventral midbrain toward spinal motoneurons in monkeys. J Physiol 2022; 600:1731-1752. [PMID: 35122444 PMCID: PMC9306604 DOI: 10.1113/jp282429] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 01/10/2022] [Indexed: 12/02/2022] Open
Abstract
Abstract Motivation boosts motor performance. Activity of the ventral midbrain (VM), consisting of the ventral tegmental area (VTA), the substantia nigra pars compacta (SNc) and the retrorubral field (RRF), plays an important role in processing motivation. However, little is known about the neural substrate bridging the VM and the spinal motor output. We hypothesized that the VM might exert a modulatory influence over the descending motor pathways. By retrograde transneuronal labelling with rabies virus, we demonstrated the existence of multisynaptic projections from the VM to the cervical enlargement in monkeys. The distribution pattern of spinal projection neurons in the VM exhibited a caudorostral gradient, in that the RRF and the caudal part of the SNc contained more retrogradely labelled neurons than the VTA and the rostral part of the SNc. Electrical stimulation of the VM induced muscle responses in the contralateral forelimb with a delay of a few milliseconds following the responses of the ipsilateral primary motor cortex (M1). The magnitude and number of evoked muscle responses were associated with the stimulus intensity and number of pulses. The muscle responses were diminished during M1 inactivation. Thus, the present study has identified a multisynaptic VM–spinal pathway that is mediated, at least in part, by the M1 and might play a pivotal role in modulatory control of the spinal motor output. Key points Motivation to obtain reward is thought to boost motor performance, and activity in the ventral midbrain is important to the motivational process. Little is known about a neural substrate bridging the ventral midbrain and the spinal motor output. Retrograde trans‐synaptic experiments revealed that the ventral midbrain projects multisynaptically to the spinal cord in macaque monkeys. Ventral midbrain activation by electrical stimulation generated cortical activity in the motor cortex and forelimb muscle activity. A multisynaptic ventral midbrain–spinal pathway most probably plays a pivotal role in modulatory control of the spinal motor output.
Collapse
Affiliation(s)
- Michiaki Suzuki
- Neural Prosthetics Project, Tokyo Metropolitan Institute of Medical Science, Setagaya, Tokyo, 156-8506, Japan.,Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi, 444-8585, Japan.,Department of Physiological Sciences, School of Life Science, SOKENDAI (The Graduate University for Advanced Studies), Hayama, Kanagawa, 240-0193, Japan
| | - Ken-Ichi Inoue
- Systems Neuroscience Section, Primate Research Institute, Kyoto University, Inuyama, Aichi, 484-8506, Japan
| | - Hiroshi Nakagawa
- Systems Neuroscience Section, Primate Research Institute, Kyoto University, Inuyama, Aichi, 484-8506, Japan.,Present address: Department of Molecular Neuroscience, World Premier International Immunology Frontier Research Center, Osaka University, Suita, Osaka, 565-0871, Japan
| | - Hiroaki Ishida
- Neural Prosthetics Project, Tokyo Metropolitan Institute of Medical Science, Setagaya, Tokyo, 156-8506, Japan.,Present address: Schizophrenia Research Project, Tokyo Metropolitan Institute of Medical Science, Setagaya, Tokyo, 156-8506, Japan
| | - Kenta Kobayashi
- Section of Viral Vector Development, National Institute for Physiological Sciences, Okazaki, Aichi, 444-8585, Japan
| | - Tadashi Isa
- Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi, 444-8585, Japan.,Department of Physiological Sciences, School of Life Science, SOKENDAI (The Graduate University for Advanced Studies), Hayama, Kanagawa, 240-0193, Japan.,Section of Viral Vector Development, National Institute for Physiological Sciences, Okazaki, Aichi, 444-8585, Japan.,Department of Neuroscience, Graduate School of Medicine, Kyoto University, Sakyo, Kyoto, 606-8501, Japan.,Institute for the Advanced Study of Human Biology (WPI-ASHBi), Kyoto University, Sakyo, Kyoto, 606-8501, Japan
| | - Masahiko Takada
- Systems Neuroscience Section, Primate Research Institute, Kyoto University, Inuyama, Aichi, 484-8506, Japan
| | - Yukio Nishimura
- Neural Prosthetics Project, Tokyo Metropolitan Institute of Medical Science, Setagaya, Tokyo, 156-8506, Japan.,Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi, 444-8585, Japan.,Department of Physiological Sciences, School of Life Science, SOKENDAI (The Graduate University for Advanced Studies), Hayama, Kanagawa, 240-0193, Japan
| |
Collapse
|
10
|
Yao Z, Hessburg JP, Francis JT. Normalization by valence and motivational intensity in the sensorimotor cortices (PMd, M1, and S1). Sci Rep 2021; 11:24221. [PMID: 34930930 PMCID: PMC8688489 DOI: 10.1038/s41598-021-03200-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Accepted: 11/26/2021] [Indexed: 12/27/2022] Open
Abstract
Our brain's ability to represent vast amounts of information, such as continuous ranges of reward spanning orders of magnitude, with limited dynamic range neurons, may be possible due to normalization. Recently our group and others have shown that the sensorimotor cortices are sensitive to reward value. Here we ask if psychological affect causes normalization of the sensorimotor cortices by modulating valence and motivational intensity. We had two non-human primates (NHP) subjects (one male bonnet macaque and one female rhesus macaque) make visually cued grip-force movements while simultaneously cueing the level of possible reward if successful, or timeout punishment, if unsuccessful. We recorded simultaneously from 96 electrodes in each the following: caudal somatosensory, rostral motor, and dorsal premotor cortices (cS1, rM1, PMd). We utilized several normalization models for valence and motivational intensity in all three regions. We found three types of divisive normalized relationships between neural activity and the representation of valence and motivation, linear, sigmodal, and hyperbolic. The hyperbolic relationships resemble receptive fields in psychological affect space, where a unit is susceptible to a small range of the valence/motivational space. We found that these cortical regions have both strong valence and motivational intensity representations.
Collapse
Affiliation(s)
- Zhao Yao
- Department of Physiology and Pharmacology, State University of New York Downstate Medical Center, Brooklyn, NY, 11203, USA
| | - John P Hessburg
- Department of Physiology and Pharmacology, State University of New York Downstate Medical Center, Brooklyn, NY, 11203, USA
| | - Joseph Thachil Francis
- Departments of Biomedical Engineering and Electrical and Computer Engineering, Cullen College of Engineering at The University of Houston, Houston, TX, 77204, USA.
| |
Collapse
|
11
|
Mirror neurons are modulated by grip force and reward expectation in the sensorimotor cortices (S1, M1, PMd, PMv). Sci Rep 2021; 11:15959. [PMID: 34354213 PMCID: PMC8342437 DOI: 10.1038/s41598-021-95536-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2020] [Accepted: 07/21/2021] [Indexed: 11/24/2022] Open
Abstract
Mirror Neurons (MNs) respond similarly when primates make or observe grasping movements. Recent work indicates that reward expectation influences rostral M1 (rM1) during manual, observational, and Brain Machine Interface (BMI) reaching movements. Previous work showed MNs are modulated by subjective value. Here we expand on the above work utilizing two non-human primates (NHPs), one male Macaca Radiata (NHP S) and one female Macaca Mulatta (NHP P), that were trained to perform a cued reward level isometric grip-force task, where the NHPs had to apply visually cued grip-force to move and transport a virtual object. We found a population of (S1 area 1–2, rM1, PMd, PMv) units that significantly represented grip-force during manual and observational trials. We found the neural representation of visually cued force was similar during observational trials and manual trials for the same units; however, the representation was weaker during observational trials. Comparing changes in neural time lags between manual and observational tasks indicated that a subpopulation fit the standard MN definition of observational neural activity lagging the visual information. Neural activity in (S1 areas 1–2, rM1, PMd, PMv) significantly represented force and reward expectation. In summary, we present results indicating that sensorimotor cortices have MNs for visually cued force and value.
Collapse
|
12
|
Moore B, Khang S, Francis JT. Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates. Front Behav Neurosci 2020; 14:541920. [PMID: 33343308 PMCID: PMC7739882 DOI: 10.3389/fnbeh.2020.541920] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2020] [Accepted: 09/30/2020] [Indexed: 11/17/2022] Open
Abstract
Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson's correlation coefficient to measure shared variability between simultaneously recorded units (32-112) and found significantly higher noise-correlation and positive correlation between the populations' signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.
Collapse
Affiliation(s)
- Brittany Moore
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| | - Sheng Khang
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| | - Joseph Thachil Francis
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
- Department of Electrical and Computer Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| |
Collapse
|
13
|
Shen X, Zhang X, Huang Y, Chen S, Wang Y. Task Learning Over Multi-Day Recording via Internally Rewarded Reinforcement Learning Based Brain Machine Interfaces. IEEE Trans Neural Syst Rehabil Eng 2020; 28:3089-3099. [PMID: 33232240 DOI: 10.1109/tnsre.2020.3039970] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Autonomous brain machine interfaces (BMIs) aim to enable paralyzed people to self-evaluate their movement intention to control external devices. Previous reinforcement learning (RL)-based decoders interpret the mapping between neural activity and movements using the external reward for well-trained subjects, and have not investigated the task learning procedure. The brain has developed a learning mechanism to identify the correct actions that lead to rewards in the new task. This internal guidance can be utilized to replace the external reference to advance BMIs as an autonomous system. In this study, we propose to build an internally rewarded reinforcement learning-based BMI framework using the multi-site recording to demonstrate the autonomous learning ability of the BMI decoder on the new task. We test the model on the neural data collected over multiple days while the rats were learning a new lever discrimination task. The primary motor cortex (M1) and medial prefrontal cortex (mPFC) spikes are interpreted by the proposed RL framework into the discrete lever press actions. The neural activity of the mPFC post the action duration is interpreted as the internal reward information, where a support vector machine is implemented to classify the reward vs. non-reward trials with a high accuracy of 87.5% across subjects. This internal reward is used to replace the external water reward to update the decoder, which is able to adapt to the nonstationary neural activity during subject learning. The multi-cortical recording allows us to take in more cortical recordings as input and uses internal critics to guide the decoder learning. Comparing with the classic decoder using M1 activity as the only input and external guidance, the proposed system with multi-cortical recordings shows a better decoding accuracy. More importantly, our internally rewarded decoder demonstrates the autonomous learning ability on the new task as the decoder successfully addresses the time-variant neural patterns while subjects are learning, and works asymptotically as the subjects' behavioral learning progresses. It reveals the potential of endowing BMIs with autonomous task learning ability in the RL framework.
Collapse
|
14
|
Taghizadeh B, Foley NC, Karimimehr S, Cohanpour M, Semework M, Sheth SA, Lashgari R, Gottlieb J. Reward uncertainty asymmetrically affects information transmission within the monkey fronto-parietal network. Commun Biol 2020; 3:594. [PMID: 33087809 PMCID: PMC7578031 DOI: 10.1038/s42003-020-01320-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Accepted: 09/25/2020] [Indexed: 01/02/2023] Open
Abstract
A central hypothesis in research on executive function is that controlled information processing is costly and is allocated according to the behavioral benefits it brings. However, while computational theories predict that the benefits of new information depend on prior uncertainty, the cellular effects of uncertainty on the executive network are incompletely understood. Using simultaneous recordings in monkeys, we describe several mechanisms by which the fronto-parietal network reacts to uncertainty. We show that the variance of expected rewards, independently of the value of the rewards, was encoded in single neuron and population spiking activity and local field potential (LFP) oscillations, and, importantly, asymmetrically affected fronto-parietal information transmission (measured through the coherence between spikes and LFPs). Higher uncertainty selectively enhanced information transmission from the parietal to the frontal lobe and suppressed it in the opposite direction, consistent with Bayesian principles that prioritize sensory information according to a decision maker’s prior uncertainty. Bahareh Taghizadeh and Nicholas Foley et al. show that individual neuronal responses, population spiking activity, and local field potential oscillations encode the variance of expected rewards independent of their value. They also demonstrate that reward uncertainty asymmetrically affects neuronal transmission within the monkey fronto-parietal network.
Collapse
Affiliation(s)
- Bahareh Taghizadeh
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences, Tehran, Iran.,School of Cognitive Sciences, Institute for Research in Fundamental Sciences, Tehran, Iran
| | - Nicholas C Foley
- Department of Neuroscience, Columbia University, New York, NY, USA.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Saeed Karimimehr
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences, Tehran, Iran.,School of Cognitive Sciences, Institute for Research in Fundamental Sciences, Tehran, Iran
| | - Michael Cohanpour
- Department of Neuroscience, Columbia University, New York, NY, USA.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Mulugeta Semework
- Department of Neuroscience, Columbia University, New York, NY, USA.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Sameer A Sheth
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
| | - Reza Lashgari
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences, Tehran, Iran.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, New York, NY, USA. .,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA. .,The Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
15
|
Mulcahy G, Atwood B, Kuznetsov A. Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states. PLoS One 2020; 15:e0228081. [PMID: 32040519 PMCID: PMC7010262 DOI: 10.1371/journal.pone.0228081] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2019] [Accepted: 01/07/2020] [Indexed: 01/06/2023] Open
Abstract
The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.
Collapse
Affiliation(s)
- Garrett Mulcahy
- Department of Mathematics, Purdue University, West Lafayette, Indiana, United States of America
| | - Brady Atwood
- Departments of Psychiatry and Pharmacology & Toxicology, IUSM, Indianapolis, Indiana, United States of America
- Indiana Alcohol Research Center, IUSM, Indianapolis, Indiana, United States of America
| | - Alexey Kuznetsov
- Indiana Alcohol Research Center, IUSM, Indianapolis, Indiana, United States of America
- Department of Mathematical Sciences, IUPUI, Indianapolis, Indiana, United States of America
| |
Collapse
|