1
|
Codol O, Michaels JA, Kashefi M, Pruszynski JA, Gribble PL. MotorNet, a Python toolbox for controlling differentiable biomechanical effectors with artificial neural networks. eLife 2024; 12:RP88591. [PMID: 39078880 PMCID: PMC11288629 DOI: 10.7554/elife.88591] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/02/2024] Open
Abstract
Artificial neural networks (ANNs) are a powerful class of computational models for unravelling neural mechanisms of brain function. However, for neural control of movement, they currently must be integrated with software simulating biomechanical effectors, leading to limiting impracticalities: (1) researchers must rely on two different platforms and (2) biomechanical effectors are not generally differentiable, constraining researchers to reinforcement learning algorithms despite the existence and potential biological relevance of faster training methods. To address these limitations, we developed MotorNet, an open-source Python toolbox for creating arbitrarily complex, differentiable, and biomechanically realistic effectors that can be trained on user-defined motor tasks using ANNs. MotorNet is designed to meet several goals: ease of installation, ease of use, a high-level user-friendly application programming interface, and a modular architecture to allow for flexibility in model building. MotorNet requires no dependencies outside Python, making it easy to get started with. For instance, it allows training ANNs on typically used motor control models such as a two joint, six muscle, planar arm within minutes on a typical desktop computer. MotorNet is built on PyTorch and therefore can implement any network architecture that is possible using the PyTorch framework. Consequently, it will immediately benefit from advances in artificial intelligence through PyTorch updates. Finally, it is open source, enabling users to create and share their own improvements, such as new effector and network architectures or custom task designs. MotorNet's focus on higher-order model and task design will alleviate overhead cost to initiate computational projects for new researchers by providing a standalone, ready-to-go framework, and speed up efforts of established computational teams by enabling a focus on concepts and ideas over implementation.
Collapse
Affiliation(s)
- Olivier Codol
- Western Institute for Neuroscience, University of Western OntarioOntarioCanada
- Department of Psychology, University of Western OntarioOntarioCanada
| | - Jonathan A Michaels
- Western Institute for Neuroscience, University of Western OntarioOntarioCanada
- Department of Physiology & Pharmacology, Schulich School of Medicine & Dentistry, University of Western OntarioOntarioCanada
- Robarts Research Institute, University of Western OntarioOntarioCanada
| | - Mehrdad Kashefi
- Western Institute for Neuroscience, University of Western OntarioOntarioCanada
- Department of Physiology & Pharmacology, Schulich School of Medicine & Dentistry, University of Western OntarioOntarioCanada
- Robarts Research Institute, University of Western OntarioOntarioCanada
| | - J Andrew Pruszynski
- Western Institute for Neuroscience, University of Western OntarioOntarioCanada
- Department of Psychology, University of Western OntarioOntarioCanada
- Department of Physiology & Pharmacology, Schulich School of Medicine & Dentistry, University of Western OntarioOntarioCanada
- Robarts Research Institute, University of Western OntarioOntarioCanada
| | - Paul L Gribble
- Western Institute for Neuroscience, University of Western OntarioOntarioCanada
- Department of Psychology, University of Western OntarioOntarioCanada
- Department of Physiology & Pharmacology, Schulich School of Medicine & Dentistry, University of Western OntarioOntarioCanada
| |
Collapse
|
2
|
Rostami V, Rost T, Schmitt FJ, van Albada SJ, Riehle A, Nawrot MP. Spiking attractor model of motor cortex explains modulation of neural and behavioral variability by prior target information. Nat Commun 2024; 15:6304. [PMID: 39060243 PMCID: PMC11282312 DOI: 10.1038/s41467-024-49889-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 06/19/2024] [Indexed: 07/28/2024] Open
Abstract
When preparing a movement, we often rely on partial or incomplete information, which can decrement task performance. In behaving monkeys we show that the degree of cued target information is reflected in both, neural variability in motor cortex and behavioral reaction times. We study the underlying mechanisms in a spiking motor-cortical attractor model. By introducing a biologically realistic network topology where excitatory neuron clusters are locally balanced with inhibitory neuron clusters we robustly achieve metastable network activity across a wide range of network parameters. In application to the monkey task, the model performs target-specific action selection and accurately reproduces the task-epoch dependent reduction of trial-to-trial variability in vivo where the degree of reduction directly reflects the amount of processed target information, while spiking irregularity remained constant throughout the task. In the context of incomplete cue information, the increased target selection time of the model can explain increased behavioral reaction times. We conclude that context-dependent neural and behavioral variability is a signum of attractor computation in the motor cortex.
Collapse
Affiliation(s)
- Vahid Rostami
- Institute of Zoology, University of Cologne, Cologne, Germany
| | - Thomas Rost
- Institute of Zoology, University of Cologne, Cologne, Germany
| | | | - Sacha Jennifer van Albada
- Institute of Zoology, University of Cologne, Cologne, Germany
- Institute for Advanced Simulation (IAS-6), Jülich Research Center, Jülich, Germany
| | - Alexa Riehle
- Institute for Advanced Simulation (IAS-6), Jülich Research Center, Jülich, Germany
- UMR7289 Institut de Neurosciences de la Timone (INT), Centre National de la Recherche Scientifique (CNRS)-Aix-Marseille Université (AMU), Marseille, France
| | | |
Collapse
|
3
|
Rodriguez AC, Perich MG, Miller L, Humphries MD. Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.05.26.542452. [PMID: 37292834 PMCID: PMC10246015 DOI: 10.1101/2023.05.26.542452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.
Collapse
Affiliation(s)
| | - Matthew G. Perich
- Département de neurosciences, Faculté de médecine, Université de Montréal, Montréal, Canada
- Québec Artificial Intelligence Institute (Mila), Québec, Canada
| | - Lee Miller
- Northwestern University, Department of Biomedical Engineering, Chicago, USA
| | - Mark D. Humphries
- School of Psychology, University of Nottingham, Nottingham, United Kingdom
| |
Collapse
|
4
|
Fortunato C, Bennasar-Vázquez J, Park J, Chang JC, Miller LE, Dudman JT, Perich MG, Gallego JA. Nonlinear manifolds underlie neural population activity during behaviour. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.07.18.549575. [PMID: 37503015 PMCID: PMC10370078 DOI: 10.1101/2023.07.18.549575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.
Collapse
Affiliation(s)
- Cátia Fortunato
- Department of Bioengineering, Imperial College London, London UK
| | | | - Junchol Park
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn VA, USA
| | - Joanna C. Chang
- Department of Bioengineering, Imperial College London, London UK
| | - Lee E. Miller
- Department of Neurosciences, Northwestern University, Chicago IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago IL, USA
- Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago IL, USA, and Shirley Ryan Ability Lab, Chicago, IL, USA
| | - Joshua T. Dudman
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn VA, USA
| | - Matthew G. Perich
- Department of Neurosciences, Faculté de médecine, Université de Montréal, Montréal, Québec, Canada
- Québec Artificial Intelligence Institute (MILA), Montréal, Québec, Canada
| | - Juan A. Gallego
- Department of Bioengineering, Imperial College London, London UK
| |
Collapse
|
5
|
Wang T, Chen Y, Zhang Y, Cui H. Multiplicative joint coding in preparatory activity for reaching sequence in macaque motor cortex. Nat Commun 2024; 15:3153. [PMID: 38605030 PMCID: PMC11009282 DOI: 10.1038/s41467-024-47511-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 04/02/2024] [Indexed: 04/13/2024] Open
Abstract
Although the motor cortex has been found to be modulated by sensory or cognitive sequences, the linkage between multiple movement elements and sequence-related responses is not yet understood. Here, we recorded neuronal activity from the motor cortex with implanted micro-electrode arrays and single electrodes while monkeys performed a double-reach task that was instructed by simultaneously presented memorized cues. We found that there existed a substantial multiplicative component jointly tuned to impending and subsequent reaches during preparation, then the coding mechanism transferred to an additive manner during execution. This multiplicative joint coding, which also spontaneously emerged in recurrent neural networks trained for double reach, enriches neural patterns for sequential movement, and might explain the linear readout of elemental movements.
Collapse
Affiliation(s)
- Tianwei Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - Yun Chen
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - Yiheng Zhang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
- Chinese Institute for Brain Research, Beijing, 102206, China
| | - He Cui
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.
- University of Chinese Academy of Sciences, Beijing, 100049, China.
- Chinese Institute for Brain Research, Beijing, 102206, China.
- Shanghai Center for Brain Science and Brain-inspired Technology, Shanghai, 200031, China.
| |
Collapse
|
6
|
Meng R, Bouchard KE. Bayesian inference of structured latent spaces from neural population activity with the orthogonal stochastic linear mixing model. PLoS Comput Biol 2024; 20:e1011975. [PMID: 38669271 PMCID: PMC11078355 DOI: 10.1371/journal.pcbi.1011975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 05/08/2024] [Accepted: 03/07/2024] [Indexed: 04/28/2024] Open
Abstract
The brain produces diverse functions, from perceiving sounds to producing arm reaches, through the collective activity of populations of many neurons. Determining if and how the features of these exogenous variables (e.g., sound frequency, reach angle) are reflected in population neural activity is important for understanding how the brain operates. Often, high-dimensional neural population activity is confined to low-dimensional latent spaces. However, many current methods fail to extract latent spaces that are clearly structured by exogenous variables. This has contributed to a debate about whether or not brains should be thought of as dynamical systems or representational systems. Here, we developed a new latent process Bayesian regression framework, the orthogonal stochastic linear mixing model (OSLMM) which introduces an orthogonality constraint amongst time-varying mixture coefficients, and provide Markov chain Monte Carlo inference procedures. We demonstrate superior performance of OSLMM on latent trajectory recovery in synthetic experiments and show superior computational efficiency and prediction performance on several real-world benchmark data sets. We primarily focus on demonstrating the utility of OSLMM in two neural data sets: μECoG recordings from rat auditory cortex during presentation of pure tones and multi-single unit recordings form monkey motor cortex during complex arm reaching. We show that OSLMM achieves superior or comparable predictive accuracy of neural data and decoding of external variables (e.g., reach velocity). Most importantly, in both experimental contexts, we demonstrate that OSLMM latent trajectories directly reflect features of the sounds and reaches, demonstrating that neural dynamics are structured by neural representations. Together, these results demonstrate that OSLMM will be useful for the analysis of diverse, large-scale biological time-series datasets.
Collapse
Affiliation(s)
- Rui Meng
- Biological Systems and Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, California, United States of America
| | - Kristofer E. Bouchard
- Biological Systems and Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, California, United States of America
- Scientific Data Division, Lawrence Berkeley National Laboratory, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, United States of America
- Redwood Center for Theoretical Neuroscience, University of California Berkeley, Berkeley, California, United States of America
| |
Collapse
|
7
|
Churchland MM, Shenoy KV. Preparatory activity and the expansive null-space. Nat Rev Neurosci 2024; 25:213-236. [PMID: 38443626 DOI: 10.1038/s41583-024-00796-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/26/2024] [Indexed: 03/07/2024]
Abstract
The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.
Collapse
Affiliation(s)
- Mark M Churchland
- Department of Neuroscience, Columbia University, New York, NY, USA.
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| |
Collapse
|
8
|
Ahmadipour P, Sani OG, Pesaran B, Shanechi MM. Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity. J Neural Eng 2024; 21:026001. [PMID: 38016450 PMCID: PMC10913727 DOI: 10.1088/1741-2552/ad1053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 10/23/2023] [Accepted: 11/28/2023] [Indexed: 11/30/2023]
Abstract
Objective.Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales.Approach.Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior.Main results.We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior.Significance.Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.
Collapse
Affiliation(s)
- Parima Ahmadipour
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Omid G Sani
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Alfred E. Mann Department of Biomedical Engineering, and the Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
9
|
Ji X, Elmoznino E, Deane G, Constant A, Dumas G, Lajoie G, Simon J, Bengio Y. Sources of richness and ineffability for phenomenally conscious states. Neurosci Conscious 2024; 2024:niae001. [PMID: 38487679 PMCID: PMC10939345 DOI: 10.1093/nc/niae001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 01/03/2024] [Accepted: 01/23/2024] [Indexed: 03/17/2024] Open
Abstract
Conscious states-state that there is something it is like to be in-seem both rich or full of detail and ineffable or hard to fully describe or recall. The problem of ineffability, in particular, is a longstanding issue in philosophy that partly motivates the explanatory gap: the belief that consciousness cannot be reduced to underlying physical processes. Here, we provide an information theoretic dynamical systems perspective on the richness and ineffability of consciousness. In our framework, the richness of conscious experience corresponds to the amount of information in a conscious state and ineffability corresponds to the amount of information lost at different stages of processing. We describe how attractor dynamics in working memory would induce impoverished recollections of our original experiences, how the discrete symbolic nature of language is insufficient for describing the rich and high-dimensional structure of experiences, and how similarity in the cognitive function of two individuals relates to improved communicability of their experiences to each other. While our model may not settle all questions relating to the explanatory gap, it makes progress toward a fully physicalist explanation of the richness and ineffability of conscious experience-two important aspects that seem to be part of what makes qualitative character so puzzling.
Collapse
Affiliation(s)
- Xu Ji
- Mila - Quebec AI Institute, Montreal, Quebec H2S 3H1, Canada
- Department of Computer science and operations Research, University of Montreal, Pavillon André-Aisenstadt 2920, chemin de la Tour, Montreal, Quebec H3T 1J4, Canada
| | - Eric Elmoznino
- Mila - Quebec AI Institute, Montreal, Quebec H2S 3H1, Canada
- Department of Computer science and operations Research, University of Montreal, Pavillon André-Aisenstadt 2920, chemin de la Tour, Montreal, Quebec H3T 1J4, Canada
| | - George Deane
- Department of Philosophy, University of Montreal, Pavillon 2910, boul. Édouard-Montpetit, Montreal, Quebec H3C 3J7, Canada
| | - Axel Constant
- School of Engineering and Informatics, University of Sussex, Sussex House, Falmer, East Sussex BN1 9RH, United Kingdom
| | - Guillaume Dumas
- Mila - Quebec AI Institute, Montreal, Quebec H2S 3H1, Canada
- Department of Psychiatry and Addiction, University of Montreal, Pavillon Roger-Gaudry 2900, boul. Édouard-Montpetit, Montreal, Quebec H3T 1J4, Canada
| | - Guillaume Lajoie
- Mila - Quebec AI Institute, Montreal, Quebec H2S 3H1, Canada
- Department of Mathematics and Statistics, University of Montreal, Pavillon André-Aisenstadt (AA-5190) 2920, chemin de la Tour, Montreal, Quebec H3T 1J4, Canada
| | - Jonathan Simon
- Department of Philosophy, University of Montreal, Pavillon 2910, boul. Édouard-Montpetit, Montreal, Quebec H3C 3J7, Canada
| | - Yoshua Bengio
- Mila - Quebec AI Institute, Montreal, Quebec H2S 3H1, Canada
- Department of Computer science and operations Research, University of Montreal, Pavillon André-Aisenstadt 2920, chemin de la Tour, Montreal, Quebec H3T 1J4, Canada
- CIFAR - Canadian Institute for Advanced Research, MaRS Centre, West Tower 661 University Ave., Suite 505, Toronto, Ontario M5G 1M1, Canada
| |
Collapse
|
10
|
Chae S, Sohn JW, Kim SP. Differential Formation of Motor Cortical Dynamics during Movement Preparation According to the Predictability of Go Timing. J Neurosci 2024; 44:e1353232024. [PMID: 38233217 PMCID: PMC10883619 DOI: 10.1523/jneurosci.1353-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 12/10/2023] [Accepted: 01/08/2024] [Indexed: 01/19/2024] Open
Abstract
The motor cortex not only executes but also prepares movement, as motor cortical neurons exhibit preparatory activity that predicts upcoming movements. In movement preparation, animals adopt different strategies in response to uncertainties existing in nature such as the unknown timing of when a predator will attack-an environmental cue informing "go." However, how motor cortical neurons cope with such uncertainties is less understood. In this study, we aim to investigate whether and how preparatory activity is altered depending on the predictability of "go" timing. We analyze firing activities of the anterior lateral motor cortex in male mice during two auditory delayed-response tasks each with predictable or unpredictable go timing. When go timing is unpredictable, preparatory activities immediately reach and stay in a neural state capable of producing movement anytime to a sudden go cue. When go timing is predictable, preparation activity reaches the movement-producible state more gradually, to secure more accurate decisions. Surprisingly, this preparation process entails a longer reaction time. We find that as preparatory activity increases in accuracy, it takes longer for a neural state to transition from the end of preparation to the start of movement. Our results suggest that the motor cortex fine-tunes preparatory activity for more accurate movement using the predictability of go timing.
Collapse
Affiliation(s)
- Soyoung Chae
- Ulsan National Institute of Science and Technology, Ulsan 44929, South Korea
| | - Jeong-Woo Sohn
- Catholic Kwandong University, Gangwon-do 25601, South Korea
| | - Sung-Phil Kim
- Ulsan National Institute of Science and Technology, Ulsan 44929, South Korea
| |
Collapse
|
11
|
Vahidi P, Sani OG, Shanechi MM. Modeling and dissociation of intrinsic and input-driven neural population dynamics underlying behavior. Proc Natl Acad Sci U S A 2024; 121:e2212887121. [PMID: 38335258 PMCID: PMC10873612 DOI: 10.1073/pnas.2212887121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Accepted: 12/03/2023] [Indexed: 02/12/2024] Open
Abstract
Neural dynamics can reflect intrinsic dynamics or dynamic inputs, such as sensory inputs or inputs from other brain regions. To avoid misinterpreting temporally structured inputs as intrinsic dynamics, dynamical models of neural activity should account for measured inputs. However, incorporating measured inputs remains elusive in joint dynamical modeling of neural-behavioral data, which is important for studying neural computations of behavior. We first show how training dynamical models of neural activity while considering behavior but not input or input but not behavior may lead to misinterpretations. We then develop an analytical learning method for linear dynamical models that simultaneously accounts for neural activity, behavior, and measured inputs. The method provides the capability to prioritize the learning of intrinsic behaviorally relevant neural dynamics and dissociate them from both other intrinsic dynamics and measured input dynamics. In data from a simulated brain with fixed intrinsic dynamics that performs different tasks, the method correctly finds the same intrinsic dynamics regardless of the task while other methods can be influenced by the task. In neural datasets from three subjects performing two different motor tasks with task instruction sensory inputs, the method reveals low-dimensional intrinsic neural dynamics that are missed by other methods and are more predictive of behavior and/or neural activity. The method also uniquely finds that the intrinsic behaviorally relevant neural dynamics are largely similar across the different subjects and tasks, whereas the overall neural dynamics are not. These input-driven dynamical models of neural-behavioral data can uncover intrinsic dynamics that may otherwise be missed.
Collapse
Affiliation(s)
- Parsa Vahidi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA90089
| | - Omid G. Sani
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA90089
| | - Maryam M. Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA90089
- Neuroscience Graduate Program, University of Southern California, Los Angeles, CA90089
- Thomas Lord Department of Computer Science and Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA90089
| |
Collapse
|
12
|
Kuzmina E, Kriukov D, Lebedev M. Neuronal travelling waves explain rotational dynamics in experimental datasets and modelling. Sci Rep 2024; 14:3566. [PMID: 38347042 PMCID: PMC10861525 DOI: 10.1038/s41598-024-53907-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Accepted: 02/06/2024] [Indexed: 02/15/2024] Open
Abstract
Spatiotemporal properties of neuronal population activity in cortical motor areas have been subjects of experimental and theoretical investigations, generating numerous interpretations regarding mechanisms for preparing and executing limb movements. Two competing models, representational and dynamical, strive to explain the relationship between movement parameters and neuronal activity. A dynamical model uses the jPCA method that holistically characterizes oscillatory activity in neuron populations by maximizing the data rotational dynamics. Different rotational dynamics interpretations revealed by the jPCA approach have been proposed. Yet, the nature of such dynamics remains poorly understood. We comprehensively analyzed several neuronal-population datasets and found rotational dynamics consistently accounted for by a traveling wave pattern. For quantifying rotation strength, we developed a complex-valued measure, the gyration number. Additionally, we identified parameters influencing rotation extent in the data. Our findings suggest that rotational dynamics and traveling waves are typically the same phenomena, so reevaluation of the previous interpretations where they were considered separate entities is needed.
Collapse
Affiliation(s)
- Ekaterina Kuzmina
- Skolkovo Institute of Science and Technology, Vladimir Zelman Center for Neurobiology and Brain Rehabilitation, Moscow, Russia, 121205.
- Artificial Intelligence Research Institute (AIRI), Moscow, Russia.
| | - Dmitrii Kriukov
- Artificial Intelligence Research Institute (AIRI), Moscow, Russia
- Skolkovo Institute of Science and Technology, Center for Molecular and Cellular Biology, Moscow, Russia, 121205
| | - Mikhail Lebedev
- Faculty of Mechanics and Mathematics, Lomonosov Moscow State University, Moscow, Russia, 119992
- Sechenov Institute of Evolutionary Physiology and Biochemistry of the Russian Academy of Sciences, Saint-Petersburg, Russia, 194223
| |
Collapse
|
13
|
Zimnik AJ, Cora Ames K, An X, Driscoll L, Lara AH, Russo AA, Susoy V, Cunningham JP, Paninski L, Churchland MM, Glaser JI. Identifying Interpretable Latent Factors with Sparse Component Analysis. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.05.578988. [PMID: 38370650 PMCID: PMC10871230 DOI: 10.1101/2024.02.05.578988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/20/2024]
Abstract
In many neural populations, the computationally relevant signals are posited to be a set of 'latent factors' - signals shared across many individual neurons. Understanding the relationship between neural activity and behavior requires the identification of factors that reflect distinct computational roles. Methods for identifying such factors typically require supervision, which can be suboptimal if one is unsure how (or whether) factors can be grouped into distinct, meaningful sets. Here, we introduce Sparse Component Analysis (SCA), an unsupervised method that identifies interpretable latent factors. SCA seeks factors that are sparse in time and occupy orthogonal dimensions. With these simple constraints, SCA facilitates surprisingly clear parcellations of neural activity across a range of behaviors. We applied SCA to motor cortex activity from reaching and cycling monkeys, single-trial imaging data from C. elegans, and activity from a multitask artificial network. SCA consistently identified sets of factors that were useful in describing network computations.
Collapse
Affiliation(s)
- Andrew J Zimnik
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
| | - K Cora Ames
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
| | - Xinyue An
- Department of Neurology, Northwestern University, Chicago, IL, USA
- Interdepartmental Neuroscience Program, Northwestern University, Chicago, IL, USA
| | - Laura Driscoll
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Allen Institute for Neural Dynamics, Allen Institute, Seattle, CA, USA
| | - Antonio H Lara
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
| | - Abigail A Russo
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
| | - Vladislav Susoy
- Department of Physics, Harvard University, Cambridge, MA, USA
- Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - John P Cunningham
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
- Department of Statistics, Columbia University, New York, NY, USA
| | - Liam Paninski
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
- Department of Statistics, Columbia University, New York, NY, USA
| | - Mark M Churchland
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA
- Kavli Institute for Brain Science, Columbia University Medical Center, New York, NY, USA
| | - Joshua I Glaser
- Department of Neurology, Northwestern University, Chicago, IL, USA
- Department of Computer Science, Northwestern University, Evanston, IL, USA
| |
Collapse
|
14
|
Gort J. Emergence of Universal Computations Through Neural Manifold Dynamics. Neural Comput 2024; 36:227-270. [PMID: 38101328 DOI: 10.1162/neco_a_01631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 09/05/2023] [Indexed: 12/17/2023]
Abstract
There is growing evidence that many forms of neural computation may be implemented by low-dimensional dynamics unfolding at the population scale. However, neither the connectivity structure nor the general capabilities of these embedded dynamical processes are currently understood. In this work, the two most common formalisms of firing-rate models are evaluated using tools from analysis, topology, and nonlinear dynamics in order to provide plausible explanations for these problems. It is shown that low-rank structured connectivities predict the formation of invariant and globally attracting manifolds in all these models. Regarding the dynamics arising in these manifolds, it is proved they are topologically equivalent across the considered formalisms. This letter also shows that under the low-rank hypothesis, the flows emerging in neural manifolds, including input-driven systems, are universal, which broadens previous findings. It explores how low-dimensional orbits can bear the production of continuous sets of muscular trajectories, the implementation of central pattern generators, and the storage of memory states. These dynamics can robustly simulate any Turing machine over arbitrary bounded memory strings, virtually endowing rate models with the power of universal computation. In addition, the letter shows how the low-rank hypothesis predicts the parsimonious correlation structure observed in cortical activity. Finally, it discusses how this theory could provide a useful tool from which to study neuropsychological phenomena using mathematical methods.
Collapse
Affiliation(s)
- Joan Gort
- Facultat de Psicologia, Universitat Autònoma de Barcelona, 08193, Bellaterra, Barcelona, Spain
| |
Collapse
|
15
|
Shinn M. Phantom oscillations in principal component analysis. Proc Natl Acad Sci U S A 2023; 120:e2311420120. [PMID: 37988465 PMCID: PMC10691246 DOI: 10.1073/pnas.2311420120] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 10/18/2023] [Indexed: 11/23/2023] Open
Abstract
Principal component analysis (PCA) is a dimensionality reduction method that is known for being simple and easy to interpret. Principal components are often interpreted as low-dimensional patterns in high-dimensional space. However, this simple interpretation fails for timeseries, spatial maps, and other continuous data. In these cases, nonoscillatory data may have oscillatory principal components. Here, we show that two common properties of data cause oscillatory principal components: smoothness and shifts in time or space. These two properties implicate almost all neuroscience data. We show how the oscillations produced by PCA, which we call "phantom oscillations," impact data analysis. We also show that traditional cross-validation does not detect phantom oscillations, so we suggest procedures that do. Our findings are supported by a collection of mathematical proofs. Collectively, our work demonstrates that patterns which emerge from high-dimensional data analysis may not faithfully represent the underlying data.
Collapse
Affiliation(s)
- Maxwell Shinn
- University College London (UCL) Queen Square Institute of Neurology, University College London, LondonWC1E 6BT, United Kingdom
| |
Collapse
|
16
|
Jarne C, Laje R. Exploring weight initialization, diversity of solutions, and degradation in recurrent neural networks trained for temporal and decision-making tasks. J Comput Neurosci 2023; 51:407-431. [PMID: 37561278 DOI: 10.1007/s10827-023-00857-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Revised: 05/26/2023] [Accepted: 06/27/2023] [Indexed: 08/11/2023]
Abstract
Recurrent Neural Networks (RNNs) are frequently used to model aspects of brain function and structure. In this work, we trained small fully-connected RNNs to perform temporal and flow control tasks with time-varying stimuli. Our results show that different RNNs can solve the same task by converging to different underlying dynamics and also how the performance gracefully degrades as either network size is decreased, interval duration is increased, or connectivity damage is induced. For the considered tasks, we explored how robust the network obtained after training can be according to task parameterization. In the process, we developed a framework that can be useful to parameterize other tasks of interest in computational neuroscience. Our results are useful to quantify different aspects of the models, which are normally used as black boxes and need to be understood in order to model the biological response of cerebral cortex areas.
Collapse
Affiliation(s)
- Cecilia Jarne
- Universidad Nacional de Quilmes, Departamento de Ciencia y Tecnología, Bernal, Buenos Aires, Argentina.
- CONICET, Buenos Aires, Argentina.
- Center for Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.
| | - Rodrigo Laje
- Universidad Nacional de Quilmes, Departamento de Ciencia y Tecnología, Bernal, Buenos Aires, Argentina
- CONICET, Buenos Aires, Argentina
| |
Collapse
|
17
|
Safaie M, Chang JC, Park J, Miller LE, Dudman JT, Perich MG, Gallego JA. Preserved neural dynamics across animals performing similar behaviour. Nature 2023; 623:765-771. [PMID: 37938772 PMCID: PMC10665198 DOI: 10.1038/s41586-023-06714-0] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Accepted: 10/04/2023] [Indexed: 11/09/2023]
Abstract
Animals of the same species exhibit similar behaviours that are advantageously adapted to their body and environment. These behaviours are shaped at the species level by selection pressures over evolutionary timescales. Yet, it remains unclear how these common behavioural adaptations emerge from the idiosyncratic neural circuitry of each individual. The overall organization of neural circuits is preserved across individuals1 because of their common evolutionarily specified developmental programme2-4. Such organization at the circuit level may constrain neural activity5-8, leading to low-dimensional latent dynamics across the neural population9-11. Accordingly, here we suggested that the shared circuit-level constraints within a species would lead to suitably preserved latent dynamics across individuals. We analysed recordings of neural populations from monkey and mouse motor cortex to demonstrate that neural dynamics in individuals from the same species are surprisingly preserved when they perform similar behaviour. Neural population dynamics were also preserved when animals consciously planned future movements without overt behaviour12 and enabled the decoding of planned and ongoing movement across different individuals. Furthermore, we found that preserved neural dynamics extend beyond cortical regions to the dorsal striatum, an evolutionarily older structure13,14. Finally, we used neural network models to demonstrate that behavioural similarity is necessary but not sufficient for this preservation. We posit that these emergent dynamics result from evolutionary constraints on brain development and thus reflect fundamental properties of the neural basis of behaviour.
Collapse
Affiliation(s)
- Mostafa Safaie
- Department of Bioengineering, Imperial College London, London, UK
| | - Joanna C Chang
- Department of Bioengineering, Imperial College London, London, UK
| | - Junchol Park
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, TX, USA
| | - Lee E Miller
- Departments of Physiology, Biomedical Engineering and Physical Medicine and Rehabilitation, Northwestern University and Shirley Ryan Ability Lab, Chicago, IL, USA
| | - Joshua T Dudman
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, TX, USA
| | - Matthew G Perich
- Département de Neurosciences, Faculté de Médecine, Université de Montréal, Montreal, Quebec, Canada.
- Mila, Quebec Artificial Intelligence Institute, Montreal, Quebec, Canada.
| | - Juan A Gallego
- Department of Bioengineering, Imperial College London, London, UK.
| |
Collapse
|
18
|
Stephen EP, Li Y, Metzger S, Oganian Y, Chang EF. Latent neural dynamics encode temporal context in speech. Hear Res 2023; 437:108838. [PMID: 37441880 PMCID: PMC11182421 DOI: 10.1016/j.heares.2023.108838] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 06/15/2023] [Accepted: 07/03/2023] [Indexed: 07/15/2023]
Abstract
Direct neural recordings from human auditory cortex have demonstrated encoding for acoustic-phonetic features of consonants and vowels. Neural responses also encode distinct acoustic amplitude cues related to timing, such as those that occur at the onset of a sentence after a silent period or the onset of the vowel in each syllable. Here, we used a group reduced rank regression model to show that distributed cortical responses support a low-dimensional latent state representation of temporal context in speech. The timing cues each capture more unique variance than all other phonetic features and exhibit rotational or cyclical dynamics in latent space from activity that is widespread over the superior temporal gyrus. We propose that these spatially distributed timing signals could serve to provide temporal context for, and possibly bind across time, the concurrent processing of individual phonetic features, to compose higher-order phonological (e.g. word-level) representations.
Collapse
Affiliation(s)
- Emily P Stephen
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States; Department of Mathematics and Statistics, Boston University, Boston, MA 02215, United States
| | - Yuanning Li
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States; School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Sean Metzger
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States
| | - Yulia Oganian
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States; Center for Integrative Neuroscience, University of Tübingen, Tübingen, Germany
| | - Edward F Chang
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA 94143, United States.
| |
Collapse
|
19
|
Athalye VR, Khanna P, Gowda S, Orsborn AL, Costa RM, Carmena JM. Invariant neural dynamics drive commands to control different movements. Curr Biol 2023; 33:2962-2976.e15. [PMID: 37402376 PMCID: PMC10527529 DOI: 10.1016/j.cub.2023.06.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 04/24/2023] [Accepted: 06/09/2023] [Indexed: 07/06/2023]
Abstract
It has been proposed that the nervous system has the capacity to generate a wide variety of movements because it reuses some invariant code. Previous work has identified that dynamics of neural population activity are similar during different movements, where dynamics refer to how the instantaneous spatial pattern of population activity changes in time. Here, we test whether invariant dynamics of neural populations are actually used to issue the commands that direct movement. Using a brain-machine interface (BMI) that transforms rhesus macaques' motor-cortex activity into commands for a neuroprosthetic cursor, we discovered that the same command is issued with different neural-activity patterns in different movements. However, these different patterns were predictable, as we found that the transitions between activity patterns are governed by the same dynamics across movements. These invariant dynamics are low dimensional, and critically, they align with the BMI, so that they predict the specific component of neural activity that actually issues the next command. We introduce a model of optimal feedback control (OFC) that shows that invariant dynamics can help transform movement feedback into commands, reducing the input that the neural population needs to control movement. Altogether our results demonstrate that invariant dynamics drive commands to control a variety of movements and show how feedback can be integrated with invariant dynamics to issue generalizable commands.
Collapse
Affiliation(s)
- Vivek R Athalye
- Zuckerman Mind Brain Behavior Institute, Departments of Neuroscience and Neurology, Columbia University, New York, NY 10027, USA.
| | - Preeya Khanna
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94158, USA.
| | - Suraj Gowda
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Amy L Orsborn
- Departments of Bioengineering, Electrical and Computer Engineering, University of Washington, Seattle, Seattle, WA 98195, USA
| | - Rui M Costa
- Zuckerman Mind Brain Behavior Institute, Departments of Neuroscience and Neurology, Columbia University, New York, NY 10027, USA.
| | - Jose M Carmena
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA 94720, USA; Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA; UC Berkeley-UCSF Joint Graduate Program in Bioengineering, University of California, Berkeley, Berkeley, CA 94720, USA.
| |
Collapse
|
20
|
Schmidt MD, Glasmachers T, Iossifidis I. The concepts of muscle activity generation driven by upper limb kinematics. Biomed Eng Online 2023; 22:63. [PMID: 37355651 DOI: 10.1186/s12938-023-01116-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 05/16/2023] [Indexed: 06/26/2023] Open
Abstract
BACKGROUND The underlying motivation of this work is to demonstrate that artificial muscle activity of known and unknown motion can be generated based on motion parameters, such as angular position, acceleration, and velocity of each joint (or the end-effector instead), which are similarly represented in our brains. This model is motivated by the known motion planning process in the central nervous system. That process incorporates the current body state from sensory systems and previous experiences, which might be represented as pre-learned inverse dynamics that generate associated muscle activity. METHODS We develop a novel approach utilizing recurrent neural networks that are able to predict muscle activity of the upper limbs associated with complex 3D human arm motions. Therefore, motion parameters such as joint angle, velocity, acceleration, hand position, and orientation, serve as input for the models. In addition, these models are trained on multiple subjects (n=5 including , 3 male in the age of 26±2 years) and thus can generalize across individuals. In particular, we distinguish between a general model that has been trained on several subjects, a subject-specific model, and a specific fine-tuned model using a transfer learning approach to adapt the model to a new subject. Estimators such as mean square error MSE, correlation coefficient r, and coefficient of determination R2 are used to evaluate the goodness of fit. We additionally assess performance by developing a new score called the zero-line score. The present approach was compared with multiple other architectures. RESULTS The presented approach predicts the muscle activity for previously through different subjects with remarkable high precision and generalizing nicely for new motions that have not been trained before. In an exhausting comparison, our recurrent network outperformed all other architectures. In addition, the high inter-subject variation of the recorded muscle activity was successfully handled using a transfer learning approach, resulting in a good fit for the muscle activity for a new subject. CONCLUSIONS The ability of this approach to efficiently predict muscle activity contributes to the fundamental understanding of motion control. Furthermore, this approach has great potential for use in rehabilitation contexts, both as a therapeutic approach and as an assistive device. The predicted muscle activity can be utilized to guide functional electrical stimulation, allowing specific muscles to be targeted and potentially improving overall rehabilitation outcomes.
Collapse
Affiliation(s)
- Marie D Schmidt
- Faculty of Electrical Engineering and Information Technology, Ruhr-University Bochum, Bochum, Germany.
- Institute of Computer Science, University of Applied Science Ruhr West, Mülheim an der Ruhr, Germany.
| | | | - Ioannis Iossifidis
- Institute of Computer Science, University of Applied Science Ruhr West, Mülheim an der Ruhr, Germany
| |
Collapse
|
21
|
Bachschmid-Romano L, Hatsopoulos NG, Brunel N. Interplay between external inputs and recurrent dynamics during movement preparation and execution in a network model of motor cortex. eLife 2023; 12:77690. [PMID: 37166452 PMCID: PMC10174693 DOI: 10.7554/elife.77690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 03/09/2023] [Indexed: 05/12/2023] Open
Abstract
The primary motor cortex has been shown to coordinate movement preparation and execution through computations in approximately orthogonal subspaces. The underlying network mechanisms, and the roles played by external and recurrent connectivity, are central open questions that need to be answered to understand the neural substrates of motor control. We develop a recurrent neural network model that recapitulates the temporal evolution of neuronal activity recorded from the primary motor cortex of a macaque monkey during an instructed delayed-reach task. In particular, it reproduces the observed dynamic patterns of covariation between neural activity and the direction of motion. We explore the hypothesis that the observed dynamics emerges from a synaptic connectivity structure that depends on the preferred directions of neurons in both preparatory and movement-related epochs, and we constrain the strength of both synaptic connectivity and external input parameters from data. While the model can reproduce neural activity for multiple combinations of the feedforward and recurrent connections, the solution that requires minimum external inputs is one where the observed patterns of covariance are shaped by external inputs during movement preparation, while they are dominated by strong direction-specific recurrent connectivity during movement execution. Our model also demonstrates that the way in which single-neuron tuning properties change over time can explain the level of orthogonality of preparatory and movement-related subspaces.
Collapse
Affiliation(s)
| | - Nicholas G Hatsopoulos
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, United States
- Committee on Computational Neuroscience, University of Chicago, Chicago, United States
| | - Nicolas Brunel
- Department of Neurobiology, Duke University, Durham, United States
- Department of Physics, Duke University, Durham, United States
- Duke Institute for Brain Sciences, Duke University, Durham, United States
- Center for Cognitive Neuroscience, Duke University, Durham, United States
| |
Collapse
|
22
|
Haggie L, Schmid L, Röhrle O, Besier T, McMorland A, Saini H. Linking cortex and contraction-Integrating models along the corticomuscular pathway. Front Physiol 2023; 14:1095260. [PMID: 37234419 PMCID: PMC10206006 DOI: 10.3389/fphys.2023.1095260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Accepted: 04/21/2023] [Indexed: 05/28/2023] Open
Abstract
Computational models of the neuromusculoskeletal system provide a deterministic approach to investigate input-output relationships in the human motor system. Neuromusculoskeletal models are typically used to estimate muscle activations and forces that are consistent with observed motion under healthy and pathological conditions. However, many movement pathologies originate in the brain, including stroke, cerebral palsy, and Parkinson's disease, while most neuromusculoskeletal models deal exclusively with the peripheral nervous system and do not incorporate models of the motor cortex, cerebellum, or spinal cord. An integrated understanding of motor control is necessary to reveal underlying neural-input and motor-output relationships. To facilitate the development of integrated corticomuscular motor pathway models, we provide an overview of the neuromusculoskeletal modelling landscape with a focus on integrating computational models of the motor cortex, spinal cord circuitry, α-motoneurons and skeletal muscle in regard to their role in generating voluntary muscle contraction. Further, we highlight the challenges and opportunities associated with an integrated corticomuscular pathway model, such as challenges in defining neuron connectivities, modelling standardisation, and opportunities in applying models to study emergent behaviour. Integrated corticomuscular pathway models have applications in brain-machine-interaction, education, and our understanding of neurological disease.
Collapse
Affiliation(s)
- Lysea Haggie
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| | - Laura Schmid
- Institute for Modelling and Simulation of Biomechanical Systems, University of Stuttgart, Stuttgart, Germany
| | - Oliver Röhrle
- Institute for Modelling and Simulation of Biomechanical Systems, University of Stuttgart, Stuttgart, Germany
- Stuttgart Center for Simulation Sciences (SC SimTech), University of Stuttgart, Stuttgart, Germany
| | - Thor Besier
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| | - Angus McMorland
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
- Department of Exercise Sciences, University of Auckland, Auckland, New Zealand
| | - Harnoor Saini
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| |
Collapse
|
23
|
Disse GD, Nandakumar B, Pauzin FP, Blumenthal GH, Kong Z, Ditterich J, Moxon KA. Neural ensemble dynamics in trunk and hindlimb sensorimotor cortex encode for the control of postural stability. Cell Rep 2023; 42:112347. [PMID: 37027302 DOI: 10.1016/j.celrep.2023.112347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Revised: 02/09/2023] [Accepted: 03/21/2023] [Indexed: 04/08/2023] Open
Abstract
The cortex has a disputed role in monitoring postural equilibrium and intervening in cases of major postural disturbances. Here, we investigate the patterns of neural activity in the cortex that underlie neural dynamics during unexpected perturbations. In both the primary sensory (S1) and motor (M1) cortices of the rat, unique neuronal classes differentially covary their responses to distinguish different characteristics of applied postural perturbations; however, there is substantial information gain in M1, demonstrating a role for higher-order computations in motor control. A dynamical systems model of M1 activity and forces generated by the limbs reveals that these neuronal classes contribute to a low-dimensional manifold comprised of separate subspaces enabled by congruent and incongruent neural firing patterns that define different computations depending on the postural responses. These results inform how the cortex engages in postural control, directing work aiming to understand postural instability after neurological disease.
Collapse
Affiliation(s)
- Gregory D Disse
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA
| | | | - Francois P Pauzin
- Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA
| | - Gary H Blumenthal
- School of Biomedical Engineering Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA
| | - Zhaodan Kong
- Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA 95616, USA
| | - Jochen Ditterich
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Neurobiology, Physiology and Behavior, University of California, Davis, Davis, CA 95616, USA
| | - Karen A Moxon
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA.
| |
Collapse
|
24
|
Marciniak Dg Agra K, Dg Agra P. F = ma. Is the macaque brain Newtonian? Cogn Neuropsychol 2023; 39:376-408. [PMID: 37045793 DOI: 10.1080/02643294.2023.2191843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/14/2023]
Abstract
Intuitive Physics, the ability to anticipate how the physical events involving mass objects unfold in time and space, is a central component of intelligent systems. Intuitive physics is a promising tool for gaining insight into mechanisms that generalize across species because both humans and non-human primates are subject to the same physical constraints when engaging with the environment. Physical reasoning abilities are widely present within the animal kingdom, but monkeys, with acute 3D vision and a high level of dexterity, appreciate and manipulate the physical world in much the same way humans do.
Collapse
Affiliation(s)
- Karolina Marciniak Dg Agra
- The Rockefeller University, Laboratory of Neural Circuits, New York, NY, USA
- Center for Brain, Minds and Machines, Cambridge, MA, USA
| | - Pedro Dg Agra
- The Rockefeller University, Laboratory of Neural Circuits, New York, NY, USA
- Center for Brain, Minds and Machines, Cambridge, MA, USA
| |
Collapse
|
25
|
Vahidi P, Sani OG, Shanechi MM. Modeling and dissociation of intrinsic and input-driven neural population dynamics underlying behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.03.14.532554. [PMID: 36993213 PMCID: PMC10055042 DOI: 10.1101/2023.03.14.532554] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Neural dynamics can reflect intrinsic dynamics or dynamic inputs, such as sensory inputs or inputs from other regions. To avoid misinterpreting temporally-structured inputs as intrinsic dynamics, dynamical models of neural activity should account for measured inputs. However, incorporating measured inputs remains elusive in joint dynamical modeling of neural-behavioral data, which is important for studying neural computations of a specific behavior. We first show how training dynamical models of neural activity while considering behavior but not input, or input but not behavior may lead to misinterpretations. We then develop a novel analytical learning method that simultaneously accounts for neural activity, behavior, and measured inputs. The method provides the new capability to prioritize the learning of intrinsic behaviorally relevant neural dynamics and dissociate them from both other intrinsic dynamics and measured input dynamics. In data from a simulated brain with fixed intrinsic dynamics that performs different tasks, the method correctly finds the same intrinsic dynamics regardless of task while other methods can be influenced by the change in task. In neural datasets from three subjects performing two different motor tasks with task instruction sensory inputs, the method reveals low-dimensional intrinsic neural dynamics that are missed by other methods and are more predictive of behavior and/or neural activity. The method also uniquely finds that the intrinsic behaviorally relevant neural dynamics are largely similar across the three subjects and two tasks whereas the overall neural dynamics are not. These input-driven dynamical models of neural-behavioral data can uncover intrinsic dynamics that may otherwise be missed.
Collapse
|
26
|
DePasquale B, Sussillo D, Abbott LF, Churchland MM. The centrality of population-level factors to network computation is demonstrated by a versatile approach for training spiking networks. Neuron 2023; 111:631-649.e10. [PMID: 36630961 PMCID: PMC10118067 DOI: 10.1016/j.neuron.2022.12.007] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 06/17/2022] [Accepted: 12/05/2022] [Indexed: 01/12/2023]
Abstract
Neural activity is often described in terms of population-level factors extracted from the responses of many neurons. Factors provide a lower-dimensional description with the aim of shedding light on network computations. Yet, mechanistically, computations are performed not by continuously valued factors but by interactions among neurons that spike discretely and variably. Models provide a means of bridging these levels of description. We developed a general method for training model networks of spiking neurons by leveraging factors extracted from either data or firing-rate-based networks. In addition to providing a useful model-building framework, this formalism illustrates how reliable and continuously valued factors can arise from seemingly stochastic spiking. Our framework establishes procedures for embedding this property in network models with different levels of realism. The relationship between spikes and factors in such networks provides a foundation for interpreting (and subtly redefining) commonly used quantities such as firing rates.
Collapse
Affiliation(s)
- Brian DePasquale
- Princeton Neuroscience Institute, Princeton University, Princeton NJ, USA; Department of Neuroscience, Columbia University, New York, NY, USA; Center for Theoretical Neuroscience, Columbia University, New York, NY, USA.
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - L F Abbott
- Department of Neuroscience, Columbia University, New York, NY, USA; Center for Theoretical Neuroscience, Columbia University, New York, NY, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA; Department of Physiology and Cellular Biophysics, Columbia University, New York, NY, USA; Kavli Institute for Brain Science, Columbia University, New York, NY, USA
| | - Mark M Churchland
- Department of Neuroscience, Columbia University, New York, NY, USA; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA; Kavli Institute for Brain Science, Columbia University, New York, NY, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA
| |
Collapse
|
27
|
Varley TF, Sporns O, Schaffelhofer S, Scherberger H, Dann B. Information-processing dynamics in neural networks of macaque cerebral cortex reflect cognitive state and behavior. Proc Natl Acad Sci U S A 2023; 120:e2207677120. [PMID: 36603032 PMCID: PMC9926243 DOI: 10.1073/pnas.2207677120] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 11/29/2022] [Indexed: 01/06/2023] Open
Abstract
One of the essential functions of biological neural networks is the processing of information. This includes everything from processing sensory information to perceive the environment, up to processing motor information to interact with the environment. Due to methodological limitations, it has been historically unclear how information processing changes during different cognitive or behavioral states and to what extent information is processed within or between the network of neurons in different brain areas. In this study, we leverage recent advances in the calculation of information dynamics to explore neural-level processing within and between the frontoparietal areas AIP, F5, and M1 during a delayed grasping task performed by three macaque monkeys. While information processing was high within all areas during all cognitive and behavioral states of the task, interareal processing varied widely: During visuomotor transformation, AIP and F5 formed a reciprocally connected processing unit, while no processing was present between areas during the memory period. Movement execution was processed globally across all areas with predominance of processing in the feedback direction. Furthermore, the fine-scale network structure reconfigured at the neuron level in response to different grasping conditions, despite no differences in the overall amount of information present. These results suggest that areas dynamically form higher-order processing units according to the cognitive or behavioral demand and that the information-processing network is hierarchically organized at the neuron level, with the coarse network structure determining the behavioral state and finer changes reflecting different conditions.
Collapse
Affiliation(s)
- Thomas F. Varley
- Department of Psychological & Brain Sciences, Indiana University47405-7007, Bloomington, IN
- School of Informatics, Computing, and Engineering, Indiana University47405-7007, Bloomington, IN
| | - Olaf Sporns
- Department of Psychological & Brain Sciences, Indiana University47405-7007, Bloomington, IN
| | - Stefan Schaffelhofer
- Neurobiology Laboratory, German Primate Center37077, Goettingen, Germany
- Faculty of Biology and Psychology, University of Goettingen37073, Goettingen, Germany
| | - Hansjörg Scherberger
- Neurobiology Laboratory, German Primate Center37077, Goettingen, Germany
- Faculty of Biology and Psychology, University of Goettingen37073, Goettingen, Germany
| | - Benjamin Dann
- Neurobiology Laboratory, German Primate Center37077, Goettingen, Germany
| |
Collapse
|
28
|
Brennan C, Aggarwal A, Pei R, Sussillo D, Proekt A. One dimensional approximations of neuronal dynamics reveal computational strategy. PLoS Comput Biol 2023; 19:e1010784. [PMID: 36607933 PMCID: PMC9821456 DOI: 10.1371/journal.pcbi.1010784] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Accepted: 12/01/2022] [Indexed: 01/07/2023] Open
Abstract
The relationship between neuronal activity and computations embodied by it remains an open question. We develop a novel methodology that condenses observed neuronal activity into a quantitatively accurate, simple, and interpretable model and validate it on diverse systems and scales from single neurons in C. elegans to fMRI in humans. The model treats neuronal activity as collections of interlocking 1-dimensional trajectories. Despite their simplicity, these models accurately predict future neuronal activity and future decisions made by human participants. Moreover, the structure formed by interconnected trajectories-a scaffold-is closely related to the computational strategy of the system. We use these scaffolds to compare the computational strategy of primates and artificial systems trained on the same task to identify specific conditions under which the artificial agent learns the same strategy as the primate. The computational strategy extracted using our methodology predicts specific errors on novel stimuli. These results show that our methodology is a powerful tool for studying the relationship between computation and neuronal activity across diverse systems.
Collapse
Affiliation(s)
- Connor Brennan
- Department of Neuroscience, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America
| | - Adeeti Aggarwal
- Department of Neuroscience, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America
| | - Rui Pei
- Department of Psychology, Stanford University, Palo Alto, California, United States of America
| | - David Sussillo
- Stanford Neurosciences Institute, Stanford University, Palo Alto, California, United States of America
- Department of Electrical Engineering, Stanford University, Palo Alto, California, United States of America
| | - Alex Proekt
- Department of Anesthesiology and Critical Care, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America
| |
Collapse
|
29
|
D'Aleo R, Rouse AG, Schieber MH, Sarma SV. Cortico-cortical drive in a coupled premotor-primary motor cortex dynamical system. Cell Rep 2022; 41:111849. [PMID: 36543147 PMCID: PMC11271678 DOI: 10.1016/j.celrep.2022.111849] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 06/13/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
In the conventional view of sensorimotor control, the premotor cortex (PM) plans actions that are executed by the primary motor cortex (M1). This notion arises in part from many experiments that have imposed a preparatory "planning" period, during which PM becomes active without M1. But during many natural movements, PM and M1 are co-activated, making it difficult to distinguish their functional roles. We leverage coupled dynamical systems models (cDSMs) to uncover interactions between PM and M1 during movements performed with no preparatory period. We build cDSMs using neural and behavioral data recorded from two non-human primates as they performed a reach-grasp-manipulate task. PM and M1 interact dynamically throughout these movements. Whereas PM drives the M1 in some situations, in other situations, M1 drives PM activity, contrary to the conventional assumption. Our DSM framework provides additional predictions differentiating the roles of PM and M1 in controlling movement.
Collapse
Affiliation(s)
- Raina D'Aleo
- Department of Neuroscience, The Johns Hopkins University, Baltimore, MD 21218, USA; Institute for Computational Medicine, Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA.
| | - Adam G Rouse
- Department of Neurosurgery, University of Kansas, Kansas City, KS 66160, USA
| | - Marc H Schieber
- Department of Neuroscience, University of Rochester, Rochester, NY 14642, USA; Department of Neurology, University of Rochester, Rochester, NY 14642, USA
| | - Sridevi V Sarma
- Institute for Computational Medicine, Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA.
| |
Collapse
|
30
|
Zhang X, Long X, Zhang SJ, Chen ZS. Excitatory-inhibitory recurrent dynamics produce robust visual grids and stable attractors. Cell Rep 2022; 41:111777. [PMID: 36516752 PMCID: PMC9805366 DOI: 10.1016/j.celrep.2022.111777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Revised: 09/28/2022] [Accepted: 11/14/2022] [Indexed: 12/15/2022] Open
Abstract
Spatially modulated grid cells have been recently found in the rat secondary visual cortex (V2) during active navigation. However, the computational mechanism and functional significance of V2 grid cells remain unknown. To address the knowledge gap, we train a biologically inspired excitatory-inhibitory recurrent neural network to perform a two-dimensional spatial navigation task with multisensory input. We find grid-like responses in both excitatory and inhibitory RNN units, which are robust with respect to spatial cues, dimensionality of visual input, and activation function. Population responses reveal a low-dimensional, torus-like manifold and attractor. We find a link between functional grid clusters with similar receptive fields and structured excitatory-to-excitatory connections. Additionally, multistable torus-like attractors emerged with increasing sparsity in inter- and intra-subnetwork connectivity. Finally, irregular grid patterns are found in recurrent neural network (RNN) units during a visual sequence recognition task. Together, our results suggest common computational mechanisms of V2 grid cells for spatial and non-spatial tasks.
Collapse
Affiliation(s)
- Xiaohan Zhang
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY, USA
| | - Xiaoyang Long
- Department of Neurosurgery, Xinqiao Hospital, Chongqing, China
| | - Sheng-Jia Zhang
- Department of Neurosurgery, Xinqiao Hospital, Chongqing, China
| | - Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY, USA; Department of Neurosurgery, Xinqiao Hospital, Chongqing, China; Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, USA.
| |
Collapse
|
31
|
Melbaum S, Russo E, Eriksson D, Schneider A, Durstewitz D, Brox T, Diester I. Conserved structures of neural activity in sensorimotor cortex of freely moving rats allow cross-subject decoding. Nat Commun 2022; 13:7420. [PMID: 36456557 PMCID: PMC9715555 DOI: 10.1038/s41467-022-35115-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Accepted: 11/17/2022] [Indexed: 12/04/2022] Open
Abstract
Our knowledge about neuronal activity in the sensorimotor cortex relies primarily on stereotyped movements that are strictly controlled in experimental settings. It remains unclear how results can be carried over to less constrained behavior like that of freely moving subjects. Toward this goal, we developed a self-paced behavioral paradigm that encouraged rats to engage in different movement types. We employed bilateral electrophysiological recordings across the entire sensorimotor cortex and simultaneous paw tracking. These techniques revealed behavioral coupling of neurons with lateralization and an anterior-posterior gradient from the premotor to the primary sensory cortex. The structure of population activity patterns was conserved across animals despite the severe under-sampling of the total number of neurons and variations in electrode positions across individuals. We demonstrated cross-subject and cross-session generalization in a decoding task through alignments of low-dimensional neural manifolds, providing evidence of a conserved neuronal code.
Collapse
Affiliation(s)
- Svenja Melbaum
- grid.5963.9Computer Vision Group, Department of Computer Science, University of Freiburg, 79110 Freiburg, Germany ,grid.5963.9IMBIT//BrainLinks-BrainTools, University of Freiburg, Georges-Köhler-Allee 201, 79110 Freiburg, Germany
| | - Eleonora Russo
- grid.410607.4Department of Psychiatry and Psychotherapy, University Medical Center, Johannes Gutenberg University, 55131 Mainz, Germany ,grid.7700.00000 0001 2190 4373Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, 68159 Mannheim, Germany
| | - David Eriksson
- grid.5963.9IMBIT//BrainLinks-BrainTools, University of Freiburg, Georges-Köhler-Allee 201, 79110 Freiburg, Germany ,grid.5963.9Optophysiology Lab, Faculty of Biology, University of Freiburg, 79110 Freiburg, Germany
| | - Artur Schneider
- grid.5963.9IMBIT//BrainLinks-BrainTools, University of Freiburg, Georges-Köhler-Allee 201, 79110 Freiburg, Germany ,grid.5963.9Optophysiology Lab, Faculty of Biology, University of Freiburg, 79110 Freiburg, Germany
| | - Daniel Durstewitz
- grid.7700.00000 0001 2190 4373Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, 68159 Mannheim, Germany
| | - Thomas Brox
- grid.5963.9Computer Vision Group, Department of Computer Science, University of Freiburg, 79110 Freiburg, Germany ,grid.5963.9IMBIT//BrainLinks-BrainTools, University of Freiburg, Georges-Köhler-Allee 201, 79110 Freiburg, Germany
| | - Ilka Diester
- grid.5963.9IMBIT//BrainLinks-BrainTools, University of Freiburg, Georges-Köhler-Allee 201, 79110 Freiburg, Germany ,grid.5963.9Optophysiology Lab, Faculty of Biology, University of Freiburg, 79110 Freiburg, Germany ,grid.5963.9Bernstein Center Freiburg, University of Freiburg, 79104 Freiburg, Germany
| |
Collapse
|
32
|
Rajalingham R, Piccato A, Jazayeri M. Recurrent neural networks with explicit representation of dynamic latent variables can mimic behavioral patterns in a physical inference task. Nat Commun 2022; 13:5865. [PMID: 36195614 PMCID: PMC9532407 DOI: 10.1038/s41467-022-33581-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Accepted: 09/22/2022] [Indexed: 11/09/2022] Open
Abstract
Primates can richly parse sensory inputs to infer latent information. This ability is hypothesized to rely on establishing mental models of the external world and running mental simulations of those models. However, evidence supporting this hypothesis is limited to behavioral models that do not emulate neural computations. Here, we test this hypothesis by directly comparing the behavior of primates (humans and monkeys) in a ball interception task to that of a large set of recurrent neural network (RNN) models with or without the capacity to dynamically track the underlying latent variables. Humans and monkeys exhibit similar behavioral patterns. This primate behavioral pattern is best captured by RNNs endowed with dynamic inference, consistent with the hypothesis that the primate brain uses dynamic inferences to support flexible physical predictions. Moreover, our work highlights a general strategy for using model neural systems to test computational hypotheses of higher brain function.
Collapse
Affiliation(s)
- Rishi Rajalingham
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA
| | - Aída Piccato
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA.,Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139-4307, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA. .,Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139-4307, USA.
| |
Collapse
|
33
|
Vaccari FE, Diomedi S, Filippini M, Hadjidimitrakis K, Fattori P. New insights on single-neuron selectivity in the era of population-level approaches. Front Integr Neurosci 2022; 16:929052. [PMID: 36249900 PMCID: PMC9554653 DOI: 10.3389/fnint.2022.929052] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Accepted: 09/02/2022] [Indexed: 11/13/2022] Open
Abstract
In the past, neuroscience was focused on individual neurons seen as the functional units of the nervous system, but this approach fell short over time to account for new experimental evidence, especially for what concerns associative and motor cortices. For this reason and thanks to great technological advances, a part of modern research has shifted the focus from the responses of single neurons to the activity of neural ensembles, now considered the real functional units of the system. However, on a microscale, individual neurons remain the computational components of these networks, thus the study of population dynamics cannot prescind from studying also individual neurons which represent their natural substrate. In this new framework, ideas such as the capability of single cells to encode a specific stimulus (neural selectivity) may become obsolete and need to be profoundly revised. One step in this direction was made by introducing the concept of “mixed selectivity,” the capacity of single cells to integrate multiple variables in a flexible way, allowing individual neurons to participate in different networks. In this review, we outline the most important features of mixed selectivity and we also present recent works demonstrating its presence in the associative areas of the posterior parietal cortex. Finally, in discussing these findings, we present some open questions that could be addressed by future studies.
Collapse
Affiliation(s)
| | - Stefano Diomedi
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
| | - Matteo Filippini
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
- Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, Italy
- *Correspondence: Patrizia Fattori
| | | | - Patrizia Fattori
- Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
- Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, Italy
- Matteo Filippini
| |
Collapse
|
34
|
Ramezanian-Panahi M, Abrevaya G, Gagnon-Audet JC, Voleti V, Rish I, Dumas G. Generative Models of Brain Dynamics. Front Artif Intell 2022; 5:807406. [PMID: 35910192 PMCID: PMC9335006 DOI: 10.3389/frai.2022.807406] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 06/10/2022] [Indexed: 01/28/2023] Open
Abstract
This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.
Collapse
Affiliation(s)
| | - Germán Abrevaya
- Mila-Quebec AI Institute, Montréal, QC, Canada
- Departamento de Física, Facultad de Ciencias Exactas y Naturales, Instituto de Física de Buenos Aires (IFIBA), CONICET, Universidad de Buenos Aires, Buenos Aires, Argentina
| | | | - Vikram Voleti
- Mila-Quebec AI Institute, Montréal, QC, Canada
- Université de Montréal, Montréal, QC, Canada
| | - Irina Rish
- Mila-Quebec AI Institute, Montréal, QC, Canada
- Université de Montréal, Montréal, QC, Canada
| | - Guillaume Dumas
- Mila-Quebec AI Institute, Montréal, QC, Canada
- Université de Montréal, Montréal, QC, Canada
- Department of Psychiatry, CHU Sainte-Justine Research Center, Mila-Quebec AI Institute, Université de Montréal, Montréal, QC, Canada
| |
Collapse
|
35
|
Saxena S, Russo AA, Cunningham J, Churchland MM. Motor cortex activity across movement speeds is predicted by network-level strategies for generating muscle activity. eLife 2022; 11:67620. [PMID: 35621264 PMCID: PMC9197394 DOI: 10.7554/elife.67620] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Accepted: 05/26/2022] [Indexed: 12/02/2022] Open
Abstract
Learned movements can be skillfully performed at different paces. What neural strategies produce this flexibility? Can they be predicted and understood by network modeling? We trained monkeys to perform a cycling task at different speeds, and trained artificial recurrent networks to generate the empirical muscle-activity patterns. Network solutions reflected the principle that smooth well-behaved dynamics require low trajectory tangling. Network solutions had a consistent form, which yielded quantitative and qualitative predictions. To evaluate predictions, we analyzed motor cortex activity recorded during the same task. Responses supported the hypothesis that the dominant neural signals reflect not muscle activity, but network-level strategies for generating muscle activity. Single-neuron responses were better accounted for by network activity than by muscle activity. Similarly, neural population trajectories shared their organization not with muscle trajectories, but with network solutions. Thus, cortical activity could be understood based on the need to generate muscle activity via dynamics that allow smooth, robust control over movement speed.
Collapse
Affiliation(s)
- Shreya Saxena
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, United States.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States.,Grossman Center for the Statistics of Mind, Columbia University, New York, United States.,Center for Theoretical Neuroscience, Columbia University, New York, United States.,Department of Statistics, Columbia University, New York, United States
| | - Abigail A Russo
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States.,Department of Neuroscience, Columbia University, New York, United States
| | - John Cunningham
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States.,Grossman Center for the Statistics of Mind, Columbia University, New York, United States.,Center for Theoretical Neuroscience, Columbia University, New York, United States.,Department of Statistics, Columbia University, New York, United States
| | - Mark M Churchland
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States.,Grossman Center for the Statistics of Mind, Columbia University, New York, United States.,Department of Neuroscience, Columbia University, New York, United States.,Kavli Institute for Brain Science, Columbia University, New York, United States
| |
Collapse
|
36
|
Wang T, Chen Y, Cui H. From Parametric Representation to Dynamical System: Shifting Views of the Motor Cortex in Motor Control. Neurosci Bull 2022; 38:796-808. [PMID: 35298779 PMCID: PMC9276910 DOI: 10.1007/s12264-022-00832-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Accepted: 11/29/2021] [Indexed: 11/01/2022] Open
Abstract
In contrast to traditional representational perspectives in which the motor cortex is involved in motor control via neuronal preference for kinetics and kinematics, a dynamical system perspective emerging in the last decade views the motor cortex as a dynamical machine that generates motor commands by autonomous temporal evolution. In this review, we first look back at the history of the representational and dynamical perspectives and discuss their explanatory power and controversy from both empirical and computational points of view. Here, we aim to reconcile the above perspectives, and evaluate their theoretical impact, future direction, and potential applications in brain-machine interfaces.
Collapse
Affiliation(s)
- Tianwei Wang
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China.,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China.,University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Yun Chen
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China.,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China.,University of Chinese Academy of Sciences, Beijing, 100049, China
| | - He Cui
- Center for Excellence in Brain Science and Intelligent Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, 200031, China. .,Shanghai Center for Brain and Brain-inspired Intelligence Technology, Shanghai, 200031, China. .,University of Chinese Academy of Sciences, Beijing, 100049, China.
| |
Collapse
|
37
|
Investigation of Neural Substrates of Erroneous Behavior in a Delayed-Response Task. eNeuro 2022; 9:ENEURO.0490-21.2022. [PMID: 35365501 PMCID: PMC9007410 DOI: 10.1523/eneuro.0490-21.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 03/08/2022] [Accepted: 03/24/2022] [Indexed: 11/29/2022] Open
Abstract
Motor cortical neurons exhibit persistent selective activities (selectivity) during motor planning. Experimental perturbation of selectivity results in the failure of short-term memory retention and consequent behavioral biases, demonstrating selectivity as a neural characteristic of encoding previous sensory input or future action. However, even without experimental manipulation, animals occasionally fail to maintain short-term memory leading to erroneous choice. Here, we investigated neural substrates that lead to the incorrect formation of selectivity during short-term memory. We analyzed neuronal activities in anterior lateral motor cortex (ALM) of mice, a region known to be engaged in motor planning while mice performed the tactile delayed-response task. We found that highly selective neurons lost their selectivity while originally nonselective neurons showed selectivity during the error trials where mice licked toward incorrect direction. We assumed that those alternations would reflect changes in intrinsic properties of population activity. Thus, we estimated an intrinsic manifold shared by neuronal population (shared space), using factor analysis (FA) and measured the association of individual neurons with the shared space by communality, the variance of neuronal activity accounted for by the shared space. We found a positive correlation between selectivity and communality over ALM neurons, which disappeared in erroneous behavior. Notably, neurons showing selectivity alternations between correct and incorrect licking also underwent proportional changes in communality. Our results demonstrated that the extent to which an ALM neuron is associated with the intrinsic manifolds of population activity may elucidate its selectivity and that disruption of this association may alter selectivity, likely leading to erroneous behavior.
Collapse
|
38
|
Cyclic, Condition-Independent Activity in Primary Motor Cortex Predicts Corrective Movement Behavior. eNeuro 2022; 9:ENEURO.0354-21.2022. [PMID: 35346960 PMCID: PMC9014981 DOI: 10.1523/eneuro.0354-21.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 02/23/2022] [Accepted: 03/16/2022] [Indexed: 11/21/2022] Open
Abstract
Reaching movements are known to have large condition-independent (CI) neural activity and cyclic neural dynamics. A new precision center-out task was performed by rhesus macaques to test the hypothesis that cyclic, CI neural activity in the primary motor cortex (M1) occurs not only during initial reaching movements but also during subsequent corrective movements. Corrective movements were observed to be discrete with time courses and bell-shaped speed profiles similar to the initial movements. CI cyclic neural trajectories were similar and repeated for initial and each additional corrective submovement. The phase of the cyclic CI neural activity predicted the time of peak movement speed more accurately than regression of instantaneous firing rate, even when the subject made multiple corrective movements. Rather than being controlled as continuations of the initial reach, a discrete cycle of motor cortex activity encodes each corrective submovement.
Collapse
|
39
|
Schroeder KE, Perkins SM, Wang Q, Churchland MM. Cortical Control of Virtual Self-Motion Using Task-Specific Subspaces. J Neurosci 2022; 42:220-239. [PMID: 34716229 PMCID: PMC8802935 DOI: 10.1523/jneurosci.2687-20.2021] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 09/18/2021] [Accepted: 10/17/2021] [Indexed: 11/21/2022] Open
Abstract
Brain-machine interfaces (BMIs) for reaching have enjoyed continued performance improvements, yet there remains significant need for BMIs that control other movement classes. Recent scientific findings suggest that the intrinsic covariance structure of neural activity depends strongly on movement class, potentially necessitating different decode algorithms across classes. To address this possibility, we developed a self-motion BMI based on cortical activity as monkeys cycled a hand-held pedal to progress along a virtual track. Unlike during reaching, we found no high-variance dimensions that directly correlated with to-be-decoded variables. This was due to no neurons having consistent correlations between their responses and kinematic variables. Yet we could decode a single variable-self-motion-by nonlinearly leveraging structure that spanned multiple high-variance neural dimensions. Resulting online BMI-control success rates approached those during manual control. These findings make two broad points regarding how to build decode algorithms that harmonize with the empirical structure of neural activity in motor cortex. First, even when decoding from the same cortical region (e.g., arm-related motor cortex), different movement classes may need to employ very different strategies. Although correlations between neural activity and hand velocity are prominent during reaching tasks, they are not a fundamental property of motor cortex and cannot be counted on to be present in general. Second, although one generally desires a low-dimensional readout, it can be beneficial to leverage a multidimensional high-variance subspace. Fully embracing this approach requires highly nonlinear approaches tailored to the task at hand, but can produce near-native levels of performance.SIGNIFICANCE STATEMENT Many brain-machine interface decoders have been constructed for controlling movements normally performed with the arm. Yet it is unclear how these will function beyond the reach-like scenarios where they were developed. Existing decoders implicitly assume that neural covariance structure, and correlations with to-be-decoded kinematic variables, will be largely preserved across tasks. We find that the correlation between neural activity and hand kinematics, a feature typically exploited when decoding reach-like movements, is essentially absent during another task performed with the arm: cycling through a virtual environment. Nevertheless, the use of a different strategy, one focused on leveraging the highest-variance neural signals, supported high performance real-time brain-machine interface control.
Collapse
Affiliation(s)
- Karen E Schroeder
- Department of Neuroscience, Columbia University Medical Center, New York, New York
- Zuckerman Institute, Columbia University, New York, New York
| | - Sean M Perkins
- Zuckerman Institute, Columbia University, New York, New York
- Department of Biomedical Engineering, Columbia University, New York, New York
| | - Qi Wang
- Department of Biomedical Engineering, Columbia University, New York, New York
| | - Mark M Churchland
- Department of Neuroscience, Columbia University Medical Center, New York, New York
- Zuckerman Institute, Columbia University, New York, New York
- Kavli Institute for Brain Science, Columbia University Medical Center, New York, New York
- Grossman Center for the Statistics of Mind, Columbia University, New York, New York
| |
Collapse
|
40
|
Abstract
Investigating how an artificial network of neurons controls a simulated arm suggests that rotational patterns of activity in the motor cortex may rely on sensory feedback from the moving limb.
Collapse
Affiliation(s)
- Omid G Sani
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, United States
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, United States.,Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, United States.,Neuroscience Graduate Program, University of Southern California, Los Angeles, United States
| |
Collapse
|
41
|
A Phase Control Method for the Dynamical Attractor of the HR Neuron Model: The Rotation-Transition Process and Its Experimental Realization. Neural Process Lett 2021. [DOI: 10.1007/s11063-021-10568-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
42
|
Kalidindi HT, Cross KP, Lillicrap TP, Omrani M, Falotico E, Sabes PN, Scott SH. Rotational dynamics in motor cortex are consistent with a feedback controller. eLife 2021; 10:e67256. [PMID: 34730516 PMCID: PMC8691841 DOI: 10.7554/elife.67256] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 10/28/2021] [Indexed: 11/13/2022] Open
Abstract
Recent studies have identified rotational dynamics in motor cortex (MC), which many assume arise from intrinsic connections in MC. However, behavioral and neurophysiological studies suggest that MC behaves like a feedback controller where continuous sensory feedback and interactions with other brain areas contribute substantially to MC processing. We investigated these apparently conflicting theories by building recurrent neural networks that controlled a model arm and received sensory feedback from the limb. Networks were trained to counteract perturbations to the limb and to reach toward spatial targets. Network activities and sensory feedback signals to the network exhibited rotational structure even when the recurrent connections were removed. Furthermore, neural recordings in monkeys performing similar tasks also exhibited rotational structure not only in MC but also in somatosensory cortex. Our results argue that rotational structure may also reflect dynamics throughout the voluntary motor system involved in online control of motor actions.
Collapse
Affiliation(s)
| | - Kevin P Cross
- Centre for Neuroscience Studies, Queen's UniversityKingstonCanada
| | - Timothy P Lillicrap
- Centre for Computation, Mathematics and Physics, University College LondonLondonUnited Kingdom
| | - Mohsen Omrani
- Centre for Neuroscience Studies, Queen's UniversityKingstonCanada
| | - Egidio Falotico
- The BioRobotics Institute, Scuola Superiore Sant'AnnaPisaItaly
| | - Philip N Sabes
- Department of Physiology, University of California, San FranciscoSan FranciscoUnited States
| | - Stephen H Scott
- Centre for Neuroscience Studies, Queen's UniversityKingstonCanada
| |
Collapse
|
43
|
Abstract
A central goal of neuroscience is to understand the representations formed by brain activity patterns and their connection to behaviour. The classic approach is to investigate how individual neurons encode stimuli and how their tuning determines the fidelity of the neural representation. Tuning analyses often use the Fisher information to characterize the sensitivity of neural responses to small changes of the stimulus. In recent decades, measurements of large populations of neurons have motivated a complementary approach, which focuses on the information available to linear decoders. The decodable information is captured by the geometry of the representational patterns in the multivariate response space. Here we review neural tuning and representational geometry with the goal of clarifying the relationship between them. The tuning induces the geometry, but different sets of tuned neurons can induce the same geometry. The geometry determines the Fisher information, the mutual information and the behavioural performance of an ideal observer in a range of psychophysical tasks. We argue that future studies can benefit from considering both tuning and geometry to understand neural codes and reveal the connections between stimuli, brain activity and behaviour.
Collapse
|
44
|
Ebitz RB, Hayden BY. The population doctrine in cognitive neuroscience. Neuron 2021; 109:3055-3068. [PMID: 34416170 PMCID: PMC8725976 DOI: 10.1016/j.neuron.2021.07.011] [Citation(s) in RCA: 69] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 07/02/2021] [Accepted: 07/13/2021] [Indexed: 01/08/2023]
Abstract
A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.
Collapse
Affiliation(s)
- R Becket Ebitz
- Department of Neurosciences, Faculté de médecine, Université de Montréal, Montréal, QC, Canada.
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
45
|
Abstract
Significant experimental, computational, and theoretical work has identified rich structure within the coordinated activity of interconnected neural populations. An emerging challenge now is to uncover the nature of the associated computations, how they are implemented, and what role they play in driving behavior. We term this computation through neural population dynamics. If successful, this framework will reveal general motifs of neural population activity and quantitatively describe how neural population dynamics implement computations necessary for driving goal-directed behavior. Here, we start with a mathematical primer on dynamical systems theory and analytical tools necessary to apply this perspective to experimental data. Next, we highlight some recent discoveries resulting from successful application of dynamical systems. We focus on studies spanning motor control, timing, decision-making, and working memory. Finally, we briefly discuss promising recent lines of investigation and future directions for the computation through neural population dynamics framework.
Collapse
Affiliation(s)
- Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, California 94305, USA; .,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA
| | - Matthew D Golub
- Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA.,Google AI, Google Inc., Mountain View, California 94305, USA
| | - Krishna V Shenoy
- Department of Bioengineering, Stanford University, Stanford, California 94305, USA; .,Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA.,Department of Neurobiology, Bio-X Institute, Neurosciences Program, and Howard Hughes Medical Institute, Stanford University, Stanford, California 94305, USA
| |
Collapse
|
46
|
Diomedi S, Vaccari FE, Galletti C, Hadjidimitrakis K, Fattori P. Motor-like neural dynamics in two parietal areas during arm reaching. Prog Neurobiol 2021; 205:102116. [PMID: 34217822 DOI: 10.1016/j.pneurobio.2021.102116] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 05/18/2021] [Accepted: 06/29/2021] [Indexed: 10/20/2022]
Abstract
The classical view on motor control makes a clear distinction between the role of motor cortex in controlling muscles and parietal cortex in processing movement plans and goals. However, the strong parieto-frontal connections argue against such clear-cut separation of function. Modern dynamical approaches revealed that population activity in motor cortex can be captured by a limited number of patterns, called neural states that are preserved across diverse motor behaviors. Whether such dynamics are also present in parietal cortex is unclear. Here, we studied neural dynamics in the primate parietal cortex during arm movements and found three main states temporally coupled to the planning, execution and target holding epochs. Strikingly, as reported recently in motor cortex, execution was subdivided into distinct, arm acceleration- and deceleration-related, states. These results suggest that dynamics across parieto-frontal areas are highly consistent and hint that parietal population activity largely reflects timing constraints while motor actions unfold.
Collapse
Affiliation(s)
- S Diomedi
- Dept. of Biomedical and Neuromotor Sciences, University of Bologna, Italy
| | - F E Vaccari
- Dept. of Biomedical and Neuromotor Sciences, University of Bologna, Italy
| | - C Galletti
- Dept. of Biomedical and Neuromotor Sciences, University of Bologna, Italy
| | - K Hadjidimitrakis
- Dept. of Biomedical and Neuromotor Sciences, University of Bologna, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Italy.
| | - P Fattori
- Dept. of Biomedical and Neuromotor Sciences, University of Bologna, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Italy.
| |
Collapse
|
47
|
Eberle H, Hayashi Y, Kurazume R, Takei T, An Q. Modeling of hyper-adaptability: from motor coordination to rehabilitation. Adv Robot 2021. [DOI: 10.1080/01691864.2021.1943710] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Harry Eberle
- Department of Ortho and MSK Science Division of Surgery & Interventional Science, Faculty of Medical Sciences, University College London, London, UK
| | - Yoshikatsu Hayashi
- Biomedical Engineering, School of Biological Sciences, University of Reading, Reading, UK
| | - Ryo Kurazume
- Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan
| | - Tomohiko Takei
- Graduate School of Medicine, Hakubi Center for Advanced Research, Kyoto University, Kyoto, Japan
| | - Qi An
- Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan
| |
Collapse
|
48
|
Kao TC, Sadabadi MS, Hennequin G. Optimal anticipatory control as a theory of motor preparation: A thalamo-cortical circuit model. Neuron 2021; 109:1567-1581.e12. [PMID: 33789082 PMCID: PMC8111422 DOI: 10.1016/j.neuron.2021.03.009] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 10/09/2020] [Accepted: 03/05/2021] [Indexed: 11/21/2022]
Abstract
Across a range of motor and cognitive tasks, cortical activity can be accurately described by low-dimensional dynamics unfolding from specific initial conditions on every trial. These "preparatory states" largely determine the subsequent evolution of both neural activity and behavior, and their importance raises questions regarding how they are, or ought to be, set. Here, we formulate motor preparation as optimal anticipatory control of future movements and show that the solution requires a form of internal feedback control of cortical circuit dynamics. In contrast to a simple feedforward strategy, feedback control enables fast movement preparation by selectively controlling the cortical state in the small subspace that matters for the upcoming movement. Feedback but not feedforward control explains the orthogonality between preparatory and movement activity observed in reaching monkeys. We propose a circuit model in which optimal preparatory control is implemented as a thalamo-cortical loop gated by the basal ganglia.
Collapse
Affiliation(s)
- Ta-Chu Kao
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK.
| | - Mahdieh S Sadabadi
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK; Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, UK
| | - Guillaume Hennequin
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK.
| |
Collapse
|
49
|
Orban GA, Lanzilotto M, Bonini L. From Observed Action Identity to Social Affordances. Trends Cogn Sci 2021; 25:493-505. [PMID: 33745819 DOI: 10.1016/j.tics.2021.02.012] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 02/24/2021] [Accepted: 02/27/2021] [Indexed: 01/08/2023]
Abstract
Others' observed actions cause continuously changing retinal images, making it challenging to build neural representations of action identity. The monkey anterior intraparietal area (AIP) and its putative human homologue (phAIP) host neurons selective for observed manipulative actions (OMAs). The neuronal activity of both AIP and phAIP allows a stable readout of OMA identity across visual formats, but human neurons exhibit greater invariance and generalize from observed actions to action verbs. These properties stem from the convergence in AIP of superior temporal signals concerning: (i) observed body movements; and (ii) the changes in the body-object relationship. We propose that evolutionarily preserved mechanisms underlie the specification of observed-actions identity and the selection of motor responses afforded by them, thereby promoting social behavior.
Collapse
Affiliation(s)
- G A Orban
- Department of Medicine and Surgery, University of Parma, Parma, Italy
| | - M Lanzilotto
- Department of Psychology, University of Turin, Turin, Italy
| | - L Bonini
- Department of Medicine and Surgery, University of Parma, Parma, Italy.
| |
Collapse
|
50
|
Khanna P, Totten D, Novik L, Roberts J, Morecraft RJ, Ganguly K. Low-frequency stimulation enhances ensemble co-firing and dexterity after stroke. Cell 2021; 184:912-930.e20. [PMID: 33571430 DOI: 10.1016/j.cell.2021.01.023] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2020] [Revised: 09/08/2020] [Accepted: 01/15/2021] [Indexed: 12/31/2022]
Abstract
Electrical stimulation is a promising tool for modulating brain networks. However, it is unclear how stimulation interacts with neural patterns underlying behavior. Specifically, how might external stimulation that is not sensitive to the state of ongoing neural dynamics reliably augment neural processing and improve function? Here, we tested how low-frequency epidural alternating current stimulation (ACS) in non-human primates recovering from stroke interacted with task-related activity in perilesional cortex and affected grasping. We found that ACS increased co-firing within task-related ensembles and improved dexterity. Using a neural network model, we found that simulated ACS drove ensemble co-firing and enhanced propagation of neural activity through parts of the network with impaired connectivity, suggesting a mechanism to link increased co-firing to enhanced dexterity. Together, our results demonstrate that ACS restores neural processing in impaired networks and improves dexterity following stroke. More broadly, these results demonstrate approaches to optimize stimulation to target neural dynamics.
Collapse
Affiliation(s)
- Preeya Khanna
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94158, USA; California National Primate Research Center, University of California, Davis, Davis, CA 95616, USA
| | - Douglas Totten
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94158, USA; California National Primate Research Center, University of California, Davis, Davis, CA 95616, USA
| | - Lisa Novik
- California National Primate Research Center, University of California, Davis, Davis, CA 95616, USA
| | - Jeffrey Roberts
- California National Primate Research Center, University of California, Davis, Davis, CA 95616, USA
| | - Robert J Morecraft
- Laboratory of Neurological Sciences, Division of Basic Biomedical Sciences, Sanford School of Medicine, The University of South Dakota, Vermillion, SD 57069, USA
| | - Karunesh Ganguly
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94158, USA; California National Primate Research Center, University of California, Davis, Davis, CA 95616, USA.
| |
Collapse
|