1
|
Sadras N, Pesaran B, Shanechi MM. Event detection and classification from multimodal time series with application to neural data. J Neural Eng 2024; 21:026049. [PMID: 38513289 DOI: 10.1088/1741-2552/ad3678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 03/21/2024] [Indexed: 03/23/2024]
Abstract
The detection of events in time-series data is a common signal-processing problem. When the data can be modeled as a known template signal with an unknown delay in Gaussian noise, detection of the template signal can be done with a traditional matched filter. However, in many applications, the event of interest is represented in multimodal data consisting of both Gaussian and point-process time series. Neuroscience experiments, for example, can simultaneously record multimodal neural signals such as local field potentials (LFPs), which can be modeled as Gaussian, and neuronal spikes, which can be modeled as point processes. Currently, no method exists for event detection from such multimodal data, and as such our objective in this work is to develop a method to meet this need. Here we address this challenge by developing the multimodal event detector (MED) algorithm which simultaneously estimates event times and classes. To do this, we write a multimodal likelihood function for Gaussian and point-process observations and derive the associated maximum likelihood estimator of simultaneous event times and classes. We additionally introduce a cross-modal scaling parameter to account for model mismatch in real datasets. We validate this method in extensive simulations as well as in a neural spike-LFP dataset recorded during an eye-movement task, where the events of interest are eye movements with unknown times and directions. We show that the MED can successfully detect eye movement onset and classify eye movement direction. Further, the MED successfully combines information across data modalities, with multimodal performance exceeding unimodal performance. This method can facilitate applications such as the discovery of latent events in multimodal neural population activity and the development of brain-computer interfaces for naturalistic settings without constrained tasks or prior knowledge of event times.
Collapse
Affiliation(s)
- Nitin Sadras
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Alfred E. Mann Department of Biomedical Engineering, and the Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
2
|
Johnston R, Smith MA. Brain-wide arousal signals are segregated from movement planning in the superior colliculus. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.26.591284. [PMID: 38746466 PMCID: PMC11092505 DOI: 10.1101/2024.04.26.591284] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
The superior colliculus (SC) is traditionally considered a brain region that functions as an interface between processing visual inputs and generating eye movement outputs. Although its role as a primary reflex center is thought to be conserved across vertebrate species, evidence suggests that the SC has evolved to support higher-order cognitive functions including spatial attention. When it comes to oculomotor areas such as the SC, it is critical that high precision fixation and eye movements are maintained even in the presence of signals related to ongoing changes in cognition and brain state, both of which have the potential to interfere with eye position encoding and movement generation. In this study, we recorded spiking responses of neuronal populations in the SC while monkeys performed a memory-guided saccade task and found that the activity of some of the neurons fluctuated over tens of minutes. By leveraging the statistical power afforded by high-dimensional neuronal recordings, we were able to identify a low-dimensional pattern of activity that was correlated with the subjects' arousal levels. Importantly, we found that the spiking responses of deep-layer SC neurons were less correlated with this brain-wide arousal signal, and that neural activity associated with changes in pupil size and saccade tuning did not overlap in population activity space with movement initiation signals. Taken together, these findings provide a framework for understanding how signals related to cognition and arousal can be embedded in the population activity of oculomotor structures without compromising the fidelity of the motor output.
Collapse
Affiliation(s)
- Richard Johnston
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, USA
| | - Matthew A. Smith
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, USA
| |
Collapse
|
3
|
Losey DM, Hennig JA, Oby ER, Golub MD, Sadtler PT, Quick KM, Ryu SI, Tyler-Kabara EC, Batista AP, Yu BM, Chase SM. Learning leaves a memory trace in motor cortex. Curr Biol 2024; 34:1519-1531.e4. [PMID: 38531360 PMCID: PMC11097210 DOI: 10.1016/j.cub.2024.03.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2023] [Revised: 12/06/2023] [Accepted: 03/04/2024] [Indexed: 03/28/2024]
Abstract
How are we able to learn new behaviors without disrupting previously learned ones? To understand how the brain achieves this, we used a brain-computer interface (BCI) learning paradigm, which enables us to detect the presence of a memory of one behavior while performing another. We found that learning to use a new BCI map altered the neural activity that monkeys produced when they returned to using a familiar BCI map in a way that was specific to the learning experience. That is, learning left a "memory trace" in the primary motor cortex. This memory trace coexisted with proficient performance under the familiar map, primarily by altering neural activity in dimensions that did not impact behavior. Forming memory traces might be how the brain is able to provide for the joint learning of multiple behaviors without interference.
Collapse
Affiliation(s)
- Darby M Losey
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Jay A Hennig
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Emily R Oby
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Matthew D Golub
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA; Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA 98195, USA
| | - Patrick T Sadtler
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Kristin M Quick
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA; Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA 94301, USA
| | - Elizabeth C Tyler-Kabara
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA 15213, USA; Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15213, USA; Department of Neurosurgery, Dell Medical School, University of Texas at Austin, Austin, TX 78712, USA
| | - Aaron P Batista
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA.
| | - Byron M Yu
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA.
| | - Steven M Chase
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA.
| |
Collapse
|
4
|
Dekleva BM, Chowdhury RH, Batista AP, Chase SM, Yu BM, Boninger ML, Collinger JL. Motor cortex retains and reorients neural dynamics during motor imagery. Nat Hum Behav 2024; 8:729-742. [PMID: 38287177 PMCID: PMC11089477 DOI: 10.1038/s41562-023-01804-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Accepted: 12/13/2023] [Indexed: 01/31/2024]
Abstract
The most prominent characteristic of motor cortex is its activation during movement execution, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioural and imaging studies, it is unknown how the specific activity patterns and temporal dynamics in motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people who retain some residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population activity into three orthogonal subspaces, where one was similarly active during both action and imagery, and the others were active only during a single task type-action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamic features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by reorienting the components related to motor output and/or feedback into a unique, output-null imagery subspace.
Collapse
Affiliation(s)
- Brian M Dekleva
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Physical Medicine & Rehabilitation, University of Pittsburgh, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
| | - Raeed H Chowdhury
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Aaron P Batista
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Steven M Chase
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Byron M Yu
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Michael L Boninger
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Physical Medicine & Rehabilitation, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Jennifer L Collinger
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, USA.
- Department of Physical Medicine & Rehabilitation, University of Pittsburgh, Pittsburgh, PA, USA.
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA.
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA.
| |
Collapse
|
5
|
Churchland MM, Shenoy KV. Preparatory activity and the expansive null-space. Nat Rev Neurosci 2024; 25:213-236. [PMID: 38443626 DOI: 10.1038/s41583-024-00796-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/26/2024] [Indexed: 03/07/2024]
Abstract
The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.
Collapse
Affiliation(s)
- Mark M Churchland
- Department of Neuroscience, Columbia University, New York, NY, USA.
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| |
Collapse
|
6
|
Hasnain MA, Birnbaum JE, Nunez JLU, Hartman EK, Chandrasekaran C, Economo MN. Separating cognitive and motor processes in the behaving mouse. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.23.554474. [PMID: 37662199 PMCID: PMC10473744 DOI: 10.1101/2023.08.23.554474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
The cognitive processes supporting complex animal behavior are closely associated with ubiquitous movements responsible for our posture, facial expressions, ability to actively sample our sensory environments, and other critical processes. These movements are strongly related to neural activity across much of the brain and are often highly correlated with ongoing cognitive processes, making it challenging to dissociate the neural dynamics that support cognitive processes from those supporting related movements. In such cases, a critical issue is whether cognitive processes are separable from related movements, or if they are driven by common neural mechanisms. Here, we demonstrate how the separability of cognitive and motor processes can be assessed, and, when separable, how the neural dynamics associated with each component can be isolated. We establish a novel two-context behavioral task in mice that involves multiple cognitive processes and show that commonly observed dynamics taken to support cognitive processes are strongly contaminated by movements. When cognitive and motor components are isolated using a novel approach for subspace decomposition, we find that they exhibit distinct dynamical trajectories. Further, properly accounting for movement revealed that largely separate populations of cells encode cognitive and motor variables, in contrast to the 'mixed selectivity' often reported. Accurately isolating the dynamics associated with particular cognitive and motor processes will be essential for developing conceptual and computational models of neural circuit function and evaluating the function of the cell types of which neural circuits are composed.
Collapse
Affiliation(s)
- Munib A. Hasnain
- Department of Biomedical Engineering, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
| | - Jaclyn E. Birnbaum
- Graduate Program for Neuroscience, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
| | | | - Emma K. Hartman
- Department of Biomedical Engineering, Boston University, Boston, MA
| | - Chandramouli Chandrasekaran
- Department of Psychological and Brain Sciences, Boston University, Boston, MA
- Department of Neurobiology & Anatomy, Boston University, Boston, MA
- Center for Systems Neuroscience, Boston University, Boston, MA
| | - Michael N. Economo
- Department of Biomedical Engineering, Boston University, Boston, MA
- Center for Neurophotonics, Boston University, Boston, MA
- Center for Systems Neuroscience, Boston University, Boston, MA
| |
Collapse
|
7
|
Pun TK, Khoshnevis M, Hosman T, Wilson GH, Kapitonava A, Kamdar F, Henderson JM, Simeral JD, Vargas-Irwin CE, Harrison MT, Hochberg LR. Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.29.582733. [PMID: 38496552 PMCID: PMC10942277 DOI: 10.1101/2024.02.29.582733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method to measure instability in neural data without needing to label user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.
Collapse
|
8
|
Ahmadipour P, Sani OG, Pesaran B, Shanechi MM. Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity. J Neural Eng 2024; 21:026001. [PMID: 38016450 PMCID: PMC10913727 DOI: 10.1088/1741-2552/ad1053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 10/23/2023] [Accepted: 11/28/2023] [Indexed: 11/30/2023]
Abstract
Objective.Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales.Approach.Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior.Main results.We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior.Significance.Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.
Collapse
Affiliation(s)
- Parima Ahmadipour
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Omid G Sani
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Alfred E. Mann Department of Biomedical Engineering, and the Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
9
|
Song CY, Shanechi MM. Unsupervised learning of stationary and switching dynamical system models from Poisson observations. J Neural Eng 2023; 20:066029. [PMID: 38083862 PMCID: PMC10714100 DOI: 10.1088/1741-2552/ad038d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 09/15/2023] [Accepted: 10/16/2023] [Indexed: 12/18/2023]
Abstract
Objective. Investigating neural population dynamics underlying behavior requires learning accurate models of the recorded spiking activity, which can be modeled with a Poisson observation distribution. Switching dynamical system models can offer both explanatory power and interpretability by piecing together successive regimes of simpler dynamics to capture more complex ones. However, in many cases, reliable regime labels are not available, thus demanding accurate unsupervised learning methods for Poisson observations. Existing learning methods, however, rely on inference of latent states in neural activity using the Laplace approximation, which may not capture the broader properties of densities and may lead to inaccurate learning. Thus, there is a need for new inference methods that can enable accurate model learning.Approach. To achieve accurate model learning, we derive a novel inference method based on deterministic sampling for Poisson observations called the Poisson Cubature Filter (PCF) and embed it in an unsupervised learning framework. This method takes a minimum mean squared error approach to estimation. Terms that are difficult to find analytically for Poisson observations are approximated in a novel way with deterministic sampling based on numerical integration and cubature rules.Main results. PCF enabled accurate unsupervised learning in both stationary and switching dynamical systems and largely outperformed prior Laplace approximation-based learning methods in both simulations and motor cortical spiking data recorded during a reaching task. These improvements were larger for smaller data sizes, showing that PCF-based learning was more data efficient and enabled more reliable regime identification. In experimental data and unsupervised with respect to behavior, PCF-based learning uncovered interpretable behavior-relevant regimes unlike prior learning methods.Significance. The developed unsupervised learning methods for switching dynamical systems can accurately uncover latent regimes and states in population spiking activity, with important applications in both basic neuroscience and neurotechnology.
Collapse
Affiliation(s)
- Christian Y Song
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America
- Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
- Thomas Lord Department of Computer Science, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
10
|
Rizvi A, Bell K, Yang D, Montenegro MP, Kim H, Bao S, Wright DL, Buchanan JJ, Lei Y. Effects of transcranial direct current stimulation over human motor cortex on cognitive-motor and sensory-motor functions. Sci Rep 2023; 13:20968. [PMID: 38017091 PMCID: PMC10684512 DOI: 10.1038/s41598-023-48070-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 11/22/2023] [Indexed: 11/30/2023] Open
Abstract
The primary motor cortex (M1) is broadly acknowledged for its crucial role in executing voluntary movements. Yet, its contributions to cognitive and sensory functions remain largely unexplored. Transcranial direct current stimulation (tDCS) is a noninvasive neurostimulation method that can modify brain activity, thereby enabling the establishment of a causal link between M1 activity and behavior. This study aimed to investigate the online effects of tDCS over M1 on cognitive-motor and sensory-motor functions. Sixty-four healthy participants underwent either anodal or sham tDCS while concurrently performing a set of standardized robotic tasks. These tasks provided sensitive and objective assessments of brain functions, including action selection, inhibitory control, cognitive control of visuomotor skills, proprioceptive sense, and bimanual coordination. Our results revealed that anodal tDCS applied to M1 enhances decision-making capacity in selecting appropriate motor actions and avoiding distractors compared to sham stimulation, suggesting improved action selection and inhibitory control capabilities. Furthermore, anodal tDCS reduces the movement time required to accomplish bimanual movements, suggesting enhanced bimanual performance. However, we found no impact of anodal tDCS on cognitive control of visuomotor skills and proprioceptive sense. This study suggests that augmenting M1 activity via anodal tDCS influences cognitive-motor and sensory-motor functions in a task-dependent manner.
Collapse
Affiliation(s)
- Aoun Rizvi
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Kara Bell
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Daniel Yang
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Maria P Montenegro
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Hakjoo Kim
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Shancheng Bao
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - David L Wright
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - John J Buchanan
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA
| | - Yuming Lei
- Program of Motor Neuroscience, Department of Kinesiology and Sport Management, Texas A&M University, College Station, TX, 77843, USA.
| |
Collapse
|
11
|
Ahmadipour P, Sani OG, Pesaran B, Shanechi MM. Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.26.542509. [PMID: 37398400 PMCID: PMC10312539 DOI: 10.1101/2023.05.26.542509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/04/2023]
Abstract
Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales. Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical subspace identification method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and spike-LFP population activity recorded during a naturalistic reach and grasp behavior. We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower computational cost while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity. Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest.
Collapse
|
12
|
Zhang Y, Bi K, Li J, Wang Y, Fang F. Dyadic visual perceptual learning on orientation discrimination. Curr Biol 2023:S0960-9822(23)00552-3. [PMID: 37224810 DOI: 10.1016/j.cub.2023.04.070] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 02/24/2023] [Accepted: 04/26/2023] [Indexed: 05/26/2023]
Abstract
The belief that learning can be modulated by social context is mainly supported by high-level value-based learning studies. However, whether social context can even modulate low-level learning such as visual perceptual learning (VPL) is still unknown. Unlike traditional VPL studies in which participants were trained singly, here, we developed a novel dyadic VPL paradigm in which paired participants were trained with the same orientation discrimination task and could monitor each other's performance. We found that the social context (i.e., dyadic training) led to a greater behavioral performance improvement and a faster learning rate compared with the single training. Interestingly, the facilitating effects could be modulated by the performance difference between paired participants. Functional magnetic resonance imaging (fMRI) results showed that, compared with the single training, social cognition areas including bilateral parietal cortex and dorsolateral prefrontal cortex displayed a different activity pattern and enhanced functional connectivities to early visual cortex (EVC) during the dyadic training. Furthermore, the dyadic training resulted in more refined orientation representation in primary visual cortex (V1), which was closely associated with the greater behavioral performance improvement. Taken together, we demonstrate that the social context, learning with a partner, can remarkably augment the plasticity of low-level visual information process by means of reshaping the neural activities in EVC and social cognition areas, as well as their functional interplays.
Collapse
Affiliation(s)
- Yifei Zhang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China; Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, China
| | - Keyan Bi
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China; Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, China
| | - Jian Li
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China
| | - Yizhou Wang
- Center on Frontiers of Computing Studies, School of Computer Science, Peking University, Beijing 100871, China
| | - Fang Fang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing 100871, China; PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China; Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing 100871, China; Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, China.
| |
Collapse
|
13
|
Smoulder AL, Marino PJ, Oby ER, Snyder SE, Miyata H, Pavlovsky NP, Bishop WE, Yu BM, Chase SM, Batista AP. A neural basis of choking under pressure. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.16.537007. [PMID: 37090659 PMCID: PMC10120738 DOI: 10.1101/2023.04.16.537007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/25/2023]
Abstract
Incentives tend to drive improvements in performance. But when incentives get too high, we can "choke under pressure" and underperform when it matters most. What neural processes might lead to choking under pressure? We studied Rhesus monkeys performing a challenging reaching task in which they underperform when an unusually large "jackpot" reward is at stake. We observed a collapse in neural information about upcoming movements for jackpot rewards: in the motor cortex, neural planning signals became less distinguishable for different reach directions when a jackpot reward was made available. We conclude that neural signals of reward and motor planning interact in the motor cortex in a manner that can explain why we choke under pressure. One-Sentence Summary In response to exceptionally large reward cues, animals can "choke under pressure", and this corresponds to a collapse in the neural information about upcoming movements.
Collapse
|
14
|
Song CY, Hsieh HL, Pesaran B, Shanechi MM. Modeling and inference methods for switching regime-dependent dynamical systems with multiscale neural observations. J Neural Eng 2022; 19. [PMID: 36261030 DOI: 10.1088/1741-2552/ac9b94] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 10/19/2022] [Indexed: 01/11/2023]
Abstract
Objective.Realizing neurotechnologies that enable long-term neural recordings across multiple spatial-temporal scales during naturalistic behaviors requires new modeling and inference methods that can simultaneously address two challenges. First, the methods should aggregate information across all activity scales from multiple recording sources such as spiking and field potentials. Second, the methods should detect changes in the regimes of behavior and/or neural dynamics during naturalistic scenarios and long-term recordings. Prior regime detection methods are developed for a single scale of activity rather than multiscale activity, and prior multiscale methods have not considered regime switching and are for stationary cases.Approach.Here, we address both challenges by developing a switching multiscale dynamical system model and the associated filtering and smoothing methods. This model describes the encoding of an unobserved brain state in multiscale spike-field activity. It also allows for regime-switching dynamics using an unobserved regime state that dictates the dynamical and encoding parameters at every time-step. We also design the associated switching multiscale inference methods that estimate both the unobserved regime and brain states from simultaneous spike-field activity.Main results.We validate the methods in both extensive numerical simulations and prefrontal spike-field data recorded in a monkey performing saccades for fluid rewards. We show that these methods can successfully combine the spiking and field potential observations to simultaneously track the regime and brain states accurately. Doing so, these methods lead to better state estimation compared with single-scale switching methods or stationary multiscale methods. Also, for single-scale linear Gaussian observations, the new switching smoother can better generalize to diverse system settings compared to prior switching smoothers.Significance.These modeling and inference methods effectively incorporate both regime-detection and multiscale observations. As such, they could facilitate investigation of latent switching neural population dynamics and improve future brain-machine interfaces by enabling inference in naturalistic scenarios where regime-dependent multiscale activity and behavior arise.
Collapse
Affiliation(s)
- Christian Y Song
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Han-Lin Hsieh
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| | - Bijan Pesaran
- Departments of Neurosurgery, Neuroscience, and Bioengineering, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Maryam M Shanechi
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America.,Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, United States of America.,Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America.,Department of Computer Science, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States of America
| |
Collapse
|
15
|
Zippi EL, You AK, Ganguly K, Carmena JM. Selective modulation of cortical population dynamics during neuroprosthetic skill learning. Sci Rep 2022; 12:15948. [PMID: 36153356 PMCID: PMC9509316 DOI: 10.1038/s41598-022-20218-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 09/09/2022] [Indexed: 01/23/2023] Open
Abstract
Brain-machine interfaces (BMIs) provide a framework for studying how cortical population dynamics evolve over learning in a task in which the mapping between neural activity and behavior is precisely defined. Learning to control a BMI is associated with the emergence of coordinated neural dynamics in populations of neurons whose activity serves as direct input to the BMI decoder (direct subpopulation). While previous work shows differential modification of firing rate modulation in this population relative to a population whose activity was not directly input to the BMI decoder (indirect subpopulation), little is known about how learning-related changes in cortical population dynamics within these groups compare.To investigate this, we monitored both direct and indirect subpopulations as two macaque monkeys learned to control a BMI. We found that while the combined population increased coordinated neural dynamics, this increase in coordination was primarily driven by changes in the direct subpopulation. These findings suggest that motor cortex refines cortical dynamics by increasing neural variance throughout the entire population during learning, with a more pronounced coordination of firing activity in subpopulations that are causally linked to behavior.
Collapse
Affiliation(s)
- Ellen L. Zippi
- grid.47840.3f0000 0001 2181 7878Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720 USA
| | - Albert K. You
- grid.47840.3f0000 0001 2181 7878Department of Electrical Engineering and Computer Sciences, University of California Berkeley, Berkeley, CA 94720 USA
| | - Karunesh Ganguly
- grid.410372.30000 0004 0419 2775Neurology and Rehabilitation Service, San Francisco VA Medical Center, San Francisco, CA 94121 USA ,grid.266102.10000 0001 2297 6811Department of Neurology, University of California, San Francisco, CA 94143 USA
| | - Jose M. Carmena
- grid.47840.3f0000 0001 2181 7878Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720 USA ,grid.47840.3f0000 0001 2181 7878Department of Electrical Engineering and Computer Sciences, University of California Berkeley, Berkeley, CA 94720 USA
| |
Collapse
|
16
|
Small, correlated changes in synaptic connectivity may facilitate rapid motor learning. Nat Commun 2022; 13:5163. [PMID: 36056006 PMCID: PMC9440011 DOI: 10.1038/s41467-022-32646-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Accepted: 08/08/2022] [Indexed: 11/08/2022] Open
Abstract
Animals rapidly adapt their movements to external perturbations, a process paralleled by changes in neural activity in the motor cortex. Experimental studies suggest that these changes originate from altered inputs (Hinput) rather than from changes in local connectivity (Hlocal), as neural covariance is largely preserved during adaptation. Since measuring synaptic changes in vivo remains very challenging, we used a modular recurrent neural network to qualitatively test this interpretation. As expected, Hinput resulted in small activity changes and largely preserved covariance. Surprisingly given the presumed dependence of stable covariance on preserved circuit connectivity, Hlocal led to only slightly larger changes in activity and covariance, still within the range of experimental recordings. This similarity is due to Hlocal only requiring small, correlated connectivity changes for successful adaptation. Simulations of tasks that impose increasingly larger behavioural changes revealed a growing difference between Hinput and Hlocal, which could be exploited when designing future experiments.
Collapse
|
17
|
Sadeh S, Clopath C. Contribution of behavioural variability to representational drift. eLife 2022; 11:77907. [PMID: 36040010 PMCID: PMC9481246 DOI: 10.7554/elife.77907] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Accepted: 08/24/2022] [Indexed: 11/25/2022] Open
Abstract
Neuronal responses to similar stimuli change dynamically over time, raising the question of how internal representations can provide a stable substrate for neural coding. Recent work has suggested a large degree of drift in neural representations even in sensory cortices, which are believed to store stable representations of the external world. While the drift of these representations is mostly characterized in relation to external stimuli, the behavioural state of the animal (for instance, the level of arousal) is also known to strongly modulate the neural activity. We therefore asked how the variability of such modulatory mechanisms can contribute to representational changes. We analysed large-scale recording of neural activity from the Allen Brain Observatory, which was used before to document representational drift in the mouse visual cortex. We found that, within these datasets, behavioural variability significantly contributes to representational changes. This effect was broadcasted across various cortical areas in the mouse, including the primary visual cortex, higher order visual areas, and even regions not primarily linked to vision like hippocampus. Our computational modelling suggests that these results are consistent with independent modulation of neural activity by behaviour over slower timescales. Importantly, our analysis suggests that reliable but variable modulation of neural representations by behaviour can be misinterpreted as representational drift if neuronal representations are only characterized in the stimulus space and marginalized over behavioural parameters.
Collapse
Affiliation(s)
- Sadra Sadeh
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | - Claudia Clopath
- Department of Bioengineering, Imperial College London, London, United Kingdom
| |
Collapse
|
18
|
Johnston R, Snyder AC, Khanna SB, Issar D, Smith MA. The eyes reflect an internal cognitive state hidden in the population activity of cortical neurons. Cereb Cortex 2022; 32:3331-3346. [PMID: 34963140 PMCID: PMC9340396 DOI: 10.1093/cercor/bhab418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Revised: 10/19/2021] [Accepted: 10/20/2021] [Indexed: 01/01/2023] Open
Abstract
Decades of research have shown that global brain states such as arousal can be indexed by measuring the properties of the eyes. The spiking responses of neurons throughout the brain have been associated with the pupil, small fixational saccades, and vigor in eye movements, but it has been difficult to isolate how internal states affect the eyes, and vice versa. While recording from populations of neurons in the visual and prefrontal cortex (PFC), we recently identified a latent dimension of neural activity called "slow drift," which appears to reflect a shift in a global brain state. Here, we asked if slow drift is correlated with the action of the eyes in distinct behavioral tasks. We recorded from visual cortex (V4) while monkeys performed a change detection task, and PFC, while they performed a memory-guided saccade task. In both tasks, slow drift was associated with the size of the pupil and the microsaccade rate, two external indicators of the internal state of the animal. These results show that metrics related to the action of the eyes are associated with a dominant and task-independent mode of neural activity that can be accessed in the population activity of neurons across the cortex.
Collapse
Affiliation(s)
- Richard Johnston
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Adam C Snyder
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, 14627, USA
- Department of Neuroscience, University of Rochester, Rochester, NY, 14642, USA
- Center for Visual Science, University of Rochester, Rochester, NY, 14627, USA
| | - Sanjeev B Khanna
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, 15261, USA
| | - Deepa Issar
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Matthew A Smith
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| |
Collapse
|
19
|
Abstract
Neurophysiological measurements suggest that human information processing is evinced by neuronal activity. However, the quantitative relationship between the activity of a brain region and its information processing capacity remains unclear. We introduce and validate a mathematical model of the information processing capacity of a brain region in terms of neuronal activity, input storage capacity, and the arrival rate of afferent information. We applied the model to fMRI data obtained from a flanker paradigm in young and old subjects. Our analysis showed that—for a given cognitive task and subject—higher information processing capacity leads to lower neuronal activity and faster responses. Crucially, processing capacity—as estimated from fMRI data—predicted task and age-related differences in reaction times, speaking to the model’s predictive validity. This model offers a framework for modelling of brain dynamics in terms of information processing capacity, and may be exploited for studies of predictive coding and Bayes-optimal decision-making.
Collapse
|
20
|
Ashwood ZC, Roy NA, Stone IR, Urai AE, Churchland AK, Pouget A, Pillow JW. Mice alternate between discrete strategies during perceptual decision-making. Nat Neurosci 2022; 25:201-212. [PMID: 35132235 PMCID: PMC8890994 DOI: 10.1038/s41593-021-01007-z] [Citation(s) in RCA: 57] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Accepted: 12/17/2021] [Indexed: 12/21/2022]
Abstract
Classical models of perceptual decision-making assume that subjects use a single, consistent strategy to form decisions, or that decision-making strategies evolve slowly over time. Here we present new analyses suggesting that this common view is incorrect. We analyzed data from mouse and human decision-making experiments and found that choice behavior relies on an interplay among multiple interleaved strategies. These strategies, characterized by states in a hidden Markov model, persist for tens to hundreds of trials before switching, and often switch multiple times within a session. The identified decision-making strategies were highly consistent across mice and comprised a single 'engaged' state, in which decisions relied heavily on the sensory stimulus, and several biased states in which errors frequently occurred. These results provide a powerful alternate explanation for 'lapses' often observed in rodent behavioral experiments, and suggest that standard measures of performance mask the presence of major changes in strategy across trials.
Collapse
Affiliation(s)
- Zoe C Ashwood
- Deptartment of Computer Science, Princeton University, Princeton, NJ, USA.
- Princeton Neuroscience Institute, Princeton, NJ, USA.
| | | | - Iris R Stone
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Anne E Urai
- Cognitive Psychology Unit, Leiden University, Leiden, Netherlands
| | - Anne K Churchland
- David Geffen School of Medicine, The University of California, Los Angeles, Los Angeles, CA, USA
| | - Alexandre Pouget
- Faculty of Medicine & Deptartment of Basic Neurosciences, University of Geneva, Geneva, Switzerland
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
21
|
Going beyond primary motor cortex to improve brain–computer interfaces. Trends Neurosci 2022; 45:176-183. [DOI: 10.1016/j.tins.2021.12.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Revised: 12/01/2021] [Accepted: 12/19/2021] [Indexed: 01/08/2023]
|
22
|
Hennig JA, Oby ER, Losey DM, Batista AP, Yu BM, Chase SM. How learning unfolds in the brain: toward an optimization view. Neuron 2021; 109:3720-3735. [PMID: 34648749 PMCID: PMC8639641 DOI: 10.1016/j.neuron.2021.09.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Revised: 08/25/2021] [Accepted: 09/02/2021] [Indexed: 12/17/2022]
Abstract
How do changes in the brain lead to learning? To answer this question, consider an artificial neural network (ANN), where learning proceeds by optimizing a given objective or cost function. This "optimization framework" may provide new insights into how the brain learns, as many idiosyncratic features of neural activity can be recapitulated by an ANN trained to perform the same task. Nevertheless, there are key features of how neural population activity changes throughout learning that cannot be readily explained in terms of optimization and are not typically features of ANNs. Here we detail three of these features: (1) the inflexibility of neural variability throughout learning, (2) the use of multiple learning processes even during simple tasks, and (3) the presence of large task-nonspecific activity changes. We propose that understanding the role of these features in the brain will be key to describing biological learning using an optimization framework.
Collapse
Affiliation(s)
- Jay A Hennig
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA.
| | - Emily R Oby
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Darby M Losey
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Aaron P Batista
- Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Byron M Yu
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Steven M Chase
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| |
Collapse
|
23
|
Kim MK, Sohn JW, Kim SP. Finding Kinematics-Driven Latent Neural States From Neuronal Population Activity for Motor Decoding. IEEE Trans Neural Syst Rehabil Eng 2021; 29:2027-2036. [PMID: 34550888 DOI: 10.1109/tnsre.2021.3114367] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
While intracortical brain-machine interfaces (BMIs) demonstrate feasibility to restore mobility to people with paralysis, it is still challenging to maintain high-performance decoding in clinical BMIs. One of the main obstacles for high-performance BMI is the noise-prone nature of traditional decoding methods that connect neural response explicitly with physical quantity, such as velocity. In contrast, the recent development of latent neural state model enables a robust readout of large-scale neuronal population activity contents. However, these latent neural states do not necessarily contain kinematic information useful for decoding. Therefore, this study proposes a new approach to finding kinematics-dependent latent factors by extracting latent factors' kinematics-dependent components using linear regression. We estimated these components from the population activity through nonlinear mapping. The proposed kinematics-dependent latent factors generate neural trajectories that discriminate latent neural states before and after the motion onset. We compared the decoding performance of the proposed analysis model with the results from other popular models. They are factor analysis (FA), Gaussian process factor analysis (GPFA), latent factor analysis via dynamical systems (LFADS), preferential subspace identification (PSID), and neuronal population firing rates. The proposed analysis model results in higher decoding accuracy than do the others ( % improvement on average). Our approach may pave a new way to extract latent neural states specific to kinematic information from motor cortices, potentially improving decoding performance for online intracortical BMIs.
Collapse
|
24
|
Li MM, Fan JT, Cheng SG, Yang LF, Yang L, Wang LF, Shang ZG, Wan H. Enhanced Hippocampus-Nidopallium Caudolaterale Connectivity during Route Formation in Goal-Directed Spatial Learning of Pigeons. Animals (Basel) 2021; 11:ani11072003. [PMID: 34359131 PMCID: PMC8300203 DOI: 10.3390/ani11072003] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 06/29/2021] [Accepted: 06/29/2021] [Indexed: 11/16/2022] Open
Abstract
Goal-directed spatial learning is crucial for the survival of animals, in which the formation of the route from the current location to the goal is one of the central problems. A distributed brain network comprising the hippocampus and prefrontal cortex has been shown to support such capacity, yet it is not fully understood how the most similar brain regions in birds, the hippocampus (Hp) and nidopallium caudolaterale (NCL), cooperate during route formation in goal-directed spatial learning. Hence, we examined neural activity in the Hp-NCL network of pigeons and explored the connectivity dynamics during route formation in a goal-directed spatial task. We found that behavioral changes in spatial learning during route formation are accompanied by modifications in neural patterns in the Hp-NCL network. Specifically, as pigeons learned to solve the task, the spectral power in both regions gradually decreased. Meanwhile, elevated hippocampal theta (5 to 12 Hz) connectivity and depressed connectivity in NCL were also observed. Lastly, the interregional functional connectivity was found to increase with learning, specifically in the theta frequency band during route formation. These results provide insight into the dynamics of the Hp-NCL network during spatial learning, serving to reveal the potential mechanism of avian spatial navigation.
Collapse
Affiliation(s)
- Meng-Meng Li
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Jian-Tao Fan
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Shu-Guan Cheng
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Li-Fang Yang
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Long Yang
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Liao-Feng Wang
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
| | - Zhi-Gang Shang
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
- Institute of Medical Engineering Technology and Data Mining, Zhengzhou University, Zhengzhou 450001, China
- Correspondence: (Z.-G.S.); (H.W.); Tel.: +86-0371-67781417 (Z.-G.S.); +86-0371-67781421 (H.W.)
| | - Hong Wan
- School of Electrical Engineering, Zhengzhou University, Zhengzhou 450001, China; (M.-M.L.); (J.-T.F.); (S.-G.C.); (L.-F.Y.); (L.Y.); (L.-F.W.)
- Henan Key Laboratory of Brain Science and Brain-Computer Interface Technology, Zhengzhou 450001, China
- Correspondence: (Z.-G.S.); (H.W.); Tel.: +86-0371-67781417 (Z.-G.S.); +86-0371-67781421 (H.W.)
| |
Collapse
|