1
|
McCart JD, Sedler AR, Versteeg C, Mifsud D, Rigotti-Thompson M, Pandarinath C. Diffusion-Based Generation of Neural Activity from Disentangled Latent Codes. ARXIV 2024:arXiv:2407.21195v1. [PMID: 39130199 PMCID: PMC11312623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/13/2024]
Abstract
Recent advances in recording technology have allowed neuroscientists to monitor activity from thousands of neurons simultaneously. Latent variable models are increasingly valuable for distilling these recordings into compact and interpretable representations. Here we propose a new approach to neural data analysis that leverages advances in conditional generative modeling to enable the unsupervised inference of disentangled behavioral variables from recorded neural activity. Our approach builds on InfoDiffusion, which augments diffusion models with a set of latent variables that capture important factors of variation in the data. We apply our model, called Generating Neural Observations Conditioned on Codes with High Information (GNOCCHI), to time series neural data and test its application to synthetic and biological recordings of neural activity during reaching. In comparison to a VAE-based sequential autoencoder, GNOCCHI learns higher-quality latent spaces that are more clearly structured and more disentangled with respect to key behavioral variables. These properties enable accurate generation of novel samples (unseen behavioral conditions) through simple linear traversal of the latent spaces produced by GNOCCHI. Our work demonstrates the potential of unsupervised, information-based models for the discovery of interpretable latent spaces from neural data, enabling researchers to generate high-quality samples from unseen conditions.
Collapse
Affiliation(s)
- Jonathan D McCart
- Center for Machine Learning, Georgia Tech
- Department of Biomedical Engineering, Georgia Tech and Emory University
| | - Andrew R Sedler
- Center for Machine Learning, Georgia Tech
- Department of Biomedical Engineering, Georgia Tech and Emory University
| | | | - Domenick Mifsud
- Center for Machine Learning, Georgia Tech
- Department of Biomedical Engineering, Georgia Tech and Emory University
| | | | - Chethan Pandarinath
- Center for Machine Learning, Georgia Tech
- Department of Biomedical Engineering, Georgia Tech and Emory University
- Department of Neurosurgery, Emory University School of Medicine
| |
Collapse
|
2
|
Hira R. Closed-loop experiments and brain machine interfaces with multiphoton microscopy. NEUROPHOTONICS 2024; 11:033405. [PMID: 38375331 PMCID: PMC10876015 DOI: 10.1117/1.nph.11.3.033405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 01/22/2024] [Accepted: 01/29/2024] [Indexed: 02/21/2024]
Abstract
In the field of neuroscience, the importance of constructing closed-loop experimental systems has increased in conjunction with technological advances in measuring and controlling neural activity in live animals. We provide an overview of recent technological advances in the field, focusing on closed-loop experimental systems where multiphoton microscopy-the only method capable of recording and controlling targeted population activity of neurons at a single-cell resolution in vivo-works through real-time feedback. Specifically, we present some examples of brain machine interfaces (BMIs) using in vivo two-photon calcium imaging and discuss applications of two-photon optogenetic stimulation and adaptive optics to real-time BMIs. We also consider conditions for realizing future optical BMIs at the synaptic level, and their possible roles in understanding the computational principles of the brain.
Collapse
Affiliation(s)
- Riichiro Hira
- Tokyo Medical and Dental University, Graduate School of Medical and Dental Sciences, Department of Physiology and Cell Biology, Tokyo, Japan
| |
Collapse
|
3
|
Li Y, An X, Qian Y, Xu XH, Zhao S, Mohan H, Bachschmid-Romano L, Brunel N, Whishaw IQ, Huang ZJ. Cortical network and projection neuron types that articulate serial order in a skilled motor behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.25.563871. [PMID: 37961483 PMCID: PMC10634836 DOI: 10.1101/2023.10.25.563871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
Skilled motor behaviors require orderly coordination of multiple constituent movements with sensory cues towards achieving a goal, but the underlying brain circuit mechanisms remain unclear. Here we show that target-guided reach-grasp-to-drink (RGD) in mice involves the ordering and coordination of a set of forelimb and oral actions. Cortex-wide activity imaging of multiple glutamatergic projection neuron (PN) types uncovered a network, involving the secondary motor cortex (MOs), forelimb primary motor and somatosensory cortex, that tracked RGD movements. Photo-inhibition highlighted MOs in coordinating RGD movements. Within the MOs, population neural trajectories tracked RGD progression and single neuron activities integrated across constituent movements. Notably, MOs intratelencephalic, pyramidal tract, and corticothalamic PN activities correlated with action coordination, showed distinct neural dynamics trajectories, and differentially contributed to movement coordination. Our results delineate a cortical network and key areas, PN types, and neural dynamics therein that articulate the serial order and coordination of a skilled behavior.
Collapse
Affiliation(s)
- Yi Li
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 1 1724, USA
| | - Xu An
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 1 1724, USA
| | - Yongjun Qian
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 1 1724, USA
| | - X. Hermione Xu
- Department of Biomedical Engineering, Duke University, Durham, NC 27710, USA
| | - Shengli Zhao
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
| | - Hemanth Mohan
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 1 1724, USA
| | | | - Nicolas Brunel
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
| | - Ian Q. Whishaw
- Department of Neuroscience, Canadian Centre for Behavioural Research, University of Lethbridge, Lethbridge, AB, TIK 3M4, Canada
| | - Z. Josh Huang
- Department of Neurobiology, Duke University, Durham, NC 27710, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 1 1724, USA
- Department of Biomedical Engineering, Duke University, Durham, NC 27710, USA
| |
Collapse
|
4
|
Cai C, Dong C, Friedrich J, Rozsa M, Pnevmatikakis EA, Giovannucci A. FIOLA: an accelerated pipeline for fluorescence imaging online analysis. Nat Methods 2023; 20:1417-1425. [PMID: 37679524 DOI: 10.1038/s41592-023-01964-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 06/19/2023] [Indexed: 09/09/2023]
Abstract
Optical microscopy methods such as calcium and voltage imaging enable fast activity readout of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This gap not only prevents the execution of real-time closed-loop experiments, but also hampers fast experiment-analysis-theory turnover for high-throughput imaging modalities. Reliable extraction of neural activity from fluorescence imaging frames at speeds compatible with indicator dynamics and imaging modalities poses a challenge. We therefore developed FIOLA, a framework for fluorescence imaging online analysis that extracts neuronal activity from calcium and voltage imaging movies at speeds one order of magnitude faster than state-of-the-art methods. FIOLA exploits algorithms optimized for parallel processing on GPUs and CPUs. We demonstrate reliable and scalable performance of FIOLA on both simulated and real calcium and voltage imaging datasets. Finally, we present an online experimental scenario to provide guidance in setting FIOLA parameters and to highlight the trade-offs of our approach.
Collapse
Affiliation(s)
- Changjia Cai
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Cynthia Dong
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | - Marton Rozsa
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | | | - Andrea Giovannucci
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
- Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
- Closed-Loop Engineering for Advanced Rehabilitation (CLEAR), North Carolina State University, Raleigh, NC, USA.
| |
Collapse
|
5
|
Sedler AR, Versteeg C, Pandarinath C. Expressive architectures enhance interpretability of dynamics-based neural population models. NEURONS, BEHAVIOR, DATA ANALYSIS, AND THEORY 2023; 2023:10.51628/001c.73987. [PMID: 38699512 PMCID: PMC11065448 DOI: 10.51628/001c.73987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2024]
Abstract
Artificial neural networks that can recover latent dynamics from recorded neural activity may provide a powerful avenue for identifying and interpreting the dynamical motifs underlying biological computation. Given that neural variance alone does not uniquely determine a latent dynamical system, interpretable architectures should prioritize accurate and low-dimensional latent dynamics. In this work, we evaluated the performance of sequential autoencoders (SAEs) in recovering latent chaotic attractors from simulated neural datasets. We found that SAEs with widely-used recurrent neural network (RNN)-based dynamics were unable to infer accurate firing rates at the true latent state dimensionality, and that larger RNNs relied upon dynamical features not present in the data. On the other hand, SAEs with neural ordinary differential equation (NODE)-based dynamics inferred accurate rates at the true latent state dimensionality, while also recovering latent trajectories and fixed point structure. Ablations reveal that this is mainly because NODEs (1) allow use of higher-capacity multi-layer perceptrons (MLPs) to model the vector field and (2) predict the derivative rather than the next state. Decoupling the capacity of the dynamics model from its latent dimensionality enables NODEs to learn the requisite low-D dynamics where RNN cells fail. Additionally, the fact that the NODE predicts derivatives imposes a useful autoregressive prior on the latent states. The suboptimal interpretability of widely-used RNN-based dynamics may motivate substitution for alternative architectures, such as NODE, that enable learning of accurate dynamics in low-dimensional latent spaces.
Collapse
Affiliation(s)
- Andrew R. Sedler
- Center for Machine Learning, Georgia Institute of Technology, Atlanta, GA, USA
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA
| | - Christopher Versteeg
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA
| | - Chethan Pandarinath
- Center for Machine Learning, Georgia Institute of Technology, Atlanta, GA, USA
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA
| |
Collapse
|
6
|
Koh TH, Bishop WE, Kawashima T, Jeon BB, Srinivasan R, Mu Y, Wei Z, Kuhlman SJ, Ahrens MB, Chase SM, Yu BM. Dimensionality reduction of calcium-imaged neuronal population activity. NATURE COMPUTATIONAL SCIENCE 2023; 3:71-85. [PMID: 37476302 PMCID: PMC10358781 DOI: 10.1038/s43588-022-00390-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 12/05/2022] [Indexed: 07/22/2023]
Abstract
Calcium imaging has been widely adopted for its ability to record from large neuronal populations. To summarize the time course of neural activity, dimensionality reduction methods, which have been applied extensively to population spiking activity, may be particularly useful. However, it is unclear if the dimensionality reduction methods applied to spiking activity are appropriate for calcium imaging. We thus carried out a systematic study of design choices based on standard dimensionality reduction methods. We also developed a method to perform deconvolution and dimensionality reduction simultaneously (Calcium Imaging Linear Dynamical System, CILDS). CILDS most accurately recovered the single-trial, low-dimensional time courses from simulated calcium imaging data. CILDS also outperformed the other methods on calcium imaging recordings from larval zebrafish and mice. More broadly, this study represents a foundation for summarizing calcium imaging recordings of large neuronal populations using dimensionality reduction in diverse experimental settings.
Collapse
Affiliation(s)
- Tze Hui Koh
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Center for the Neural Basis of Cognition, PA
| | - William E. Bishop
- Center for the Neural Basis of Cognition, PA
- Department of Machine Learning, Carnegie Mellon University, PA
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Takashi Kawashima
- Janelia Research Campus, Howard Hughes Medical Institute, VA
- Department of Brain Sciences, Weizmann Institute of Science, Israel
| | - Brian B. Jeon
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Center for the Neural Basis of Cognition, PA
| | - Ranjani Srinivasan
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Department of Electrical and Computer Engineering, Johns Hopkins University, MD
| | - Yu Mu
- Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, China
| | - Ziqiang Wei
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Sandra J. Kuhlman
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
- Department of Biological Sciences, Carnegie Mellon University, PA
| | - Misha B. Ahrens
- Janelia Research Campus, Howard Hughes Medical Institute, VA
| | - Steven M. Chase
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
| | - Byron M. Yu
- Department of Biomedical Engineering, Carnegie Mellon University, PA
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, PA
- Department of Electrical and Computer Engineering, Carnegie Mellon University, PA
| |
Collapse
|