1
|
Cai C, Dong C, Friedrich J, Rozsa M, Pnevmatikakis EA, Giovannucci A. FIOLA: an accelerated pipeline for fluorescence imaging online analysis. Nat Methods 2023; 20:1417-1425. [PMID: 37679524 DOI: 10.1038/s41592-023-01964-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 06/19/2023] [Indexed: 09/09/2023]
Abstract
Optical microscopy methods such as calcium and voltage imaging enable fast activity readout of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This gap not only prevents the execution of real-time closed-loop experiments, but also hampers fast experiment-analysis-theory turnover for high-throughput imaging modalities. Reliable extraction of neural activity from fluorescence imaging frames at speeds compatible with indicator dynamics and imaging modalities poses a challenge. We therefore developed FIOLA, a framework for fluorescence imaging online analysis that extracts neuronal activity from calcium and voltage imaging movies at speeds one order of magnitude faster than state-of-the-art methods. FIOLA exploits algorithms optimized for parallel processing on GPUs and CPUs. We demonstrate reliable and scalable performance of FIOLA on both simulated and real calcium and voltage imaging datasets. Finally, we present an online experimental scenario to provide guidance in setting FIOLA parameters and to highlight the trade-offs of our approach.
Collapse
Affiliation(s)
- Changjia Cai
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Cynthia Dong
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | - Marton Rozsa
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | | | - Andrea Giovannucci
- Joint Department of Biomedical Engineering UNC/NCSU, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
- Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
- Closed-Loop Engineering for Advanced Rehabilitation (CLEAR), North Carolina State University, Raleigh, NC, USA.
| |
Collapse
|
2
|
Cai C, Friedrich J, Singh A, Eybposh MH, Pnevmatikakis EA, Podgorski K, Giovannucci A. VolPy: Automated and scalable analysis pipelines for voltage imaging datasets. PLoS Comput Biol 2021; 17:e1008806. [PMID: 33852574 PMCID: PMC8075204 DOI: 10.1371/journal.pcbi.1008806] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Revised: 04/26/2021] [Accepted: 02/16/2021] [Indexed: 12/19/2022] Open
Abstract
Voltage imaging enables monitoring neural activity at sub-millisecond and sub-cellular scale, unlocking the study of subthreshold activity, synchrony, and network dynamics with unprecedented spatio-temporal resolution. However, high data rates (>800MB/s) and low signal-to-noise ratios create bottlenecks for analyzing such datasets. Here we present VolPy, an automated and scalable pipeline to pre-process voltage imaging datasets. VolPy features motion correction, memory mapping, automated segmentation, denoising and spike extraction, all built on a highly parallelizable, modular, and extensible framework optimized for memory and speed. To aid automated segmentation, we introduce a corpus of 24 manually annotated datasets from different preparations, brain areas and voltage indicators. We benchmark VolPy against ground truth segmentation, simulations and electrophysiology recordings, and we compare its performance with existing algorithms in detecting spikes. Our results indicate that VolPy's performance in spike extraction and scalability are state-of-the-art.
Collapse
Affiliation(s)
- Changjia Cai
- Joint Department of Biomedical Engineering at University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, North Carolina, United States of America
| | - Johannes Friedrich
- Flatiron Institute, Simons Foundation, New York, New York, United States of America
| | - Amrita Singh
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, United States of America
| | - M. Hossein Eybposh
- Joint Department of Biomedical Engineering at University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, North Carolina, United States of America
| | | | - Kaspar Podgorski
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, United States of America
| | - Andrea Giovannucci
- Joint Department of Biomedical Engineering at University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, North Carolina, United States of America
- Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States of America
| |
Collapse
|
3
|
Friedrich J, Giovannucci A, Pnevmatikakis EA. Online analysis of microendoscopic 1-photon calcium imaging data streams. PLoS Comput Biol 2021; 17:e1008565. [PMID: 33507937 PMCID: PMC7842953 DOI: 10.1371/journal.pcbi.1008565] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2020] [Accepted: 11/27/2020] [Indexed: 11/18/2022] Open
Abstract
In vivo calcium imaging through microendoscopic lenses enables imaging of neuronal populations deep within the brains of freely moving animals. Previously, a constrained matrix factorization approach (CNMF-E) has been suggested to extract single-neuronal activity from microendoscopic data. However, this approach relies on offline batch processing of the entire video data and is demanding both in terms of computing and memory requirements. These drawbacks prevent its applicability to the analysis of large datasets and closed-loop experimental settings. Here we address both issues by introducing two different online algorithms for extracting neuronal activity from streaming microendoscopic data. Our first algorithm, OnACID-E, presents an online adaptation of the CNMF-E algorithm, which dramatically reduces its memory and computation requirements. Our second algorithm proposes a convolution-based background model for microendoscopic data that enables even faster (real time) processing. Our approach is modular and can be combined with existing online motion artifact correction and activity deconvolution methods to provide a highly scalable pipeline for microendoscopic data analysis. We apply our algorithms on four previously published typical experimental datasets and show that they yield similar high-quality results as the popular offline approach, but outperform it with regard to computing time and memory requirements. They can be used instead of CNMF-E to process pre-recorded data with boosted speeds and dramatically reduced memory requirements. Further, they newly enable online analysis of live-streaming data even on a laptop. Calcium imaging methods enable researchers to measure the activity of genetically-targeted large-scale neuronal subpopulations. Whereas previous methods required the specimen to be stable, e.g. anesthetized or head-fixed, new brain imaging techniques using microendoscopic lenses and miniaturized microscopes have enabled deep brain imaging in freely moving mice. However, the very large background fluctuations, the inevitable movements and distortions of imaging field, and the extensive spatial overlaps of fluorescent signals complicate the goal of efficiently extracting accurate estimates of neural activity from the observed video data. Further, current activity extraction methods are computationally expensive due to the complex background model and are typically applied to imaging data long after the experiment is complete. Moreover, in some scenarios it is necessary to perform experiments in real-time and closed-loop—analyzing data on-the-fly to guide the next experimental steps or to control feedback –, and this calls for new methods for accurate real-time processing. Here we address both issues by adapting a popular extraction method to operate online and extend it to utilize GPU hardware that enables real time processing. Our algorithms yield similar high-quality results as the original offline approach, but outperform it with regard to computing time and memory requirements. Our results enable faster and scalable analysis, and open the door to new closed-loop experiments in deep brain areas and on freely-moving preparations. Our algorithms can be used for newly enabled real-time analysis of streaming data, as well as swapped in directly to replace the computationally costly offline approach.
Collapse
Affiliation(s)
- Johannes Friedrich
- Flatiron Institute, Simons Foundation, New York, New York, United States of America
- * E-mail:
| | - Andrea Giovannucci
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University; and UNC Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States of America
| | | |
Collapse
|
4
|
Taxidis J, Pnevmatikakis EA, Dorian CC, Mylavarapu AL, Arora JS, Samadian KD, Hoffberg EA, Golshani P. Differential Emergence and Stability of Sensory and Temporal Representations in Context-Specific Hippocampal Sequences. Neuron 2020; 108:984-998.e9. [PMID: 32949502 DOI: 10.1016/j.neuron.2020.08.028] [Citation(s) in RCA: 44] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 07/04/2020] [Accepted: 08/27/2020] [Indexed: 12/15/2022]
Abstract
Hippocampal spiking sequences encode external stimuli and spatiotemporal intervals, linking sequential experiences in memory, but the dynamics controlling the emergence and stability of such diverse representations remain unclear. Using two-photon calcium imaging in CA1 while mice performed an olfactory working-memory task, we recorded stimulus-specific sequences of "odor-cells" encoding olfactory stimuli followed by "time-cells" encoding time points in the ensuing delay. Odor-cells were reliably activated and retained stable fields during changes in trial structure and across days. Time-cells exhibited sparse and dynamic fields that remapped in both cases. During task training, but not in untrained task exposure, time-cell ensembles increased in size, whereas odor-cell numbers remained stable. Over days, sequences drifted to new populations with cell activity progressively converging to a field and then diverging from it. Therefore, CA1 employs distinct regimes to encode external cues versus their variable temporal relationships, which may be necessary to construct maps of sequential experiences.
Collapse
MESH Headings
- Action Potentials
- Animals
- CA1 Region, Hippocampal/chemistry
- CA1 Region, Hippocampal/cytology
- CA1 Region, Hippocampal/physiology
- Cues
- Male
- Memory, Short-Term/drug effects
- Memory, Short-Term/physiology
- Mice
- Mice, 129 Strain
- Mice, Inbred C57BL
- Mice, Transgenic
- Microscopy, Fluorescence, Multiphoton/methods
- Odorants
- Smell/drug effects
- Smell/physiology
- Time Factors
Collapse
Affiliation(s)
- Jiannis Taxidis
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA; Integrative Center for Learning and Memory, Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA.
| | | | - Conor C Dorian
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Apoorva L Mylavarapu
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Jagmeet S Arora
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Kian D Samadian
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Emily A Hoffberg
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - Peyman Golshani
- Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA; Integrative Center for Learning and Memory, Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA; Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, Los Angeles, CA, USA; West Los Angeles Veteran Affairs Medical Center, Los Angeles, CA, USA; Intellectual and Developmental Disabilities Research Center, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
5
|
Lacefield CO, Pnevmatikakis EA, Paninski L, Bruno RM. Reinforcement Learning Recruits Somata and Apical Dendrites across Layers of Primary Sensory Cortex. Cell Rep 2020; 26:2000-2008.e2. [PMID: 30784583 PMCID: PMC7001879 DOI: 10.1016/j.celrep.2019.01.093] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2016] [Revised: 09/27/2018] [Accepted: 01/24/2019] [Indexed: 01/20/2023] Open
Abstract
The mammalian brain can form associations between behaviorally relevant stimuli in an animal’s environment. While such learning is thought to primarily involve high-order association cortex, even primary sensory areas receive long-range connections carrying information that could contribute to high-level representations. Here, we imaged layer 1 apical dendrites in the barrel cortex of mice performing a whisker-based operant behavior. In addition to sensory-motor events, calcium signals in apical dendrites of layers 2/3 and 5 neurons and in layer 2/3 somata track the delivery of rewards, both choice related and randomly administered. Reward-related tuft-wide dendritic spikes emerge gradually with training and are task specific. Learning recruits cells whose intrinsic activity coincides with the time of reinforcement. Layer 4 largely lacked reward-related signals, suggesting a source other than the primary thalamus. Our results demonstrate that a sensory cortex can acquire a set of associations outside its immediate sensory modality and linked to salient behavioral events. Previously, the only known triggers of apical dendritic spikes were “bottom-up”events, such as appropriate sensory stimuli or an animal’s location in space. Lacefield et al. show that reinforced associations are powerful triggers of apical dendrite activity and that reward can manipulate perceptions at their earliest stages of cortical processing.
Collapse
Affiliation(s)
- Clay O Lacefield
- Department of Neuroscience, Mortimer Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Kavli Institute for Brain Science, Columbia University, New York, NY 10027, USA
| | | | - Liam Paninski
- Department of Neuroscience, Mortimer Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Kavli Institute for Brain Science, Columbia University, New York, NY 10027, USA; Department of Statistics, Columbia University, New York, NY 10027, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY 10027, USA
| | - Randy M Bruno
- Department of Neuroscience, Mortimer Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Kavli Institute for Brain Science, Columbia University, New York, NY 10027, USA.
| |
Collapse
|
6
|
Abstract
Calcium imaging is a popular tool among neuroscientists because of its capability to monitor in vivo large neural populations across weeks with single neuron and single spike resolution. Before any downstream analysis, the data needs to be pre-processed to extract the location and activity of the neurons and processes in the observed field of view. The ever increasing size of calcium imaging datasets necessitates scalable analysis pipelines that are reproducible and fully automated. This review focuses on recent methods for addressing the pre-processing problems that arise in calcium imaging data analysis, and available software tools for high throughput analysis pipelines.
Collapse
|
7
|
Giovannucci A, Friedrich J, Gunn P, Kalfon J, Brown BL, Koay SA, Taxidis J, Najafi F, Gauthier JL, Zhou P, Khakh BS, Tank DW, Chklovskii DB, Pnevmatikakis EA. CaImAn an open source tool for scalable calcium imaging data analysis. eLife 2019; 8:38173. [PMID: 30652683 PMCID: PMC6342523 DOI: 10.7554/elife.38173] [Citation(s) in RCA: 345] [Impact Index Per Article: 69.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2018] [Accepted: 11/23/2018] [Indexed: 12/11/2022] Open
Abstract
Advances in fluorescence microscopy enable monitoring larger brain areas in-vivo with finer time resolution. The resulting data rates require reproducible analysis pipelines that are reliable, fully automated, and scalable to datasets generated over the course of months. We present CaImAn, an open-source library for calcium imaging data analysis. CaImAn provides automatic and scalable methods to address problems common to pre-processing, including motion correction, neural activity identification, and registration across different sessions of data collection. It does this while requiring minimal user intervention, with good scalability on computers ranging from laptops to high-performance computing clusters. CaImAn is suitable for two-photon and one-photon imaging, and also enables real-time analysis on streaming data. To benchmark the performance of CaImAn we collected and combined a corpus of manual annotations from multiple labelers on nine mouse two-photon datasets. We demonstrate that CaImAn achieves near-human performance in detecting locations of active neurons. The human brain contains billions of cells called neurons that rapidly carry information from one part of the brain to another. Progress in medical research and healthcare is hindered by the difficulty in understanding precisely which neurons are active at any given time. New brain imaging techniques and genetic tools allow researchers to track the activity of thousands of neurons in living animals over many months. However, these experiments produce large volumes of data that researchers currently have to analyze manually, which can take a long time and generate irreproducible results. There is a need to develop new computational tools to analyze such data. The new tools should be able to operate on standard computers rather than just specialist equipment as this would limit the use of the solutions to particularly well-funded research teams. Ideally, the tools should also be able to operate in real-time as several experimental and therapeutic scenarios, like the control of robotic limbs, require this. To address this need, Giovannucci et al. developed a new software package called CaImAn to analyze brain images on a large scale. Firstly, the team developed algorithms that are suitable to analyze large sets of data on laptops and other standard computing equipment. These algorithms were then adapted to operate online in real-time. To test how well the new software performs against manual analysis by human researchers, Giovannucci et al. asked several trained human annotators to identify active neurons that were round or donut-shaped in several sets of imaging data from mouse brains. Each set of data was independently analyzed by three or four researchers who then discussed any neurons they disagreed on to generate a ‘consensus annotation’. Giovannucci et al. then used CaImAn to analyze the same sets of data and compared the results to the consensus annotations. This demonstrated that CaImAn is nearly as good as human researchers at identifying active neurons in brain images. CaImAn provides a quicker method to analyze large sets of brain imaging data and is currently used by over a hundred laboratories across the world. The software is open source, meaning that it is freely-available and that users are encouraged to customize it and collaborate with other users to develop it further.
Collapse
Affiliation(s)
- Andrea Giovannucci
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, United States
| | - Johannes Friedrich
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, United States.,Department of Statistics, Columbia University, New York, United States.,Center for Theoretical Neuroscience, Columbia University, New York, United States
| | - Pat Gunn
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, United States
| | | | - Brandon L Brown
- Department of Physiology, University of California, Los Angeles, Los Angeles, United States
| | - Sue Ann Koay
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Jiannis Taxidis
- Department of Neurology, University of California, Los Angeles, Los Angeles, United States
| | | | - Jeffrey L Gauthier
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Pengcheng Zhou
- Department of Statistics, Columbia University, New York, United States.,Center for Theoretical Neuroscience, Columbia University, New York, United States
| | - Baljit S Khakh
- Department of Physiology, University of California, Los Angeles, Los Angeles, United States.,Department of Neurobiology, University of California, Los Angeles, Los Angeles, United States
| | - David W Tank
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Dmitri B Chklovskii
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, United States
| | | |
Collapse
|
8
|
Zhou P, Resendez SL, Rodriguez-Romaguera J, Jimenez JC, Neufeld SQ, Giovannucci A, Friedrich J, Pnevmatikakis EA, Stuber GD, Hen R, Kheirbek MA, Sabatini BL, Kass RE, Paninski L. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. eLife 2018; 7:e28728. [PMID: 29469809 PMCID: PMC5871355 DOI: 10.7554/elife.28728] [Citation(s) in RCA: 321] [Impact Index Per Article: 53.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2017] [Accepted: 02/20/2018] [Indexed: 12/12/2022] Open
Abstract
In vivo calcium imaging through microendoscopic lenses enables imaging of previously inaccessible neuronal populations deep within the brains of freely moving animals. However, it is computationally challenging to extract single-neuronal activity from microendoscopic data, because of the very large background fluctuations and high spatial overlaps intrinsic to this recording modality. Here, we describe a new constrained matrix factorization approach to accurately separate the background and then demix and denoise the neuronal signals of interest. We compared the proposed method against previous independent components analysis and constrained nonnegative matrix factorization approaches. On both simulated and experimental data recorded from mice, our method substantially improved the quality of extracted cellular signals and detected more well-isolated neural signals, especially in noisy data regimes. These advances can in turn significantly enhance the statistical power of downstream analyses, and ultimately improve scientific conclusions derived from microendoscopic data.
Collapse
Affiliation(s)
- Pengcheng Zhou
- Center for the Neural Basis of CognitionCarnegie Mellon UniversityPittsburghUnited States
- Department of StatisticsColumbia UniversityNew YorkUnited States
- Machine Learning DepartmentCarnegie Mellon UniversityPittsburghUnited States
- Grossman Center for the Statistics of MindColumbia UniversityNew YorkUnited States
- Center for Theoretical NeuroscienceColumbia UniversityNew YorkUnited States
| | - Shanna L Resendez
- Department of PsychiatryUniversity of North Carolina at Chapel HillChapel HillUnited States
| | | | - Jessica C Jimenez
- Department of NeuroscienceColumbia UniversityNew YorkUnited States
- Division of Integrative Neuroscience, Department of PsychiatryNew York State Psychiatric InstituteNew YorkUnited States
- Department of Psychiatry & PharmacologyColumbia UniversityNew YorkUnited States
| | - Shay Q Neufeld
- Department of NeurobiologyHarvard Medical School, Howard Hughes Medical InstituteBostonUnited States
| | - Andrea Giovannucci
- Center for Computational BiologyFlatiron Institute, Simons FoundationNew YorkUnited States
| | - Johannes Friedrich
- Center for Computational BiologyFlatiron Institute, Simons FoundationNew YorkUnited States
| | | | - Garret D Stuber
- Department of PsychiatryUniversity of North Carolina at Chapel HillChapel HillUnited States
- Department of Cell Biology and PhysiologyUniversity of North Carolina at Chapel HillChapel HillUnited States
- Neuroscience CenterUniversity of North Carolina at Chapel HillChapel HillUnited States
| | - Rene Hen
- Department of NeuroscienceColumbia UniversityNew YorkUnited States
- Division of Integrative Neuroscience, Department of PsychiatryNew York State Psychiatric InstituteNew YorkUnited States
- Department of Psychiatry & PharmacologyColumbia UniversityNew YorkUnited States
| | - Mazen A Kheirbek
- Weill Institute for NeurosciencesUniversity of California, San FranciscoSan FranciscoUnited States
- Neuroscience Graduate ProgramUniversity of CaliforniaSan FranciscoUnited States
- Kavli Institute for Fundamental NeuroscienceUniversity of California, San FranciscoSan FranciscoUnited States
- Department of PsychiatryUniversity of California, San FranciscoSan FranciscoUnited States
| | - Bernardo L Sabatini
- Department of NeurobiologyHarvard Medical School, Howard Hughes Medical InstituteBostonUnited States
| | - Robert E Kass
- Center for the Neural Basis of CognitionCarnegie Mellon UniversityPittsburghUnited States
- Machine Learning DepartmentCarnegie Mellon UniversityPittsburghUnited States
- Department of StatisticsCarnegie Mellon UniversityPittsburghUnited States
| | - Liam Paninski
- Department of StatisticsColumbia UniversityNew YorkUnited States
- Grossman Center for the Statistics of MindColumbia UniversityNew YorkUnited States
- Center for Theoretical NeuroscienceColumbia UniversityNew YorkUnited States
- Department of NeuroscienceColumbia UniversityNew YorkUnited States
- Kavli Institute for Brain ScienceColumbia UniversityNew YorkUnited States
- Neurotechnology CenterColumbia UniversityNew YorkUnited States
| |
Collapse
|
9
|
Picardo MA, Merel J, Katlowitz KA, Vallentin D, Okobi DE, Benezra SE, Clary RC, Pnevmatikakis EA, Paninski L, Long MA. Population-Level Representation of a Temporal Sequence Underlying Song Production in the Zebra Finch. Neuron 2017; 90:866-76. [PMID: 27196976 DOI: 10.1016/j.neuron.2016.02.016] [Citation(s) in RCA: 73] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2015] [Revised: 01/14/2016] [Accepted: 02/04/2016] [Indexed: 12/13/2022]
Abstract
The zebra finch brain features a set of clearly defined and hierarchically arranged motor nuclei that are selectively responsible for producing singing behavior. One of these regions, a critical forebrain structure called HVC, contains premotor neurons that are active at precise time points during song production. However, the neural representation of this behavior at a population level remains elusive. We used two-photon microscopy to monitor ensemble activity during singing, integrating across multiple trials by adopting a Bayesian inference approach to more precisely estimate burst timing. Additionally, we examined spiking and motor-related synaptic inputs using intracellular recordings during singing. With both experimental approaches, we find that premotor events do not occur preferentially at the onsets or offsets of song syllables or at specific subsyllabic motor landmarks. These results strongly support the notion that HVC projection neurons collectively exhibit a temporal sequence during singing that is uncoupled from ongoing movements.
Collapse
Affiliation(s)
- Michel A Picardo
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Josh Merel
- Department of Statistics and Center for Theoretical Neuroscience, Columbia University, New York, NY 10027, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY 10027, USA
| | - Kalman A Katlowitz
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Daniela Vallentin
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Daniel E Okobi
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Sam E Benezra
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Rachel C Clary
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Eftychios A Pnevmatikakis
- Department of Statistics and Center for Theoretical Neuroscience, Columbia University, New York, NY 10027, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY 10027, USA; Simons Center for Data Analysis, Simons Foundation, New York, NY 10010, USA
| | - Liam Paninski
- Department of Statistics and Center for Theoretical Neuroscience, Columbia University, New York, NY 10027, USA; Grossman Center for the Statistics of Mind, Columbia University, New York, NY 10027, USA
| | - Michael A Long
- New York University Neuroscience Institute and Department of Otolaryngology, New York University Langone Medical Center, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10003, USA.
| |
Collapse
|
10
|
Pnevmatikakis EA, Giovannucci A. NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. J Neurosci Methods 2017; 291:83-94. [PMID: 28782629 DOI: 10.1016/j.jneumeth.2017.07.031] [Citation(s) in RCA: 397] [Impact Index Per Article: 56.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2017] [Revised: 07/27/2017] [Accepted: 07/28/2017] [Indexed: 12/16/2022]
Abstract
BACKGROUND Motion correction is a challenging pre-processing problem that arises early in the analysis pipeline of calcium imaging data sequences. The motion artifacts in two-photon microscopy recordings can be non-rigid, arising from the finite time of raster scanning and non-uniform deformations of the brain medium. NEW METHOD We introduce an algorithm for fast Non-Rigid Motion Correction (NoRMCorre) based on template matching. NoRMCorre operates by splitting the field of view (FOV) into overlapping spatial patches along all directions. The patches are registered at a sub-pixel resolution for rigid translation against a regularly updated template. The estimated alignments are subsequently up-sampled to create a smooth motion field for each frame that can efficiently approximate non-rigid artifacts in a piecewise-rigid manner. EXISTING METHODS Existing approaches either do not scale well in terms of computational performance or are targeted to non-rigid artifacts arising just from the finite speed of raster scanning, and thus cannot correct for non-rigid motion observable in datasets from a large FOV. RESULTS NoRMCorre can be run in an online mode resulting in comparable to or even faster than real time motion registration of streaming data. We evaluate its performance with simple yet intuitive metrics and compare against other non-rigid registration methods on simulated data and in vivo two-photon calcium imaging datasets. Open source Matlab and Python code is also made available. CONCLUSIONS The proposed method and accompanying code can be useful for solving large scale image registration problems in calcium imaging, especially in the presence of non-rigid deformations.
Collapse
Affiliation(s)
| | - Andrea Giovannucci
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, NY 10010, USA
| |
Collapse
|
11
|
Giovannucci A, Pnevmatikakis EA, Deverett B, Pereira T, Fondriest J, Brady MJ, Wang SSH, Abbas W, Parés P, Masip D. Automated gesture tracking in head-fixed mice. J Neurosci Methods 2017; 300:184-195. [PMID: 28728948 DOI: 10.1016/j.jneumeth.2017.07.014] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2017] [Revised: 06/25/2017] [Accepted: 07/13/2017] [Indexed: 11/15/2022]
Abstract
BACKGROUND The preparation consisting of a head-fixed mouse on a spherical or cylindrical treadmill offers unique advantages in a variety of experimental contexts. Head fixation provides the mechanical stability necessary for optical and electrophysiological recordings and stimulation. Additionally, it can be combined with virtual environments such as T-mazes, enabling these types of recording during diverse behaviors. NEW METHOD In this paper we present a low-cost, easy-to-build acquisition system, along with scalable computational methods to quantitatively measure behavior (locomotion and paws, whiskers, and tail motion patterns) in head-fixed mice locomoting on cylindrical or spherical treadmills. EXISTING METHODS Several custom supervised and unsupervised methods have been developed for measuring behavior in mice. However, to date there is no low-cost, turn-key, general-purpose, and scalable system for acquiring and quantifying behavior in mice. RESULTS We benchmark our algorithms against ground truth data generated either by manual labeling or by simpler methods of feature extraction. We demonstrate that our algorithms achieve good performance, both in supervised and unsupervised settings. CONCLUSIONS We present a low-cost suite of tools for behavioral quantification, which serve as valuable complements to recording and stimulation technologies being developed for the head-fixed mouse preparation.
Collapse
Affiliation(s)
- A Giovannucci
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, NY, USA; Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA.
| | - E A Pnevmatikakis
- Center for Computational Biology, Flatiron Institute, Simons Foundation, New York, NY, USA
| | - B Deverett
- Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA; Robert Wood Johnson Medical School, New Brunswick, NJ, USA
| | - T Pereira
- Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA
| | - J Fondriest
- Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA; Robert Wood Johnson Medical School, New Brunswick, NJ, USA
| | - M J Brady
- Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA; Robert Wood Johnson Medical School, New Brunswick, NJ, USA
| | - S S-H Wang
- Princeton Neuroscience Institute and Department of Molecular Biology, Princeton University, Princeton, NJ, USA; Robert Wood Johnson Medical School, New Brunswick, NJ, USA
| | - W Abbas
- Department of Computer Science, Universitat Oberta de Catalunya, Barcelona, Spain
| | - P Parés
- Department of Computer Science, Universitat Oberta de Catalunya, Barcelona, Spain
| | - D Masip
- Department of Computer Science, Universitat Oberta de Catalunya, Barcelona, Spain
| |
Collapse
|
12
|
Pnevmatikakis EA, Rad KR, Huggins J, Paninski L. Fast Kalman Filtering and Forward–Backward Smoothing via a Low-Rank Perturbative Approach. J Comput Graph Stat 2014. [DOI: 10.1080/10618600.2012.760461] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
13
|
Pnevmatikakis EA, Kelleher K, Chen R, Saggau P, Josić K, Paninski L. Fast spatiotemporal smoothing of calcium measurements in dendritic trees. PLoS Comput Biol 2012; 8:e1002569. [PMID: 22787437 PMCID: PMC3386185 DOI: 10.1371/journal.pcbi.1002569] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2011] [Accepted: 05/08/2012] [Indexed: 11/18/2022] Open
Abstract
We discuss methods for fast spatiotemporal smoothing of calcium signals in dendritic trees, given single-trial, spatially localized imaging data obtained via multi-photon microscopy. By analyzing the dynamics of calcium binding to probe molecules and the effects of the imaging procedure, we show that calcium concentration can be estimated up to an affine transformation, i.e., an additive and multiplicative constant. To obtain a full spatiotemporal estimate, we model calcium dynamics within the cell using a functional approach. The evolution of calcium concentration is represented through a smaller set of hidden variables that incorporate fast transients due to backpropagating action potentials (bAPs), or other forms of stimulation. Because of the resulting state space structure, inference can be done in linear time using forward-backward maximum-a-posteriori methods. Non-negativity constraints on the calcium concentration can also be incorporated using a log-barrier method that does not affect the computational scaling. Moreover, by exploiting the neuronal tree structure we show that the cost of the algorithm is also linear in the size of the dendritic tree, making the approach applicable to arbitrarily large trees. We apply this algorithm to data obtained from hippocampal CA1 pyramidal cells with experimentally evoked bAPs, some of which were paired with excitatory postsynaptic potentials (EPSPs). The algorithm recovers the timing of the bAPs and provides an estimate of the induced calcium transient throughout the tree. The proposed methods could be used to further understand the interplay between bAPs and EPSPs in synaptic strength modification. More generally, this approach allows us to infer the concentration on intracellular calcium across the dendritic tree from noisy observations at a discrete set of points in space.
Collapse
Affiliation(s)
- Eftychios A Pnevmatikakis
- Department of Statistics and Center for Theoretical Neuroscience, Columbia University, New York, New York, USA.
| | | | | | | | | | | |
Collapse
|
14
|
Abstract
We investigate architectures for time encoding and time decoding of visual stimuli such as natural and synthetic video streams (movies, animation). The architecture for time encoding is akin to models of the early visual system. It consists of a bank of filters in cascade with single-input multi-output neural circuits. Neuron firing is based on either a threshold-and-fire or an integrate-and-fire spiking mechanism with feedback. We show that analog information is represented by the neural circuits as projections on a set of band-limited functions determined by the spike sequence. Under Nyquist-type and frame conditions, the encoded signal can be recovered from these projections with arbitrary precision. For the video time encoding machine architecture, we demonstrate that band-limited video streams of finite energy can be faithfully recovered from the spike trains and provide a stable algorithm for perfect recovery. The key condition for recovery calls for the number of neurons in the population to be above a threshold value.
Collapse
Affiliation(s)
- Aurel A Lazar
- Department of Electrical Engineering, Columbia University, New York, NY 10027, USA.
| | | |
Collapse
|
15
|
Lazar AA, Pnevmatikakis EA, Zhou Y. Encoding natural scenes with neural circuits with random thresholds. Vision Res 2010; 50:2200-12. [PMID: 20350565 DOI: 10.1016/j.visres.2010.03.015] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2009] [Revised: 03/20/2010] [Accepted: 03/22/2010] [Indexed: 10/19/2022]
Abstract
We present a general framework for the reconstruction of natural video scenes encoded with a population of spiking neural circuits with random thresholds. The natural scenes are modeled as space-time functions that belong to a space of trigonometric polynomials. The visual encoding system consists of a bank of filters, modeling the visual receptive fields, in cascade with a population of neural circuits, modeling encoding in the early visual system. The neuron models considered include integrate-and-fire neurons and ON-OFF neuron pairs with threshold-and-fire spiking mechanisms. All thresholds are assumed to be random. We demonstrate that neural spiking is akin to taking noisy measurements on the stimulus both for time-varying and space-time-varying stimuli. We formulate the reconstruction problem as the minimization of a suitable cost functional in a finite-dimensional vector space and provide an explicit algorithm for stimulus recovery. We also present a general solution using the theory of smoothing splines in Reproducing Kernel Hilbert Spaces. We provide examples of both synthetic video as well as for natural scenes and demonstrate that the quality of the reconstruction degrades gracefully as the threshold variability of the neurons increases.
Collapse
Affiliation(s)
- Aurel A Lazar
- Department of Electrical Engineering, Columbia University, New York, NY 10027, USA.
| | | | | |
Collapse
|
16
|
Lazar AA, Pnevmatikakis EA. Consistent recovery of sensory stimuli encoded with MIMO neural circuits. Comput Intell Neurosci 2010; 2010:469658. [PMID: 19809513 PMCID: PMC2754078 DOI: 10.1155/2010/469658] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/01/2009] [Accepted: 06/24/2009] [Indexed: 11/17/2022]
Abstract
We consider the problem of reconstructing finite energy stimuli encoded with a population of spiking leaky integrate-and-fire neurons. The reconstructed signal satisfies a consistency condition: when passed through the same neuron, it triggers the same spike train as the original stimulus. The recovered stimulus has to also minimize a quadratic smoothness optimality criterion. We formulate the reconstruction as a spline interpolation problem for scalar as well as vector valued stimuli and show that the recovery has a unique solution. We provide explicit reconstruction algorithms for stimuli encoded with single as well as a population of integrate-and-fire neurons. We demonstrate how our reconstruction algorithms can be applied to stimuli encoded with ON-OFF neural circuits with feedback. Finally, we extend the formalism to multi-input multi-output neural circuits and demonstrate that vector-valued finite energy signals can be efficiently encoded by a neural population provided that its size is beyond a threshold value. Examples are given that demonstrate the potential applications of our methodology to systems neuroscience and neuromorphic engineering.
Collapse
Affiliation(s)
- Aurel A Lazar
- Department of Electrical Engineering, Columbia University, New York, NY 10027, USA.
| | | |
Collapse
|
17
|
Lazar AA, Pnevmatikakis EA. Reconstruction and classification of stimuli encoded with neural circuits with feedback. BMC Neurosci 2009. [DOI: 10.1186/1471-2202-10-s1-p123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|
18
|
Lazar AA, Pnevmatikakis EA. Reconstruction of Sensory Stimuli Encoded with Integrate-and-Fire Neurons with Random Thresholds. EURASIP J Adv Signal Process 2009; 2009:682930. [PMID: 24077610 PMCID: PMC3783269 DOI: 10.1155/2009/682930] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
We present a general approach to the reconstruction of sensory stimuli encoded with leaky integrate-and-fire neurons with random thresholds. The stimuli are modeled as elements of a Reproducing Kernel Hilbert Space. The reconstruction is based on finding a stimulus that minimizes a regularized quadratic optimality criterion. We discuss in detail the reconstruction of sensory stimuli modeled as absolutely continuous functions as well as stimuli with absolutely continuous first-order derivatives. Reconstruction results are presented for stimuli encoded with single as well as a population of neurons. Examples are given that demonstrate the performance of the reconstruction algorithms as a function of threshold variability.
Collapse
Affiliation(s)
- Aurel A Lazar
- Department of Electrical Engineering, Columbia University, New York, NY 10027, USA
| | | |
Collapse
|
19
|
Abstract
We consider a formal model of stimulus encoding with a circuit consisting of a bank of filters and an ensemble of integrate-and-fire neurons. Such models arise in olfactory systems, vision, and hearing. We demonstrate that bandlimited stimuli can be faithfully represented with spike trains generated by the ensemble of neurons. We provide a stimulus reconstruction scheme based on the spike times of the ensemble of neurons and derive conditions for perfect recovery. The key result calls for the spike density of the neural population to be above the Nyquist rate. We also show that recovery is perfect if the number of neurons in the population is larger than a threshold value. Increasing the number of neurons to achieve a faithful representation of the sensory world is consistent with basic neurobiological thought. Finally we demonstrate that in general, the problem of faithful recovery of stimuli from the spike train of single neurons is ill posed. The stimulus can be recovered, however, from the information contained in the spike train of a population of neurons.
Collapse
Affiliation(s)
- Aurel A Lazar
- Department of Electrical Engineering, Columbia University, New York, NY 10027, USA.
| | | |
Collapse
|
20
|
Lazar AA, Pnevmatikakis EA. A simple spiking retina model for exact video stimulus representation. BMC Neurosci 2008. [DOI: 10.1186/1471-2202-9-s1-p130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|