1
|
Ruffini N, Altahini S, Weißbach S, Weber N, Milkovits J, Wierczeiko A, Backhaus H, Stroh A. ViNe-Seg: deep-learning-assisted segmentation of visible neurons and subsequent analysis embedded in a graphical user interface. Bioinformatics 2024; 40:btae177. [PMID: 38569889 PMCID: PMC11034984 DOI: 10.1093/bioinformatics/btae177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 03/06/2024] [Accepted: 04/02/2024] [Indexed: 04/05/2024] Open
Abstract
SUMMARY Segmentation of neural somata is a crucial and usually the most time-consuming step in the analysis of optical functional imaging of neuronal microcircuits. In recent years, multiple auto-segmentation tools have been developed to improve the speed and consistency of the segmentation process, mostly, using deep learning approaches. Current segmentation tools, while advanced, still encounter challenges in producing accurate segmentation results, especially in datasets with a low signal-to-noise ratio. This has led to a reliance on manual segmentation techniques. However, manual methods, while customized to specific laboratory protocols, can introduce variability due to individual differences in interpretation, potentially affecting dataset consistency across studies. In response to this challenge, we present ViNe-Seg: a deep-learning-based semi-automatic segmentation tool that offers (i) detection of visible neurons, irrespective of their activity status; (ii) the ability to perform segmentation during an ongoing experiment; (iii) a user-friendly graphical interface that facilitates expert supervision, ensuring precise identification of Regions of Interest; (iv) an array of segmentation models with the option of training custom models and sharing them with the community; and (v) seamless integration of subsequent analysis steps. AVAILABILITY AND IMPLEMENTATION ViNe-Seg code and documentation are publicly available at https://github.com/NiRuff/ViNe-Seg and can be installed from https://pypi.org/project/ViNeSeg/.
Collapse
Affiliation(s)
- Nicolas Ruffini
- Institute for Human Genetics, University Medical Center, Johannes Gutenberg University, 55131 Mainz, Germany
- Leibniz Institute for Resilience Research, Leibniz Association, 55122 Mainz, Germany
| | - Saleh Altahini
- Leibniz Institute for Resilience Research, Leibniz Association, 55122 Mainz, Germany
| | - Stephan Weißbach
- Institute of Developmental Biology and Neurobiology (iDN), Johannes Gutenberg University, 55128 Mainz, Germany
| | - Nico Weber
- Fraunhofer Institute for Industrial Mathematics ITWM, 67663 Kaiserslautern, Germany
| | - Jonas Milkovits
- Institute of Developmental Biology and Neurobiology (iDN), Johannes Gutenberg University, 55128 Mainz, Germany
| | - Anna Wierczeiko
- Institute for Human Genetics, University Medical Center, Johannes Gutenberg University, 55131 Mainz, Germany
- Leibniz Institute for Resilience Research, Leibniz Association, 55122 Mainz, Germany
| | - Hendrik Backhaus
- Leibniz Institute for Resilience Research, Leibniz Association, 55122 Mainz, Germany
| | - Albrecht Stroh
- Leibniz Institute for Resilience Research, Leibniz Association, 55122 Mainz, Germany
| |
Collapse
|
2
|
Shvedov NR, Analoui S, Dafalias T, Bedell BL, Gardner TJ, Scott BB. In vivo imaging in transgenic songbirds reveals superdiffusive neuron migration in the adult brain. Cell Rep 2024; 43:113759. [PMID: 38345898 DOI: 10.1016/j.celrep.2024.113759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 12/01/2023] [Accepted: 01/23/2024] [Indexed: 03/02/2024] Open
Abstract
Neuron migration is a key phase of neurogenesis, critical for the assembly and function of neuronal circuits. In songbirds, this process continues throughout life, but how these newborn neurons disperse through the adult brain is unclear. We address this question using in vivo two-photon imaging in transgenic zebra finches that express GFP in young neurons and other cell types. In juvenile and adult birds, migratory cells are present at a high density, travel in all directions, and make frequent course changes. Notably, these dynamic migration patterns are well fit by a superdiffusive model. Simulations reveal that these superdiffusive dynamics are sufficient to disperse new neurons throughout the song nucleus HVC. These results suggest that superdiffusive migration may underlie the formation and maintenance of nuclear brain structures in the postnatal brain and indicate that transgenic songbirds are a useful resource for future studies into the mechanisms of adult neurogenesis.
Collapse
Affiliation(s)
- Naomi R Shvedov
- Graduate Program for Neuroscience, Boston University, Boston, MA 02215, USA
| | - Sina Analoui
- Department of Psychological and Brain Sciences, Boston University, Boston, MA 02215, USA
| | - Theresia Dafalias
- Graduate Program for Neuroscience, Boston University, Boston, MA 02215, USA
| | - Brooke L Bedell
- Department of Psychological and Brain Sciences, Boston University, Boston, MA 02215, USA
| | - Timothy J Gardner
- Phil and Penny Knight Campus for Accelerating Scientific Impact, University of Oregon, Eugene, OR 97403, USA
| | - Benjamin B Scott
- Department of Psychological and Brain Sciences, Boston University, Boston, MA 02215, USA; Neurophotonics Center, Photonics Center, and Center for Systems Neuroscience, Boston University, Boston, MA 02215, USA.
| |
Collapse
|
3
|
Li X, Hu X, Chen X, Fan J, Zhao Z, Wu J, Wang H, Dai Q. Spatial redundancy transformer for self-supervised fluorescence image denoising. NATURE COMPUTATIONAL SCIENCE 2023; 3:1067-1080. [PMID: 38177722 PMCID: PMC10766531 DOI: 10.1038/s43588-023-00568-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 11/07/2023] [Indexed: 01/06/2024]
Abstract
Fluorescence imaging with high signal-to-noise ratios has become the foundation of accurate visualization and analysis of biological phenomena. However, the inevitable noise poses a formidable challenge to imaging sensitivity. Here we provide the spatial redundancy denoising transformer (SRDTrans) to remove noise from fluorescence images in a self-supervised manner. First, a sampling strategy based on spatial redundancy is proposed to extract adjacent orthogonal training pairs, which eliminates the dependence on high imaging speed. Second, we designed a lightweight spatiotemporal transformer architecture to capture long-range dependencies and high-resolution features at low computational cost. SRDTrans can restore high-frequency information without producing oversmoothed structures and distorted fluorescence traces. Finally, we demonstrate the state-of-the-art denoising performance of SRDTrans on single-molecule localization microscopy and two-photon volumetric calcium imaging. SRDTrans does not contain any assumptions about the imaging process and the sample, thus can be easily extended to various imaging modalities and biological applications.
Collapse
Affiliation(s)
- Xinyang Li
- Department of Automation, Tsinghua University, Beijing, China
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China
| | - Xiaowan Hu
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
| | - Xingye Chen
- Department of Automation, Tsinghua University, Beijing, China
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China
- Research Institute for Frontier Science, Beihang University, Beijing, China
| | - Jiaqi Fan
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Electronic Engineering, Tsinghua University, Beijing, China
| | - Zhifeng Zhao
- Department of Automation, Tsinghua University, Beijing, China
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China
| | - Jiamin Wu
- Department of Automation, Tsinghua University, Beijing, China.
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China.
- Beijing Key Laboratory of Multi-dimension and Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing, China.
- IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China.
| | - Haoqian Wang
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China.
- The Shenzhen Institute of Future Media Technology, Shenzhen, China.
| | - Qionghai Dai
- Department of Automation, Tsinghua University, Beijing, China.
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China.
- Beijing Key Laboratory of Multi-dimension and Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing, China.
- IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China.
| |
Collapse
|
4
|
Kim SJ, Affan RO, Frostig H, Scott BB, Alexander AS. Advances in cellular resolution microscopy for brain imaging in rats. NEUROPHOTONICS 2023; 10:044304. [PMID: 38076724 PMCID: PMC10704261 DOI: 10.1117/1.nph.10.4.044304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 09/23/2023] [Accepted: 11/07/2023] [Indexed: 02/12/2024]
Abstract
Rats are used in neuroscience research because of their physiological similarities with humans and accessibility as model organisms, trainability, and behavioral repertoire. In particular, rats perform a wide range of sophisticated social, cognitive, motor, and learning behaviors within the contexts of both naturalistic and laboratory environments. Further progress in neuroscience can be facilitated by using advanced imaging methods to measure the complex neural and physiological processes during behavior in rats. However, compared with the mouse, the rat nervous system offers a set of challenges, such as larger brain size, decreased neuron density, and difficulty with head restraint. Here, we review recent advances in in vivo imaging techniques in rats with a special focus on open-source solutions for calcium imaging. Finally, we provide suggestions for both users and developers of in vivo imaging systems for rats.
Collapse
Affiliation(s)
- Su Jin Kim
- Johns Hopkins University, Department of Psychological and Brain Sciences, Baltimore, Maryland, United States
| | - Rifqi O. Affan
- Boston University, Center for Systems Neuroscience, Department of Psychological and Brain Sciences, Boston, Massachusetts, United States
- Boston University, Graduate Program in Neuroscience, Boston, Massachusetts, United States
| | - Hadas Frostig
- Boston University, Center for Systems Neuroscience, Department of Psychological and Brain Sciences, Boston, Massachusetts, United States
| | - Benjamin B. Scott
- Boston University, Center for Systems Neuroscience, Department of Psychological and Brain Sciences, Boston, Massachusetts, United States
- Boston University, Neurophotonics Center and Photonics Center, Boston, Massachusetts, United States
| | - Andrew S. Alexander
- University of California Santa Barbara, Department of Psychological and Brain Sciences, Santa Barbara, California, United States
| |
Collapse
|
5
|
Bouchard C, Bernatchez R, Lavoie-Cardinal F. Addressing annotation and data scarcity when designing machine learning strategies for neurophotonics. NEUROPHOTONICS 2023; 10:044405. [PMID: 37636490 PMCID: PMC10447257 DOI: 10.1117/1.nph.10.4.044405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 07/19/2023] [Accepted: 07/20/2023] [Indexed: 08/29/2023]
Abstract
Machine learning has revolutionized the way data are processed, allowing information to be extracted in a fraction of the time it would take an expert. In the field of neurophotonics, machine learning approaches are used to automatically detect and classify features of interest in complex images. One of the key challenges in applying machine learning methods to the field of neurophotonics is the scarcity of available data and the complexity associated with labeling them, which can limit the performance of data-driven algorithms. We present an overview of various strategies, such as weakly supervised learning, active learning, and domain adaptation that can be used to address the problem of labeled data scarcity in neurophotonics. We provide a comprehensive overview of the strengths and limitations of each approach and discuss their potential applications to bioimaging datasets. In addition, we highlight how different strategies can be combined to increase model performance on those datasets. The approaches we describe can help to improve the accessibility of machine learning-based analysis with limited number of annotated images for training and can enable researchers to extract more meaningful insights from small datasets.
Collapse
Affiliation(s)
- Catherine Bouchard
- CERVO Brain Research Centre, Québec, Québec, Canada
- Université Laval, Institute Intelligence and Data, Québec, Québec, Canada
| | - Renaud Bernatchez
- CERVO Brain Research Centre, Québec, Québec, Canada
- Université Laval, Institute Intelligence and Data, Québec, Québec, Canada
| | - Flavie Lavoie-Cardinal
- CERVO Brain Research Centre, Québec, Québec, Canada
- Université Laval, Institute Intelligence and Data, Québec, Québec, Canada
- Université Laval, Département de psychiatrie et de neurosciences, Québec, Québec, Canada
| |
Collapse
|
6
|
Eom M, Han S, Park P, Kim G, Cho ES, Sim J, Lee KH, Kim S, Tian H, Böhm UL, Lowet E, Tseng HA, Choi J, Lucia SE, Ryu SH, Rózsa M, Chang S, Kim P, Han X, Piatkevich KD, Choi M, Kim CH, Cohen AE, Chang JB, Yoon YG. Statistically unbiased prediction enables accurate denoising of voltage imaging data. Nat Methods 2023; 20:1581-1592. [PMID: 37723246 PMCID: PMC10555843 DOI: 10.1038/s41592-023-02005-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 08/10/2023] [Indexed: 09/20/2023]
Abstract
Here we report SUPPORT (statistically unbiased prediction utilizing spatiotemporal information in imaging data), a self-supervised learning method for removing Poisson-Gaussian noise in voltage imaging data. SUPPORT is based on the insight that a pixel value in voltage imaging data is highly dependent on its spatiotemporal neighboring pixels, even when its temporally adjacent frames alone do not provide useful information for statistical prediction. Such dependency is captured and used by a convolutional neural network with a spatiotemporal blind spot to accurately denoise voltage imaging data in which the existence of the action potential in a time frame cannot be inferred by the information in other frames. Through simulations and experiments, we show that SUPPORT enables precise denoising of voltage imaging data and other types of microscopy image while preserving the underlying dynamics within the scene.
Collapse
Affiliation(s)
- Minho Eom
- School of Electrical Engineering, KAIST, Daejeon, Republic of Korea
| | - Seungjae Han
- School of Electrical Engineering, KAIST, Daejeon, Republic of Korea
| | - Pojeong Park
- Department of Chemistry and Chemical Biology, Harvard University, Cambridge, MA, USA
| | - Gyuri Kim
- School of Electrical Engineering, KAIST, Daejeon, Republic of Korea
| | - Eun-Seo Cho
- School of Electrical Engineering, KAIST, Daejeon, Republic of Korea
| | - Jueun Sim
- Department of Materials Science and Engineering, KAIST, Daejeon, Republic of Korea
| | - Kang-Han Lee
- Department of Biology, Chungnam National University, Daejeon, Republic of Korea
| | - Seonghoon Kim
- School of Biological Sciences, Seoul National University, Seoul, Republic of Korea
- Institute of Molecular Biology and Genetics, Seoul National University, Seoul, Republic of Korea
| | - He Tian
- Department of Chemistry and Chemical Biology, Harvard University, Cambridge, MA, USA
| | - Urs L Böhm
- Department of Chemistry and Chemical Biology, Harvard University, Cambridge, MA, USA
- Einstein Center for Neurosciences, NeuroCure Cluster of Excellence, Charité University of Medicine Berlin, Berlin, Germany
| | - Eric Lowet
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Hua-An Tseng
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Jieun Choi
- Graduate School of Medical Science and Engineering, KAIST, Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, Daejeon, Republic of Korea
| | - Stephani Edwina Lucia
- Graduate School of Medical Science and Engineering, KAIST, Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, Daejeon, Republic of Korea
| | - Seung Hyun Ryu
- Interdisciplinary Program in Neuroscience, Seoul National University, Seoul, Republic of Korea
| | - Márton Rózsa
- Allen Institute for Neural Dynamics, Seattle, WA, USA
| | - Sunghoe Chang
- Department of Physiology and Biomedical Sciences, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Pilhan Kim
- Graduate School of Medical Science and Engineering, KAIST, Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, Daejeon, Republic of Korea
- Graduate School of Nanoscience and Technology, KAIST, Daejeon, Republic of Korea
| | - Xue Han
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Kiryl D Piatkevich
- Research Center for Industries of the Future and School of Life Sciences, Westlake University, Hangzhou, China
- Westlake Laboratory of Life Sciences and Biomedicine, Hangzhou, China
- Institute of Basic Medical Sciences, Westlake Institute for Advanced Study, Hangzhou, China
| | - Myunghwan Choi
- School of Biological Sciences, Seoul National University, Seoul, Republic of Korea
- Institute of Molecular Biology and Genetics, Seoul National University, Seoul, Republic of Korea
| | - Cheol-Hee Kim
- Department of Biology, Chungnam National University, Daejeon, Republic of Korea
| | - Adam E Cohen
- Department of Chemistry and Chemical Biology, Harvard University, Cambridge, MA, USA
- Department of Physics, Harvard University, Cambridge, MA, USA
| | - Jae-Byum Chang
- Department of Materials Science and Engineering, KAIST, Daejeon, Republic of Korea
| | - Young-Gyu Yoon
- School of Electrical Engineering, KAIST, Daejeon, Republic of Korea.
- KAIST Institute for Health Science and Technology, Daejeon, Republic of Korea.
- Department of Semiconductor System Engineering, KAIST, Daejeon, Republic of Korea.
| |
Collapse
|
7
|
Zhao Z, Zhou Y, Liu B, He J, Zhao J, Cai Y, Fan J, Li X, Wang Z, Lu Z, Wu J, Qi H, Dai Q. Two-photon synthetic aperture microscopy for minimally invasive fast 3D imaging of native subcellular behaviors in deep tissue. Cell 2023; 186:2475-2491.e22. [PMID: 37178688 DOI: 10.1016/j.cell.2023.04.016] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 02/21/2023] [Accepted: 04/10/2023] [Indexed: 05/15/2023]
Abstract
Holistic understanding of physio-pathological processes requires noninvasive 3D imaging in deep tissue across multiple spatial and temporal scales to link diverse transient subcellular behaviors with long-term physiogenesis. Despite broad applications of two-photon microscopy (TPM), there remains an inevitable tradeoff among spatiotemporal resolution, imaging volumes, and durations due to the point-scanning scheme, accumulated phototoxicity, and optical aberrations. Here, we harnessed the concept of synthetic aperture radar in TPM to achieve aberration-corrected 3D imaging of subcellular dynamics at a millisecond scale for over 100,000 large volumes in deep tissue, with three orders of magnitude reduction in photobleaching. With its advantages, we identified direct intercellular communications through migrasome generation following traumatic brain injury, visualized the formation process of germinal center in the mouse lymph node, and characterized heterogeneous cellular states in the mouse visual cortex, opening up a horizon for intravital imaging to understand the organizations and functions of biological systems at a holistic level.
Collapse
Affiliation(s)
- Zhifeng Zhao
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China; Hangzhou Zhuoxi Institute of Brain and Intelligence, Hangzhou 311100, China
| | - Yiliang Zhou
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China; Hangzhou Zhuoxi Institute of Brain and Intelligence, Hangzhou 311100, China
| | - Bo Liu
- Tsinghua-Peking Center for Life Sciences, Beijing 100084, China; Laboratory of Dynamic Immunobiology, Institute for Immunology, Tsinghua University, Beijing 100084, China; Department of Basic Medical Sciences, School of Medicine, Tsinghua University, Beijing 100084, China
| | - Jing He
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China
| | - Jiayin Zhao
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China
| | - Yeyi Cai
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China
| | - Jingtao Fan
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China
| | - Xinyang Li
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; Hangzhou Zhuoxi Institute of Brain and Intelligence, Hangzhou 311100, China; Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China
| | - Zilin Wang
- Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Department of Anesthesiology, the First Medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Zhi Lu
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China; Hangzhou Zhuoxi Institute of Brain and Intelligence, Hangzhou 311100, China
| | - Jiamin Wu
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China.
| | - Hai Qi
- Tsinghua-Peking Center for Life Sciences, Beijing 100084, China; Laboratory of Dynamic Immunobiology, Institute for Immunology, Tsinghua University, Beijing 100084, China; Department of Basic Medical Sciences, School of Medicine, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory for Immunological Research on Chronic Diseases, Tsinghua University, Beijing 100084, China; Beijing Frontier Research Center for Biological Structure, Tsinghua University, Beijing 100084, China.
| | - Qionghai Dai
- Department of Automation, Tsinghua University, Beijing 100084, China; Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing 100084, China; Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing 100084, China; IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China.
| |
Collapse
|
8
|
Nöbauer T, Zhang Y, Kim H, Vaziri A. Mesoscale volumetric light-field (MesoLF) imaging of neuroactivity across cortical areas at 18 Hz. Nat Methods 2023; 20:600-609. [PMID: 36823333 PMCID: PMC11057224 DOI: 10.1038/s41592-023-01789-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Accepted: 01/24/2023] [Indexed: 02/25/2023]
Abstract
Various implementations of mesoscopes provide optical access for calcium imaging across multi-millimeter fields of view in the mammalian brain; however, capturing the activity of the neuronal population within such fields of view near-simultaneously and in a volumetric fashion has remained challenging as approaches for imaging scattering brain tissues typically are based on sequential acquisition. Here we present a modular, mesoscale light-field (MesoLF) imaging hardware and software solution that allows recording from thousands of neurons within volumes of ⌀ 4 × 0.2 mm, located at up to 350 µm depth in the mouse cortex, at 18 volumes per second and an effective voxel rate of ~40 megavoxels per second. Using our optical design and computational approach we show recording of ~10,000 neurons across multiple cortical areas in mice using workstation-grade computing resources.
Collapse
Affiliation(s)
- Tobias Nöbauer
- Laboratory of Neurotechnology and Biophysics, The Rockefeller University, New York, NY, USA
| | - Yuanlong Zhang
- Laboratory of Neurotechnology and Biophysics, The Rockefeller University, New York, NY, USA
- Department of Automation, Tsinghua University, Beijing, China
| | - Hyewon Kim
- Laboratory of Neurotechnology and Biophysics, The Rockefeller University, New York, NY, USA
| | - Alipasha Vaziri
- Laboratory of Neurotechnology and Biophysics, The Rockefeller University, New York, NY, USA.
- The Kavli Neural Systems Institute, The Rockefeller University, New York, NY, USA.
| |
Collapse
|
9
|
Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol 2023; 41:282-292. [PMID: 36163547 PMCID: PMC9931589 DOI: 10.1038/s41587-022-01450-8] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 07/29/2022] [Indexed: 11/09/2022]
Abstract
A fundamental challenge in fluorescence microscopy is the photon shot noise arising from the inevitable stochasticity of photon detection. Noise increases measurement uncertainty and limits imaging resolution, speed and sensitivity. To achieve high-sensitivity fluorescence imaging beyond the shot-noise limit, we present DeepCAD-RT, a self-supervised deep learning method for real-time noise suppression. Based on our previous framework DeepCAD, we reduced the number of network parameters by 94%, memory consumption by 27-fold and processing time by a factor of 20, allowing real-time processing on a two-photon microscope. A high imaging signal-to-noise ratio can be acquired with tenfold fewer photons than in standard imaging approaches. We demonstrate the utility of DeepCAD-RT in a series of photon-limited experiments, including in vivo calcium imaging of mice, zebrafish larva and fruit flies, recording of three-dimensional (3D) migration of neutrophils after acute brain injury and imaging of 3D dynamics of cortical ATP release. DeepCAD-RT will facilitate the morphological and functional interrogation of biological dynamics with a minimal photon budget.
Collapse
|
10
|
Machado TA, Kauvar IV, Deisseroth K. Multiregion neuronal activity: the forest and the trees. Nat Rev Neurosci 2022; 23:683-704. [PMID: 36192596 PMCID: PMC10327445 DOI: 10.1038/s41583-022-00634-0] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/25/2022] [Indexed: 12/12/2022]
Abstract
The past decade has witnessed remarkable advances in the simultaneous measurement of neuronal activity across many brain regions, enabling fundamentally new explorations of the brain-spanning cellular dynamics that underlie sensation, cognition and action. These recently developed multiregion recording techniques have provided many experimental opportunities, but thoughtful consideration of methodological trade-offs is necessary, especially regarding field of view, temporal acquisition rate and ability to guarantee cellular resolution. When applied in concert with modern optogenetic and computational tools, multiregion recording has already made possible fundamental biological discoveries - in part via the unprecedented ability to perform unbiased neural activity screens for principles of brain function, spanning dozens of brain areas and from local to global scales.
Collapse
Affiliation(s)
- Timothy A Machado
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Isaac V Kauvar
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University, Stanford, CA, USA.
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA.
| |
Collapse
|
11
|
de Kraker L, Seignette K, Thamizharasu P, van den Boom BJ, Ferreira Pica I, Willuhn I, Levelt CN, Togt CVD. SpecSeg is a versatile toolbox that segments neurons and neurites in chronic calcium imaging datasets based on low-frequency cross-spectral power. CELL REPORTS METHODS 2022; 2:100299. [PMID: 36313805 PMCID: PMC9606108 DOI: 10.1016/j.crmeth.2022.100299] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 02/11/2022] [Accepted: 08/29/2022] [Indexed: 11/25/2022]
Abstract
Imaging calcium signals in neurons of animals using single- or multi-photon microscopy facilitates the study of coding in large neural populations. Such experiments produce massive datasets requiring powerful methods to extract responses from hundreds of neurons. We present SpecSeg, an open-source toolbox for (1) segmentation of regions of interest (ROIs) representing neuronal structures, (2) inspection and manual editing of ROIs, (3) neuropil correction and signal extraction, and (4) matching of ROIs in sequential recordings. ROI segmentation in SpecSeg is based on temporal cross-correlations of low-frequency components derived by Fourier analysis of each pixel with its neighbors. The approach is user-friendly, intuitive, and insightful and enables ROI detection around neurons or neurites. It works for single- (miniscope) and multi-photon microscopy data, eliminating the need for separate toolboxes. SpecSeg thus provides an efficient and versatile approach for analyzing calcium responses in neuronal structures imaged over prolonged periods of time.
Collapse
Affiliation(s)
- Leander de Kraker
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
| | - Koen Seignette
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
| | - Premnath Thamizharasu
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
| | - Bastijn J.G. van den Boom
- Netherlands Institute for Neuroscience, Neuromodulation and Behavior Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
- Department of Psychiatry, Amsterdam UMC, University of Amsterdam, Meibergdreef 5, 1105 AZ Amsterdam, the Netherlands
| | - Ildefonso Ferreira Pica
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
| | - Ingo Willuhn
- Netherlands Institute for Neuroscience, Neuromodulation and Behavior Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
- Department of Psychiatry, Amsterdam UMC, University of Amsterdam, Meibergdreef 5, 1105 AZ Amsterdam, the Netherlands
| | - Christiaan N. Levelt
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
- Department of Molecular and Cellular Neurobiology, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, de Boelelaan 1085, 1081 HV Amsterdam, the Netherlands
| | - Chris van der Togt
- Netherlands Institute for Neuroscience, Molecular Visual Plasticity Group, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, the Netherlands
| |
Collapse
|
12
|
Benisty H, Song A, Mishne G, Charles AS. Review of data processing of functional optical microscopy for neuroscience. NEUROPHOTONICS 2022; 9:041402. [PMID: 35937186 PMCID: PMC9351186 DOI: 10.1117/1.nph.9.4.041402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 07/15/2022] [Indexed: 05/04/2023]
Abstract
Functional optical imaging in neuroscience is rapidly growing with the development of optical systems and fluorescence indicators. To realize the potential of these massive spatiotemporal datasets for relating neuronal activity to behavior and stimuli and uncovering local circuits in the brain, accurate automated processing is increasingly essential. We cover recent computational developments in the full data processing pipeline of functional optical microscopy for neuroscience data and discuss ongoing and emerging challenges.
Collapse
Affiliation(s)
- Hadas Benisty
- Yale Neuroscience, New Haven, Connecticut, United States
| | - Alexander Song
- Max Planck Institute for Intelligent Systems, Stuttgart, Germany
| | - Gal Mishne
- UC San Diego, Halıcığlu Data Science Institute, Department of Electrical and Computer Engineering and the Neurosciences Graduate Program, La Jolla, California, United States
| | - Adam S. Charles
- Johns Hopkins University, Kavli Neuroscience Discovery Institute, Center for Imaging Science, Department of Biomedical Engineering, Department of Neuroscience, and Mathematical Institute for Data Science, Baltimore, Maryland, United States
| |
Collapse
|
13
|
Xue Y, Yang Q, Hu G, Guo K, Tian L. Deep-learning-augmented computational miniature mesoscope. OPTICA 2022; 9:1009-1021. [PMID: 36506462 PMCID: PMC9731182 DOI: 10.1364/optica.464700] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 08/02/2022] [Indexed: 05/30/2023]
Abstract
Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a trade-off between field of view (FOV), resolution, and system complexity, and thus cannot fulfill the emerging need for miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed a computational miniature mesoscope (CM2) that exploits a computational imaging strategy to enable single-shot, 3D high-resolution imaging across a wide FOV in a miniaturized platform. Here, we present CM2 V2, which significantly advances both the hardware and computation. We complement the 3 × 3 microlens array with a hybrid emission filter that improves the imaging contrast by 5×, and design a 3D-printed free-form collimator for the LED illuminator that improves the excitation efficiency by 3×. To enable high-resolution reconstruction across a large volume, we develop an accurate and efficient 3D linear shift-variant (LSV) model to characterize spatially varying aberrations. We then train a multimodule deep learning model called CM2Net, using only the 3D-LSV simulator. We quantify the detection performance and localization accuracy of CM2Net to reconstruct fluorescent emitters under different conditions in simulation. We then show that CM2Net generalizes well to experiments and achieves accurate 3D reconstruction across a ~7-mm FOV and 800-μm depth, and provides ~6-μm lateral and ~25-μm axial resolution. This provides an ~8× better axial resolution and ~1400× faster speed compared to the previous model-based algorithm. We anticipate this simple, low-cost computational miniature imaging system will be useful for many large-scale 3D fluorescence imaging applications.
Collapse
Affiliation(s)
- Yujia Xue
- Department of Electrical and Computer Engineering, Boston University, Boston, Massachusetts 02215, USA
| | - Qianwan Yang
- Department of Electrical and Computer Engineering, Boston University, Boston, Massachusetts 02215, USA
| | - Guorong Hu
- Department of Electrical and Computer Engineering, Boston University, Boston, Massachusetts 02215, USA
| | - Kehan Guo
- Department of Electrical and Computer Engineering, Boston University, Boston, Massachusetts 02215, USA
| | - Lei Tian
- Department of Electrical and Computer Engineering, Boston University, Boston, Massachusetts 02215, USA
- Department of Biomedical Engineering, Boston University, Boston, Massachusetts 02215, USA
- Neurophotonics Center, Boston University, Boston, Massachusetts 02215, USA
| |
Collapse
|
14
|
Voelcker B, Pancholi R, Peron S. Transformation of primary sensory cortical representations from layer 4 to layer 2. Nat Commun 2022; 13:5484. [PMID: 36123376 PMCID: PMC9485231 DOI: 10.1038/s41467-022-33249-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 09/08/2022] [Indexed: 11/24/2022] Open
Abstract
Sensory input arrives from thalamus in cortical layer (L) 4, which outputs predominantly to superficial layers. L4 to L2 thus constitutes one of the earliest cortical feedforward networks. Despite extensive study, the transformation performed by this network remains poorly understood. We use two-photon calcium imaging to record neural activity in L2-4 of primary vibrissal somatosensory cortex (vS1) as mice perform an object localization task with two whiskers. Touch responses sparsen and become more reliable from L4 to L2, with nearly half of the superficial touch response confined to ~1 % of excitatory neurons. These highly responsive neurons have broad receptive fields and can more accurately decode stimulus features. They participate disproportionately in ensembles, small subnetworks with elevated pairwise correlations. Thus, from L4 to L2, cortex transitions from distributed probabilistic coding to sparse and robust ensemble-based coding, resulting in more efficient and accurate representations.
Collapse
Affiliation(s)
- Bettina Voelcker
- Center for Neural Science, New York University, 4 Washington Place Rm. 621, New York, NY, 10003, USA.,Neuroscience Institute, NYU Medical Center, 435 E 30th St., New York, NY, 10016, USA
| | - Ravi Pancholi
- Center for Neural Science, New York University, 4 Washington Place Rm. 621, New York, NY, 10003, USA.,Neuroscience Institute, NYU Medical Center, 435 E 30th St., New York, NY, 10016, USA
| | - Simon Peron
- Center for Neural Science, New York University, 4 Washington Place Rm. 621, New York, NY, 10003, USA. .,Neuroscience Institute, NYU Medical Center, 435 E 30th St., New York, NY, 10016, USA.
| |
Collapse
|
15
|
Grienberger C, Giovannucci A, Zeiger W, Portera-Cailliau C. Two-photon calcium imaging of neuronal activity. NATURE REVIEWS. METHODS PRIMERS 2022; 2:67. [PMID: 38124998 PMCID: PMC10732251 DOI: 10.1038/s43586-022-00147-1] [Citation(s) in RCA: 26] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/07/2022] [Indexed: 12/23/2023]
Abstract
In vivo two-photon calcium imaging (2PCI) is a technique used for recording neuronal activity in the intact brain. It is based on the principle that, when neurons fire action potentials, intracellular calcium levels rise, which can be detected using fluorescent molecules that bind to calcium. This Primer is designed for scientists who are considering embarking on experiments with 2PCI. We provide the reader with a background on the basic concepts behind calcium imaging and on the reasons why 2PCI is an increasingly powerful and versatile technique in neuroscience. The Primer explains the different steps involved in experiments with 2PCI, provides examples of what ideal preparations should look like and explains how data are analysed. We also discuss some of the current limitations of the technique, and the types of solutions to circumvent them. Finally, we conclude by anticipating what the future of 2PCI might look like, emphasizing some of the analysis pipelines that are being developed and international efforts for data sharing.
Collapse
Affiliation(s)
- Christine Grienberger
- Department of Biology and Volen National Center for Complex Systems, Brandeis University, Waltham, MA, USA
| | - Andrea Giovannucci
- Joint Department of Biomedical Engineering University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, NC, USA
- UNC Neuroscience Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - William Zeiger
- Department of Neurology, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
| | - Carlos Portera-Cailliau
- Department of Neurology, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
- Department of Neurobiology, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
| |
Collapse
|
16
|
Moore JJ, Robert V, Rashid SK, Basu J. Assessing Local and Branch-specific Activity in Dendrites. Neuroscience 2022; 489:143-164. [PMID: 34756987 PMCID: PMC9125998 DOI: 10.1016/j.neuroscience.2021.10.022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Revised: 10/09/2021] [Accepted: 10/21/2021] [Indexed: 01/12/2023]
Abstract
Dendrites are elaborate neural processes which integrate inputs from various sources in space and time. While decades of work have suggested an independent role for dendrites in driving nonlinear computations for the cell, only recently have technological advances enabled us to capture the variety of activity in dendrites and their coupling dynamics with the soma. Under certain circumstances, activity generated in a given dendritic branch remains isolated, such that the soma or even sister dendrites are not privy to these localized signals. Such branch-specific activity could radically increase the capacity and flexibility of coding for the cell as a whole. Here, we discuss these forms of localized and branch-specific activity, their functional relevance in plasticity and behavior, and their supporting biophysical and circuit-level mechanisms. We conclude by showcasing electrical and optical approaches in hippocampal area CA3, using original experimental data to discuss experimental and analytical methodology and key considerations to take when investigating the functional relevance of independent dendritic activity.
Collapse
Affiliation(s)
- Jason J Moore
- Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA
| | - Vincent Robert
- Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA
| | - Shannon K Rashid
- Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA
| | - Jayeeta Basu
- Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA; Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, NY 10016, USA; Department of Psychiatry, New York University Grossman School of Medicine, New York, NY 10016, USA.
| |
Collapse
|
17
|
Liu W, Pan J, Xu Y, Wang M, Jia H, Zhang K, Chen X, Li X, Liao X. Fast and Accurate Motion Correction for Two-Photon Ca 2+ Imaging in Behaving Mice. Front Neuroinform 2022; 16:851188. [PMID: 35559212 PMCID: PMC9088923 DOI: 10.3389/fninf.2022.851188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2022] [Accepted: 03/07/2022] [Indexed: 11/13/2022] Open
Abstract
Two-photon Ca2+ imaging is a widely used technique for investigating brain functions across multiple spatial scales. However, the recording of neuronal activities is affected by movement of the brain during tasks in which the animal is behaving normally. Although post-hoc image registration is the commonly used approach, the recent developments of online neuroscience experiments require real-time image processing with efficient motion correction performance, posing new challenges in neuroinformatics. We propose a fast and accurate image density feature-based motion correction method to address the problem of imaging animal during behaviors. This method is implemented by first robustly estimating and clustering the density features from two-photon images. Then, it takes advantage of the temporal correlation in imaging data to update features of consecutive imaging frames with efficient calculations. Thus, motion artifacts can be quickly and accurately corrected by matching the features and obtaining the transformation parameters for the raw images. Based on this efficient motion correction strategy, our algorithm yields promising computational efficiency on imaging datasets with scales ranging from dendritic spines to neuronal populations. Furthermore, we show that the proposed motion correction method outperforms other methods by evaluating not only computational speed but also the quality of the correction performance. Specifically, we provide a powerful tool to perform motion correction for two-photon Ca2+ imaging data, which may facilitate online imaging experiments in the future.
Collapse
Affiliation(s)
- Weiyi Liu
- Brain Research Center and State Key Laboratory of Trauma, Burns, and Combined Injury, Third Military Medical University, Chongqing, China
| | - Junxia Pan
- Brain Research Center and State Key Laboratory of Trauma, Burns, and Combined Injury, Third Military Medical University, Chongqing, China
| | - Yuanxu Xu
- Brain Research Center and State Key Laboratory of Trauma, Burns, and Combined Injury, Third Military Medical University, Chongqing, China
| | - Meng Wang
- Center for Neurointelligence, School of Medicine, Chongqing University, Chongqing, China
| | - Hongbo Jia
- Advanced Institute for Brain and Intelligence, Guangxi University, Nanning, China
- Brain Research Instrument Innovation Center, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Kuan Zhang
- Brain Research Center and State Key Laboratory of Trauma, Burns, and Combined Injury, Third Military Medical University, Chongqing, China
| | - Xiaowei Chen
- Brain Research Center and State Key Laboratory of Trauma, Burns, and Combined Injury, Third Military Medical University, Chongqing, China
| | - Xingyi Li
- Center for Neurointelligence, School of Medicine, Chongqing University, Chongqing, China
| | - Xiang Liao
- Center for Neurointelligence, School of Medicine, Chongqing University, Chongqing, China
| |
Collapse
|
18
|
Triplett MA, Goodhill GJ. Inference of Multiplicative Factors Underlying Neural Variability in Calcium Imaging Data. Neural Comput 2022; 34:1143-1169. [PMID: 35344990 DOI: 10.1162/neco_a_01492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 01/11/2022] [Indexed: 11/04/2022]
Abstract
Understanding brain function requires disentangling the high-dimensional activity of populations of neurons. Calcium imaging is an increasingly popular technique for monitoring such neural activity, but computational tools for interpreting extracted calcium signals are lacking. While there has been a substantial development of factor-analysis-type methods for neural spike train analysis, similar methods targeted at calcium imaging data are only beginning to emerge. Here we develop a flexible modeling framework that identifies low-dimensional latent factors in calcium imaging data with distinct additive and multiplicative modulatory effects. Our model includes spike-and-slab sparse priors that regularize additive factor activity and gaussian process priors that constrain multiplicative effects to vary only gradually, allowing for the identification of smooth and interpretable changes in multiplicative gain. These factors are estimated from the data using a variational expectation-maximization algorithm that requires a differentiable reparameterization of both continuous and discrete latent variables. After demonstrating our method on simulated data, we apply it to experimental data from the zebrafish optic tectum, uncovering low-dimensional fluctuations in multiplicative excitability that govern trial-to-trial variation in evoked responses.
Collapse
Affiliation(s)
- Marcus A Triplett
- Queensland Brain Institute and School of Mathematics and Physics, University of Queensland, St Lucia, QLD 4072, Australia
| | - Geoffrey J Goodhill
- Queensland Brain Institute and School of Mathematics and Physics, University of Queensland, St Lucia, QLD 4072, Australia
| |
Collapse
|
19
|
Sità L, Brondi M, Lagomarsino de Leon Roig P, Curreli S, Panniello M, Vecchia D, Fellin T. A deep-learning approach for online cell identification and trace extraction in functional two-photon calcium imaging. Nat Commun 2022; 13:1529. [PMID: 35318335 PMCID: PMC8940911 DOI: 10.1038/s41467-022-29180-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Accepted: 02/24/2022] [Indexed: 12/11/2022] Open
Abstract
In vivo two-photon calcium imaging is a powerful approach in neuroscience. However, processing two-photon calcium imaging data is computationally intensive and time-consuming, making online frame-by-frame analysis challenging. This is especially true for large field-of-view (FOV) imaging. Here, we present CITE-On (Cell Identification and Trace Extraction Online), a convolutional neural network-based algorithm for fast automatic cell identification, segmentation, identity tracking, and trace extraction in two-photon calcium imaging data. CITE-On processes thousands of cells online, including during mesoscopic two-photon imaging, and extracts functional measurements from most neurons in the FOV. Applied to publicly available datasets, the offline version of CITE-On achieves performance similar to that of state-of-the-art methods for offline analysis. Moreover, CITE-On generalizes across calcium indicators, brain regions, and acquisition parameters in anesthetized and awake head-fixed mice. CITE-On represents a powerful tool to speed up image analysis and facilitate closed-loop approaches, for example in combined all-optical imaging and manipulation experiments. Processing of two-photon calcium imaging data is generally time-consuming, especially for large fields of view. Here, the authors present CITE-On, a tool based on a convolutional neural network, enabling online automatic cell identification, segmentation, identity tracking, and trace extraction.
Collapse
Affiliation(s)
- Luca Sità
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy.
| | - Marco Brondi
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy.
| | - Pedro Lagomarsino de Leon Roig
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy.,University of Genova, Genova, Italy
| | - Sebastiano Curreli
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy
| | - Mariangela Panniello
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy
| | - Dania Vecchia
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy
| | - Tommaso Fellin
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy.
| |
Collapse
|
20
|
Janiak FK, Bartel P, Bale MR, Yoshimatsu T, Komulainen E, Zhou M, Staras K, Prieto-Godino LL, Euler T, Maravall M, Baden T. Non-telecentric two-photon microscopy for 3D random access mesoscale imaging. Nat Commun 2022; 13:544. [PMID: 35087041 PMCID: PMC8795402 DOI: 10.1038/s41467-022-28192-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2020] [Accepted: 01/04/2022] [Indexed: 01/07/2023] Open
Abstract
Diffraction-limited two-photon microscopy permits minimally invasive optical monitoring of neuronal activity. However, most conventional two-photon microscopes impose significant constraints on the size of the imaging field-of-view and the specific shape of the effective excitation volume, thus limiting the scope of biological questions that can be addressed and the information obtainable. Here, employing a non-telecentric optical design, we present a low-cost, easily implemented and flexible solution to address these limitations, offering a several-fold expanded three-dimensional field of view. Moreover, rapid laser-focus control via an electrically tunable lens allows near-simultaneous imaging of remote regions separated in three dimensions and permits the bending of imaging planes to follow natural curvatures in biological structures. Crucially, our core design is readily implemented (and reversed) within a matter of hours, making it highly suitable as a base platform for further development. We demonstrate the application of our system for imaging neuronal activity in a variety of examples in zebrafish, mice and fruit flies.
Collapse
Affiliation(s)
- F K Janiak
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK.
| | - P Bartel
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - M R Bale
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - T Yoshimatsu
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - E Komulainen
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - M Zhou
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - K Staras
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | | | - T Euler
- Institute of Ophthalmic Research, University of Tübingen, Tübingen, Germany
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany
| | - M Maravall
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK
| | - T Baden
- Sussex Neuroscience, School of Life Sciences, University of Sussex, Brighton, UK.
- Institute of Ophthalmic Research, University of Tübingen, Tübingen, Germany.
| |
Collapse
|
21
|
Bao Y, Redington E, Agarwal A, Gong Y. Decontaminate Traces From Fluorescence Calcium Imaging Videos Using Targeted Non-negative Matrix Factorization. Front Neurosci 2022; 15:797421. [PMID: 35126042 PMCID: PMC8815790 DOI: 10.3389/fnins.2021.797421] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Accepted: 12/06/2021] [Indexed: 01/26/2023] Open
Abstract
Fluorescence microscopy and genetically encoded calcium indicators help understand brain function by recording large-scale in vivo videos in assorted animal models. Extracting the fluorescent transients that represent active periods of individual neurons is a key step when analyzing imaging videos. Non-specific calcium sources and background adjacent to segmented neurons contaminate the neurons’ temporal traces with false transients. We developed and characterized a novel method, temporal unmixing of calcium traces (TUnCaT), to quickly and accurately unmix the calcium signals of neighboring neurons and background. Our algorithm used background subtraction to remove the false transients caused by background fluctuations, and then applied targeted non-negative matrix factorization to remove the false transients caused by neighboring calcium sources. TUnCaT was more accurate than existing algorithms when processing multiple experimental and simulated datasets. TUnCaT’s speed was faster than or comparable to existing algorithms.
Collapse
Affiliation(s)
- Yijun Bao
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
- *Correspondence: Yijun Bao,
| | - Emily Redington
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Agnim Agarwal
- North Carolina School of Science and Mathematics, Durham, NC, United States
| | - Yiyang Gong
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
- Department of Neurobiology, Duke University, Durham, NC, United States
- Yiyang Gong,
| |
Collapse
|
22
|
Charles AS, Cermak N, Affan RO, Scott BB, Schiller J, Mishne G. GraFT: Graph Filtered Temporal Dictionary Learning for Functional Neural Imaging. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:3509-3524. [PMID: 35533160 PMCID: PMC9278524 DOI: 10.1109/tip.2022.3171414] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Optical imaging of calcium signals in the brain has enabled researchers to observe the activity of hundreds-to-thousands of individual neurons simultaneously. Current methods predominantly use morphological information, typically focusing on expected shapes of cell bodies, to better identify neurons in the field-of-view. The explicit shape constraints limit the applicability of automated cell identification to other important imaging scales with more complex morphologies, e.g., dendritic or widefield imaging. Specifically, fluorescing components may be broken up, incompletely found, or merged in ways that do not accurately describe the underlying neural activity. Here we present Graph Filtered Temporal Dictionary (GraFT), a new approach that frames the problem of isolating independent fluorescing components as a dictionary learning problem. Specifically, we focus on the time-traces-the main quantity used in scientific discovery-and learn a time trace dictionary with the spatial maps acting as the presence coefficients encoding which pixels the time-traces are active in. Furthermore, we present a novel graph filtering model which redefines connectivity between pixels in terms of their shared temporal activity, rather than spatial proximity. This model greatly eases the ability of our method to handle data with complex non-local spatial structure. We demonstrate important properties of our method, such as robustness to morphology, simultaneously detecting different neuronal types, and implicitly inferring number of neurons, on both synthetic data and real data examples. Specifically, we demonstrate applications of our method to calcium imaging both at the dendritic, somatic, and widefield scales.
Collapse
|
23
|
Meamardoost S, Bhattacharya M, Hwang EJ, Komiyama T, Mewes C, Wang L, Zhang Y, Gunawan R. FARCI: Fast and Robust Connectome Inference. Brain Sci 2021; 11:1556. [PMID: 34942857 PMCID: PMC8699247 DOI: 10.3390/brainsci11121556] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2021] [Revised: 11/12/2021] [Accepted: 11/18/2021] [Indexed: 11/17/2022] Open
Abstract
The inference of neuronal connectome from large-scale neuronal activity recordings, such as two-photon Calcium imaging, represents an active area of research in computational neuroscience. In this work, we developed FARCI (Fast and Robust Connectome Inference), a MATLAB package for neuronal connectome inference from high-dimensional two-photon Calcium fluorescence data. We employed partial correlations as a measure of the functional association strength between pairs of neurons to reconstruct a neuronal connectome. We demonstrated using in silico datasets from the Neural Connectomics Challenge (NCC) and those generated using the state-of-the-art simulator of Neural Anatomy and Optimal Microscopy (NAOMi) that FARCI provides an accurate connectome and its performance is robust to network sizes, missing neurons, and noise levels. Moreover, FARCI is computationally efficient and highly scalable to large networks. In comparison with the best performing connectome inference algorithm in the NCC, Generalized Transfer Entropy (GTE), and Fluorescence Single Neuron and Network Analysis Package (FluoroSNNAP), FARCI produces more accurate networks over different network sizes, while providing significantly better computational speed and scaling.
Collapse
Affiliation(s)
- Saber Meamardoost
- Department of Chemical and Biological Engineering, University at Buffalo, Buffalo, NY 14260, USA;
| | | | - Eun Jung Hwang
- Neurobiology Section, Center for Neural Circuits and Behavior, Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA; (E.J.H.); (T.K.)
- Cell Biology and Anatomy Discipline, Center for Brain Function and Repair, Chicago Medical School, Rosalind Franklin University of Medicine and Science, North Chicago, IL 60064, USA
| | - Takaki Komiyama
- Neurobiology Section, Center for Neural Circuits and Behavior, Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, USA; (E.J.H.); (T.K.)
| | - Claudia Mewes
- Department of Physics and Astronomy, University of Alabama, Tuscaloosa, AL 35487, USA;
| | - Linbing Wang
- Department of Civil and Environmental Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA;
| | - Ying Zhang
- Department of Cell and Molecular Biology, University of Rhode Island, Kingston, RI 02881, USA;
| | - Rudiyanto Gunawan
- Department of Chemical and Biological Engineering, University at Buffalo, Buffalo, NY 14260, USA;
| |
Collapse
|
24
|
Lecoq J, Oliver M, Siegle JH, Orlova N, Ledochowitsch P, Koch C. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat Methods 2021; 18:1401-1408. [PMID: 34650233 PMCID: PMC8833814 DOI: 10.1038/s41592-021-01285-2] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 08/30/2021] [Indexed: 11/09/2022]
Abstract
Progress in many scientific disciplines is hindered by the presence of independent noise. Technologies for measuring neural activity (calcium imaging, extracellular electrophysiology and functional magnetic resonance imaging (fMRI)) operate in domains in which independent noise (shot noise and/or thermal noise) can overwhelm physiological signals. Here, we introduce DeepInterpolation, a general-purpose denoising algorithm that trains a spatiotemporal nonlinear interpolation model using only raw noisy samples. Applying DeepInterpolation to two-photon calcium imaging data yielded up to six times more neuronal segments than those computed from raw data with a 15-fold increase in the single-pixel signal-to-noise ratio (SNR), uncovering single-trial network dynamics that were previously obscured by noise. Extracellular electrophysiology recordings processed with DeepInterpolation yielded 25% more high-quality spiking units than those computed from raw data, while DeepInterpolation produced a 1.6-fold increase in the SNR of individual voxels in fMRI datasets. Denoising was attained without sacrificing spatial or temporal resolution and without access to ground truth training data. We anticipate that DeepInterpolation will provide similar benefits in other domains in which independent noise contaminates spatiotemporally structured datasets.
Collapse
Affiliation(s)
- Jérôme Lecoq
- MindScope Program, Allen Institute, Seattle, WA, USA.
| | | | | | | | | | - Christof Koch
- MindScope Program, Allen Institute, Seattle, WA, USA
| |
Collapse
|
25
|
Vishniakou I, Seelig JD. Differentiable model-based adaptive optics for two-photon microscopy. OPTICS EXPRESS 2021; 29:21418-21427. [PMID: 34265930 DOI: 10.1364/oe.424344] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Accepted: 06/01/2021] [Indexed: 06/13/2023]
Abstract
Aberrations limit scanning fluorescence microscopy when imaging in scattering materials such as biological tissue. Model-based approaches for adaptive optics take advantage of a computational model of the optical setup. Such models can be combined with the optimization techniques of machine learning frameworks to find aberration corrections, as was demonstrated for focusing a laser beam through aberrations onto a camera [Opt. Express2826436 (26436)10.1364/OE.403487]. Here, we extend this approach to two-photon scanning microscopy. The developed sensorless technique finds corrections for aberrations in scattering samples and will be useful for a range of imaging application, for example in brain tissue.
Collapse
|
26
|
Denis J, Dard RF, Quiroli E, Cossart R, Picardo MA. DeepCINAC: A Deep-Learning-Based Python Toolbox for Inferring Calcium Imaging Neuronal Activity Based on Movie Visualization. eNeuro 2020; 7:ENEURO.0038-20.2020. [PMID: 32699072 PMCID: PMC7438055 DOI: 10.1523/eneuro.0038-20.2020] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2020] [Revised: 07/06/2020] [Accepted: 07/10/2020] [Indexed: 11/21/2022] Open
Abstract
Two-photon calcium imaging is now widely used to infer neuronal dynamics from changes in fluorescence of an indicator. However, state-of-the-art computational tools are not optimized for the reliable detection of fluorescence transients from highly synchronous neurons located in densely packed regions such as the CA1 pyramidal layer of the hippocampus during early postnatal stages of development. Indeed, the latest analytical tools often lack proper benchmark measurements. To meet this challenge, we first developed a graphical user interface (GUI) allowing for a precise manual detection of all calcium transients from imaged neurons based on the visualization of the calcium imaging movie. Then, we analyzed movies from mouse pups using a convolutional neural network (CNN) with an attention process and a bidirectional long-short term memory (LSTM) network. This method is able to reach human performance and offers a better F1 score (harmonic mean of sensitivity and precision) than CaImAn to infer neural activity in the developing CA1 without any user intervention. It also enables automatically identifying activity originating from GABAergic neurons. Overall, DeepCINAC offers a simple, fast and flexible open-source toolbox for processing a wide variety of calcium imaging datasets while providing the tools to evaluate its performance.
Collapse
Affiliation(s)
| | | | | | - Rosa Cossart
- Aix Marseille Univ, INSERM, INMED, Marseille 13273, France
| | | |
Collapse
|