Automatic Cell Segmentation by Adaptive Thresholding (ACSAT) for Large-Scale Calcium Imaging Datasets

Accurate neuron segmentation method for one-photon calcium imaging videos combining convolutional neural networks and clustering

One-photon fluorescent calcium imaging helps understand brain functions by recording large-scale neural activities in freely moving animals. Automatic, fast, and accurate active neuron segmentation algorithms are essential to extract and interpret information from these videos. One-photon imaging videos' low resolution, high noise, and high background fluctuation pose significant challenges. Here, we develop a software pipeline to address the challenges of processing one-photon calcium imaging videos. We extend our previous two-photon active neuron segmentation algorithm, Shallow U-Net Neuron Segmentation (SUNS), to better suppress background fluctuations in one-photon videos. We also develop additional neuron extraction (ANE) to locate small or dim neurons missed by SUNS. To train our segmentation method, we create ground truth neurons by developing a manual labeling pipeline assisted with semi-automatic refinement. Our method is more accurate and faster than state-of-the-art techniques when processing simulated videos and multiple experimental datasets acquired over various brain regions with different imaging conditions.

NeuroSeg-III: efficient neuron segmentation in two-photon Ca2+ imaging data using self-supervised learning

Two-photon Ca2+ imaging technology increasingly plays an essential role in neuroscience research. However, the requirement for extensive professional annotation poses a significant challenge to improving the performance of neuron segmentation models. Here, we present NeuroSeg-III, an innovative self-supervised learning approach specifically designed to achieve fast and precise segmentation of neurons in imaging data. This approach consists of two modules: a self-supervised pre-training network and a segmentation network. After pre-training the encoder of the segmentation network via a self-supervised learning method without any annotated data, we only need to fine-tune the segmentation network with a small amount of annotated data. The segmentation network is designed with YOLOv8s, FasterNet, efficient multi-scale attention mechanism (EMA), and bi-directional feature pyramid network (BiFPN), which enhanced the model's segmentation accuracy while reducing the computational cost and parameters. The generalization of our approach was validated across different Ca2+ indicators and scales of imaging data. Significantly, the proposed neuron segmentation approach exhibits exceptional speed and accuracy, surpassing the current state-of-the-art benchmarks when evaluated using a publicly available dataset. The results underscore the effectiveness of NeuroSeg-III, with employing an efficient training strategy tailored for two-photon Ca2+ imaging data and delivering remarkable precision in neuron segmentation.

Dilated Heterogeneous Convolution for Cell Detection and Segmentation Based on Mask R-CNN

Owing to the variable shapes, large size difference, uneven grayscale, and dense distribution among biological cells in an image, it is very difficult to accurately detect and segment cells. Especially, it is a serious challenge for some microscope imaging devices with limited resources owing to a large number of learning parameters and computational burden when using the standard Mask R-CNN. In this work, we propose a mask R-DHCNN for cell detection and segmentation. More specifically, Dilation Heterogeneous Convolution (DHConv) is proposed by designing a novel convolutional kernel structure (i.e., DHConv), which integrates the strengths of the heterogeneous kernel structure and dilated convolution. Then, the traditional homogeneous convolution structure of the standard Mask R-CNN is replaced with the proposed DHConv module to it adapt to shape and size differences encountered in cell detection and segmentation tasks. Finally, a series of comparison and ablation experiments are conducted on various biological cell datasets (such as U373, GoTW1, SIM+, and T24) to verify the effectiveness of the proposed method. The results show that the proposed method can obtain better performance than some state-of-the-art methods in multiple metrics (including AP, Precision, Recall, Dice, and PQ) while maintaining competitive FLOPs and FPS.

DeepSeeded: Volumetric Segmentation of Dense Cell Populations with a Cascade of Deep Neural Networks in Bacterial Biofilm Applications

Accurate and automatic segmentation of individual cell instances in microscopy images is a vital step for quantifying the cellular attributes, which can subsequently lead to new discoveries in biomedical research. In recent years, data-driven deep learning techniques have shown promising results in this task. Despite the success of these techniques, many fail to accurately segment cells in microscopy images with high cell density and low signal-to-noise ratio. In this paper, we propose a novel 3D cell segmentation approach DeepSeeded, a cascaded deep learning architecture that estimates seeds for a classical seeded watershed segmentation. The cascaded architecture enhances the cell interior and border information using Euclidean distance transforms and detects the cell seeds by performing voxel-wise classification. The data-driven seed estimation process proposed here allows segmenting touching cell instances from a dense, intensity-inhomogeneous microscopy image volume. We demonstrate the performance of the proposed method in segmenting 3D microscopy images of a particularly dense cell population called bacterial biofilms. Experimental results on synthetic and two real biofilm datasets suggest that the proposed method leads to superior segmentation results when compared to state-of-the-art deep learning methods and a classical method.

An Image Processing Algorithm for Facile and Reproducible Quantification of Vomocytosis

Vomocytosis is a process that occurs when internalized fungal pathogens escape from phagocytes without compromising the viability of the pathogen and the host cell. Manual quantification of time-lapse microscopy videos is currently used as the standard to study pathogen behavior and vomocytosis incidence. However, human-driven quantification of vomocytosis (and the closely related phenomenon, exocytosis) is incredibly burdensome, especially when a large volume of cells and interactions needs to be analyzed. In this study, we designed a MATLAB algorithm that measures the extent of colocalization between the phagocyte and fungal cell (Cryptococcus neoformans; CN) and rapidly reports the occurrence of vomocytosis in a high throughput manner. Our code processes multichannel, time-lapse microscopy videos of cocultured CN and immune cells that have each been fluorescently stained with unique dyes and provides quantitative readouts of the spatiotemporally dynamic process that is vomocytosis. This study also explored metrics, such as the rate of change of pathogen colocalization with the host cell, that could potentially be used to predict vomocytosis occurrence based on the quantitative data collected. Ultimately, the algorithm quantifies vomocytosis events and reduces the time for video analysis from over 1 h to just 10 min, a reduction in labor of 83%, while simultaneously minimizing human error. This tool significantly minimizes the vomocytosis analysis pipeline, accelerates our ability to elucidate unstudied aspects of this phenomenon, and expedites our ability to characterize CN strains for the study of their epidemiology and virulence.

The autism spectrum disorder risk gene NEXMIF over-synchronizes hippocampal CA1 network and alters neuronal coding

Mutations in autism spectrum disorder (ASD) risk genes disrupt neural network dynamics that ultimately lead to abnormal behavior. To understand how ASD-risk genes influence neural circuit computation during behavior, we analyzed the hippocampal network by performing large-scale cellular calcium imaging from hundreds of individual CA1 neurons simultaneously in transgenic mice with total knockout of the X-linked ASD-risk gene NEXMIF (neurite extension and migration factor). As NEXMIF knockout in mice led to profound learning and memory deficits, we examined the CA1 network during voluntary locomotion, a fundamental component of spatial memory. We found that NEXMIF knockout does not alter the overall excitability of individual neurons but exaggerates movement-related neuronal responses. To quantify network functional connectivity changes, we applied closeness centrality analysis from graph theory to our large-scale calcium imaging datasets, in addition to using the conventional pairwise correlation analysis. Closeness centrality analysis considers both the number of connections and the connection strength between neurons within a network. We found that in wild-type mice the CA1 network desynchronizes during locomotion, consistent with increased network information coding during active behavior. Upon NEXMIF knockout, CA1 network is over-synchronized regardless of behavioral state and fails to desynchronize during locomotion, highlighting how perturbations in ASD-implicated genes create abnormal network synchronization that could contribute to ASD-related behaviors.

Computer vision meets microfluidics: a label-free method for high-throughput cell analysis

In this paper, we review the integration of microfluidic chips and computer vision, which has great potential to advance research in the life sciences and biology, particularly in the analysis of cell imaging data. Microfluidic chips enable the generation of large amounts of visual data at the single-cell level, while computer vision techniques can rapidly process and analyze these data to extract valuable information about cellular health and function. One of the key advantages of this integrative approach is that it allows for noninvasive and low-damage cellular characterization, which is important for studying delicate or fragile microbial cells. The use of microfluidic chips provides a highly controlled environment for cell growth and manipulation, minimizes experimental variability and improves the accuracy of data analysis. Computer vision can be used to recognize and analyze target species within heterogeneous microbial populations, which is important for understanding the physiological status of cells in complex biological systems. As hardware and artificial intelligence algorithms continue to improve, computer vision is expected to become an increasingly powerful tool for in situ cell analysis. The use of microelectromechanical devices in combination with microfluidic chips and computer vision could enable the development of label-free, automatic, low-cost, and fast cellular information recognition and the high-throughput analysis of cellular responses to different compounds, for broad applications in fields such as drug discovery, diagnostics, and personalized medicine.

NeuroSeg-II: A deep learning approach for generalized neuron segmentation in two-photon Ca2+ imaging

The development of two-photon microscopy and Ca²⁺ indicators has enabled the recording of multiscale neuronal activities in vivo and thus advanced the understanding of brain functions. However, it is challenging to perform automatic, accurate, and generalized neuron segmentation when processing a large amount of imaging data. Here, we propose a novel deep-learning-based neural network, termed as NeuroSeg-II, to conduct automatic neuron segmentation for in vivo two-photon Ca²⁺ imaging data. This network architecture is based on Mask region-based convolutional neural network (R-CNN) but has enhancements of an attention mechanism and modified feature hierarchy modules. We added an attention mechanism module to focus the computation on neuron regions in imaging data. We also enhanced the feature hierarchy to extract feature information at diverse levels. To incorporate both spatial and temporal information in our data processing, we fused the images from average projection and correlation map extracting the temporal information of active neurons, and the integrated information was expressed as two-dimensional (2D) images. To achieve a generalized neuron segmentation, we conducted a hybrid learning strategy by training our model with imaging data from different labs, including multiscale data with different Ca²⁺ indicators. The results showed that our approach achieved promising segmentation performance across different imaging scales and Ca²⁺ indicators, even including the challenging data of large field-of-view mesoscopic images. By comparing state-of-the-art neuron segmentation methods for two-photon Ca²⁺ imaging data, we showed that our approach achieved the highest accuracy with a publicly available dataset. Thus, NeuroSeg-II enables good segmentation accuracy and a convenient training and testing process.

Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A Survey

Calcium imaging has rapidly become a methodology of choice for real-time in vivo neuron analysis. Its application to large sets of data requires automated tools to annotate and segment cells, allowing scalable image segmentation under reproducible criteria. In this paper, we review and summarize the most recent methods for computational segmentation of calcium imaging. The contributions of the paper are three-fold: we provide an overview of the main algorithms taxonomized in three categories (signal processing, matrix factorization and machine learning-based approaches), we highlight the main advantages and disadvantages of each category and we provide a summary of the performance of the methods that have been tested on public benchmarks (with links to the public code when available).

Physics-based machine learning for subcellular segmentation in living cells

Segmenting subcellular structures in living cells from fluorescence microscope images is a ground truth (GT)-deficient problem. The microscopes’ three-dimensional blurring function, finite optical resolution due to light diffraction, finite pixel resolution and the complex morphological manifestations of the structures all contribute to GT-hardness. Unsupervised segmentation approaches are quite inaccurate. Therefore, manual segmentation relying on heuristics and experience remains the preferred approach. However, this process is tedious, given the countless structures present inside a single cell, and generating analytics across a large population of cells or performing advanced artificial intelligence tasks such as tracking are greatly limited. Here we bring modelling and deep learning to a nexus for solving this GT-hard problem, improving both the accuracy and speed of subcellular segmentation. We introduce a simulation-supervision approach empowered by physics-based GT, which presents two advantages. First, the physics-based GT resolves the GT-hardness. Second, computational modelling of all the relevant physical aspects assists the deep learning models in learning to compensate, to a great extent, for the limitations of physics and the instrument. We show extensive results on the segmentation of small vesicles and mitochondria in diverse and independent living- and fixed-cell datasets. We demonstrate the adaptability of the approach across diverse microscopes through transfer learning, and illustrate biologically relevant applications of automated analytics and motion analysis.

Graph-Theoretic Post-Processing of Segmentation With Application to Dense Biofilms

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

Region-specific effects of ultrasound on individual neurons in the awake mammalian brain

Ultrasound modulates brain activity. However, it remains unclear how ultrasound affects individual neurons in the brain, where neural circuit architecture is intact and different brain regions exhibit distinct tissue properties. Using a high-resolution calcium imaging technique, we characterized the effect of ultrasound stimulation on thousands of individual neurons in the hippocampus and the motor cortex of awake mice. We found that brief 100-ms-long ultrasound pulses increase intracellular calcium in a large fraction of individual neurons in both brain regions. Ultrasound-evoked calcium response in hippocampal neurons exhibits a rapid onset with a latency shorter than 50 ms. The evoked response in the hippocampus is shorter in duration and smaller in magnitude than that in the motor cortex. These results demonstrate that noninvasive ultrasound stimulation transiently increases intracellular calcium in individual neurons in awake mice, and the evoked response profiles are brain region specific.

Segmentation of Neurons from Fluorescence Calcium Recordings Beyond Real-time

Fluorescent genetically encoded calcium indicators and two-photon microscopy help understand brain function by generating large-scale in vivo recordings in multiple animal models. Automatic, fast, and accurate active neuron segmentation is critical when processing these videos. In this work, we developed and characterized a novel method, Shallow U-Net Neuron Segmentation (SUNS), to quickly and accurately segment active neurons from two-photon fluorescence imaging videos. We used temporal filtering and whitening schemes to extract temporal features associated with active neurons, and used a compact shallow U-Net to extract spatial features of neurons. Our method was both more accurate and an order of magnitude faster than state-of-the-art techniques when processing multiple datasets acquired by independent experimental groups; the difference in accuracy was enlarged when processing datasets containing few manually marked ground truths. We also developed an online version, potentially enabling real-time feedback neuroscience experiments.

Distinct neuronal populations contribute to trace conditioning and extinction learning in the hippocampal CA1

Trace conditioning and extinction learning depend on the hippocampus, but it remains unclear how neural activity in the hippocampus is modulated during these two different behavioral processes. To explore this question, we performed calcium imaging from a large number of individual CA1 neurons during both trace eye-blink conditioning and subsequent extinction learning in mice. Our findings reveal that distinct populations of CA1 cells contribute to trace conditioned learning versus extinction learning, as learning emerges. Furthermore, we examined network connectivity by calculating co-activity between CA1 neuron pairs and found that CA1 network connectivity patterns also differ between conditioning and extinction, even though the overall connectivity density remains constant. Together, our results demonstrate that distinct populations of hippocampal CA1 neurons, forming different sub-networks with unique connectivity patterns, encode different aspects of learning.

Comparative Effects of Event Detection Methods on the Analysis and Interpretation of Ca2+ Imaging Data

Calcium imaging has gained substantial popularity as a tool to profile the activity of multiple simultaneously active cells at high spatiotemporal resolution. Among the diverse approaches to processing of Ca²⁺ imaging data is an often subjective decision of how to quantify baseline fluorescence or F ₀. We examine the effect of popular F ₀ determination methods on the interpretation of neuronal and astrocyte activity in a single dataset of rats trained to self-administer intravenous infusions of cocaine and compare them with an F ₀-independent wavelet ridgewalking event detection approach. We find that the choice of the processing method has a profound impact on the interpretation of widefield imaging results. All of the dF/F ₀ thresholding methods tended to introduce spurious events and fragment individual transients, leading to smaller calculated event durations and larger event frequencies. Analysis of simulated datasets confirmed these observations and indicated substantial intermethod variability as to the events classified as significant. Additionally, most dF/F ₀ methods on their own were unable to adequately account for bleaching of fluorescence, although the F ₀ smooth approach and the wavelet ridgewalking algorithm both did so. In general, the choice of the processing method led to dramatically different quantitative and sometimes opposing qualitative interpretations of the effects of cocaine self-administration both at the level of individual cells and at the level of cell networks. Significantly different distributions of event duration, amplitude, frequency, and network measures were found across the majority of dF/F ₀ approaches. The wavelet ridgewalking algorithm broadly outperformed dF/F ₀-based methods for both neuron and astrocyte recordings. These results indicate the need for heightened awareness of the limitations and tendencies associated with decisions to use particular Ca²⁺ image processing pipelines. Both quantification and interpretation of the effects of experimental manipulations are strongly sensitive to such decisions.

Precision Calcium Imaging of Dense Neural Populations via a Cell-Body-Targeted Calcium Indicator

Methods for one-photon fluorescent imaging of calcium dynamics can capture the activity of hundreds of neurons across large fields of view at a low equipment complexity and cost. In contrast to two-photon methods, however, one-photon methods suffer from higher levels of crosstalk from neuropil, resulting in a decreased signal-to-noise ratio and artifactual correlations of neural activity. We address this problem by engineering cell-body-targeted variants of the fluorescent calcium indicators GCaMP6f and GCaMP7f. We screened fusions of GCaMP to natural, as well as artificial, peptides and identified fusions that localized GCaMP to within 50 μm of the cell body of neurons in mice and larval zebrafish. One-photon imaging of soma-targeted GCaMP in dense neural circuits reported fewer artifactual spikes from neuropil, an increased signal-to-noise ratio, and decreased artifactual correlation across neurons. Thus, soma-targeting of fluorescent calcium indicators facilitates usage of simple, powerful, one-photon methods for imaging neural calcium dynamics.

A Teensy microcontroller-based interface for optical imaging camera control during behavioral experiments

BACKGROUND

Systems neuroscience experiments often require the integration of precisely timed data acquisition and behavioral monitoring. While specialized commercial systems have been designed to meet various needs of data acquisition and device control, they often fail to offer flexibility to interface with new instruments and variable behavioral experimental designs.

NEW METHOD

We developed a Teensy 3.2 microcontroller-based interface that is easily programmable, and offers high-speed, precisely timed behavioral data acquisition and digital and analog outputs for controlling sCMOS cameras and other devices.

RESULTS

We demonstrate the flexibility and the temporal precision of the Teensy interface in two experimental settings. In one example, we used the Teensy interface to record an animal's directional movement on a spherical treadmill, while delivering repeated digital pulses that can be used to control image acquisition from a sCMOS camera. In another example, we used the Teensy interface to deliver an auditory stimulus and a gentle eye puff at precise times in a trace conditioning eye blink behavioral paradigm, while delivering repeated digital pulses to trigger camera image acquisition.

COMPARISON WITH EXISTING METHODS

This interface allows high-speed and temporally precise digital data acquisition and device control during diverse behavioral experiments.

CONCLUSION

The Teensy interface, consisting of a Teensy 3.2 and custom software functions, provides a temporally precise, low-cost, and flexible platform to integrate sCMOS camera control into behavioral experiments.