Improved detection of soma location and morphology in fluorescence microscopy images of neurons

Connecto-informatics at the mesoscale: current advances in image processing and analysis for mapping the brain connectivity

Mapping neural connections within the brain has been a fundamental goal in neuroscience to understand better its functions and changes that follow aging and diseases. Developments in imaging technology, such as microscopy and labeling tools, have allowed researchers to visualize this connectivity through high-resolution brain-wide imaging. With this, image processing and analysis have become more crucial. However, despite the wealth of neural images generated, access to an integrated image processing and analysis pipeline to process these data is challenging due to scattered information on available tools and methods. To map the neural connections, registration to atlases and feature extraction through segmentation and signal detection are necessary. In this review, our goal is to provide an updated overview of recent advances in these image-processing methods, with a particular focus on fluorescent images of the mouse brain. Our goal is to outline a pathway toward an integrated image-processing pipeline tailored for connecto-informatics. An integrated workflow of these image processing will facilitate researchers' approach to mapping brain connectivity to better understand complex brain networks and their underlying brain functions. By highlighting the image-processing tools available for fluroscent imaging of the mouse brain, this review will contribute to a deeper grasp of connecto-informatics, paving the way for better comprehension of brain connectivity and its implications.

Machine learning approach for recognition and morphological analysis of isolated astrocytes in phase contrast microscopy

Astrocytes are glycolytically active cells in the central nervous system playing a crucial role in various brain processes from homeostasis to neurotransmission. Astrocytes possess a complex branched morphology, frequently examined by fluorescent microscopy. However, staining and fixation may impact the properties of astrocytes, thereby affecting the accuracy of the experimental data of astrocytes dynamics and morphology. On the other hand, phase contrast microscopy can be used to study astrocytes morphology without affecting them, but the post-processing of the resulting low-contrast images is challenging. The main result of this work is a novel approach for recognition and morphological analysis of unstained astrocytes based on machine-learning recognition of microscopic images. We conducted a series of experiments involving the cultivation of isolated astrocytes from the rat brain cortex followed by microscopy. Using the proposed approach, we tracked the temporal evolution of the average total length of branches, branching, and area per astrocyte in our experiments. We believe that the proposed approach and the obtained experimental data will be of interest and benefit to the scientific communities in cell biology, biophysics, and machine learning.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

A Deep Learning Approach for Neuronal Cell Body Segmentation in Neurons Expressing GCaMP Using a Swin Transformer

Neuronal cell body analysis is crucial for quantifying changes in neuronal sizes under different physiological and pathologic conditions. Neuronal cell body detection and segmentation mainly rely on manual or pseudo-manual annotations. Manual annotation of neuronal boundaries is time-consuming, requires human expertise, and has intra/interobserver variances. Also, determining where the neuron's cell body ends and where the axons and dendrites begin is taxing. We developed a deep-learning-based approach that uses a state-of-the-art shifted windows (Swin) transformer for automated, reproducible, fast, and unbiased 2D detection and segmentation of neuronal somas imaged in mouse acute brain slices by multiphoton microscopy. We tested our Swin algorithm during different experimental conditions of low and high signal fluorescence. Our algorithm achieved a mean Dice score of 0.91, a precision of 0.83, and a recall of 0.86. Compared with two different convolutional neural networks, the Swin transformer outperformed them in detecting the cell boundaries of GCamP6s expressing neurons. Thus, our Swin transform algorithm can assist in the fast and accurate segmentation of fluorescently labeled neuronal cell bodies in thick acute brain slices. Using our flexible algorithm, researchers can better study the fluctuations in neuronal soma size during physiological and pathologic conditions.

3D Soma Detection in Large-Scale Whole Brain Images via a Two-Stage Neural Network

3D soma detection in whole brain images is a critical step for neuron reconstruction. However, existing soma detection methods are not suitable for whole mouse brain images with large amounts of data and complex structure. In this paper, we propose a two-stage deep neural network to achieve fast and accurate soma detection in large-scale and high-resolution whole mouse brain images (more than 1TB). For the first stage, a lightweight Multi-level Cross Classification Network (MCC-Net) is proposed to filter out images without somas and generate coarse candidate images by combining the advantages of the multi convolution layer's feature extraction ability. It can speed up the detection of somas and reduce the computational complexity. For the second stage, to further obtain the accurate locations of somas in the whole mouse brain images, the Scale Fusion Segmentation Network (SFS-Net) is developed to segment soma regions from candidate images. Specifically, the SFS-Net captures multi-scale context information and establishes a complementary relationship between encoder and decoder by combining the encoder-decoder structure and a 3D Scale-Aware Pyramid Fusion (SAPF) module for better segmentation performance. The experimental results on three whole mouse brain images verify that the proposed method can achieve excellent performance and provide the reconstruction of neurons with beneficial information. Additionally, we have established a public dataset named WBMSD, including 798 high-resolution and representative images ( 256 ×256 ×256 voxels) from three whole mouse brain images, dedicated to the research of soma detection, which will be released along with this paper.

Review of data processing of functional optical microscopy for neuroscience

Functional optical imaging in neuroscience is rapidly growing with the development of optical systems and fluorescence indicators. To realize the potential of these massive spatiotemporal datasets for relating neuronal activity to behavior and stimuli and uncovering local circuits in the brain, accurate automated processing is increasingly essential. We cover recent computational developments in the full data processing pipeline of functional optical microscopy for neuroscience data and discuss ongoing and emerging challenges.

Foreground Estimation in Neuronal Images With a Sparse-Smooth Model for Robust Quantification

3D volume imaging has been regarded as a basic tool to explore the organization and function of the neuronal system. Foreground estimation from neuronal image is essential in the quantification and analysis of neuronal image such as soma counting, neurite tracing and neuron reconstruction. However, the complexity of neuronal structure itself and differences in the imaging procedure, including different optical systems and biological labeling methods, result in various and complex neuronal images, which greatly challenge foreground estimation from neuronal image. In this study, we propose a robust sparse-smooth model (RSSM) to separate the foreground and the background of neuronal image. The model combines the different smoothness levels of the foreground and the background, and the sparsity of the foreground. These prior constraints together contribute to the robustness of foreground estimation from a variety of neuronal images. We demonstrate the proposed RSSM method could promote some best available tools to trace neurites or locate somas from neuronal images with their default parameters, and the quantified results are similar or superior to the results that generated from the original images. The proposed method is proved to be robust in the foreground estimation from different neuronal images, and helps to improve the usability of current quantitative tools on various neuronal images with several applications.

DeepRayburst for Automatic Shape Analysis of Tree-Like Structures in Biomedical Images

Precise quantification of tree-like structures from biomedical images, such as neuronal shape reconstruction and retinal blood vessel caliber estimation, is increasingly important in understanding normal function and pathologic processes in biology. Some handcrafted methods have been proposed for this purpose in recent years. However, they are designed only for a specific application. In this paper, we propose a shape analysis algorithm, DeepRayburst, that can be applied to many different applications based on a Multi-Feature Rayburst Sampling (MFRS) and a Dual Channel Temporal Convolutional Network (DC-TCN). Specifically, we first generate a Rayburst Sampling (RS) core containing a set of multidirectional rays. Then the MFRS is designed by extending each ray of the RS to multiple parallel rays which extract a set of feature sequences. A Gaussian kernel is then used to fuse these feature sequences and outputs one feature sequence. Furthermore, we design a DC-TCN to make the rays terminate on the surface of tree-like structures according to the fused feature sequence. Finally, by analyzing the distribution patterns of the terminated rays, the algorithm can serve multiple shape analysis applications of tree-like structures. Experiments on three different applications, including soma shape reconstruction, neuronal shape reconstruction, and vessel caliber estimation, confirm that the proposed method outperforms other state-of-the-art shape analysis methods, which demonstrate its flexibility and robustness.

ANMAF: an automated neuronal morphology analysis framework using convolutional neural networks

Measurement of neuronal size is challenging due to their complex histology. Current practice includes manual or pseudo-manual measurement of somatic areas, which is labor-intensive and prone to human biases and intra-/inter-observer variances. We developed a novel high-throughput neuronal morphology analysis framework (ANMAF), using convolutional neural networks (CNN) to automatically contour the somatic area of fluorescent neurons in acute brain slices. Our results demonstrate considerable agreements between human annotators and ANMAF on detection, segmentation, and the area of somatic regions in neurons expressing a genetically encoded fluorophore. However, in contrast to humans, who exhibited significant variability in repeated measurements, ANMAF produced consistent neuronal contours. ANMAF was generalizable across different imaging protocols and trainable even with a small number of humanly labeled neurons. Our framework can facilitate more rigorous and quantitative studies of neuronal morphology by enabling the segmentation of many fluorescent neurons in thick brain slices in a standardized manner.

Accurate Neuronal Soma Segmentation Using 3D Multi-Task Learning U-Shaped Fully Convolutional Neural Networks

Neuronal soma segmentation is a crucial step for the quantitative analysis of neuronal morphology. Automated neuronal soma segmentation methods have opened up the opportunity to improve the time-consuming manual labeling required during the neuronal soma morphology reconstruction for large-scale images. However, the presence of touching neuronal somata and variable soma shapes in images brings challenges for automated algorithms. This study proposes a neuronal soma segmentation method combining 3D U-shaped fully convolutional neural networks with multi-task learning. Compared to existing methods, this technique applies multi-task learning to predict the soma boundary to split touching somata, and adopts U-shaped architecture convolutional neural network which is effective for a limited dataset. The contour-aware multi-task learning framework is applied to the proposed method to predict the masks of neuronal somata and boundaries simultaneously. In addition, a spatial attention module is embedded into the multi-task model to improve neuronal soma segmentation results. The Nissl-stained dataset captured by the micro-optical sectioning tomography system is used to validate the proposed method. Following comparison to four existing segmentation models, the proposed method outperforms the others notably in both localization and segmentation. The novel method has potential for high-throughput neuronal soma segmentation in large-scale optical imaging data for neuron morphology quantitative analysis.

Myelin detection in fluorescence microscopy images using machine learning

BACKGROUND

The myelin sheath produced by glial cells insulates the axons, and supports the function of the nervous system. Myelin sheath degeneration causes neurodegenerative disorders, such as multiple sclerosis (MS). There are no therapies for MS that promote remyelination. Drug discovery frequently involves screening thousands of compounds. However, this is not feasible for remyelination drugs, since myelin quantification is a manual labor-intensive endeavor. Therefore, the development of assistive software for expedited myelin detection is instrumental for MS drug discovery by enabling high-content image-based drug screens.

NEW METHOD

In this study, we developed a machine learning based expedited myelin detection approach in fluorescence microscopy images. Multi-channel three-dimensional microscopy images of a mouse stem cell-based myelination assay were labeled by experts. A spectro-spatial feature extraction method was introduced to represent local dependencies of voxels both in spatial and spectral domains. Feature extraction yielded two data set of over forty-seven thousand annotated images in total.

RESULTS

Myelin detection performances of 23 different supervised machine learning techniques including a customized-convolutional neural network (CNN), were assessed using various train/test split ratios of the data sets. The highest accuracy values of 98.84±0.09% and 98.46±0.11% were achieved by Boosted Trees and customized-CNN, respectively.

COMPARISON WITH EXISTING METHODS

Our approach can detect myelin in a common experimental setup. Myelin extending in any orientation in 3 dimensions is segmented from 3 channel z-stack fluorescence images.

CONCLUSIONS

Our results suggest that the proposed expedited myelin detection approach is a feasible and robust method for remyelination drug screening.

A multistep deep learning framework for the automated detection and segmentation of astrocytes in fluorescent images of brain tissue

While astrocytes have been traditionally described as passive supportive cells, studies during the last decade have shown they are active players in many aspects of CNS physiology and function both in normal and disease states. However, the precise mechanisms regulating astrocytes function and interactions within the CNS are still poorly understood. This knowledge gap is due in large part to the limitations of current image analysis tools that cannot process astrocyte images efficiently and to the lack of methods capable of quantifying their complex morphological characteristics. To provide an unbiased and accurate framework for the quantitative analysis of fluorescent images of astrocytes, we introduce a new automated image processing pipeline whose main novelties include an innovative module for cell detection based on multiscale directional filters and a segmentation routine that leverages deep learning and sparse representations to reduce the need of training data and improve performance. Extensive numerical tests show that our method performs very competitively with respect to state-of-the-art methods also in challenging images where astrocytes are clustered together. Our code is released open source and freely available to the scientific community.

Cell numbers, distribution, shape, and regional variation throughout the murine hippocampal formation from the adult brain Allen Reference Atlas

Quantifying the distribution of cells in every brain region is fundamental to attaining a comprehensive census of distinct neuronal and glial types. Until recently, estimating neuron numbers involved time-consuming procedures that were practically limited to stereological sampling. Progress in open-source image recognition software, growth in computing power, and unprecedented neuroinformatics developments now offer the potentially paradigm-shifting alternative of comprehensive cell-by-cell analysis in an entire brain region. The Allen Brain Atlas provides free digital access to complete series of raw Nissl-stained histological section images along with regional delineations. Automated cell segmentation of these data enables reliable and reproducible high-throughput quantification of regional variations in cell count, density, size, and shape at whole-system scale. While this strategy is directly applicable to any regions of the mouse brain, we first deploy it here on the closed-loop circuit of the hippocampal formation: the medial and lateral entorhinal cortices; dentate gyrus (DG); areas Cornu Ammonis 3 (CA3), CA2, and CA1; and dorsal and ventral subiculum. Using two independent image processing pipelines and the adult mouse reference atlas, we report the first cellular-level soma segmentation in every sub-region and non-principal layer of the left hippocampal formation through the full rostral-caudal extent. It is important to note that our techniques excluded the layers with the largest number of cells, DG granular and CA pyramidal, due to dense packing. The numerical estimates for the remaining layers are corroborated by traditional stereological sampling on a data subset and well match sparse published reports.

Morphological Segmentation Analysis and Texture-based Support Vector Machines Classification on Mice Liver Fibrosis Microscopic Images

Background:

To reduce the intensity of the work of doctors, pre-classification work needs to be issued. In this paper, a novel and related liver microscopic image classification analysis method is proposed.

Objective:

For quantitative analysis, segmentation is carried out to extract the quantitative information of special organisms in the image for further diagnosis, lesion localization, learning and treating anatomical abnormalities and computer-guided surgery. </P><P> Methods: In the current work, entropy-based features of microscopic fibrosis mice’ liver images were analyzed using fuzzy c-cluster, k-means and watershed algorithms based on distance transformations and gradient. A morphological segmentation based on a local threshold was deployed to determine the fibrosis areas of images.

Results:

The segmented target region using the proposed method achieved high effective microscopy fibrosis images segmenting of mice liver in terms of the running time, dice ratio and precision. The image classification experiments were conducted using Gray Level Co-occurrence Matrix (GLCM). The best classification model derived from the established characteristics was GLCM which performed the highest accuracy of classification using a developed Support Vector Machine (SVM). The training model using 11 features was found to be accurate when only trained by 8 GLCMs.

Conclusion:

The research illustrated that the proposed method is a new feasible research approach for microscopy mice liver image segmentation and classification using intelligent image analysis techniques. It is also reported that the average computational time of the proposed approach was only 2.335 seconds, which outperformed other segmentation algorithms with 0.8125 dice ratio and 0.5253 precision.</P>

Quantitative Determination of Cellular-and Neurite Motility Speed in Dense Cell Cultures

Mobility quantification of single cells and cellular processes in dense cultures is a challenge, because single cell tracking is impossible. We developed a software for cell structure segmentation and implemented 2 algorithms to measure motility speed. Complex algorithms were tested to separate cells and cellular components, an important prerequisite for the acquisition of meaningful motility data. Plasma membrane segmentation was performed to measure membrane contraction dynamics and organelle trafficking. The discriminative performance and sensitivity of the algorithms were tested on different cell types and calibrated on computer-simulated cells to obtain absolute values for cellular velocity. Both motility algorithms had advantages in different experimental setups, depending on the complexity of the cellular movement. The correlation algorithm (COPRAMove) performed best under most tested conditions and appeared less sensitive to variable cell densities, brightness and focus changes than the differentiation algorithm (DiffMove). In summary, our software can be used successfully to analyze and quantify cellular and subcellular movements in dense cell cultures.

Soma Detection in 3D Images of Neurons using Machine Learning Technique

Computing and analyzing the neuronal structure is essential to studying connectome. Two important tasks for such analysis are finding the soma and constructing the neuronal structure. Finding the soma is considered more important because it is required for some neuron tracing algorithms. We describe a robust automatic soma detection method developed based on the machine learning technique. Images of neurons were three-dimensional confocal microscopic images in the FlyCircuit database. The testing data were randomly selected raw images that contained noises and partial neuronal structures. The number of somas in the images was not known in advance. Our method tries to identify all the somas in the images. Experimental results showed that the method is efficient and robust.

Automated 3D Soma Segmentation with Morphological Surface Evolution for Neuron Reconstruction

The automatic neuron reconstruction is important since it accelerates the collection of 3D neuron models for the neuronal morphological studies. The majority of the previous neuron reconstruction methods only focused on tracing neuron fibres without considering the somatic surface. Thus, topological errors often present around the soma area in the results obtained by these tracing methods. Segmentation of the soma structures can be embedded in the existing neuron tracing methods to reduce such topological errors. In this paper, we present a novel method to segment the soma structures with complex geometry. It can be applied along with the existing methods in a fully automated pipeline. An approximate bounding block is firstly estimated based on a geodesic distance transform. Then the soma segmentation is obtained by evolving the surface with a set of morphological operators inside the initial bounding region. By evaluating the methods against the challenging images released by the BigNeuron project, we showed that the proposed method can outperform the existing soma segmentation methods regarding the accuracy. We also showed that the soma segmentation can be used for enhancing the results of existing neuron tracing methods.

Automated sorting of neuronal trees in fluorescent images of neuronal networks using NeuroTreeTracer

Fluorescence confocal microscopy has become increasingly more important in neuroscience due to its applications in image-based screening and profiling of neurons. Multispectral confocal imaging is useful to simultaneously probe for distribution of multiple analytes over networks of neurons. However, current automated image analysis algorithms are not designed to extract single-neuron arbors in images where neurons are not separated, hampering the ability map fluorescence signals at the single cell level. To overcome this limitation, we introduce NeuroTreeTracer - a novel image processing framework aimed at automatically extracting and sorting single-neuron traces in fluorescent images of multicellular neuronal networks. This method applies directional multiscale filters for automated segmentation of neurons and soma detection, and includes a novel tracing routine that sorts neuronal trees in the image by resolving network connectivity even when neurites appear to intersect. By extracting each neuronal tree, NeuroTreetracer enables to automatically quantify the spatial distribution of analytes of interest in the subcellular compartments of individual neurons. This software is released open-source and freely available with the goal to facilitate applications in neuron screening and profiling.