Imagining the future of bioimage analysis

Made to measure: An introduction to quantifying microscopy data in the life sciences

Images are at the core of most modern biological experiments and are used as a major source of quantitative information. Numerous algorithms are available to process images and make them more amenable to be measured. Yet the nature of the quantitative output that is useful for a given biological experiment is uniquely dependent upon the question being investigated. Here, we discuss the 3 main types of information that can be extracted from microscopy data: intensity, morphology, and object counts or categorical labels. For each, we describe where they come from, how they can be measured, and what may affect the relevance of these measurements in downstream data analysis. Acknowledging that what makes a measurement 'good' is ultimately down to the biological question being investigated, this review aims at providing readers with a toolkit to challenge how they quantify their own data and be critical of conclusions drawn from quantitative bioimage analysis experiments.

Machine learning aided single cell image analysis improves understanding of morphometric heterogeneity of human mesenchymal stem cells

The multipotent stem cells of our body have been largely harnessed in biotherapeutics. However, as they are derived from multiple anatomical sources, from different tissues, human mesenchymal stem cells (hMSCs) are a heterogeneous population showing ambiguity in their in vitro behavior. Intra-clonal population heterogeneity has also been identified and pre-clinical mechanistic studies suggest that these cumulatively depreciate the therapeutic effects of hMSC transplantation. Although various biomarkers identify these specific stem cell populations, recent artificial intelligence-based methods have capitalized on the cellular morphologies of hMSCs, opening a new approach to understand their attributes. A robust and rapid platform is required to accommodate and eliminate the heterogeneity observed in the cell population, to standardize the quality of hMSC therapeutics globally. Here, we report our primary findings of morphological heterogeneity observed within and across two sources of hMSCs namely, stem cells from human exfoliated deciduous teeth (SHEDs) and human Wharton jelly mesenchymal stem cells (hWJ MSCs), using real-time single-cell images generated on immunophenotyping by imaging flow cytometry (IFC). We used the ImageJ software for identification and comparison between the two types of hMSCs using statistically significant morphometric descriptors that are biologically relevant. To expand on these insights, we have further applied deep learning methods and successfully report the development of a Convolutional Neural Network-based image classifier. In our research, we introduced a machine learning methodology to streamline the entire procedure, utilizing convolutional neural networks and transfer learning for binary classification, achieving an accuracy rate of 97.54%. We have also critically discussed the challenges, comparisons between solutions and future directions of machine learning in hMSC classification in biotherapeutics.

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets

Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models.

Automated staging of zebrafish embryos with deep learning

The zebrafish (Danio rerio) is an important biomedical model organism used in many disciplines. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype. However, the detection and quantification of these delays is often achieved through manual observation, which is both time-consuming and subjective. We present KimmelNet, a deep learning model trained to predict embryo age (hours post fertilisation) from 2D brightfield images. KimmelNet's predictions agree closely with established staging methods and can detect developmental delays between populations with high confidence using as few as 100 images. Moreover, KimmelNet generalises to previously unseen data, with transfer learning enhancing its performance. With the ability to analyse tens of thousands of standard brightfield microscopy images on a timescale of minutes, we envisage that KimmelNet will be a valuable resource for the developmental biology community. Furthermore, the approach we have used could easily be adapted to generate models for other organisms.

Heterosynaptic plasticity-induced modulation of synapses

Plasticity is a common feature of synapses that is stated in different ways and occurs through several mechanisms. The regular action of the brain needs to be balanced in several neuronal and synaptic features, one of which is synaptic plasticity. The different homeostatic processes, including the balance between excitation/inhibition or homeostasis of synaptic weights at the single-neuron level, may obtain this. Homosynaptic Hebbian-type plasticity causes associative alterations of synapses. Both homosynaptic and heterosynaptic plasticity characterize the corresponding aspects of adjustable synapses, and both are essential for the regular action of neural systems and their plastic synapses.In this review, we will compare homo- and heterosynaptic plasticity and the main factors affecting the direction of plastic changes. This review paper will also discuss the diverse functions of the different kinds of heterosynaptic plasticity and their properties. We argue that a complementary system of heterosynaptic plasticity demonstrates an essential cellular constituent for homeostatic modulation of synaptic weights and neuronal activity.

Whole-brain Optical Imaging: A Powerful Tool for Precise Brain Mapping at the Mesoscopic Level

The mammalian brain is a highly complex network that consists of millions to billions of densely-interconnected neurons. Precise dissection of neural circuits at the mesoscopic level can provide important structural information for understanding the brain. Optical approaches can achieve submicron lateral resolution and achieve "optical sectioning" by a variety of means, which has the natural advantage of allowing the observation of neural circuits at the mesoscopic level. Automated whole-brain optical imaging methods based on tissue clearing or histological sectioning surpass the limitation of optical imaging depth in biological tissues and can provide delicate structural information in a large volume of tissues. Combined with various fluorescent labeling techniques, whole-brain optical imaging methods have shown great potential in the brain-wide quantitative profiling of cells, circuits, and blood vessels. In this review, we summarize the principles and implementations of various whole-brain optical imaging methods and provide some concepts regarding their future development.

Internalization of Pegylated Er:Y2O3 Nanoparticles inside HCT-116 Cancer Cells: Implications for Imaging and Drug Delivery

Lanthanide-doped nanoparticles, featuring sharp emission peaks with narrow bandwidth, exhibit high downconversion luminescence intensity, making them highly valuable in the fields of bioimaging and drug delivery. High-crystallinity Y2O3 nanoparticles (NPs) doped with Er3+ ions were functionalized by using a pegylation procedure to confer water solubility and biocompatibility. The NPs were thoroughly characterized using transmission electron microscopy (TEM), inductively coupled plasma mass spectrometry (ICP-MS), and photoluminescence measurements. The pegylated nanoparticles were studied both from a toxicological perspective and to demonstrate their internalization within HCT-116 cancer cells. Cell viability tests allowed for the identification of the "optimal" concentration, which yields a detectable fluorescence signal without being toxic to the cells. The internalization process was investigated using a combined approach involving confocal microscopy and ICP-MS. The obtained data clearly indicate the efficient internalization of NPs into the cells with emission intensity showing a strong correlation with the concentrations of nanoparticles delivered to the cells. Overall, this research contributes significantly to the fields of nanotechnology and biomedical research, with noteworthy implications for imaging and drug delivery applications.

The promise of digital healthcare technologies

Digital health technologies have been in use for many years in a wide spectrum of healthcare scenarios. This narrative review outlines the current use and the future strategies and significance of digital health technologies in modern healthcare applications. It covers the current state of the scientific field (delineating major strengths, limitations, and applications) and envisions the future impact of relevant emerging key technologies. Furthermore, we attempt to provide recommendations for innovative approaches that would accelerate and benefit the research, translation and utilization of digital health technologies.

Phenomics as an approach to Comparative Developmental Physiology

The dynamic nature of developing organisms and how they function presents both opportunity and challenge to researchers, with significant advances in understanding possible by adopting innovative approaches to their empirical study. The information content of the phenotype during organismal development is arguably greater than at any other life stage, incorporating change at a broad range of temporal, spatial and functional scales and is of broad relevance to a plethora of research questions. Yet, effectively measuring organismal development, and the ontogeny of physiological regulations and functions, and their responses to the environment, remains a significant challenge. "Phenomics", a global approach to the acquisition of phenotypic data at the scale of the whole organism, is uniquely suited as an approach. In this perspective, we explore the synergies between phenomics and Comparative Developmental Physiology (CDP), a discipline of increasing relevance to understanding sensitivity to drivers of global change. We then identify how organismal development itself provides an excellent model for pushing the boundaries of phenomics, given its inherent complexity, comparably smaller size, relative to adult stages, and the applicability of embryonic development to a broad suite of research questions using a diversity of species. Collection, analysis and interpretation of whole organismal phenotypic data are the largest obstacle to capitalising on phenomics for advancing our understanding of biological systems. We suggest that phenomics within the context of developing organismal form and function could provide an effective scaffold for addressing grand challenges in CDP and phenomics.

Denoising Autoencoder Trained on Simulation-Derived Structures for Noise Reduction in Chromatin Scanning Transmission Electron Microscopy

Scanning transmission electron microscopy tomography with ChromEM staining (ChromSTEM), has allowed for the three-dimensional study of genome organization. By leveraging convolutional neural networks and molecular dynamics simulations, we have developed a denoising autoencoder (DAE) capable of postprocessing experimental ChromSTEM images to provide nucleosome-level resolution. Our DAE is trained on synthetic images generated from simulations of the chromatin fiber using the 1-cylinder per nucleosome (1CPN) model of chromatin. We find that our DAE is capable of removing noise commonly found in high-angle annular dark field (HAADF) STEM experiments and is able to learn structural features driven by the physics of chromatin folding. The DAE outperforms other well-known denoising algorithms without degradation of structural features and permits the resolution of α-tetrahedron tetranucleosome motifs that induce local chromatin compaction and mediate DNA accessibility. Notably, we find no evidence for the 30 nm fiber, which has been suggested to serve as the higher-order structure of the chromatin fiber. This approach provides high-resolution STEM images that allow for the resolution of single nucleosomes and organized domains within chromatin dense regions comprising of folding motifs that modulate the accessibility of DNA to external biological machinery.

Unbiased single-cell morphology with self-supervised vision transformers

Accurately quantifying cellular morphology at scale could substantially empower existing single-cell approaches. However, measuring cell morphology remains an active field of research, which has inspired multiple computer vision algorithms over the years. Here, we show that DINO, a vision-transformer based, self-supervised algorithm, has a remarkable ability for learning rich representations of cellular morphology without manual annotations or any other type of supervision. We evaluate DINO on a wide variety of tasks across three publicly available imaging datasets of diverse specifications and biological focus. We find that DINO encodes meaningful features of cellular morphology at multiple scales, from subcellular and single-cell resolution, to multi-cellular and aggregated experimental groups. Importantly, DINO successfully uncovers a hierarchy of biological and technical factors of variation in imaging datasets. The results show that DINO can support the study of unknown biological variation, including single-cell heterogeneity and relationships between samples, making it an excellent tool for image-based biological discovery.

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. We observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

Deep learning techniques and mathematical modeling allow 3D analysis of mitotic spindle dynamics

Time-lapse microscopy movies have transformed the study of subcellular dynamics. However, manual analysis of movies can introduce bias and variability, obscuring important insights. While automation can overcome such limitations, spatial and temporal discontinuities in time-lapse movies render methods such as 3D object segmentation and tracking difficult. Here, we present SpinX, a framework for reconstructing gaps between successive image frames by combining deep learning and mathematical object modeling. By incorporating expert feedback through selective annotations, SpinX identifies subcellular structures, despite confounding neighbor-cell information, non-uniform illumination, and variable fluorophore marker intensities. The automation and continuity introduced here allows the precise 3D tracking and analysis of spindle movements with respect to the cell cortex for the first time. We demonstrate the utility of SpinX using distinct spindle markers, cell lines, microscopes, and drug treatments. In summary, SpinX provides an exciting opportunity to study spindle dynamics in a sophisticated way, creating a framework for step changes in studies using time-lapse microscopy.

Automated staging of zebrafish embryos using machine learning

The zebrafish ( Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

Automated staging of zebrafish embryos using machine learning

The zebrafish ( Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

Automated staging of zebrafish embryos using machine learning

The zebrafish (Danio rerio), is an important biomedical model organism used in many disciplines, including development, disease modeling and toxicology, to better understand vertebrate biology. The phenomenon of developmental delay in zebrafish embryos has been widely reported as part of a mutant or treatment-induced phenotype, and accurate characterization of such delays is imperative. Despite this, the only way at present to identify and quantify these delays is through manual observation, which is both time-consuming and subjective. Machine learning approaches in biology are rapidly becoming part of the toolkit used by researchers to address complex questions. In this work, we introduce a machine learning-based classifier that has been trained to detect temporal developmental differences across groups of zebrafish embryos. Our classifier is capable of rapidly analyzing thousands of images, allowing comparisons of developmental temporal rates to be assessed across and between experimental groups of embryos. Finally, as our classifier uses images obtained from a standard live-imaging widefield microscope and camera set-up, we envisage it will be readily accessible to the zebrafish community, and prove to be a valuable resource.

Untargeted Metabolomics for Integrative Taxonomy: Metabolomics, DNA Marker-Based Sequencing, and Phenotype Bioimaging

Integrative taxonomy is a fundamental part of biodiversity and combines traditional morphology with additional methods such as DNA sequencing or biochemistry. Here, we aim to establish untargeted metabolomics for use in chemotaxonomy. We used three thallose liverwort species Riccia glauca, R. sorocarpa, and R. warnstorfii (order Marchantiales, Ricciaceae) with Lunularia cruciata (order Marchantiales, Lunulariacea) as an outgroup. Liquid chromatography high-resolution mass-spectrometry (UPLC/ESI-QTOF-MS) with data-dependent acquisition (DDA-MS) were integrated with DNA marker-based sequencing of the trnL-trnF region and high-resolution bioimaging. Our untargeted chemotaxonomy methodology enables us to distinguish taxa based on chemophenetic markers at different levels of complexity: (1) molecules, (2) compound classes, (3) compound superclasses, and (4) molecular descriptors. For the investigated Riccia species, we identified 71 chemophenetic markers at the molecular level, a characteristic composition in 21 compound classes, and 21 molecular descriptors largely indicating electron state, presence of chemical motifs, and hydrogen bonds. Our untargeted approach revealed many chemophenetic markers at different complexity levels that can provide more mechanistic insight into phylogenetic delimitation of species within a clade than genetic-based methods coupled with traditional morphology-based information. However, analytical and bioinformatics analysis methods still need to be better integrated to link the chemophenetic information at multiple scales.

A fully automatic target detection and quantification strategy based on object detection convolutional neural network YOLOv3 for one-step X-ray image grading

Methods for automatic image analysis are demanded for dealing with the explosively increased imaging data in clinics. Osteoarthritis (OA) is a typical disease diagnosed based on X-ray imaging. Herein, we propose a novel modeling strategy based on YOLO version 3 (YOLOv3) for automatic simultaneous localization of knee joints and quantification of radiographic knee OA. As an advanced deep convolutional neural network (CNN) algorithm for target detection, YOLOv3 enables simultaneous small object detection and quantification due to its unique residual connection and feature map merging. Hence, a unified CNN model is built for the elegant integration of knee joint detection and corresponding OA severity grading using the YOLOv3 framework. We achieve desirable accuracy in knee OA grading using the public and clinical datasets. It provides improvements in the precision, recall, F₁ score and diagnostic accuracy of knee OA as well. Because of the fully automatic target detection and quantification, the time of handling an image is merely 40 ms from inputting the image to getting its label, supporting quick clinic decisions. It, thus, affords convenient and efficient image analysis for daily clinical diagnosis.

Fisheye transformation enhances deep-learning-based single-cell phenotyping by including cellular microenvironment

Incorporating information about the surroundings can have a significant impact on successfully determining the class of an object. This is of particular interest when determining the phenotypes of cells, for example, in the context of high-throughput screens. We hypothesized that an ideal approach would consider the fully featured view of the cell of interest, include its neighboring microenvironment, and give lesser weight to cells that are far from the cell of interest. To satisfy these criteria, we present an approach with a transformation similar to those characteristic of fisheye cameras. Using this transformation with proper settings, we could significantly increase the accuracy of single-cell phenotyping, both in the case of cell culture and tissue-based microscopy images, and we present improved results on a dataset containing images of wild animals.

Differential diagnosis of thyroid nodule capsules using random forest guided selection of image features

Microscopic evaluation of tissue sections stained with hematoxylin and eosin is the current gold standard for diagnosing thyroid pathology. Digital pathology is gaining momentum providing the pathologist with additional cues to traditional routes when placing a diagnosis, therefore it is extremely important to develop new image analysis methods that can extract image features with diagnostic potential. In this work, we use histogram and texture analysis to extract features from microscopic images acquired on thin thyroid nodule capsules sections and demonstrate how they enable the differential diagnosis of thyroid nodules. Targeted thyroid nodules are benign (i.e., follicular adenoma) and malignant (i.e., papillary thyroid carcinoma and its sub-type arising within a follicular adenoma). Our results show that the considered image features can enable the quantitative characterization of the collagen capsule surrounding thyroid nodules and provide an accurate classification of the latter's type using random forest.

A comprehensive review of computational and image analysis techniques for quantitative evaluation of striated muscle tissue architecture

Unbiased evaluation of morphology is crucial to understanding development, mechanics, and pathology of striated muscle tissues. Indeed, the ability of striated muscles to contract and the strength of their contraction is dependent on their tissue-, cellular-, and cytoskeletal-level organization. Accordingly, the study of striated muscles often requires imaging and assessing aspects of their architecture at multiple different spatial scales. While an expert may be able to qualitatively appraise tissues, it is imperative to have robust, repeatable tools to quantify striated myocyte morphology and behavior that can be used to compare across different labs and experiments. There has been a recent effort to define the criteria used by experts to evaluate striated myocyte architecture. In this review, we will describe metrics that have been developed to summarize distinct aspects of striated muscle architecture in multiple different tissues, imaged with various modalities. Additionally, we will provide an overview of metrics and image processing software that needs to be developed. Importantly to any lab working on striated muscle platforms, characterization of striated myocyte morphology using the image processing pipelines discussed in this review can be used to quantitatively evaluate striated muscle tissues and contribute to a robust understanding of the development and mechanics of striated muscles.

Minimizing Molecular Misidentification in Imaging Low-Abundance Protein Interactions Using Spectroscopic Single-Molecule Localization Microscopy

Super-resolution microscopy can capture spatiotemporal organizations of protein interactions with resolution down to 10 nm; however, the analyses of more than two proteins involving low-abundance protein are challenging because spectral crosstalk and heterogeneities of individual fluorescent labels result in molecular misidentification. Here we developed a deep learning-based imaging analysis method for spectroscopic single-molecule localization microscopy to minimize molecular misidentification in three-color super-resolution imaging. We characterized the 3-fold reduction of molecular misidentification in the new imaging method using pure samples of different photoswitchable fluorophores and visualized three distinct subcellular proteins in U2-OS cell lines. We further validated the protein counts and interactions of TOMM20, DRP1, and SUMO1 in a well-studied biological process, Staurosporine-induced apoptosis, by comparing the imaging results with Western-blot analyses of different subcellular portions.

Reference bioimaging to assess the phenotypic trait diversity of bryophytes within the family Scapaniaceae

Macro- and microscopic images of organisms are pivotal in biodiversity research. Despite that bioimages have manifold applications such as assessing the diversity of form and function, FAIR bioimaging data in the context of biodiversity are still very scarce, especially for difficult taxonomic groups such as bryophytes. Here, we present a high-quality reference dataset containing macroscopic and bright-field microscopic images documenting various phenotypic characters of the species belonging to the liverwort family of Scapaniaceae occurring in Europe. To encourage data reuse in biodiversity and adjacent research areas, we annotated the imaging data with machine-actionable metadata using community-accepted semantics. Furthermore, raw imaging data are retained and any contextual image processing like multi-focus image fusion and stitching were documented to foster good scientific practices through source tracking and provenance. The information contained in the raw images are also of particular interest for machine learning and image segmentation used in bioinformatics and computational ecology. We expect that this richly annotated reference dataset will encourage future studies to follow our principles.

Measurement(s)	phenotype
Technology Type(s)	bright-field microscopy
Factor Type(s)	taxonomic identification of different species
Sample Characteristic - Organism	Scapaniaceae

HeartCV: a tool for transferrable, automated measurement of heart rate and heart rate variability in transparent animals

Heart function is a key component of whole-organismal physiology. Bioimaging is commonly, but not exclusively, used for quantifying heart function in transparent individuals, including early developmental stages of aquatic animals, many of which are transparent. However, a central limitation of many imaging-related methods is the lack of transferability between species, life-history stages and experimental approaches. Furthermore, locating the heart in mobile individuals remains challenging. Here, we present HeartCV: an open-source Python package for automated measurement of heart rate and heart rate variability that integrates automated localization and is transferrable across a wide range of species. We demonstrate the efficacy of HeartCV by comparing its outputs with measurements made manually for a number of very different species with contrasting heart morphologies. Lastly, we demonstrate the applicability of the software to different experimental approaches and to different dataset types, such as those corresponding to longitudinal studies.

Mars, a molecule archive suite for reproducible analysis and reporting of single-molecule properties from bioimages

The rapid development of new imaging approaches is generating larger and more complex datasets, revealing the time evolution of individual cells and biomolecules. Single-molecule techniques, in particular, provide access to rare intermediates in complex, multistage molecular pathways. However, few standards exist for processing these information-rich datasets, posing challenges for wider dissemination. Here, we present Mars, an open-source platform for storing and processing image-derived properties of biomolecules. Mars provides Fiji/ImageJ2 commands written in Java for common single-molecule analysis tasks using a Molecule Archive architecture that is easily adapted to complex, multistep analysis workflows. Three diverse workflows involving molecule tracking, multichannel fluorescence imaging, and force spectroscopy, demonstrate the range of analysis applications. A comprehensive graphical user interface written in JavaFX enhances biomolecule feature exploration by providing charting, tagging, region highlighting, scriptable dashboards, and interactive image views. The interoperability of ImageJ2 ensures Molecule Archives can easily be opened in multiple environments, including those written in Python using PyImageJ, for interactive scripting and visualization. Mars provides a flexible solution for reproducible analysis of image-derived properties, facilitating the discovery and quantitative classification of new biological phenomena with an open data format accessible to everyone.

A role for artificial intelligence in molecular imaging of infection and inflammation

The detection of occult infections and low-grade inflammation in clinical practice remains challenging and much depending on readers’ expertise. Although molecular imaging, like [¹⁸F]FDG PET or radiolabeled leukocyte scintigraphy, offers quantitative and reproducible whole body data on inflammatory responses its interpretation is limited to visual analysis. This often leads to delayed diagnosis and treatment, as well as untapped areas of potential application. Artificial intelligence (AI) offers innovative approaches to mine the wealth of imaging data and has led to disruptive breakthroughs in other medical domains already. Here, we discuss how AI-based tools can improve the detection sensitivity of molecular imaging in infection and inflammation but also how AI might push the data analysis beyond current application toward predicting outcome and long-term risk assessment.

Applications of Neural Networks in Biomedical Data Analysis

Neural networks for deep-learning applications, also called artificial neural networks, are important tools in science and industry. While their widespread use was limited because of inadequate hardware in the past, their popularity increased dramatically starting in the early 2000s when it became possible to train increasingly large and complex networks. Today, deep learning is widely used in biomedicine from image analysis to diagnostics. This also includes special topics, such as forensics. In this review, we discuss the latest networks and how they work, with a focus on the analysis of biomedical data, particularly biomarkers in bioimage data. We provide a summary on numerous technical aspects, such as activation functions and frameworks. We also present a data analysis of publications about neural networks to provide a quantitative insight into the use of network types and the number of journals per year to determine the usage in different scientific fields.

Live cell microscopy: From image to insight

Live-cell microscopy is a powerful tool that can reveal cellular behavior as well as the underlying molecular processes. A key advantage of microscopy is that by visualizing biological processes, it can provide direct insights. Nevertheless, live-cell imaging can be technically challenging and prone to artifacts. For a successful experiment, many careful decisions are required at all steps from hardware selection to downstream image analysis. Facing these questions can be particularly intimidating due to the requirement for expertise in multiple disciplines, ranging from optics, biophysics, and programming to cell biology. In this review, we aim to summarize the key points that need to be considered when setting up and analyzing a live-cell imaging experiment. While we put a particular focus on yeast, many of the concepts discussed are applicable also to other organisms. In addition, we discuss reporting and data sharing strategies that we think are critical to improve reproducibility in the field.

GIANI: open-source software for automated analysis of 3D microscopy images

The study of cellular and developmental processes in physiologically relevant three-dimensional (3D) systems facilitates an understanding of mechanisms underlying cell fate, disease and injury. While cutting-edge microscopy technologies permit the routine acquisition of 3D datasets, there is currently a limited number of open-source software packages to analyse such images. Here, we describe General Image Analysis of Nuclei-based Images (GIANI; https://djpbarry.github.io/Giani), new software for the analysis of 3D images. The design primarily facilitates segmentation of nuclei and cells, followed by quantification of morphology and protein expression. GIANI enables routine and reproducible batch-processing of large numbers of images, and comes with scripting and command line tools. We demonstrate the utility of GIANI by quantifying cell morphology and protein expression in confocal images of mouse early embryos and by segmenting nuclei from light-sheet microscopy images of the flour beetle embryo. We also validate the performance of the software using simulated data. More generally, we anticipate that GIANI will be a useful tool for researchers in a variety of biomedical fields.

Summary: General Image Analysis of Nuclei-based Images (GIANI) is a new plugin for the popular FIJI platform, designed for the automated analysis of 3D microscopy images of a wide range of sample types.

Fluorescent Carbon Dot-Supported Imaging-Based Biomedicine: A Comprehensive Review

Carbon dots (CDs) provide distinctive advantages of strong fluorescence, good photostability, high water solubility, and outstanding biocompatibility, and thus are widely exploited as potential imaging agents for in vitro and in vivo bioimaging. Imaging is absolutely necessary when discovering the structure and function of cells, detecting biomarkers in diagnosis, tracking the progress of ongoing disease, treating various tumors, and monitoring therapeutic efficacy, making it an important approach in modern biomedicine. Numerous investigations of CDs have been intensively studied for utilization in bioimaging-supported medical sciences. However, there is still no article highlighting the potential importance of CD-based bioimaging to support various biomedical applications. Herein, we summarize the development of CDs as fluorescence (FL) nanoprobes with different FL colors for potential bioimaging-based applications in living cells, tissue, and organisms, including the bioimaging of various cell types and targets, bioimaging-supported sensing of metal ions and biomolecules, and FL imaging-guided tumor therapy. Current CD-based microscopic techniques and their advantages are also highlighted. This review discusses the significance of advanced CD-supported imaging-based in vitro and in vivo investigations, suggests the potential of CD-based imaging for biomedicine, and encourages the effective selection and development of superior probes and platforms for further biomedical applications.

Petabyte-Scale Multi-Morphometry of Single Neurons for Whole Brains

Recent advances in brain imaging allow producing large amounts of 3-D volumetric data from which morphometry data is reconstructed and measured. Fine detailed structural morphometry of individual neurons, including somata, dendrites, axons, and synaptic connectivity based on digitally reconstructed neurons, is essential for cataloging neuron types and their connectivity. To produce quality morphometry at large scale, it is highly desirable but extremely challenging to efficiently handle petabyte-scale high-resolution whole brain imaging database. Here, we developed a multi-level method to produce high quality somatic, dendritic, axonal, and potential synaptic morphometry, which was made possible by utilizing necessary petabyte hardware and software platform to optimize both the data and workflow management. Our method also boosts data sharing and remote collaborative validation. We highlight a petabyte application dataset involving 62 whole mouse brains, from which we identified 50,233 somata of individual neurons, profiled the dendrites of 11,322 neurons, reconstructed the full 3-D morphology of 1,050 neurons including their dendrites and full axons, and detected 1.9 million putative synaptic sites derived from axonal boutons. Analysis and simulation of these data indicate the promise of this approach for modern large-scale morphology applications.

“Tonga”: A Novel Toolbox for Straightforward Bioimage Analysis

Techniques to acquire and analyze biological images are central to life science. However, the workflow downstream of imaging can be complex and involve several tools, leading to creation of very specialized scripts and pipelines that are difficult to reproduce by other users. Although many commercial and open-source software are available, non-expert users are often challenged by a knowledge gap in setting up analysis pipelines and selecting correct tools for extracting data from images. Moreover, a significant share of everyday image analysis requires simple tools, such as precise segmentation, cell counting, and recording of fluorescent intensities. Hence, there is a need for user-friendly platforms for everyday image analysis that do not require extensive prior knowledge on bioimage analysis or coding. We set out to create a bioimage analysis software that has a straightforward interface and covers common analysis tasks such as object segmentation and analysis, in a practical, reproducible, and modular fashion. We envision our software being useful for analysis of cultured cells, histological sections, and high-content data.

OpenCell: Endogenous tagging for the cartography of human cellular organization

Elucidating the wiring diagram of the human cell is a central goal of the postgenomic era. We combined genome engineering, confocal live-cell imaging, mass spectrometry, and data science to systematically map the localization and interactions of human proteins. Our approach provides a data-driven description of the molecular and spatial networks that organize the proteome. Unsupervised clustering of these networks delineates functional communities that facilitate biological discovery. We found that remarkably precise functional information can be derived from protein localization patterns, which often contain enough information to identify molecular interactions, and that RNA binding proteins form a specific subgroup defined by unique interaction and localization properties. Paired with a fully interactive website (opencell.czbiohub.org), our work constitutes a resource for the quantitative cartography of human cellular organization.

User-Accessible Machine Learning Approaches for Cell Segmentation and Analysis in Tissue

Advanced image analysis with machine and deep learning has improved cell segmentation and classification for novel insights into biological mechanisms. These approaches have been used for the analysis of cells in situ, within tissue, and confirmed existing and uncovered new models of cellular microenvironments in human disease. This has been achieved by the development of both imaging modality specific and multimodal solutions for cellular segmentation, thus addressing the fundamental requirement for high quality and reproducible cell segmentation in images from immunofluorescence, immunohistochemistry and histological stains. The expansive landscape of cell types-from a variety of species, organs and cellular states-has required a concerted effort to build libraries of annotated cells for training data and novel solutions for leveraging annotations across imaging modalities and in some cases led to questioning the requirement for single cell demarcation all together. Unfortunately, bleeding-edge approaches are often confined to a few experts with the necessary domain knowledge. However, freely available, and open-source tools and libraries of trained machine learning models have been made accessible to researchers in the biomedical sciences as software pipelines, plugins for open-source and free desktop and web-based software solutions. The future holds exciting possibilities with expanding machine learning models for segmentation via the brute-force addition of new training data or the implementation of novel network architectures, the use of machine and deep learning in cell and neighborhood classification for uncovering cellular microenvironments, and the development of new strategies for the use of machine and deep learning in biomedical research.

Probabilistic spatial analysis in quantitative microscopy with uncertainty-aware cell detection using deep Bayesian regression

The investigation of biological systems with three-dimensional microscopy demands automatic cell identification methods that not only are accurate but also can imply the uncertainty in their predictions. The use of deep learning to regress density maps is a popular successful approach for extracting cell coordinates from local peaks in a postprocessing step, which then, however, hinders any meaningful probabilistic output. We propose a framework that can operate on large microscopy images and output probabilistic predictions (i) by integrating deep Bayesian learning for the regression of uncertainty-aware density maps, where peak detection algorithms generate cell proposals, and (ii) by learning a mapping from prediction proposals to a probabilistic space that accurately represents the chances of a successful prediction. Using these calibrated predictions, we propose a probabilistic spatial analysis with Monte Carlo sampling. We demonstrate this in a bone marrow dataset, where our proposed methods reveal spatial patterns that are otherwise undetectable.

3D fluorescence microscopy data synthesis for segmentation and benchmarking

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.

Foreground Estimation in Neuronal Images With a Sparse-Smooth Model for Robust Quantification

3D volume imaging has been regarded as a basic tool to explore the organization and function of the neuronal system. Foreground estimation from neuronal image is essential in the quantification and analysis of neuronal image such as soma counting, neurite tracing and neuron reconstruction. However, the complexity of neuronal structure itself and differences in the imaging procedure, including different optical systems and biological labeling methods, result in various and complex neuronal images, which greatly challenge foreground estimation from neuronal image. In this study, we propose a robust sparse-smooth model (RSSM) to separate the foreground and the background of neuronal image. The model combines the different smoothness levels of the foreground and the background, and the sparsity of the foreground. These prior constraints together contribute to the robustness of foreground estimation from a variety of neuronal images. We demonstrate the proposed RSSM method could promote some best available tools to trace neurites or locate somas from neuronal images with their default parameters, and the quantified results are similar or superior to the results that generated from the original images. The proposed method is proved to be robust in the foreground estimation from different neuronal images, and helps to improve the usability of current quantitative tools on various neuronal images with several applications.

Analysis of clonogenic growth in vitro

The clonogenic assay measures the capacity of single cells to form colonies in vitro. It is widely used to identify and quantify self-renewing mammalian cells derived from in vitro cultures as well as from ex vivo tissue preparations of different origins. Varying research questions and the heterogeneous growth requirements of individual cell model systems led to the development of several assay principles and formats that differ with regard to their conceptual setup, 2D or 3D culture conditions, optional cytotoxic treatments and subsequent mathematical analysis. The protocol presented here is based on the initial clonogenic assay protocol as developed by Puck and Marcus more than 60 years ago. It updates and extends the 2006 Nature Protocols article by Franken et al. It discusses different strategies and principles to analyze clonogenic growth in vitro and presents the clonogenic assay in a modular protocol framework enabling a diversity of formats and measures to optimize determination of clonogenic growth parameters. We put particular focus on the phenomenon of cellular cooperation and consideration of how this can affect the mathematical analysis of survival data. This protocol is applicable to any mammalian cell model system from which single-cell suspensions can be prepared and which contains at least a small fraction of cells with self-renewing capacity in vitro. Depending on the cell system used, the entire procedure takes ~2-10 weeks, with a total hands-on time of <20 h per biological replicate.

Bidirectional Mapping-Based Domain Adaptation for Nucleus Detection in Cross-Modality Microscopy Images

Cell or nucleus detection is a fundamental task in microscopy image analysis and has recently achieved state-of-the-art performance by using deep neural networks. However, training supervised deep models such as convolutional neural networks (CNNs) usually requires sufficient annotated image data, which is prohibitively expensive or unavailable in some applications. Additionally, when applying a CNN to new datasets, it is common to annotate individual cells/nuclei in those target datasets for model re-learning, leading to inefficient and low-throughput image analysis. To tackle these problems, we present a bidirectional, adversarial domain adaptation method for nucleus detection on cross-modality microscopy image data. Specifically, the method learns a deep regression model for individual nucleus detection with both source-to-target and target-to-source image translation. In addition, we explicitly extend this unsupervised domain adaptation method to a semi-supervised learning situation and further boost the nucleus detection performance. We evaluate the proposed method on three cross-modality microscopy image datasets, which cover a wide variety of microscopy imaging protocols or modalities, and obtain a significant improvement in nucleus detection compared to reference baseline approaches. In addition, our semi-supervised method is very competitive with recent fully supervised learning models trained with all real target training labels.

Visualising an invisible symbiosis

Despite the vast abundance and global importance of plant and microbial species, the large majority go unnoticed and unappreciated by humans, contributing to pressing issues including the neglect of study and research of these organisms, the lack of interest and support for their protection and conservation, low microbial and botanical literacy in society, and a growing disconnect between people and nature. The invisibility of many of these organisms is a key factor in their oversight by society, but also points to a solution: sharing the wealth of visual data produced during scientific research with a broader audience. Here, we discuss how the invisible can be visualised for a public audience, and the benefits it can bring.

SUMMARY

Whether too small, slow or concealed, the majority of species on Earth go unseen by humans. One such rather unobservable group of organisms are the arbuscular mycorrhizal (AM) fungi, who form beneficial symbioses with plants. AM symbiosis is ubiquitous and vitally important globally in ecosystem functioning, but partly as a consequence of its invisibility, it receives disproportionally little attention and appreciation. Yet AM fungi, and other unseen organisms, need not remain overlooked: from decades of scientific research there exists a goldmine of visual data, which if shared effectively we believe can alleviate the issues of low awareness. Here, we use examples from our experience of public engagement with AM symbiosis as well as evidence from the literature to outline the diverse ways in which invisible organisms can be visualised for a broad audience. We highlight outcomes and knock-on consequences of this visualisation, ranging from improved human mental health to environmental protection, making the case for researchers to share their images more widely for the benefit of plants (and fungi and other overlooked organisms), people and planet.

HRMAn 2.0: Next-generation artificial intelligence-driven analysis for broad host-pathogen interactions

To study the dynamics of infection processes, it is common to manually enumerate imaging-based infection assays. However, manual counting of events from imaging data is biased, error-prone and a laborious task. We recently presented HRMAn (Host Response to Microbe Analysis), an automated image analysis program using state-of-the-art machine learning and artificial intelligence algorithms to analyse pathogen growth and host defence behaviour. With HRMAn, we can quantify intracellular infection by pathogens such as Toxoplasma gondii and Salmonella in a variety of cell types in an unbiased and highly reproducible manner, measuring multiple parameters including pathogen growth, pathogen killing and activation of host cell defences. Since HRMAn is based on the KNIME Analytics platform, it can easily be adapted to work with other pathogens and produce more readouts from quantitative imaging data. Here we showcase improvements to HRMAn resulting in the release of HRMAn 2.0 and new applications of HRMAn 2.0 for the analysis of host-pathogen interactions using the established pathogen T. gondii and further extend it for use with the bacterial pathogen Chlamydia trachomatis and the fungal pathogen Cryptococcus neoformans.

Cell Tracking for Organoids: Lessons From Developmental Biology

Organoids have emerged as powerful model systems to study organ development and regeneration at the cellular level. Recently developed microscopy techniques that track individual cells through space and time hold great promise to elucidate the organizational principles of organs and organoids. Applied extensively in the past decade to embryo development and 2D cell cultures, cell tracking can reveal the cellular lineage trees, proliferation rates, and their spatial distributions, while fluorescent markers indicate differentiation events and other cellular processes. Here, we review a number of recent studies that exemplify the power of this approach, and illustrate its potential to organoid research. We will discuss promising future routes, and the key technical challenges that need to be overcome to apply cell tracking techniques to organoid biology.

Image-based cell phenotyping with deep learning

A cell's phenotype is the culmination of several cellular processes through a complex network of molecular interactions that ultimately result in a unique morphological signature. Visual cell phenotyping is the characterization and quantification of these observable cellular traits in images. Recently, cellular phenotyping has undergone a massive overhaul in terms of scale, resolution, and throughput, which is attributable to advances across electronic, optical, and chemical technologies for imaging cells. Coupled with the rapid acceleration of deep learning-based computational tools, these advances have opened up new avenues for innovation across a wide variety of high-throughput cell biology applications. Here, we review applications wherein deep learning is powering the recognition, profiling, and prediction of visual phenotypes to answer important biological questions. As the complexity and scale of imaging assays increase, deep learning offers computational solutions to elucidate the details of previously unexplored cellular phenotypes.

Highlights from the 2016-2020 NEUBIAS training schools for Bioimage Analysts: a success story and key asset for analysts and life scientists

NEUBIAS, the European Network of Bioimage Analysts, was created in 2016 with the goal of improving the communication and the knowledge transfer among the various stakeholders involved in the acquisition, processing and analysis of biological image data, and to promote the establishment and recognition of the profession of Bioimage Analyst. One of the most successful initiatives of the NEUBIAS programme was its series of 15 training schools, which trained over 400 new Bioimage Analysts, coming from over 40 countries. Here we outline the rationale behind the innovative three-level program of the schools, the curriculum, the trainer recruitment and turnover strategy, the outcomes for the community and the career path of analysts, including some success stories. We discuss the future of the materials created during this programme and some of the new initiatives emanating from the community of NEUBIAS-trained analysts, such as the NEUBIAS Academy. Overall, we elaborate on how this training programme played a key role in collectively leveraging Bioimaging and Life Science research by bringing the latest innovations into structured, frequent and intensive training activities, and on why we believe this should become a model to further develop in Life Sciences.

Data science in cell imaging

Cell imaging has entered the 'Big Data' era. New technologies in light microscopy and molecular biology have led to an explosion in high-content, dynamic and multidimensional imaging data. Similar to the 'omics' fields two decades ago, our current ability to process, visualize, integrate and mine this new generation of cell imaging data is becoming a critical bottleneck in advancing cell biology. Computation, traditionally used to quantitatively test specific hypotheses, must now also enable iterative hypothesis generation and testing by deciphering hidden biologically meaningful patterns in complex, dynamic or high-dimensional cell image data. Data science is uniquely positioned to aid in this process. In this Perspective, we survey the rapidly expanding new field of data science in cell imaging. Specifically, we highlight how data science tools are used within current image analysis pipelines, propose a computation-first approach to derive new hypotheses from cell image data, identify challenges and describe the next frontiers where we believe data science will make an impact. We also outline steps to ensure broad access to these powerful tools - democratizing infrastructure availability, developing sensitive, robust and usable tools, and promoting interdisciplinary training to both familiarize biologists with data science and expose data scientists to cell imaging.

High-definition imaging using line-illumination modulation microscopy

The microscopic visualization of large-scale three-dimensional (3D) samples by optical microscopy requires overcoming challenges in imaging quality and speed and in big data acquisition and management. We report a line-illumination modulation (LiMo) technique for imaging thick tissues with high throughput and low background. Combining LiMo with thin tissue sectioning, we further develop a high-definition fluorescent micro-optical sectioning tomography (HD-fMOST) method that features an average signal-to-noise ratio of 110, leading to substantial improvement in neuronal morphology reconstruction. We achieve a >30-fold lossless data compression at a voxel resolution of 0.32 × 0.32 × 1.00 μm³, enabling online data storage to a USB drive or in the cloud, and high-precision (95% accuracy) brain-wide 3D cell counting in real time. These results highlight the potential of HD-fMOST to facilitate large-scale acquisition and analysis of whole-brain high-resolution datasets.

Constructing and optimizing 3D atlases from 2D data with application to the developing mouse brain

3D imaging data necessitate 3D reference atlases for accurate quantitative interpretation. Existing computational methods to generate 3D atlases from 2D-derived atlases result in extensive artifacts, while manual curation approaches are labor-intensive. We present a computational approach for 3D atlas construction that substantially reduces artifacts by identifying anatomical boundaries in the underlying imaging data and using these to guide 3D transformation. Anatomical boundaries also allow extension of atlases to complete edge regions. Applying these methods to the eight developmental stages in the Allen Developing Mouse Brain Atlas (ADMBA) led to more comprehensive and accurate atlases. We generated imaging data from 15 whole mouse brains to validate atlas performance and observed qualitative and quantitative improvement (37% greater alignment between atlas and anatomical boundaries). We provide the pipeline as the MagellanMapper software and the eight 3D reconstructed ADMBA atlases. These resources facilitate whole-organ quantitative analysis between samples and across development.

The research community needs precise, reliable 3D atlases of organs to pinpoint where biological structures and processes are located. For instance, these maps are essential to understand where specific genes are turned on or off, or the spatial organization of various groups of cells over time.

For centuries, atlases have been built by thinly ‘slicing up’ an organ, and then precisely representing each 2D layer. Yet this approach is imperfect: each layer may be accurate on its own, but inevitable mismatches appear between the slices when viewed in 3D or from another angle.

Advances in microscopy now allow entire organs to be imaged in 3D. Comparing these images with atlases could help to detect subtle differences that indicate or underlie disease. However, this is only possible if 3D maps are accurate and do not feature mismatches between layers. To create an atlas without such artifacts, one approach consists in starting from scratch and manually redrawing the maps in 3D, a labor-intensive method that discards a large body of well-established atlases.

Instead, Young et al. set out to create an automated method which could help to refine existing ‘layer-based’ atlases, releasing software that anyone can use to improve current maps. The package was created by harnessing eight atlases in the Allen Developing Mouse Brain Atlas, and then using the underlying anatomical images to resolve discrepancies between layers or fill out any missing areas. Known as MagellanMapper, the software was extensively tested to demonstrate the accuracy of the maps it creates, including comparison to whole-brain imaging data from 15 mouse brains. Armed with this new software, researchers can improve the accuracy of their atlases, helping them to understand the structure of organs at the level of the cell and giving them insight into a broad range of human disorders.

A robust unsupervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei

Cell morphology encodes essential information on many underlying biological processes. It is commonly used by clinicians and researchers in the study, diagnosis, prognosis, and treatment of human diseases. Quantification of cell morphology has seen tremendous advances in recent years. However, effectively defining morphological shapes and evaluating the extent of morphological heterogeneity within cell populations remain challenging. Here we present a protocol and software for the analysis of cell and nuclear morphology from fluorescence or bright-field images using the VAMPIRE algorithm ( https://github.com/kukionfr/VAMPIRE_open ). This algorithm enables the profiling and classification of cells into shape modes based on equidistant points along cell and nuclear contours. Examining the distributions of cell morphologies across automatically identified shape modes provides an effective visualization scheme that relates cell shapes to cellular subtypes based on endogenous and exogenous cellular conditions. In addition, these shape mode distributions offer a direct and quantitative way to measure the extent of morphological heterogeneity within cell populations. This protocol is highly automated and fast, with the ability to quantify the morphologies from 2D projections of cells seeded both on 2D substrates or embedded within 3D microenvironments, such as hydrogels and tissues. The complete analysis pipeline can be completed within 60 minutes for a dataset of ~20,000 cells/2,400 images.

webTDat: A Web-Based, Real-Time, 3D Visualization Framework for Mesoscopic Whole-Brain Images

The popularity of mesoscopic whole-brain imaging techniques has increased dramatically, but these techniques generate teravoxel-sized volumetric image data. Visualizing or interacting with these massive data is both necessary and essential in the bioimage analysis pipeline; however, due to their size, researchers have difficulty using typical computers to process them. The existing solutions do not consider applying web visualization and three-dimensional (3D) volume rendering methods simultaneously to reduce the number of data copy operations and provide a better way to visualize 3D structures in bioimage data. Here, we propose webTDat, an open-source, web-based, real-time 3D visualization framework for mesoscopic-scale whole-brain imaging datasets. webTDat uses an advanced rendering visualization method designed with an innovative data storage format and parallel rendering algorithms. webTDat loads the primary information in the image first and then decides whether it needs to load the secondary information in the image. By performing validation on TB-scale whole-brain datasets, webTDat achieves real-time performance during web visualization. The webTDat framework also provides a rich interface for annotation, making it a useful tool for visualizing mesoscopic whole-brain imaging data.