Visualization and analysis of 3D microscopic images

Tissue clearing applications in memory engram research

A memory engram is thought to be the physical substrate of the memory trace within the brain, which is generally depicted as a neuronal ensemble activated by learning to fire together during encoding and retrieval. It has been postulated that engram cell ensembles are functionally interconnected across multiple brain regions to store a single memory as an "engram complex", but visualizing this engram complex across the whole brain has for long been hindered by technical limitations. With the recent development of tissue clearing techniques, advanced light-sheet microscopy, and automated 3D image analysis, it has now become possible to generate a brain-wide map of engram cells and thereby to visualize the "engram complex". In this review, we first provide a comprehensive summary of brain-wide engram mapping studies to date. We then compile a guide on implementing the optimal tissue clearing technique for engram tagging approaches, paying particular attention to visualize engram reactivation as a critical mnemonic property, for which whole-brain multiplexed immunostaining becomes a challenging prerequisite. Finally, we highlight the potential of tissue clearing to simultaneously shed light on both the circuit connectivity and molecular underpinnings of engram cells in a single snapshot. In doing so, novel brain regions and circuits can be identified for subsequent functional manipulation, thus providing an opportunity to robustly examine the "engram complex" underlying memory storage.

Immersive and interactive visualization of 3D spatio-temporal data using a space time hypercube: Application to cell division and morphogenesis analysis

The analysis of multidimensional time-varying datasets faces challenges, notably regarding the representation of the data and the visualization of temporal variations. We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data. Additionally, we propose a set of tools allowing the user to filter and manipulate the 3D STC which benefits the visualization, exploration and interaction possibilities offered by VR. Finally, we evaluated the proposed visualization method in the context of 3D temporal cell imaging data analysis, through a user study (n = 5) reporting the feedback from five biologists. These domain experts also accompanied the application design as consultants, providing insights on how the STC visualization could be used for the exploration of complex 3D temporal morphogenesis data.

Vaa3D-x for cross-platform teravoxel-scale immersive exploration of multidimensional image data

SUMMARY

Vaa3D is a software package that has been widely used to visualize and analyze multidimensional microscopic images in a number of cutting edge bioimage informatics applications. However, due to many recent updates of both software development environments and operating systems, it was highly requested to maintain Vaa3D and disseminate it on latest operating systems. In addition, there has never been a showcase about how to use Vaa3D's cross-platform visualization and immersive exploration functions for multidimensional and teravoxel-scale images. Here, we introduce a newly developed version of the software, called Vaa3D-x, to address all the above issues.

AVAILABILITY AND IMPLEMENTATION

Vaa3D-x is released in both binary and Open-Source available at vaa3d.org and GitHub (https://github.com/Vaa3D).

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

IMAGE-IN: Interactive web-based multidimensional 3D visualizer for multi-modal microscopy images

Advances in microscopy hardware and storage capabilities lead to increasingly larger multidimensional datasets. The multiple dimensions are commonly associated with space, time, and color channels. Since "seeing is believing", it is important to have easy access to user-friendly visualization software. Here we present IMAGE-IN, an interactive web-based multidimensional (N-D) viewer designed specifically for confocal laser scanning microscopy (CLSM) and focused ion beam scanning electron microscopy (FIB-SEM) data, with the goal of assisting biologists in their visualization and analysis tasks and promoting digital workflows. This new visualization platform includes intuitive multidimensional opacity fine-tuning, shading on/off, multiple blending modes for volume viewers, and the ability to handle multichannel volumetric data in volume and surface views. The software accepts a sequence of image files or stacked 3D images as input and offers a variety of viewing options ranging from 3D volume/surface rendering to multiplanar reconstruction approaches. We evaluate the performance by comparing the loading and rendering timings of a heterogeneous dataset of multichannel CLSM and FIB-SEM images on two devices with installed graphic cards, as well as comparing rendered image quality between ClearVolume (the ImageJ open-source desktop viewer), Napari (the Python desktop viewer), Imaris (the closed-source desktop viewer), and our proposed IMAGE-IN web viewer.

When light meets biology - how the specimen affects quantitative microscopy

Fluorescence microscopy images should not be treated as perfect representations of biology. Many factors within the biospecimen itself can drastically affect quantitative microscopy data. Whereas some sample-specific considerations, such as photobleaching and autofluorescence, are more commonly discussed, a holistic discussion of sample-related issues (which includes less-routine topics such as quenching, scattering and biological anisotropy) is required to appropriately guide life scientists through the subtleties inherent to bioimaging. Here, we consider how the interplay between light and a sample can cause common experimental pitfalls and unanticipated errors when drawing biological conclusions. Although some of these discrepancies can be minimized or controlled for, others require more pragmatic considerations when interpreting image data. Ultimately, the power lies in the hands of the experimenter. The goal of this Review is therefore to survey how biological samples can skew quantification and interpretation of microscopy data. Furthermore, we offer a perspective on how to manage many of these potential pitfalls.

In vivo imaging of mammary epithelial cell dynamics in response to lineage-biased Wnt/β-catenin activation

Real-time in vivo imaging provides an essential window into the spatiotemporal cellular events contributing to tissue development and pathology. By coupling longitudinal intravital imaging with genetic lineage tracing, here we capture the earliest cellular events arising in response to active Wnt/β-catenin signaling and the ensuing impact on the organization and differentiation of the mammary epithelium. This enables us to interrogate how Wnt/β-catenin regulates the dynamics of distinct subpopulations of mammary epithelial cells in vivo and in real time. We show that β-catenin stabilization, when targeted to either the mammary luminal or basal epithelial lineage, leads to cellular rearrangements that precipitate the formation of hyperplastic lesions that undergo squamous transdifferentiation. These results enhance our understanding of the earliest stages of hyperplastic lesion formation in vivo and reveal that, in mammary neoplastic development, β-catenin activation dictates a hair follicle/epidermal differentiation program independently of the targeted cell of origin.

Regulation of FGF-2, FGF-18 and Transcription Factor Activity by Perlecan in the Maturational Development of Transitional Rudiment and Growth Plate Cartilages and in the Maintenance of Permanent Cartilage Homeostasis

The aim of this study was to highlight the roles of perlecan in the regulation of the development of the rudiment developmental cartilages and growth plate cartilages, and also to show how perlecan maintains permanent articular cartilage homeostasis. Cartilage rudiments are transient developmental templates containing chondroprogenitor cells that undergo proliferation, matrix deposition, and hypertrophic differentiation. Growth plate cartilage also undergoes similar changes leading to endochondral bone formation, whereas permanent cartilage is maintained as an articular structure and does not undergo maturational changes. Pericellular and extracellular perlecan-HS chains interact with growth factors, morphogens, structural matrix glycoproteins, proteases, and inhibitors to promote matrix stabilization and cellular proliferation, ECM remodelling, and tissue expansion. Perlecan has mechanotransductive roles in cartilage that modulate chondrocyte responses in weight-bearing environments. Nuclear perlecan may modulate chromatin structure and transcription factor access to DNA and gene regulation. Snail-1, a mesenchymal marker and transcription factor, signals through FGFR-3 to promote chondrogenesis and maintain Acan and type II collagen levels in articular cartilage, but prevents further tissue expansion. Pre-hypertrophic growth plate chondrocytes also express high Snail-1 levels, leading to cessation of Acan and CoI2A1 synthesis and appearance of type X collagen. Perlecan differentially regulates FGF-2 and FGF-18 to maintain articular cartilage homeostasis, rudiment and growth plate cartilage growth, and maturational changes including mineralization, contributing to skeletal growth.

Computational Methods for Single-Cell Imaging and Omics Data Integration

Integrating single cell omics and single cell imaging allows for a more effective characterisation of the underlying mechanisms that drive a phenotype at the tissue level, creating a comprehensive profile at the cellular level. Although the use of imaging data is well established in biomedical research, its primary application has been to observe phenotypes at the tissue or organ level, often using medical imaging techniques such as MRI, CT, and PET. These imaging technologies complement omics-based data in biomedical research because they are helpful for identifying associations between genotype and phenotype, along with functional changes occurring at the tissue level. Single cell imaging can act as an intermediary between these levels. Meanwhile new technologies continue to arrive that can be used to interrogate the genome of single cells and its related omics datasets. As these two areas, single cell imaging and single cell omics, each advance independently with the development of novel techniques, the opportunity to integrate these data types becomes more and more attractive. This review outlines some of the technologies and methods currently available for generating, processing, and analysing single-cell omics- and imaging data, and how they could be integrated to further our understanding of complex biological phenomena like ageing. We include an emphasis on machine learning algorithms because of their ability to identify complex patterns in large multidimensional data.

Visualizing and Quantifying Data from Time-Lapse Imaging Experiments

One obvious feature of life is that it is highly dynamic. The dynamics can be captured by movies that are made by acquiring images at regular time intervals, a method that is also known as time-lapse imaging. Looking at movies is a great way to learn more about the dynamics in cells, tissue, and organisms. However, science is different from Netflix, in that it aims for a quantitative understanding of the dynamics. The quantification is important for the comparison of dynamics and to study effects of perturbations. Here, we provide detailed processing and analysis methods that we commonly use to analyze and visualize our time-lapse imaging data. All methods use freely available open-source software and use example data that is available from an online data repository. The step-by-step guides together with example data allow for fully reproducible workflows that can be modified and adjusted to visualize and quantify other data from time-lapse imaging experiments.

What Are the Potential Roles of Nuclear Perlecan and Other Heparan Sulphate Proteoglycans in the Normal and Malignant Phenotype

The recent discovery of nuclear and perinuclear perlecan in annulus fibrosus and nucleus pulposus cells and its known matrix stabilizing properties in tissues introduces the possibility that perlecan may also have intracellular stabilizing or regulatory roles through interactions with nuclear envelope or cytoskeletal proteins or roles in nucleosomal-chromatin organization that may regulate transcriptional factors and modulate gene expression. The nucleus is a mechano-sensor organelle, and sophisticated dynamic mechanoresponsive cytoskeletal and nuclear envelope components support and protect the nucleus, allowing it to perceive and respond to mechano-stimulation. This review speculates on the potential roles of perlecan in the nucleus based on what is already known about nuclear heparan sulphate proteoglycans. Perlecan is frequently found in the nuclei of tumour cells; however, its specific role in these diseased tissues is largely unknown. The aim of this review is to highlight probable roles for this intriguing interactive regulatory proteoglycan in the nucleus of normal and malignant cell types.

Factors to consider when interrogating 3D culture models with plate readers or automated microscopes

Along with the increased use of more physiologically relevant three-dimensional cell culture models comes the responsibility of researchers to validate new assay methods that measure events in structures that are physically larger and more complex compared to monolayers of cells. It should not be assumed that assays designed using monolayers of cells will work for cells cultured as larger three-dimensional masses. The size and barriers for penetration of molecules through the layers of cells result in a different microenvironment for the cells in the outer layer compared to the center of three-dimensional structures. Diffusion rates for nutrients and oxygen may limit metabolic activity which is often measured as a marker for cell viability. For assays that lyse cells, the penetration of reagents to achieve uniform cell lysis must be considered. For live cell fluorescent imaging assays, the diffusion of fluorescent probes and penetration of photons of light for probe excitation and fluorescent emission must be considered. This review will provide an overview of factors to consider when implementing assays to interrogate three dimensional cell culture models.

webTDat: A Web-Based, Real-Time, 3D Visualization Framework for Mesoscopic Whole-Brain Images

The popularity of mesoscopic whole-brain imaging techniques has increased dramatically, but these techniques generate teravoxel-sized volumetric image data. Visualizing or interacting with these massive data is both necessary and essential in the bioimage analysis pipeline; however, due to their size, researchers have difficulty using typical computers to process them. The existing solutions do not consider applying web visualization and three-dimensional (3D) volume rendering methods simultaneously to reduce the number of data copy operations and provide a better way to visualize 3D structures in bioimage data. Here, we propose webTDat, an open-source, web-based, real-time 3D visualization framework for mesoscopic-scale whole-brain imaging datasets. webTDat uses an advanced rendering visualization method designed with an innovative data storage format and parallel rendering algorithms. webTDat loads the primary information in the image first and then decides whether it needs to load the secondary information in the image. By performing validation on TB-scale whole-brain datasets, webTDat achieves real-time performance during web visualization. The webTDat framework also provides a rich interface for annotation, making it a useful tool for visualizing mesoscopic whole-brain imaging data.

Focus on biosensors: Looking through the lens of quantitative biology

In recent years, plant biologists interested in quantifying molecules and molecular events in vivo have started to complement reporter systems with genetically encoded fluorescent biosensors (GEFBs) that directly sense an analyte. Such biosensors can allow measurements at the level of individual cells and over time. This information is proving valuable to mathematical modellers interested in representing biological phenomena in silico, because improved measurements can guide improved model construction and model parametrisation. Advances in synthetic biology have accelerated the pace of biosensor development, and the simultaneous expression of spectrally compatible biosensors now allows quantification of multiple nodes in signalling networks. For biosensors that directly respond to stimuli, targeting to specific cellular compartments allows the observation of differential accumulation of analytes in distinct organelles, bringing insights to reactive oxygen species/calcium signalling and photosynthesis research. In conjunction with improved image analysis methods, advances in biosensor imaging can help close the loop between experimentation and mathematical modelling.

Image analysis of PVC / TiO2 nanocomposites SEM micrographs

The present paper deals with image analysis of Scanning Electron Microscope (SEM) micrographs of PVC / TiO₂ nanocomposites prepared through solution casting technique at different wt% of TiO₂ nanofillers dispersed within the PVC matrix. The qualitative and quantitative dispersion of TiO₂ nanofillers is estimated through the proposed algorithm based on global threshold obtained from Otsu's method. The PVC-TiO₂- PVC molecular region map and TiO₂ nanofillers dispersion maps are obtained by background elimination technique with predefined threshold value and a global threshold value, respectively. The particle size distribution histograms are obtained in terms of TiO₂ nanofillers population versus available PVC/TiO₂ matrix areal spread. This study is carried out to correlate qualitative and quantitative dispersion of TiO₂ nanofillers within the PVC matrix with the variation of specimen properties accordingly.

Whole-Brain Image Analysis and Anatomical Atlas 3D Generation Using MagellanMapper

MagellanMapper is a software suite designed for visual inspection and end-to-end automated processing of large-volume, 3D brain imaging datasets in a memory-efficient manner. The rapidly growing number of large-volume, high-resolution datasets necessitates visualization of raw data at both macro- and microscopic levels to assess the quality of data, as well as automated processing to quantify data in an unbiased manner for comparison across a large number of samples. To facilitate these analyses, MagellanMapper provides both a graphical user interface for manual inspection and a command-line interface for automated image processing. At the macroscopic level, the graphical interface allows researchers to view full volumetric images simultaneously in each dimension and to annotate anatomical label placements. At the microscopic level, researchers can inspect regions of interest at high resolution to build ground truth data of cellular locations such as nuclei positions. Using the command-line interface, researchers can automate cell detection across volumetric images, refine anatomical atlas labels to fit underlying histology, register these atlases to sample images, and perform statistical analyses by anatomical region. MagellanMapper leverages established open-source computer vision libraries and is itself open source and freely available for download and extension. © 2020 Wiley Periodicals LLC. Basic Protocol 1: MagellanMapper installation Alternate Protocol: Alternative methods for MagellanMapper installation Basic Protocol 2: Import image files into MagellanMapper Basic Protocol 3: Region of interest visualization and annotation Basic Protocol 4: Explore an atlas along all three dimensions and register to a sample brain Basic Protocol 5: Automated 3D anatomical atlas construction Basic Protocol 6: Whole-tissue cell detection and quantification by anatomical label Support Protocol: Import a tiled microscopy image in proprietary format into MagellanMapper.

A bird's-eye view of deep learning in bioimage analysis

Deep learning of artificial neural networks has become the de facto standard approach to solving data analysis problems in virtually all fields of science and engineering. Also in biology and medicine, deep learning technologies are fundamentally transforming how we acquire, process, analyze, and interpret data, with potentially far-reaching consequences for healthcare. In this mini-review, we take a bird's-eye view at the past, present, and future developments of deep learning, starting from science at large, to biomedical imaging, and bioimage analysis in particular.

Reprint of "Multi-modal image cytometry approach - From dynamic to whole organ imaging"

Optical imaging is a valuable tool to visualise biological processes in the context of the tissue. Each imaging modality provides the biologist with different types of information - cell dynamics and migration over time can be tracked with time-lapse imaging (e.g. intra-vital imaging); an overview of whole tissues can be acquired using optical clearing in conjunction with light sheet microscopy; finer details such as cellular morphology and fine nerve tortuosity can be imaged at higher resolution using the confocal microscope. Multi-modal imaging combined with image cytometry - a form of quantitative analysis of image datasets - provides an objective basis for comparing between sample groups. Here, we provide an overview of technical aspects to look out for in an image cytometry workflow, and discuss issues related to sample preparation, image post-processing and analysis for intra-vital and whole organ imaging.

Enhanced translation expands the endo-lysosome size and promotes antigen presentation during phagocyte activation

The mechanisms that govern organelle adaptation and remodelling remain poorly defined. The endo-lysosomal system degrades cargo from various routes, including endocytosis, phagocytosis, and autophagy. For phagocytes, endosomes and lysosomes (endo-lysosomes) are kingpin organelles because they are essential to kill pathogens and process and present antigens. During phagocyte activation, endo-lysosomes undergo a morphological transformation, going from a collection of dozens of globular structures to a tubular network in a process that requires the phosphatidylinositol-3-kinase-AKT-mechanistic target of rapamycin (mTOR) signalling pathway. Here, we show that the endo-lysosomal system undergoes an expansion in volume and holding capacity during phagocyte activation within 2 h of lipopolysaccharides (LPS) stimulation. Endo-lysosomal expansion was paralleled by an increase in lysosomal protein levels, but this was unexpectedly largely independent of the transcription factor EB (TFEB) and transcription factor E3 (TFE3), which are known to scale up lysosome biogenesis. Instead, we demonstrate a hitherto unappreciated mechanism of acute organelle expansion via mTOR Complex 1 (mTORC1)-dependent increase in translation, which appears to be mediated by both S6Ks and 4E-BPs. Moreover, we show that stimulation of RAW 264.7 macrophage cell line with LPS alters translation of a subset but not all of mRNAs encoding endo-lysosomal proteins, thereby suggesting that endo-lysosome expansion is accompanied by functional remodelling. Importantly, mTORC1-dependent increase in translation activity was necessary for efficient and rapid antigen presentation by dendritic cells. Collectively, we identified a previously unknown and functionally relevant mechanism for endo-lysosome expansion that relies on mTORC1-dependent translation to stimulate endo-lysosome biogenesis in response to an infection signal.

Activation of phagocytes rapidly expands the endo-lysosomal system and promotes antigen presentation. Endo-lysosome expansion was driven by mTORC1-dependent enhanced translation, revealing regulated translation as a mechanism to remodel membrane organelles in response to external signals and stresses.

Molecular and living cell dynamic assays with optical microscopy imaging techniques

Generally, the message elucidated by the conventional analytical methods overlooks the heterogeneity of single objects, where the behavior of individual molecules is shielded. With the advent of optical microscopy imaging techniques, it is possible to identify, visualize and track individual molecules or nanoparticles under a biological environment with high temporal and spatial resolution. In this work, we summarize the commonly adopted optical microscopy techniques for bio-analytical assays in living cells, including total internal reflection fluorescence microscopy (TIRFM), super-resolution optical microscopy (SRM), and dark-field optical microscopy (DFM). The basic principles of these methods and some recent interesting applications in molecular detection and single-particle tracking are introduced. Moreover, the development in high-dimensional optical microscopy to achieve three-dimensional (3-D) as well as sub-diffraction localization and tracking of biomolecules is also highlighted.

Layer-based visualization and biomedical information exploration of multi-channel large histological data

BACKGROUND AND OBJECTIVE

Modern microscopes can acquire multi-channel large histological data from tissues of human beings or animals, which contain rich biomedical information for disease diagnosis and biological feature analysis. However, due to the large size, fuzzy tissue structure, and complicated multiple elements integrated in the image color space, it is still a challenge for current software systems to effectively calculate histological data, show the inner tissue structures and unveil hidden biomedical information. Therefore, we developed new algorithms and a software platform to address this issue.

METHODS

This paper presents a multi-channel biomedical data computing and visualization system that can efficiently process large 3D histological images acquired from high-resolution microscopes. A novelty of our system is that it can dynamically display a volume of interest and extract tissue information using a layer-based data navigation scheme. During the data exploring process, the actual resolution of the loaded data can be dynamically determined and updated, and data rendering is synchronized in four display windows at each data layer, where 2D textures are extracted from the imaging volume and mapped onto the displayed clipping planes in 3D space.

RESULTS

To test the efficiency and scalability of this system, we performed extensive evaluations using several different hardware systems and large histological color datasets acquired from a CryoViz 3D digital system. The experimental results demonstrated that our system can deliver interactive data navigation speed and display detailed imaging information in real time, which is beyond the capability of commonly available biomedical data exploration software platforms.

CONCLUSION

Taking advantage of both CPU (central processing unit) main memory and GPU (graphics processing unit) graphics memory, the presented software platform can efficiently compute, process and visualize very large biomedical data and enhance data information. The performance of this system can satisfactorily address the challenges of navigating and interrogating volumetric multi-spectral large histological image at multiple resolution levels.

CellProfiler 3.0: Next-generation image processing for biology

CellProfiler has enabled the scientific research community to create flexible, modular image analysis pipelines since its release in 2005. Here, we describe CellProfiler 3.0, a new version of the software supporting both whole-volume and plane-wise analysis of three-dimensional (3D) image stacks, increasingly common in biomedical research. CellProfiler's infrastructure is greatly improved, and we provide a protocol for cloud-based, large-scale image processing. New plugins enable running pretrained deep learning models on images. Designed by and for biologists, CellProfiler equips researchers with powerful computational tools via a well-documented user interface, empowering biologists in all fields to create quantitative, reproducible image analysis workflows.

Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking

The digital reconstruction of single neurons from 3D confocal microscopic images is an important tool for understanding the neuron morphology and function. However the accurate automatic neuron reconstruction remains a challenging task due to the varying image quality and the complexity in the neuronal arborisation. Targeting the common challenges of neuron tracing, we propose a novel automatic 3D neuron reconstruction algorithm, named Rivulet, which is based on the multi-stencils fast-marching and iterative back-tracking. The proposed Rivulet algorithm is capable of tracing discontinuous areas without being interrupted by densely distributed noises. By evaluating the proposed pipeline with the data provided by the Diadem challenge and the recent BigNeuron project, Rivulet is shown to be robust to challenging microscopic imagestacks. We discussed the algorithm design in technical details regarding the relationships between the proposed algorithm and the other state-of-the-art neuron tracing algorithms.

Two-photon microscopy imaging of oxidative stress in human living erythrocytes

Red blood cells (RBCs) are known to be the most suitable cells to study oxidative stress, which is implicated in the etiopathology of many human diseases. The goal of the current study was to develop a new effective approach for assessing oxidative stress in human living RBCs using two-photon microscopy. To mimic oxidative stress in human living RBCs, an in vitro model was generated followed by two-photon microscopy imaging. The results revealed that oxidative stress is clearly visible on the two-photon microscopy images of RBCs under oxidative stress compared to no fluorescence in controls (P<0.0001). This novel approach for oxidative stress investigation in human living RBCs could efficiently be applied in clinical research and antioxidant compounds testing.

Open data and digital morphology

Over the past two decades, the development of methods for visualizing and analysing specimens digitally, in three and even four dimensions, has transformed the study of living and fossil organisms. However, the initial promise that the widespread application of such methods would facilitate access to the underlying digital data has not been fully achieved. The underlying datasets for many published studies are not readily or freely available, introducing a barrier to verification and reproducibility, and the reuse of data. There is no current agreement or policy on the amount and type of data that should be made available alongside studies that use, and in some cases are wholly reliant on, digital morphology. Here, we propose a set of recommendations for minimum standards and additional best practice for three-dimensional digital data publication, and review the issues around data storage, management and accessibility.

Quantitative analyses of the 3D nuclear landscape recorded with super-resolved fluorescence microscopy

Recent advancements of super-resolved fluorescence microscopy have revolutionized microscopic studies of cells, including the exceedingly complex structural organization of cell nuclei in space and time. In this paper we describe and discuss tools for (semi-) automated, quantitative 3D analyses of the spatial nuclear organization. These tools allow the quantitative assessment of highly resolved different chromatin compaction levels in individual cell nuclei, which reflect functionally different regions or sub-compartments of the 3D nuclear landscape, and measurements of absolute distances between sites of different chromatin compaction. In addition, these tools allow 3D mapping of specific DNA/RNA sequences and nuclear proteins relative to the 3D chromatin compaction maps and comparisons of multiple cell nuclei. The tools are available in the free and open source R packages nucim and bioimagetools. We discuss the use of masks for the segmentation of nuclei and the use of DNA stains, such as DAPI, as a proxy for local differences in chromatin compaction. We further discuss the limitations of 3D maps of the nuclear landscape as well as problems of the biological interpretation of such data.

Acquisition, Visualization and Potential Applications of 3D Data in Anatomic Pathology

Although human anatomy and histology are naturally three-dimensional (3D), commonly used diagnostic and educational tools are technologically restricted to providing two-dimensional representations (e.g. gross photography and glass slides). This limitation may be overcome by employing techniques to acquire and display 3D data, which refers to the digital information used to describe a 3D object mathematically. There are several established and experimental strategies to capture macroscopic and microscopic 3D data. In addition, recent hardware and software innovations have propelled the visualization of 3D models, including virtual and augmented reality. Accompanying these advances are novel clinical and non-clinical applications of 3D data in pathology. Medical education and research stand to benefit a great deal from utilizing 3D data as it can change our understanding of complex anatomical and histological structures. Although these technologies are yet to be adopted in routine surgical pathology, forensic pathology has embraced 3D scanning and model reconstruction. In this review, we intend to provide a general overview of the technologies and emerging applications involved with 3D data.

Topology-aware illumination design for volume rendering

Background

Direct volume rendering is one of flexible and effective approaches to inspect large volumetric data such as medical and biological images. In conventional volume rendering, it is often time consuming to set up a meaningful illumination environment. Moreover, conventional illumination approaches usually assign same values of variables of an illumination model to different structures manually and thus neglect the important illumination variations due to structure differences.

Results

We introduce a novel illumination design paradigm for volume rendering on the basis of topology to automate illumination parameter definitions meaningfully. The topological features are extracted from the contour tree of an input volumetric data. The automation of illumination design is achieved based on four aspects of attenuation, distance, saliency, and contrast perception. To better distinguish structures and maximize illuminance perception differences of structures, a two-phase topology-aware illuminance perception contrast model is proposed based on the psychological concept of Just-Noticeable-Difference.

Conclusions

The proposed approach allows meaningful and efficient automatic generations of illumination in volume rendering. Our results showed that our approach is more effective in depth and shape depiction, as well as providing higher perceptual differences between structures.

Getting the whole picture: High content screening using three-dimensional cellular model systems and whole animal assays

Phenotypic or High Content Screening (HCS) is becoming more widely used for primary screening campaigns in drug discovery. Currently the vast majority of HCS campaigns are using cell lines grown in well-established monolayer cultures (2D tissue culture). There is widespread recognition that the more biologically relevant 3D tissue culture technologies such as spheroids and organoids and even whole animal assays will eventually be run as primary HCS. Upgrading the IT infrastructure to cope with the increase in data volumes requires investments in hardware (and software) and this will be manageable. However, the main bottleneck for the effective adoption and use of 3D tissue culture and whole animal assays in HCS is anticipated to be the development of software for the analysis of 3D images. In this review we summarize the current state of the available software and how they may be applied to analyzing 3D images obtained from a HCS campaign. © 2016 International Society for Advancement of Cytometry.

Analysis of in vivo single cell behavior by high throughput, human-in-the-loop segmentation of three-dimensional images

BACKGROUND

Analysis of single cells in their native environment is a powerful method to address key questions in developmental systems biology. Confocal microscopy imaging of intact tissues, followed by automatic image segmentation, provides a means to conduct cytometric studies while at the same time preserving crucial information about the spatial organization of the tissue and morphological features of the cells. This technique is rapidly evolving but is still not in widespread use among research groups that do not specialize in technique development, perhaps in part for lack of tools that automate repetitive tasks while allowing experts to make the best use of their time in injecting their domain-specific knowledge.

RESULTS

Here we focus on a well-established stem cell model system, the C. elegans gonad, as well as on two other model systems widely used to study cell fate specification and morphogenesis: the pre-implantation mouse embryo and the developing mouse olfactory epithelium. We report a pipeline that integrates machine-learning-based cell detection, fast human-in-the-loop curation of these detections, and running of active contours seeded from detections to segment cells. The procedure can be bootstrapped by a small number of manual detections, and outperforms alternative pieces of software we benchmarked on C. elegans gonad datasets. Using cell segmentations to quantify fluorescence contents, we report previously-uncharacterized cell behaviors in the model systems we used. We further show how cell morphological features can be used to identify cell cycle phase; this provides a basis for future tools that will streamline cell cycle experiments by minimizing the need for exogenous cell cycle phase labels.

CONCLUSIONS

High-throughput 3D segmentation makes it possible to extract rich information from images that are routinely acquired by biologists, and provides insights - in particular with respect to the cell cycle - that would be difficult to derive otherwise.

Procedures for the quantification of whole-tissue immunofluorescence images obtained at single-cell resolution during murine tubular organ development

Whole-tissue quantification at single-cell resolution has become an inevitable approach for further quantitative understanding of morphogenesis in organ development. The feasibility of the approach has been dramatically increased by recent technological improvements in optical tissue clearing and microscopy. However, the series of procedures required for this approach to lead to successful whole-tissue quantification is far from developed. To provide the appropriate procedure, we here show tips for each critical step of the entire process, including fixation for immunofluorescence, optical clearing, and digital image processing, using developing murine internal organs such as epididymis, kidney, and lung as an example. Through comparison of fixative solutions and of clearing methods, we found optimal conditions to achieve clearer deep-tissue imaging of specific immunolabeled targets and explain what methods result in vivid volume imaging. In addition, we demonstrated that three-dimensional digital image processing after optical clearing produces objective quantitative data for the whole-tissue analysis, focusing on the spatial distribution of mitotic cells in the epididymal tubule. The procedure for the whole-tissue quantification shown in this article should contribute to systematic measurements of cellular processes in developing organs, accelerating the further understanding of morphogenesis at the single cell level.

The ImageJ ecosystem: An open platform for biomedical image analysis

Technology in microscopy advances rapidly, enabling increasingly affordable, faster, and more precise quantitative biomedical imaging, which necessitates correspondingly more-advanced image processing and analysis techniques. A wide range of software is available-from commercial to academic, special-purpose to Swiss army knife, small to large-but a key characteristic of software that is suitable for scientific inquiry is its accessibility. Open-source software is ideal for scientific endeavors because it can be freely inspected, modified, and redistributed; in particular, the open-software platform ImageJ has had a huge impact on the life sciences, and continues to do so. From its inception, ImageJ has grown significantly due largely to being freely available and its vibrant and helpful user community. Scientists as diverse as interested hobbyists, technical assistants, students, scientific staff, and advanced biology researchers use ImageJ on a daily basis, and exchange knowledge via its dedicated mailing list. Uses of ImageJ range from data visualization and teaching to advanced image processing and statistical analysis. The software's extensibility continues to attract biologists at all career stages as well as computer scientists who wish to effectively implement specific image-processing algorithms. In this review, we use the ImageJ project as a case study of how open-source software fosters its suites of software tools, making multitudes of image-analysis technology easily accessible to the scientific community. We specifically explore what makes ImageJ so popular, how it impacts the life sciences, how it inspires other projects, and how it is self-influenced by coevolving projects within the ImageJ ecosystem.

3D reconstruction of neurons in electron microscopy images

With the prevalence of brain-related diseases like Alzheimer in an increasing ageing population, Connectomics, the study of connections between neurons of the human brain, has emerged as a novel and challenging research topic. Accurate and fully automatic algorithms are needed to deal with the increasing amount of data from the brain images. This paper presents an automatic 3D neuron reconstruction technique where neurons within each slice image are first segmented and then linked across multiple slices within the publicly available Electron Microscopy dataset (SNEMI3D). First, random Forest classifier is adapted on top of superpixels for the neuron segmentation within each slice image. The maximum overlap between two consecutive images is then calculated for neuron linking, where the adjacency matrix of two different labeling of the segments is used to distinguish neuron merging and splitting. Experiments over the SNEMI3D dataset show that the proposed technique is efficient and accurate.

neuTube 1.0: A New Design for Efficient Neuron Reconstruction Software Based on the SWC Format

Compared to other existing tools, the novel software we present has some unique features such as comprehensive editing functions and the combination of seed-based tracing and path searching algorithms, as well as their availability in parallel 2D and 3D visualization. These features allow the user to reconstruct neuronal morphology efficiently in a comfortable “What You See Is What You Get” (WYSIWYG) way.

Brain circuit mapping requires digital reconstruction of neuronal morphologies in complicated networks. Despite recent advances in automatic algorithms, reconstruction of neuronal structures is still a bottleneck in circuit mapping due to a lack of appropriate software for both efficient reconstruction and user-friendly editing. Here we present a new software design based on the SWC format, a standardized neuromorphometric format that has been widely used for analyzing neuronal morphologies or sharing neuron reconstructions via online archives such as NeuroMorpho.org. We have also implemented the design in our open-source software called neuTube 1.0. As specified by the design, the software is equipped with parallel 2D and 3D visualization and intuitive neuron tracing/editing functions, allowing the user to efficiently reconstruct neurons from fluorescence image data and edit standard neuron structure files produced by any other reconstruction software. We show the advantages of neuTube 1.0 by comparing it to two other software tools, namely Neuromantic and Neurostudio. The software is available for free at http://www.neutracing.com, which also hosts complete software documentation and video tutorials.

Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis

Three-dimensional (3D) bioimaging, visualization and data analysis are in strong need of powerful 3D exploration techniques. We develop virtual finger (VF) to generate 3D curves, points and regions-of-interest in the 3D space of a volumetric image with a single finger operation, such as a computer mouse stroke, or click or zoom from the 2D-projection plane of an image as visualized with a computer. VF provides efficient methods for acquisition, visualization and analysis of 3D images for roundworm, fruitfly, dragonfly, mouse, rat and human. Specifically, VF enables instant 3D optical zoom-in imaging, 3D free-form optical microsurgery, and 3D visualization and annotation of terabytes of whole-brain image volumes. VF also leads to orders of magnitude better efficiency of automated 3D reconstruction of neurons and similar biostructures over our previous systems. We use VF to generate from images of 1,107 Drosophila GAL4 lines a projectome of a Drosophila brain.

Three-dimensional visualization of nanostructured surfaces and bacterial attachment using Autodesk® Maya®

There has been a growing interest in understanding the ways in which bacteria interact with nano-structured surfaces. As a result, there is a need for innovative approaches to enable researchers to visualize the biological processes taking place, despite the fact that it is not possible to directly observe these processes. We present a novel approach for the three-dimensional visualization of bacterial interactions with nano-structured surfaces using the software package Autodesk Maya. Our approach comprises a semi-automated stage, where actual surface topographic parameters, obtained using an atomic force microscope, are imported into Maya via a custom Python script, followed by a ‘creative stage', where the bacterial cells and their interactions with the surfaces are visualized using available experimental data. The ‘Dynamics' and ‘nDynamics' capabilities of the Maya software allowed the construction and visualization of plausible interaction scenarios. This capability provides a practical aid to knowledge discovery, assists in the dissemination of research results, and provides an opportunity for an improved public understanding. We validated our approach by graphically depicting the interactions between the two bacteria being used for modeling purposes, Staphylococcus aureus and Pseudomonas aeruginosa, with different titanium substrate surfaces that are routinely used in the production of biomedical devices.

Extensible visualization and analysis for multidimensional images using Vaa3D

Open-Source 3D Visualization-Assisted Analysis (Vaa3D) is a software platform for the visualization and analysis of large-scale multidimensional images. In this protocol we describe how to use several popular features of Vaa3D, including (i) multidimensional image visualization, (ii) 3D image object generation and quantitative measurement, (iii) 3D image comparison, fusion and management, (iv) visualization of heterogeneous images and respective surface objects and (v) extension of Vaa3D functions using its plug-in interface. We also briefly demonstrate how to integrate these functions for complicated applications of microscopic image visualization and quantitative analysis using three exemplar pipelines, including an automated pipeline for image filtering, segmentation and surface generation; an automated pipeline for 3D image stitching; and an automated pipeline for neuron morphology reconstruction, quantification and comparison. Once a user is familiar with Vaa3D, visualization usually runs in real time and analysis takes less than a few minutes for a simple data set.

Building quantitative, three-dimensional atlases of gene expression and morphology at cellular resolution

Animals comprise dynamic three-dimensional arrays of cells that express gene products in intricate spatial and temporal patterns that determine cellular differentiation and morphogenesis. A rigorous understanding of these developmental processes requires automated methods that quantitatively record and analyze complex morphologies and their associated patterns of gene expression at cellular resolution. Here we summarize light microscopy-based approaches to establish permanent, quantitative datasets-atlases-that record this information. We focus on experiments that capture data for whole embryos or large areas of tissue in three dimensions, often at multiple time points. We compare and contrast the advantages and limitations of different methods and highlight some of the discoveries made. We emphasize the need for interdisciplinary collaborations and integrated experimental pipelines that link sample preparation, image acquisition, image analysis, database design, visualization, and quantitative analysis.

Evaluation of the effectiveness of simple nuclei-segmentation methods on Caenorhabditis elegans embryogenesis images

BACKGROUND

For the analysis of spatio-temporal dynamics, various automated processing methods have been developed for nuclei segmentation. These methods tend to be complex for segmentation of images with crowded nuclei, preventing the simple reapplication of the methods to other problems. Thus, it is useful to evaluate the ability of simple methods to segment images with various degrees of crowded nuclei.

RESULTS

Here, we selected six simple methods from various watershed based and local maxima detection based methods that are frequently used for nuclei segmentation, and evaluated their segmentation accuracy for each developmental stage of the Caenorhabditis elegans. We included a 4D noise filter, in addition to 2D and 3D noise filters, as a pre-processing step to evaluate the potential of simple methods as widely as possible. By applying the methods to image data between the 50- to 500-cell developmental stages at 50-cell intervals, the error rate for nuclei detection could be reduced to ≤ 2.1% at every stage until the 350-cell stage. The fractions of total errors throughout the stages could be reduced to ≤ 2.4%. The error rates improved at most of the stages and the total errors improved when a 4D noise filter was used. The methods with the least errors were two watershed-based methods with 4D noise filters. For all the other methods, the error rate and the fraction of errors could be reduced to ≤ 4.2% and ≤ 4.1%, respectively. The minimum error rate for each stage between the 400- to 500-cell stages ranged from 6.0% to 8.4%. However, similarities between the computational and manual segmentations measured by volume overlap and Hausdorff distance were not good. The methods were also applied to Drosophila and zebrafish embryos and found to be effective.

CONCLUSIONS

The simple segmentation methods were found to be useful for detecting nuclei until the 350-cell stage, but not very useful after the 400-cell stage. The incorporation of a 4D noise filter to the simple methods could improve their performances. Error types and the temporal biases of errors were dependent on the methods used. Combining multiple simple methods could also give good segmentations.

A high-fidelity cell lineage tracing method for obtaining systematic spatiotemporal gene expression patterns in Caenorhabditis elegans

Advances in microscopy and fluorescent reporters have allowed us to detect the onset of gene expression on a cell-by-cell basis in a systemic fashion. This information, however, is often encoded in large repositories of images, and developing ways to extract this spatiotemporal expression data is a difficult problem that often uses complex domain-specific methods for each individual data set. We present a more unified approach that incorporates general previous information into a hierarchical probabilistic model to extract spatiotemporal gene expression from 4D confocal microscopy images of developing Caenorhabditis elegans embryos. This approach reduces the overall error rate of our automated lineage tracing pipeline by 3.8-fold, allowing us to routinely follow the C. elegans lineage to later stages of development, where individual neuronal subspecification becomes apparent. Unlike previous methods that often use custom approaches that are organism specific, our method uses generalized linear models and extensions of standard reversible jump Markov chain Monte Carlo methods that can be readily extended to other organisms for a variety of biological inference problems relating to cell fate specification. This modeling approach is flexible and provides tractable avenues for incorporating additional previous information into the model for similar difficult high-fidelity/low error tolerance image analysis problems for systematically applied genomic experiments.

Freely-available, true-color volume rendering software and cryohistology data sets for virtual exploration of the temporal bone anatomy

BACKGROUND

Cadaveric dissection of temporal bone anatomy is not always possible or feasible in certain educational environments. Volume rendering using CT and/or MRI helps understanding spatial relationships, but they suffer in nonrealistic depictions especially regarding color of anatomical structures. Freely available, nonstained histological data sets and software which are able to render such data sets in realistic color could overcome this limitation and be a very effective teaching tool.

METHODS

With recent availability of specialized public-domain software, volume rendering of true-color, histological data sets is now possible. We present both feasibility as well as step-by-step instructions to allow processing of publicly available data sets (Visible Female Human and Visible Ear) into easily navigable 3-dimensional models using free software.

RESULTS

Example renderings are shown to demonstrate the utility of these free methods in virtual exploration of the complex anatomy of the temporal bone. After exploring the data sets, the Visible Ear appears more natural than the Visible Human.

CONCLUSION

We provide directions for an easy-to-use, open-source software in conjunction with freely available histological data sets. This work facilitates self-education of spatial relationships of anatomical structures inside the human temporal bone as well as it allows exploration of surgical approaches prior to cadaveric testing and/or clinical implementation.