Image-derived, three-dimensional generative models of cellular organization

Learning Morphological, Spatial, and Dynamic Models of Cellular Components

In this chapter, we describe protocols for using the CellOrganizer software on the Jupyter Notebook platform to analyze and model cell and organelle shape and spatial arrangement. CellOrganizer is an open-source system for using microscope images to learn statistical models of the structure of cell components and how those components are organized relative to each other. Such models capture the statistical variation in the organization of cellular components by jointly modeling the distributions of their number, shape, and spatial distributions. These models can be created for different cell types or conditions and compared to reflect differences in their spatial organizations. The models are also generative, in that they can be used to synthesize new cell instances reflecting what a model learned and to provide well-structured cell geometries that can be used for biochemical simulations.

A rapid, high-throughput, viral infectivity assay using automated brightfield microscopy with machine learning

Infectivity assays are essential for the development of viral vaccines, antiviral therapies, and the manufacture of biologicals. Traditionally, these assays take 2-7 days and require several manual processing steps after infection. We describe an automated viral infectivity assay (AVIATM), using convolutional neural networks (CNNs) and high-throughput brightfield microscopy on 96-well plates that can quantify infection phenotypes within hours, before they are manually visible, and without sample preparation. CNN models were trained on HIV, influenza A virus, coronavirus 229E, vaccinia viruses, poliovirus, and adenoviruses, which together span the four major categories of virus (DNA, RNA, enveloped, and non-enveloped). A sigmoidal function, fit between virus dilution curves and CNN predictions, results in sensitivity ranges comparable to or better than conventional plaque or TCID50 assays, and a precision of ∼10%, which is considerably better than conventional infectivity assays. Because this technology is based on sensitizing CNNs to specific phenotypes of infection, it has potential as a rapid, broad-spectrum tool for virus characterization, and potentially identification.

Cellular harmonics for the morphology-invariant analysis of molecular organization at the cell surface

The spatiotemporal organization of membrane-associated molecules is central to the regulation of cellular signals. Powerful new microscopy techniques enable the three-dimensional visualization of localization and activation of these molecules; however, the quantitative interpretation and comparison of molecular organization on the three-dimensional cell surface remains challenging because cells themselves vary greatly in morphology. Here we introduce u-signal3D, a framework to assess the spatial scales of molecular organization at the cell surface in a cell-morphology-invariant manner. We validated the framework by analyzing synthetic signaling patterns painted onto observed cell morphologies, as well as measured distributions of cytoskeletal and signaling molecules. To demonstrate the framework's versatility, we further compared the spatial organization of cell surface signals both within, and between, cell populations, and powered an upstream machine-learning-based analysis of signaling motifs.

Using Simulated Training Data of Voxel-Level Generative Models to Improve 3D Neuron Reconstruction

Reconstructing neuron morphologies from fluorescence microscope images plays a critical role in neuroscience studies. It relies on image segmentation to produce initial masks either for further processing or final results to represent neuronal morphologies. This has been a challenging step due to the variation and complexity of noisy intensity patterns in neuron images acquired from microscopes. Whereas progresses in deep learning have brought the goal of accurate segmentation much closer to reality, creating training data for producing powerful neural networks is often laborious. To overcome the difficulty of obtaining a vast number of annotated data, we propose a novel strategy of using two-stage generative models to simulate training data with voxel-level labels. Trained upon unlabeled data by optimizing a novel objective function of preserving predefined labels, the models are able to synthesize realistic 3D images with underlying voxel labels. We showed that these synthetic images could train segmentation networks to obtain even better performance than manually labeled data. To demonstrate an immediate impact of our work, we further showed that segmentation results produced by networks trained upon synthetic data could be used to improve existing neuron reconstruction methods.

A deep generative model of 3D single-cell organization

We introduce a framework for end-to-end integrative modeling of 3D single-cell multi-channel fluorescent image data of diverse subcellular structures. We employ stacked conditional β-variational autoencoders to first learn a latent representation of cell morphology, and then learn a latent representation of subcellular structure localization which is conditioned on the learned cell morphology. Our model is flexible and can be trained on images of arbitrary subcellular structures and at varying degrees of sparsity and reconstruction fidelity. We train our full model on 3D cell image data and explore design trade-offs in the 2D setting. Once trained, our model can be used to predict plausible locations of structures in cells where these structures were not imaged. The trained model can also be used to quantify the variation in the location of subcellular structures by generating plausible instantiations of each structure in arbitrary cell geometries. We apply our trained model to a small drug perturbation screen to demonstrate its applicability to new data. We show how the latent representations of drugged cells differ from unperturbed cells as expected by on-target effects of the drugs.

Data science in cell imaging

Cell imaging has entered the 'Big Data' era. New technologies in light microscopy and molecular biology have led to an explosion in high-content, dynamic and multidimensional imaging data. Similar to the 'omics' fields two decades ago, our current ability to process, visualize, integrate and mine this new generation of cell imaging data is becoming a critical bottleneck in advancing cell biology. Computation, traditionally used to quantitatively test specific hypotheses, must now also enable iterative hypothesis generation and testing by deciphering hidden biologically meaningful patterns in complex, dynamic or high-dimensional cell image data. Data science is uniquely positioned to aid in this process. In this Perspective, we survey the rapidly expanding new field of data science in cell imaging. Specifically, we highlight how data science tools are used within current image analysis pipelines, propose a computation-first approach to derive new hypotheses from cell image data, identify challenges and describe the next frontiers where we believe data science will make an impact. We also outline steps to ensure broad access to these powerful tools - democratizing infrastructure availability, developing sensitive, robust and usable tools, and promoting interdisciplinary training to both familiarize biologists with data science and expose data scientists to cell imaging.

Probing the characteristics and biofunctional effects of disease-affected cells and drug response via machine learning applications

Drug-induced transformations in disease characteristics at the cellular and molecular level offers the opportunity to predict and evaluate the efficacy of pharmaceutical ingredients whilst enabling the optimal design of new and improved drugs with enhanced pharmacokinetics and pharmacodynamics. Machine learning is a promising in-silico tool used to simulate cells with specific disease properties and to determine their response toward drug uptake. Differences in the properties of normal and infected cells, including biophysical, biochemical and physiological characteristics, plays a key role in developing fundamental cellular probing platforms for machine learning applications. Cellular features can be extracted periodically from both the drug treated, infected, and normal cells via image segmentations in order to probe dynamic differences in cell behavior. Cellular segmentation can be evaluated to reflect the levels of drug effect on a distinct cell or group of cells via probability scoring. This article provides an account for the use of machine learning methods to probe differences in the biophysical, biochemical and physiological characteristics of infected cells in response to pharmacokinetics uptake of drug ingredients for application in cancer, diabetes and neurodegenerative disease therapies.

Computational modeling of cellular structures using conditional deep generative networks

MOTIVATION

Cellular function is closely related to the localizations of its sub-structures. It is, however, challenging to experimentally label all sub-cellular structures simultaneously in the same cell. This raises the need of building a computational model to learn the relationships among these sub-cellular structures and use reference structures to infer the localizations of other structures.

RESULTS

We formulate such a task as a conditional image generation problem and propose to use conditional generative adversarial networks for tackling it. We employ an encoder-decoder network as the generator and propose to use skip connections between the encoder and decoder to provide spatial information to the decoder. To incorporate the conditional information in a variety of different ways, we develop three different types of skip connections, known as the self-gated connection, encoder-gated connection and label-gated connection. The proposed skip connections are built based on the conditional information using gating mechanisms. By learning a gating function, the network is able to control what information should be passed through the skip connections from the encoder to the decoder. Since the gate parameters are also learned automatically, we expect that only useful spatial information is transmitted to the decoder to help image generation. We perform both qualitative and quantitative evaluations to assess the effectiveness of our proposed approaches. Experimental results show that our cGAN-based approaches have the ability to generate the desired sub-cellular structures correctly. Our results also demonstrate that the proposed approaches outperform the existing approach based on adversarial auto-encoders, and the new skip connections lead to improved performance. In addition, the localizations of generated sub-cellular structures by our approaches are consistent with observations in biological experiments.

AVAILABILITY AND IMPLEMENTATION

The source code and more results are available at https://github.com/divelab/cgan/.

An image-based data-driven analysis of cellular architecture in a developing tissue

Quantitative microscopy is becoming increasingly crucial in efforts to disentangle the complexity of organogenesis, yet adoption of the potent new toolbox provided by modern data science has been slow, primarily because it is often not directly applicable to developmental imaging data. We tackle this issue with a newly developed algorithm that uses point cloud-based morphometry to unpack the rich information encoded in 3D image data into a straightforward numerical representation. This enabled us to employ data science tools, including machine learning, to analyze and integrate cell morphology, intracellular organization, gene expression and annotated contextual knowledge. We apply these techniques to construct and explore a quantitative atlas of cellular architecture for the zebrafish posterior lateral line primordium, an experimentally tractable model of complex self-organized organogenesis. In doing so, we are able to retrieve both previously established and novel biologically relevant patterns, demonstrating the potential of our data-driven approach.

Image-derived models of cell organization changes during differentiation and drug treatments

PC12 cells are a popular model system to study changes driving and accompanying neuronal differentiation. While attention has been paid to changes in transcriptional regulation and protein signaling, much less is known about the changes in organization that accompany PC12 differentiation. Fluorescence microscopy can provide extensive information about these changes, although it is difficult to continuously observe changes over many days of differentiation. We describe a generative model of differentiation-associated changes in cell and nuclear shape and their relationship to mitochondrial distribution constructed from images of different cells at discrete time points. We show that the model accurately represents complex cell and nuclear shapes and learn a regression model that relates cell and nuclear shape to mitochondrial distribution; the predictive accuracy of the model increases during differentiation. Most importantly, we propose a method, based on cell matching and interpolation, to produce realistic simulations of the dynamics of cell differentiation from only static images. We also found that the distribution of cell shapes is hollow: most shapes are very different from the average shape. Finally, we show how the method can be used to model nuclear shape changes of human-induced pluripotent stem cells resulting from drug treatments.

Applications and Challenges of Machine Learning to Enable Realistic Cellular Simulations

In this perspective, we examine three key aspects of an end-to-end pipeline for realistic cellular simulations: reconstruction and segmentation of cellular structures; generation of cellular structures; and mesh generation, simulation, and data analysis. We highlight some of the relevant prior work in these distinct but overlapping areas, with a particular emphasis on current use of machine learning technologies, as well as on future opportunities.

3D Shape Modeling for Cell Nuclear Morphological Analysis and Classification

Quantitative analysis of morphological changes in a cell nucleus is important for the understanding of nuclear architecture and its relationship with pathological conditions such as cancer. However, dimensionality of imaging data, together with a great variability of nuclear shapes, presents challenges for 3D morphological analysis. Thus, there is a compelling need for robust 3D nuclear morphometric techniques to carry out population-wide analysis. We propose a new approach that combines modeling, analysis, and interpretation of morphometric characteristics of cell nuclei and nucleoli in 3D. We used robust surface reconstruction that allows accurate approximation of 3D object boundary. Then, we computed geometric morphological measures characterizing the form of cell nuclei and nucleoli. Using these features, we compared over 450 nuclei with about 1,000 nucleoli of epithelial and mesenchymal prostate cancer cells, as well as 1,000 nuclei with over 2,000 nucleoli from serum-starved and proliferating fibroblast cells. Classification of sets of 9 and 15 cells achieved accuracy of 95.4% and 98%, respectively, for prostate cancer cells, and 95% and 98% for fibroblast cells. To our knowledge, this is the first attempt to combine these methods for 3D nuclear shape modeling and morphometry into a highly parallel pipeline workflow for morphometric analysis of thousands of nuclei and nucleoli in 3D.

A quantitative approach for analyzing the spatio-temporal distribution of 3D intracellular events in fluorescence microscopy

Analysis of the spatial distribution of endomembrane trafficking is fundamental to understand the mechanisms controlling cellular dynamics, cell homeostasy, and cell interaction with its external environment in normal and pathological situations. We present a semi-parametric framework to quantitatively analyze and visualize the spatio-temporal distribution of intracellular events from different conditions. From the spatial coordinates of intracellular features such as segmented subcellular structures or vesicle trajectories, QuantEv automatically estimates weighted densities that are easy to interpret and performs a comprehensive statistical analysis from distribution distances. We apply this approach to study the spatio-temporal distribution of moving Rab6 fluorescently labeled membranes with respect to their direction of movement in crossbow- and disk-shaped cells. We also investigate the position of the generating hub of Rab11-positive membranes and the effect of actin disruption on Rab11 trafficking in coordination with cell shape.

Proteins are the workhorses of the body, performing a range of roles that are essential for life. Often, this requires these molecules to move from one location to another inside a cell. Scientists are interested in following an individual protein in a living cell ‘in real time’, as this helps understand what this protein does.

Scientists can track the whereabouts of a protein by ‘tagging’ it with a fluorescent molecule that emits light which can be picked up by a powerful microscope. This process is repeated many times on different samples. Finally, researchers have to analyze all the resulting images, and conduct statistical analysis to draw robust conclusions about the overall trajectories of the proteins. This process often relies on experts assessing the images, and it is therefore time-consuming and not easily scalable or applied to other experiments.

To help with this, Pécot et al. have developed QuantEV, a free algorithm that can analyze proteins’ paths within a cell, and then return statistical graphs and 3D visualizations. The program also gives access to the statistical procedure that was used, which means that different experiments can be compared.

Pécot et al. used the method to follow the Rab6 protein in cells of different shapes, and found that the conformation of the cell influences where Rab6 is located. For example, in crossbow-shaped cells, Rab6 is found more often toward the three tips of the crossbow, while its distribution is uniform in cells that look like disks. Another experiment examined where the protein Rab11 is normally placed, and how this changes when the cell’s skeleton is artificially disrupted. Both studies help to gain an insight into the behavior of the cellular structures in which Rab6 and Rab11 are embedded.

Following proteins in the cell is an increasingly popular method, and there is therefore a growing amount of data to process. QuantEV should make it easier for biologists to analyze their results, which could help them to have a better grasp on how cells work in various circumstances.

Subcellular protein expression models for microsatellite instability in colorectal adenocarcinoma tissue images

Background

New bioimaging techniques capable of visualising the co-location of numerous proteins within individual cells have been proposed to study tumour heterogeneity of neighbouring cells within the same tissue specimen. These techniques have highlighted the need to better understand the interplay between proteins in terms of their colocalisation.

Results

We recently proposed a cellular-level model of the healthy and cancerous colonic crypt microenvironments. Here, we extend the model to include detailed models of protein expression to generate synthetic multiplex fluorescence data. As a first step, we present models for various cell organelles learned from real immunofluorescence data from the Human Protein Atlas. Comparison between the distribution of various features obtained from the real and synthetic organelles has shown very good agreement. This has included both features that have been used as part of the model input and ones that have not been explicitly considered. We then develop models for six proteins which are important colorectal cancer biomarkers and are associated with microsatellite instability, namely MLH1, PMS2, MSH2, MSH6, P53 and PTEN. The protein models include their complex expression patterns and which cell phenotypes express them. The models have been validated by comparing distributions of real and synthesised parameters and by application of frameworks for analysing multiplex immunofluorescence image data.

Conclusions

The six proteins have been chosen as a case study to illustrate how the model can be used to generate synthetic multiplex immunofluorescence data. Further proteins could be included within the model in a similar manner to enable the study of a larger set of proteins of interest and their interactions. To the best of our knowledge, this is the first model for expression of multiple proteins in anatomically intact tissue, rather than within cells in culture.

Large-scale image-based screening and profiling of cellular phenotypes

Cellular phenotypes are observable characteristics of cells resulting from the interactions of intrinsic and extrinsic chemical or biochemical factors. Image-based phenotypic screens under large numbers of basal or perturbed conditions can be used to study the influences of these factors on cellular phenotypes. Hundreds to thousands of phenotypic descriptors can also be quantified from the images of cells under each of these experimental conditions. Therefore, huge amounts of data can be generated, and the analysis of these data has become a major bottleneck in large-scale phenotypic screens. Here, we review current experimental and computational methods for large-scale image-based phenotypic screens. Our focus is on phenotypic profiling, a computational procedure for constructing quantitative and compact representations of cellular phenotypes based on the images collected in these screens. © 2016 International Society for Advancement of Cytometry.

Strategy and software for the statistical spatial analysis of 3D intracellular distributions

The localization of proteins in specific domains or compartments in the 3D cellular space is essential for many fundamental processes in eukaryotic cells. Deciphering spatial organization principles within cells is a challenging task, in particular because of the large morphological variations between individual cells. We present here an approach for normalizing variations in cell morphology and for statistically analyzing spatial distributions of intracellular compartments from collections of 3D images. The method relies on the processing and analysis of 3D geometrical models that are generated from image stacks and that are used to build representations at progressively increasing levels of integration, ultimately revealing statistical significant traits of spatial distributions. To make this methodology widely available to end-users, we implemented our algorithmic pipeline into a user-friendly, multi-platform, and freely available software. To validate our approach, we generated 3D statistical maps of endomembrane compartments at subcellular resolution within an average epidermal root cell from collections of image stacks. This revealed unsuspected polar distribution patterns of organelles that were not detectable in individual images. By reversing the classical 'measure-then-average' paradigm, one major benefit of the proposed strategy is the production and display of statistical 3D representations of spatial organizations, thus fully preserving the spatial dimension of image data and at the same time allowing their integration over individual observations. The approach and software are generic and should be of general interest for experimental and modeling studies of spatial organizations at multiple scales (subcellular, cellular, tissular) in biological systems.

Point process models for localization and interdependence of punctate cellular structures

Accurate representations of cellular organization for multiple eukaryotic cell types are required for creating predictive models of dynamic cellular function. To this end, we have previously developed the CellOrganizer platform, an open source system for generative modeling of cellular components from microscopy images. CellOrganizer models capture the inherent heterogeneity in the spatial distribution, size, and quantity of different components among a cell population. Furthermore, CellOrganizer can generate quantitatively realistic synthetic images that reflect the underlying cell population. A current focus of the project is to model the complex, interdependent nature of organelle localization. We built upon previous work on developing multiple non-parametric models of organelles or structures that show punctate patterns. The previous models described the relationships between the subcellular localization of puncta and the positions of cell and nuclear membranes and microtubules. We extend these models to consider the relationship to the endoplasmic reticulum (ER), and to consider the relationship between the positions of different puncta of the same type. Our results do not suggest that the punctate patterns we examined are dependent on ER position or inter- and intra-class proximity. With these results, we built classifiers to update previous assignments of proteins to one of 11 patterns in three distinct cell lines. Our generative models demonstrate the ability to construct statistically accurate representations of puncta localization from simple cellular markers in distinct cell types, capturing the complex phenomena of cellular structure interaction with little human input. This protocol represents a novel approach to vesicular protein annotation, a field that is often neglected in high-throughput microscopy. These results suggest that spatial point process models provide useful insight with respect to the spatial dependence between cellular structures. © 2016 International Society for Advancement of Cytometry.

Building cell models and simulations from microscope images

The use of fluorescence microscopy has undergone a major revolution over the past twenty years, both with the development of dramatic new technologies and with the widespread adoption of image analysis and machine learning methods. Many open source software tools provide the ability to use these methods in a wide range of studies, and many molecular and cellular phenotypes can now be automatically distinguished. This article presents the next major challenge in microscopy automation, the creation of accurate models of cell organization directly from images, and reviews the progress that has been made towards this challenge.

Unbiased Rare Event Sampling in Spatial Stochastic Systems Biology Models Using a Weighted Ensemble of Trajectories

The long-term goal of connecting scales in biological simulation can be facilitated by scale-agnostic methods. We demonstrate that the weighted ensemble (WE) strategy, initially developed for molecular simulations, applies effectively to spatially resolved cell-scale simulations. The WE approach runs an ensemble of parallel trajectories with assigned weights and uses a statistical resampling strategy of replicating and pruning trajectories to focus computational effort on difficult-to-sample regions. The method can also generate unbiased estimates of non-equilibrium and equilibrium observables, sometimes with significantly less aggregate computing time than would be possible using standard parallelization. Here, we use WE to orchestrate particle-based kinetic Monte Carlo simulations, which include spatial geometry (e.g., of organelles, plasma membrane) and biochemical interactions among mobile molecular species. We study a series of models exhibiting spatial, temporal and biochemical complexity and show that although WE has important limitations, it can achieve performance significantly exceeding standard parallel simulation—by orders of magnitude for some observables.

Stochastic simulations (simulations where randomness plays a role) of even simple biological systems are often so computationally intensive that it is impossible, in practice, to simulate them exhaustively and gather good statistics about the likelihood of different outcomes. The difficulty is compounded for the observation of rare events in these simulations; unfortunately, rare events, such as state transitions and barrier crossings, are often those of particular interest. Using the weighted ensemble (WE) method, we are able to enhance the characterization of rare events in cell biology simulations, but in such a way that the statistics for these events remain unbiased. The histogram of outcomes that WE produces has the same shape as a naive one, but the resolution of events in the tails of the histogram is greatly improved. This improved resolution in rare event statistics can be used to infer unbiased estimates of long timescale dynamics from short simulations, and we show that using a weighted ensemble can result in a reduction in total simulation time needed to sample certain events of interest in spatial, stochastic models of biological systems.

Automated Learning of Subcellular Variation among Punctate Protein Patterns and a Generative Model of Their Relation to Microtubules

Characterizing the spatial distribution of proteins directly from microscopy images is a difficult problem with numerous applications in cell biology (e.g. identifying motor-related proteins) and clinical research (e.g. identification of cancer biomarkers). Here we describe the design of a system that provides automated analysis of punctate protein patterns in microscope images, including quantification of their relationships to microtubules. We constructed the system using confocal immunofluorescence microscopy images from the Human Protein Atlas project for 11 punctate proteins in three cultured cell lines. These proteins have previously been characterized as being primarily located in punctate structures, but their images had all been annotated by visual examination as being simply "vesicular". We were able to show that these patterns could be distinguished from each other with high accuracy, and we were able to assign to one of these subclasses hundreds of proteins whose subcellular localization had not previously been well defined. In addition to providing these novel annotations, we built a generative approach to modeling of punctate distributions that captures the essential characteristics of the distinct patterns. Such models are expected to be valuable for representing and summarizing each pattern and for constructing systems biology simulations of cell behaviors.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Joint modeling of cell and nuclear shape variation

Modeling cell shape variation is critical to our understanding of cell biology. Previous work has demonstrated the utility of nonrigid image registration methods for the construction of nonparametric nuclear shape models in which pairwise deformation distances are measured between all shapes and are embedded into a low-dimensional shape space. Using these methods, we explore the relationship between cell shape and nuclear shape. We find that these are frequently dependent on each other and use this as the motivation for the development of combined cell and nuclear shape space models, extending nonparametric cell representations to multiple-component three-dimensional cellular shapes and identifying modes of joint shape variation. We learn a first-order dynamics model to predict cell and nuclear shapes, given shapes at a previous time point. We use this to determine the effects of endogenous protein tags or drugs on the shape dynamics of cell lines and show that tagged C1QBP reduces the correlation between cell and nuclear shape. To reduce the computational cost of learning these models, we demonstrate the ability to reconstruct shape spaces using a fraction of computed pairwise distances. The open-source tools provide a powerful basis for future studies of the molecular basis of cell organization.

Extracting meaning from biological imaging data

Biological imaging continues to improve, capturing continually longer-term, richer, and more complex data, penetrating deeper into live tissue. How do we gain insight into the dynamic processes of disease and development from terabytes of multidimensional image data? Here I describe a collaborative approach to extracting meaning from biological imaging data. The collaboration consists of teams of biologists and engineers working together. Custom computational tools are built to best exploit application-specific knowledge in order to visualize and analyze large and complex data sets. The image data are summarized, extracting and modeling the features that capture the objects and relationships in the data. The summarization is validated, the results visualized, and errors corrected as needed. Finally, the customized analysis and visualization tools together with the image data and the summarization results are shared. This Perspective provides a brief guide to the mathematical ideas that rigorously quantify the notion of extracting meaning from biological image, and to the practical approaches that have been used to apply these ideas to a wide range of applications in cell and tissue optical imaging.

Toward the virtual cell: automated approaches to building models of subcellular organization "learned" from microscopy images

We review state-of-the-art computational methods for constructing, from image data, generative statistical models of cellular and nuclear shapes and the arrangement of subcellular structures and proteins within them. These automated approaches allow consistent analysis of images of cells for the purposes of learning the range of possible phenotypes, discriminating between them, and informing further investigation. Such models can also provide realistic geometry and initial protein locations to simulations in order to better understand cellular and subcellular processes. To determine the structures of cellular components and how proteins and other molecules are distributed among them, the generative modeling approach described here can be coupled with high throughput imaging technology to infer and represent subcellular organization from data with few a priori assumptions. We also discuss potential improvements to these methods and future directions for research.

From microscopy data to in silico environments for in vivo-oriented simulations

ABSTRACT

: In our previous study, we introduced a combination methodology of Fluorescence Correlation Spectroscopy (FCS) and Transmission Electron Microscopy (TEM), which is powerful to investigate the effect of intracellular environment to biochemical reaction processes. Now, we developed a reconstruction method of realistic simulation spaces based on our TEM images. Interactive raytracing visualization of this space allows the perception of the overall 3D structure, which is not directly accessible from 2D TEM images. Simulation results show that the diffusion in such generated structures strongly depends on image post-processing. Frayed structures corresponding to noisy images hinder the diffusion much stronger than smooth surfaces from denoised images. This means that the correct identification of noise or structure is significant to reconstruct appropriate reaction environment in silico in order to estimate realistic behaviors of reactants in vivo. Static structures lead to anomalous diffusion due to the partial confinement. In contrast, mobile crowding agents do not lead to anomalous diffusion at moderate crowding levels. By varying the mobility of these non-reactive obstacles (NRO), we estimated the relationship between NRO diffusion coefficient (Dnro) and the anomaly in the tracer diffusion (α). For Dnro=21.96 to 44.49 μm2/s, the simulation results match the anomaly obtained from FCS measurements. This range of the diffusion coefficient from simulations is compatible with the range of the diffusion coefficient of structural proteins in the cytoplasm. In addition, we investigated the relationship between the radius of NRO and anomalous diffusion coefficient of tracers by the comparison between different simulations. The radius of NRO has to be 58 nm when the polymer moves with the same diffusion speed as a reactant, which is close to the radius of functional protein complexes in a cell.

Analysis of spatial point patterns in nuclear biology

There is considerable interest in cell biology in determining whether, and to what extent, the spatial arrangement of nuclear objects affects nuclear function. A common approach to address this issue involves analyzing a collection of images produced using some form of fluorescence microscopy. We assume that these images have been successfully pre-processed and a spatial point pattern representation of the objects of interest within the nuclear boundary is available. Typically in these scenarios, the number of objects per nucleus is low, which has consequences on the ability of standard analysis procedures to demonstrate the existence of spatial preference in the pattern. There are broadly two common approaches to look for structure in these spatial point patterns. First a spatial point pattern for each image is analyzed individually, or second a simple normalization is performed and the patterns are aggregated. In this paper we demonstrate using synthetic spatial point patterns drawn from predefined point processes how difficult it is to distinguish a pattern from complete spatial randomness using these techniques and hence how easy it is to miss interesting spatial preferences in the arrangement of nuclear objects. The impact of this problem is also illustrated on data related to the configuration of PML nuclear bodies in mammalian fibroblast cells.

CellOrganizer: Image-derived models of subcellular organization and protein distribution

This chapter describes approaches for learning models of subcellular organization from images. The primary utility of these models is expected to be from incorporation into complex simulations of cell behaviors. Most current cell simulations do not consider spatial organization of proteins at all, or treat each organelle type as a single, idealized compartment. The ability to build generative models for all proteins in a proteome and use them for spatially accurate simulations is expected to improve the accuracy of models of cell behaviors. A second use, of potentially equal importance, is expected to be in testing and comparing software for analyzing cell images. The complexity and sophistication of algorithms used in cell-image-based screens and assays (variously referred to as high-content screening, high-content analysis, or high-throughput microscopy) is continuously increasing, and generative models can be used to produce images for testing these algorithms in which the expected answer is known.