Evaluation of methods for generative modeling of cell and nuclear shape

Interpretable representation learning for 3D multi-piece intracellular structures using point clouds

A key challenge in understanding subcellular organization is quantifying interpretable measurements of intracellular structures with complex multi-piece morphologies in an objective, robust and generalizable manner. Here we introduce a morphology-appropriate representation learning framework that uses 3D rotation invariant autoencoders and point clouds. This framework is used to learn representations of complex multi-piece morphologies that are independent of orientation, compact, and easy to interpret. We apply our framework to intracellular structures with punctate morphologies (e.g. DNA replication foci) and polymorphic morphologies (e.g. nucleoli). We systematically compare our framework to image-based autoencoders across several intracellular structure datasets, including a synthetic dataset with pre-defined rules of organization. We explore the trade-offs in the performance of different models by performing multi-metric benchmarking across efficiency, generative capability, and representation expressivity metrics. We find that our framework, which embraces the underlying morphology of multi-piece structures, facilitates the unsupervised discovery of sub-clusters for each structure. We show how our approach can also be applied to phenotypic profiling using a dataset of nucleolar images following drug perturbations. We implement and provide all representation learning models using CytoDL, a python package for flexible and configurable deep learning experiments.

Morphological profiling for drug discovery in the era of deep learning

Morphological profiling is a valuable tool in phenotypic drug discovery. The advent of high-throughput automated imaging has enabled the capturing of a wide range of morphological features of cells or organisms in response to perturbations at the single-cell resolution. Concurrently, significant advances in machine learning and deep learning, especially in computer vision, have led to substantial improvements in analyzing large-scale high-content images at high throughput. These efforts have facilitated understanding of compound mechanism of action, drug repurposing, characterization of cell morphodynamics under perturbation, and ultimately contributing to the development of novel therapeutics. In this review, we provide a comprehensive overview of the recent advances in the field of morphological profiling. We summarize the image profiling analysis workflow, survey a broad spectrum of analysis strategies encompassing feature engineering- and deep learning-based approaches, and introduce publicly available benchmark datasets. We place a particular emphasis on the application of deep learning in this pipeline, covering cell segmentation, image representation learning, and multimodal learning. Additionally, we illuminate the application of morphological profiling in phenotypic drug discovery and highlight potential challenges and opportunities in this field.

Morphological entropy encodes cellular migration strategies on multiple length scales

Cell migration is crucial for numerous physiological and pathological processes. A cell adapts its morphology, including the overall and nuclear morphology, in response to various cues in complex microenvironments, such as topotaxis and chemotaxis during migration. Thus, the dynamics of cellular morphology can encode migration strategies, from which diverse migration mechanisms can be inferred. However, deciphering the mechanisms behind cell migration encoded in morphology dynamics remains a challenging problem. Here, we present a powerful universal metric, the Cell Morphological Entropy (CME), developed by combining parametric morphological analysis with Shannon entropy. The utility of CME, which accurately quantifies the complex cellular morphology at multiple length scales through the deviation from a perfectly circular shape, is illustrated using a variety of normal and tumor cell lines in different in vitro microenvironments. Our results show how geometric constraints affect the MDA-MB-231 cell nucleus, the emerging interactions of MCF-10A cells migrating on collagen gel, and the critical transition from proliferation to invasion in tumor spheroids. The analysis demonstrates that the CME-based approach provides an effective and physically interpretable tool to measure morphology in real-time across multiple length scales. It provides deeper insight into cell migration and contributes to the understanding of different behavioral modes and collective cell motility in more complex microenvironments.

Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles

Cell and organelle shape are driven by diverse genetic and environmental factors and thus accurate quantification of cellular morphology is essential to experimental cell biology. Autoencoders are a popular tool for unsupervised biological image analysis because they learn a low-dimensional representation that maps images to feature vectors to generate a semantically meaningful embedding space of morphological variation. The learned feature vectors can also be used for clustering, dimensionality reduction, outlier detection, and supervised learning problems. Shape properties do not change with orientation, and thus we argue that representation learning methods should encode this orientation invariance. We show that conventional autoencoders are sensitive to orientation, which can lead to suboptimal performance on downstream tasks. To address this, we develop O2-variational autoencoder (O2-VAE), an unsupervised method that learns robust, orientation-invariant representations. We use O2-VAE to discover morphology subgroups in segmented cells and mitochondria, detect outlier cells, and rapidly characterise cellular shape and texture in large datasets, including in a newly generated synthetic benchmark.

Spherical harmonics analysis reveals cell shape-fate relationships in zebrafish lateral line neuromasts

Cell shape is a powerful readout of cell state, fate and function. We describe a custom workflow to perform semi-automated, 3D cell and nucleus segmentation, and spherical harmonics and principal components analysis to distill cell and nuclear shape variation into discrete biologically meaningful parameters. We apply these methods to analyze shape in the neuromast cells of the zebrafish lateral line system, finding that shapes vary with cell location and identity. The distinction between hair cells and support cells accounted for much of the variation, which allowed us to train classifiers to predict cell identity from shape features. Using transgenic markers for support cell subpopulations, we found that subtypes had different shapes from each other. To investigate how loss of a neuromast cell type altered cell shape distributions, we examined atoh1a mutants that lack hair cells. We found that mutant neuromasts lacked the cell shape phenotype associated with hair cells, but did not exhibit a mutant-specific cell shape. Our results demonstrate the utility of using 3D cell shape features to characterize, compare and classify cells in a living developing organism.

Learning Morphological, Spatial, and Dynamic Models of Cellular Components

In this chapter, we describe protocols for using the CellOrganizer software on the Jupyter Notebook platform to analyze and model cell and organelle shape and spatial arrangement. CellOrganizer is an open-source system for using microscope images to learn statistical models of the structure of cell components and how those components are organized relative to each other. Such models capture the statistical variation in the organization of cellular components by jointly modeling the distributions of their number, shape, and spatial distributions. These models can be created for different cell types or conditions and compared to reflect differences in their spatial organizations. The models are also generative, in that they can be used to synthesize new cell instances reflecting what a model learned and to provide well-structured cell geometries that can be used for biochemical simulations.

Cellular harmonics for the morphology-invariant analysis of molecular organization at the cell surface

The spatiotemporal organization of membrane-associated molecules is central to the regulation of cellular signals. Powerful new microscopy techniques enable the three-dimensional visualization of localization and activation of these molecules; however, the quantitative interpretation and comparison of molecular organization on the three-dimensional cell surface remains challenging because cells themselves vary greatly in morphology. Here we introduce u-signal3D, a framework to assess the spatial scales of molecular organization at the cell surface in a cell-morphology-invariant manner. We validated the framework by analyzing synthetic signaling patterns painted onto observed cell morphologies, as well as measured distributions of cytoskeletal and signaling molecules. To demonstrate the framework's versatility, we further compared the spatial organization of cell surface signals both within, and between, cell populations, and powered an upstream machine-learning-based analysis of signaling motifs.

Integrated intracellular organization and its variations in human iPS cells

Understanding how a subset of expressed genes dictates cellular phenotype is a considerable challenge owing to the large numbers of molecules involved, their combinatorics and the plethora of cellular behaviours that they determine1,2. Here we reduced this complexity by focusing on cellular organization-a key readout and driver of cell behaviour3,4-at the level of major cellular structures that represent distinct organelles and functional machines, and generated the WTC-11 hiPSC Single-Cell Image Dataset v1, which contains more than 200,000 live cells in 3D, spanning 25 key cellular structures. The scale and quality of this dataset permitted the creation of a generalizable analysis framework to convert raw image data of cells and their structures into dimensionally reduced, quantitative measurements that can be interpreted by humans, and to facilitate data exploration. This framework embraces the vast cell-to-cell variability that is observed within a normal population, facilitates the integration of cell-by-cell structural data and allows quantitative analyses of distinct, separable aspects of organization within and across different cell populations. We found that the integrated intracellular organization of interphase cells was robust to the wide range of variation in cell shape in the population; that the average locations of some structures became polarized in cells at the edges of colonies while maintaining the 'wiring' of their interactions with other structures; and that, by contrast, changes in the location of structures during early mitotic reorganization were accompanied by changes in their wiring.

Using Optimal Transport to Improve Spherical Harmonic Quantification of Complex Biological Shapes

The knowledge of the anatomical shape of both gross and microscopic structures is the key to understanding the effects of disease processes on cellular structure. Geometric morphometric methods, such as Procrustes superimposition, and Spherical Harmonics (SPHARM), have been used to capture the biological shape variation and group differences in morphology. Previous SPHARM-MAT techniques use the CALD algorithm to parameterize the mesh surface. It starts from initial mapping and performs local and global smoothing methods alternately to control the area and length distortions simultaneously. However, this parameterization may not be sufficient in complex morphological cases. To bridge this gap, we propose SPHARM-OT, an enhanced SPHARM surface modeling method using optimal transport (OT) for spherical parameterization. First, the genus 0 3D objects are conformally mapped onto a sphere. Then the optimal transport theory via spherical power diagram is introduced to minimize the area distortion. This new algorithm can effectively reduce the area distortion and lead to a better reconstruction result. We demonstrate the effectiveness of the method by applying it to the human sphenoidal paranasal sinuses.

Improving and evaluating deep learning models of cellular organization

MOTIVATION

Cells contain dozens of major organelles and thousands of other structures, many of which vary extensively in their number, size, shape and spatial distribution. This complexity and variation dramatically complicates the use of both traditional and deep learning methods to build accurate models of cell organization. Most cellular organelles are distinct objects with defined boundaries that do not overlap, while the pixel resolution of most imaging methods is n sufficient to resolve these boundaries. Thus while cell organization is conceptually object-based, most current methods are pixel-based. Using extensive image collections in which particular organelles were fluorescently labeled, deep learning methods can be used to build conditional autoencoder models for particular organelles. A major advance occurred with the use of a U-net approach to make multiple models all conditional upon a common reference, unlabeled image, allowing the relationships between different organelles to be at least partially inferred.

RESULTS

We have developed improved Generative Adversarial Networks-based approaches for learning these models and have also developed novel criteria for evaluating how well synthetic cell images reflect the properties of real images. The first set of criteria measure how well models preserve the expected property that organelles do not overlap. We also developed a modified loss function that allows retraining of the models to minimize that overlap. The second set of criteria uses object-based modeling to compare object shape and spatial distribution between synthetic and real images. Our work provides the first demonstration that, at least for some organelles, deep learning models can capture object-level properties of cell images.

AVAILABILITY AND IMPLEMENTATION

http://murphylab.cbd.cmu.edu/Software/2022_insilico.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Non-destructive monitoring of 3D cell cultures: new technologies and applications

3D cell cultures are becoming the new standard for cell-based in vitro research, due to their higher transferrability toward in vivo biology. The lack of established techniques for the non-destructive quantification of relevant variables, however, constitutes a major barrier to the adoption of these technologies, as it increases the resources needed for the experimentation and reduces its accuracy. In this review, we aim at addressing this limitation by providing an overview of different non-destructive approaches for the evaluation of biological features commonly quantified in a number of studies and applications. In this regard, we will cover cell viability, gene expression, population distribution, cell morphology and interactions between the cells and the environment. This analysis is expected to promote the use of the showcased technologies, together with the further development of these and other monitoring methods for 3D cell cultures. Overall, an extensive technology shift is required, in order for monolayer cultures to be superseded, but the potential benefit derived from an increased accuracy of in vitro studies, justifies the effort and the investment.

High-Throughput 3D Phenotyping of Plant Shoot Apical Meristems From Tissue-Resolution Data

Confocal imaging is a well-established method for investigating plant phenotypes on the tissue and organ level. However, many differences are difficult to assess by visual inspection and researchers rely extensively on ad hoc manual quantification techniques and qualitative assessment. Here we present a method for quantitatively phenotyping large samples of plant tissue morphologies using triangulated isosurfaces. We successfully demonstrate the applicability of the approach using confocal imaging of aerial organs in Arabidopsis thaliana. Automatic identification of flower primordia using the surface curvature as an indication of outgrowth allows for high-throughput quantification of divergence angles and further analysis of individual flowers. We demonstrate the throughput of our method by quantifying geometric features of 1065 flower primordia from 172 plants, comparing auxin transport mutants to wild type. Additionally, we find that a paraboloid provides a simple geometric parameterisation of the shoot inflorescence domain with few parameters. We utilise parameterisation methods to provide a computational comparison of the shoot apex defined by a fluorescent reporter of the central zone marker gene CLAVATA3 with the apex defined by the paraboloid. Finally, we analyse the impact of mutations which alter mechanical properties on inflorescence dome curvature and compare the results with auxin transport mutants. Our results suggest that region-specific expression domains of genes regulating cell wall biosynthesis and local auxin transport can be important in maintaining the wildtype tissue shape. Altogether, our results indicate a general approach to parameterise and quantify plant development in 3D, which is applicable also in cases where data resolution is limited, and cell segmentation not possible. This enables researchers to address fundamental questions of plant development by quantitative phenotyping with high throughput, consistency and reproducibility.

MIRIAM: A machine and deep learning single-cell segmentation and quantification pipeline for multi-dimensional tissue images

Increasingly, highly multiplexed tissue imaging methods are used to profile protein expression at the single‐cell level. However, a critical limitation is the lack of robust cell segmentation tools for tissue sections. We present Multiplexed Image Resegmentation of Internal Aberrant Membranes (MIRIAM) that combines (a) a pipeline for cell segmentation and quantification that incorporates machine learning‐based pixel classification to define cellular compartments, (b) a novel method for extending incomplete cell membranes, and (c) a deep learning‐based cell shape descriptor. Using human colonic adenomas as an example, we show that MIRIAM is superior to widely utilized segmentation methods and provides a pipeline that is broadly applicable to different imaging platforms and tissue types.

Learning Cellular Phenotypes through Supervision

Image-based cell phenotyping is an important and open problem in computational pathology. The two principal challenges are: 1) making the cell cluster properties insensitive to experimental settings (like seed point and feature selection) and 2) ensuring that the phenotypes emerging are biologically relevant and support clinical reporting. To gauge robustness, we first compare the consistency of the phenotypes using self-supervised and supervised features. Through case classification, we analyse the relevance of the self-supervised and supervised feature sets with respect to the clinical diagnosis. In addition, we demonstrate how we can add model explainability through Shapley values to identify more disease relevant cellular phenotypes and measure their importance in context of the disease. Here, myeloproliferative neoplasms, a haematopoietic stem cell disorder, where one particular cell type is of diagnostic relevance is used as an exemplar. The experiments conducted on a set of bone marrow trephines demonstrate an improvement of 7.4 % in accuracy for case classification using cellular phenotypes derived from the supervised scenario.

Red blood cell phenotyping from 3D confocal images using artificial neural networks

The investigation of cell shapes mostly relies on the manual classification of 2D images, causing a subjective and time consuming evaluation based on a portion of the cell surface. We present a dual-stage neural network architecture for analyzing fine shape details from confocal microscopy recordings in 3D. The system, tested on red blood cells, uses training data from both healthy donors and patients with a congenital blood disease, namely hereditary spherocytosis. Characteristic shape features are revealed from the spherical harmonics spectrum of each cell and are automatically processed to create a reproducible and unbiased shape recognition and classification. The results show the relation between the particular genetic mutation causing the disease and the shape profile. With the obtained 3D phenotypes, we suggest our method for diagnostics and theragnostics of blood diseases. Besides the application employed in this study, our algorithms can be easily adapted for the 3D shape phenotyping of other cell types and extend their use to other applications, such as industrial automated 3D quality control.

A robust unsupervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei

Cell morphology encodes essential information on many underlying biological processes. It is commonly used by clinicians and researchers in the study, diagnosis, prognosis, and treatment of human diseases. Quantification of cell morphology has seen tremendous advances in recent years. However, effectively defining morphological shapes and evaluating the extent of morphological heterogeneity within cell populations remain challenging. Here we present a protocol and software for the analysis of cell and nuclear morphology from fluorescence or bright-field images using the VAMPIRE algorithm ( https://github.com/kukionfr/VAMPIRE_open ). This algorithm enables the profiling and classification of cells into shape modes based on equidistant points along cell and nuclear contours. Examining the distributions of cell morphologies across automatically identified shape modes provides an effective visualization scheme that relates cell shapes to cellular subtypes based on endogenous and exogenous cellular conditions. In addition, these shape mode distributions offer a direct and quantitative way to measure the extent of morphological heterogeneity within cell populations. This protocol is highly automated and fast, with the ability to quantify the morphologies from 2D projections of cells seeded both on 2D substrates or embedded within 3D microenvironments, such as hydrogels and tissues. The complete analysis pipeline can be completed within 60 minutes for a dataset of ~20,000 cells/2,400 images.

Of Cell Shapes and Motion: The Physical Basis of Animal Cell Migration

Motile cells have developed a variety of migration modes relying on diverse traction-force-generation mechanisms. Before the behavior of intracellular components could be easily imaged, cell movements were mostly classified by different types of cellular shape dynamics. Indeed, even though some types of cells move without any significant change in shape, most cell propulsion mechanisms rely on global or local deformations of the cell surface. In this review, focusing mostly on metazoan cells, we discuss how different types of local and global shape changes underlie distinct migration modes. We then discuss mechanical differences between force-generation mechanisms and finish by speculating on how they may have evolved.

Image-derived models of cell organization changes during differentiation and drug treatments

PC12 cells are a popular model system to study changes driving and accompanying neuronal differentiation. While attention has been paid to changes in transcriptional regulation and protein signaling, much less is known about the changes in organization that accompany PC12 differentiation. Fluorescence microscopy can provide extensive information about these changes, although it is difficult to continuously observe changes over many days of differentiation. We describe a generative model of differentiation-associated changes in cell and nuclear shape and their relationship to mitochondrial distribution constructed from images of different cells at discrete time points. We show that the model accurately represents complex cell and nuclear shapes and learn a regression model that relates cell and nuclear shape to mitochondrial distribution; the predictive accuracy of the model increases during differentiation. Most importantly, we propose a method, based on cell matching and interpolation, to produce realistic simulations of the dynamics of cell differentiation from only static images. We also found that the distribution of cell shapes is hollow: most shapes are very different from the average shape. Finally, we show how the method can be used to model nuclear shape changes of human-induced pluripotent stem cells resulting from drug treatments.

Applications and Challenges of Machine Learning to Enable Realistic Cellular Simulations

In this perspective, we examine three key aspects of an end-to-end pipeline for realistic cellular simulations: reconstruction and segmentation of cellular structures; generation of cellular structures; and mesh generation, simulation, and data analysis. We highlight some of the relevant prior work in these distinct but overlapping areas, with a particular emphasis on current use of machine learning technologies, as well as on future opportunities.

Capturing Single-Cell Phenotypic Variation via Unsupervised Representation Learning

We propose a novel variational autoencoder (VAE) framework for learning representations of cell images for the domain of image-based profiling, important for new therapeutic discovery. Previously, generative adversarial network-based (GAN) approaches were proposed to enable biologists to visualize structural variations in cells that drive differences in populations. However, while the images were realistic, they did not provide direct reconstructions from representations, and their performance in downstream analysis was poor. We address these limitations in our approach by adding an adversarial-driven similarity constraint applied to the standard VAE framework, and a progressive training procedure that allows higher quality reconstructions than standard VAE's. The proposed models improve classification accuracy by 22% (to 90%) compared to the best reported GAN model, making it competitive with other models that have higher quality representations, but lack the ability to synthesize images. This provides researchers a new tool to match cellular phenotypes effectively, and also to gain better insight into cellular structure variations that are driving differences between populations of cells.