PhenoRipper: software for rapidly profiling microscopy images

High-Content Image-Based Screening and Deep Learning for the Detection of Anti-Inflammatory Drug Leads

We developed a high-content image-based screen that utilizes the pro-inflammatory stimulus lipopolysaccharide (LPS) and murine macrophages (RAW264.7) with the goal of enabling the identification of novel anti-inflammatory lead compounds. We screened 2,259 bioactive compounds with annotated mechanisms of action (MOA) to identify compounds that block the LPS-induced phenotype in macrophages. We utilized a set of seven fluorescence microscopy probes to generate images that were used to train and optimize a deep neural network classifier to distinguish between unstimulated and LPS-stimulated macrophages. The top hits from the deep learning classifier were validated using a linear classifier trained on individual cells and subsequently investigated in a multiplexed cytokine secretion assay. All 12 hits significantly modulated the expression of at least one cytokine upon LPS stimulation. Seven of these were allosteric inhibitors of the mitogen-activated protein kinase kinase (MEK1/2) and showed similar effects on cytokine expression. This deep learning morphological assay identified compounds that modulate the innate immune response to LPS and may aid in identifying new anti-inflammatory drug leads.

Statistical Analysis of Multiplex Immunofluorescence and Immunohistochemistry Imaging Data

Advances in multiplexed single-cell immunofluorescence (mIF) and multiplex immunohistochemistry (mIHC) imaging technologies have enabled the analysis of cell-to-cell spatial relationships that promise to revolutionize our understanding of tissue-based diseases and autoimmune disorders. Multiplex images are collected as multichannel TIFF files; then denoised, segmented to identify cells and nuclei, normalized across slides with protein markers to correct for batch effects, and phenotyped; and then tissue composition and spatial context at the cellular level are analyzed. This chapter discusses methods and software infrastructure for image processing and statistical analysis of mIF/mIHC data.

A comprehensive review of computational and image analysis techniques for quantitative evaluation of striated muscle tissue architecture

Unbiased evaluation of morphology is crucial to understanding development, mechanics, and pathology of striated muscle tissues. Indeed, the ability of striated muscles to contract and the strength of their contraction is dependent on their tissue-, cellular-, and cytoskeletal-level organization. Accordingly, the study of striated muscles often requires imaging and assessing aspects of their architecture at multiple different spatial scales. While an expert may be able to qualitatively appraise tissues, it is imperative to have robust, repeatable tools to quantify striated myocyte morphology and behavior that can be used to compare across different labs and experiments. There has been a recent effort to define the criteria used by experts to evaluate striated myocyte architecture. In this review, we will describe metrics that have been developed to summarize distinct aspects of striated muscle architecture in multiple different tissues, imaged with various modalities. Additionally, we will provide an overview of metrics and image processing software that needs to be developed. Importantly to any lab working on striated muscle platforms, characterization of striated myocyte morphology using the image processing pipelines discussed in this review can be used to quantitatively evaluate striated muscle tissues and contribute to a robust understanding of the development and mechanics of striated muscles.

Recent Microscopy Advances and the Applications to Huntington’s Disease Research

 Huntingtin is a 3144 amino acid protein defined as a scaffold protein with many intracellular locations that suggest functions in these compartments. Expansion of the CAG DNA tract in the huntingtin first exon is the cause of Huntington’s disease. An important tool in understanding the biological functions of huntingtin is molecular imaging at the single-cell level by microscopy and nanoscopy. The evolution of these technologies has accelerated since the Nobel Prize in Chemistry was awarded in 2014 for super-resolution nanoscopy. We are in a new era of light imaging at the single-cell level, not just for protein location, but also for protein conformation and biochemical function. Large-scale microscopy-based screening is also being accelerated by a coincident development of machine-based learning that offers a framework for truly unbiased data acquisition and analysis at very large scales. This review will summarize the newest technologies in light, electron, and atomic force microscopy in the context of unique challenges with huntingtin cell biology and biochemistry.

Functional characterization of variants of unknown significance in a spinocerebellar ataxia patient using an unsupervised machine learning pipeline

CAG-expanded ATXN7 has been previously defined in the pathogenesis of spinocerebellar ataxia type 7 (SCA7), a polyglutamine expansion autosomal dominant cerebellar ataxia. Pathology in SCA7 occurs as a result of a CAG triplet repeat expansion in excess of 37 in the first exon of ATXN7, which encodes ataxin-7. SCA7 presents clinically with spinocerebellar ataxia and cone-rod dystrophy. Here, we present a novel spinocerebellar ataxia variant occurring in a patient with mutations in both ATXN7 and TOP1MT, which encodes mitochondrial topoisomerase I (top1mt). Using machine-guided, unbiased microscopy image analysis, we demonstrate alterations in ataxin-7 subcellular localization, and through high-fidelity measurements of cellular respiration, bioenergetic defects in association with top1mt mutations. We identify ataxin-7 Q35P and top1mt R111W as deleterious mutations, potentially contributing to disease states. We recapitulate our mutations through Drosophila genetic models. Our work provides important insight into the cellular biology of ataxin-7 and top1mt and offers insight into the pathogenesis of spinocerebellar ataxia applicable to multiple subtypes of the illness. Moreover, our study demonstrates an effective pipeline for the characterization of previously unreported genetic variants at the level of cell biology.

Computational analyses of mechanism of action (MoA): data, methods and integration

The elucidation of a compound's Mechanism of Action (MoA) is a challenging task in the drug discovery process, but it is important in order to rationalise phenotypic findings and to anticipate potential side-effects. Bioinformatic approaches, advances in machine learning techniques and the increasing deposition of high-throughput data in public databases have significantly contributed to recent advances in the field, but it is not straightforward to decide which data and methods are most suitable to use in a given case. In this review, we focus on these methods and data and their applications in generating MoA hypotheses for subsequent experimental validation. We discuss compound-specific data such as -omics, cell morphology and bioactivity data, as well as commonly used supplementary prior knowledge such as network and pathway data, and provide information on databases where this data can be accessed. In terms of methodologies, we discuss both well-established methods (connectivity mapping, pathway enrichment) as well as more developing methods (neural networks and multi-omics integration). Finally, we review case studies where the MoA of a compound was successfully suggested from computational analysis by incorporating multiple data modalities and/or methodologies. Our aim for this review is to provide researchers with insights into the benefits and drawbacks of both the data and methods in terms of level of understanding, biases and interpretation - and to highlight future avenues of investigation which we foresee will improve the field of MoA elucidation, including greater public access to -omics data and methodologies which are capable of data integration.

A deep generative model of 3D single-cell organization

We introduce a framework for end-to-end integrative modeling of 3D single-cell multi-channel fluorescent image data of diverse subcellular structures. We employ stacked conditional β-variational autoencoders to first learn a latent representation of cell morphology, and then learn a latent representation of subcellular structure localization which is conditioned on the learned cell morphology. Our model is flexible and can be trained on images of arbitrary subcellular structures and at varying degrees of sparsity and reconstruction fidelity. We train our full model on 3D cell image data and explore design trade-offs in the 2D setting. Once trained, our model can be used to predict plausible locations of structures in cells where these structures were not imaged. The trained model can also be used to quantify the variation in the location of subcellular structures by generating plausible instantiations of each structure in arbitrary cell geometries. We apply our trained model to a small drug perturbation screen to demonstrate its applicability to new data. We show how the latent representations of drugged cells differ from unperturbed cells as expected by on-target effects of the drugs.

Quantitative Evaluation of Cardiac Cell Interactions and Responses to Cyclic Strain

The heart has a dynamic mechanical environment contributed by its unique cellular composition and the resultant complex tissue structure. In pathological heart tissue, both the mechanics and cell composition can change and influence each other. As a result, the interplay between the cell phenotype and mechanical stimulation needs to be considered to understand the biophysical cell interactions and organization in healthy and diseased myocardium. In this work, we hypothesized that the overall tissue organization is controlled by varying densities of cardiomyocytes and fibroblasts in the heart. In order to test this hypothesis, we utilized a combination of mechanical strain, co-cultures of different cell types, and inhibitory drugs that block intercellular junction formation. To accomplish this, an image analysis pipeline was developed to automatically measure cell type-specific organization relative to the stretch direction. The results indicated that cardiac cell type-specific densities influence the overall organization of heart tissue such that it is possible to model healthy and fibrotic heart tissue in vitro. This study provides insight into how to mimic the dynamic mechanical environment of the heart in engineered tissue as well as providing valuable information about the process of cardiac remodeling and repair in diseased hearts.

Investigation of Different Free Image Analysis Software for High-Throughput Droplet Detection

Droplet microfluidics has revealed innovative strategies in biology and chemistry. This advancement has delivered novel quantification methods, such as droplet digital polymerase chain reaction (ddPCR) and an antibiotic heteroresistance analysis tool. For droplet analysis, researchers often use image-based detection techniques. Unfortunately, the analysis of images may require specific tools or programming skills to produce the expected results. In order to address the issue, we explore the potential use of standalone freely available software to perform image-based droplet detection. We select the four most popular software and classify them into rule-based and machine learning-based types after assessing the software's modules. We test and evaluate the software's (i) ability to detect droplets, (ii) accuracy and precision, and (iii) overall components and supporting material. In our experimental setting, we find that the rule-based type of software is better suited for image-based droplet detection. The rule-based type of software also has a simpler workflow or pipeline, especially aimed for non-experienced users. In our case, CellProfiler (CP) offers the most user-friendly experience for both single image and batch processing analyses.

Emerging machine learning approaches to phenotyping cellular motility and morphodynamics

Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.

Image-based cell phenotyping with deep learning

A cell's phenotype is the culmination of several cellular processes through a complex network of molecular interactions that ultimately result in a unique morphological signature. Visual cell phenotyping is the characterization and quantification of these observable cellular traits in images. Recently, cellular phenotyping has undergone a massive overhaul in terms of scale, resolution, and throughput, which is attributable to advances across electronic, optical, and chemical technologies for imaging cells. Coupled with the rapid acceleration of deep learning-based computational tools, these advances have opened up new avenues for innovation across a wide variety of high-throughput cell biology applications. Here, we review applications wherein deep learning is powering the recognition, profiling, and prediction of visual phenotypes to answer important biological questions. As the complexity and scale of imaging assays increase, deep learning offers computational solutions to elucidate the details of previously unexplored cellular phenotypes.

High-Throughput Methods in the Discovery and Study of Biomaterials and Materiobiology

The complex interaction of cells with biomaterials (i.e., materiobiology) plays an increasingly pivotal role in the development of novel implants, biomedical devices, and tissue engineering scaffolds to treat diseases, aid in the restoration of bodily functions, construct healthy tissues, or regenerate diseased ones. However, the conventional approaches are incapable of screening the huge amount of potential material parameter combinations to identify the optimal cell responses and involve a combination of serendipity and many series of trial-and-error experiments. For advanced tissue engineering and regenerative medicine, highly efficient and complex bioanalysis platforms are expected to explore the complex interaction of cells with biomaterials using combinatorial approaches that offer desired complex microenvironments during healing, development, and homeostasis. In this review, we first introduce materiobiology and its high-throughput screening (HTS). Then we present an in-depth of the recent progress of 2D/3D HTS platforms (i.e., gradient and microarray) in the principle, preparation, screening for materiobiology, and combination with other advanced technologies. The Compendium for Biomaterial Transcriptomics and high content imaging, computational simulations, and their translation toward commercial and clinical uses are highlighted. In the final section, current challenges and future perspectives are discussed. High-throughput experimentation within the field of materiobiology enables the elucidation of the relationships between biomaterial properties and biological behavior and thereby serves as a potential tool for accelerating the development of high-performance biomaterials.

Rapid 3D phenotypic analysis of neurons and organoids using data-driven cell segmentation-free machine learning

Phenotypic profiling of large three-dimensional microscopy data sets has not been widely adopted due to the challenges posed by cell segmentation and feature selection. The computational demands of automated processing further limit analysis of hard-to-segment images such as of neurons and organoids. Here we describe a comprehensive shallow-learning framework for automated quantitative phenotyping of three-dimensional (3D) image data using unsupervised data-driven voxel-based feature learning, which enables computationally facile classification, clustering and advanced data visualization. We demonstrate the analysis potential on complex 3D images by investigating the phenotypic alterations of: neurons in response to apoptosis-inducing treatments and morphogenesis for oncogene-expressing human mammary gland acinar organoids. Our novel implementation of image analysis algorithms called Phindr3D allowed rapid implementation of data-driven voxel-based feature learning into 3D high content analysis (HCA) operations and constitutes a major practical advance as the computed assignments represent the biology while preserving the heterogeneity of the underlying data. Phindr3D is provided as Matlab code and as a stand-alone program (https://github.com/DWALab/Phindr3D).

Fluorescence microscopy is a fundamental technology for cell biology. However, unbiased quantitative phenotypic analysis of microscopy images of cells grown in 3D organoids or in dense culture conditions in large enough numbers to reach statistical clarity remains a fundamental challenge. Here, we report that using data-driven voxel-based features and machine learning it is possible to analyze complex 3D image data without compressing them to 2D, identifying individual cells or using computationally intensive deep learning techniques. Further, we present methods for analyzing this data by classification or clustering. Together these techniques provide the means for facile discovery and interpretation of meaningful patterns in a high dimensional feature space without complex image processing and prior knowledge or assumptions about the feature space. Our method enables novel opportunities for rapid large-scale multivariate phenotypic microscopy image analysis in 3D using a standard desktop computer.

High-Content Imaging to Phenotype Human Primary and iPSC-Derived Cells

Increasingly powerful microscopy, liquid handling, and computational techniques have enabled cell imaging in high throughput. Microscopy images are quantified using high-content analysis platforms linking object features to cell behavior. This can be attempted on physiologically relevant cell models, including stem cells and primary cells, in complex environments, and conceivably in the presence of perturbations. Recently, substantial focus has been devoted to cell profiling for cell therapy, assays for drug discovery or biomarker identification for clinical decision-making protocols, bringing this wealth of information into translational applications. In this chapter, we focus on two protocols enabling to (1) benchmark human cells, in particular human endothelial cells as a case study and (2) extract cells from blood for follow-up experiments including image-based drug testing. We also present concepts of high-content imaging and discuss the benefits and challenges, with the aim of enabling readers to tailor existing pipelines and bring such approaches closer to translational research and the clinic.

Image-based high-content screening in drug discovery

While target-based drug discovery strategies rely on the precise knowledge of the identity and function of the drug targets, phenotypic drug discovery (PDD) approaches allow the identification of novel drugs based on knowledge of a distinct phenotype. Image-based high-content screening (HCS) is a potent PDD strategy that characterizes small-molecule effects through the quantification of features that depict cellular changes among or within cell populations, thereby generating valuable data sets for subsequent data analysis. However, these data can be complex, making image analysis from large HCS campaigns challenging. Technological advances in image acquisition, processing, and analysis as well as machine-learning (ML) approaches for the analysis of multidimensional data sets have rendered HCS as a viable technology for small-molecule drug discovery. Here, we discuss HCS concepts, current workflows as well as opportunities and challenges of image-based phenotypic screening and data analysis.

Computational modeling of cellular structures using conditional deep generative networks

MOTIVATION

Cellular function is closely related to the localizations of its sub-structures. It is, however, challenging to experimentally label all sub-cellular structures simultaneously in the same cell. This raises the need of building a computational model to learn the relationships among these sub-cellular structures and use reference structures to infer the localizations of other structures.

RESULTS

We formulate such a task as a conditional image generation problem and propose to use conditional generative adversarial networks for tackling it. We employ an encoder-decoder network as the generator and propose to use skip connections between the encoder and decoder to provide spatial information to the decoder. To incorporate the conditional information in a variety of different ways, we develop three different types of skip connections, known as the self-gated connection, encoder-gated connection and label-gated connection. The proposed skip connections are built based on the conditional information using gating mechanisms. By learning a gating function, the network is able to control what information should be passed through the skip connections from the encoder to the decoder. Since the gate parameters are also learned automatically, we expect that only useful spatial information is transmitted to the decoder to help image generation. We perform both qualitative and quantitative evaluations to assess the effectiveness of our proposed approaches. Experimental results show that our cGAN-based approaches have the ability to generate the desired sub-cellular structures correctly. Our results also demonstrate that the proposed approaches outperform the existing approach based on adversarial auto-encoders, and the new skip connections lead to improved performance. In addition, the localizations of generated sub-cellular structures by our approaches are consistent with observations in biological experiments.

AVAILABILITY AND IMPLEMENTATION

The source code and more results are available at https://github.com/divelab/cgan/.

An image-based data-driven analysis of cellular architecture in a developing tissue

Quantitative microscopy is becoming increasingly crucial in efforts to disentangle the complexity of organogenesis, yet adoption of the potent new toolbox provided by modern data science has been slow, primarily because it is often not directly applicable to developmental imaging data. We tackle this issue with a newly developed algorithm that uses point cloud-based morphometry to unpack the rich information encoded in 3D image data into a straightforward numerical representation. This enabled us to employ data science tools, including machine learning, to analyze and integrate cell morphology, intracellular organization, gene expression and annotated contextual knowledge. We apply these techniques to construct and explore a quantitative atlas of cellular architecture for the zebrafish posterior lateral line primordium, an experimentally tractable model of complex self-organized organogenesis. In doing so, we are able to retrieve both previously established and novel biologically relevant patterns, demonstrating the potential of our data-driven approach.

Phenotyping analysis of maize stem using micro-computed tomography at the elongation and tasseling stages

BACKGROUND

Micro-computed tomography (μCT) bring a new opportunity to accurately quantify micro phenotypic traits of maize stem, also provide comparable benchmark to evaluate its dynamic development at the different growth stages. The progressive accumulation of stem biomass brings manifest structure changes of maize stem and vascular bundles, which are closely related with maize varietal characteristics and growth stages. Thus, micro-phenotyping (μPhenotyping) of maize stems is not only valuable to evaluate bio-mechanics and water-transport performance of maize, but also yield growth-based traits for quantitative traits loci (QTL) and functional genes location in molecular breeding.

RESULT

In this study, maize stems of 20 maize cultivars and two growth stages were imaged using μCT scanning technology. According to the observable differences of maize stems from the elongation and tasseling stages, function zones of maize stem were firstly defined to describe the substance accumulation of maize stems. And then a set of image-based μPhenotyping pipelines were implemented to quantify maize stem and vascular bundles at the two stages. The coefficient of determination (R2) of counting vascular bundles was higher than 0.95. Based on the uniform contour representation, intensity-related, geometry-related and distribution-related traits of vascular bundles were respectively evaluated in function zones and structure layers. And growth-related traits of the slice, epidermis, periphery and inner zones were also used to describe the dynamic growth of maize stem. Statistical analysis demonstrated the presented method was suitable to the phenotyping analysis of maize stem for multiple growth stages.

CONCLUSIONS

The novel descriptors of function zones provide effective phenotypic references to quantify the differences between growth stages; and the detection and identification of vascular bundles based on function zones are more robust to determine the adaptive image analysis pipeline. Developing robust and effective image-based phenotyping method to assess the traits of stem and vascular bundles, is highly relevant for understanding the relationship between maize phenomics and genomics.

A novel rind puncture technique to measure rind thickness and diameter in plant stalks

BACKGROUND

Measurements of rind and culm thickness and stem radius/diameter are important to biomechanical, ecological and physiological plant studies. However, many methods of measuring rind thickness and diameter are labor intensive and induce plant fatality. A novel rind puncture methodology for obtaining measurements of rind thickness and diameter has been developed. The suitability of the new method for implementation in plant studies is presented.

RESULTS

The novel rind puncture technique was used to obtain measurements of rind thickness and diameter for samples of Poison Hemlock (Conium maculatum). The rind puncture measurements were strongly correlated with caliper measurements (R² > 0.97) and photographic image analysis measurements (R² > 0.84). The capacity for high throughput measurements using the rind puncture technique was determined to exceed that of caliper measurements and image analysis techniques.

CONCLUSIONS

The rind puncture technique shows promise as a high throughput method for determining rind thickness and diameter as it is cost effective and non-lethal. The authors are currently working to develop a custom handheld apparatus to allow the novel rind puncture method to be used in field work. High throughput field-based measurements of rind thickness and diameter are needed to help address the problem of stalk lodging (failure of grain crops to remain upright until harvest).

Phenotypic Variation Between Stromal Cells Differentially Impacts Engineered Cardiac Tissue Function

IMPACT STATEMENT

Understanding the relationship between parenchymal and supporting cell populations is paramount to recapitulate the multicellular complexity of native tissues. Incorporation of stromal cells is widely recognized to be necessary for the stable formation of stem cell-derived cardiac tissues; yet, the types of stromal cells used have varied widely. This study systematically characterized several stromal populations and found that stromal phenotype and morphology was highly variable depending on cell source and exerted differential impacts on cardiac tissue function and induced pluripotent stem cell-cardiomyocyte phenotype. Therefore, the choice of supporting stromal population can differentially impact the phenotypic or functional performance of engineered cardiac tissues.

A multi-modal data resource for investigating topographic heterogeneity in patient-derived xenograft tumors

Patient-derived xenografts (PDXs) are an essential pre-clinical resource for investigating tumor biology. However, cellular heterogeneity within and across PDX tumors can strongly impact the interpretation of PDX studies. Here, we generated a multi-modal, large-scale dataset to investigate PDX heterogeneity in metastatic colorectal cancer (CRC) across tumor models, spatial scales and genomic, transcriptomic, proteomic and imaging assay modalities. To showcase this dataset, we present analysis to assess sources of PDX variation, including anatomical orientation within the implanted tumor, mouse contribution, and differences between replicate PDX tumors. A unique aspect of our dataset is deep characterization of intra-tumor heterogeneity via immunofluorescence imaging, which enables investigation of variation across multiple spatial scales, from subcellular to whole tumor levels. Our study provides a benchmark data resource to investigate PDX models of metastatic CRC and serves as a template for future, quantitative investigations of spatial heterogeneity within and across PDX tumor models.

Phenotypic Image Analysis Software Tools for Exploring and Understanding Big Image Data from Cell-Based Assays

Phenotypic image analysis is the task of recognizing variations in cell properties using microscopic image data. These variations, produced through a complex web of interactions between genes and the environment, may hold the key to uncover important biological phenomena or to understand the response to a drug candidate. Today, phenotypic analysis is rarely performed completely by hand. The abundance of high-dimensional image data produced by modern high-throughput microscopes necessitates computational solutions. Over the past decade, a number of software tools have been developed to address this need. They use statistical learning methods to infer relationships between a cell's phenotype and data from the image. In this review, we examine the strengths and weaknesses of non-commercial phenotypic image analysis software, cover recent developments in the field, identify challenges, and give a perspective on future possibilities.

Considerations for an In Vitro, Cell-Based Testing Platform for Detection of Drug-Induced Inotropic Effects in Early Drug Development. Part 2: Designing and Fabricating Microsystems for Assaying Cardiac Contractility With Physiological Relevance Using Human iPSC-Cardiomyocytes

Contractility of the myocardium engines the pumping function of the heart and is enabled by the collective contractile activity of its muscle cells: cardiomyocytes. The effects of drugs on the contractility of human cardiomyocytes in vitro can provide mechanistic insight that can support the prediction of clinical cardiac drug effects early in drug development. Cardiomyocytes differentiated from human-induced pluripotent stem cells have high potential for overcoming the current limitations of contractility assays because they attach easily to extracellular materials and last long in culture, while having human- and patient-specific properties. Under these conditions, contractility measurements can be non-destructive and minimally invasive, which allow assaying sub-chronic effects of drugs. For this purpose, the function of cardiomyocytes in vitro must reflect physiological settings, which is not observed in cultured cardiomyocytes derived from induced pluripotent stem cells because of the fetal-like properties of their contractile machinery. Primary cardiomyocytes or tissues of human origin fully represent physiological cellular properties, but are not easily available, do not last long in culture, and do not attach easily to force sensors or mechanical actuators. Microengineered cellular systems with a more mature contractile function have been developed in the last 5 years to overcome this limitation of stem cell-derived cardiomyocytes, while simultaneously measuring contractile endpoints with integrated force sensors/actuators and image-based techniques. Known effects of engineered microenvironments on the maturity of cardiomyocyte contractility have also been discovered in the development of these systems. Based on these discoveries, we review here design criteria of microengineered platforms of cardiomyocytes derived from pluripotent stem cells for measuring contractility with higher physiological relevance. These criteria involve the use of electromechanical, chemical and morphological cues, co-culture of different cell types, and three-dimensional cellular microenvironments. We further discuss the use and the current challenges for developing and improving these novel technologies for predicting clinical effects of drugs based on contractility measurements with cardiomyocytes differentiated from induced pluripotent stem cells. Future research should establish contexts of use in drug development for novel contractility assays with stem cell-derived cardiomyocytes.

Biomedical Image Processing with Containers and Deep Learning: An Automated Analysis Pipeline: Data architecture, artificial intelligence, automated processing, containerization, and clusters orchestration ease the transition from data acquisition to insights in medium-to-large datasets

Here, a streamlined, scalable, laboratory approach is discussed that enables medium-to-large dataset analysis. The presented approach combines data management, artificial intelligence, containerization, cluster orchestration, and quality control in a unified analytic pipeline. The unique combination of these individual building blocks creates a new and powerful analysis approach that can readily be applied to medium-to-large datasets by researchers to accelerate the pace of research. The proposed framework is applied to a project that counts the number of plasmonic nanoparticles bound to peripheral blood mononuclear cells in dark-field microscopy images. By using the techniques presented in this article, the images are automatically processed overnight, without user interaction, streamlining the path from experiment to conclusions.

Tools to reverse-engineer multicellular systems: case studies using the fruit fly

Reverse-engineering how complex multicellular systems develop and function is a grand challenge for systems bioengineers. This challenge has motivated the creation of a suite of bioengineering tools to develop increasingly quantitative descriptions of multicellular systems. Here, we survey a selection of these tools including microfluidic devices, imaging and computer vision techniques. We provide a selected overview of the emerging cross-talk between engineering methods and quantitative investigations within developmental biology. In particular, the review highlights selected recent examples from the Drosophila system, an excellent platform for understanding the interplay between genetics and biophysics. In sum, the integrative approaches that combine multiple advances in these fields are increasingly necessary to enable a deeper understanding of how to analyze both natural and synthetic multicellular systems.

A fully-automated, robust, and versatile algorithm for long-term budding yeast segmentation and tracking

Live cell time-lapse microscopy, a widely-used technique to study gene expression and protein dynamics in single cells, relies on segmentation and tracking of individual cells for data generation. The potential of the data that can be extracted from this technique is limited by the inability to accurately segment a large number of cells from such microscopy images and track them over long periods of time. Existing segmentation and tracking algorithms either require additional dyes or markers specific to segmentation or they are highly specific to one imaging condition and cell morphology and/or necessitate manual correction. Here we introduce a fully automated, fast and robust segmentation and tracking algorithm for budding yeast that overcomes these limitations. Full automatization is achieved through a novel automated seeding method, which first generates coarse seeds, then automatically fine-tunes cell boundaries using these seeds and automatically corrects segmentation mistakes. Our algorithm can accurately segment and track individual yeast cells without any specific dye or biomarker. Moreover, we show how existing channels devoted to a biological process of interest can be used to improve the segmentation. The algorithm is versatile in that it accurately segments not only cycling cells with smooth elliptical shapes, but also cells with arbitrary morphologies (e.g. sporulating and pheromone treated cells). In addition, the algorithm is independent of the specific imaging method (bright-field/phase) and objective used (40X/63X/100X). We validate our algorithm's performance on 9 cases each entailing a different imaging condition, objective magnification and/or cell morphology. Taken together, our algorithm presents a powerful segmentation and tracking tool that can be adapted to numerous budding yeast single-cell studies.

High-throughput micro-phenotyping measurements applied to assess stalk lodging in maize (Zea mays L.)

BACKGROUND

The biomechanical properties of maize stalks largely determine their lodging resistance, which affects crop yield per unit area. However, the quantitative and qualitative relationship between micro-phenotypes and the biomechanics of maize stalks is still under examined. In particular, the roles of the number, geometry, and distribution of vascular bundles of stalks in maize lodging resistance remain unclear. Research on these biomechanical properties will benefit from high-resolution micro-phenotypic image acquisition capabilities, which have been improved by modern X-ray imaging devices such as micro-CT and the development of micro-phenotyping analysis software. Hence, high-throughput image analysis and accurate quantification of anatomical phenotypes of stalks are necessary.

RESULTS

We have updated VesselParser version 1.0 to version 2.0 and have improved its performance, accuracy, and computation strategies. Anatomical characteristics of the second and third stalk internodes of the cultivars 'Jingke968' and 'Jingdan38' were analyzed using VesselParser 2.0. The relationships between lodging resistance and anatomical phenotypes of stalks between the two different maize varieties were investigated. The total area of vascular bundles in the peripheral layer, auxiliary axis diameter, and total area of vascular bundles were revealed to have the highest correlation with mechanical properties, and anatomical phenotypes of maize stalk were better predictors of mechanical properties than macro features observed optically from direct measurement, such as diameter and perimeter.

CONCLUSIONS

This study demonstrates the utility of VesselParser 2.0 in assessing stalk mechanical properties. The combination of anatomical phenotypes and mechanical behavior research provides unique insights into the problem of stalk lodging, showing that micro phenotypes of vascular bundles are good predictors of maize stalk mechanical properties that may be important indices for the evaluation and identification of the biomechanical properties to improve lodging resistance of future maize varieties.

Machine learning and image-based profiling in drug discovery

The increase in imaging throughput, new analytical frameworks and high-performance computational resources open new avenues for data-rich phenotypic profiling of small molecules in drug discovery. Image-based profiling assays assessing single-cell phenotypes have been used to explore mechanisms of action, target efficacy and toxicity of small molecules. Technological advances to generate large data sets together with new machine learning approaches for the analysis of high-dimensional profiling data create opportunities to improve many steps in drug discovery. In this review, we will discuss how recent studies applied machine learning approaches in functional profiling workflows with a focus on chemical genetics. While their utility in image-based screening and profiling is predictably evident, examples of novel insights beyond the status quo based on the applications of machine learning approaches are just beginning to emerge. To enable discoveries, future studies also need to develop methodologies that lower the entry barriers to high-throughput profiling experiments by streamlining image-based profiling assays and providing applications for advanced learning technologies such as easy to deploy deep neural networks.

N6-Furfuryladenine is protective in Huntington's disease models by signaling huntingtin phosphorylation

The huntingtin N17 domain is a modulator of mutant huntingtin toxicity and is hypophosphorylated in Huntington's disease (HD). We conducted high-content analysis to find compounds that could restore N17 phosphorylation. One lead compound from this screen was N6-furfuryladenine (N6FFA). N6FFA was protective in HD model neurons, and N6FFA treatment of an HD mouse model corrects HD phenotypes and eliminates cortical mutant huntingtin inclusions. We show that N6FFA restores N17 phosphorylation levels by being salvaged to a triphosphate form by adenine phosphoribosyltransferase (APRT) and used as a phosphate donor by casein kinase 2 (CK2). N6FFA is a naturally occurring product of oxidative DNA damage. Phosphorylated huntingtin functionally redistributes and colocalizes with CK2, APRT, and N6FFA DNA adducts at sites of induced DNA damage. We present a model in which this natural product compound is salvaged to provide a triphosphate substrate to signal huntingtin phosphorylation via CK2 during low-ATP stress under conditions of DNA damage, with protective effects in HD model systems.

Morphometrics of complex cell shapes: lobe contribution elliptic Fourier analysis (LOCO-EFA)

Quantifying cell morphology is fundamental to the statistical study of cell populations, and can help unravel mechanisms underlying cell and tissue morphogenesis. Current methods, however, require extensive human intervention, are highly parameter sensitive, or produce metrics that are difficult to interpret biologically. We therefore developed a method, lobe contribution elliptical Fourier analysis (LOCO-EFA), which generates from digitalised two-dimensional cell outlines meaningful descriptors that can be directly matched to morphological features. This is shown by studying well-defined geometric shapes as well as actual biological cells from plant and animal tissues. LOCO-EFA provides a tool to phenotype efficiently and objectively populations of cells, here demonstrated by applying it to the complex shaped pavement cells of Arabidopsis thaliana wild-type and speechless leaves, and Drosophila amnioserosa cells. To validate our method's applicability to large populations, we analysed computer-generated tissues. By controlling in silico cell shape, we explored the potential impact of cell packing on individual cell shape, quantifying through LOCO-EFA deviations between the specified shape of single cells in isolation and the resultant shape when they interact within a confluent tissue.

Inferring cell state by quantitative motility analysis reveals a dynamic state system and broken detailed balance

Cell populations display heterogeneous and dynamic phenotypic states at multiple scales. Similar to molecular features commonly used to explore cell heterogeneity, cell behavior is a rich phenotypic space that may allow for identification of relevant cell states. Inference of cell state from cell behavior across a time course may enable the investigation of dynamics of transitions between heterogeneous cell states, a task difficult to perform with destructive molecular observations. Cell motility is one such easily observed cell behavior with known biomedical relevance. To investigate heterogenous cell states and their dynamics through the lens of cell behavior, we developed Heteromotility, a software tool to extract quantitative motility features from timelapse cell images. In mouse embryonic fibroblasts (MEFs), myoblasts, and muscle stem cells (MuSCs), Heteromotility analysis identifies multiple motility phenotypes within the population. In all three systems, the motility state identity of individual cells is dynamic. Quantification of state transitions reveals that MuSCs undergoing activation transition through progressive motility states toward the myoblast phenotype. Transition rates during MuSC activation suggest non-linear kinetics. By probability flux analysis, we find that this MuSC motility state system breaks detailed balance, while the MEF and myoblast systems do not. Balanced behavior state transitions can be captured by equilibrium formalisms, while unbalanced switching between states violates equilibrium conditions and would require an external driving force. Our data indicate that the system regulating cell behavior can be decomposed into a set of attractor states which depend on the identity of the cell, together with a set of transitions between states. These results support a conceptual view of cell populations as dynamical systems, responding to inputs from signaling pathways and generating outputs in the form of state transitions and observable motile behaviors.

Quality Control for High-Throughput Imaging Experiments Using Machine Learning in Cellprofiler

Robust high-content screening of visual cellular phenotypes has been enabled by automated microscopy and quantitative image analysis. The identification and removal of common image-based aberrations is critical to the screening workflow. Out-of-focus images, debris, and auto-fluorescing samples can cause artifacts such as focus blur and image saturation, contaminating downstream analysis and impairing identification of subtle phenotypes. Here, we describe an automated quality control protocol implemented in validated open-source software, leveraging the suite of image-based measurements generated by CellProfiler and the machine-learning functionality of CellProfiler Analyst.

A deep learning and novelty detection framework for rapid phenotyping in high-content screening

Supervised machine learning is a powerful and widely used method for analyzing high-content screening data. Despite its accuracy, efficiency, and versatility, supervised machine learning has drawbacks, most notably its dependence on a priori knowledge of expected phenotypes and time-consuming classifier training. We provide a solution to these limitations with CellCognition Explorer, a generic novelty detection and deep learning framework. Application to several large-scale screening data sets on nuclear and mitotic cell morphologies demonstrates that CellCognition Explorer enables discovery of rare phenotypes without user training, which has broad implications for improved assay development in high-content screening.

Data-analysis strategies for image-based cell profiling

Image-based cell profiling is a high-throughput strategy for the quantification of phenotypic differences among a variety of cell populations. It paves the way to studying biological systems on a large scale by using chemical and genetic perturbations. The general workflow for this technology involves image acquisition with high-throughput microscopy systems and subsequent image processing and analysis. Here, we introduce the steps required to create high-quality image-based (i.e., morphological) profiles from a collection of microscopy images. We recommend techniques that have proven useful in each stage of the data analysis process, on the basis of the experience of 20 laboratories worldwide that are refining their image-based cell-profiling methodologies in pursuit of biological discovery. The recommended techniques cover alternatives that may suit various biological goals, experimental designs, and laboratories' preferences.

Epithelial Ovarian Cancer Diagnosis of Second-Harmonic Generation Images: A Semiautomatic Collagen Fibers Quantification Protocol

A vast number of human pathologic conditions are directly or indirectly related to tissular collagen structure remodeling. The nonlinear optical microscopy second-harmonic generation has become a powerful tool for imaging biological tissues with anisotropic hyperpolarized structures, such as collagen. During the past years, several quantification methods to analyze and evaluate these images have been developed. However, automated or semiautomated solutions are necessary to ensure objectivity and reproducibility of such analysis. This work describes automation and improvement methods for calculating the anisotropy (using fast Fourier transform analysis and the gray-level co-occurrence matrix). These were applied to analyze biopsy samples of human ovarian epithelial cancer at different stages of malignancy (mucinous, serous, mixed, and endometrial subtypes). The semiautomation procedure enabled us to design a diagnostic protocol that recognizes between healthy and pathologic tissues, as well as between different tumor types.

Machine learning and computer vision approaches for phenotypic profiling

Grys et al. review computer vision and machine-learning methods that have been applied to phenotypic profiling of image-based data. Descriptions are provided for segmentation, feature extraction, selection, and dimensionality reduction, as well as clustering, outlier detection, and classification of data.

With recent advances in high-throughput, automated microscopy, there has been an increased demand for effective computational strategies to analyze large-scale, image-based data. To this end, computer vision approaches have been applied to cell segmentation and feature extraction, whereas machine-learning approaches have been developed to aid in phenotypic classification and clustering of data acquired from biological images. Here, we provide an overview of the commonly used computer vision and machine-learning methods for generating and categorizing phenotypic profiles, highlighting the general biological utility of each approach.

Deep phenotyping unveils hidden traits and genetic relations in subtle mutants

Discovering mechanistic insights from phenotypic information is critical for the understanding of biological processes. For model organisms, unlike in cell culture, this is currently bottlenecked by the non-quantitative nature and perceptive biases of human observations, and the limited number of reporters that can be simultaneously incorporated in live animals. An additional challenge is that isogenic populations exhibit significant phenotypic heterogeneity. These difficulties limit genetic approaches to many biological questions. To overcome these bottlenecks, we developed tools to extract complex phenotypic traits from images of fluorescently labelled subcellular landmarks, using C. elegans synapses as a test case. By population-wide comparisons, we identified subtle but relevant differences inaccessible to subjective conceptualization. Furthermore, the models generated testable hypotheses of how individual alleles relate to known mechanisms or belong to new pathways. We show that our model not only recapitulates current knowledge in synaptic patterning but also identifies novel alleles overlooked by traditional methods.

Experimenter scoring of cellular imaging data can be biased. This study describes an automated and unbiased multidimensional phenotyping method that relies on machine learning and complex feature computation of imaging data, and identifies weak alleles affecting synapse morphology in live C. elegans.

Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes

In morphological profiling, quantitative data are extracted from microscopy images of cells to identify biologically relevant similarities and differences among samples based on these profiles. This protocol describes the design and execution of experiments using Cell Painting, which is a morphological profiling assay that multiplexes six fluorescent dyes, imaged in five channels, to reveal eight broadly relevant cellular components or organelles. Cells are plated in multiwell plates, perturbed with the treatments to be tested, stained, fixed, and imaged on a high-throughput microscope. Next, an automated image analysis software identifies individual cells and measures ∼1,500 morphological features (various measures of size, shape, texture, intensity, and so on) to produce a rich profile that is suitable for the detection of subtle phenotypes. Profiles of cell populations treated with different experimental perturbations can be compared to suit many goals, such as identifying the phenotypic impact of chemical or genetic perturbations, grouping compounds and/or genes into functional pathways, and identifying signatures of disease. Cell culture and image acquisition takes 2 weeks; feature extraction and data analysis take an additional 1-2 weeks.

CP-CHARM: segmentation-free image classification made accessible

Background

Automated classification using machine learning often relies on features derived from segmenting individual objects, which can be difficult to automate. WND-CHARM is a previously developed classification algorithm in which features are computed on the whole image, thereby avoiding the need for segmentation. The algorithm obtained encouraging results but requires considerable computational expertise to execute. Furthermore, some benchmark sets have been shown to be subject to confounding artifacts that overestimate classification accuracy.

Results

We developed CP-CHARM, a user-friendly image-based classification algorithm inspired by WND-CHARM in (i) its ability to capture a wide variety of morphological aspects of the image, and (ii) the absence of requirement for segmentation. In order to make such an image-based classification method easily accessible to the biological research community, CP-CHARM relies on the widely-used open-source image analysis software CellProfiler for feature extraction. To validate our method, we reproduced WND-CHARM’s results and ensured that CP-CHARM obtained comparable performance. We then successfully applied our approach on cell-based assay data and on tissue images. We designed these new training and test sets to reduce the effect of batch-related artifacts.

Conclusions

The proposed method preserves the strengths of WND-CHARM - it extracts a wide variety of morphological features directly on whole images thereby avoiding the need for cell segmentation, but additionally, it makes the methods easily accessible for researchers without computational expertise by implementing them as a CellProfiler pipeline. It has been demonstrated to perform well on a wide range of bioimage classification problems, including on new datasets that have been carefully selected and annotated to minimize batch effects. This provides for the first time a realistic and reliable assessment of the whole image classification strategy.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-016-0895-y) contains supplementary material, which is available to authorized users.

Label-free cell cycle analysis for high-throughput imaging flow cytometry

Imaging flow cytometry combines the high-throughput capabilities of conventional flow cytometry with single-cell imaging. Here we demonstrate label-free prediction of DNA content and quantification of the mitotic cell cycle phases by applying supervised machine learning to morphological features extracted from brightfield and the typically ignored darkfield images of cells from an imaging flow cytometer. This method facilitates non-destructive monitoring of cells avoiding potentially confounding effects of fluorescent stains while maximizing available fluorescence channels. The method is effective in cell cycle analysis for mammalian cells, both fixed and live, and accurately assesses the impact of a cell cycle mitotic phase blocking agent. As the same method is effective in predicting the DNA content of fission yeast, it is likely to have a broad application to other cell types.

Imaging flow cytometry enables high-throughput acquisition of fluorescence, brightfield and darkfield images of biological cells. Here, Blasi et al. demonstrate that applying machine learning algorithms on brightfield and darkfield images can detect cellular phenotypes without the need for fluorescent stains, enabling label-free assays.

Use of a Machine Learning-Based High Content Analysis Approach to Identify Photoreceptor Neurite Promoting Molecules

High content analysis (HCA) has become a leading methodology in phenotypic drug discovery efforts. Typical HCA workflows include imaging cells using an automated microscope and analyzing the data using algorithms designed to quantify one or more specific phenotypes of interest. Due to the richness of high content data, unappreciated phenotypic changes may be discovered in existing image sets using interactive machine-learning based software systems. Primary postnatal day four retinal cells from the photoreceptor (PR) labeled QRX-EGFP reporter mice were isolated, seeded, treated with a set of 234 profiled kinase inhibitors and then cultured for 1 week. The cells were imaged with an Acumen plate-based laser cytometer to determine the number and intensity of GFP-expressing, i.e. PR, cells. Wells displaying intensities and counts above threshold values of interest were re-imaged at a higher resolution with an INCell2000 automated microscope. The images were analyzed with an open source HCA analysis tool, PhenoRipper (Rajaram et al., Nat Methods 9:635-637, 2012), to identify the high GFP-inducing treatments that additionally resulted in diverse phenotypes compared to the vehicle control samples. The pyrimidinopyrimidone kinase inhibitor CHEMBL-1766490, a pan kinase inhibitor whose major known targets are p38α and the Src family member lck, was identified as an inducer of photoreceptor neuritogenesis by using the open-source HCA program PhenoRipper. This finding was corroborated using a cell-based method of image analysis that measures quantitative differences in the mean neurite length in GFP expressing cells. Interacting with data using machine learning algorithms may complement traditional HCA approaches by leading to the discovery of small molecule-induced cellular phenotypes in addition to those upon which the investigator is initially focusing.

Slide Set: Reproducible image analysis and batch processing with ImageJ

Most imaging studies in the biological sciences rely on analyses that are relatively simple. However, manual repetition of analysis tasks across multiple regions in many images can complicate even the simplest analysis, making record keeping difficult, increasing the potential for error, and limiting reproducibility. While fully automated solutions are necessary for very large data sets, they are sometimes impractical for the small- and medium-sized data sets common in biology. Here we present the Slide Set plugin for ImageJ, which provides a framework for reproducible image analysis and batch processing. Slide Set organizes data into tables, associating image files with regions of interest and other relevant information. Analysis commands are automatically repeated over each image in the data set, and multiple commands can be chained together for more complex analysis tasks. All analysis parameters are saved, ensuring transparency and reproducibility. Slide Set includes a variety of built-in analysis commands and can be easily extended to automate other ImageJ plugins, reducing the manual repetition of image analysis without the set-up effort or programming expertise required for a fully automated solution.

Studying Cellular Signal Transduction with OMIC Technologies

In the gulf between genotype and phenotype exists proteins and, in particular, protein signal transduction systems. These systems use a relatively limited parts list to respond to a much longer list of extracellular, environmental, and/or mechanical cues with rapidity and specificity. Most signaling networks function in a highly non-linear and often contextual manner. Furthermore, these processes occur dynamically across space and time. Because of these complexities, systems and "OMIC" approaches are essential for the study of signal transduction. One challenge in using OMIC-scale approaches to study signaling is that the "signal" can take different forms in different situations. Signals are encoded in diverse ways such as protein-protein interactions, enzyme activities, localizations, or post-translational modifications to proteins. Furthermore, in some cases, signals may be encoded only in the dynamics, duration, or rates of change of these features. Accordingly, systems-level analyses of signaling may need to integrate multiple experimental and/or computational approaches. As the field has progressed, the non-triviality of integrating experimental and computational analyses has become apparent. Successful use of OMIC methods to study signaling will require the "right" experiments and the "right" modeling approaches, and it is critical to consider both in the design phase of the project. In this review, we discuss common OMIC and modeling approaches for studying signaling, emphasizing the philosophical and practical considerations for effectively merging these two types of approaches to maximize the probability of obtaining reliable and novel insights into signaling biology.

CAST: An automated segmentation and tracking tool for the analysis of transcriptional kinetics from single-cell time-lapse recordings

Fluorescence and bioluminescence time-lapse imaging allows to investigate a vast range of cellular processes at single-cell or even subcellular resolution. In particular, time-lapse imaging can provide uniquely detailed information on the fine kinetics of transcription, as well as on biological oscillations such as the circadian and cell cycles. However, we face a paucity of automated methods to quantify time-lapse imaging data with single-cell precision, notably throughout multiple cell cycles. We developed CAST (Cell Automated Segmentation and Tracking platform) to automatically and robustly detect the position and size of cells or nuclei, quantify the corresponding light signals, while taking into account both cell divisions (lineage tracking) and migration events. We present here how CAST analyzes bioluminescence data from a short-lived transcriptional luciferase reporter. However, our flexible and modular implementation makes it easily adaptable to a wide variety of time-lapse recordings. We exemplify how CAST efficiently quantifies single-cell gene expression over multiple cell cycles using mouse NIH3T3 culture cells with a luminescence expression driven by the Bmal1 promoter, a central gene of the circadian oscillator. We further illustrate how such data can be used to quantify transcriptional bursting in conditions of lengthened circadian period, revealing thereby remarkably similar bursting signature compared to the endogenous circadian condition despite marked period lengthening. In summary, we establish CAST as novel tool for the efficient segmentation, signal quantification, and tracking of time-lapse images from mammalian cell culture.

Ultra high content image analysis and phenotype profiling of 3D cultured micro-tissues

In many situations, 3D cell cultures mimic the natural organization of tissues more closely than 2D cultures. Conventional methods for phenotyping such 3D cultures use either single or multiple simple parameters based on morphology and fluorescence staining intensity. However, due to their simplicity many details are not taken into account which limits system-level study of phenotype characteristics. Here, we have developed a new image analysis platform to automatically profile 3D cell phenotypes with 598 parameters including morphology, topology, and texture parameters such as wavelet and image moments. As proof of concept, we analyzed mouse breast cancer cells (4T1 cells) in a 384-well plate format following exposure to a diverse set of compounds at different concentrations. The result showed concentration dependent phenotypic trajectories for different biologically active compounds that could be used to classify compounds based on their biological target. To demonstrate the wider applicability of our method, we analyzed the phenotypes of a collection of 44 human breast cancer cell lines cultured in 3D and showed that our method correctly distinguished basal-A, basal-B, luminal and ERBB2+ cell lines in a supervised nearest neighbor classification method.