Machine learning in cell biology - teaching computers to recognize phenotypes

Machine learning-enabled multiplexed microfluidic sensors

High-throughput, cost-effective, and portable devices can enhance the performance of point-of-care tests. Such devices are able to acquire images from samples at a high rate in combination with microfluidic chips in point-of-care applications. However, interpreting and analyzing the large amount of acquired data is not only a labor-intensive and time-consuming process, but also prone to the bias of the user and low accuracy. Integrating machine learning (ML) with the image acquisition capability of smartphones as well as increasing computing power could address the need for high-throughput, accurate, and automatized detection, data processing, and quantification of results. Here, ML-supported diagnostic technologies are presented. These technologies include quantification of colorimetric tests, classification of biological samples (cells and sperms), soft sensors, assay type detection, and recognition of the fluid properties. Challenges regarding the implementation of ML methods, including the required number of data points, image acquisition prerequisites, and execution of data-limited experiments are also discussed.

Functional genomics, genetic risk profiling and cell phenotypes in neurodegenerative disease

Human genetics provides unbiased insights into the causes of human disease, which can be used to create a foundation for effective ways to more accurately diagnose patients, stratify patients for more successful clinical trials, discover and develop new therapies, and ultimately help patients choose the safest and most promising therapeutic option based on their risk profile. But the process for translating basic observations from human genetics studies into pathogenic disease mechanisms and treatments is laborious and complex, and this challenge has particularly slowed the development of interventions for neurodegenerative disease. In this review, we discuss the many steps in the process, the important considerations at each stage, and some of the latest tools and technologies that are available to help investigators translate insights from human genetics into diagnostic and therapeutic strategies that will lead to the sort of advances in clinical care that make a difference for patients.

A bird's-eye view of deep learning in bioimage analysis

Deep learning of artificial neural networks has become the de facto standard approach to solving data analysis problems in virtually all fields of science and engineering. Also in biology and medicine, deep learning technologies are fundamentally transforming how we acquire, process, analyze, and interpret data, with potentially far-reaching consequences for healthcare. In this mini-review, we take a bird's-eye view at the past, present, and future developments of deep learning, starting from science at large, to biomedical imaging, and bioimage analysis in particular.

Automated interpretation of time-lapse quantitative phase image by machine learning to study cellular dynamics during epithelial-mesenchymal transition

SIGNIFICANCE

Machine learning is increasingly being applied to the classification of microscopic data. In order to detect some complex and dynamic cellular processes, time-resolved live-cell imaging might be necessary. Incorporating the temporal information into the classification process may allow for a better and more specific classification.

AIM

We propose a methodology for cell classification based on the time-lapse quantitative phase images (QPIs) gained by digital holographic microscopy (DHM) with the goal of increasing performance of classification of dynamic cellular processes.

APPROACH

The methodology was demonstrated by studying epithelial-mesenchymal transition (EMT) which entails major and distinct time-dependent morphological changes. The time-lapse QPIs of EMT were obtained over a 48-h period and specific novel features representing the dynamic cell behavior were extracted. The two distinct end-state phenotypes were classified by several supervised machine learning algorithms and the results were compared with the classification performed on single-time-point images.

RESULTS

In comparison to the single-time-point approach, our data suggest the incorporation of temporal information into the classification of cell phenotypes during EMT improves performance by nearly 9% in terms of accuracy, and further indicate the potential of DHM to monitor cellular morphological changes.

CONCLUSIONS

Proposed approach based on the time-lapse images gained by DHM could improve the monitoring of live cell behavior in an automated fashion and could be further developed into a tool for high-throughput automated analysis of unique cell behavior.

Probing the characteristics and biofunctional effects of disease-affected cells and drug response via machine learning applications

Drug-induced transformations in disease characteristics at the cellular and molecular level offers the opportunity to predict and evaluate the efficacy of pharmaceutical ingredients whilst enabling the optimal design of new and improved drugs with enhanced pharmacokinetics and pharmacodynamics. Machine learning is a promising in-silico tool used to simulate cells with specific disease properties and to determine their response toward drug uptake. Differences in the properties of normal and infected cells, including biophysical, biochemical and physiological characteristics, plays a key role in developing fundamental cellular probing platforms for machine learning applications. Cellular features can be extracted periodically from both the drug treated, infected, and normal cells via image segmentations in order to probe dynamic differences in cell behavior. Cellular segmentation can be evaluated to reflect the levels of drug effect on a distinct cell or group of cells via probability scoring. This article provides an account for the use of machine learning methods to probe differences in the biophysical, biochemical and physiological characteristics of infected cells in response to pharmacokinetics uptake of drug ingredients for application in cancer, diabetes and neurodegenerative disease therapies.

Quantifying drug-induced structural toxicity in hepatocytes and cardiomyocytes derived from hiPSCs using a deep learning method

Cardiac and hepatic toxicity result from induced disruption of the functioning of cardiomyocytes and hepatocytes, respectively, which is tightly related to the organization of their subcellular structures. Cellular structure can be analyzed from microscopy imaging data. However, subtle or complex structural changes that are not easily perceived may be missed by conventional image-analysis techniques. Here we report the evaluation of PhenoTox, an image-based deep-learning method of quantifying drug-induced structural changes using human hepatocytes and cardiomyocytes derived from human induced pluripotent stem cells. We assessed the ability of the deep learning method to detect variations in the organization of cellular structures from images of fixed or live cells. We also evaluated the power and sensitivity of the method for detecting toxic effects of drugs by conducting a set of experiments using known toxicants and other methods of screening for cytotoxic effects. Moreover, we used PhenoTox to characterize the effects of tamoxifen and doxorubicin-which cause liver toxicity-on hepatocytes. PhenoTox revealed differences related to loss of cytochrome P450 3A4 activity, for which it showed greater sensitivity than a caspase 3/7 assay. Finally, PhenoTox detected structural toxicity in cardiomyocytes, which was correlated with contractility defects induced by doxorubicin, erlotinib, and sorafenib. Taken together, the results demonstrated that PhenoTox can capture the subtle morphological changes that are early signs of toxicity in both hepatocytes and cardiomyocytes.

Machine Learning Representation of Loss of Eye Regularity in a Drosophila Neurodegenerative Model

The fruit fly compound eye is a premier experimental system for modeling human neurodegenerative diseases. The disruption of the retinal geometry has been historically assessed using time-consuming and poorly reliable techniques such as histology or pseudopupil manual counting. Recent semiautomated quantification approaches rely either on manual region-of-interest delimitation or engineered features to estimate the extent of degeneration. This work presents a fully automated classification pipeline of bright-field images based on orientated gradient descriptors and machine learning techniques. An initial region-of-interest extraction is performed, applying morphological kernels and Euclidean distance-to-centroid thresholding. Image classification algorithms are trained on these regions (support vector machine, decision trees, random forest, and convolutional neural network), and their performance is evaluated on independent, unseen datasets. The combinations of oriented gradient + gaussian kernel Support Vector Machine [0.97 accuracy and 0.98 area under the curve (AUC)] and fine-tuned pre-trained convolutional neural network (0.98 accuracy and 0.99 AUC) yielded the best results overall. The proposed method provides a robust quantification framework that can be generalized to address the loss of regularity in biological patterns similar to the Drosophila eye surface and speeds up the processing of large sample batches.

CytoCensus, mapping cell identity and division in tissues and organs using machine learning

A major challenge in cell and developmental biology is the automated identification and quantitation of cells in complex multilayered tissues. We developed CytoCensus: an easily deployed implementation of supervised machine learning that extends convenient 2D 'point-and-click' user training to 3D detection of cells in challenging datasets with ill-defined cell boundaries. In tests on such datasets, CytoCensus outperforms other freely available image analysis software in accuracy and speed of cell detection. We used CytoCensus to count stem cells and their progeny, and to quantify individual cell divisions from time-lapse movies of explanted Drosophila larval brains, comparing wild-type and mutant phenotypes. We further illustrate the general utility and future potential of CytoCensus by analysing the 3D organisation of multiple cell classes in Zebrafish retinal organoids and cell distributions in mouse embryos. CytoCensus opens the possibility of straightforward and robust automated analysis of developmental phenotypes in complex tissues.

SUMOylated PRC1 controls histone H3.3 deposition and genome integrity of embryonic heterochromatin

Chromatin integrity is essential for cellular homeostasis. Polycomb group proteins modulate chromatin states and transcriptionally repress developmental genes to maintain cell identity. They also repress repetitive sequences such as major satellites and constitute an alternative state of pericentromeric constitutive heterochromatin at paternal chromosomes (pat‐PCH) in mouse pre‐implantation embryos. Remarkably, pat‐PCH contains the histone H3.3 variant, which is absent from canonical PCH at maternal chromosomes, which is marked by histone H3 lysine 9 trimethylation (H3K9me3), HP1, and ATRX proteins. Here, we show that SUMO2‐modified CBX2‐containing Polycomb Repressive Complex 1 (PRC1) recruits the H3.3‐specific chaperone DAXX to pat‐PCH, enabling H3.3 incorporation at these loci. Deficiency of Daxx or PRC1 components Ring1 and Rnf2 abrogates H3.3 incorporation, induces chromatin decompaction and breakage at PCH of exclusively paternal chromosomes, and causes their mis‐segregation. Complementation assays show that DAXX‐mediated H3.3 deposition is required for chromosome stability in early embryos. DAXX also regulates repression of PRC1 target genes during oogenesis and early embryogenesis. The study identifies a novel critical role for Polycomb in ensuring heterochromatin integrity and chromosome stability in mouse early development.

Predicting cell lineages using autoencoders and optimal transport

Lineage tracing involves the identification of all ancestors and descendants of a given cell, and is an important tool for studying biological processes such as development and disease progression. However, in many settings, controlled time-course experiments are not feasible, for example when working with tissue samples from patients. Here we present ImageAEOT, a computational pipeline based on autoencoders and optimal transport for predicting the lineages of cells using time-labeled datasets from different stages of a cellular process. Given a single-cell image from one of the stages, ImageAEOT generates an artificial lineage of this cell based on the population characteristics of the other stages. These lineages can be used to connect subpopulations of cells through the different stages and identify image-based features and biomarkers underlying the biological process. To validate our method, we apply ImageAEOT to a benchmark task based on nuclear and chromatin images during the activation of fibroblasts by tumor cells in engineered 3D tissues. We further validate ImageAEOT on chromatin images of various breast cancer cell lines and human tissue samples, thereby linking alterations in chromatin condensation patterns to different stages of tumor progression. Our results demonstrate the promise of computational methods based on autoencoding and optimal transport principles for lineage tracing in settings where existing experimental strategies cannot be used.

Many key biological processes, such as development and disease progression, require analyzing lineages of cells backwards as well as forwards in time. However, current single-cell experiments tend to be destructive to cells, so that a single lineage can only be measured at one point in time. In this work, we introduce a computational framework for predicting the lineage of cells from a single snapshot in time based on measurements of other cells at other time points. The method generates these lineages by computing the most plausible path for a population of cells to transition from one time point to the next, assuming that a cell is more likely to transition to similar cells compared to dissimilar cells. We validate our computational method on imaging data of fibroblasts and cancer cells, though our method could also be applied to other modalities of single-cell data such as genomics and transcriptomics as well as multi-modal single-cell datasets.

Classifying changes in LN-18 glial cell morphology: a supervised machine learning approach to analyzing cell microscopy data via FIJI and WEKA

In cell-based research, the process of visually monitoring cells generates large image datasets that need to be evaluated for quantifiable information in order to track the effectiveness of treatments in vitro. With the traditional, end-point assay-based approach being error-prone, and existing computational approaches being complex, we tested existing machine learning frameworks to find methods that are relatively simple, yet powerful enough to accomplish the goal of analyzing cell microscopy data. This paper details the machine learning pipeline for pixel-based classification and object-based classification. Furthermore, it compares the performances of three classifiers. The classifiers evaluated were the fast-random forest (RF), the sequential minimal optimization (SMO), and the Bayesian network (BN). Images were first preprocessed using smoothing and contrast methods found in FIJI. For pixel-based classification, the preprocessed images were fed into the Trainable Waikato Segmentation (TWS). For object-based classification, training and classification were conducted within the Waikato Environment for Knowledge Analysis (WEKA) interface. All classifiers' performance was evaluated using the WEKA experimental explorer. In terms of performance, the BN had the lowest classification accuracy for both the pixel-based and object-based model. The object-based SMO classifier had the best performance with the lowest mean absolute error of 0.05. The TWS and WEKA interface allows users to easily create and train classifiers for image analysis. However, for analyzing large image datasets, they are not ideal. Grapical abstract.

Application and Comparison of Supervised Learning Strategies to Classify Polarity of Epithelial Cell Spheroids in 3D Culture

Three-dimensional culture systems that allow generation of monolayered epithelial cell spheroids are widely used to study epithelial function in vitro. Epithelial spheroid formation is applied to address cellular consequences of (mono)-genetic disorders, that is, ciliopathies, in toxicity testing, or to develop treatment options aimed to restore proper epithelial cell characteristics and function. With the potential of a high-throughput method, the main obstacle to efficient application of the spheroid formation assay so far is the laborious, time-consuming, and bias-prone analysis of spheroid images by individuals. Hundredths of multidimensional fluorescence images are blinded, rated by three persons, and subsequently, differences in ratings are compared and discussed. Here, we apply supervised learning and compare strategies based on machine learning versus deep learning. While deep learning approaches can directly process raw image data, machine learning requires transformed data of features extracted from fluorescence images. We verify the accuracy of both strategies on a validation data set, analyse an experimental data set, and observe that different strategies can be very accurate. Deep learning, however, is less sensitive to overfitting and experimental batch-to-batch variations, thus providing a rather powerful and easily adjustable classification tool.

High-Content Image-Based Single-Cell Phenotypic Analysis for the Testicular Toxicity Prediction Induced by Bisphenol A and Its Analogs Bisphenol S, Bisphenol AF, and Tetrabromobisphenol A in a Three-Dimensional Testicular Cell Co-culture Model

Emerging data indicate that structural analogs of bisphenol A (BPA) such as bisphenol S (BPS), tetrabromobisphenol A (TBBPA), and bisphenol AF (BPAF) have been introduced into the market as substitutes for BPA. Our previous study compared in vitro testicular toxicity using murine C18-4 spermatogonial cells and found that BPAF and TBBPA exhibited higher spermatogonial toxicities as compared with BPA and BPS. Recently, we developed a novel in vitro three-dimensional (3D) testicular cell co-culture model, enabling the classification of reproductive toxic substances. In this study, we applied the testicular cell co-culture model and employed a high-content image (HCA)-based single-cell analysis to further compare the testicular toxicities of BPA and its analogs. We also developed a machine learning (ML)-based HCA pipeline to examine the complex phenotypic changes associated with testicular toxicities. We found dose- and time-dependent changes in a wide spectrum of adverse endpoints, including nuclear morphology, DNA synthesis, DNA damage, and cytoskeletal structure in a single-cell-based analysis. The co-cultured testicular cells were more sensitive than the C18 spermatogonial cells in response to BPA and its analogs. Unlike conventional population-averaged assays, single-cell-based assays not only showed the levels of the averaged population, but also revealed changes in the sub-population. Machine learning-based phenotypic analysis revealed that treatment of BPA and its analogs resulted in the loss of spatial cytoskeletal structure, and an accumulation of M phase cells in a dose- and time-dependent manner. Furthermore, treatment of BPAF-induced multinucleated cells, which were associated with altered DNA damage response and impaired cellular F-actin filaments. Overall, we demonstrated a new and effective means to evaluate multiple toxic endpoints in the testicular co-culture model through the combination of ML and high-content image-based single-cell analysis. This approach provided an in-depth analysis of the multi-dimensional HCA data and provided an unbiased quantitative analysis of the phenotypes of interest.

Rotation Invariant Clustering of 3D Cell Nuclei Shapes

Cellular imaging with confocal fluorescence laser microscopy gave rise to many new insights into the cellular machinery. One interesting observation suggests that morphology of cell nucleus plays a key role for neutrophilic function, which is an essential part of the innate immune system of most mammals. Due to the increasing availability of high resolution 3D images coming from the microscope, machine learning becomes a promising tool for automatically discovering underlying hidden structures. Here, the major difficulty consists of selecting an appropriate representation for characterizing the morphology of cell nucleus. In this work we tackle this problem and propose a fully unsupervised mechanism for finding structure in high-throughput 3D image data. The key component of our approach is based on Generic Fourier Transform (GFT) for 2D images, which for 3D involves spherical coordinate transformation prior to fast Discrete Fourier Transformation. On top on GFT we apply dimensionality reduction with Principal Component Analysis, followed by generative cluster analysis with a Gaussian Mixture Model. We validate our new approach first on a synthetic 3D-MNIST dataset with random rotations, where quantitative and qualitative results confirm the applicability of the proposed pipeline for exploring shape space in a purely unsupervised manner. Then we apply our proposed technique to a new collected dataset of high resolution 3D images of neutrophile nuclei suggesting a clustering model with six significant clusters of morphological cell nuclei prototypes. We visualize differences in the cell shape clusters by providing prototypical examples of neutrophilic cell nuclei.

Systems biology and big data in asthma and allergy: recent discoveries and emerging challenges

Asthma is a common condition caused by immune and respiratory dysfunction, and it is often linked to allergy. A systems perspective may prove helpful in unravelling the complexity of asthma and allergy. Our aim is to give an overview of systems biology approaches used in allergy and asthma research. Specifically, we describe recent "omic"-level findings, and examine how these findings have been systematically integrated to generate further insight.Current research suggests that allergy is driven by genetic and epigenetic factors, in concert with environmental factors such as microbiome and diet, leading to early-life disturbance in immunological development and disruption of balance within key immuno-inflammatory pathways. Variation in inherited susceptibility and exposures causes heterogeneity in manifestations of asthma and other allergic diseases. Machine learning approaches are being used to explore this heterogeneity, and to probe the pathophysiological patterns or "endotypes" that correlate with subphenotypes of asthma and allergy. Mathematical models are being built based on genomic, transcriptomic and proteomic data to predict or discriminate disease phenotypes, and to describe the biomolecular networks behind asthma.The use of systems biology in allergy and asthma research is rapidly growing, and has so far yielded fruitful results. However, the scale and multidisciplinary nature of this research means that it is accompanied by new challenges. Ultimately, it is hoped that systems medicine, with its integration of omics data into clinical practice, can pave the way to more precise, personalised and effective management of asthma.

Phenotyping analysis of maize stem using micro-computed tomography at the elongation and tasseling stages

BACKGROUND

Micro-computed tomography (μCT) bring a new opportunity to accurately quantify micro phenotypic traits of maize stem, also provide comparable benchmark to evaluate its dynamic development at the different growth stages. The progressive accumulation of stem biomass brings manifest structure changes of maize stem and vascular bundles, which are closely related with maize varietal characteristics and growth stages. Thus, micro-phenotyping (μPhenotyping) of maize stems is not only valuable to evaluate bio-mechanics and water-transport performance of maize, but also yield growth-based traits for quantitative traits loci (QTL) and functional genes location in molecular breeding.

RESULT

In this study, maize stems of 20 maize cultivars and two growth stages were imaged using μCT scanning technology. According to the observable differences of maize stems from the elongation and tasseling stages, function zones of maize stem were firstly defined to describe the substance accumulation of maize stems. And then a set of image-based μPhenotyping pipelines were implemented to quantify maize stem and vascular bundles at the two stages. The coefficient of determination (R2) of counting vascular bundles was higher than 0.95. Based on the uniform contour representation, intensity-related, geometry-related and distribution-related traits of vascular bundles were respectively evaluated in function zones and structure layers. And growth-related traits of the slice, epidermis, periphery and inner zones were also used to describe the dynamic growth of maize stem. Statistical analysis demonstrated the presented method was suitable to the phenotyping analysis of maize stem for multiple growth stages.

CONCLUSIONS

The novel descriptors of function zones provide effective phenotypic references to quantify the differences between growth stages; and the detection and identification of vascular bundles based on function zones are more robust to determine the adaptive image analysis pipeline. Developing robust and effective image-based phenotyping method to assess the traits of stem and vascular bundles, is highly relevant for understanding the relationship between maize phenomics and genomics.

Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl

Segmenting the nuclei of cells in microscopy images is often the first step in the quantitative analysis of imaging data for biological and biomedical applications. Many bioimage analysis tools can segment nuclei in images but need to be selected and configured for every experiment. The 2018 Data Science Bowl attracted 3,891 teams worldwide to make the first attempt to build a segmentation method that could be applied to any two-dimensional light microscopy image of stained nuclei across experiments, with no human interaction. Top participants in the challenge succeeded in this task, developing deep-learning-based models that identified cell nuclei across many image types and experimental conditions without the need to manually adjust segmentation parameters. This represents an important step toward configuration-free bioimage analysis software tools.

Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group Article*: Opinion on the Application of Artificial Intelligence and Machine Learning to Digital Toxicologic Pathology

Toxicologic pathology is transitioning from analog to digital methods. This transition seems inevitable due to a host of ongoing social and medical technological forces. Of these, artificial intelligence (AI) and in particular machine learning (ML) are globally disruptive, rapidly growing sectors of technology whose impact on the long-established field of histopathology is quickly being realized. The development of increasing numbers of algorithms, peering ever deeper into the histopathological space, has demonstrated to the scientific community that AI pathology platforms are now poised to truly impact the future of precision and personalized medicine. However, as with all great technological advances, there are implementation and adoption challenges. This review aims to define common and relevant AI and ML terminology, describe data generation and interpretation, outline current and potential future business cases, discuss validation and regulatory hurdles, and most importantly, propose how overcoming the challenges of this burgeoning technology may shape toxicologic pathology for years to come, enabling pathologists to contribute even more effectively to answering scientific questions and solving global health issues. [Box: see text]

Applications of Computer Modeling and Simulation in Cartilage Tissue Engineering

BACKGROUND

Advances in cartilage tissue engineering have demonstrated noteworthy potential for developing cartilage for implantation onto sites impacted by joint degeneration and injury. To supplement resource-intensive in vivo and in vitro studies required for cartilage tissue engineering, computational models and simulations can assist in enhancing experimental design.

METHODS

Research articles pertinent to cartilage tissue engineering and computer modeling were identified, reviewed, and summarized. Various applications of computer modeling for cartilage tissue engineering are highlighted, limitations of in silico modeling are addressed, and suggestions for future work are enumerated.

RESULTS

Computational modeling can help better characterize shear stresses generated by bioreactor fluid flow, refine scaffold geometry, customize the mechanical properties of engineered cartilage tissue, and model rates of cell growth and dynamics. Thus, results from in silico studies can help resourcefully enhance in vitro and in vivo studies; however, the limitations of these studies, such as the underlying assumptions and simplifications applied in each model, should always be addressed and justified where applicable. In silico models should also seek validation and verification when possible.

CONCLUSION

Future studies may adopt similar approaches to supplement in vitro trials and further investigate effects of mechanical stimulation on chondrocyte and stem cell dynamics. Additionally, as precision medicine, machine learning, and powerful open-source software become more popular and accessible, applications of multi-scale and multiphysics computational models in cartilage tissue engineering are expected to increase.

Morphology-based classification of mycobacteria-infected macrophages with convolutional neural network: reveal EsxA-induced morphologic changes indistinguishable by naked eyes

EsxA is an essential virulence factor for Mycobacterium tuberculosis (Mtb) pathogenesis as well as an important biomarker for Mtb detection. In this study, we use light microscopy and deep learning-based image analysis to classify the morphologic changes of macrophages infected by Mycobacterium marinum (Mm), a surrogate model for Mtb. Macrophages were infected either with the mCherry-expressing Mm wild type strain (Mm(WT)), or a mutant strain with deletion of the esxA-esxB operon (Mm(ΔEsxA:B)). The mCherry serves as an infection marker to train the convolution neural network (CNN) and to validate the classification results. Data show that CNN can distinguish the Mm(WT)-infected cells from uninfected cells with an accuracy of 92.4% at 2 hours postinfection (hpi). However, the accuracy at 12 and 24 hpi is decreased to ∼75% and ∼83%, respectively, suggesting dynamic morphologic changes through different stages of infection. The accuracy of discriminating Mm(ΔEsxA:B)-infected cells from uninfected cells is lower than 80% at all time, which is consistent to attenuated virulence of Mm(ΔEsxA:B). Interestingly, CNN distinguishes Mm(WT)-infected cells from Mm(ΔEsxA:B)-infected cells with ∼90% accuracy, implicating EsxA induces unique morphologic changes in macrophages. Deconvolutional analysis successfully reconstructed the morphologic features used by CNN for classification, which are indistinguishable to naked eyes and distinct from intracellular mycobacteria. This study presents a deep learning-aided imaging analytical tool that can accurately detect virulent mycobacteria-infected macrophages by cellular morphologic changes. The observed morphologic changes induced by EsxA warrant further studies to fill the gap from molecular actions of bacterial virulence factors to cellular morphology.

Phenotypic Image Analysis Software Tools for Exploring and Understanding Big Image Data from Cell-Based Assays

Phenotypic image analysis is the task of recognizing variations in cell properties using microscopic image data. These variations, produced through a complex web of interactions between genes and the environment, may hold the key to uncover important biological phenomena or to understand the response to a drug candidate. Today, phenotypic analysis is rarely performed completely by hand. The abundance of high-dimensional image data produced by modern high-throughput microscopes necessitates computational solutions. Over the past decade, a number of software tools have been developed to address this need. They use statistical learning methods to infer relationships between a cell's phenotype and data from the image. In this review, we examine the strengths and weaknesses of non-commercial phenotypic image analysis software, cover recent developments in the field, identify challenges, and give a perspective on future possibilities.

Towards pixel-to-pixel deep nucleus detection in microscopy images

BACKGROUND

Nucleus is a fundamental task in microscopy image analysis and supports many other quantitative studies such as object counting, segmentation, tracking, etc. Deep neural networks are emerging as a powerful tool for biomedical image computing; in particular, convolutional neural networks have been widely applied to nucleus/cell detection in microscopy images. However, almost all models are tailored for specific datasets and their applicability to other microscopy image data remains unknown. Some existing studies casually learn and evaluate deep neural networks on multiple microscopy datasets, but there are still several critical, open questions to be addressed.

RESULTS

We analyze the applicability of deep models specifically for nucleus detection across a wide variety of microscopy image data. More specifically, we present a fully convolutional network-based regression model and extensively evaluate it on large-scale digital pathology and microscopy image datasets, which consist of 23 organs (or cancer diseases) and come from multiple institutions. We demonstrate that for a specific target dataset, training with images from the same types of organs might be usually necessary for nucleus detection. Although the images can be visually similar due to the same staining technique and imaging protocol, deep models learned with images from different organs might not deliver desirable results and would require model fine-tuning to be on a par with those trained with target data. We also observe that training with a mixture of target and other/non-target data does not always mean a higher accuracy of nucleus detection, and it might require proper data manipulation during model training to achieve good performance.

CONCLUSIONS

We conduct a systematic case study on deep models for nucleus detection in a wide variety of microscopy images, aiming to address several important but previously understudied questions. We present and extensively evaluate an end-to-end, pixel-to-pixel fully convolutional regression network and report a few significant findings, some of which might have not been reported in previous studies. The model performance analysis and observations would be helpful to nucleus detection in microscopy images.

Predicting the future direction of cell movement with convolutional neural networks

Image-based deep learning systems, such as convolutional neural networks (CNNs), have recently been applied to cell classification, producing impressive results; however, application of CNNs has been confined to classification of the current cell state from the image. Here, we focused on cell movement where current and/or past cell shape can influence the future cell movement. We demonstrate that CNNs prospectively predicted the future direction of cell movement with high accuracy from a single image patch of a cell at a certain time. Furthermore, by visualizing the image features that were learned by the CNNs, we could identify morphological features, e.g., the protrusions and trailing edge that have been experimentally reported to determine the direction of cell movement. Our results indicate that CNNs have the potential to predict the future direction of cell movement from current cell shape, and can be used to automatically identify those morphological features that influence future cell movement.

Automated Neuron Detection in High-Content Fluorescence Microscopy Images Using Machine Learning

The study of neuronal morphology in relation to function, and the development of effective medicines to positively impact this relationship in patients suffering from neurodegenerative diseases, increasingly involves image-based high-content screening and analysis. The first critical step toward fully automated high-content image analyses in such studies is to detect all neuronal cells and distinguish them from possible non-neuronal cells or artifacts in the images. Here we investigate the performance of well-established machine learning techniques for this purpose. These include support vector machines, random forests, k-nearest neighbors, and generalized linear model classifiers, operating on an extensive set of image features extracted using the compound hierarchy of algorithms representing morphology, and the scale-invariant feature transform. We present experiments on a dataset of rat hippocampal neurons from our own studies to find the most suitable classifier(s) and subset(s) of features in the common practical setting where there is very limited annotated data for training. The results indicate that a random forests classifier using the right feature subset ranks best for the considered task, although its performance is not statistically significantly better than some support vector machine based classification models.

Uncertainty reduction in biochemical kinetic models: Enforcing desired model properties

A persistent obstacle for constructing kinetic models of metabolism is uncertainty in the kinetic properties of enzymes. Currently, available methods for building kinetic models can cope indirectly with uncertainties by integrating data from different biological levels and origins into models. In this study, we use the recently proposed computational approach iSCHRUNK (in Silico Approach to Characterization and Reduction of Uncertainty in the Kinetic Models), which combines Monte Carlo parameter sampling methods and machine learning techniques, in the context of Bayesian inference. Monte Carlo parameter sampling methods allow us to exploit synergies between different data sources and generate a population of kinetic models that are consistent with the available data and physicochemical laws. The machine learning allows us to data-mine the a priori generated kinetic parameters together with the integrated datasets and derive posterior distributions of kinetic parameters consistent with the observed physiology. In this work, we used iSCHRUNK to address a design question: can we identify which are the kinetic parameters and what are their values that give rise to a desired metabolic behavior? Such information is important for a wide variety of studies ranging from biotechnology to medicine. To illustrate the proposed methodology, we performed Metabolic Control Analysis, computed the flux control coefficients of the xylose uptake (XTR), and identified parameters that ensure a rate improvement of XTR in a glucose-xylose co-utilizing S. cerevisiae strain. Our results indicate that only three kinetic parameters need to be accurately characterized to describe the studied physiology, and ultimately to design and control the desired responses of the metabolism. This framework paves the way for a new generation of methods that will systematically integrate the wealth of available omics data and efficiently extract the information necessary for metabolic engineering and synthetic biology decisions.

Kinetic models are the most promising tool for understanding the complex dynamic behavior of living cells. The primary goal of kinetic models is to capture the properties of the metabolic networks as a whole, and thus we need large-scale models for dependable in silico analyses of metabolism. However, uncertainty in kinetic parameters impedes the development of kinetic models, and uncertainty levels increase with the model size. Tools that will address the issues with parameter uncertainty and that will be able to reduce the uncertainty propagation through the system are therefore needed. In this work, we applied a method called iSCHRUNK that combines parameter sampling and machine learning techniques to characterize the uncertainties and uncover intricate relationships between the parameters of kinetic models and the responses of the metabolic network. The proposed method allowed us to identify a small number of parameters that determine the responses in the network regardless of the values of other parameters. As a consequence, in future studies of metabolism, it will be sufficient to explore a reduced kinetic space, and more comprehensive analyses of large-scale and genome-scale metabolic networks will be computationally tractable.

Analytics-statistics mixed training and its fitness to semisupervised manufacturing

While there have been many studies using machine learning (ML) algorithms to predict process outcomes and device performance in semiconductor manufacturing, the extensively developed technology computer-aided design (TCAD) physical models should play a more significant role in conjunction with ML. While TCAD models have been effective in predicting the trends of experiments, a machine learning statistical model is more capable of predicting the anomalous effects that can be dependent on the chambers, machines, fabrication environment, and specific layouts. In this paper, we use an analytics-statistics mixed training (ASMT) approach using TCAD. Under this method, the TCAD models are incorporated into the machine learning training procedure. The mixed dataset with the experimental and TCAD results improved the prediction in terms of accuracy. With the application of ASMT to the BOSCH process, we show that the mean square error (MSE) can be effectively decreased when the analytics-statistics mixed training (ASMT) scheme is used instead of the classic neural network (NN) used in the baseline study. In this method, statistical induction and analytical deduction can be combined to increase the prediction accuracy of future intelligent semiconductor manufacturing.

Integrating Imaging and Omics: Computational Methods and Challenges

Fluorescence microscopy imaging has long been complementary to DNA sequencing- and mass spectrometry–based omics in biomedical research, but these approaches are now converging. On the one hand, omics methods are moving from in vitro methods that average across large cell populations to in situ molecular characterization tools with single-cell sensitivity. On the other hand, fluorescence microscopy imaging has moved from a morphological description of tissues and cells to quantitative molecular profiling with single-molecule resolution. Recent technological developments underpinned by computational methods have started to blur the lines between imaging and omics and have made their direct correlation and seamless integration an exciting possibility. As this trend continues rapidly, it will allow us to create comprehensive molecular profiles of living systems with spatial and temporal context and subcellular resolution. Key to achieving this ambitious goal will be novel computational methods and successfully dealing with the challenges of data integration and sharing as well as cloud-enabled big data analysis.

Nuclear morphometrics and chromatin condensation patterns as disease biomarkers using a mobile microscope

Current cancer diagnosis involves the use of nuclear morphology and chromatin condensation signatures for accurate advanced stage classification. While such diagnostic approaches rely on high resolution imaging of the cell nucleus using expensive microscopy systems, developing portable mobile microscopes to visualize nuclear and chromatin condensation patterns is desirable at clinical settings with limited infrastructure. In this study, we develop a portable fluorescent mobile microscope capable of acquiring high resolution images of the nucleus and chromatin. Using this we extracted nuclear morphometric and chromatin texture based features and were able to discriminate between normal and cancer cells with similar accuracy as wide-field fluorescence microscopy. We were also able to detect subtle changes in nuclear and chromatin features in cells subjected to compressive forces, cytoskeletal perturbations and cytokine stimulation, thereby highlighting the sensitivity of the portable microscope. Taken together, we present a versatile platform to exploit nuclear morphometrics and chromatin condensation features as physical biomarkers for point-of-care diagnostic solutions.

The application of convolutional neural network to stem cell biology

Induced pluripotent stem cells (iPSC) are one the most prominent innovations of medical research in the last few decades. iPSCs can be easily generated from human somatic cells and have several potential uses in regenerative medicine, disease modeling, drug screening, and precision medicine. However, further innovation is still required to realize their full potential. Machine learning is an algorithm that learns from large datasets for pattern formation and classification. Deep learning, a form of machine learning, uses a multilayered neural network that mimics human neural circuit structure. Deep neural networks can automatically extract features from an image, although classical machine learning methods still require feature extraction by a human expert. Deep learning technology has developed recently; in particular, the accuracy of an image classification task by using a convolutional neural network (CNN) has exceeded that of humans since 2015. CNN is now used to address several tasks including medical issues. We believe that CNN would also have a great impact on the research of stem cell biology. iPSCs are utilized after their differentiation to specific cells, which are characterized by molecular techniques such as immunostaining or lineage tracing. Each cell shows a characteristic morphology; thus, a morphology-based identification system of cell type by CNN would be an alternative technique. The development of CNN enables the automation of identifying cell types from phase contrast microscope images without molecular labeling, which will be applied to several researches and medical science. Image classification is a strong field among deep learning tasks, and several medical tasks will be solved by deep learning-based programs in the future.

From observing to predicting single-cell structure and function with high-throughput/high-content microscopy

In the past 15 years, cell-based microscopy has evolved its focus from observing cell function to aiming to predict it. In particular-powered by breakthroughs in computer vision, large-scale image analysis and machine learning-high-throughput and high-content microscopy imaging have enabled to uniquely harness single-cell information to systematically discover and annotate genes and regulatory pathways, uncover systems-level interactions and causal links between cellular processes, and begin to clarify and predict causal cellular behaviour and decision making. Here we review these developments, discuss emerging trends in the field, and describe how single-cell 'omics and single-cell microscopy are imminently in an intersecting trajectory. The marriage of these two fields will make possible an unprecedented understanding of cell and tissue behaviour and function.

Cardiac tissue engineering: state-of-the-art methods and outlook

The purpose of this review is to assess the state-of-the-art fabrication methods, advances in genome editing, and the use of machine learning to shape the prospective growth in cardiac tissue engineering. Those interdisciplinary emerging innovations would move forward basic research in this field and their clinical applications. The long-entrenched challenges in this field could be addressed by novel 3-dimensional (3D) scaffold substrates for cardiomyocyte (CM) growth and maturation. Stem cell-based therapy through genome editing techniques can repair gene mutation, control better maturation of CMs or even reveal its molecular clock. Finally, machine learning and precision control for improvements of the construct fabrication process and optimization in tissue-specific clonal selections with an outlook of cardiac tissue engineering are also presented.

A two-stage method for automated detection of ring-like endosomes in fluorescent microscopy images

Endosomes are subcellular organelles which serve as important transport compartments in eukaryotic cells. Fluorescence microscopy is a widely applied technology to study endosomes at the subcellular level. In general, a microscopy image can contain a large number of organelles and endosomes in particular. Detecting and annotating endosomes in fluorescence microscopy images is a critical part in the study of subcellular trafficking processes. Such annotation is usually performed by human inspection, which is time-consuming and prone to inaccuracy if carried out by inexperienced analysts. This paper proposes a two-stage method for automated detection of ring-like endosomes. The method consists of a localization stage cascaded by an identification stage. Given a test microscopy image, the localization stage generates a voting-map by locally comparing the query endosome patches and the test image based on a bag-of-words model. Using the voting-map, a number of candidate patches of endosomes are determined. Subsequently, in the identification stage, a support vector machine (SVM) is trained using the endosome patches and the background pattern patches. Each of the candidate patches is classified by the SVM to rule out those patches of endosome-like background patterns. The performance of the proposed method is evaluated with real microscopy images of human myeloid endothelial cells. It is shown that the proposed method significantly outperforms several state-of-the-art competing methods using multiple performance metrics.

Supervised learning to quantify amyloidosis in whole brains of an Alzheimer's disease mouse model acquired with optical projection tomography

Alzheimer's disease (AD) is characterized by amyloidosis of brain tissues. This phenomenon is studied with genetically-modified mouse models. We propose a method to quantify amyloidosis in whole 5xFAD mouse brains, a model of AD. We use optical projection tomography (OPT) and a random forest voxel classifier to segment and measure amyloid plaques. We validate our method in a preliminary cross-sectional study, where we measure 6136 ± 1637, 8477 ± 3438, and 17267 ± 4241 plaques (AVG ± SD) at 11, 17, and 31 weeks. Overall, this method can be used in the evaluation of new treatments against AD.

Unsupervised quantification of tissue immunofluorescence in animal models of multiple sclerosis - Instructions for use

BACKGROUND

In the analysis of animal models of CNS diseases such as experimental autoimmune encephalomyelitis (EAE), immunostaining and histopathology are important readouts. However, the complex morphological features of a tissue staining are often reduced to a single measure which relies on tedious manual planimetry. Furthermore, the measure itself and co-variables such as the region being analysed are chosen in a human decision-making process, which introduces bias.

NEW METHOD

First aim of the present study is to provide an open-source workflow for the high-throughput, unsupervised quantification of different stainings in the spinal cord. We evaluate different EAE models, spinal cord regions and different time points of disease. By applying random forest classification, we compare different measures.

RESULTS

Exemplified for glial reactivity, we show that measures and variables interact and that their values are non-normally distributed, hampering the common use of parametric tests. Furthermore, we demonstrate that one-dimensional measures are insufficient descriptors for immunofluorescence data in EAE and thus need to be considered as partly invalid.

COMPARISON WITH EXISTING METHODS

We show in a systematic analysis of EAE studies that currently published immunohistological outcomes are highly incompatible regarding methodology and statistics. Furthermore, they lack the report of important information necessary for reproducibility and do not use unsupervised automatic analysis.

CONCLUSIONS

Our results discover relevant caveats in the currently used methods of immunofluorescence analysis. The provided step-by-step instructions and open-source code are intended to serve as a framework for sensitive, unbiased immunofluorescence analysis of tissue sections in translational research.

Towards Building a Plant Cell Atlas

Enormous societal challenges, such as feeding and providing energy for a growing population in a dramatically changing climate, necessitate technological advances in plant science. Plant cells are fundamental organizational units that mediate the production, transport, and storage of our primary food sources, and they sequester a significant proportion of the world's carbon. New technologies allow comprehensive descriptions of cells that could accelerate research across fields of plant science. Complementary to the efforts towards understanding the cellular diversity in human brain and immune systems, a Plant Cell Atlas (PCA) that maps molecular machineries to cellular and subcellular domains, follows their dynamic movements, and describes their interactions would accelerate discovery in plant science and help to solve imminent societal problems.

Deep Learning in Image Cytometry: A Review

Artificial intelligence, deep convolutional neural networks, and deep learning are all niche terms that are increasingly appearing in scientific presentations as well as in the general media. In this review, we focus on deep learning and how it is applied to microscopy image data of cells and tissue samples. Starting with an analogy to neuroscience, we aim to give the reader an overview of the key concepts of neural networks, and an understanding of how deep learning differs from more classical approaches for extracting information from image data. We aim to increase the understanding of these methods, while highlighting considerations regarding input data requirements, computational resources, challenges, and limitations. We do not provide a full manual for applying these methods to your own data, but rather review previously published articles on deep learning in image cytometry, and guide the readers toward further reading on specific networks and methods, including new methods not yet applied to cytometry data. © 2018 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

Comparison between two programs for image analysis, machine learning and subsequent classification

In the early 1950s, flow cytometry was developed as the first method for automated quantitative cellular analysis. In the early 1990s, the first equipment for image cytometry (laser scanning cytometry, LSC) became commercially available. As flow cytometry was considered the gold standard, various studies found that the results of flow cytometry and LSC generated comparable results. One of the first programs for image analysis that included morphological parameters was ImageJ, published in 1997. One of the newer programs for image analysis that is not limited to fluorescence images is the free software CellProfiler. In 2008, the same group published a new software, CellProfiler Analyst. One part of CellProfiler Analyst is a supervised machine-learning-based classifier that allows users to conduct imaging-based diagnoses, e.g., cellular diagnosis based on morphology. Another relatively new, free software for image analysis is QuPath. The aim of the present study was to compare two free programs for conducting image analysis, CellProfiler and QuPath, and the subsequent classification based on machine learning. For this study, images of renal tissue were analyzed, and the identified objects were classified. The same images were loaded in both software programs. Advanced statistical analysis was used to compare the two methods. The Bland-Altman assay showed that all of the differences were within the mean ± 1.96 * standard deviation, i.e., the differences are normally distributed, and the software programs are comparable. For the analyzed samples (renal tissue stained with HIF and TUNEL), the use of QuPath was easier because it offers image analysis without a previous processing of the images (e.g., conversion to grayscale, inverted intensities) and an unsupervised machine learning process.

Automated morphometry toolbox for analysis of microscopic model organisms using simple bright-field imaging

Model organisms with compact genomes, such as yeast and C aenorhabditis elegans, are particularly useful for understanding organism growth and life/cell cycle. Organism morphology is a critical parameter to measure in monitoring growth and stage in the life cycle. However, manual measurements are both time consuming and potentially inaccurate, due to variations among users and user fatigue. In this paper we present an automated method to segment bright-field images of fission yeast, budding yeast, and C. elegans roundworm, reporting a wide range of morphometric parameters, such as length, width, eccentricity, and others. Comparisons between automated and manual methods on fission yeast reveal good correlation in size values, with the 95% confidence interval lying between -0.8 and +0.6 μm in cell length, similar to the 95% confidence interval between two manual users. In a head-to-head comparison with other published algorithms on multiple datasets, our method achieves more accurate and robust results with substantially less computation time. We demonstrate the method's versatility on several model organisms, and demonstrate its utility through automated analysis of changes in fission yeast growth due to single kinase deletions. The algorithm has additionally been implemented as a stand-alone executable program to aid dissemination to other researchers.

Pixel-to-Pixel Learning With Weak Supervision for Single-Stage Nucleus Recognition in Ki67 Images

OBJECTIVE

Nucleus recognition is a critical yet challenging step in histopathology image analysis, for example, in Ki67 immunohistochemistry stained images. Although many automated methods have been proposed, most use a multi-stage processing pipeline to categorize nuclei, leading to cumbersome, low-throughput, and error-prone assessments. To address this issue, we propose a novel deep fully convolutional network for single-stage nucleus recognition.

METHODS

Instead of conducting direct pixel-wise classification, we formulate nucleus identification as a deep structured regression model. For each input image, it produces multiple proximity maps, each of which corresponds to one nucleus category and exhibits strong responses in central regions of the nuclei. In addition, by taking into consideration the nucleus distribution in histopathology images, we further introduce an auxiliary task, region of interest (ROI) extraction, to assist and boost the nucleus quantification with weak ROI annotation. The proposed network can be learned in an end-to-end, pixel-to-pixel manner for simultaneous nucleus detection and classification.

RESULTS

We have evaluated this network on a pancreatic neuroendocrine tumor Ki67 image dataset, and the experiments demonstrate that our method outperforms recent state-of-the-art approaches.

CONCLUSION

We present a new, pixel-to-pixel deep neural network with two sibling branches for effective nucleus recognition and observe that learning with another relevant task, ROI extraction, can further boost individual nucleus localization and classification.

SIGNIFICANCE

Our method provides a clean, single-stage nucleus recognition pipeline for histopathology image analysis, especially a new perspective for Ki67 image quantification, which would potentially benefit individual object quantification in whole-slide images.

IMACEL: A cloud-based bioimage analysis platform for morphological analysis and image classification

Automated quantitative image analysis is essential for all fields of life science research. Although several software programs and algorithms have been developed for bioimage processing, an advanced knowledge of image processing techniques and high-performance computing resources are required to use them. Hence, we developed a cloud-based image analysis platform called IMACEL, which comprises morphological analysis and machine learning-based image classification. The unique click-based user interface of IMACEL’s morphological analysis platform enables researchers with limited resources to evaluate particles rapidly and quantitatively without prior knowledge of image processing. Because all the image processing and machine learning algorithms are performed on high-performance virtual machines, users can access the same analytical environment from anywhere. A validation study of the morphological analysis and image classification of IMACEL was performed. The results indicate that this platform is an accessible and potentially powerful tool for the quantitative evaluation of bioimages that will lower the barriers to life science research.

Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes

The quantification and identification of cellular phenotypes from high-content microscopy images has proven to be very useful for understanding biological activity in response to different drug treatments. The traditional approach has been to use classical image analysis to quantify changes in cell morphology, which requires several nontrivial and independent analysis steps. Recently, convolutional neural networks have emerged as a compelling alternative, offering good predictive performance and the possibility to replace traditional workflows with a single network architecture. In this study, we applied the pretrained deep convolutional neural networks ResNet50, InceptionV3, and InceptionResnetV2 to predict cell mechanisms of action in response to chemical perturbations for two cell profiling datasets from the Broad Bioimage Benchmark Collection. These networks were pretrained on ImageNet, enabling much quicker model training. We obtain higher predictive accuracy than previously reported, between 95% and 97%. The ability to quickly and accurately distinguish between different cell morphologies from a scarce amount of labeled data illustrates the combined benefit of transfer learning and deep convolutional neural networks for interrogating cell-based images.

Arrayed functional genetic screenings in pluripotency reprogramming and differentiation

Thoroughly understanding the molecular mechanisms responsible for the biological properties of pluripotent stem cells, as well as for the processes involved in reprograming, differentiation, and transition between Naïve and Primed pluripotent states, is of great interest in basic and applied research. Although pluripotent cells have been extensively characterized in terms of their transcriptome and miRNome, a comprehensive understanding of how these gene products specifically impact their biology, depends on gain- or loss-of-function experimental approaches capable to systematically interrogate their function. We review all studies carried up to date that used arrayed screening approaches to explore the function of these genetic elements on those biological contexts, using focused or genome-wide genetic libraries. We further discuss the limitations and advantages of approaches based on assays with population-level primary readouts, derived from single-parameter plate readers, or cell-level primary readouts, obtained using multiparametric flow cytometry or quantitative fluorescence microscopy (i.e., high-content screening). Finally, we discuss technical limitation and future perspectives, highlighting how the integration of screening data may lead to major advances in the field of stem cell research and therapy.

Machine Learning: Advanced Image Segmentation Using ilastik

Segmentation is one of the most ubiquitous problems in biological image analysis. Here we present a machine learning-based solution to it as implemented in the open source ilastik toolkit. We give a broad description of the underlying theory and demonstrate two workflows: Pixel Classification and Autocontext. We illustrate their use on a challenging problem in electron microscopy image segmentation. After following this walk-through, we expect the readers to be able to apply the necessary steps to their own data and segment their images by either workflow.