Advanced Cell Classifier: User-Friendly Machine-Learning-Based Software for Discovering Phenotypes in High-Content Imaging Data

Piximi - An Images to Discovery web tool for bioimages and beyond

Deep learning has greatly accelerated research in biological image analysis yet it often requires programming skills and specialized tool installation. Here we present Piximi, a modern, no-programming image analysis tool leveraging deep learning. Implemented as a web application at Piximi.app, Piximi requires no installation and can be accessed by any modern web browser. Its client-only architecture preserves the security of researcher data by running all computation locally. Piximi offers four core modules: a deep learning classifier, an image annotator, measurement modules, and pre-trained deep learning segmentation modules. Piximi is interoperable with existing tools and workflows by supporting import and export of common data and model formats. The intuitive researcher interface and easy access to Piximi allows biological researchers to obtain insights into images within just a few minutes. Piximi aims to bring deep learning-powered image analysis to a broader community by eliminating barriers to entry.

Cooperative polarization of MCAM/CD146 and ERM family proteins in melanoma

The WRAMP structure is a protein network associated with tail-end actomyosin contractility, membrane retraction, and directional persistence during cell migration. A marker of WRAMP structures is melanoma cell adhesion molecule (MCAM) which dynamically polarizes to the cell rear. However, factors that mediate MCAM polarization are still unknown. In this study, BioID using MCAM as bait identifies the ERM family proteins, moesin, ezrin, and radixin, as WRAMP structure components. We also present a novel image analysis pipeline, Protein Polarity by Percentile ("3P"), which classifies protein polarization using machine learning and facilitates quantitative analysis. Using 3P, we find that depletion of moesin, and to a lesser extent ezrin, decreases the proportion of cells with polarized MCAM. Furthermore, although copolarized MCAM and ERM proteins show high spatial overlap, 3P identifies subpopulations with ERM proteins closer to the cell periphery. Live-cell imaging confirms that MCAM and ERM protein polarization is tightly coordinated, but ERM proteins enrich at the cell edge first. Finally, deletion of a juxtamembrane segment in MCAM previously shown to promote ERM protein interactions impedes MCAM polarization. Our findings highlight the requirement for ERM proteins in recruitment of MCAM to WRAMP structures and an advanced computational tool to characterize protein polarization.

From pixels to insights: Machine learning and deep learning for bioimage analysis

Bioimage analysis plays a critical role in extracting information from biological images, enabling deeper insights into cellular structures and processes. The integration of machine learning and deep learning techniques has revolutionized the field, enabling the automated, reproducible, and accurate analysis of biological images. Here, we provide an overview of the history and principles of machine learning and deep learning in the context of bioimage analysis. We discuss the essential steps of the bioimage analysis workflow, emphasizing how machine learning and deep learning have improved preprocessing, segmentation, feature extraction, object tracking, and classification. We provide examples that showcase the application of machine learning and deep learning in bioimage analysis. We examine user-friendly software and tools that enable biologists to leverage these techniques without extensive computational expertise. This review is a resource for researchers seeking to incorporate machine learning and deep learning in their bioimage analysis workflows and enhance their research in this rapidly evolving field.

Image-based and machine learning-guided multiplexed serology test for SARS-CoV-2

We present a miniaturized immunofluorescence assay (mini-IFA) for measuring antibody response in patient blood samples. The method utilizes machine learning-guided image analysis and enables simultaneous measurement of immunoglobulin M (IgM), IgA, and IgG responses against different viral antigens in an automated and high-throughput manner. The assay relies on antigens expressed through transfection, enabling use at a low biosafety level and fast adaptation to emerging pathogens. Using severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) as the model pathogen, we demonstrate that this method allows differentiation between vaccine-induced and infection-induced antibody responses. Additionally, we established a dedicated web page for quantitative visualization of sample-specific results and their distribution, comparing them with controls and other samples. Our results provide a proof of concept for the approach, demonstrating fast and accurate measurement of antibody responses in a research setup with prospects for clinical diagnostics.

High-content imaging as a tool to quantify and characterize malaria parasites

In 2021, Plasmodium falciparum was responsible for 619,000 reported malaria-related deaths. Resistance has been detected to every clinically used antimalarial, urging the development of novel antimalarials with uncompromised mechanisms of actions. High-content imaging allows researchers to collect and quantify numerous phenotypic properties at the single-cell level, and machine learning-based approaches enable automated classification and clustering of cell populations. By combining these technologies, we developed a method capable of robustly differentiating and quantifying P. falciparum asexual blood stages. These phenotypic properties also allow for the quantification of changes in parasite morphology. Here, we demonstrate that our analysis can be used to quantify schizont nuclei, a phenotype that previously had to be enumerated manually. By monitoring stage progression and quantifying parasite phenotypes, our method can discern stage specificity of new compounds, thus providing insight into the compound's mode of action.

Phosphorylation of PACSIN2 at S313 Regulates Podocyte Architecture in Coordination with N-WASP

Changes in the dynamic architecture of podocytes, the glomerular epithelial cells, lead to kidney dysfunction. Previous studies on protein kinase C and casein kinase 2 substrates in neurons 2 (PACSIN2), a known regulator of endocytosis and cytoskeletal organization, reveal a connection between PACSIN2 and kidney pathogenesis. Here, we show that the phosphorylation of PACSIN2 at serine 313 (S313) is increased in the glomeruli of rats with diabetic kidney disease. We found that phosphorylation at S313 is associated with kidney dysfunction and increased free fatty acids rather than with high glucose and diabetes alone. Phosphorylation of PACSIN2 emerged as a dynamic process that fine-tunes cell morphology and cytoskeletal arrangement, in cooperation with the regulator of the actin cytoskeleton, Neural Wiskott-Aldrich syndrome protein (N-WASP). PACSIN2 phosphorylation decreased N-WASP degradation while N-WASP inhibition triggered PACSIN2 phosphorylation at S313. Functionally, pS313-PACSIN2 regulated actin cytoskeleton rearrangement depending on the type of cell injury and the signaling pathways involved. Collectively, this study indicates that N-WASP induces phosphorylation of PACSIN2 at S313, which serves as a mechanism whereby cells regulate active actin-related processes. The dynamic phosphorylation of S313 is needed to regulate cytoskeletal reorganization.

Automated high-content imaging in iPSC-derived neuronal progenitors

Induced pluripotent stem cells (iPSCs) have great potential as physiological disease models for human disorders where access to primary cells is difficult, such as neurons. In recent years, many protocols have been developed for the generation of iPSCs and the differentiation into specialised cell subtypes of interest. More recently, these models have been modified to allow large-scale phenotyping and high-content screening of small molecule compounds in iPSC-derived neuronal cells. Here, we describe the automated seeding of day 11 ventral midbrain progenitor cells into 96-well plates, administration of compounds, automated staining for immunofluorescence, the acquisition of images on a high-content screening platform and workflows for image analysis.

Correlative Fluorescence and Raman Microscopy to Define Mitotic Stages at the Single-Cell Level: Opportunities and Limitations in the AI Era

Nowadays, morphology and molecular analyses at the single-cell level have a fundamental role in understanding biology better. These methods are utilized for cell phenotyping and in-depth studies of cellular processes, such as mitosis. Fluorescence microscopy and optical spectroscopy techniques, including Raman micro-spectroscopy, allow researchers to examine biological samples at the single-cell level in a non-destructive manner. Fluorescence microscopy can give detailed morphological information about the localization of stained molecules, while Raman microscopy can produce label-free images at the subcellular level; thus, it can reveal the spatial distribution of molecular fingerprints, even in live samples. Accordingly, the combination of correlative fluorescence and Raman microscopy (CFRM) offers a unique approach for studying cellular stages at the single-cell level. However, subcellular spectral maps are complex and challenging to interpret. Artificial intelligence (AI) may serve as a valuable solution to characterize the molecular backgrounds of phenotypes and biological processes by finding the characteristic patterns in spectral maps. The major contributions of the manuscript are: (I) it gives a comprehensive review of the literature focusing on AI techniques in Raman-based cellular phenotyping; (II) via the presentation of a case study, a new neural network-based approach is described, and the opportunities and limitations of AI, specifically deep learning, are discussed regarding the analysis of Raman spectroscopy data to classify mitotic cellular stages based on their spectral maps.

Statistical Analysis of Multiplex Immunofluorescence and Immunohistochemistry Imaging Data

Advances in multiplexed single-cell immunofluorescence (mIF) and multiplex immunohistochemistry (mIHC) imaging technologies have enabled the analysis of cell-to-cell spatial relationships that promise to revolutionize our understanding of tissue-based diseases and autoimmune disorders. Multiplex images are collected as multichannel TIFF files; then denoised, segmented to identify cells and nuclei, normalized across slides with protein markers to correct for batch effects, and phenotyped; and then tissue composition and spatial context at the cellular level are analyzed. This chapter discusses methods and software infrastructure for image processing and statistical analysis of mIF/mIHC data.

Fisheye transformation enhances deep-learning-based single-cell phenotyping by including cellular microenvironment

Incorporating information about the surroundings can have a significant impact on successfully determining the class of an object. This is of particular interest when determining the phenotypes of cells, for example, in the context of high-throughput screens. We hypothesized that an ideal approach would consider the fully featured view of the cell of interest, include its neighboring microenvironment, and give lesser weight to cells that are far from the cell of interest. To satisfy these criteria, we present an approach with a transformation similar to those characteristic of fisheye cameras. Using this transformation with proper settings, we could significantly increase the accuracy of single-cell phenotyping, both in the case of cell culture and tissue-based microscopy images, and we present improved results on a dataset containing images of wild animals.

Unleashing high content screening in hit detection – benchmarking AI workflows including novelty detection

Complex mixtures containing natural products are still an interesting source of novel drug candidates. High content screening (HCS) is a popular tool to screen for such. In particular, multiplexed HCS assays promise comprehensive bioactivity profiles, but generate also high amounts of data. Yet, only some machine learning (ML) applications for data analysis are available and these usually require a profound knowledge of the underlying cell biology. Unfortunately, there are no applications that simply predict if samples are biologically active or not (any kind of bioactivity). Within this work, we benchmark ML algorithms for binary classification, starting with classical ML models, which are the standard classifiers of the scikit-learn library or ensemble models of these classifiers (a total of 92 models tested). Followed by a partial least square regression (PLSR)-based classification (44 tested models in total) and simple artificial neural networks (ANNs) with dense layers (72 tested models in total). In addition, a novelty detection (ND) was examined, which is supposed to handle unknown patterns. For the final analysis the models, with and without upstream ND, were tested with two independent data sets. In our analysis, a stacking model, an ensamble model of class ML algorithms, performed best to predict new and unknown data. ND improved the predictions of the models and was useful to handle unknown patterns. Importantly, the classifier presented here can be easily rebuilt and be adapted to the data and demands of other groups. The hit detector (ND + stacking model) is universal and suitable for a broader application to support the search for new drug candidates.

Segmentation, tracking and cell cycle analysis of live-cell imaging data with Cell-ACDC

Background

High-throughput live-cell imaging is a powerful tool to study dynamic cellular processes in single cells but creates a bottleneck at the stage of data analysis, due to the large amount of data generated and limitations of analytical pipelines. Recent progress on deep learning dramatically improved cell segmentation and tracking. Nevertheless, manual data validation and correction is typically still required and tools spanning the complete range of image analysis are still needed.

Results

We present Cell-ACDC, an open-source user-friendly GUI-based framework written in Python, for segmentation, tracking and cell cycle annotations. We included state-of-the-art deep learning models for single-cell segmentation of mammalian and yeast cells alongside cell tracking methods and an intuitive, semi-automated workflow for cell cycle annotation of single cells. Using Cell-ACDC, we found that mTOR activity in hematopoietic stem cells is largely independent of cell volume. By contrast, smaller cells exhibit higher p38 activity, consistent with a role of p38 in regulation of cell size. Additionally, we show that, in S. cerevisiae, histone Htb1 concentrations decrease with replicative age.

Conclusions

Cell-ACDC provides a framework for the application of state-of-the-art deep learning models to the analysis of live cell imaging data without programming knowledge. Furthermore, it allows for visualization and correction of segmentation and tracking errors as well as annotation of cell cycle stages. We embedded several smart algorithms that make the correction and annotation process fast and intuitive. Finally, the open-source and modularized nature of Cell-ACDC will enable simple and fast integration of new deep learning-based and traditional methods for cell segmentation, tracking, and downstream image analysis.

Source code: https://github.com/SchmollerLab/Cell_ACDC

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01372-6.

LiveCellMiner: A new tool to analyze mitotic progression

Live-cell imaging has become state of the art to accurately identify the nature of mitotic and cell cycle defects. Low- and high-throughput microscopy setups have yield huge data amounts of cells recorded in different experimental and pathological conditions. Tailored semi-automated and automated image analysis approaches allow the analysis of high-content screening data sets, saving time and avoiding bias. However, they were mostly designed for very specific experimental setups, which restricts their flexibility and usability. The general need for dedicated experiment-specific user-annotated training sets and experiment-specific user-defined segmentation parameters remains a major bottleneck for fully automating the analysis process. In this work we present LiveCellMiner, a highly flexible open-source software tool to automatically extract, analyze and visualize both aggregated and time-resolved image features with potential biological relevance. The software tool allows analysis across high-content data sets obtained in different platforms, in a quantitative and unbiased manner. As proof of principle application, we analyze here the dynamic chromatin and tubulin cytoskeleton features in human cells passing through mitosis highlighting the versatile and flexible potential of this tool set.

Precise measurement of nanoscopic septin ring structures with deep learning-assisted quantitative superresolution microscopy

The combination of image analysis and superresolution microscopy methods allows for unprecedented insight into the organization of macromolecular assemblies in cells. Advances in deep learning (DL)-based object recognition enable the automated processing of large amounts of data, resulting in high accuracy through averaging. However, while the analysis of highly symmetric structures of constant size allows for a resolution approaching the dimensions of structural biology, DL-based image recognition may introduce bias. This prohibits the development of readouts for processes that involve significant changes in size or shape of amorphous macromolecular complexes. Here we address this problem by using changes of septin ring structures in single molecule localization-based superresolution microscopy data as a paradigm. We identify potential sources of bias resulting from different training approaches by rigorous testing of trained models using real or simulated data covering a wide range of possible results. In a quantitative comparison of our models, we find that a trade-off exists between measurement accuracy and the range of recognized phenotypes. Using our thus verified models, we find that septin ring size can be explained by the number of subunits they are assembled from alone. Furthermore, we provide a new experimental system for the investigation of septin polymerization.

Smart imaging to empower brain-wide neuroscience at single-cell levels

A deep understanding of the neuronal connectivity and networks with detailed cell typing across brain regions is necessary to unravel the mechanisms behind the emotional and memorial functions as well as to find the treatment of brain impairment. Brain-wide imaging with single-cell resolution provides unique advantages to access morphological features of a neuron and to investigate the connectivity of neuron networks, which has led to exciting discoveries over the past years based on animal models, such as rodents. Nonetheless, high-throughput systems are in urgent demand to support studies of neural morphologies at larger scale and more detailed level, as well as to enable research on non-human primates (NHP) and human brains. The advances in artificial intelligence (AI) and computational resources bring great opportunity to 'smart' imaging systems, i.e., to automate, speed up, optimize and upgrade the imaging systems with AI and computational strategies. In this light, we review the important computational techniques that can support smart systems in brain-wide imaging at single-cell resolution.

Proteome-wide landscape of solubility limits in a bacterial cell

Proteins are prone to aggregate when expressed above their solubility limits. Aggregation may occur rapidly, potentially as early as proteins emerge from the ribosome, or slowly, following synthesis. However, in vivo data on aggregation rates are scarce. Here, we classified the Escherichia coli proteome into rapidly and slowly aggregating proteins using an in vivo image-based screen coupled with machine learning. We find that the majority (70%) of cytosolic proteins that become insoluble upon overexpression have relatively low rates of aggregation and are unlikely to aggregate co-translationally. Remarkably, such proteins exhibit higher folding rates compared to rapidly aggregating proteins, potentially implying that they aggregate after reaching their folded states. Furthermore, we find that a substantial fraction (~ 35%) of the proteome remain soluble at concentrations much higher than those found naturally, indicating a large margin of safety to tolerate gene expression changes. We show that high disorder content and low surface stickiness are major determinants of high solubility and are favored in abundant bacterial proteins. Overall, our study provides a global view of aggregation rates and hence solubility limits of proteins in a bacterial cell.

Genome-wide RNAi screen identifies novel players in human 60S subunit biogenesis including key enzymes of polyamine metabolism

Ribosome assembly is an essential process that is linked to human congenital diseases and tumorigenesis. While great progress has been made in deciphering mechanisms governing ribosome biogenesis in eukaryotes, an inventory of factors that support ribosome synthesis in human cells is still missing, in particular regarding the maturation of the large 60S subunit. Here, we performed a genome-wide RNAi screen using an imaging-based, single cell assay to unravel the cellular machinery promoting 60S subunit assembly in human cells. Our screen identified a group of 310 high confidence factors. These highlight the conservation of the process across eukaryotes and reveal the intricate connectivity of 60S subunit maturation with other key cellular processes, including splicing, translation, protein degradation, chromatin organization and transcription. Intriguingly, we also identified a cluster of hits comprising metabolic enzymes of the polyamine synthesis pathway. We demonstrate that polyamines, which have long been used as buffer additives to support ribosome assembly in vitro, are required for 60S maturation in living cells. Perturbation of polyamine metabolism results in early defects in 60S but not 40S subunit maturation. Collectively, our data reveal a novel function for polyamines in living cells and provide a rich source for future studies on ribosome synthesis.

A digital pathology tool for quantification of color features in histologic specimens

In preclinical research, histological analysis of tissue samples is often limited to qualitative or semiquantitative scoring assessments. The reliability of this analysis can be impaired by the subjectivity of these approaches, even when read by experienced pathologists. Furthermore, the laborious nature of manual image assessments often leads to the analysis being restricted to a relatively small number of images that may not accurately represent the whole sample. Thus, there is a clear need for automated image analysis tools that can provide robust and rapid quantification of histologic samples from paraffin-embedded or cryopreserved tissues. To address this need, we have developed a color image analysis algorithm (DigiPath) to quantify distinct color features in histologic sections. We demonstrate the utility of this tool across multiple types of tissue samples and pathologic features, and compare results from our program to other quantitative approaches such as color thresholding and hand tracing. We believe this tool will enable more thorough and reliable characterization of histological samples to facilitate better rigor and reproducibility in tissue-based analyses.

Investigation of Different Free Image Analysis Software for High-Throughput Droplet Detection

Droplet microfluidics has revealed innovative strategies in biology and chemistry. This advancement has delivered novel quantification methods, such as droplet digital polymerase chain reaction (ddPCR) and an antibiotic heteroresistance analysis tool. For droplet analysis, researchers often use image-based detection techniques. Unfortunately, the analysis of images may require specific tools or programming skills to produce the expected results. In order to address the issue, we explore the potential use of standalone freely available software to perform image-based droplet detection. We select the four most popular software and classify them into rule-based and machine learning-based types after assessing the software's modules. We test and evaluate the software's (i) ability to detect droplets, (ii) accuracy and precision, and (iii) overall components and supporting material. In our experimental setting, we find that the rule-based type of software is better suited for image-based droplet detection. The rule-based type of software also has a simpler workflow or pipeline, especially aimed for non-experienced users. In our case, CellProfiler (CP) offers the most user-friendly experience for both single image and batch processing analyses.

CellProfiler Analyst 3.0: Accessible data exploration and machine learning for image analysis

 

Image-based experiments can yield many thousands of individual measurements describing each object of interest, such as cells in microscopy screens. CellProfiler Analyst is a free, open-source software package designed for the exploration of quantitative image-derived data and the training of machine learning classifiers with an intuitive user interface. We have now released CellProfiler Analyst 3.0, which in addition to enhanced performance adds support for neural network classifiers, identifying rare object subsets, and direct transfer of objects of interest from visualisation tools into the Classifier tool for use as training data. This release also increases interoperability with the recently released CellProfiler 4, making it easier for users to detect and measure particular classes of objects in their analyses.

AVAILABILITY

CellProfiler Analyst binaries for Windows and MacOS are freely available for download at https://cellprofileranalyst.org/. Source code is implemented in Python 3 and is available at https://github.com/CellProfiler/CellProfiler-Analyst/. A sample data set is available at https://cellprofileranalyst.org/examples, based on images freely available from the Broad Bioimage Benchmark Collection (BBBC).

The human melanoma proteome atlas-Defining the molecular pathology

The MM500 study is an initiative to map the protein levels in malignant melanoma tumor samples, focused on in-depth histopathology coupled to proteome characterization. The protein levels and localization were determined for a broad spectrum of diverse, surgically isolated melanoma tumors originating from multiple body locations. More than 15,500 proteoforms were identified by mass spectrometry, from which chromosomal and subcellular localization was annotated within both primary and metastatic melanoma. The data generated by global proteomic experiments covered 72% of the proteins identified in the recently reported high stringency blueprint of the human proteome. This study contributes to the NIH Cancer Moonshot initiative combining detailed histopathological presentation with the molecular characterization for 505 melanoma tumor samples, localized in 26 organs from 232 patients.

Emerging machine learning approaches to phenotyping cellular motility and morphodynamics

Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.

Regression plane concept for analysing continuous cellular processes with machine learning

Biological processes are inherently continuous, and the chance of phenotypic discovery is significantly restricted by discretising them. Using multi-parametric active regression we introduce the Regression Plane (RP), a user-friendly discovery tool enabling class-free phenotypic supervised machine learning, to describe and explore biological data in a continuous manner. First, we compare traditional classification with regression in a simulated experimental setup. Second, we use our framework to identify genes involved in regulating triglyceride levels in human cells. Subsequently, we analyse a time-lapse dataset on mitosis to demonstrate that the proposed methodology is capable of modelling complex processes at infinite resolution. Finally, we show that hemocyte differentiation in Drosophila melanogaster has continuous characteristics.

DeepMIB: User-friendly and open-source software for training of deep learning network for biological image segmentation

We present DeepMIB, a new software package that is capable of training convolutional neural networks for segmentation of multidimensional microscopy datasets on any workstation. We demonstrate its successful application for segmentation of 2D and 3D electron and multicolor light microscopy datasets with isotropic and anisotropic voxels. We distribute DeepMIB as both an open-source multi-platform Matlab code and as compiled standalone application for Windows, MacOS and Linux. It comes in a single package that is simple to install and use as it does not require knowledge of programming. DeepMIB is suitable for everyone interested of bringing a power of deep learning into own image segmentation workflows.

Convolutional Neural Network-Based Artificial Intelligence for Classification of Protein Localization Patterns

Identifying localization of proteins and their specific subpopulations associated with certain cellular compartments is crucial for understanding protein function and interactions with other macromolecules. Fluorescence microscopy is a powerful method to assess protein localizations, with increasing demand of automated high throughput analysis methods to supplement the technical advancements in high throughput imaging. Here, we study the applicability of deep neural network-based artificial intelligence in classification of protein localization in 13 cellular subcompartments. We use deep learning-based on convolutional neural network and fully convolutional network with similar architectures for the classification task, aiming at achieving accurate classification, but importantly, also comparison of the networks. Our results show that both types of convolutional neural networks perform well in protein localization classification tasks for major cellular organelles. Yet, in this study, the fully convolutional network outperforms the convolutional neural network in classification of images with multiple simultaneous protein localizations. We find that the fully convolutional network, using output visualizing the identified localizations, is a very useful tool for systematic protein localization assessment.

A robust unsupervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei

Cell morphology encodes essential information on many underlying biological processes. It is commonly used by clinicians and researchers in the study, diagnosis, prognosis, and treatment of human diseases. Quantification of cell morphology has seen tremendous advances in recent years. However, effectively defining morphological shapes and evaluating the extent of morphological heterogeneity within cell populations remain challenging. Here we present a protocol and software for the analysis of cell and nuclear morphology from fluorescence or bright-field images using the VAMPIRE algorithm ( https://github.com/kukionfr/VAMPIRE_open ). This algorithm enables the profiling and classification of cells into shape modes based on equidistant points along cell and nuclear contours. Examining the distributions of cell morphologies across automatically identified shape modes provides an effective visualization scheme that relates cell shapes to cellular subtypes based on endogenous and exogenous cellular conditions. In addition, these shape mode distributions offer a direct and quantitative way to measure the extent of morphological heterogeneity within cell populations. This protocol is highly automated and fast, with the ability to quantify the morphologies from 2D projections of cells seeded both on 2D substrates or embedded within 3D microenvironments, such as hydrogels and tissues. The complete analysis pipeline can be completed within 60 minutes for a dataset of ~20,000 cells/2,400 images.

High-Content Imaging to Phenotype Human Primary and iPSC-Derived Cells

Increasingly powerful microscopy, liquid handling, and computational techniques have enabled cell imaging in high throughput. Microscopy images are quantified using high-content analysis platforms linking object features to cell behavior. This can be attempted on physiologically relevant cell models, including stem cells and primary cells, in complex environments, and conceivably in the presence of perturbations. Recently, substantial focus has been devoted to cell profiling for cell therapy, assays for drug discovery or biomarker identification for clinical decision-making protocols, bringing this wealth of information into translational applications. In this chapter, we focus on two protocols enabling to (1) benchmark human cells, in particular human endothelial cells as a case study and (2) extract cells from blood for follow-up experiments including image-based drug testing. We also present concepts of high-content imaging and discuss the benefits and challenges, with the aim of enabling readers to tailor existing pipelines and bring such approaches closer to translational research and the clinic.

Neuropilin-1 is a host factor for SARS-CoV-2 infection

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 (COVID-19), uses the viral spike (S) protein for host cell attachment and entry. The host protease furin cleaves the full-length precursor S glycoprotein into two associated polypeptides: S1 and S2. Cleavage of S generates a polybasic Arg-Arg-Ala-Arg carboxyl-terminal sequence on S1, which conforms to a C-end rule (CendR) motif that binds to cell surface neuropilin-1 (NRP1) and NRP2 receptors. We used x-ray crystallography and biochemical approaches to show that the S1 CendR motif directly bound NRP1. Blocking this interaction by RNA interference or selective inhibitors reduced SARS-CoV-2 entry and infectivity in cell culture. NRP1 thus serves as a host factor for SARS-CoV-2 infection and may potentially provide a therapeutic target for COVID-19.

Hypothesis-driven quantitative fluorescence microscopy - the importance of reverse-thinking in experimental design

One of the challenges in modern fluorescence microscopy is to reconcile the conventional utilization of microscopes as exploratory instruments with their emerging and rapidly expanding role as a quantitative tools. The contribution of microscopy to observational biology will remain enormous owing to the improvements in acquisition speed, imaging depth, resolution and biocompatibility of modern imaging instruments. However, the use of fluorescence microscopy to facilitate the quantitative measurements necessary to challenge hypotheses is a relatively recent concept, made possible by advanced optics, functional imaging probes and rapidly increasing computational power. We argue here that to fully leverage the rapidly evolving application of microscopes in hypothesis-driven biology, we not only need to ensure that images are acquired quantitatively but must also re-evaluate how microscopy-based experiments are designed. In this Opinion, we present a reverse logic that guides the design of quantitative fluorescence microscopy experiments. This unique approach starts from identifying the results that would quantitatively inform the hypothesis and map the process backward to microscope selection. This ensures that the quantitative aspects of testing the hypothesis remain the central focus of the entire experimental design.

Neuropilin-1 is a host factor for SARS-CoV-2 infection

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 (COVID-19), uses the viral spike (S) protein for host cell attachment and entry. The host protease furin cleaves the full-length precursor S glycoprotein into two associated polypeptides: S1 and S2. Cleavage of S generates a polybasic Arg-Arg-Ala-Arg carboxyl-terminal sequence on S1, which conforms to a C-end rule (CendR) motif that binds to cell surface neuropilin-1 (NRP1) and NRP2 receptors. We used x-ray crystallography and biochemical approaches to show that the S1 CendR motif directly bound NRP1. Blocking this interaction by RNA interference or selective inhibitors reduced SARS-CoV-2 entry and infectivity in cell culture. NRP1 thus serves as a host factor for SARS-CoV-2 infection and may potentially provide a therapeutic target for COVID-19.

Keras R-CNN: library for cell detection in biological images using deep neural networks

BACKGROUND

A common yet still manual task in basic biology research, high-throughput drug screening and digital pathology is identifying the number, location, and type of individual cells in images. Object detection methods can be useful for identifying individual cells as well as their phenotype in one step. State-of-the-art deep learning for object detection is poised to improve the accuracy and efficiency of biological image analysis.

RESULTS

We created Keras R-CNN to bring leading computational research to the everyday practice of bioimage analysts. Keras R-CNN implements deep learning object detection techniques using Keras and Tensorflow ( https://github.com/broadinstitute/keras-rcnn ). We demonstrate the command line tool's simplified Application Programming Interface on two important biological problems, nucleus detection and malaria stage classification, and show its potential for identifying and classifying a large number of cells. For malaria stage classification, we compare results with expert human annotators and find comparable performance.

CONCLUSIONS

Keras R-CNN is a Python package that performs automated cell identification for both brightfield and fluorescence images and can process large image sets. Both the package and image datasets are freely available on GitHub and the Broad Bioimage Benchmark Collection.

Probing the characteristics and biofunctional effects of disease-affected cells and drug response via machine learning applications

Drug-induced transformations in disease characteristics at the cellular and molecular level offers the opportunity to predict and evaluate the efficacy of pharmaceutical ingredients whilst enabling the optimal design of new and improved drugs with enhanced pharmacokinetics and pharmacodynamics. Machine learning is a promising in-silico tool used to simulate cells with specific disease properties and to determine their response toward drug uptake. Differences in the properties of normal and infected cells, including biophysical, biochemical and physiological characteristics, plays a key role in developing fundamental cellular probing platforms for machine learning applications. Cellular features can be extracted periodically from both the drug treated, infected, and normal cells via image segmentations in order to probe dynamic differences in cell behavior. Cellular segmentation can be evaluated to reflect the levels of drug effect on a distinct cell or group of cells via probability scoring. This article provides an account for the use of machine learning methods to probe differences in the biophysical, biochemical and physiological characteristics of infected cells in response to pharmacokinetics uptake of drug ingredients for application in cancer, diabetes and neurodegenerative disease therapies.

Image-based high-content screening in drug discovery

While target-based drug discovery strategies rely on the precise knowledge of the identity and function of the drug targets, phenotypic drug discovery (PDD) approaches allow the identification of novel drugs based on knowledge of a distinct phenotype. Image-based high-content screening (HCS) is a potent PDD strategy that characterizes small-molecule effects through the quantification of features that depict cellular changes among or within cell populations, thereby generating valuable data sets for subsequent data analysis. However, these data can be complex, making image analysis from large HCS campaigns challenging. Technological advances in image acquisition, processing, and analysis as well as machine-learning (ML) approaches for the analysis of multidimensional data sets have rendered HCS as a viable technology for small-molecule drug discovery. Here, we discuss HCS concepts, current workflows as well as opportunities and challenges of image-based phenotypic screening and data analysis.

Software tools for 3D nuclei segmentation and quantitative analysis in multicellular aggregates

Today, we are fully immersed into the era of 3D biology. It has been extensively demonstrated that 3D models: (a) better mimic the physiology of human tissues; (b) can effectively replace animal models; (c) often provide more reliable results than 2D ones. Accordingly, anti-cancer drug screenings and toxicology studies based on multicellular 3D biological models, the so-called "-oids" (e.g. spheroids, tumoroids, organoids), are blooming in the literature. However, the complex nature of these systems limit the manual quantitative analyses of single cells' behaviour in the culture. Accordingly, the demand for advanced software tools that are able to perform phenotypic analysis is fundamental. In this work, we describe the freely accessible tools that are currently available for biologists and researchers interested in analysing the effects of drugs/treatments on 3D multicellular -oids at a single-cell resolution level. In addition, using publicly available nuclear stained datasets we quantitatively compare the segmentation performance of 9 specific tools.

The palette of techniques for cell cycle analysis

The cell division cycle is the generational period of cellular growth and propagation. Cell cycle progression needs to be highly regulated to preserve genomic fidelity while increasing cell number. In multicellular organisms, the cell cycle must also coordinate with cell fate specification during development and tissue homeostasis. Altered cell cycle dynamics play a central role also in a number of pathophysiological processes. Thus, extensive effort has been made to define the biochemical machineries that execute the cell cycle and their regulation, as well as implementing more sensitive and accurate cell cycle measurements. Here, we review the available techniques for cell cycle analysis, revisiting the assumptions behind conventional population-based measurements and discussing new tools to better address cell cycle heterogeneity in the single-cell era. We weigh the strengths, weaknesses, and trade-offs of methods designed to measure temporal aspects of the cell cycle. Finally, we discuss emerging techniques for capturing cell cycle speed at single-cell resolution in live animals.

Combining Supervised and Unsupervised Machine Learning Methods for Phenotypic Functional Genomics Screening

There has been an increase in the use of machine learning and artificial intelligence (AI) for the analysis of image-based cellular screens. The accuracy of these analyses, however, is greatly dependent on the quality of the training sets used for building the machine learning models. We propose that unsupervised exploratory methods should first be applied to the data set to gain a better insight into the quality of the data. This improves the selection and labeling of data for creating training sets before the application of machine learning. We demonstrate this using a high-content genome-wide small interfering RNA screen. We perform an unsupervised exploratory data analysis to facilitate the identification of four robust phenotypes, which we subsequently use as a training set for building a high-quality random forest machine learning model to differentiate four phenotypes with an accuracy of 91.1% and a kappa of 0.85. Our approach enhanced our ability to extract new knowledge from the screen when compared with the use of unsupervised methods alone.

Superpixel-Based Conditional Random Fields (SuperCRF): Incorporating Global and Local Context for Enhanced Deep Learning in Melanoma Histopathology

Computational pathology-based cell classification algorithms are revolutionizing the study of the tumor microenvironment and can provide novel predictive/prognosis biomarkers crucial for the delivery of precision oncology. Current algorithms used on hematoxylin and eosin slides are based on individual cell nuclei morphology with limited local context features. Here, we propose a novel multi-resolution hierarchical framework (SuperCRF) inspired by the way pathologists perceive regional tissue architecture to improve cell classification and demonstrate its clinical applications. We develop SuperCRF by training a state-of-art deep learning spatially constrained- convolution neural network (SC-CNN) to detect and classify cells from 105 high-resolution (20×) H&E-stained slides of The Cancer Genome Atlas melanoma dataset and subsequently, a conditional random field (CRF) by combining cellular neighborhood with tumor regional classification from lower resolution images (5, 1.25×) given by a superpixel-based machine learning framework. SuperCRF led to an 11.85% overall improvement in the accuracy of the state-of-art deep learning SC-CNN cell classifier. Consistent with a stroma-mediated immune suppressive microenvironment, SuperCRF demonstrated that (i) a high ratio of lymphocytes to all lymphocytes within the stromal compartment (p = 0.026) and (ii) a high ratio of stromal cells to all cells (p < 0.0001 compared to p = 0.039 for SC-CNN only) are associated with poor survival in patients with melanoma. SuperCRF improves cell classification by introducing global and local context-based information and can be implemented in combination with any single-cell classifier. SuperCRF provides valuable tools to study the tumor microenvironment and identify predictors of survival and response to therapy.

Phenotypic Image Analysis Software Tools for Exploring and Understanding Big Image Data from Cell-Based Assays

Phenotypic image analysis is the task of recognizing variations in cell properties using microscopic image data. These variations, produced through a complex web of interactions between genes and the environment, may hold the key to uncover important biological phenomena or to understand the response to a drug candidate. Today, phenotypic analysis is rarely performed completely by hand. The abundance of high-dimensional image data produced by modern high-throughput microscopes necessitates computational solutions. Over the past decade, a number of software tools have been developed to address this need. They use statistical learning methods to infer relationships between a cell's phenotype and data from the image. In this review, we examine the strengths and weaknesses of non-commercial phenotypic image analysis software, cover recent developments in the field, identify challenges, and give a perspective on future possibilities.

Imaging, Visualization, and Computation in Developmental Biology

Embryonic development is highly complex and dynamic, requiring the coordination of numerous molecular and cellular events at precise times and places. Advances in imaging technology have made it possible to follow developmental processes at cellular, tissue, and organ levels over time as they take place in the intact embryo. Parallel innovations of in vivo probes permit imaging to report on molecular, physiological, and anatomical events of embryogenesis, but the resulting multidimensional data sets pose significant challenges for extracting knowledge. In this review, we discuss recent and emerging advances in imaging technologies, in vivo labeling, and data processing that offer the greatest potential for jointly deciphering the intricate cellular dynamics and the underlying molecular mechanisms. Our discussion of the emerging area of “image-omics” highlights both the challenges of data analysis and the promise of more fully embracing computation and data science for rapidly advancing our understanding of biology.

Techniques for Studying Decoding of Single Cell Dynamics

Cells must be able to interpret signals they encounter and reliably generate an appropriate response. It has long been known that the dynamics of transcription factor and kinase activation can play a crucial role in selecting an individual cell's response. The study of cellular dynamics has expanded dramatically in the last few years, with dynamics being discovered in novel pathways, new insights being revealed about the importance of dynamics, and technological improvements increasing the throughput and capabilities of single cell measurements. In this review, we highlight the important developments in this field, with a focus on the methods used to make new discoveries. We also include a discussion on improvements in methods for engineering and measuring single cell dynamics and responses. Finally, we will briefly highlight some of the many challenges and avenues of research that are still open.

eDetect: A Fast Error Detection and Correction Tool for Live Cell Imaging Data Analysis

Live cell imaging has been widely used to generate data for quantitative understanding of cellular dynamics. Various applications have been developed to perform automated imaging data analysis, which often requires tedious manual correction. It remains a challenge to develop an efficient curation method that can analyze massive imaging datasets with high accuracy. Here, we present eDetect, a fast error detection and correction tool that provides a powerful and convenient solution for the curation of live cell imaging analysis results. In eDetect, we propose a gating strategy to distinguish correct and incorrect image analysis results by visualizing image features based on principal component analysis. We demonstrate that this approach can substantially accelerate the data correction process and improve the accuracy of imaging data analysis. eDetect is well documented and designed to be user friendly for non-expert users. It is freely available at https://sites.google.com/view/edetect/ and https://github.com/Zi-Lab/eDetect.

Environmental properties of cells improve machine learning-based phenotype recognition accuracy

To answer major questions of cell biology, it is often essential to understand the complex phenotypic composition of cellular systems precisely. Modern automated microscopes produce vast amounts of images routinely, making manual analysis nearly impossible. Due to their efficiency, machine learning-based analysis software have become essential tools to perform single-cell-level phenotypic analysis of large imaging datasets. However, an important limitation of such methods is that they do not use the information gained from the cellular micro- and macroenvironment: the algorithmic decision is based solely on the local properties of the cell of interest. Here, we present how various features from the surrounding environment contribute to identifying a cell and how such additional information can improve single-cell-level phenotypic image analysis. The proposed methodology was tested for different sizes of Euclidean and nearest neighbour-based cellular environments both on tissue sections and cell cultures. Our experimental data verify that the surrounding area of a cell largely determines its entity. This effect was found to be especially strong for established tissues, while it was somewhat weaker in the case of cell cultures. Our analysis shows that combining local cellular features with the properties of the cell’s neighbourhood significantly improves the accuracy of machine learning-based phenotyping.

Development of image analysis software for quantification of viable cells in microchips

Over the past few years, image analysis has emerged as a powerful tool for analyzing various cell biology parameters in an unprecedented and highly specific manner. The amount of data that is generated requires automated methods for the processing and analysis of all the resulting information. The software available so far are suitable for the processing of fluorescence and phase contrast images, but often do not provide good results from transmission light microscopy images, due to the intrinsic variation of the acquisition of images technique itself (adjustment of brightness / contrast, for instance) and the variability between image acquisition introduced by operators / equipment. In this contribution, it has been presented an image processing software, Python based image analysis for cell growth (PIACG), that is able to calculate the total area of the well occupied by cells with fusiform and rounded morphology in response to different concentrations of fetal bovine serum in microfluidic chips, from microscopy images in transmission light, in a highly efficient way.

Hsp70-associated chaperones have a critical role in buffering protein production costs

Proteins are necessary for cellular growth. Concurrently, however, protein production has high energetic demands associated with transcription and translation. Here, we propose that activity of molecular chaperones shape protein burden, that is the fitness costs associated with expression of unneeded proteins. To test this hypothesis, we performed a genome-wide genetic interaction screen in baker's yeast. Impairment of transcription, translation, and protein folding rendered cells hypersensitive to protein burden. Specifically, deletion of specific regulators of the Hsp70-associated chaperone network increased protein burden. In agreement with expectation, temperature stress, increased mistranslation and a chemical misfolding agent all substantially enhanced protein burden. Finally, unneeded protein perturbed interactions between key components of the Hsp70-Hsp90 network involved in folding of native proteins. We conclude that specific chaperones contribute to protein burden. Our work indicates that by minimizing the damaging impact of gratuitous protein overproduction, chaperones enable tolerance to massive changes in genomic expression.

Intelligent image-based in situ single-cell isolation

Quantifying heterogeneities within cell populations is important for many fields including cancer research and neurobiology; however, techniques to isolate individual cells are limited. Here, we describe a high-throughput, non-disruptive, and cost-effective isolation method that is capable of capturing individually targeted cells using widely available techniques. Using high-resolution microscopy, laser microcapture microscopy, image analysis, and machine learning, our technology enables scalable molecular genetic analysis of single cells, targetable by morphology or location within the sample.

The isolation of single cells while retaining context is important for quantifying cellular heterogeneity but technically challenging. Here, the authors develop a high-throughput, scalable workflow for microscopy-based single cell isolation using machine-learning, high-throughput microscopy and laser capture microdissection.