Cellpose: a generalist algorithm for cellular segmentation

Dynamic FRET-FLIM based screening of signal transduction pathways

Fluorescence Lifetime Imaging (FLIM) is an intrinsically quantitative method to screen for protein-protein interactions and is frequently used to record the outcome of signal transduction events. With new highly sensitive and photon efficient FLIM instrumentation, the technique also becomes attractive to screen, with high temporal resolution, for fast changes in Förster Resonance Energy Transfer (FRET), such as those occurring upon activation of cell signaling. The second messenger cyclic adenosine monophosphate (cAMP) is rapidly formed following activation of certain cell surface receptors. cAMP is subsequently degraded by a set of phosphodiesterases (PDEs) which display cell-type specific expression and may also affect baseline levels of the messenger. To study which specific PDEs contribute most to cAMP regulation, we knocked down individual PDEs and recorded breakdown rates of cAMP levels following transient stimulation in HeLa cells stably expressing the FRET/FLIM sensor, Epac-SH189. Many hundreds of cells were recorded at 5 s intervals for each condition. FLIM time traces were calculated for every cell, and decay kinetics were obtained. cAMP clearance was significantly slower when PDE3A and, to a lesser amount, PDE10A were knocked down, identifying these isoforms as dominant in HeLa cells. However, taking advantage of the quantitative FLIM data, we found that knockdown of individual PDEs has a very limited effect on baseline cAMP levels. By combining photon-efficient FLIM instrumentation with optimized sensors, systematic gene knockdown and an automated open-source analysis pipeline, our study demonstrates that dynamic screening of transient cell signals has become feasible. The quantitative platform described here provides detailed kinetic analysis of cellular signals in individual cells with unprecedented throughput.

Graph-Theoretic Post-Processing of Segmentation With Application to Dense Biofilms

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

Plexin-B2 orchestrates collective stem cell dynamics via actomyosin contractility, cytoskeletal tension and adhesion

During morphogenesis, molecular mechanisms that orchestrate biomechanical dynamics across cells remain unclear. Here, we show a role of guidance receptor Plexin-B2 in organizing actomyosin network and adhesion complexes during multicellular development of human embryonic stem cells and neuroprogenitor cells. Plexin-B2 manipulations affect actomyosin contractility, leading to changes in cell stiffness and cytoskeletal tension, as well as cell-cell and cell-matrix adhesion. We have delineated the functional domains of Plexin-B2, RAP1/2 effectors, and the signaling association with ERK1/2, calcium activation, and YAP mechanosensor, thus providing a mechanistic link between Plexin-B2-mediated cytoskeletal tension and stem cell physiology. Plexin-B2-deficient stem cells exhibit premature lineage commitment, and a balanced level of Plexin-B2 activity is critical for maintaining cytoarchitectural integrity of the developing neuroepithelium, as modeled in cerebral organoids. Our studies thus establish a significant function of Plexin-B2 in orchestrating cytoskeletal tension and cell-cell/cell-matrix adhesion, therefore solidifying the importance of collective cell mechanics in governing stem cell physiology and tissue morphogenesis.

Cell segmentation in imaging-based spatial transcriptomics

Single-molecule spatial transcriptomics protocols based on in situ sequencing or multiplexed RNA fluorescent hybridization can reveal detailed tissue organization. However, distinguishing the boundaries of individual cells in such data is challenging and can hamper downstream analysis. Current methods generally approximate cells positions using nuclei stains. We describe a segmentation method, Baysor, that optimizes two-dimensional (2D) or three-dimensional (3D) cell boundaries considering joint likelihood of transcriptional composition and cell morphology. While Baysor can take into account segmentation based on co-stains, it can also perform segmentation based on the detected transcripts alone. To evaluate performance, we extend multiplexed error-robust fluorescence in situ hybridization (MERFISH) to incorporate immunostaining of cell boundaries. Using this and other benchmarks, we show that Baysor segmentation can, in some cases, nearly double the number of cells compared to existing tools while reducing segmentation artifacts. We demonstrate that Baysor performs well on data acquired using five different protocols, making it a useful general tool for analysis of imaging-based spatial transcriptomics.

Application of Machine Learning Techniques to an Agent-Based Model of Pantoea

Agent-based modeling (ABM) is a powerful simulation technique which describes a complex dynamic system based on its interacting constituent entities. While the flexibility of ABM enables broad application, the complexity of real-world models demands intensive computing resources and computational time; however, a metamodel may be constructed to gain insight at less computational expense. Here, we developed a model in NetLogo to describe the growth of a microbial population consisting of Pantoea. We applied 13 parameters that defined the model and actively changed seven of the parameters to modulate the evolution of the population curve in response to these changes. We efficiently performed more than 3,000 simulations using a Python wrapper, NL4Py. Upon evaluation of the correlation between the active parameters and outputs by random forest regression, we found that the parameters which define the depth of medium and glucose concentration affect the population curves significantly. Subsequently, we constructed a metamodel, a dense neural network, to predict the simulation outputs from the active parameters and found that it achieves high prediction accuracy, reaching an R² coefficient of determination value up to 0.92. Our approach of using a combination of ABM with random forest regression and neural network reduces the number of required ABM simulations. The simplified and refined metamodels may provide insights into the complex dynamic system before their transition to more sophisticated models that run on high-performance computing systems. The ultimate goal is to build a bridge between simulation and experiment, allowing model validation by comparing the simulated data to experimental data in microbiology.

NuMorph: Tools for cortical cellular phenotyping in tissue-cleared whole-brain images

Tissue-clearing methods allow every cell in the mouse brain to be imaged without physical sectioning. However, the computational tools currently available for cell quantification in cleared tissue images have been limited to counting sparse cell populations in stereotypical mice. Here, we introduce NuMorph, a group of analysis tools to quantify all nuclei and nuclear markers within the mouse cortex after clearing and imaging by light-sheet microscopy. We apply NuMorph to investigate two distinct mouse models: a Topoisomerase 1 (Top1) model with severe neurodegenerative deficits and a Neurofibromin 1 (Nf1) model with a more subtle brain overgrowth phenotype. In each case, we identify differential effects of gene deletion on individual cell-type counts and distribution across cortical regions that manifest as alterations of gross brain morphology. These results underline the value of whole-brain imaging approaches, and the tools are widely applicable for studying brain structure phenotypes at cellular resolution.

IFITM1 expression determines extracellular vesicle uptake in colorectal cancer

The majority of colorectal cancer (CRC) patients carry mutations in the APC gene, which lead to the unregulated activation of the Wnt pathway. Extracellular vesicles (EV) are considered potential therapeutic tools. Although CRC is a genetically heterogeneous disease, the significance of the intra-tumor heterogeneity in EV uptake of CRC cells is not yet known. By using mouse and patient-derived organoids, the currently available best model of capturing cellular heterogeneity, we found that Apc mutation induced the expression of interferon-induced transmembrane protein 1 (Ifitm1), a membrane protein that plays a major role in cellular antiviral responses. Importantly, organoids derived from IFITM1^high CRC cells contained more proliferating cells and they had a markedly reduced uptake of fibroblast EVs as compared to IFITM1^low/- cells. In contrast, there was no difference in the intensity of EV release between CRC subpopulations with high and low IFITM1 levels. Importantly, the difference in cell proliferation between these two subpopulations disappeared in the presence of fibroblast-derived EVs, proving the functional relevance of the enhanced EV uptake by IFITM1^low CRC cells. Furthermore, inactivating IFITM1 resulted in an enhanced EV uptake, highlighting the importance of this molecule in establishing the cellular difference for EV effects. Collectively, we identified CRC cells with functional difference in their EV uptake ability that must be taken into consideration when using EVs as therapeutic tools for targeting cancer cells.

Imaging developmental cell cycles

The last decade has seen a major expansion in development of live biosensors, the tools needed to genetically encode them into model organisms, and the microscopic techniques used to visualize them. When combined, these offer us powerful tools with which to make fundamental discoveries about complex biological processes. In this review, we summarize the availability of biosensors to visualize an essential cellular process, the cell cycle, and the techniques for single-cell tracking and quantification of these reporters. We also highlight studies investigating the connection of cellular behavior to the cell cycle, particularly through live imaging, and anticipate exciting discoveries with the combination of these technologies in developmental contexts.

Subdiffraction-resolution fluorescence imaging of immunological synapse formation between NK cells and A. fumigatus by expansion microscopy

Expansion microscopy (ExM) enables super-resolution fluorescence imaging on standard microscopes by physical expansion of the sample. However, the investigation of interactions between different organisms such as mammalian and fungal cells by ExM remains challenging because different cell types require different expansion protocols to ensure identical, ideally isotropic expansion of both partners. Here, we introduce an ExM method that enables super-resolved visualization of the interaction between NK cells and Aspergillus fumigatus hyphae. 4-fold expansion in combination with confocal fluorescence imaging allows us to resolve details of cytoskeleton rearrangement as well as NK cells' lytic granules triggered by contact with an RFP-expressing A. fumigatus strain. In particular, subdiffraction-resolution images show polarized degranulation upon contact formation and the presence of LAMP1 surrounding perforin at the NK cell-surface post degranulation. Our data demonstrate that optimized ExM protocols enable the investigation of immunological synapse formation between two different species with so far unmatched spatial resolution.

Avoiding a replication crisis in deep-learning-based bioimage analysis

Deep learning algorithms are powerful tools to analyse, restore and transform bioimaging data, increasingly used in life sciences research. These approaches now outperform most other algorithms for a broad range of image analysis tasks. In particular, one of the promises of deep learning is the possibility to provide parameter-free, one-click data analysis achieving expert-level performances in a fraction of the time previously required. However, as with most new and upcoming technologies, the potential for inappropriate use is raising concerns among the biomedical research community. This perspective aims to provide a short overview of key concepts that we believe are important for researchers to consider when using deep learning for their microscopy studies. These comments are based on our own experience gained while optimising various deep learning tools for bioimage analysis and discussions with colleagues from both the developer and user community. In particular, we focus on describing how results obtained using deep learning can be validated and discuss what should, in our views, be considered when choosing a suitable tool. We also suggest what aspects of a deep learning analysis would need to be reported in publications to describe the use of such tools to guarantee that the work can be reproduced. We hope this perspective will foster further discussion between developers, image analysis specialists, users and journal editors to define adequate guidelines and ensure that this transformative technology is used appropriately.

Advances in spatial transcriptomic data analysis

Spatial transcriptomics is a rapidly growing field that promises to comprehensively characterize tissue organization and architecture at the single-cell or subcellular resolution. Such information provides a solid foundation for mechanistic understanding of many biological processes in both health and disease that cannot be obtained by using traditional technologies. The development of computational methods plays important roles in extracting biological signals from raw data. Various approaches have been developed to overcome technology-specific limitations such as spatial resolution, gene coverage, sensitivity, and technical biases. Downstream analysis tools formulate spatial organization and cell-cell communications as quantifiable properties, and provide algorithms to derive such properties. Integrative pipelines further assemble multiple tools in one package, allowing biologists to conveniently analyze data from beginning to end. In this review, we summarize the state of the art of spatial transcriptomic data analysis methods and pipelines, and discuss how they operate on different technological platforms.

Tight Junction ZO Proteins Maintain Tissue Fluidity, Ensuring Efficient Collective Cell Migration

Tight junctions (TJs) are essential components of epithelial tissues connecting neighboring cells to provide protective barriers. While their general function to seal compartments is well understood, their role in collective cell migration is largely unexplored. Here, the importance of the TJ zonula occludens (ZO) proteins ZO1 and ZO2 for epithelial migration is investigated employing video microscopy in conjunction with velocimetry, segmentation, cell tracking, and atomic force microscopy/spectroscopy. The results indicate that ZO proteins are necessary for fast and coherent migration. In particular, ZO1 and 2 loss (dKD) induces actomyosin remodeling away from the central cortex towards the periphery of individual cells, resulting in altered viscoelastic properties. A tug-of-war emerges between two subpopulations of cells with distinct morphological and mechanical properties: 1) smaller and highly contractile cells with an outward bulging apical membrane, and 2) larger, flattened cells, which, due to tensile stress, display a higher proliferation rate. In response, the cell density increases, leading to crowding-induced jamming and more small cells over time. Co-cultures comprising wildtype and dKD cells migrate inefficiently due to phase separation based on differences in contractility rather than differential adhesion. This study shows that ZO proteins are necessary for efficient collective cell migration by maintaining tissue fluidity and controlling proliferation.

Self-supervised pretraining for transferable quantitative phase image cell segmentation

In this paper, a novel U-Net-based method for robust adherent cell segmentation for quantitative phase microscopy image is designed and optimised. We designed and evaluated four specific post-processing pipelines. To increase the transferability to different cell types, non-deep learning transfer with adjustable parameters is used in the post-processing step. Additionally, we proposed a self-supervised pretraining technique using nonlabelled data, which is trained to reconstruct multiple image distortions and improved the segmentation performance from 0.67 to 0.70 of object-wise intersection over union. Moreover, we publish a new dataset of manually labelled images suitable for this task together with the unlabelled data for self-supervised pretraining.

Spatiotemporal dynamics of PIEZO1 localization controls keratinocyte migration during wound healing

Keratinocytes, the predominant cell type of the epidermis, migrate to reinstate the epithelial barrier during wound healing. Mechanical cues are known to regulate keratinocyte re-epithelialization and wound healing; however, the underlying molecular transducers and biophysical mechanisms remain elusive. Here, we show through molecular, cellular, and organismal studies that the mechanically activated ion channel PIEZO1 regulates keratinocyte migration and wound healing. Epidermal-specific Piezo1 knockout mice exhibited faster wound closure while gain-of-function mice displayed slower wound closure compared to littermate controls. By imaging the spatiotemporal localization dynamics of endogenous PIEZO1 channels, we find that channel enrichment at some regions of the wound edge induces a localized cellular retraction that slows keratinocyte collective migration. In migrating single keratinocytes, PIEZO1 is enriched at the rear of the cell, where maximal retraction occurs, and we find that chemical activation of PIEZO1 enhances retraction during single as well as collective migration. Our findings uncover novel molecular mechanisms underlying single and collective keratinocyte migration that may suggest a potential pharmacological target for wound treatment. More broadly, we show that nanoscale spatiotemporal dynamics of Piezo1 channels can control tissue-scale events, a finding with implications beyond wound healing to processes as diverse as development, homeostasis, disease, and repair.

An Open-Source Framework for Automated High-Throughput Cell Biology Experiments

Modern data analysis methods, such as optimization algorithms or deep learning have been successfully applied to a number of biotechnological and medical questions. For these methods to be efficient, a large number of high-quality and reproducible experiments needs to be conducted, requiring a high degree of automation. Here, we present an open-source hardware and low-cost framework that allows for automatic high-throughput generation of large amounts of cell biology data. Our design consists of an epifluorescent microscope with automated XY stage for moving a multiwell plate containing cells and a perfusion manifold allowing programmed application of up to eight different solutions. Our system is very flexible and can be adapted easily for individual experimental needs. To demonstrate the utility of the system, we have used it to perform high-throughput Ca2+ imaging and large-scale fluorescent labeling experiments.

Mechanics of neural tube morphogenesis

The neural tube is an important model system of morphogenesis representing the developmental module of out-of-plane epithelial deformation. As the embryonic precursor of the central nervous system, the neural tube also holds keys to many defects and diseases. Recent advances begin to reveal how genetic, cellular and environmental mechanisms work in concert to ensure correct neural tube shape. A physical model is emerging where these factors converge at the regulation of the mechanical forces and properties within and around the tissue that drive tube formation towards completion. Here we review the dynamics and mechanics of neural tube morphogenesis and discuss the underlying cellular behaviours from the viewpoint of tissue mechanics. We will also highlight some of the conceptual and technical next steps.

A high-throughput DNA FISH protocol to visualize genome regions in human cells

Here, we describe an end-to-end high-throughput imaging protocol to visualize genomic loci in cells at high throughput using DNA fluorescence in situ hybridization, automated microscopy, and computational analysis. This is particularly useful for quantifying patterns of heterogeneity in relative gene positioning or differences within subpopulations of cells. We focus on important experimental design and execution steps in this one-week protocol, suggest ways to ensure and verify data quality, and provide practical solutions to common problems.

For complete details on the generation and use of this protocol, please refer to Finn et al. (2019).

•

DNA fluorescence in situ hybridization technique for high-content imaging

•

Optimized for use in a 384-well plate format with cultured cells

•

One-week protocol to assess thousands of cells per condition in tens of conditions

•

Thorough discussion of troubleshooting for new probes, probe types, and cell types

Deep learning for bioimage analysis in developmental biology

Deep learning has transformed the way large and complex image datasets can be processed, reshaping what is possible in bioimage analysis. As the complexity and size of bioimage data continues to grow, this new analysis paradigm is becoming increasingly ubiquitous. In this Review, we begin by introducing the concepts needed for beginners to understand deep learning. We then review how deep learning has impacted bioimage analysis and explore the open-source resources available to integrate it into a research project. Finally, we discuss the future of deep learning applied to cell and developmental biology. We analyze how state-of-the-art methodologies have the potential to transform our understanding of biological systems through new image-based analysis and modelling that integrate multimodal inputs in space and time.

CellProfiler 4: improvements in speed, utility and usability

Background

Imaging data contains a substantial amount of information which can be difficult to evaluate by eye. With the expansion of high throughput microscopy methodologies producing increasingly large datasets, automated and objective analysis of the resulting images is essential to effectively extract biological information from this data. CellProfiler is a free, open source image analysis program which enables researchers to generate modular pipelines with which to process microscopy images into interpretable measurements.

Results

Herein we describe CellProfiler 4, a new version of this software with expanded functionality. Based on user feedback, we have made several user interface refinements to improve the usability of the software. We introduced new modules to expand the capabilities of the software. We also evaluated performance and made targeted optimizations to reduce the time and cost associated with running common large-scale analysis pipelines.

Conclusions

CellProfiler 4 provides significantly improved performance in complex workflows compared to previous versions. This release will ensure that researchers will have continued access to CellProfiler’s powerful computational tools in the coming years.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-021-04344-9.

Application of 3D MAPs pipeline identifies the morphological sequence chondrocytes undergo and the regulatory role of GDF5 in this process

The activity of epiphyseal growth plates, which drives long bone elongation, depends on extensive changes in chondrocyte size and shape during differentiation. Here, we develop a pipeline called 3D Morphometric Analysis for Phenotypic significance (3D MAPs), which combines light-sheet microscopy, segmentation algorithms and 3D morphometric analysis to characterize morphogenetic cellular behaviors while maintaining the spatial context of the growth plate. Using 3D MAPs, we create a 3D image database of hundreds of thousands of chondrocytes. Analysis reveals broad repertoire of morphological changes, growth strategies and cell organizations during differentiation. Moreover, identifying a reduction in Smad 1/5/9 activity together with multiple abnormalities in cell growth, shape and organization provides an explanation for the shortening of Gdf5 KO tibias. Overall, our findings provide insight into the morphological sequence that chondrocytes undergo during differentiation and highlight the ability of 3D MAPs to uncover cellular mechanisms that may regulate this process.

A graph-based cell tracking algorithm with few manually tunable parameters and automated segmentation error correction

Automatic cell segmentation and tracking enables to gain quantitative insights into the processes driving cell migration. To investigate new data with minimal manual effort, cell tracking algorithms should be easy to apply and reduce manual curation time by providing automatic correction of segmentation errors. Current cell tracking algorithms, however, are either easy to apply to new data sets but lack automatic segmentation error correction, or have a vast set of parameters that needs either manual tuning or annotated data for parameter tuning. In this work, we propose a tracking algorithm with only few manually tunable parameters and automatic segmentation error correction. Moreover, no training data is needed. We compare the performance of our approach to three well-performing tracking algorithms from the Cell Tracking Challenge on data sets with simulated, degraded segmentation—including false negatives, over- and under-segmentation errors. Our tracking algorithm can correct false negatives, over- and under-segmentation errors as well as a mixture of the aforementioned segmentation errors. On data sets with under-segmentation errors or a mixture of segmentation errors our approach performs best. Moreover, without requiring additional manual tuning, our approach ranks several times in the top 3 on the 6^th edition of the Cell Tracking Challenge.

Computational challenges and opportunities in spatially resolved transcriptomic data analysis

Spatially resolved transcriptomic data demand new computational analysis methods to derive biological insights. Here, we comment on these associated computational challenges as well as highlight the opportunities for standardized benchmarking metrics and data-sharing infrastructure in spurring innovation moving forward.

The yeast mitochondrial succinylome: Implications for regulation of mitochondrial nucleoids

Acylation modifications, such as the succinylation of lysine, are post-translational modifications and a powerful means of regulating protein activity. Some acylations occur nonenzymatically, driven by an increase in the concentration of acyl group donors. Lysine succinylation has a profound effect on the corresponding site within the protein, as it dramatically changes the charge of the residue. In eukaryotes, it predominantly affects mitochondrial proteins because the donor of succinate, succinyl-CoA, is primarily generated in the tricarboxylic acid cycle. Although numerous succinylated mitochondrial proteins have been identified in Saccharomyces cerevisiae, a more detailed characterization of the yeast mitochondrial succinylome is still lacking. Here, we performed a proteomic MS analysis of purified yeast mitochondria and detected 314 succinylated mitochondrial proteins with 1763 novel succinylation sites. The mitochondrial nucleoid, a complex of mitochondrial DNA and mitochondrial proteins, is one of the structures whose protein components are affected by succinylation. We found that Abf2p, the principal component of mitochondrial nucleoids responsible for compacting mitochondrial DNA in S. cerevisiae, can be succinylated in vivo on at least thirteen lysine residues. Abf2p succinylation in vitro inhibits its DNA-binding activity and reduces its sensitivity to digestion by the ATP-dependent ScLon protease. We conclude that changes in the metabolic state of a cell resulting in an increase in the concentration of tricarboxylic acid intermediates may affect mitochondrial functions.

Transient Oxygen-Glucose Deprivation Causes Region- and Cell Type-Dependent Functional Deficits in the Mouse Hippocampus In Vitro

Neurons are highly vulnerable to conditions of hypoxia-ischemia (HI) such as stroke or transient ischemic attacks. Recovery of cognitive and behavioral functions requires re-emergence of coordinated network activity, which, in turn, relies on the well-orchestrated interaction of pyramidal cells (PYRs) and interneurons. We therefore modelled HI in the mouse hippocampus, a particularly vulnerable region showing marked loss of PYR and fast-spiking interneurons (FSIs) after hypoxic-ischemic insults. Transient oxygen-glucose deprivation (OGD) in ex vivo hippocampal slices led to a rapid loss of neuronal activity and spontaneous network oscillations (sharp wave-ripple complexes; SPW-Rs), and to the occurrence of a spreading depolarization. Following reperfusion, both SPW-R and neuronal spiking resumed, but FSI activity remained strongly reduced compared with PYR. Whole-cell recordings in CA1 PYR revealed, however, a similar reduction of both EPSCs and IPSCs, leaving inhibition-excitation (I/E) balance unaltered. At the network level, SPW-R incidence was strongly reduced and the remaining network events showed region-specific changes including reduced ripple energy in CA3 and increased ripple frequency in CA1. Together, our data show that transient hippocampal energy depletion results in severe functional alterations at the cellular and network level. While I/E balance is maintained, synaptic activity, interneuron spiking and coordinated network patterns remain reduced. Such alterations may be network-level correlates of cognitive and functional deficits after cerebral HI.

LIVECell-A large-scale dataset for label-free live cell segmentation

Light microscopy combined with well-established protocols of two-dimensional cell culture facilitates high-throughput quantitative imaging to study biological phenomena. Accurate segmentation of individual cells in images enables exploration of complex biological questions, but can require sophisticated imaging processing pipelines in cases of low contrast and high object density. Deep learning-based methods are considered state-of-the-art for image segmentation but typically require vast amounts of annotated data, for which there is no suitable resource available in the field of label-free cellular imaging. Here, we present LIVECell, a large, high-quality, manually annotated and expert-validated dataset of phase-contrast images, consisting of over 1.6 million cells from a diverse set of cell morphologies and culture densities. To further demonstrate its use, we train convolutional neural network-based models using LIVECell and evaluate model segmentation accuracy with a proposed a suite of benchmarks.

Object-Guided Instance Segmentation With Auxiliary Feature Refinement for Biological Images

Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. In this paper, we propose a novel box-based instance segmentation method. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region. However, existing methods can hardly differentiate the target from its neighboring objects within the same bounding box region due to their similar textures and low-contrast boundaries. To deal with this problem, in this paper, we propose an object-guided instance segmentation method. Our method first detects the center points of the objects, from which the bounding box parameters are then predicted. To perform segmentation, an object-guided coarse-to-fine segmentation branch is built along with the detection branch. The segmentation branch reuses the object features as guidance to separate target object from the neighboring ones within the same bounding box region. To further improve the segmentation quality, we design an auxiliary feature refinement module that densely samples and refines point-wise features in the boundary regions. Experimental results on three biological image datasets demonstrate the advantages of our method. The code will be available at https://github.com/yijingru/ObjGuided-Instance-Segmentation.

Sequence-based features that are determinant for tail-anchored membrane protein sorting in eukaryotes

The correct targeting and insertion of tail-anchored (TA) integral membrane proteins is critical for cellular homeostasis. TA proteins are defined by a hydrophobic transmembrane domain (TMD) at their C-terminus and are targeted to either the ER or mitochondria. Derived from experimental measurements of a few TA proteins, there has been little examination of the TMD features that determine localization. As a result, the localization of many TA proteins are misclassified by the simple heuristic of overall hydrophobicity. Because ER-directed TMDs favor arrangement of hydrophobic residues to one side, we sought to explore the role of geometric hydrophobic properties. By curating TA proteins with experimentally determined localizations and assessing hypotheses for recognition, we bioinformatically and experimentally verify that a hydrophobic face is the most accurate singular metric for separating ER and mitochondria-destined yeast TA proteins. A metric focusing on an 11 residue segment of the TMD performs well when classifying human TA proteins. The most inclusive predictor uses both hydrophobicity and C-terminal charge in tandem. This work provides context for previous observations and opens the door for more detailed mechanistic experiments to determine the molecular factors driving this recognition.

RNA transport from transcription to localized translation: a single molecule perspective

Transport of mRNAs is an important step of gene expression, which brings the genetic message from the DNA in the nucleus to a precise cytoplasmic location in a regulated fashion. Perturbation of this process can lead to pathologies such as developmental and neurological disorders. In this review, we discuss recent advances in the field of mRNA transport made using single molecule fluorescent imaging approaches. We present an overview of these approaches in fixed and live cells and their input in understanding the key steps of mRNA journey: transport across the nucleoplasm, export through the nuclear pores and delivery to its final cytoplasmic location. This review puts a particular emphasis on the coupling of mRNA transport with translation, such as localization-dependent translational regulation and translation-dependent mRNA localization. We also highlight the recently discovered translation factories, and how cellular and viral RNAs can hijack membrane transport systems to travel in the cytoplasm.

Advances and opportunities in image analysis of bacterial cells and communities

The cellular morphology and sub-cellular spatial structure critically influence the function of microbial cells. Similarly, the spatial arrangement of genotypes and phenotypes in microbial communities has important consequences for cooperation, competition, and community functions. Fluorescence microscopy techniques are widely used to measure spatial structure inside living cells and communities, which often results in large numbers of images that are difficult or impossible to analyze manually. The rapidly evolving progress in computational image analysis has recently enabled the quantification of a large number of properties of single cells and communities, based on traditional analysis techniques and convolutional neural networks. Here, we provide a brief introduction to core concepts of automated image processing, recent software tools and how to validate image analysis results. We also discuss recent advances in image analysis of microbial cells and communities, and how these advances open up opportunities for quantitative studies of spatiotemporal processes in microbiology, based on image cytometry and adaptive microscope control.

Strategies for Accurate Cell Type Identification in CODEX Multiplexed Imaging Data

Multiplexed imaging is a recently developed and powerful single-cell biology research tool. However, it presents new sources of technical noise that are distinct from other types of single-cell data, necessitating new practices for single-cell multiplexed imaging processing and analysis, particularly regarding cell-type identification. Here we created single-cell multiplexed imaging datasets by performing CODEX on four sections of the human colon (ascending, transverse, descending, and sigmoid) using a panel of 47 oligonucleotide-barcoded antibodies. After cell segmentation, we implemented five different normalization techniques crossed with four unsupervised clustering algorithms, resulting in 20 unique cell-type annotations for the same dataset. We generated two standard annotations: hand-gated cell types and cell types produced by over-clustering with spatial verification. We then compared these annotations at four levels of cell-type granularity. First, increasing cell-type granularity led to decreased labeling accuracy; therefore, subtle phenotype annotations should be avoided at the clustering step. Second, accuracy in cell-type identification varied more with normalization choice than with clustering algorithm. Third, unsupervised clustering better accounted for segmentation noise during cell-type annotation than hand-gating. Fourth, Z-score normalization was generally effective in mitigating the effects of noise from single-cell multiplexed imaging. Variation in cell-type identification will lead to significant differential spatial results such as cellular neighborhood analysis; consequently, we also make recommendations for accurately assigning cell-type labels to CODEX multiplexed imaging.

Spatial omics and multiplexed imaging to explore cancer biology

Understanding intratumoral heterogeneity-the molecular variation among cells within a tumor-promises to address outstanding questions in cancer biology and improve the diagnosis and treatment of specific cancer subtypes. Single-cell analyses, especially RNA sequencing and other genomics modalities, have been transformative in revealing novel biomarkers and molecular regulators associated with tumor growth, metastasis and drug resistance. However, these approaches fail to provide a complete picture of tumor biology, as information on cellular location within the tumor microenvironment is lost. New technologies leveraging multiplexed fluorescence, DNA, RNA and isotope labeling enable the detection of tens to thousands of cancer subclones or molecular biomarkers within their native spatial context. The expeditious growth in these techniques, along with methods for multiomics data integration, promises to yield a more comprehensive understanding of cell-to-cell variation within and between individual tumors. Here we provide the current state and future perspectives on the spatial technologies expected to drive the next generation of research and diagnostic and therapeutic strategies for cancer.

CODEX multiplexed tissue imaging with DNA-conjugated antibodies

Advances in multiplexed imaging technologies have drastically improved our ability to characterize healthy and diseased tissues at the single-cell level. Co-detection by indexing (CODEX) relies on DNA-conjugated antibodies and the cyclic addition and removal of complementary fluorescently labeled DNA probes and has been used so far to simultaneously visualize up to 60 markers in situ. CODEX enables a deep view into the single-cell spatial relationships in tissues and is intended to spur discovery in developmental biology, disease and therapeutic design. Herein, we provide optimized protocols for conjugating purified antibodies to DNA oligonucleotides, validating the conjugation by CODEX staining and executing the CODEX multicycle imaging procedure for both formalin-fixed, paraffin-embedded (FFPE) and fresh-frozen tissues. In addition, we describe basic image processing and data analysis procedures. We apply this approach to an FFPE human tonsil multicycle experiment. The hands-on experimental time for antibody conjugation is ~4.5 h, validation of DNA-conjugated antibodies with CODEX staining takes ~6.5 h and preparation for a CODEX multicycle experiment takes ~8 h. The multicycle imaging and data analysis time depends on the tissue size, number of markers in the panel and computational complexity.

A deep learning approach to quantify auditory hair cells

Hearing loss affects millions of people worldwide. Yet, there are still no curative therapies for sensorineural hearing loss. Frequent causes of sensorineural hearing loss are due to damage or loss of the sensory hair cells, the spiral ganglion neurons, or the synapses between them. Culturing the organ of Corti allows the study of all these structures in an experimental model, which is easy to manipulate. Therefore, the in vitro culture of the neonatal mammalian organ of Corti remains a frequently used experimental system, in which hair cell survival is routinely assessed. However, the analysis of the surviving hair cells is commonly performed via manual counting, which is a time-consuming process and the inter-rater reliability can be an issue. Here, we describe a deep learning approach to quantify hair cell survival in the murine organ of Corti explants. We used StarDist, a publicly available platform and plugin for Fiji (Fiji is just ImageJ), to train and apply our own custom deep learning model. We successfully validated our model in untreated, cisplatin, and gentamicin treated organ of Corti explants. Therefore, deep learning is a valuable approach for quantifying hair cell survival in organ of Corti explants. Moreover, we also demonstrate how the publicly available Fiji plugin StarDist can be efficiently used for this purpose.

HLA-DR cancer cells expression correlates with T cell infiltration and is enriched in lung adenocarcinoma with indolent behavior

Lung adenocarcinoma (ADC) is a heterogeneous group of tumors associated with different survival rates, even when detected at an early stage. Here, we aim to investigate whether CyTOF identifies cellular and molecular predictors of tumor behavior. We developed and validated a CyTOF panel of 34 antibodies in four ADC cell lines and PBMC. We tested our panel in a set of 10 ADCs, classified into long- (LPS) (n = 4) and short-predicted survival (SPS) (n = 6) based on radiomics features. We identified cellular subpopulations of epithelial cancer cells (ECC) and their microenvironment and validated our results by multiplex immunofluorescence (mIF) applied to a tissue microarray (TMA) of LPS and SPS ADCs. The antibody panel captured the phenotypical differences in ADC cell lines and PBMC. LPS ADCs had a higher proportion of immune cells. ECC clusters (ECCc) were identified and uncovered two ADC groups. ECCc with high HLA-DR expression were correlated with CD4+ and CD8+ T cells, with LPS samples being enriched for those clusters. We confirmed a positive correlation between HLA-DR expression on ECC and T cell number by mIF staining on TMA slides. Spatial analysis demonstrated shorter distances from T cells to the nearest ECC in LPS. Our results demonstrate a distinctive cellular profile of ECC and their microenvironment in ADC. We showed that HLA-DR expression in ECC is correlated with T cell infiltration, and that a set of ADCs with high abundance of HLA-DR+ ECCc and T cells is enriched in LPS samples. This suggests new insights into the role of antigen presenting tumor cells in tumorigenesis.

Active flows and deformable surfaces in development

We review progress in active hydrodynamic descriptions of flowing media on curved and deformable manifolds: the state-of-the-art in continuum descriptions of single-layers of epithelial and/or other tissues during development. First, after a brief overview of activity, flows and hydrodynamic descriptions, we highlight the generic challenge of identifying the dependence on dynamical variables of so-called active kinetic coefficients- active counterparts to dissipative Onsager coefficients. We go on to describe some of the subtleties concerning how curvature and active flows interact, and the issues that arise when surfaces are deformable. We finish with a broad discussion around the utility of such theories in developmental biology. This includes limitations to analytical techniques, challenges associated with numerical integration, fitting-to-data and inference, and potential tools for the future, such as discrete differential geometry.

Fully unsupervised deep mode of action learning for phenotyping high-content cellular images

MOTIVATION

The identification and discovery of phenotypes from high content screening (HCS) images is a challenging task. Earlier works use image analysis pipelines to extract biological features, supervised training methods or generate features with neural networks pretrained on non-cellular images. We introduce a novel unsupervised deep learning algorithm to cluster cellular images with similar Mode-of-Action (MOA) together using only the images' pixel intensity values as input. It corrects for batch effect during training. Importantly, our method does not require the extraction of cell candidates and works from the entire images directly.

RESULTS

The method achieves competitive results on the labelled subset of the BBBC021 dataset with an accuracy of 97.09% for correctly classifying the MOA by nearest neighbors matching. Importantly, we can train our approach on unannotated datasets. Therefore, our method can discover novel MOAs and annotate unlabelled compounds. The ability to train end-to-end on the full resolution images makes our method easy to apply and allows it to further distinguish treatments by their effect on proliferation.

AVAILABILITY

Our code is available at https://github.com/Novartis/UMM-Discovery.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Budding epithelial morphogenesis driven by cell-matrix versus cell-cell adhesion

Many embryonic organs undergo epithelial morphogenesis to form tree-like hierarchical structures. However, it remains unclear what drives the budding and branching of stratified epithelia, such as in the embryonic salivary gland and pancreas. Here, we performed live-organ imaging of mouse embryonic salivary glands at single-cell resolution to reveal that budding morphogenesis is driven by expansion and folding of a distinct epithelial surface cell sheet characterized by strong cell-matrix adhesions and weak cell-cell adhesions. Profiling of single-cell transcriptomes of this epithelium revealed spatial patterns of transcription underlying these cell adhesion differences. We then synthetically reconstituted budding morphogenesis by experimentally suppressing E-cadherin expression and inducing basement membrane formation in 3D spheroid cultures of engineered cells, which required β1-integrin-mediated cell-matrix adhesion for successful budding. Thus, stratified epithelial budding, the key first step of branching morphogenesis, is driven by an overall combination of strong cell-matrix adhesion and weak cell-cell adhesion by peripheral epithelial cells.

HRMAn 2.0: Next-generation artificial intelligence-driven analysis for broad host-pathogen interactions

To study the dynamics of infection processes, it is common to manually enumerate imaging-based infection assays. However, manual counting of events from imaging data is biased, error-prone and a laborious task. We recently presented HRMAn (Host Response to Microbe Analysis), an automated image analysis program using state-of-the-art machine learning and artificial intelligence algorithms to analyse pathogen growth and host defence behaviour. With HRMAn, we can quantify intracellular infection by pathogens such as Toxoplasma gondii and Salmonella in a variety of cell types in an unbiased and highly reproducible manner, measuring multiple parameters including pathogen growth, pathogen killing and activation of host cell defences. Since HRMAn is based on the KNIME Analytics platform, it can easily be adapted to work with other pathogens and produce more readouts from quantitative imaging data. Here we showcase improvements to HRMAn resulting in the release of HRMAn 2.0 and new applications of HRMAn 2.0 for the analysis of host-pathogen interactions using the established pathogen T. gondii and further extend it for use with the bacterial pathogen Chlamydia trachomatis and the fungal pathogen Cryptococcus neoformans.

A variable-gain stochastic pooling motif mediates information transfer from receptor assemblies into NF-κB

A myriad of inflammatory cytokines regulate signaling pathways to maintain cellular homeostasis. The IκB kinase (IKK) complex is an integration hub for cytokines that govern nuclear factor κB (NF-κB) signaling. In response to inflammation, IKK is activated through recruitment to receptor-associated protein assemblies. How and what information IKK complexes transmit about the milieu are open questions. Here, we track dynamics of IKK complexes and nuclear NF-κB to identify upstream signaling features that determine same-cell responses. Experiments and modeling of single complexes reveal their size, number, and timing relays cytokine-specific control over shared signaling mechanisms with feedback regulation that is independent of transcription. Our results provide evidence for variable-gain stochastic pooling, a noise-reducing motif that enables cytokine-specific regulation and parsimonious information transfer. We propose that emergent properties of stochastic pooling are general principles of receptor signaling that have evolved for constructive information transmission in noisy molecular environments.

Evaluation of Deep Learning Architectures for Complex Immunofluorescence Nuclear Image Segmentation

Separating and labeling each nuclear instance (instance-aware segmentation) is the key challenge in nuclear image segmentation. Deep Convolutional Neural Networks have been demonstrated to solve nuclear image segmentation tasks across different imaging modalities, but a systematic comparison on complex immunofluorescence images has not been performed. Deep learning based segmentation requires annotated datasets for training, but annotated fluorescence nuclear image datasets are rare and of limited size and complexity. In this work, we evaluate and compare the segmentation effectiveness of multiple deep learning architectures (U-Net, U-Net ResNet, Cellpose, Mask R-CNN, KG instance segmentation) and two conventional algorithms (Iterative h-min based watershed, Attributed relational graphs) on complex fluorescence nuclear images of various types. We propose and evaluate a novel strategy to create artificial images to extend the training set. Results show that instance-aware segmentation architectures and Cellpose outperform the U-Net architectures and conventional methods on complex images in terms of F1 scores, while the U-Net architectures achieve overall higher mean Dice scores. Training with additional artificially generated images improves recall and F1 scores for complex images, thereby leading to top F1 scores for three out of five sample preparation types. Mask R-CNN trained on artificial images achieves the overall highest F1 score on complex images of similar conditions to the training set images while Cellpose achieves the overall highest F1 score on complex images of new imaging conditions. We provide quantitative results demonstrating that images annotated by under-graduates are sufficient for training instance-aware segmentation architectures to efficiently segment complex fluorescence nuclear images.

Maximizing content across scales: Moving multimodal microscopy and mesoscopy toward molecular imaging

Molecular imaging aims to depict the molecules in living patients. However, because this aim is still far beyond reach, patchworks of different solutions need to be used to tackle this overarching goal. From the vast toolbox of imaging techniques, we focus on those recent advances in optical microscopy that image molecules and cells at the submicron to centimeter scale. Mesoscopic imaging covers the "imaging gap" between techniques such as confocal microscopy and magnetic resonance imagingthat image entire live samples but with limited resolution. Microscopy focuses on the cellular level; mesoscopy visualizes the organization of molecules and cells into tissues and organs. The correlation between these techniques allows us to combine disciplines ranging from whole body imaging to basic research of model systems. We review current developments focused on improving microscopic and mesoscopic imaging technologies and on hardware and software that push the current sensitivity and resolution boundaries.

Cell Tracking for Organoids: Lessons From Developmental Biology

Organoids have emerged as powerful model systems to study organ development and regeneration at the cellular level. Recently developed microscopy techniques that track individual cells through space and time hold great promise to elucidate the organizational principles of organs and organoids. Applied extensively in the past decade to embryo development and 2D cell cultures, cell tracking can reveal the cellular lineage trees, proliferation rates, and their spatial distributions, while fluorescent markers indicate differentiation events and other cellular processes. Here, we review a number of recent studies that exemplify the power of this approach, and illustrate its potential to organoid research. We will discuss promising future routes, and the key technical challenges that need to be overcome to apply cell tracking techniques to organoid biology.

Automatic and unbiased segmentation and quantification of myofibers in skeletal muscle

Skeletal muscle has the remarkable ability to regenerate. However, with age and disease muscle strength and function decline. Myofiber size, which is affected by injury and disease, is a critical measurement to assess muscle health. Here, we test and apply Cellpose, a recently developed deep learning algorithm, to automatically segment myofibers within murine skeletal muscle. We first show that tissue fixation is necessary to preserve cellular structures such as primary cilia, small cellular antennae, and adipocyte lipid droplets. However, fixation generates heterogeneous myofiber labeling, which impedes intensity-based segmentation. We demonstrate that Cellpose efficiently delineates thousands of individual myofibers outlined by a variety of markers, even within fixed tissue with highly uneven myofiber staining. We created a novel ImageJ plugin (LabelsToRois) that allows processing of multiple Cellpose segmentation images in batch. The plugin also contains a semi-automatic erosion function to correct for the area bias introduced by the different stainings, thereby identifying myofibers as accurately as human experts. We successfully applied our segmentation pipeline to uncover myofiber regeneration differences between two different muscle injury models, cardiotoxin and glycerol. Thus, Cellpose combined with LabelsToRois allows for fast, unbiased, and reproducible myofiber quantification for a variety of staining and fixation conditions.

Methodology for comprehensive cell-level analysis of wound healing experiments using deep learning in MATLAB

BACKGROUND

Endothelial healing after deployment of cardiovascular devices is particularly important in the context of clinical outcome. It is therefore of great interest to develop tools for a precise prediction of endothelial growth after injury in the process of implant deployment. For experimental investigation of re-endothelialization in vitro cell migration assays are routinely used. However, semi-automatic analyses of live cell images are often based on gray value distributions and are as such limited by image quality and user dependence. The rise of deep learning algorithms offers promising opportunities for application in medical image analysis. Here, we present an intelligent cell detection (iCD) approach for comprehensive assay analysis to obtain essential characteristics on cell and population scale.

RESULTS

In an in vitro wound healing assay, we compared conventional analysis methods with our iCD approach. Therefore we determined cell density and cell velocity on cell scale and the movement of the cell layer as well as the gap closure between two cell monolayers on population scale. Our data demonstrate that cell density analysis based on deep learning algorithms is superior to an adaptive threshold method regarding robustness against image distortion. In addition, results on cell scale obtained with iCD are in agreement with manually velocity detection, while conventional methods, such as Cell Image Velocimetry (CIV), underestimate cell velocity by a factor of 0.5. Further, we found that iCD analysis of the monolayer movement gave results just as well as manual freehand detection, while conventional methods again shows more frayed leading edge detection compared to manual detection. Analysis of monolayer edge protrusion by ICD also produced results, which are close to manual estimation with an relative error of 11.7%. In comparison, the conventional Canny method gave a relative error of 76.4%.

CONCLUSION

The results of our experiments indicate that deep learning algorithms such as our iCD have the ability to outperform conventional methods in the field of wound healing analysis. The combined analysis on cell and population scale using iCD is very well suited for timesaving and high quality wound healing analysis enabling the research community to gain detailed understanding of endothelial movement.

Integrating barcoded neuroanatomy with spatial transcriptional profiling enables identification of gene correlates of projections

Functional circuits consist of neurons with diverse axonal projections and gene expression. Understanding the molecular signature of projections requires high-throughput interrogation of both gene expression and projections to multiple targets in the same cells at cellular resolution, which is difficult to achieve using current technology. Here, we introduce BARseq2, a technique that simultaneously maps projections and detects multiplexed gene expression by in situ sequencing. We determined the expression of cadherins and cell-type markers in 29,933 cells and the projections of 3,164 cells in both the mouse motor cortex and auditory cortex. Associating gene expression and projections in 1,349 neurons revealed shared cadherin signatures of homologous projections across the two cortical areas. These cadherins were enriched across multiple branches of the transcriptomic taxonomy. By correlating multigene expression and projections to many targets in single neurons with high throughput, BARseq2 provides a potential path to uncovering the molecular logic underlying neuronal circuits.

Joint cell segmentation and cell type annotation for spatial transcriptomics

RNA hybridization-based spatial transcriptomics provides unparalleled detection sensitivity. However, inaccuracies in segmentation of image volumes into cells cause misassignment of mRNAs which is a major source of errors. Here, we develop JSTA, a computational framework for joint cell segmentation and cell type annotation that utilizes prior knowledge of cell type-specific gene expression. Simulation results show that leveraging existing cell type taxonomy increases RNA assignment accuracy by more than 45%. Using JSTA, we were able to classify cells in the mouse hippocampus into 133 (sub)types revealing the spatial organization of CA1, CA3, and Sst neuron subtypes. Analysis of within cell subtype spatial differential gene expression of 80 candidate genes identified 63 with statistically significant spatial differential gene expression across 61 (sub)types. Overall, our work demonstrates that known cell type expression patterns can be leveraged to improve the accuracy of RNA hybridization-based spatial transcriptomics while providing highly granular cell (sub)type information. The large number of newly discovered spatial gene expression patterns substantiates the need for accurate spatial transcriptomic measurements that can provide information beyond cell (sub)type labels.

Segmentor: a tool for manual refinement of 3D microscopy annotations

BACKGROUND

Recent advances in tissue clearing techniques, combined with high-speed image acquisition through light sheet microscopy, enable rapid three-dimensional (3D) imaging of biological specimens, such as whole mouse brains, in a matter of hours. Quantitative analysis of such 3D images can help us understand how changes in brain structure lead to differences in behavior or cognition, but distinguishing densely packed features of interest, such as nuclei, from background can be challenging. Recent deep learning-based nuclear segmentation algorithms show great promise for automated segmentation, but require large numbers of accurate manually labeled nuclei as training data.

RESULTS

We present Segmentor, an open-source tool for reliable, efficient, and user-friendly manual annotation and refinement of objects (e.g., nuclei) within 3D light sheet microscopy images. Segmentor employs a hybrid 2D-3D approach for visualizing and segmenting objects and contains features for automatic region splitting, designed specifically for streamlining the process of 3D segmentation of nuclei. We show that editing simultaneously in 2D and 3D using Segmentor significantly decreases time spent on manual annotations without affecting accuracy as compared to editing the same set of images with only 2D capabilities.

CONCLUSIONS

Segmentor is a tool for increased efficiency of manual annotation and refinement of 3D objects that can be used to train deep learning segmentation algorithms, and is available at https://www.nucleininja.org/ and https://github.com/RENCI/Segmentor .

Image-based cell phenotyping with deep learning

A cell's phenotype is the culmination of several cellular processes through a complex network of molecular interactions that ultimately result in a unique morphological signature. Visual cell phenotyping is the characterization and quantification of these observable cellular traits in images. Recently, cellular phenotyping has undergone a massive overhaul in terms of scale, resolution, and throughput, which is attributable to advances across electronic, optical, and chemical technologies for imaging cells. Coupled with the rapid acceleration of deep learning-based computational tools, these advances have opened up new avenues for innovation across a wide variety of high-throughput cell biology applications. Here, we review applications wherein deep learning is powering the recognition, profiling, and prediction of visual phenotypes to answer important biological questions. As the complexity and scale of imaging assays increase, deep learning offers computational solutions to elucidate the details of previously unexplored cellular phenotypes.

High-Content Imaging to Phenotype Antimicrobial Effects on Individual Bacteria at Scale

High-content imaging (HCI) is a technique for screening multiple cells in high resolution to detect subtle morphological and phenotypic variation. The method has been commonly deployed on model eukaryotic cellular systems, often for screening new drugs and targets. HCI is not commonly utilized for studying bacterial populations but may be a powerful tool in understanding and combatting antimicrobial resistance. Consequently, we developed a high-throughput method for phenotyping bacteria under antimicrobial exposure at the scale of individual bacterial cells. Imaging conditions were optimized on an Opera Phenix confocal microscope (Perkin Elmer), and novel analysis pipelines were established for both Gram-negative bacilli and Gram-positive cocci. The potential of this approach was illustrated using isolates of Klebsiella pneumoniae, Salmonella enterica serovar Typhimurium, and Staphylococcus aureus HCI enabled the detection and assessment of subtle morphological characteristics, undetectable through conventional phenotypical methods, that could reproducibly distinguish between bacteria exposed to different classes of antimicrobials with distinct modes of action (MOAs). In addition, distinctive responses were observed between susceptible and resistant isolates. By phenotyping single bacterial cells, we observed intrapopulation differences, which may be critical in identifying persistence or emerging resistance during antimicrobial treatment. The work presented here outlines a comprehensive method for investigating morphological changes at scale in bacterial populations under specific perturbation.IMPORTANCE High-content imaging (HCI) is a microscopy technique that permits the screening of multiple cells simultaneously in high resolution to detect subtle morphological and phenotypic variation. The power of this methodology is that it can generate large data sets comprised of multiple parameters taken from individual cells subjected to a range of different conditions. We aimed to develop novel methods for using HCI to study bacterial cells exposed to a range of different antibiotic classes. Using an Opera Phenix confocal microscope (Perkin Elmer) and novel analysis pipelines, we created a method to study the morphological characteristics of Klebsiella pneumoniae, Salmonella enterica serovar Typhimurium, and Staphylococcus aureus when exposed to antibacterial drugs with differing modes of action. By imaging individual bacterial cells at high resolution and scale, we observed intrapopulation differences associated with different antibiotics. The outlined methods are highly relevant for how we begin to better understand and combat antimicrobial resistance.