SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches

High-throughput image processing software for the study of nuclear architecture and gene expression

High-throughput imaging (HTI) generates complex imaging datasets from a large number of experimental perturbations. Commercial HTI software programs for image analysis workflows typically do not allow full customization and adoption of new image processing algorithms in the analysis modules. While open-source HTI analysis platforms provide individual modules in the workflow, like nuclei segmentation, spot detection, or cell tracking, they are often limited in integrating novel analysis modules or algorithms. Here, we introduce the High-Throughput Image Processing Software (HiTIPS) to expand the range and customization of existing HTI analysis capabilities. HiTIPS incorporates advanced image processing and machine learning algorithms for automated cell and nuclei segmentation, spot signal detection, nucleus tracking, nucleus registration, spot tracking, and quantification of spot signal intensity. Furthermore, HiTIPS features a graphical user interface that is open to integration of new analysis modules for existing analysis pipelines and to adding new analysis modules. To demonstrate the utility of HiTIPS, we present three examples of image analysis workflows for high-throughput DNA FISH, immunofluorescence (IF), and live-cell imaging of transcription in single cells. Altogether, we demonstrate that HiTIPS is a user-friendly, flexible, and open-source HTI software platform for a variety of cell biology applications.

Accurate single-molecule spot detection for image-based spatial transcriptomics with weakly supervised deep learning

Image-based spatial transcriptomics methods enable transcriptome-scale gene expression measurements with spatial information but require complex, manually tuned analysis pipelines. We present Polaris, an analysis pipeline for image-based spatial transcriptomics that combines deep-learning models for cell segmentation and spot detection with a probabilistic gene decoder to quantify single-cell gene expression accurately. Polaris offers a unifying, turnkey solution for analyzing spatial transcriptomics data from multiplexed error-robust FISH (MERFISH), sequential fluorescence in situ hybridization (seqFISH), or in situ RNA sequencing (ISS) experiments. Polaris is available through the DeepCell software library (https://github.com/vanvalenlab/deepcell-spots) and https://www.deepcell.org.

Accurate single-molecule spot detection for image-based spatial transcriptomics with weakly supervised deep learning

Image-based spatial transcriptomics methods enable transcriptome-scale gene expression measurements with spatial information but require complex, manually-tuned analysis pipelines. We present Polaris, an analysis pipeline for image-based spatial transcriptomics that combines deep learning models for cell segmentation and spot detection with a probabilistic gene decoder to quantify single-cell gene expression accurately. Polaris offers a unifying, turnkey solution for analyzing spatial transcriptomics data from MERFSIH, seqFISH, or ISS experiments. Polaris is available through the DeepCell software library (https://github.com/vanvalenlab/deepcell-spots) and https://www.deepcell.org.

Artificial Intelligence-Assisted Diagnostic Cytology and Genomic Testing for Hematologic Disorders

Artificial intelligence (AI) is a rapidly evolving field of computer science that involves the development of computational programs that can mimic human intelligence. In particular, machine learning and deep learning models have enabled the identification and grouping of patterns within data, leading to the development of AI systems that have been applied in various areas of hematology, including digital pathology, alpha thalassemia patient screening, cytogenetics, immunophenotyping, and sequencing. These AI-assisted methods have shown promise in improving diagnostic accuracy and efficiency, identifying novel biomarkers, and predicting treatment outcomes. However, limitations such as limited databases, lack of validation and standardization, systematic errors, and bias prevent AI from completely replacing manual diagnosis in hematology. In addition, the processing of large amounts of patient data and personal information by AI poses potential data privacy issues, necessitating the development of regulations to evaluate AI systems and address ethical concerns in clinical AI systems. Nonetheless, with continued research and development, AI has the potential to revolutionize the field of hematology and improve patient outcomes. To fully realize this potential, however, the challenges facing AI in hematology must be addressed and overcome.

Deep learning in spatial transcriptomics: Learning from the next next-generation sequencing

Spatial transcriptomics (ST) technologies are rapidly becoming the extension of single-cell RNA sequencing (scRNAseq), holding the potential of profiling gene expression at a single-cell resolution while maintaining cellular compositions within a tissue. Having both expression profiles and tissue organization enables researchers to better understand cellular interactions and heterogeneity, providing insight into complex biological processes that would not be possible with traditional sequencing technologies. Data generated by ST technologies are inherently noisy, high-dimensional, sparse, and multi-modal (including histological images, count matrices, etc.), thus requiring specialized computational tools for accurate and robust analysis. However, many ST studies currently utilize traditional scRNAseq tools, which are inadequate for analyzing complex ST datasets. On the other hand, many of the existing ST-specific methods are built upon traditional statistical or machine learning frameworks, which have shown to be sub-optimal in many applications due to the scale, multi-modality, and limitations of spatially resolved data (such as spatial resolution, sensitivity, and gene coverage). Given these intricacies, researchers have developed deep learning (DL)-based models to alleviate ST-specific challenges. These methods include new state-of-the-art models in alignment, spatial reconstruction, and spatial clustering, among others. However, DL models for ST analysis are nascent and remain largely underexplored. In this review, we provide an overview of existing state-of-the-art tools for analyzing spatially resolved transcriptomics while delving deeper into the DL-based approaches. We discuss the new frontiers and the open questions in this field and highlight domains in which we anticipate transformational DL applications.

Pooled image-base screening of mitochondria with microraft isolation distinguishes pathogenic mitofusin 2 mutations

Most human genetic variation is classified as variants of uncertain significance. While advances in genome editing have allowed innovation in pooled screening platforms, many screens deal with relatively simple readouts (viability, fluorescence) and cannot identify the complex cellular phenotypes that underlie most human diseases. In this paper, we present a generalizable functional genomics platform that combines high-content imaging, machine learning, and microraft isolation in a method termed "Raft-Seq". We highlight the efficacy of our platform by showing its ability to distinguish pathogenic point mutations of the mitochondrial regulator Mitofusin 2, even when the cellular phenotype is subtle. We also show that our platform achieves its efficacy using multiple cellular features, which can be configured on-the-fly. Raft-Seq enables a way to perform pooled screening on sets of mutations in biologically relevant cells, with the ability to physically capture any cell with a perturbed phenotype and expand it clonally, directly from the primary screen.

Nuclear position modulates long-range chromatin interactions

The human genome is non-randomly organized within the cell nucleus. Spatial mapping of genome folding by biochemical methods and imaging has revealed extensive variation in locus interaction frequencies between cells in a population and between homologs within an individual cell. Commonly used mapping approaches typically examine either the relative position of genomic sites to each other or the position of individual loci relative to nuclear landmarks. Whether the frequency of specific chromatin-chromatin interactions is affected by where in the nuclear space a locus is located is unknown. Here, we have simultaneously mapped at the single cell level the interaction frequencies and radial position of more than a hundred locus pairs using high-throughput imaging to ask whether the location within the nucleus affects interaction frequency. We find strong enrichment of many interactions at specific radial positions. Position-dependency of interactions was cell-type specific, correlated with local chromatin type, and cell-type-specific enriched associations were marked by increased variability, sometimes without a significant decrease in mean spatial distance. These observations demonstrate that the folding of the chromatin fiber, which brings genomically distant loci into proximity, and the position of that chromatin fiber relative to nuclear landmarks, are closely linked.

Large-scale mapping of positional changes of hypoxia-responsive genes upon activation

Chromosome structure and nuclear organization are important factors in the regulation of gene expression. Transcription of a gene is influenced by local and global chromosome features such as chromatin condensation status. The relationship between the 3D position of a gene in the nucleus and its activity is less clear. Here we used high-throughput imaging to perform a large-scale analysis of the spatial location of nearly 100 hypoxia-responsive genes to determine whether their location and activity state are correlated. Radial distance analysis demonstrated that the majority of Hypoxia-Inducible Factor (HIF)- and CREB-dependent hypoxia-responsive genes are located in the intermediate region of the nucleus, and some of them changed their radial position in hypoxia. Analysis of the relative distances among a subset of HIF target genes revealed that some gene pairs altered their relative location to each other on hypoxic treatment, suggesting higher-order chromatin rearrangements. While these changes in location occurred in response to hypoxic activation of the target genes, they did not correlate with the extent of their activation. These results suggest that induction of the hypoxia-responsive gene expression program is accompanied by spatial alterations of the genome, but that radial and relative gene positions are not directly related to gene activity.

MYC amplifies gene expression through global changes in transcription factor dynamics

The MYC oncogene has been studied for decades, yet there is still intense debate over how this transcription factor controls gene expression. Here, we seek to answer these questions with an in vivo readout of discrete events of gene expression in single cells. We engineered an optogenetic variant of MYC (Pi-MYC) and combined this tool with single-molecule RNA and protein imaging techniques to investigate the role of MYC in modulating transcriptional bursting and transcription factor binding dynamics in human cells. We find that the immediate consequence of MYC overexpression is an increase in the duration rather than in the frequency of bursts, a functional role that is different from the majority of human transcription factors. We further propose that the mechanism by which MYC exerts global effects on the active period of genes is by altering the binding dynamics of transcription factors involved in RNA polymerase II complex assembly and productive elongation.

The 3D Genome: From Structure to Function

The genome is the most functional part of a cell, and genomic contents are organized in a compact three-dimensional (3D) structure. The genome contains millions of nucleotide bases organized in its proper frame. Rapid development in genome sequencing and advanced microscopy techniques have enabled us to understand the 3D spatial organization of the genome. Chromosome capture methods using a ligation approach and the visualization tool of a 3D genome browser have facilitated detailed exploration of the genome. Topologically associated domains (TADs), lamin-associated domains, CCCTC-binding factor domains, cohesin, and chromatin structures are the prominent identified components that encode the 3D structure of the genome. Although TADs are the major contributors to 3D genome organization, they are absent in Arabidopsis. However, a few research groups have reported the presence of TAD-like structures in the plant kingdom.

A high-throughput DNA FISH protocol to visualize genome regions in human cells

Here, we describe an end-to-end high-throughput imaging protocol to visualize genomic loci in cells at high throughput using DNA fluorescence in situ hybridization, automated microscopy, and computational analysis. This is particularly useful for quantifying patterns of heterogeneity in relative gene positioning or differences within subpopulations of cells. We focus on important experimental design and execution steps in this one-week protocol, suggest ways to ensure and verify data quality, and provide practical solutions to common problems.

For complete details on the generation and use of this protocol, please refer to Finn et al. (2019).

•

DNA fluorescence in situ hybridization technique for high-content imaging

•

Optimized for use in a 384-well plate format with cultured cells

•

One-week protocol to assess thousands of cells per condition in tens of conditions

•

Thorough discussion of troubleshooting for new probes, probe types, and cell types

deepBlink: threshold-independent detection and localization of diffraction-limited spots

Detection of diffraction-limited spots in single-molecule microscopy images is traditionally performed with mathematical operators designed for idealized spots. This process requires manual tuning of parameters that is time-consuming and not always reliable. We have developed deepBlink, a neural network-based method to detect and localize spots automatically. We demonstrate that deepBlink outperforms other state-of-the-art methods across six publicly available datasets containing synthetic and experimental data.

Deep probabilistic tracking of particles in fluorescence microscopy images

Tracking of particles in temporal fluorescence microscopy image sequences is of fundamental importance to quantify dynamic processes of intracellular structures as well as virus structures. We introduce a probabilistic deep learning approach for fluorescent particle tracking, which is based on a recurrent neural network that mimics classical Bayesian filtering. Compared to previous deep learning methods for particle tracking, our approach takes into account uncertainty, both aleatoric and epistemic uncertainty. Thus, information about the reliability of the computed trajectories is determined. Manual tuning of tracking parameters is not necessary and prior knowledge about the noise statistics is not required. Short and long-term temporal dependencies of individual object dynamics are exploited for state prediction, and assigned detections are used to update the predicted states. For correspondence finding, we introduce a neural network which computes assignment probabilities jointly across multiple detections as well as determines the probabilities of missing detections. Training requires only simulated data and therefore tedious manual annotation of ground truth is not needed. We performed a quantitative performance evaluation based on synthetic and real 2D as well as 3D fluorescence microscopy images. We used image data of the Particle Tracking Challenge as well as real time-lapse fluorescence microscopy images displaying virus structures and chromatin structures. It turned out that our approach yields state-of-the-art results or improves the tracking results compared to previous methods.

Pinpointing Cell Identity in Time and Space

Mammalian cells display a broad spectrum of phenotypes, morphologies, and functional niches within biological systems. Our understanding of mechanisms at the individual cellular level, and how cells function in concert to form tissues, organs and systems, has been greatly facilitated by centuries of extensive work to classify and characterize cell types. Classic histological approaches are now complemented with advanced single-cell sequencing and spatial transcriptomics for cell identity studies. Emerging data suggests that additional levels of information should be considered, including the subcellular spatial distribution of molecules such as RNA and protein, when classifying cells. In this Perspective piece we describe the importance of integrating cell transcriptional state with tissue and subcellular spatial and temporal information for thorough characterization of cell type and state. We refer to recent studies making use of single cell RNA-seq and/or image-based cell characterization, which highlight a need for such in-depth characterization of cell populations. We also describe the advances required in experimental, imaging and analytical methods to address these questions. This Perspective concludes by framing this argument in the context of projects such as the Human Cell Atlas, and related fields of cancer research and developmental biology.

Integration of the ImageJ Ecosystem in the KNIME Analytics Platform

Open-source software tools are often used for analysis of scientific image data due to their flexibility and transparency in dealing with rapidly evolving imaging technologies. The complex nature of image analysis problems frequently requires many tools to be used in conjunction, including image processing and analysis, data processing, machine learning and deep learning, statistical analysis of the results, visualization, correlation to heterogeneous but related data, and more. However, the development, and therefore application, of these computational tools is impeded by a lack of integration across platforms. Integration of tools goes beyond convenience, as it is impractical for one tool to anticipate and accommodate the current and future needs of every user. This problem is emphasized in the field of bioimage analysis, where various rapidly emerging methods are quickly being adopted by researchers. ImageJ is a popular open-source image analysis platform, with contributions from a global community resulting in hundreds of specialized routines for a wide array of scientific tasks. ImageJ's strength lies in its accessibility and extensibility, allowing researchers to easily improve the software to solve their image analysis tasks. However, ImageJ is not designed for development of complex end-to-end image analysis workflows. Scientists are often forced to create highly specialized and hard-to-reproduce scripts to orchestrate individual software fragments and cover the entire life-cycle of an analysis of an image dataset. KNIME Analytics Platform, a user-friendly data integration, analysis, and exploration workflow system, was designed to handle huge amounts of heterogeneous data in a platform-agnostic, computing environment and has been successful in meeting complex end-to-end demands in several communities, such as cheminformatics and mass spectrometry. Similar needs within the bioimage analysis community led to the creation of the KNIME Image Processing extension which integrates ImageJ into KNIME Analytics Platform, enabling researchers to develop reproducible and scalable workflows, integrating a diverse range of analysis tools. Here we present how users and developers alike can leverage the ImageJ ecosystem via the KNIME Image Processing extension to provide robust and extensible image analysis within KNIME workflows. We illustrate the benefits of this integration with examples, as well as representative scientific use cases.

A Recurrent Neural Network for Particle Tracking in Microscopy Images Using Future Information, Track Hypotheses, and Multiple Detections

Automatic tracking of particles in time-lapse fluorescence microscopy images is essential for quantifying the dynamic behavior of subcellular structures and virus structures. We introduce a novel particle tracking approach based on a deep recurrent neural network architecture that exploits past and future information in both forward and backward direction. Assignment probabilities are determined jointly across multiple detections, and the probability of missing detections is computed. In addition, existence probabilities are determined by the network to handle track initiation and termination. For correspondence finding, track hypotheses are propagated to future time points so that information at later time points can be used to resolve ambiguities. A handcrafted similarity measure and handcrafted motion features are not necessary. Manually labeled data is not required for network training. We evaluated the performance of our approach using image data of the Particle Tracking Challenge as well as real fluorescence microscopy image sequences of virus structures. It turned out that the proposed approach outperforms previous methods.

Transcriptional Bursting and Co-bursting Regulation by Steroid Hormone Release Pattern and Transcription Factor Mobility

Genes are transcribed in a discontinuous pattern referred to as RNA bursting, but the mechanisms regulating this process are unclear. Although many physiological signals, including glucocorticoid hormones, are pulsatile, the effects of transient stimulation on bursting are unknown. Here we characterize RNA synthesis from single-copy glucocorticoid receptor (GR)-regulated transcription sites (TSs) under pulsed (ultradian) and constant hormone stimulation. In contrast to constant stimulation, pulsed stimulation induces restricted bursting centered around the hormonal pulse. Moreover, we demonstrate that transcription factor (TF) nuclear mobility determines burst duration, whereas its bound fraction determines burst frequency. Using 3D tracking of TSs, we directly correlate TF binding and RNA synthesis at a specific promoter. Finally, we uncover a striking co-bursting pattern between TSs located at proximal and distal positions in the nucleus. Together, our data reveal a dynamic interplay between TF mobility and RNA bursting that is responsive to stimuli strength, type, modality, and duration.

Automated detection of the HER2 gene amplification status in Fluorescence in situ hybridization images for the diagnostics of cancer tissues

The human epidermal growth factor receptor 2 (HER2) gene amplification status is a crucial marker for evaluating clinical therapies of breast or gastric cancer. We propose a deep learning-based pipeline for the detection, localization and classification of interphase nuclei depending on their HER2 gene amplification state in Fluorescence in situ hybridization (FISH) images. Our pipeline combines two RetinaNet-based object localization networks which are trained (1) to detect and classify interphase nuclei into distinct classes normal, low-grade and high-grade and (2) to detect and classify FISH signals into distinct classes HER2 or centromere of chromosome 17 (CEN17). By independently classifying each nucleus twice, the two-step pipeline provides both robustness and interpretability for the automated detection of the HER2 amplification status. The accuracy of our deep learning-based pipeline is on par with that of three pathologists and a set of 57 validation images containing several hundreds of nuclei are accurately classified. The automatic pipeline is a first step towards assisting pathologists in evaluating the HER2 status of tumors using FISH images, for analyzing FISH images in retrospective studies, and for optimizing the documentation of each tumor sample by automatically annotating and reporting of the HER2 gene amplification specificities.

DoGNet: A deep architecture for synapse detection in multiplexed fluorescence images

Neuronal synapses transmit electrochemical signals between cells through the coordinated action of presynaptic vesicles, ion channels, scaffolding and adapter proteins, and membrane receptors. In situ structural characterization of numerous synaptic proteins simultaneously through multiplexed imaging facilitates a bottom-up approach to synapse classification and phenotypic description. Objective automation of efficient and reliable synapse detection within these datasets is essential for the high-throughput investigation of synaptic features. Convolutional neural networks can solve this generalized problem of synapse detection, however, these architectures require large numbers of training examples to optimize their thousands of parameters. We propose DoGNet, a neural network architecture that closes the gap between classical computer vision blob detectors, such as Difference of Gaussians (DoG) filters, and modern convolutional networks. DoGNet is optimized to analyze highly multiplexed microscopy data. Its small number of training parameters allows DoGNet to be trained with few examples, which facilitates its application to new datasets without overfitting. We evaluate the method on multiplexed fluorescence imaging data from both primary mouse neuronal cultures and mouse cortex tissue slices. We show that DoGNet outperforms convolutional networks with a low-to-moderate number of training examples, and DoGNet is efficiently transferred between datasets collected from separate research groups. DoGNet synapse localizations can then be used to guide the segmentation of individual synaptic protein locations and spatial extents, revealing their spatial organization and relative abundances within individual synapses. The source code is publicly available: https://github.com/kulikovv/dognet.

Effects of human sex chromosome dosage on spatial chromosome organization

Sex chromosome aneuploidies (SCAs) are common genetic syndromes characterized by the presence of an aberrant number of X and Y chromosomes due to meiotic defects. These conditions impact the structure and function of diverse tissues, but the proximal effects of SCAs on genome organization are unknown. Here, to determine the consequences of SCAs on global genome organization, we have analyzed multiple architectural features of chromosome organization in a comprehensive set of primary cells from SCA patients with various ratios of X and Y chromosomes by use of imaging-based high-throughput chromosome territory mapping (HiCTMap). We find that X chromosome supernumeracy does not affect the size, volume, or nuclear position of the Y chromosome or an autosomal chromosome. In contrast, the active X chromosome undergoes architectural changes as a function of increasing X copy number as measured by a decrease in size and an increase in circularity, which is indicative of chromatin compaction. In Y chromosome supernumeracy, Y chromosome size is reduced suggesting higher chromatin condensation. The radial positioning of chromosomes is unaffected in SCA karyotypes. Taken together, these observations document changes in genome architecture in response to alterations in sex chromosome numbers and point to trans-effects of dosage compensation on chromosome organization.