Unsupervised discovery of tissue architecture in multiplexed imaging

STModule: identifying tissue modules to uncover spatial components and characteristics of transcriptomic landscapes

Here we present STModule, a Bayesian method developed to identify tissue modules from spatially resolved transcriptomics that reveal spatial components and essential characteristics of tissues. STModule uncovers diverse expression signals in transcriptomic landscapes such as cancer, intraepithelial neoplasia, immune infiltration, outcome-related molecular features and various cell types, which facilitate downstream analysis and provide insights into tumor microenvironments, disease mechanisms, treatment development, and histological organization of tissues. STModule captures a broader spectrum of biological signals compared to other methods and detects novel spatial components. The tissue modules characterized by gene sets demonstrate greater robustness and transferability across different biopsies. STModule: https://github.com/rwang-z/STModule.git .

Characterization of tumour heterogeneity through segmentation-free representation learning on multiplexed imaging data

High-dimensional multiplexed imaging can reveal the spatial organization of tumour tissues at the molecular level. However, owing to the scale and information complexity of the imaging data, it is challenging to discover and thoroughly characterize the heterogeneity of tumour microenvironments. Here we show that self-supervised representation learning on data from imaging mass cytometry can be leveraged to distinguish morphological differences in tumour microenvironments and to precisely characterize distinct microenvironment signatures. We used self-supervised masked image modelling to train a vision transformer that directly takes high-dimensional multiplexed mass-cytometry images. In contrast with traditional spatial analyses relying on cellular segmentation, the vision transformer is segmentation-free, uses pixel-level information, and retains information on the local morphology and biomarker distribution. By applying the vision transformer to a lung-tumour dataset, we identified and validated a monocytic signature that is associated with poor prognosis.

Identification and characterization of cell niches in tissue from spatial omics data at single-cell resolution

Deciphering the features, structure, and functions of the cell niche in tissues remains a major challenge. Here, we present scNiche, a computational framework to identify and characterize cell niches from spatial omics data at single-cell resolution. We benchmark scNiche with both simulated and biological datasets, and demonstrate that scNiche can effectively and robustly identify cell niches while outperforming other existing methods. In spatial proteomics data from human triple-negative breast cancer, scNiche reveals the influence of the microenvironment on cellular phenotypes, and further dissects patient-specific niches with distinct cellular compositions or phenotypic characteristics. By analyzing mouse liver spatial transcriptomics data across normal and early-onset liver failure donors, scNiche uncovers disease-specific liver injury niches, and further delineates the niche remodeling from normal liver to liver failure. Overall, scNiche enables decoding the cellular microenvironment in tissues from single-cell spatial omics data.

Spatial Patterning Analysis of Cellular Ensembles (SPACE) finds complex spatial organization at the cell and tissue levels

Spatial patterns of cells and other biological elements drive physiologic and pathologic processes within tissues. While many imaging and transcriptomic methods document tissue organization, discerning these patterns is challenging, especially when they involve multiple elements in complex arrangements. To address this challenge, we present Spatial Patterning Analysis of Cellular Ensembles (SPACE), an R package for analysis of high-plex spatial data. SPACE is compatible with any data collection modality that records values (i.e., categorical cell/structure types or quantitative expression levels) at fixed spatial coordinates (i.e., 2d pixels or 3d voxels). SPACE detects not only broad patterns of co-occurrence but also context-dependent associations, quantitative gradients and orientations, and other organizational complexities. Via a robust information theoretic framework, SPACE explores all possible ensembles of tissue elements-single elements, pairs, triplets, and so on-and ranks the most strongly patterned ensembles. For single images, rankings reflect differences from random assortment. For sets of images, rankings reflect differences across sample groups (e.g., genotypes, treatments, timepoints, etc.). Further tools then characterize the nature of each pattern for intuitive interpretation. We validate SPACE and demonstrate its advantages using murine lymph node images for which ground truth has been defined. We then detect new patterns across varied datasets, including tumors and tuberculosis granulomas.

Powerful and accurate case-control analysis of spatial molecular data with deep learning-defined tissue microniches

As spatial molecular data grow in scope and resolution, there is a pressing need to identify key spatial structures associated with disease. Current approaches often rely on hand-crafted features such as local abundances of manually annotated, discrete cell types, which may overlook important signals. Here we introduce variational inference-based microniche analysis (VIMA), a method that combines deep learning with principled statistics to discover associated spatial features with greater flexibility and precision. VIMA uses a variational autoencoder to extract numerical "fingerprints" from small tissue patches that capture their biological content. It uses these fingerprints to define a large number of "microniches" - small, potentially overlapping groups of tissue patches with highly similar biology that span multiple samples. It then uses rigorous statistics to identify microniches whose abundance correlates with case-control status. We show in simulations that VIMA is well calibrated and more powerful and accurate than other approaches. We then apply VIMA to a 140-gene spatial transcriptomics dataset in Alzheimer's dementia, a 54-marker CO-Detection by indEXing (CODEX) dataset in ulcerative colitis (UC), and a 7-marker immunohistochemistry dataset in rheumatoid arthritis (RA), in each case recapitulating known biology and identifying novel spatial features of disease.

Frontiers in fluorescence imaging: tools for the in situ sensing of disease biomarkers

Fluorescence imaging has been recognized as a powerful tool for the real-time detection and specific imaging of biomarkers within living systems, which is crucial for early diagnosis and treatment evaluation of major diseases. Over the years, significant advancements in this field have been achieved, particularly with the development of novel fluorescent probes and advanced imaging technologies such as NIR-II imaging, super-resolution imaging, and 3D imaging. These technologies have enabled deeper tissue penetration, higher image contrast, and more accurate detection of disease-related biomarkers. Despite these advancements, challenges such as improving probe specificity, enhancing imaging depth and resolution, and optimizing signal-to-noise ratios still remain. The emergence of artificial intelligence (AI) has injected new vitality into the designs and performances of fluorescent probes, offering new tools for more precise disease diagnosis. This review will not only discuss chemical modifications of classic fluorophores and in situ visualization of various biomarkers including metal ions, reactive species, and enzymes, but also share some breakthroughs in AI-driven fluorescence imaging, aiming to provide a comprehensive understanding of these advancements. Future prospects of fluorescence imaging for biomarkers including the potential impact of AI in this rapidly evolving field are also highlighted.

Profiling cell identity and tissue architecture with single-cell and spatial transcriptomics

Single-cell transcriptomics has broadened our understanding of cellular diversity and gene expression dynamics in healthy and diseased tissues. Recently, spatial transcriptomics has emerged as a tool to contextualize single cells in multicellular neighbourhoods and to identify spatially recurrent phenotypes, or ecotypes. These technologies have generated vast datasets with targeted-transcriptome and whole-transcriptome profiles of hundreds to millions of cells. Such data have provided new insights into developmental hierarchies, cellular plasticity and diverse tissue microenvironments, and spurred a burst of innovation in computational methods for single-cell analysis. In this Review, we discuss recent advancements, ongoing challenges and prospects in identifying and characterizing cell states and multicellular neighbourhoods. We discuss recent progress in sample processing, data integration, identification of subtle cell states, trajectory modelling, deconvolution and spatial analysis. Furthermore, we discuss the increasing application of deep learning, including foundation models, in analysing single-cell and spatial transcriptomics data. Finally, we discuss recent applications of these tools in the fields of stem cell biology, immunology, and tumour biology, and the future of single-cell and spatial transcriptomics in biological research and its translation to the clinic.

Decoding Spatial Tissue Architecture: A Scalable Bayesian Topic Model for Multiplexed Imaging Analysis

Recent progress in multiplexed tissue imaging is advancing the study of tumor microenvironments to enhance our understanding of treatment response and disease progression. Cellular neighborhood analysis is a popular computational approach for these complex image data. Despite its popularity, there are significant challenges, including high computational demands that limit feasibility for large-scale applications and the lack of a principled strategy for integrative analysis across images. This absence hampers the precise and consistent identification of spatial features and tracking of their dynamics over disease progression. To overcome these challenges, we introduce SpatialTopic, a spatial topic model designed to decode high-level spatial architecture across multiplexed tissue images. SpatialTopic integrates both cell type and spatial information within a topic modelling framework, originally developed for natural language processing and adapted for computer vision. Spatial information is incorporated into the flexible design of documents, representing densely overlapping regions in images. We employ an efficient collapsed Gibbs sampling algorithm for model inference. We benchmarked the performance against five state-of-the-art algorithms through various case studies using different single-cell spatial transcriptomic and proteomic imaging platforms across different tissue types. We show that SpatialTopic is highly scalable on large-scale image datasets with millions of cells, along with high precision and interpretability. Our findings demonstrate that SpatialTopic consistently identifies biologically and clinically significant spatial "topics" such as tertiary lymphoid structures (TLSs) and tracks dynamic changes in spatial features over disease progression. Its computational efficiency and broad applicability across various molecular imaging platforms will enhance the analysis of large-scale tissue imaging datasets.

High-parametric protein maps reveal the spatial organization in early-developing human lung

The respiratory system, including the lungs, is essential for terrestrial life. While recent research has advanced our understanding of lung development, much still relies on animal models and transcriptome analyses. In this study conducted within the Human Developmental Cell Atlas (HDCA) initiative, we describe the protein-level spatiotemporal organization of the lung during the first trimester of human gestation. Using high-parametric tissue imaging with a 30-plex antibody panel, we analyzed human lung samples from 6 to 13 post-conception weeks, generating data from over 2 million cells across five developmental timepoints. We present a resource detailing spatially resolved cell type composition of the developing human lung, including proliferative states, immune cell patterns, spatial arrangement traits, and their temporal evolution. This represents an extensive single-cell resolved protein-level examination of the developing human lung and provides a valuable resource for further research into the developmental roots of human respiratory health and disease.

Consensus tissue domain detection in spatial omics data using multiplex image labeling with regional morphology (MILWRM)

Spatially resolved molecular assays provide high dimensional genetic, transcriptomic, proteomic, and epigenetic information in situ and at various resolutions. Pairing these data across modalities with histological features enables powerful studies of tissue pathology in the context of an intact microenvironment and tissue structure. Increasing dimensions across molecular analytes and samples require new data science approaches to functionally annotate spatially resolved molecular data. A specific challenge is data-driven cross-sample domain detection that allows for analysis within and between consensus tissue compartments across high volumes of multiplex datasets stemming from tissue atlasing efforts. Here, we present MILWRM (multiplex image labeling with regional morphology)-a Python package for rapid, multi-scale tissue domain detection and annotation at the image- or spot-level. We demonstrate MILWRM's utility in identifying histologically distinct compartments in human colonic polyps, lymph nodes, mouse kidney, and mouse brain slices through spatially-informed clustering in two different spatial data modalities from different platforms. We used tissue domains detected in human colonic polyps to elucidate the molecular distinction between polyp subtypes, and explored the ability of MILWRM to identify anatomical regions of the brain tissue and their respective distinct molecular profiles.

Cross-expression analysis reveals patterns of coordinated gene expression in spatial transcriptomics

Spatial transcriptomics promises to transform our understanding of tissue biology by molecularly profiling individual cells in situ. A fundamental question they allow us to ask is how nearby cells orchestrate their gene expression. To investigate this, we introduce cross-expression, a novel framework for discovering gene pairs that coordinate their expression across neighboring cells. Just as co-expression quantifies synchronized gene expression within the same cells, cross-expression measures coordinated gene expression between spatially adjacent cells, allowing us to understand tissue gene expression programs with single cell resolution. Using this framework, we recover ligand-receptor partners and discover gene combinations marking anatomical regions. More generally, we create cross-expression networks to find gene modules with orchestrated expression patterns. Finally, we provide an efficient R package to facilitate cross-expression analysis, quantify effect sizes, and generate novel visualizations to better understand spatial gene expression programs.

Characterization of tumor heterogeneity through segmentation-free representation learning

The interaction between tumors and their microenvironment is complex and heterogeneous. Recent developments in high-dimensional multiplexed imaging have revealed the spatial organization of tumor tissues at the molecular level. However, the discovery and thorough characterization of the tumor microenvironment (TME) remains challenging due to the scale and complexity of the images. Here, we propose a self-supervised representation learning framework, CANVAS, that enables discovery of novel types of TMEs. CANVAS is a vision transformer that directly takes high-dimensional multiplexed images and is trained using self-supervised masked image modeling. In contrast to traditional spatial analysis approaches which rely on cell segmentations, CANVAS is segmentation-free, utilizes pixel-level information, and retains local morphology and biomarker distribution information. This approach allows the model to distinguish subtle morphological differences, leading to precise separation and characterization of distinct TME signatures. We applied CANVAS to a lung tumor dataset and identified and validated a monocytic signature that is associated with poor prognosis.

A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics

Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT-DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.

Discovery and generalization of tissue structures from spatial omics data

Tissues are organized into anatomical and functional units at different scales. New technologies for high-dimensional molecular profiling in situ have enabled the characterization of structure-function relationships in increasing molecular detail. However, it remains a challenge to consistently identify key functional units across experiments, tissues, and disease contexts, a task that demands extensive manual annotation. Here, we present spatial cellular graph partitioning (SCGP), a flexible method for the unsupervised annotation of tissue structures. We further present a reference-query extension pipeline, SCGP-Extension, that generalizes reference tissue structure labels to previously unseen samples, performing data integration and tissue structure discovery. Our experiments demonstrate reliable, robust partitioning of spatial data in a wide variety of contexts and best-in-class accuracy in identifying expertly annotated structures. Downstream analysis on SCGP-identified tissue structures reveals disease-relevant insights regarding diabetic kidney disease, skin disorder, and neoplastic diseases, underscoring its potential to drive biological insight and discovery from spatial datasets.

CNTools: A computational toolbox for cellular neighborhood analysis from multiplexed images

Recent studies show that cellular neighborhoods play an important role in evolving biological events such as cancer and diabetes. Therefore, it is critical to accurately and efficiently identify cellular neighborhoods from spatially-resolved single-cell transcriptomic data or single-cell resolution tissue imaging data. In this work, we develop CNTools, a computational toolbox for end-to-end cellular neighborhood analysis on annotated cell images, comprising both the identification and analysis steps. It includes state-of-the-art cellular neighborhood identification methods and post-identification smoothing techniques, with our newly proposed Cellular Neighbor Embedding (CNE) method and Naive Smoothing technique, as well as several established downstream analysis approaches. We applied CNTools on three real-world CODEX datasets and evaluated identification methods with smoothing techniques quantitatively and qualitatively. It shows that CNE with Naive Smoothing overall outperformed other methods and revealed more convincing biological insights. We also provided suggestions on how to choose proper identification methods and smoothing techniques according to input data.

Multi-parametric atlas of the pre-metastatic liver for prediction of metastatic outcome in early-stage pancreatic cancer

Metastasis occurs frequently after resection of pancreatic cancer (PaC). In this study, we hypothesized that multi-parametric analysis of pre-metastatic liver biopsies would classify patients according to their metastatic risk, timing and organ site. Liver biopsies obtained during pancreatectomy from 49 patients with localized PaC and 19 control patients with non-cancerous pancreatic lesions were analyzed, combining metabolomic, tissue and single-cell transcriptomics and multiplex imaging approaches. Patients were followed prospectively (median 3 years) and classified into four recurrence groups; early (<6 months after resection) or late (>6 months after resection) liver metastasis (LiM); extrahepatic metastasis (EHM); and disease-free survivors (no evidence of disease (NED)). Overall, PaC livers exhibited signs of augmented inflammation compared to controls. Enrichment of neutrophil extracellular traps (NETs), Ki-67 upregulation and decreased liver creatine significantly distinguished those with future metastasis from NED. Patients with future LiM were characterized by scant T cell lobular infiltration, less steatosis and higher levels of citrullinated H3 compared to patients who developed EHM, who had overexpression of interferon target genes (MX1 and NR1D1) and an increase of CD11B+ natural killer (NK) cells. Upregulation of sortilin-1 and prominent NETs, together with the lack of T cells and a reduction in CD11B+ NK cells, differentiated patients with early-onset LiM from those with late-onset LiM. Liver profiles of NED closely resembled those of controls. Using the above parameters, a machine-learning-based model was developed that successfully predicted the metastatic outcome at the time of surgery with 78% accuracy. Therefore, multi-parametric profiling of liver biopsies at the time of PaC diagnosis may determine metastatic risk and organotropism and guide clinical stratification for optimal treatment selection.

Deep cell phenotyping and spatial analysis of multiplexed imaging with TRACERx-PHLEX

The growing scale and dimensionality of multiplexed imaging require reproducible and comprehensive yet user-friendly computational pipelines. TRACERx-PHLEX performs deep learning-based cell segmentation (deep-imcyto), automated cell-type annotation (TYPEx) and interpretable spatial analysis (Spatial-PHLEX) as three independent but interoperable modules. PHLEX generates single-cell identities, cell densities within tissue compartments, marker positivity calls and spatial metrics such as cellular barrier scores, along with summary graphs and spatial visualisations. PHLEX was developed using imaging mass cytometry (IMC) in the TRACERx study, validated using published Co-detection by indexing (CODEX), IMC and orthogonal data and benchmarked against state-of-the-art approaches. We evaluated its use on different tissue types, tissue fixation conditions, image sizes and antibody panels. As PHLEX is an automated and containerised Nextflow pipeline, manual assessment, programming skills or pathology expertise are not essential. PHLEX offers an end-to-end solution in a growing field of highly multiplexed data and provides clinically relevant insights.

Cross-domain information fusion for enhanced cell population delineation in single-cell spatial-omics data

Cell population delineation and identification is an essential step in single-cell and spatial-omics studies. Spatial-omics technologies can simultaneously measure information from three complementary domains related to this task: expression levels of a panel of molecular biomarkers at single-cell resolution, relative positions of cells, and images of tissue sections, but existing computational methods for performing this task on single-cell spatial-omics datasets often relinquish information from one or more domains. The additional reliance on the availability of "atlas" training or reference datasets limits cell type discovery to well-defined but limited cell population labels, thus posing major challenges for using these methods in practice. Successful integration of all three domains presents an opportunity for uncovering cell populations that are functionally stratified by their spatial contexts at cellular and tissue levels: the key motivation for employing spatial-omics technologies in the first place. In this work, we introduce Cell Spatio- and Neighborhood-informed Annotation and Patterning (CellSNAP), a self-supervised computational method that learns a representation vector for each cell in tissue samples measured by spatial-omics technologies at the single-cell or finer resolution. The learned representation vector fuses information about the corresponding cell across all three aforementioned domains. By applying CellSNAP to datasets spanning both spatial proteomic and spatial transcriptomic modalities, and across different tissue types and disease settings, we show that CellSNAP markedly enhances de novo discovery of biologically relevant cell populations at fine granularity, beyond current approaches, by fully integrating cells' molecular profiles with cellular neighborhood and tissue image information.

Precise immunofluorescence canceling for highly multiplexed imaging to capture specific cell states

Cell states are regulated by the response of signaling pathways to receptor ligand-binding and intercellular interactions. High-resolution imaging has been attempted to explore the dynamics of these processes and, recently, multiplexed imaging has profiled cell states by achieving a comprehensive acquisition of spatial protein information from cells. However, the specificity of antibodies is still compromised when visualizing activated signals. Here, we develop Precise Emission Canceling Antibodies (PECAbs) that have cleavable fluorescent labeling. PECAbs enable high-specificity sequential imaging using hundreds of antibodies, allowing for reconstruction of the spatiotemporal dynamics of signaling pathways. Additionally, combining this approach with seq-smFISH can effectively classify cells and identify their signal activation states in human tissue. Overall, the PECAb system can serve as a comprehensive platform for analyzing complex cell processes.

Spatial Biology of Breast Cancer

Spatial findings have shaped on our understanding of breast cancer. In this review, we discuss how spatial methods, including spatial transcriptomics and proteomics and the resultant understanding of spatial relationships, have contributed to concepts regarding cancer progression and treatment. In addition to discussing traditional approaches, we examine how emerging multiplex imaging technologies have contributed to the field and how they might influence future research.

Complementing Cell Taxonomies with a Multicellular Analysis of Tissues

The application of single-cell molecular profiling coupled with spatial technologies has enabled charting of cellular heterogeneity in reference tissues and in disease. This new wave of molecular data has highlighted the expected diversity of single-cell dynamics upon shared external queues and spatial organizations. However, little is known about the relationship between single-cell heterogeneity and the emergence and maintenance of robust multicellular processes in developed tissues and its role in (patho)physiology. Here, we present emerging computational modeling strategies that use increasingly available large-scale cross-condition single-cell and spatial datasets to study multicellular organization in tissues and complement cell taxonomies. This perspective should enable us to better understand how cells within tissues collectively process information and adapt synchronized responses in disease contexts and to bridge the gap between structural changes and functions in tissues.

Cellular architecture of evolving neuroinflammatory lesions and multiple sclerosis pathology

Multiple sclerosis (MS) is a neurological disease characterized by multifocal lesions and smoldering pathology. Although single-cell analyses provided insights into cytopathology, evolving cellular processes underlying MS remain poorly understood. We investigated the cellular dynamics of MS by modeling temporal and regional rates of disease progression in mouse experimental autoimmune encephalomyelitis (EAE). By performing single-cell spatial expression profiling using in situ sequencing (ISS), we annotated disease neighborhoods and found centrifugal evolution of active lesions. We demonstrated that disease-associated (DA)-glia arise independently of lesions and are dynamically induced and resolved over the disease course. Single-cell spatial mapping of human archival MS spinal cords confirmed the differential distribution of homeostatic and DA-glia, enabled deconvolution of active and inactive lesions into sub-compartments, and identified new lesion areas. By establishing a spatial resource of mouse and human MS neuropathology at a single-cell resolution, our study unveils the intricate cellular dynamics underlying MS.

Exploring the single-cell immune landscape of kidney allograft inflammation using imaging mass cytometry

Kidney allograft inflammation, mostly attributed to rejection and infection, is an important cause of graft injury and loss. Standard histopathological assessment of allograft inflammation provides limited insights into biological processes and the immune landscape. Here, using imaging mass cytometry with a panel of 28 validated biomarkers, we explored the single-cell landscape of kidney allograft inflammation in 32 kidney transplant biopsies and 247 high-dimensional histopathology images of various phenotypes of allograft inflammation (antibody-mediated rejection, T cell-mediated rejection, BK nephropathy, and chronic pyelonephritis). Using novel analytical tools, for cell segmentation, we segmented over 900 000 cells and developed a tissue-based classifier using over 3000 manually annotated kidney microstructures (glomeruli, tubules, interstitium, and arteries). Using PhenoGraph, we identified 11 immune and 9 nonimmune clusters and found a high prevalence of memory T cell and macrophage-enriched immune populations across phenotypes. Additionally, we trained a machine learning classifier to identify spatial biomarkers that could discriminate between the different allograft inflammatory phenotypes. Further validation of imaging mass cytometry in larger cohorts and with more biomarkers will likely help interrogate kidney allograft inflammation in more depth than has been possible to date.

SpiDe-Sr: blind super-resolution network for precise cell segmentation and clustering in spatial proteomics imaging

Spatial proteomics elucidates cellular biochemical changes with unprecedented topological level. Imaging mass cytometry (IMC) is a high-dimensional single-cell resolution platform for targeted spatial proteomics. However, the precision of subsequent clinical analysis is constrained by imaging noise and resolution. Here, we propose SpiDe-Sr, a super-resolution network embedded with a denoising module for IMC spatial resolution enhancement. SpiDe-Sr effectively resists noise and improves resolution by 4 times. We demonstrate SpiDe-Sr respectively with cells, mouse and human tissues, resulting 18.95%/27.27%/21.16% increase in peak signal-to-noise ratio and 15.95%/31.63%/15.52% increase in cell extraction accuracy. We further apply SpiDe-Sr to study the tumor microenvironment of a 20-patient clinical breast cancer cohort with 269,556 single cells, and discover the invasion of Gram-negative bacteria is positively correlated with carcinogenesis markers and negatively correlated with immunological markers. Additionally, SpiDe-Sr is also compatible with fluorescence microscopy imaging, suggesting SpiDe-Sr an alternative tool for microscopy image super-resolution.

Mapping cell-to-tissue graphs across human placenta histology whole slide images using deep learning with HAPPY

Accurate placenta pathology assessment is essential for managing maternal and newborn health, but the placenta's heterogeneity and temporal variability pose challenges for histology analysis. To address this issue, we developed the 'Histology Analysis Pipeline.PY' (HAPPY), a deep learning hierarchical method for quantifying the variability of cells and micro-anatomical tissue structures across placenta histology whole slide images. HAPPY differs from patch-based features or segmentation approaches by following an interpretable biological hierarchy, representing cells and cellular communities within tissues at a single-cell resolution across whole slide images. We present a set of quantitative metrics from healthy term placentas as a baseline for future assessments of placenta health and we show how these metrics deviate in placentas with clinically significant placental infarction. HAPPY's cell and tissue predictions closely replicate those from independent clinical experts and placental biology literature.

Unsupervised and supervised discovery of tissue cellular neighborhoods from cell phenotypes

It is poorly understood how different cells in a tissue organize themselves to support tissue functions. We describe the CytoCommunity algorithm for the identification of tissue cellular neighborhoods (TCNs) based on cell phenotypes and their spatial distributions. CytoCommunity learns a mapping directly from the cell phenotype space to the TCN space using a graph neural network model without intermediate clustering of cell embeddings. By leveraging graph pooling, CytoCommunity enables de novo identification of condition-specific and predictive TCNs under the supervision of sample labels. Using several types of spatial omics data, we demonstrate that CytoCommunity can identify TCNs of variable sizes with substantial improvement over existing methods. By analyzing risk-stratified colorectal and breast cancer data, CytoCommunity revealed new granulocyte-enriched and cancer-associated fibroblast-enriched TCNs specific to high-risk tumors and altered interactions between neoplastic and immune or stromal cells within and between TCNs. CytoCommunity can perform unsupervised and supervised analyses of spatial omics maps and enable the discovery of condition-specific cell-cell communication patterns across spatial scales.

Unsupervised spatially embedded deep representation of spatial transcriptomics

Optimal integration of transcriptomics data and associated spatial information is essential towards fully exploiting spatial transcriptomics to dissect tissue heterogeneity and map out inter-cellular communications. We present SEDR, which uses a deep autoencoder coupled with a masked self-supervised learning mechanism to construct a low-dimensional latent representation of gene expression, which is then simultaneously embedded with the corresponding spatial information through a variational graph autoencoder. SEDR achieved higher clustering performance on manually annotated 10 × Visium datasets and better scalability on high-resolution spatial transcriptomics datasets than existing methods. Additionally, we show SEDR's ability to impute and denoise gene expression (URL: https://github.com/JinmiaoChenLab/SEDR/ ).

Niche-DE: niche-differential gene expression analysis in spatial transcriptomics data identifies context-dependent cell-cell interactions

Existing methods for analysis of spatial transcriptomic data focus on delineating the global gene expression variations of cell types across the tissue, rather than local gene expression changes driven by cell-cell interactions. We propose a new statistical procedure called niche-differential expression (niche-DE) analysis that identifies cell-type-specific niche-associated genes, which are differentially expressed within a specific cell type in the context of specific spatial niches. We further develop niche-LR, a method to reveal ligand-receptor signaling mechanisms that underlie niche-differential gene expression patterns. Niche-DE and niche-LR are applicable to low-resolution spot-based spatial transcriptomics data and data that is single-cell or subcellular in resolution.

CellCharter reveals spatial cell niches associated with tissue remodeling and cell plasticity

Tissues are organized in cellular niches, the composition and interactions of which can be investigated using spatial omics technologies. However, systematic analyses of tissue composition are challenged by the scale and diversity of the data. Here we present CellCharter, an algorithmic framework to identify, characterize, and compare cellular niches in spatially resolved datasets. CellCharter outperformed existing approaches and effectively identified cellular niches across datasets generated using different technologies, and comprising hundreds of samples and millions of cells. In multiple human lung cancer cohorts, CellCharter uncovered a cellular niche composed of tumor-associated neutrophil and cancer cells expressing markers of hypoxia and cell migration. This cancer cell state was spatially segregated from more proliferative tumor cell clusters and was associated with tumor-associated neutrophil infiltration and poor prognosis in independent patient cohorts. Overall, CellCharter enables systematic analyses across data types and technologies to decode the link between spatial tissue architectures and cell plasticity.

Unsupervised Analysis of Optical Imaging Data for the Discovery of Reactivity Patterns in Metal Alloy

Operando wide-field optical microscopy imaging yields a wealth of information about the reactivity of metal interfaces, yet the data are often unstructured and challenging to process. In this study, the power of unsupervised machine learning (ML) algorithms is harnessed to analyze chemical reactivity images obtained dynamically by reflectivity microscopy in combination with ex situ scanning electron microscopy to identify and cluster the chemical reactivity of particles in Al alloy. The ML analysis uncovers three distinct clusters of reactivity from unlabeled datasets. A detailed examination of representative reactivity patterns confirms the chemical communication of generated OH- fluxes within particles, as supported by statistical analysis of size distribution and finite element modelling (FEM). The ML procedures also reveal statistically significant patterns of reactivity under dynamic conditions, such as pH acidification. The results align well with a numerical model of chemical communication, underscoring the synergy between data-driven ML and physics-driven FEM approaches.

Mapping human natural killer cell development in pediatric tonsil by imaging mass cytometry and high-resolution microscopy

Natural killer (NK) cells develop from CD34+ progenitors in a stage-specific manner defined by changes in cell surface receptor expression and function. Secondary lymphoid tissues, including tonsil, are sites of human NK cell development. Here we present new insights into human NK cell development in pediatric tonsil using cyclic immunofluorescence and imaging mass cytometry. We show that NK cell subset localization and interactions are dependent on NK cell developmental stage and tissue residency. NK cell progenitors are found in the interfollicular domain in proximity to cytokine-expressing stromal cells that promote proliferation and maturation. Mature NK cells are primarily found in the T-cell rich parafollicular domain engaging in cell-cell interactions that differ depending on their stage and tissue residency. The presence of local inflammation results in changes in NK cell interactions, abundance, and localization. This study provides the first comprehensive atlas of human NK cell development in secondary lymphoid tissue.

Application of digital pathology-based advanced analytics of tumour microenvironment organisation to predict prognosis and therapeutic response

In recent years, the application of advanced analytics, especially artificial intelligence (AI), to digital H&E images, and other histological image types, has begun to radically change how histological images are used in the clinic. Alongside the recognition that the tumour microenvironment (TME) has a profound impact on tumour phenotype, the technical development of highly multiplexed immunofluorescence platforms has enhanced the biological complexity that can be captured in the TME with high precision. AI has an increasingly powerful role in the recognition and quantitation of image features and the association of such features with clinically important outcomes, as occurs in distinct stages in conventional machine learning. Deep-learning algorithms are able to elucidate TME patterns inherent in the input data with minimum levels of human intelligence and, hence, have the potential to achieve clinically relevant predictions and discovery of important TME features. Furthermore, the diverse repertoire of deep-learning algorithms able to interrogate TME patterns extends beyond convolutional neural networks to include attention-based models, graph neural networks, and multimodal models. To date, AI models have largely been evaluated retrospectively, outside the well-established rigour of prospective clinical trials, in part because traditional clinical trial methodology may not always be suitable for the assessment of AI technology. However, to enable digital pathology-based advanced analytics to meaningfully impact clinical care, specific measures of 'added benefit' to the current standard of care and validation in a prospective setting are important. This will need to be accompanied by adequate measures of explainability and interpretability. Despite such challenges, the combination of expanding datasets, increased computational power, and the possibility of integration of pre-clinical experimental insights into model development means there is exciting potential for the future progress of these AI applications. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

A review on deep learning applications in highly multiplexed tissue imaging data analysis

Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients. Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of the DL-based pipelines used in preprocessing the highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.

Complex interactions of cellular players in chronic Graft-versus-Host Disease

Chronic Graft-versus-Host Disease is a life-threatening inflammatory condition that affects many patients after allogeneic hematopoietic stem cell transplantation. Although we have made substantial progress in understanding disease pathogenesis and the role of specific immune cell subsets, treatment options are still limited. To date, we lack a global understanding of the interplay between the different cellular players involved, in the affected tissues and at different stages of disease development and progression. In this review we summarize our current knowledge on pathogenic and protective mechanisms elicited by the major involved immune subsets, being T cells, B cells, NK cells and antigen presenting cells, as well as the microbiome, with a special focus on intercellular communication of these cell types via extracellular vesicles as up-and-coming fields in chronic Graft-versus-Host Disease research. Lastly, we discuss the importance of understanding systemic and local aberrant cell communication during disease for defining better biomarkers and therapeutic targets, eventually enabling the design of personalized treatment schemes.

Interpretable spatial cell learning enhances the characterization of patient tissue microenvironments with highly multiplexed imaging data

Multiplexed imaging technologies enable highly resolved spatial characterization of cellular environments. However, exploiting these rich spatial cell datasets for biological insight is a considerable analytical challenge. In particular, effective approaches to define disease-specific microenvironments on the basis of clinical outcomes is a complex problem with immediate pathological value. Here we present InterSTELLAR, a geometric deep learning framework for multiplexed imaging data, to directly link tissue subtypes with corresponding cell communities that have clinical relevance. Using a publicly available breast cancer imaging mass cytometry dataset, InterSTELLAR allows simultaneous tissue type prediction and interested community detection, with improved performance over conventional methods. Downstream analyses demonstrate InterSTELLAR is able to capture specific pathological features from different clinical cancer subtypes. The method is able to reveal potential relationships between these regions and patient prognosis. InterSTELLAR represents an application of geometric deep learning with direct benefits for extracting enhanced microenvironment characterization for multiplexed imaging of patient samples.

A deep learning method to map tissue architecture

A new study in Nature Methods describes a computational method named UTAG (unsupervised discovery of tissue architecture with graphs) that aims to identify and quantify higher-level tissue domains from biological images without previous knowledge.