Deciphering tumor ecosystems at super resolution from spatial transcriptomics with TESLA

STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics

Recent advances in multi-modal algorithms have driven and been driven by the increasing availability of large image-text datasets, leading to significant strides in various fields, including computational pathology. However, in most existing medical image-text datasets, the text typically provides high-level summaries that may not sufficiently describe sub-tile regions within a large pathology image. For example, an image might cover an extensive tissue area containing cancerous and healthy regions, but the accompanying text might only specify that this image is a cancer slide, lacking the nuanced details needed for in-depth analysis. In this study, we introduce STimage-1K4M, a novel dataset designed to bridge this gap by providing genomic features for sub-tile images. STimage-1K4M contains 1,149 images derived from spatial transcriptomics data, which captures gene expression information at the level of individual spatial spots within a pathology image. Specifically, each image in the dataset is broken down into smaller sub-image tiles, with each tile paired with 15,000 - 30,000 dimensional gene expressions. With 4,293,195 pairs of sub-tile images and gene expressions, STimage-1K4M offers unprecedented granularity, paving the way for a wide range of advanced research in multi-modal data analysis an innovative applications in computational pathology, and beyond.

Inferring single-cell spatial gene expression with tissue morphology via explainable deep learning

The spatial arrangement of cells is vital in developmental processes and organogenesis in multicellular life forms. Deep learning models trained with spatial omics data uncover complex patterns and relationships among cells, genes, and proteins in a high-dimensional space, providing new insights into biological processes and diseases. State-of-the-art in silico spatial multi-cell gene expression methods using histological images of tissue stained with hematoxylin and eosin (H&E) to characterize cellular heterogeneity. These computational techniques offer the advantage of analyzing vast amounts of spatial data in a scalable and automated manner, thereby accelerating scientific discovery and enabling more precise medical diagnostics and treatments. In this work, we developed a vision transformer (ViT) framework to map histological signatures to spatial single-cell transcriptomic signatures, named SPiRiT ( S patial Omics P rediction and R eproducibility integrated T ransformer). Our framework was enhanced by integrating cross validation with model interpretation during hyper-parameter tuning. SPiRiT predicts single-cell spatial gene expression using the matched histopathological image tiles of human breast cancer and whole mouse pup, evaluated by Xenium (10x Genomics) datasets. Furthermore, ViT model interpretation reveals the high-resolution, high attention area (HAR) that the ViT model uses to predict the gene expression, including marker genes for invasive cancer cells ( FASN ), stromal cells ( POSTN ), and lymphocytes ( IL7R ). In an apple-to-apple comparison with the ST-Net Convolutional Neural Network algorithm, SPiRiT improved predictive accuracy by 40% using human breast cancer Visium (10x Genomics) dataset. Cancer biomarker gene prediction and expression level are highly consistent with the tumor region annotation. In summary, our work highlights the feasibility to infer spatial single-cell gene expression using tissue morphology in multiple-species, i.e., human and mouse, and multi-organs, i.e., mouse whole body morphology. Importantly, incorporating model interpretation and vision transformer is expected to serve as a general-purpose framework for spatial transcriptomics.

Tertiary lymphoid structural heterogeneity determines tumour immunity and prospects for clinical application

Tertiary lymphoid structures (TLS) are clusters of immune cells that resemble and function similarly to secondary lymphoid organs (SLOs). While TLS is generally associated with an anti-tumour immune response in most cancer types, it has also been observed to act as a pro-tumour immune response. The heterogeneity of TLS function is largely determined by the composition of tumour-infiltrating lymphocytes (TILs) and the balance of cell subsets within the tumour-associated TLS (TA-TLS). TA-TLS of varying maturity, density, and location may have opposing effects on tumour immunity. Higher maturity and/or higher density TLS are often associated with favorable clinical outcomes and immunotherapeutic response, mainly due to crosstalk between different proportions of immune cell subpopulations in TA-TLS. Therefore, TLS can be used as a marker to predict the efficacy of immunotherapy in immune checkpoint blockade (ICB). Developing efficient imaging and induction methods to study TA-TLS is crucial for enhancing anti-tumour immunity. The integration of imaging techniques with biological materials, including nanoprobes and hydrogels, alongside artificial intelligence (AI), enables non-invasive in vivo visualization of TLS. In this review, we explore the dynamic interactions among T and B cell subpopulations of varying phenotypes that contribute to the structural and functional diversity of TLS, examining both existing and emerging techniques for TLS imaging and induction, focusing on cancer immunotherapies and biomaterials. We also highlight novel therapeutic approaches of TLS that are being explored with the aim of increasing ICB treatment efficacy and predicting prognosis.

An atlas of epithelial cell states and plasticity in lung adenocarcinoma

Understanding the cellular processes that underlie early lung adenocarcinoma (LUAD) development is needed to devise intervention strategies1. Here we studied 246,102 single epithelial cells from 16 early-stage LUADs and 47 matched normal lung samples. Epithelial cells comprised diverse normal and cancer cell states, and diversity among cancer cells was strongly linked to LUAD-specific oncogenic drivers. KRAS mutant cancer cells showed distinct transcriptional features, reduced differentiation and low levels of aneuploidy. Non-malignant areas surrounding human LUAD samples were enriched with alveolar intermediate cells that displayed elevated KRT8 expression (termed KRT8+ alveolar intermediate cells (KACs) here), reduced differentiation, increased plasticity and driver KRAS mutations. Expression profiles of KACs were enriched in lung precancer cells and in LUAD cells and signified poor survival. In mice exposed to tobacco carcinogen, KACs emerged before lung tumours and persisted for months after cessation of carcinogen exposure. Moreover, they acquired Kras mutations and conveyed sensitivity to targeted KRAS inhibition in KAC-enriched organoids derived from alveolar type 2 (AT2) cells. Last, lineage-labelling of AT2 cells or KRT8+ cells following carcinogen exposure showed that KACs are possible intermediates in AT2-to-tumour cell transformation. This study provides new insights into epithelial cell states at the root of LUAD development, and such states could harbour potential targets for prevention or intervention.

Deep Learning Glioma Grading with the Tumor Microenvironment Analysis Protocol for Comprehensive Learning, Discovering, and Quantifying Microenvironmental Features

Gliomas are primary brain tumors that arise from neural stem cells, or glial precursors. Diagnosis of glioma is based on histological evaluation of pathological cell features and molecular markers. Gliomas are infiltrated by myeloid cells that accumulate preferentially in malignant tumors, and their abundance inversely correlates with survival, which is of interest for cancer immunotherapies. To avoid time-consuming and laborious manual examination of images, a deep learning approach for automatic multiclass classification of tumor grades was proposed. As an alternative way of investigating characteristics of brain tumor grades, we implemented a protocol for learning, discovering, and quantifying tumor microenvironment elements on our glioma dataset. Using only single-stained biopsies we derived characteristic differentiating tumor microenvironment phenotypic neighborhoods. The study was complicated by the small size of the available human leukocyte antigen stained on glioma tissue microarray dataset - 206 images of 5 classes - as well as imbalanced data distribution. This challenge was addressed by image augmentation for underrepresented classes. In practice, we considered two scenarios, a whole slide supervised learning classification, and an unsupervised cell-to-cell analysis looking for patterns of the microenvironment. In the supervised learning investigation, we evaluated 6 distinct model architectures. Experiments revealed that a DenseNet121 architecture surpasses the baseline's accuracy by a significant margin of 9% for the test set, achieving a score of 69%, increasing accuracy in discerning challenging WHO grade 2 and 3 cases. All experiments have been carried out in a cross-validation manner. The tumor microenvironment analysis suggested an important role for myeloid cells and their accumulation in the context of characterizing glioma grades. Those promising approaches can be used as an additional diagnostic tool to improve assessment during intraoperative examination or subtyping tissues for treatment selection, potentially easing the workflow of pathologists and oncologists.

Computational cell-cell interaction technologies drive mechanistic and biomarker discovery in the tumor microenvironment

Complex networks of cell-cell interactions (CCIs) within the tumor microenvironment (TME) play a crucial role in cancer persistence. These communication axes represent prime targets for therapeutic intervention, but our incomplete understanding of the cellular heterogeneity and interacting partners within the TME remains a stubborn barrier to complete drug responses. This review outlines recent advances in the study of CCIs that leverage single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) technologies that can clarify TME dynamics. We anticipate that these strategies will promote discovery of CCIs critical to the tumor-immune interface and will, by extension, expand the repertoire of druggable tumor biomarkers.

STGIC: A graph and image convolution-based method for spatial transcriptomic clustering

Spatial transcriptomic (ST) clustering employs spatial and transcription information to group spots spatially coherent and transcriptionally similar together into the same spatial domain. Graph convolution network (GCN) and graph attention network (GAT), fed with spatial coordinates derived adjacency and transcription profile derived feature matrix are often used to solve the problem. Our proposed method STGIC (spatial transcriptomic clustering with graph and image convolution) is designed for techniques with regular lattices on chips. It utilizes an adaptive graph convolution (AGC) to get high quality pseudo-labels and then resorts to dilated convolution framework (DCF) for virtual image converted from gene expression information and spatial coordinates of spots. The dilation rates and kernel sizes are set appropriately and updating of weight values in the kernels is made to be subject to the spatial distance from the position of corresponding elements to kernel centers so that feature extraction of each spot is better guided by spatial distance to neighbor spots. Self-supervision realized by Kullback-Leibler (KL) divergence, spatial continuity loss and cross entropy calculated among spots with high confidence pseudo-labels make up the training objective of DCF. STGIC attains state-of-the-art (SOTA) clustering performance on the benchmark dataset of 10x Visium human dorsolateral prefrontal cortex (DLPFC). Besides, it's capable of depicting fine structures of other tissues from other species as well as guiding the identification of marker genes. Also, STGIC is expandable to Stereo-seq data with high spatial resolution.

Reconstructing Spatial Transcriptomics at the Single-cell Resolution with BayesDeep

Spatially resolved transcriptomics (SRT) techniques have revolutionized the characterization of molecular profiles while preserving spatial and morphological context. However, most next-generation sequencing-based SRT techniques are limited to measuring gene expression in a confined array of spots, capturing only a fraction of the spatial domain. Typically, these spots encompass gene expression from a few to hundreds of cells, underscoring a critical need for more detailed, single-cell resolution SRT data to enhance our understanding of biological functions within the tissue context. Addressing this challenge, we introduce BayesDeep, a novel Bayesian hierarchical model that leverages cellular morphological data from histology images, commonly paired with SRT data, to reconstruct SRT data at the single-cell resolution. BayesDeep effectively model count data from SRT studies via a negative binomial regression model. This model incorporates explanatory variables such as cell types and nuclei-shape information for each cell extracted from the paired histology image. A feature selection scheme is integrated to examine the association between the morphological and molecular profiles, thereby improving the model robustness. We applied BayesDeep to two real SRT datasets, successfully demonstrating its capability to reconstruct SRT data at the single-cell resolution. This advancement not only yields new biological insights but also significantly enhances various downstream analyses, such as pseudotime and cell-cell communication.

Novel research and future prospects of artificial intelligence in cancer diagnosis and treatment

Research into the potential benefits of artificial intelligence for comprehending the intricate biology of cancer has grown as a result of the widespread use of deep learning and machine learning in the healthcare sector and the availability of highly specialized cancer datasets. Here, we review new artificial intelligence approaches and how they are being used in oncology. We describe how artificial intelligence might be used in the detection, prognosis, and administration of cancer treatments and introduce the use of the latest large language models such as ChatGPT in oncology clinics. We highlight artificial intelligence applications for omics data types, and we offer perspectives on how the various data types might be combined to create decision-support tools. We also evaluate the present constraints and challenges to applying artificial intelligence in precision oncology. Finally, we discuss how current challenges may be surmounted to make artificial intelligence useful in clinical settings in the future.

Tertiary lymphoid structures and B cells: An intratumoral immunity cycle

The generation of anti-tumor immunity in the draining lymph nodes is known as the cancer immunity cycle. Accumulating evidence supports the occurrence of such a cycle at tumor sites in the context of chronic inflammation. Here, we review the role of tertiary lymphoid structures (TLS) in the generation of T and B cell immunities, focusing on the impact of B cells that undergo full maturation, resulting in the generation of plasma cells (PCs) producing high-affinity IgG and IgA antibodies. In this context, we propose that antibodies binding to tumor cells induce macrophage or natural killer (NK)-cell-dependent apoptosis. Subsequently, released antigen-antibody complexes are internalized and processed by dendritic cells (DCs), amplifying antigen presentation to T cells. Immune complexes may also be fixed by follicular DCs (FDCs) in TLS, thereby increasing memory B cell responses. This amplification loop creates an intra-tumoral immunity cycle, capable of increasing sensitivity of tumors to immunotherapy even in cancers with low mutational burden.

SIMVI reveals intrinsic and spatial-induced states in spatial omics data

Spatial omics analyze gene expression and interaction dynamics in relation to tissue structure and function. However, existing methods cannot model the intrinsic and spatial-induced variation in spatial omics data, thus failing to identify true spatial interaction effects. Here, we present Spatial Interaction Modeling using Variational Inference (SIMVI), an annotation-free framework that disentangles cell intrinsic and spatial-induced latent variables for modeling gene expression in spatial omics data. SIMVI enables novel downstream analyses, such as clustering and differential expression analysis based on disentangled representations, spatial effect (SE) identification, SE interpretation, and transfer learning on new measurements / modalities. We benchmarked SIMVI on both simulated and real datasets and show that SIMVI uniquely generates highly accurate SE inferences in synthetic datasets and unveils intrinsic variation in complex real datasets. We applied SIMVI to spatial omics data from diverse platforms and tissues (MERFISH human cortex / mouse liver, Slide-seqv2 mouse hippocampus, Spatial-ATAC-RNA-seq) and revealed various region-specific and cell-type-specific spatial interactions. In addition, our experiments on MERFISH human cortex and spatial-ATAC-RNA-seq showcased SIMVI's power in identifying SEs for new samples / modalities. Finally, we applied SIMVI on a newly collected CosMx melanoma dataset. Using SIMVI, we identified immune cells associated with spatial-dependent interactions and revealed the underlying spatial variations associated with patient outcomes.

Leveraging spatial transcriptomics data to recover cell locations in single-cell RNA-seq with CeLEry

Single-cell RNA sequencing (scRNA-seq) has revolutionized our understanding of cellular heterogeneity in health and disease. However, the lack of physical relationships among dissociated cells has limited its applications. To address this issue, we present CeLEry (Cell Location recovEry), a supervised deep learning algorithm that leverages gene expression and spatial location relationships learned from spatial transcriptomics to recover the spatial origins of cells in scRNA-seq. CeLEry has an optional data augmentation procedure via a variational autoencoder, which improves the method's robustness and allows it to overcome noise in scRNA-seq data. We show that CeLEry can infer the spatial origins of cells in scRNA-seq at multiple levels, including 2D location and spatial domain of a cell, while also providing uncertainty estimates for the recovered locations. Our comprehensive benchmarking evaluations on multiple datasets generated from brain and cancer tissues using Visium, MERSCOPE, MERFISH, and Xenium demonstrate that CeLEry can reliably recover the spatial location information for cells using scRNA-seq data.

Pan-cancer T cell atlas links a cellular stress response state to immunotherapy resistance

Tumor-infiltrating T cells offer a promising avenue for cancer treatment, yet their states remain to be fully characterized. Here we present a single-cell atlas of T cells from 308,048 transcriptomes across 16 cancer types, uncovering previously undescribed T cell states and heterogeneous subpopulations of follicular helper, regulatory and proliferative T cells. We identified a unique stress response state, TSTR, characterized by heat shock gene expression. TSTR cells are detectable in situ in the tumor microenvironment across various cancer types, mostly within lymphocyte aggregates or potential tertiary lymphoid structures in tumor beds or surrounding tumor edges. T cell states/compositions correlated with genomic, pathological and clinical features in 375 patients from 23 cohorts, including 171 patients who received immune checkpoint blockade therapy. We also found significantly upregulated heat shock gene expression in intratumoral CD4/CD8+ cells following immune checkpoint blockade treatment, particularly in nonresponsive tumors, suggesting a potential role of TSTR cells in immunotherapy resistance. Our well-annotated T cell reference maps, web portal and automatic alignment/annotation tool could provide valuable resources for T cell therapy optimization and biomarker discovery.