Nondestructive, multiplex three-dimensional mapping of immune infiltrates in core needle biopsy

Increased Multiplexity in Optical Tissue Clearing-Based Three-Dimensional Immunofluorescence Microscopy of the Tumor Microenvironment by Light-Emitting Diode Photobleaching

Optical tissue clearing and three-dimensional (3D) immunofluorescence (IF) microscopy is transforming imaging of the complex tumor microenvironment (TME). However, current 3D IF microscopy has restricted multiplexity; only 3 or 4 cellular and noncellular TME components can be localized in cleared tumor tissue. Here we report a light-emitting diode (LED) photobleaching method and its application for 3D multiplexed optical mapping of the TME. We built a high-power LED light irradiation device and temperature-controlled chamber for completely bleaching fluorescent signals throughout optically cleared tumor tissues without compromise of tissue and protein antigen integrity. With newly developed tissue mounting and selected region-tracking methods, we established a cyclic workflow involving IF staining, tissue clearing, 3D confocal microscopy, and LED photobleaching. By registering microscope channel images generated through 3 work cycles, we produced 8-plex image data from individual 400 μm-thick tumor macrosections that visualize various vascular, immune, and cancer cells in the same TME at tissue-wide and cellular levels in 3D. Our method was also validated for quantitative 3D spatial analysis of cellular remodeling in the TME after immunotherapy. These results demonstrate that our LED photobleaching system and its workflow offer a novel approach to increase the multiplexing power of 3D IF microscopy for studying tumor heterogeneity and response to therapy.

Correlative multiscale 3D imaging of mouse primary and metastatic tumors by sequential light sheet and confocal fluorescence microscopy

Three-dimensional (3D) optical microscopy, combined with advanced tissue clearing, permits in situ interrogation of the tumor microenvironment (TME) in large volumetric tumors for preclinical cancer research. Light sheet (also known as ultramicroscopy) and confocal fluorescence microscopy are often used to achieve macroscopic and microscopic 3D images of optically cleared tumor tissues, respectively. Although each technique offers distinct fields of view (FOVs) and spatial resolution, the combination of these two optical microscopy techniques to obtain correlative multiscale 3D images from the same tumor tissues has not yet been explored. To establish correlative multiscale 3D optical microscopy, we developed a method for optically marking defined regions of interest (ROIs) within a cleared mouse tumor by employing a UV light-activated visible dye and Z-axis position-selective UV irradiation in a light sheet microscope system. By integrating this method with subsequent tissue processing, including physical ROI marking, reversal of tissue clearing, tissue macrosectioning, and multiplex immunofluorescence, we established a workflow that enables the tracking and 3D imaging of ROIs within tumor tissues through sequential light sheet and confocal fluorescence microscopy. This approach allowed for quantitative 3D spatial analysis of the immune response in the TME of a mouse mammary tumor following cancer immunotherapy at multiple spatial scales. The workflow also facilitated the direct localization of a metastatic lesion within a whole mouse brain. These results demonstrate that our ROI tracking method and its associated workflow offer a novel approach for correlative multiscale 3D optical microscopy, with the potential to provide new insights into tumor heterogeneity, metastasis, and response to therapy at various spatial levels.

Direct three-dimensional segmentation of prostate glands with nnU-Net

Significance

In recent years, we and others have developed non-destructive methods to obtain three-dimensional (3D) pathology datasets of clinical biopsies and surgical specimens. For prostate cancer risk stratification (prognostication), standard-of-care Gleason grading is based on examining the morphology of prostate glands in thin 2D sections. This motivates us to perform 3D segmentation of prostate glands in our 3D pathology datasets for the purposes of computational analysis of 3D glandular features that could offer improved prognostic performance.

Aim

To facilitate prostate cancer risk assessment, we developed a computationally efficient and accurate deep learning model for 3D gland segmentation based on open-top light-sheet microscopy datasets of human prostate biopsies stained with a fluorescent analog of hematoxylin and eosin (H&E).

Approach

For 3D gland segmentation based on our H&E-analog 3D pathology datasets, we previously developed a hybrid deep learning and computer vision-based pipeline, called image translation-assisted segmentation in 3D (ITAS3D), which required a complex two-stage procedure and tedious manual optimization of parameters. To simplify this procedure, we use the 3D gland-segmentation masks previously generated by ITAS3D as training datasets for a direct end-to-end deep learning-based segmentation model, nnU-Net. The inputs to this model are 3D pathology datasets of prostate biopsies rapidly stained with an inexpensive fluorescent analog of H&E and the outputs are 3D semantic segmentation masks of the gland epithelium, gland lumen, and surrounding stromal compartments within the tissue.

Results

nnU-Net demonstrates remarkable accuracy in 3D gland segmentations even with limited training data. Moreover, compared with the previous ITAS3D pipeline, nnU-Net operation is simpler and faster, and it can maintain good accuracy even with lower-resolution inputs.

Conclusions

Our trained DL-based 3D segmentation model will facilitate future studies to demonstrate the value of computational 3D pathology for guiding critical treatment decisions for patients with prostate cancer.

Increased multiplexity in optical tissue clearing-based 3D immunofluorescence microscopy of the tumor microenvironment by LED photobleaching

Optical tissue clearing and three-dimensional (3D) immunofluorescence (IF) microscopy have been transforming imaging of the complex tumor microenvironment (TME). However, current 3D IF microscopy has restricted multiplexity; only three or four cellular and non-cellular TME components can be localized in a cleared tumor tissue. Here we report a LED photobleaching method and its application for 3D multiplexed optical mapping of the TME. We built a high-power LED light irradiation device and temperature-controlled chamber for completely bleaching fluorescent signals throughout optically cleared tumor tissues without compromise of tissue and protein antigen integrity. With newly developed tissue mounting and selected region-tracking methods, we established a cyclic workflow involving IF staining, tissue clearing, 3D confocal microscopy, and LED photobleaching. By registering microscope channel images generated through three work cycles, we produced 8-plex image data from individual 400 μm-thick tumor macrosections that visualize various vascular, immune, and cancer cells in the same TME at tissue-wide and cellular levels in 3D. Our method was also validated for quantitative 3D spatial analysis of cellular remodeling in the TME after immunotherapy. These results demonstrate that our LED photobleaching system and its workflow offer a novel approach to increase the multiplexing power of 3D IF microscopy for studying tumor heterogeneity and response to therapy.

Enhancing personalized immune checkpoint therapy by immune archetyping and pharmacological targeting

Immune checkpoint inhibitors (ICIs) are an expanding class of immunotherapeutic agents with the potential to cure cancer. Despite the outstanding clinical response in patient subsets, most individuals become refractory or develop resistance. Patient stratification and personalized immunotherapies are limited by the absence of predictive response markers. Recent findings show that dominant patterns of immune cell composition, T-cell status and heterogeneity, and spatiotemporal distribution of immune cells within the tumor microenvironment (TME) are becoming essential determinants of prognosis and therapeutic response. In this context, ICIs also function as investigational tools and proof of concept, allowing the validation of the identified mechanisms. After reviewing the current state of ICIs, this article will explore new comprehensive predictive markers for ICIs based on recent discoveries. We will discuss the recent establishment of a classification of TMEs into immune archetypes as a tool for personalized immune profiling, allowing patient stratification before ICI treatment. We will discuss the developing comprehension of T-cell diversity and its role in shaping the immune profile of patients. We describe the potential of strategies that score the mutual spatiotemporal modulation between T-cells and other cellular components of the TME. Additionally, we will provide an overview of a range of synthetic and naturally occurring or derived small molecules. We will compare compounds that were recently identified by in silico prediction to wet lab-validated drug candidates with the potential to function as ICIs and/or modulators of the cellular components of the TME.

Panoramic Tissue Examination That Integrates 3-Dimensional Pathology Imaging and Gene Mutation: Potential Utility in Non-Small Cell Lung Cancer

Novel therapeutics have significantly improved the survival and quality of life of patients with malignancies in this century. Versatile precision diagnostic data were used to formulate personalized therapeutic strategies for patients. However, the cost of extensive information depends on the consumption of the specimen, raising the challenges of effective specimen utilization, particularly in small biopsies. In this study, we proposed a tissue-processing cascaded protocol that obtains 3-dimensional (3D) protein expression spatial distribution and mutation analysis from an identical specimen. In order to reuse the thick section tissue evaluated after the 3D pathology technique, we designed a novel high-flatness agarose-embedded method that could improve tissue utilization rate by 1.52 fold, whereas it reduced the tissue-processing time by 80% compared with the traditional paraffin-embedding method. In animal studies, we demonstrated that the protocol would not affect the results of DNA mutation analysis. Furthermore, we explored the utility of this approach in non-small cell lung cancer because it is a compelling application for this innovation. We used 35 cases including 7 cases of biopsy specimens of non-small cell lung cancer to simulate future clinical application. The cascaded protocol consumed 150-μm thickness of formalin-fixed, paraffin-embedded specimens, providing 3D histologic and immunohistochemical information approximately 38 times that of the current paraffin-embedding protocol, and 3 rounds of DNA mutation analysis, offering both essential guidance for routine diagnostic evaluation and advanced information for precision medicine. Our designed integrated workflow provides an alternative way for pathological examination and paves the way for multidimensional tumor tissue assessment.

Multiresolution 3D Optical Mapping of Immune Cell Infiltrates in Mouse Asthmatic Lung

Asthma is a chronic inflammatory airway disease driven by various infiltrating immune cell types into the lung. Optical microscopy has been used to study immune infiltrates in asthmatic lungs. Confocal laser scanning microscopy (CLSM) identifies the phenotypes and locations of individual immune cells in lung tissue sections by employing high-magnification objectives and multiplex immunofluorescence staining. In contrast, light-sheet fluorescence microscopy (LSFM) can visualize the macroscopic and mesoscopic architecture of whole-mount lung tissues in three dimensions (3D) by adopting an optical tissue-clearing method. Despite each microscopy method producing image data with unique resolution from a tissue sample, CLSM and LSFM have not been applied together because of different tissue-preparation procedures. Here, we introduce a new approach combining LSFM and CLSM into a sequential imaging pipeline. We built a new optical tissue clearing workflow in which the immersion clearing agent can be switched from an organic solvent to an aqueous sugar solution for sequential 3D LSFM and CLSM of mouse lungs. This sequential combination microscopy offered quantitative 3D spatial analyses of the distribution of immune infiltrates in the same mouse asthmatic lung tissue at the organ, tissue, and cell levels. These results show that our method facilitates multiresolution 3D fluorescence microscopy as a new imaging approach providing comprehensive spatial information for a better understanding of inflammatory lung diseases.

Deep-learning based breast cancer detection for cross-staining histopathology images

Hematoxylin and eosin (H&E) staining is the gold standard for tissue characterization in routine pathological diagnoses. However, these visible light dyes do not exclusively label the nuclei and cytoplasm, making clear-cut segmentation of staining signals challenging. Currently, fluorescent staining technology is much more common in clinical research for analyzing tissue morphology and protein distribution owing to its advantages of channel independence, multiplex labeling, and the possibility of enabling 3D tissue labeling. Although both H&E and fluorescent dyes can stain the nucleus and cytoplasm for representative tissue morphology, color variation between these two staining technologies makes cross-analysis difficult, especially with computer-assisted artificial intelligence (AI) algorithms. In this study, we applied color normalization and nucleus extraction methods to overcome the variation between staining technologies. We also developed an available workflow for using an H&E-stained segmentation AI model in the analysis of fluorescent nucleic acid staining images in breast cancer tumor recognition, resulting in 89.6% and 80.5% accuracy in recognizing specific tumor features in H&E- and fluorescent-stained pathological images, respectively. The results show that the cross-staining inference maintained the same precision level as the proposed workflow, providing an opportunity for an expansion of the application of current pathology AI models.

3D imaging for driving cancer discovery

Our understanding of the cellular composition and architecture of cancer has primarily advanced using 2D models and thin slice samples. This has granted spatial information on fundamental cancer biology and treatment response. However, tissues contain a variety of interconnected cells with different functional states and shapes, and this complex organization is impossible to capture in a single plane. Furthermore, tumours have been shown to be highly heterogenous, requiring large-scale spatial analysis to reliably profile their cellular and structural composition. Volumetric imaging permits the visualization of intact biological samples, thereby revealing the spatio-phenotypic and dynamic traits of cancer. This review focuses on new insights into cancer biology uniquely brought to light by 3D imaging and concomitant progress in cancer modelling and quantitative analysis. 3D imaging has the potential to generate broad knowledge advance from major mechanisms of tumour progression to new strategies for cancer treatment and patient diagnosis. We discuss the expected future contributions of the newest imaging trends towards these goals and the challenges faced for reaching their full application in cancer research.

Prostate Cancer Risk Stratification via Nondestructive 3D Pathology with Deep Learning-Assisted Gland Analysis

Prostate cancer treatment planning is largely dependent upon examination of core-needle biopsies. The microscopic architecture of the prostate glands forms the basis for prognostic grading by pathologists. Interpretation of these convoluted three-dimensional (3D) glandular structures via visual inspection of a limited number of two-dimensional (2D) histology sections is often unreliable, which contributes to the under- and overtreatment of patients. To improve risk assessment and treatment decisions, we have developed a workflow for nondestructive 3D pathology and computational analysis of whole prostate biopsies labeled with a rapid and inexpensive fluorescent analogue of standard hematoxylin and eosin (H&E) staining. This analysis is based on interpretable glandular features and is facilitated by the development of image translation-assisted segmentation in 3D (ITAS3D). ITAS3D is a generalizable deep learning-based strategy that enables tissue microstructures to be volumetrically segmented in an annotation-free and objective (biomarker-based) manner without requiring immunolabeling. As a preliminary demonstration of the translational value of a computational 3D versus a computational 2D pathology approach, we imaged 300 ex vivo biopsies extracted from 50 archived radical prostatectomy specimens, of which, 118 biopsies contained cancer. The 3D glandular features in cancer biopsies were superior to corresponding 2D features for risk stratification of patients with low- to intermediate-risk prostate cancer based on their clinical biochemical recurrence outcomes. The results of this study support the use of computational 3D pathology for guiding the clinical management of prostate cancer. SIGNIFICANCE: An end-to-end pipeline for deep learning-assisted computational 3D histology analysis of whole prostate biopsies shows that nondestructive 3D pathology has the potential to enable superior prognostic stratification of patients with prostate cancer.

Large-scale, three-dimensional tissue cytometry of the human kidney: a complete and accessible pipeline

The advent of personalized medicine has driven the development of novel approaches for obtaining detailed cellular and molecular information from clinical tissue samples. Tissue cytometry is a promising new technique that can be used to enumerate and characterize each cell in a tissue and, unlike flow cytometry and other single-cell techniques, does so in the context of the intact tissue, preserving spatial information that is frequently crucial to understanding a cell's physiology, function, and behavior. However, the wide-scale adoption of tissue cytometry as a research tool has been limited by the fact that published examples utilize specialized techniques that are beyond the capabilities of most laboratories. Here we describe a complete and accessible pipeline, including methods of sample preparation, microscopy, image analysis, and data analysis for large-scale three-dimensional tissue cytometry of human kidney tissues. In this workflow, multiphoton microscopy of unlabeled tissue is first conducted to collect autofluorescence and second-harmonic images. The tissue is then labeled with eight fluorescent probes, and imaged using spectral confocal microscopy. The raw 16-channel images are spectrally deconvolved into 8-channel images, and analyzed using the Volumetric Tissue Exploration and Analysis (VTEA) software developed by our group. We applied this workflow to analyze millimeter-scale tissue samples obtained from human nephrectomies and from renal biopsies from individuals diagnosed with diabetic nephropathy, generating a quantitative census of tens of thousands of cells in each. Such analyses can provide useful insights that can be linked to the biology or pathology of kidney disease. The approach utilizes common laboratory techniques, is compatible with most commercially-available confocal microscope systems and all image and data analysis is conducted using the VTEA image analysis software, which is available as a plug-in for ImageJ.

Harnessing non-destructive 3D pathology

High-throughput methods for slide-free three-dimensional (3D) pathological analyses of whole biopsies and surgical specimens offer the promise of modernizing traditional histology workflows and delivering improvements in diagnostic performance. Advanced optical methods now enable the interrogation of orders of magnitude more tissue than previously possible, where volumetric imaging allows for enhanced quantitative analyses of cell distributions and tissue structures that are prognostic and predictive. Non-destructive imaging processes can simplify laboratory workflows, potentially reducing costs, and can ensure that samples are available for subsequent molecular assays. However, the large size of the feature-rich datasets that they generate poses challenges for data management and computer-aided analysis. In this Perspective, we provide an overview of the imaging technologies that enable 3D pathology, and the computational tools-machine learning, in particular-for image processing and interpretation. We also discuss the integration of various other diagnostic modalities with 3D pathology, along with the challenges and opportunities for clinical adoption and regulatory approval.

Combined transmission, dark field and fluorescence microscopy for intact, 3D tissue analysis of biopsies

SIGNIFICANCE

Currently, tissue biopsies are sectioned into 3- to 5-μm-thick slices that are used for conventional pathology analysis. Previous work by confocal microscopy and light-sheet microscopy has shown that analyzing biopsies intact in three-dimensions (3D) is possible and may lead to a better understanding of cancer growth patterns. Although accurate, these methods require fluorescent staining of the tissue, in addition to tissue clearing. If the 3D biopsy analysis could be done sufficiently swiftly, this approach may be used for on-site assessment of the adequacy of a biopsy taken.

AIM

We aim to show that, by transmission microscopy of optically cleared tissue punches, the tissue architecture can be determined without the need for fluorescent staining.

APPROACH

Transmission microscopy is used by combining bright field microscopy with dark field and epifluorescent microscopy to compare samples that have also been analyzed by fluorescent confocal microscopy.

RESULTS

With increasing distance to the focal plane, the higher-frequency part of the spatial frequency spectrum of transmitted light is attenuated increasingly. This property is exploited for tissue segmentation, detecting whether tissue is present at a certain position in the focal plane image. Using this approach, we show that a 3D rendering of the internal cavity or tubules structure of punch biopsies, which are up to 1-mm thick, can be acquired in ≈1 min scan time per imaging modality. The images of the overall tissue architecture that are obtained are similar to those from the confocal microscopy benchmark, without requiring fluorescent staining.

CONCLUSIONS

Images of the overall tissue architecture can be obtained from transmission microcopy; they are similar to those from the confocal microscopy benchmark without requiring fluorescent staining. Tissue clearing is still needed. The total scan time of the present method is significantly shorter at a fraction of the device costs.

Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

BACKGROUND

Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of the unique colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel.

METHODS

Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and ensemble methods that employ both ColorAE and U-Net, collectively referred to as (3) ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor).

RESULTS

We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect 6 different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net into ensemble methods outperform using either ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also present a use case, wherein we apply the ColorAE:U-Net ensemble method across 3 mIHC WSIs and use the predictions to quantify all stained cell populations and perform nearest neighbor spatial analysis. Thus, we provide proof of concept that these methods can be employed to quantitatively describe the spatial distribution immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.