Dual-mode emission and transmission microscopy for virtual histochemistry using hematoxylin- and eosin-stained tissue sections

Current Landscape of Advanced Imaging Tools for Pathology Diagnostics

Histopathology relies on century-old workflows of formalin fixation, paraffin embedding, sectioning, and staining tissue specimens on glass slides. Despite being robust, this conventional process is slow, labor-intensive, and limited to providing two-dimensional views. Emerging technologies promise to enhance and accelerate histopathology. Slide-free microscopy allows rapid imaging of fresh, unsectioned specimens, overcoming slide preparation delays. Methods such as fluorescence confocal microscopy, multiphoton microscopy, along with more recent innovations including microscopy with UV surface excitation and fluorescence-imitating brightfield imaging can generate images resembling conventional histology directly from the surface of tissue specimens. Slide-free microscopy enable applications such as rapid intraoperative margin assessment and, with appropriate technology, three-dimensional histopathology. Multiomics profiling techniques, including imaging mass spectrometry and Raman spectroscopy, provide highly multiplexed molecular maps of tissues, although clinical translation remains challenging. Artificial intelligence is aiding the adoption of new imaging modalities via virtual staining, which converts methods such as slide-free microscopy into synthetic brightfield-like or even molecularly informed images. Although not yet commonplace, these emerging technologies collectively demonstrate the potential to modernize histopathology. Artificial intelligence-assisted workflows will ease the transition to new imaging modalities. With further validation, these advances may transform the century-old conventional histopathology pipeline to better serve 21st-century medicine. This review provides an overview of these enabling technology platforms and discusses their potential impact.

Highly sensitive and specific resonance Rayleigh scattering detection of esophageal cancer cells via dual-aptamer target binding strategy

The modification of EGFR aptamer (Apt 1) and HER2 aptamer (Apt 2) with gold nanoparticles (AuNPs) is reported to obtain probe I (Apt 1-AuNPs) and probe II (Apt 2-AuNPs). Taking Eca109, KYSE510, and KYSE150 cells as models, the sandwich scattering system of probe I-cell-probe II was formed by the recognition of tumor markers by the aptamer modified probe, and the resonance Rayleigh scattering (RRS) spectra were investigated. The results showed that the scattering system can be used to quantitatively detect the Eca109 cell lines in the range 5.0×10 to 5.0×10⁵ cells·mL^-1 with a detection limit of 15 cells· mL^-1.The system can also detect the KYSE510 cell lines in a linear range of 5.0×10 to 5.0×10⁵ cells·mL^-1 with a detection limit of 18 cells·mL^-1 and the KYSE150 cell lines in a linear range of 3.0×10 to 5.0×10⁵ cells·mL^-1 with a detection limit of 12 cells·mL^-1. To demonstrate the potential application of the RRS method for real sample analysis, cells were spiked into blank serum samples at concentrations from 1.0×10² to 1.0×10⁵ cells·mL^-1. The recovery was between 97.0% and 102.3%, and the RSD was between 1.1% and 4.9%, confirming the feasibility of the proposed method for ESCC cell determination.

Mapping variation of extracellular matrix in human keloid scar by label-free multiphoton imaging and machine learning

Significance

Rapid diagnosis and analysis of human keloid scar tissues in an automated manner are essential for understanding pathogenesis and formulating treatment solutions.

Aim

Our aim is to resolve the features of the extracellular matrix in human keloid scar tissues automatically for accurate diagnosis with the aid of machine learning.

Approach

Multiphoton microscopy was utilized to acquire images of collagen and elastin fibers. Morphological features, histogram, and gray-level co-occurrence matrix-based texture features were obtained to produce a total of 28 features. The minimum redundancy maximum relevancy feature selection approach was implemented to rank these features and establish feature subsets, each of which was employed to build a machine learning model through the tree-based pipeline optimization tool (TPOT).

Results

The feature importance ranking was obtained, and 28 feature subsets were acquired by incremental feature selection. The subset with the top 23 features was identified as the most accurate. Then stochastic gradient descent classifier optimized by the TPOT was generated with an accuracy of 96.15% in classifying normal, scar, and adjacent tissues. The area under curve of the classification results (scar versus normal and adjacent, normal versus scar and adjacent, and adjacent versus normal and scar) was 1.0, 1.0, and 0.99, respectively.

Conclusions

The proposed approach has great potential for future dermatological clinical diagnosis and analysis and holds promise for the development of computer-aided systems to assist dermatologists in diagnosis and treatment.

Improving quantification of renal fibrosis using Deep-DUET

Accurate quantification of renal fibrosis has profound importance in the assessment of chronic kidney disease (CKD). Visual analysis of a biopsy stained with trichrome under the microscope by a pathologist is the gold standard for evaluation of fibrosis. Trichrome helps to highlight collagen and ultimately interstitial fibrosis. However, trichrome stains are not always reproducible, can underestimate collagen content and are not sensitive to subtle fibrotic patterns. Using the Dual-mode emission and transmission (DUET) microscopy approach, it is possible to capture both brightfield and fluorescence images from the same area of a tissue stained with hematoxylin and eosin (H&E) enabling reproducible extraction of collagen with high sensitivity and specificity. Manual extraction of spectrally overlapping collagen signals from tubular epithelial cells and red blood cells is still an intensive task. We employed a UNet++ architecture for pixel-level segmentation and quantification of collagen using 760 whole slide image (WSI) patches from six cases of varying stages of fibrosis. Our trained model (Deep-DUET) used the supervised extracted collagen mask as ground truth and was able to predict the extent of collagen signal with a MSE of 0.05 in a holdout testing set while achieving an average AUC of 0.94 for predicting regions of collagen deposits. Expanding this work to the level of the WSI can greatly improve the ability of pathologists and machine learning (ML) tools to quantify the extent of renal fibrosis reproducibly and reliably.

Differential diagnosis of thyroid nodule capsules using random forest guided selection of image features

Microscopic evaluation of tissue sections stained with hematoxylin and eosin is the current gold standard for diagnosing thyroid pathology. Digital pathology is gaining momentum providing the pathologist with additional cues to traditional routes when placing a diagnosis, therefore it is extremely important to develop new image analysis methods that can extract image features with diagnostic potential. In this work, we use histogram and texture analysis to extract features from microscopic images acquired on thin thyroid nodule capsules sections and demonstrate how they enable the differential diagnosis of thyroid nodules. Targeted thyroid nodules are benign (i.e., follicular adenoma) and malignant (i.e., papillary thyroid carcinoma and its sub-type arising within a follicular adenoma). Our results show that the considered image features can enable the quantitative characterization of the collagen capsule surrounding thyroid nodules and provide an accurate classification of the latter's type using random forest.

Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging

SIGNIFICANCE

Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space.

AIM

WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics.

APPROACH

High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection.

RESULTS

Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results.

CONCLUSIONS

The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.

Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging

SIGNIFICANCE

Automatic, fast, and accurate identification of cancer on histologic slides has many applications in oncologic pathology.

AIM

The purpose of this study is to investigate hyperspectral imaging (HSI) for automatic detection of head and neck cancer nuclei in histologic slides, as well as cancer region identification based on nuclei detection.

APPROACH

A customized hyperspectral microscopic imaging system was developed and used to scan histologic slides from 20 patients with squamous cell carcinoma (SCC). Hyperspectral images and red, green, and blue (RGB) images of the histologic slides with the same field of view were obtained and registered. A principal component analysis-based nuclei segmentation method was developed to extract nuclei patches from the hyperspectral images and the coregistered RGB images. Spectra-based support vector machine and patch-based convolutional neural networks (CNNs) were implemented for nuclei classification. The CNNs were trained with RGB patches (RGB-CNN) and hyperspectral patches (HSI-CNN) of the segmented nuclei and the utility of the extra spectral information provided by HSI was evaluated. Furthermore, cancer region identification was implemented by image-wise classification based on the percentage of cancerous nuclei detected in each image.

RESULTS

RGB-CNN, which mainly used the spatial information of nuclei, resulted in a 0.81 validation accuracy and 0.74 testing accuracy. HSI-CNN, which utilized the spatial and spectral features of the nuclei, showed significant improvement in classification performance and achieved 0.89 validation accuracy as well as 0.82 testing accuracy. Furthermore, the image-wise cancer region identification based on nuclei detection could generally improve the cancer detection rate.

CONCLUSIONS

We demonstrated that the morphological and spectral information contribute to SCC nuclei differentiation and that the spectral information within hyperspectral images could improve classification performance.

Automated Structural Analysis and Quantitative Characterization of Scar Tissue Using Machine Learning

An analysis of scar tissue is necessary to understand the pathological tissue conditions during or after the wound healing process. Hematoxylin and eosin (HE) staining has conventionally been applied to understand the morphology of scar tissue. However, the scar lesions cannot be analyzed from a whole slide image. The current study aimed to develop a method for the rapid and automatic characterization of scar lesions in HE-stained scar tissues using a supervised and unsupervised learning algorithm. The supervised learning used a Mask region-based convolutional neural network (RCNN) to train a pattern from a data representation using MMDetection tools. The K-means algorithm characterized the HE-stained tissue and extracted the main features, such as the collagen density and directional variance of the collagen. The Mask RCNN model effectively predicted scar images using various backbone networks (e.g., ResNet50, ResNet101, ResNeSt50, and ResNeSt101) with high accuracy. The K-means clustering method successfully characterized the HE-stained tissue by separating the main features in terms of the collagen fiber and dermal mature components, namely, the glands, hair follicles, and nuclei. A quantitative analysis of the scar tissue in terms of the collagen density and directional variance of the collagen confirmed 50% differences between the normal and scar tissues. The proposed methods were utilized to characterize the pathological features of scar tissue for an objective histological analysis. The trained model is time-efficient when used for detection in place of a manual analysis. Machine learning-assisted analysis is expected to aid in understanding scar conditions, and to help establish an optimal treatment plan.

Real-time polarization microscopy of fibrillar collagen in histopathology

Over the past two decades, fibrillar collagen reorganization parameters such as the amount of collagen deposition, fiber angle and alignment have been widely explored in numerous studies. These parameters are now widely accepted as stromal biomarkers and linked to disease progression and survival time in several cancer types. Despite all these advances, there has not been a significant effort to make it possible for clinicians to explore these biomarkers without adding steps to the clinical workflow or by requiring high-cost imaging systems. In this paper, we evaluate previously described polychromatic polarization microscope (PPM) to visualize collagen fibers with an optically generated color representation of fiber orientation and alignment when inspecting the sample by a regular microscope with minor modifications. This system does not require stained slides, but is compatible with histological stains such as H&E. Consequently, it can be easily accommodated as part of regular pathology review of tissue slides, while providing clinically useful insight into stromal composition.

Tumor collagen framework from bright-field histology images predicts overall survival of breast carcinoma patients

Within the tumor microenvironment, specifically aligned collagen has been shown to stimulate tumor progression by directing the migration of metastatic cells along its structural framework. Tumor-associated collagen signatures (TACS) have been linked to breast cancer patient outcome. Robust and affordable methods for assessing biological information contained in collagen architecture need to be developed. We have developed a novel artificial neural network (ANN) based approach for tumor collagen segmentation from bright-field histology images and have tested it on a set of tissue microarray sections from early hormone receptor-positive invasive ductal breast carcinoma stained with Sirius Red (1 core per patient, n = 92). We designed and trained ANNs on sets of differently annotated image patches to segment collagen fibers and extracted 37 features of collagen fiber morphometry, density, orientation, texture, and fractal characteristics in the entire cohort. Independent instances of ANN models trained on highly differing annotations produced reasonably concordant collagen segmentation masks and allowed reliable prognostic Cox regression models (with likelihood ratios 14.11-22.99, at p-value < 0.05) superior to conventional clinical parameters (size of the primary tumor (T), regional lymph node status (N), histological grade (G), and patient age). Additionally, we noted statistically significant differences of collagen features between tumor grade groups, and the factor analysis revealed features resembling the TACS concept. Our proposed method offers collagen framework segmentation from bright-field histology images and provides novel image-based features for better breast cancer patient prognostication.

Machine-Learning-Based Evaluation of Intratumoral Heterogeneity and Tumor-Stroma Interface for Clinical Guidance

Assessment of intratumoral heterogeneity and tumor-host interaction within the tumor microenvironment is becoming increasingly important for innovative cancer therapy decisions because of the unique information it can generate about the state of the disease. However, its assessment and quantification are limited by ambiguous definitions of the tumor-host interface and by human cognitive capacity in current pathology practice. Advances in machine learning and artificial intelligence have opened the field of digital pathology to novel tissue image analytics and feature extraction for generation of high-capacity computational disease management models. A particular benefit is expected from machine-learning applications that can perform extraction and quantification of subvisual features of both intratumoral heterogeneity and tumor microenvironment aspects. These methods generate information about cancer cell subpopulation heterogeneity, potential tumor-host interactions, and tissue microarchitecture, derived from morphologically resolved content using both explicit and implicit features. Several studies have achieved promising diagnostic, prognostic, and predictive artificial intelligence models that often outperform current clinical and pathology criteria. However, further effort is needed for clinical adoption of such methods through development of standardizable high-capacity workflows and proper validation studies.

The Effects of Stiffness, Fluid Viscosity, and Geometry of Microenvironment in Homeostasis, Aging, and Diseases: A Brief Review

Cells sense biophysical cues in the micro-environment and respond to the cues biochemically and biophysically. Proper responses from cells are critical to maintain the homeostasis in the body. Abnormal biophysical cues will cause pathological development in the cells; pathological or aging cells, on the other hand, can alter their micro-environment to become abnormal. In this minireview, we discuss four important biophysical cues of the micro-environment-stiffness, curvature, extracellular matrix (ECM) architecture and viscosity-in terms of their roles in health, aging, and diseases.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.