Multispectral analysis tools can increase utility of RGB color images in histology

Colorectal cancer histopathology image analysis: A comparative study of prognostic values of automatically extracted morphometric nuclear features in multispectral and red-blue-green imagery

OBJECTIVES

Multispectral imaging (MSI) has been utilized to predict the prognosis of colorectal cancer (CRC) patients, however, our understanding of the prognostic value of nuclear morphological parameters of bright-field MSI in CRC is still limited. This study was designed to compare the efficiency of MSI and standard red-green-blue (RGB) images in predicting the prognosis of CRC.

METHODS

We compared the efficiency of MS and conventional RGB images on the quantitative assessment of hematoxylin-eosin (HE) stained histopathology images. A pipeline was developed using a pixel-wise support vector machine (SVM) classifier for gland-stroma segmentation, and a marker-controlled watershed algorithm was used for nuclei segmentation. The correlation between extracted morphological parameters and the five-year disease-free survival (5-DFS) was analyzed.

RESULTS

Forty-seven nuclear morphological parameters were extracted in total. Based on Kaplan-Meier analysis, eight features derived from MS images and seven featured derived from RGB images were significantly associated with 5-DFS, respectively. Compared with RGB images, MSI showed higher accuracy, precision, and Dice index in nuclei segmentation. Multivariate analysis indicated that both integrated parameters 1 (factors negatively correlated with CRC prognosis including nuclear number, circularity, eccentricity, major axis length) and 2 (factors positively correlated with CRC prognosis including nuclear average area, area perimeter, total area/total perimeter ratio, average area/perimeter ratio) in MS images were independent prognostic factors of 5-DFS, in contrast with only integrated parameter 1 (P<0.001) in RGB images. More importantly, the quantification of HE-stained MS images displayed higher accuracy in predicting 5-DFS compared with RGB images (76.9% vs 70.9%).

CONCLUSIONS

Quantitative evaluation of HE-stained MS images could yield more information and better predictive performance for CRC prognosis than conventional RGB images, thereby contributing to precision oncology.

Forage Height and Above-Ground Biomass Estimation by Comparing UAV-Based Multispectral and RGB Imagery

Crop height and biomass are the two important phenotyping traits to screen forage population types at local and regional scales. This study aims to compare the performances of multispectral and RGB sensors onboard drones for quantitative retrievals of forage crop height and biomass at very high resolution. We acquired the unmanned aerial vehicle (UAV) multispectral images (MSIs) at 1.67 cm spatial resolution and visible data (RGB) at 0.31 cm resolution and measured the forage height and above-ground biomass over the alfalfa (Medicago sativa L.) breeding trials in the Canadian Prairies. (1) For height estimation, the digital surface model (DSM) and digital terrain model (DTM) were extracted from MSI and RGB data, respectively. As the resolution of the DTM is five times less than that of the DSM, we applied an aggregation algorithm to the DSM to constrain the same spatial resolution between DSM and DTM. The difference between DSM and DTM was computed as the canopy height model (CHM), which was at 8.35 cm and 1.55 cm for MSI and RGB data, respectively. (2) For biomass estimation, the normalized difference vegetation index (NDVI) from MSI data and excess green (ExG) index from RGB data were analyzed and regressed in terms of ground measurements, leading to empirical models. The results indicate better performance of MSI for above-ground biomass (AGB) retrievals at 1.67 cm resolution and better performance of RGB data for canopy height retrievals at 1.55 cm. Although the retrieved height was well correlated with the ground measurements, a significant underestimation was observed. Thus, we developed a bias correction function to match the retrieval with the ground measurements. This study provides insight into the optimal selection of sensor for specific targeted vegetation growth traits in a forage crop.

Design and Validation of a Custom-Made Hyperspectral Microscope Imaging System for Biomedical Applications

Hyperspectral microscope imaging (HMI) is an emerging modality that integrates spatial information collected by standard laboratory microscopy and the spectral-based contrast obtained by hyperspectral imaging and may be instrumental in establishing novel quantitative diagnostic methodologies, particularly in histopathology. Further expansion of HMI capabilities hinges upon the modularity and versatility of systems and their proper standardization. In this report, we describe the design, calibration, characterization, and validation of the custom-made laboratory HMI system based on a Zeiss Axiotron fully motorized microscope and a custom-developed Czerny-Turner-type monochromator. For these important steps, we rely on a previously designed calibration protocol. Validation of the system demonstrates a performance comparable to classic spectrometry laboratory systems. We further demonstrate validation against a laboratory hyperspectral imaging system for macroscopic samples, enabling future comparison of spectral imaging results across length scales. An example of the utility of our custom-made HMI system on a standard hematoxylin and eosin-stained histology slide is also shown.

Live-cell fluorescence spectral imaging as a data science challenge

Live-cell fluorescence spectral imaging is an evolving modality of microscopy that uses specific properties of fluorophores, such as excitation or emission spectra, to detect multiple molecules and structures in intact cells. The main challenge of analyzing live-cell fluorescence spectral imaging data is the precise quantification of fluorescent molecules despite the weak signals and high noise found when imaging living cells under non-phototoxic conditions. Beyond the optimization of fluorophores and microscopy setups, quantifying multiple fluorophores requires algorithms that separate or unmix the contributions of the numerous fluorescent signals recorded at the single pixel level. This review aims to provide both the experimental scientist and the data analyst with a straightforward description of the evolution of spectral unmixing algorithms for fluorescence live-cell imaging. We show how the initial systems of linear equations used to determine the concentration of fluorophores in a pixel progressively evolved into matrix factorization, clustering, and deep learning approaches. We outline potential future trends on combining fluorescence spectral imaging with label-free detection methods, fluorescence lifetime imaging, and deep learning image analysis.

Spectroscopic and deep learning-based approaches to identify and quantify cerebral microhemorrhages

Cerebral microhemorrhages (CMHs) are associated with cerebrovascular disease, cognitive impairment, and normal aging. One method to study CMHs is to analyze histological sections (5-40 μm) stained with Prussian blue. Currently, users manually and subjectively identify and quantify Prussian blue-stained regions of interest, which is prone to inter-individual variability and can lead to significant delays in data analysis. To improve this labor-intensive process, we developed and compared three digital pathology approaches to identify and quantify CMHs from Prussian blue-stained brain sections: (1) ratiometric analysis of RGB pixel values, (2) phasor analysis of RGB images, and (3) deep learning using a mask region-based convolutional neural network. We applied these approaches to a preclinical mouse model of inflammation-induced CMHs. One-hundred CMHs were imaged using a 20 × objective and RGB color camera. To determine the ground truth, four users independently annotated Prussian blue-labeled CMHs. The deep learning and ratiometric approaches performed better than the phasor analysis approach compared to the ground truth. The deep learning approach had the most precision of the three methods. The ratiometric approach has the most versatility and maintained accuracy, albeit with less precision. Our data suggest that implementing these methods to analyze CMH images can drastically increase the processing speed while maintaining precision and accuracy.

Artificial intelligence and algorithmic computational pathology: an introduction with renal allograft examples

Whole slide imaging, which is an important technique in the field of digital pathology, has recently been the subject of increased interest and avenues for utilisation, and with more widespread whole slide image (WSI) utilisation, there will also be increased interest in and implementation of image analysis (IA) techniques. IA includes artificial intelligence (AI) and targeted or hypothesis-driven algorithms. In the overall pathology field, the number of citations related to these topics has increased in recent years. Renal pathology is one anatomical pathology subspecialty that has utilised WSIs and IA algorithms; it can be argued that renal transplant pathology could be particularly suited for whole slide imaging and IA, as renal transplant pathology is frequently classified by use of the semiquantitative Banff classification of renal allograft pathology. Hypothesis-driven/targeted algorithms have been used in the past for the assessment of a variety of features in the kidney (e.g. interstitial fibrosis, tubular atrophy, inflammation); in recent years, the amount of research has particularly increased in the area of AI/machine learning for the identification of glomeruli, for histological segmentation, and for other applications. Deep learning is the form of machine learning that is most often used for such AI approaches to the 'big data' of pathology WSIs, and deep learning methods such as artificial neural networks (ANNs)/convolutional neural networks (CNNs) are utilised. Unsupervised and supervised AI algorithms can be employed to accomplish image or semantic classification. In this review, AI and other IA algorithms applied to WSIs are discussed, and examples from renal pathology are covered, with an emphasis on renal transplant pathology.

Banff Digital Pathology Working Group: Going digital in transplant pathology

The Banff Digital Pathology Working Group (DPWG) was formed in the time leading up to and during the joint American Society for Histocompatibility and Immunogenetics/Banff Meeting, September 23-27, 2019, held in Pittsburgh, Pennsylvania. At the meeting, the 14th Banff Conference, presentations directly and peripherally related to the topic of "digital pathology" were presented; and discussions before, during, and after the meeting have resulted in a list of issues to address for the DPWG. Included are practice standardization, integrative approaches for study classification, scoring of histologic parameters (eg, interstitial fibrosis and tubular atrophy and inflammation), algorithm classification, and precision diagnosis (eg, molecular pathways and therapeutics). Since the meeting, a survey with international participation of mostly pathologists (81%) was conducted, showing that whole slide imaging is available at the majority of centers (71%) but that artificial intelligence (AI)/machine learning was only used in ≈12% of centers, with a wide variety of programs/algorithms employed. Digitalization is not just an end in itself. It also is a necessary precondition for AI and other approaches. Discussions at the meeting and the survey highlight the unmet need for a Banff DPWG and point the way toward future contributions that can be made.

Dual-mode emission and transmission microscopy for virtual histochemistry using hematoxylin- and eosin-stained tissue sections

In the clinical practice of pathology, trichrome stains are commonly used to highlight collagen and to help evaluate fibrosis. Such stains do delineate collagen deposits but are not molecularly specific and can suffer from staining inconsistencies. Moreover, performing histochemical stain evaluation requires the preparation of additional sections beyond the original hematoxylin- and eosin-stained slides, as well as additional staining steps, which together add cost, time, and workflow complications. We have developed a new microscopy approach, termed DUET (DUal-mode Emission and Transmission) that can be used to extract signals that would typically require special stains or advanced optical methods. Our preliminary analysis demonstrates the potential of using the resulting signals to generate virtual histochemical images that resemble trichrome-stained slides and can support clinical evaluation. We demonstrate advantages of this approach over images acquired from conventional trichrome-stained slides and compare them with images created using second harmonic generation microscopy.