Multi-modal Image Fusion for Multispectral Super-resolution in Microscopy

Moving towards a unified classification of glioblastomas utilizing artificial intelligence and deep machine learning integration

Glioblastoma a deadly brain cancer that is nearly universally fatal. Accurate prognostication and the successful application of emerging precision medicine in glioblastoma relies upon the resolution and exactitude of classification. We discuss limitations of our current classification systems and their inability to capture the full heterogeneity of the disease. We review the various layers of data that are available to substratify glioblastoma and we discuss how artificial intelligence and machine learning tools provide the opportunity to organize and integrate this data in a nuanced way. In doing so there is the potential to generate clinically relevant disease sub-stratifications, which could help predict neuro-oncological patient outcomes with greater certainty. We discuss limitations of this approach and how these might be overcome. The development of a comprehensive unified classification of glioblastoma would be a major advance in the field. This will require the fusion of advances in understanding glioblastoma biology with technological innovation in data processing and organization.

ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration

Establishing voxelwise semantic correspondence across distinct imaging modalities is a foundational yet formidable computer vision task. Current multi-modality registration techniques maximize hand-crafted inter-domain similarity functions, are limited in modeling nonlinear intensity-relationships and deformations, and may require significant re-engineering or underperform on new tasks, datasets, and domain pairs. This work presents ContraReg, an unsupervised contrastive representation learning approach to multi-modality deformable registration. By projecting learned multi-scale local patch features onto a jointly learned inter-domain embedding space, ContraReg obtains representations useful for non-rigid multi-modality alignment. Experimentally, ContraReg achieves accurate and robust results with smooth and invertible deformations across a series of baselines and ablations on a neonatal T1-T2 brain MRI registration task with all methods validated over a wide range of deformation regularization strengths.

Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging

SIGNIFICANCE

Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space.

AIM

WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics.

APPROACH

High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection.

RESULTS

Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results.

CONCLUSIONS

The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.