Deep Learning in Medical Hyperspectral Images: A Review

Hyperspectral imaging for in situ visual assessment of Industrial-Scale ginseng

In industrial production, the timely assessment of ginseng-derived ingredients is crucial and requires nondestructive techniques for identifying and analyzing composition. Hyperspectral imaging (HSI) effectively visualizes the three-dimensional spatial distribution of phytochemicals in dried ginseng. This study explores the in-situ prediction and visualization of moisture content (MC) and ginsenoside content (GC) in thermally processed ginseng using dual-band HSI. We collected hyperspectral images from 216 raw ginseng samples, which underwent dimensionality reduction, noise reduction, and feature enhancement via Principal Component Analysis (PCA) and Minimum Noise Separation (MNF). Linear regression models were developed following these pretreatments and evaluated using a validation set. The PCA-based models demonstrated superior performance over those based on MNF, especially in predicting GC in the near-infrared (NIR) spectrum. Similarly, models predicting MC in the visible spectrum showed favorable results. HSI enables rapid generation of distribution maps, facilitating real-time imaging for commercial applications. Repeated drying cycles and increased duration primarily affect the textural characteristics and visible color of the ginseng surface, without significantly altering its intrinsic properties. The deployment of this predictive model alongside real-time content inversion using HSI technology holds promise for integrating visual and intelligent quality monitoring in the trade of valuable herbal commodities.

Advancements in Hyperspectral Imaging and Computer-Aided Diagnostic Methods for the Enhanced Detection and Diagnosis of Head and Neck Cancer

Background/Objectives: Head and neck cancer (HNC), predominantly squamous cell carcinoma (SCC), presents a significant global health burden. Conventional diagnostic approaches often face challenges in terms of achieving early detection and accurate diagnosis. This review examines recent advancements in hyperspectral imaging (HSI), integrated with computer-aided diagnostic (CAD) techniques, to enhance HNC detection and diagnosis. Methods: A systematic review of seven rigorously selected studies was performed. We focused on CAD algorithms, such as convolutional neural networks (CNNs), support vector machines (SVMs), and linear discriminant analysis (LDA). These are applicable to the hyperspectral imaging of HNC tissues. Results: The meta-analysis findings indicate that LDA surpasses other algorithms, achieving an accuracy of 92%, sensitivity of 91%, and specificity of 93%. CNNs exhibit moderate performance, with an accuracy of 82%, sensitivity of 77%, and specificity of 86%. SVMs demonstrate the lowest performance, with an accuracy of 76% and sensitivity of 48%, but maintain a high specificity level at 89%. Additionally, in vivo studies demonstrate superior performance when compared to ex vivo studies, reporting higher accuracy (81%), sensitivity (83%), and specificity (79%). Conclusion: Despite these promising findings, challenges persist, such as HSI's sensitivity to external conditions, the need for high-resolution and high-speed imaging, and the lack of comprehensive spectral databases. Future research should emphasize dimensionality reduction techniques, the integration of multiple machine learning models, and the development of extensive spectral libraries to enhance HSI's clinical utility in HNC diagnostics. This review underscores the transformative potential of HSI and CAD techniques in revolutionizing HNC diagnostics, facilitating more accurate and earlier detection, and improving patient outcomes.

Hyperspectral retinal imaging in Alzheimer's disease and age-related macular degeneration: a review

While Alzheimer's disease and other neurodegenerative diseases have traditionally been viewed as brain disorders, there is growing evidence indicating their manifestation in the eyes as well. The retina, being a developmental extension of the brain, represents the only part of the central nervous system that can be noninvasively imaged at a high spatial resolution. The discovery of the specific pathological hallmarks of Alzheimer's disease in the retina of patients holds great promise for disease diagnosis and monitoring, particularly in the early stages where disease progression can potentially be slowed. Among various retinal imaging methods, hyperspectral imaging has garnered significant attention in this field. It offers a label-free approach to detect disease biomarkers, making it especially valuable for large-scale population screening efforts. In this review, we discuss recent advances in the field and outline the current bottlenecks and enabling technologies that could propel this field toward clinical translation.

Advancing laser ablation assessment in hyperspectral imaging through machine learning

Hyperspectral imaging (HSI) is gaining increasing relevance in medicine, with an innovative application being the intraoperative assessment of the outcome of laser ablation treatment used for minimally invasive tumor removal. However, the high dimensionality and complexity of HSI data create a need for end-to-end image processing workflows specifically tailored to handle these data. This study addresses this challenge by proposing a multi-stage workflow for the analysis of hyperspectral data and allows investigating the performance of different components and modalities for ablation detection and segmentation. To address dimensionality reduction, we integrated principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE) to capture dominant variations and reveal intricate structures, respectively. Additionally, we employed the Faster Region-based Convolutional Neural Network (Faster R-CNN) to accurately localize ablation areas. The two-stage detection process of Faster R-CNN, along with the choice of dimensionality reduction technique and data modality, significantly influenced the performance in detecting ablation areas. The evaluation of the ablation detection on an independent test set demonstrated a mean average precision of approximately 0.74, which validates the generalization ability of the models. In the segmentation component, the Mean Shift algorithm showed high quality segmentation without manual cluster definition. Our results prove that the integration of PCA, t-SNE, and Faster R-CNN enables improved interpretation of hyperspectral data, leading to the development of reliable ablation detection and segmentation systems.

Spectral library and method for sparse unmixing of hyperspectral images in fluorescence guided resection of brain tumors

Through spectral unmixing, hyperspectral imaging (HSI) in fluorescence-guided brain tumor surgery has enabled the detection and classification of tumor regions invisible to the human eye. Prior unmixing work has focused on determining a minimal set of viable fluorophore spectra known to be present in the brain and effectively reconstructing human data without overfitting. With these endmembers, non-negative least squares regression (NNLS) was commonly used to compute the abundances. However, HSI images are heterogeneous, so one small set of endmember spectra may not fit all pixels well. Additionally, NNLS is the maximum likelihood estimator only if the measurement is normally distributed, and it does not enforce sparsity, which leads to overfitting and unphysical results. In this paper, we analyzed 555666 HSI fluorescence spectra from 891 ex vivo measurements of patients with various brain tumors to show that a Poisson distribution indeed models the measured data 82% better than a Gaussian in terms of the Kullback-Leibler divergence, and that the endmember abundance vectors are sparse. With this knowledge, we introduce (1) a library of 9 endmember spectra, including PpIX (620 nm and 634 nm photostates), NADH, FAD, flavins, lipofuscin, melanin, elastin, and collagen, (2) a sparse, non-negative Poisson regression algorithm to perform physics-informed unmixing with this library without overfitting, and (3) a highly realistic spectral measurement simulation with known endmember abundances. The new unmixing method was then tested on the human and simulated data and compared to four other candidate methods. It outperforms previous methods with 25% lower error in the computed abundances on the simulated data than NNLS, lower reconstruction error on human data, better sparsity, and 31 times faster runtime than state-of-the-art Poisson regression. This method and library of endmember spectra can enable more accurate spectral unmixing to aid the surgeon better during brain tumor resection.

Spectral characterization of intraoperative renal perfusion using hyperspectral imaging and artificial intelligence

Accurate intraoperative assessment of organ perfusion is a pivotal determinant in preserving organ function e.g. during kidney surgery including partial nephrectomy or kidney transplantation. Hyperspectral imaging (HSI) has great potential to objectively describe and quantify this perfusion as opposed to conventional surrogate techniques such as ultrasound flowmeter, indocyanine green or the subjective eye of the surgeon. An established live porcine model under general anesthesia received median laparotomy and renal mobilization. Different scenarios that were measured using HSI were (1) complete, (2) gradual and (3) partial malperfusion. The differences in spectral reflectance as well as HSI oxygenation (StO2) between different perfusion states were compelling and as high as 56.9% with 70.3% (± 11.0%) for "physiological" vs. 13.4% (± 3.1%) for "venous congestion". A machine learning (ML) algorithm was able to distinguish between these perfusion states with a balanced prediction accuracy of 97.8%. Data from this porcine study including 1300 recordings across 57 individuals was compared to a human dataset of 104 recordings across 17 individuals suggesting clinical transferability. Therefore, HSI is a highly promising tool for intraoperative microvascular evaluation of perfusion states with great advantages over existing surrogate techniques. Clinical trials are required to prove patient benefit.

A proof of concept for microcirculation monitoring using machine learning based hyperspectral imaging in critically ill patients: a monocentric observational study

BACKGROUND

Impaired microcirculation is a cornerstone of sepsis development and leads to reduced tissue oxygenation, influenced by fluid and catecholamine administration during treatment. Hyperspectral imaging (HSI) is a non-invasive bedside technology for visualizing physicochemical tissue characteristics. Machine learning (ML) for skin HSI might offer an automated approach for bedside microcirculation assessment, providing an individualized tissue fingerprint of critically ill patients in intensive care. The study aimed to determine if machine learning could be utilized to automatically identify regions of interest (ROIs) in the hand, thereby distinguishing between healthy individuals and critically ill patients with sepsis using HSI.

METHODS

HSI raw data from 75 critically ill sepsis patients and from 30 healthy controls were recorded using TIVITA® Tissue System and analyzed using an automated ML approach. Additionally, patients were divided into two groups based on their SOFA scores for further subanalysis: less severely ill (SOFA ≤ 5) and severely ill (SOFA > 5). The analysis of the HSI raw data was fully-automated using MediaPipe for ROI detection (palm and fingertips) and feature extraction. HSI Features were statistically analyzed to highlight relevant wavelength combinations using Mann-Whitney-U test and Benjamini, Krieger, and Yekutieli (BKY) correction. In addition, Random Forest models were trained using bootstrapping, and feature importances were determined to gain insights regarding the wavelength importance for a model decision.

RESULTS

An automated pipeline for generating ROIs and HSI feature extraction was successfully established. HSI raw data analysis accurately distinguished healthy controls from sepsis patients. Wavelengths at the fingertips differed in the ranges of 575-695 nm and 840-1000 nm. For the palm, significant differences were observed in the range of 925-1000 nm. Feature importance plots indicated relevant information in the same wavelength ranges. Combining palm and fingertip analysis provided the highest reliability, with an AUC of 0.92 to distinguish between sepsis patients and healthy controls.

CONCLUSION

Based on this proof of concept, the integration of automated and standardized ROIs along with automated skin HSI analyzes, was able to differentiate between healthy individuals and patients with sepsis. This approach offers a reliable and objective assessment of skin microcirculation, facilitating the rapid identification of critically ill patients.

Two-Stage Deep Learning Model for Diagnosis of Lumbar Spondylolisthesis Based on Lateral X-Ray Images

BACKGROUND

Diagnosing early lumbar spondylolisthesis is challenging for many doctors because of the lack of obvious symptoms. Using deep learning (DL) models to improve the accuracy of X-ray diagnoses can effectively reduce missed and misdiagnoses in clinical practice. This study aimed to use a two-stage deep learning model, the Res-SE-Net model with the YOLOv8 algorithm, to facilitate efficient and reliable diagnosis of early lumbar spondylolisthesis based on lateral X-ray image identification.

METHODS

A total of 2424 lumbar lateral radiographs of patients treated in the Beijing Tongren Hospital between January 2021 and September 2023 were obtained. The data were labeled and mutually identified by 3 orthopedic surgeons after reshuffling in a random order and divided into a training set, validation set, and test set in a ratio of 7:2:1. We trained 2 models for automatic detection of spondylolisthesis. YOLOv8 model was used to detect the position of lumbar spondylolisthesis, and the Res-SE-Net classification method was designed to classify the clipped area and determine whether it was lumbar spondylolisthesis. The model performance was evaluated using a test set and an external dataset from Beijing Haidian Hospital. Finally, we compared model validation results with professional clinicians' evaluation.

RESULTS

The model achieved promising results, with a high diagnostic accuracy of 92.3%, precision of 93.5%, and recall of 93.1% for spondylolisthesis detection on the test set, the area under the curve (AUC) value was 0.934.

CONCLUSIONS

Our two-stage deep learning model provides doctors with a reference basis for the better diagnosis and treatment of early lumbar spondylolisthesis.

Machine Learning Method and Hyperspectral Imaging for Precise Determination of Glucose and Silicon Levels

This article introduces an algorithm for detecting glucose and silicon levels in solution. The research focuses on addressing the critical need for accurate and efficient glucose monitoring, particularly in the context of diabetic management. Understanding and monitoring silicon levels in the body is crucial due to its significant role in various physiological processes. Silicon, while often overshadowed by other minerals, plays a vital role in bone health, collagen formation, and connective tissue integrity. Moreover, recent research suggests its potential involvement in neurological health and the prevention of certain degenerative diseases. Investigating silicon levels becomes essential for a comprehensive understanding of its impact on overall health and well-being and paves the way for targeted interventions and personalized healthcare strategies. The approach presented in this paper is based on the integration of hyperspectral data and artificial intelligence techniques. The algorithm investigates the effectiveness of two distinct models utilizing SVMR and a perceptron independently. SVMR is employed to establish a robust regression model that maps input features to continuous glucose and silicon values. The study outlines the methodology, including feature selection, model training, and evaluation metrics. Experimental results demonstrate the algorithm's effectiveness at accurately predicting glucose and silicon concentrations and showcases its potential for real-world application in continuous glucose and silicon monitoring systems.

Bedside Hyperspectral Imaging and Organ Dysfunction Severity in Critically Ill COVID-19 Patients-A Prospective, Monocentric Observational Study

Hyperspectral imaging (HSI) is a non-invasive technology that provides information on biochemical tissue properties, including skin oxygenation and perfusion quality. Microcirculatory alterations are associated with organ dysfunction in septic COVID-19 patients. This prospective observational study investigated associations between skin HSI and organ dysfunction severity in critically ill COVID-19 patients. During the first seven days in the ICU, palmar HSI measurements were carried out with the TIVITA® tissue system. We report data from 52 critically ill COVID-19 patients, of whom 40 required extracorporeal membrane oxygenation (ECMO). HSI parameters for superficial tissue oxygenation (StO2) and oxygenation and perfusion quality (NPI) were persistently decreased. Hemoglobin tissue content (THI) increased, and tissue water content (TWI) was persistently elevated. Regression analysis showed strong indications for an association of NPI and weaker indications for associations of StO2, THI, and TWI with sequential organ failure assessment (SOFA) scoring. StO2 and NPI demonstrated negative associations with vasopressor support and lactate levels as well as positive associations with arterial oxygen saturation. These results suggest that skin HSI provides clinically relevant information, opening new perspectives for microcirculatory monitoring in critical care.

Unsupervised dimensionality reduction of medical hyperspectral imagery in tensor space

BACKGROUND AND OBJECTIVES

Compared with traditional RGB images, medical hyperspectral imagery (HSI) has numerous continuous narrow spectral bands, which can provide rich information for cancer diagnosis. However, the abundant spectral bands also contain a large amount of redundancy information and increase computational complexity. Thus, dimensionality reduction (DR) is essential in HSI analysis. All vector-based DR methods ignore the cubic nature of HSI resulting from vectorization. To overcome the disadvantage of vector-based DR methods, tensor-based techniques have been developed by employing multi-linear algebra.

METHODS

To fully exploit the structure features of medical HSI and enhance computational efficiency, a novel method called unsupervised dimensionality reduction via tensor-based low-rank collaborative graph embedding (TLCGE) is proposed. TLCGE introduces entropy rate superpixel (ERS) segmentation algorithm to generate superpixels. Then, a low-rank collaborative graph weight matrix is constructed on each superpixel, greatly improving the efficiency and robustness of the proposed method. After that, TLCGE reduces dimensions in tensor space to well preserve intrinsic structure of HSI.

RESULTS

The proposed TLCGE is tested on cholangiocarcinoma microscopic hyperspectral data sets. To further demonstrate the effectiveness of the proposed algorithm, other machine learning DR methods are used for comparison. Experimental results on cholangiocarcinoma microscopic hyperspectral data sets validate the effectiveness of the proposed TLCGE.

CONCLUSIONS

The proposed TLCGE is a tensor-based DR method, which can maintain the intrinsic 3-D data structure of medical HSI. By imposing the low-rank and sparse constraints on the objective function, the proposed TLCGE can fully explore the local and global structures within each superpixel. The computational efficiency of the proposed TLCGE is better than other tensor-based DR methods, which can be used as a preprocessing step in real medical HSI classification or segmentation.

Bibliometric analysis of the current status and trends on medical hyperspectral imaging

Hyperspectral imaging (HSI) is a promising technology that can provide valuable support for the advancement of the medical field. Bibliometrics can analyze a vast number of publications on both macroscopic and microscopic levels, providing scholars with essential foundations to shape future directions. The purpose of this study is to comprehensively review the existing literature on medical hyperspectral imaging (MHSI). Based on the Web of Science (WOS) database, this study systematically combs through literature using bibliometric methods and visualization software such as VOSviewer and CiteSpace to draw scientific conclusions. The analysis yielded 2,274 articles from 73 countries/regions, involving 7,401 authors, 2,037 institutions, 1,038 journals/conferences, and a total of 7,522 keywords. The field of MHSI is currently in a positive stage of development and has conducted extensive research worldwide. This research encompasses not only HSI technology but also its application to diverse medical research subjects, such as skin, cancer, tumors, etc., covering a wide range of hardware constructions and software algorithms. In addition to advancements in hardware, the future should focus on the development of algorithm standards for specific medical research targets and cultivate medical professionals of managing vast amounts of technical information.

Hyperspectral Imaging in Brain Tumor Surgery-Evidence of Machine Learning-Based Performance

BACKGROUND

Hyperspectral imaging (HSI) has the potential to enhance surgical tissue detection and diagnostics. Definite utilization of intraoperative HSI guidance demands validated machine learning and public datasets that currently do not exist. Moreover, current imaging conventions are dispersed, and evidence-based paradigms for neurosurgical HSI have not been declared.

METHODS

We presented the rationale and a detailed clinical paradigm for establishing microneurosurgical HSI guidance. In addition, a systematic literature review was conducted to summarize the current indications and performance of neurosurgical HSI systems, with an emphasis on machine learning-based methods.

RESULTS

The published data comprised a few case series or case reports aiming to classify tissues during glioma operations. For a multitissue classification problem, the highest overall accuracy of 80% was obtained using deep learning. Our HSI system was capable of intraoperative data acquisition and visualization with minimal disturbance to glioma surgery.

CONCLUSIONS

In a limited number of publications, neurosurgical HSI has demonstrated unique capabilities in contrast to the established imaging techniques. Multidisciplinary work is required to establish communicable HSI standards and clinical impact. Our HSI paradigm endorses systematic intraoperative HSI data collection, which aims to facilitate the related standards, medical device regulations, and value-based medical imaging systems.

Study on detection method of microplastics in farmland soil based on hyperspectral imaging technology

Microplastics (MPs) in farming soils can have a substantial impact on soil ecology and agricultural productivity, as well as affecting human health and the food chain cycle. As a result, it is vital to study MPs detection technologies that are rapid, efficient, and accurate in agriculture soils. This study investigated the classification and detection of MPs using hyperspectral imaging (HSI) technology and a machine learning methodology. To begin, the hyperspectral data was preprocessed using SG convolution smoothing and Z-score normalization. Second, the feature variables were extracted from the preprocessed spectral data using bootstrapping soft shrinkage, model adaptive space shrinkage, principal component analysis, isometric mapping (Isomap), genetic algorithm, successive projections algorithm (SPA), and uninformative variable elimination. Finally, three support vector machine (SVM), back propagation neural network (BPNN), and one-dimensional convolutional neural network (1D-CNN) models were developed to classify and detect three microplastic polymers: polyethylene, polypropylene, and polyvinyl chloride, as well as their combinations. According to the experimental results, the best approaches based on three models were Isomap-SVM, Isomap-BPNN, and SPA-1D-CNN. Among them, the accuracy, precision, recall and F1_score of Isomap-SVM were 0.9385, 0.9433, 0.9385 and 0.9388, respectively. The accuracy, precision, recall and F1_score of Isomap-BPNN were 0.9414, 0.9427, 0.9414 and 0.9414, respectively, while the accuracy, precision, recall and F1_score of SPA-1D-CNN were 0.9500, 0.9515, 0.9500 and 0.9500, respectively. When their classification accuracy was compared, SPA-1D-CNN had the best classification performance, with a classification accuracy of 0.9500. The findings of this study shown that the SPA-1D-CNN based on HSI technology can efficiently and accurately identify MPs in farmland soils, providing theoretical backing as well as technical means for real-time detection of MPs in farmland soils.