Medical hyperspectral imaging: a review

Computer-Assisted Differentiation between Colon-Mesocolon and Retroperitoneum Using Hyperspectral Imaging (HSI) Technology

Complete mesocolic excision (CME), which involves the adequate resection of the tumor-bearing colonic segment with “en bloc” removal of its mesocolon along embryological fascial planes is associated with superior oncological outcomes. However, CME presents a higher complication rate compared to non-CME resections due to a higher risk of vascular injury. Hyperspectral imaging (HSI) is a contrast-free optical imaging technology, which facilitates the quantitative imaging of physiological tissue parameters and the visualization of anatomical structures. This study evaluates the accuracy of HSI combined with deep learning (DL) to differentiate the colon and its mesenteric tissue from retroperitoneal tissue. In an animal study including 20 pig models, intraoperative hyperspectral images of the sigmoid colon, sigmoid mesentery, and retroperitoneum were recorded. A convolutional neural network (CNN) was trained to distinguish the two tissue classes using HSI data, validated with a leave-one-out cross-validation process. The overall recognition sensitivity of the tissues to be preserved (retroperitoneum) and the tissues to be resected (colon and mesentery) was 79.0 ± 21.0% and 86.0 ± 16.0%, respectively. Automatic classification based on HSI and CNNs is a promising tool to automatically, non-invasively, and objectively differentiate the colon and its mesentery from retroperitoneal tissue.

5D-fusion imaging for surface shape, polarization, and hyperspectral measurement

We present a five-dimensional (5D) imager that is capable of simultaneous detection of the surface shape, spectral characteristics, and polarization states of macroscopic objects, and straightforwardly fuse collected data into a 5D data set. A polarized module that uses a polarized camera obtains polarized images, while a 3D hyperspectral module reconstructs the target as a 3D point cloud using a fringe projection technique. A liquid-crystal tunable filter is placed in front of the camera of this module to acquire spectral data that can be assigned to corresponding point clouds directly. The two modules are coupled by a dual-path configuration that allows the polarization information to be merged into a comprehensive point cloud with spectral information, generating a new 5D model. The 5D imager shows excellent performance, with a spectral resolution of 10 nm, depth accuracy of 30.7 µm, and imaging time of 8 s. Sample experiments on a toy car with micro scratch defects and a yellowing plant are presented to demonstrate the capabilities of the 5D imager and its potential for use in a broad range of applications, such as industrial manufacturing inspection, plant health monitoring, and biological analysis.

Advances in Machine Learning and Hyperspectral Imaging in the Food Supply Chain

Food quality and safety are the essential hot issues of social concern. In recent years, there has been a growing demand for real-time food information, and non-destructive testing is gradually replacing traditional manual sensory testing and chemical analysis methods with lagging and destructive effects and has strong potential for application in the food supply chain. With the maturity and development of computer science and spectroscopic techniques, machine learning and hyperspectral imaging (HSI) have been widely demonstrated as efficient detection techniques that can be applied to rapidly evaluate sensory characteristics and quality attributes of food products nondestructively and efficiently. This paper first briefly described the basic concepts of hyperspectral imaging and machine learning, including the imaging process of HSI, the type of algorithms contained in machine learning, and the data processing flow. Secondly, this paper provided an objective and comprehensive overview of the current applications of machine learning and HSI in the food supply chain for sorting, packaging, transportation, storage, and sales, based on the state-of-art literature from 2017 to 2022. Finally, the potential of the technology is further discussed to provide optimized ideas for practical application.

Hyperspectral three-dimensional absorption imaging using snapshot optical tomography

Hyperspectral imaging (HSI) records a series of two-dimensional (2D) images for different wavelengths to provide the chemical fingerprint at each pixel. Combining HSI with a tomographic data acquisition method, we can obtain the chemical fingerprint of a sample at each point in three-dimensional (3D) space. The so-called 3D HSI typically suffers from low imaging throughput due to the requirement of scanning the wavelength and rotating the beam or sample. In this paper we present an optical system which captures the entire four-dimensional (4D), i.e., 3D structure and 1D spectrum, dataset of a sample with the same throughput of conventional HSI systems. Our system works by combining snapshot projection optical tomography (SPOT) which collects multiple projection images with a single snapshot, and Fourier-transform spectroscopy (FTS) which results in superior spectral resolution by collecting and processing a series of interferogram images. Using this hyperspectral SPOT system we imaged the volumetric absorbance of dyed polystyrene microbeads, oxygenated red blood cells (RBCs), and deoxygenated RBCs. The 4D optical system demonstrated in this paper provides a tool for high-throughput chemical imaging of complex microscopic specimens.

Design and Validation of a Custom-Made Laboratory Hyperspectral Imaging System for Biomedical Applications Using a Broadband LED Light Source

Hyperspectral imaging (HSI) is a promising optical modality that is already being used in numerous applications. Further expansion of the capabilities of HSI depends on the modularity and versatility of the systems, which would, inter alia, incorporate profilometry, fluorescence imaging, and Raman spectroscopy while following a rigorous calibration and verification protocols, thus offering new insights into the studied samples as well as verifiable, quantitative measurement results applicable to the development of quantitative metrics. Considering these objectives, we developed a custom-made laboratory HSI system geared toward biomedical applications. In this report, we describe the design, along with calibration, characterization, and verification protocols needed to establish such systems, with the overall goal of standardization. As an additional novelty, our HSI system uses a custom-built broadband LED-based light source for reflectance imaging, which is particularly important for biomedical applications due to the elimination of sample heating. Three examples illustrating the utility and advantages of the integrated system in biomedical applications are shown. Our attempt presents both the development of a custom-based laboratory HSI system with novel LED light source as well as a framework which may improve technological standards in HSI system design.

Exploring time series of hyperspectral images for cold water coral stress response analysis

Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a lot of undeveloped potential due to the high dimensionality and complexity of the data. If temporal effects are studied, such as in a monitoring context, the analysis becomes more challenging as time is added to the dimensions of space (image coordinates) and wavelengths. We conducted a series of laboratory experiments to investigate the impact of different stressor exposure patterns on the spectrum of the cold water coral Desmophyllum pertusum. 65 coral samples were divided into 12 groups, each group being exposed to different types and levels of particles. Hyperspectral images of the coral samples were collected at four time points from prior to exposure to 6 weeks after exposure. To investigate the relationships between the corals’ spectral signatures and controlled experimental parameters, a new software tool for interactive visual exploration was developed and applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles from exploratory data analysis, information visualization and machine learning-based dimension reduction. This combination enables users to select regions of interest (ROI) in all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We propose two HypIX workflows to find relationships in time series of hyperspectral datasets, namely morphology-based filtering workflow and embedded driven response analysis workflow. With these HypIX workflows three users identified different temporal and spatial patterns in the spectrum of corals exposed to different particle stressor conditions. Corals exposed to particles tended to have a larger change rate than control corals, which was evident as a shifted spectrum. The responses, however, were not uniform for coral samples undergoing the same exposure treatments, indicating individual tolerance levels. We also observed a good inter-observer agreement between the three HyPIX users, indicating that the proposed workflow can be applied to obtain reproducible HSI analysis results.

Deep spatial-spectral prior with an adaptive dual attention network for single-pixel hyperspectral reconstruction

Recently, single-pixel imaging has shown great promise in developing cost-effective imaging systems, where coding and reconstruction are the keys to success. However, it also brings challenges in capturing hyperspectral information accurately and instantly. Many works have attempted to improve reconstruction performance in single-pixel hyperspectral imaging by applying various hand-crafted priors, leading to sub-optimal solutions. In this paper, we present the deep spatial-spectral prior with adaptive dual attention network for single-pixel hyperspectral reconstruction. Specifically, the spindle structure of the parameter sharing method is developed to integrate information across spatial and spectral dimensions of HSI, which can synergistically and efficiently extract global and local prior information of hyperspectral images from both shallow and deep layers. Particularly, a sequential adaptive dual attention block (SADAB), i.e., spatial attention and spectral attention, are devised to adaptively rescale informative features of spatial locations and spectral channels simultaneously, which can effectively boost the reconstruction accuracy. Experiment results on public HSI datasets demonstrate that the proposed method significantly outperforms the state-of-the-art algorithm in terms of reconstruction accuracy and speed.

A Review of Pharmaceutical Robot based on Hyperspectral Technology

The quality and safety of medicinal products are related to patients’ lives and health. Therefore, quality inspection takes a key role in the pharmaceutical industry. Most of the previous solutions are based on machine vision, however, their performance is limited by the RGB sensor. The pharmaceutical visual inspection robot combined with hyperspectral imaging technology is becoming a new trend in the high-end medical quality inspection process since the hyperspectral data can provide spectral information with spatial knowledge. Yet, there is no comprehensive review about hyperspectral imaging-based medicinal products inspection. This paper focuses on the pivotal pharmaceutical applications, including counterfeit drugs detection, active component analysis of tables, and quality testing of herbal medicines and other medical materials. We discuss the technology and hardware of Raman spectroscopy and hyperspectral imaging, firstly. Furthermore, we review these technologies in pharmaceutical scenarios. Finally, the development tendency and prospect of hyperspectral imaging technology-based robots in the field of pharmaceutical quality inspection is summarized.

Noninvasive hemoglobin sensing and imaging: optical tools for disease diagnosis

SIGNIFICANCE

Measurement and imaging of hemoglobin oxygenation are used extensively in the detection and diagnosis of disease; however, the applied instruments vary widely in their depth of imaging, spatiotemporal resolution, sensitivity, accuracy, complexity, physical size, and cost. The wide variation in available instrumentation can make it challenging for end users to select the appropriate tools for their application and to understand the relative limitations of different methods.

AIM

We aim to provide a systematic overview of the field of hemoglobin imaging and sensing.

APPROACH

We reviewed the sensing and imaging methods used to analyze hemoglobin oxygenation, including pulse oximetry, spectral reflectance imaging, diffuse optical imaging, spectroscopic optical coherence tomography, photoacoustic imaging, and diffuse correlation spectroscopy.

RESULTS

We compared and contrasted the ability of different methods to determine hemoglobin biomarkers such as oxygenation while considering factors that influence their practical application.

CONCLUSIONS

We highlight key limitations in the current state-of-the-art and make suggestions for routes to advance the clinical use and interpretation of hemoglobin oxygenation information.

Adaptive denoising hyperspectral data for visualization enhancement of intraoperative tissue

The vast amount of reflectance information obtained from the hyperspectral imaging devices offers great opportunities for investigating the function and structure of human tissue. However, the captured hyperspectral data often contain various noises due to the intrinsic imperfection of associated electrical and optical imaging components. This work proposed an automatic total variation algorithm to suppress the noises while preserving the details of the spectral and spatial information. The variation of spectral images at neighboring bands was calculated for regulating the total variation of hyperspectral data so that the spectral-dependent noises can be treated differentially across all bands. Experimental results demonstrated that the proposed method could effectively remove the spectral noises, especially near the ends of those extreme bands. The noise suppressed hyperspectral data could then be used for the visualization enhancement on pathophysiological conditions of intraoperative observed anatomies such as the vessels of brain tissues.

Burn depth assessment using hyperspectral imaging in a prospective single center study

BACKGROUND

The assessment of thermal burn depth remains challenging. Over the last decades, several optical systems were developed to determine burn depth. So far, only laser doppler imaging (LDI) has been shown to be reliable while others such as infrared thermography or spectrophotometric intracutaneous analysis have been less accurate. The aim of our study is to evaluate hyperspectral imaging (HSI) as a new optical device.

METHODS

Patients suffering thermal trauma treated in a burn unit in Germany between November 2019 and September 2020 were included. Inclusion criteria were age ≥18 years, 2^nd or 3^rd degree thermal burns, written informed consent and presentation within 24 h after injury. Clinical assessment and hyperspectral imaging were performed 24, 48 and 72 h after the injury. Patients in whom secondary wound closure was complete within 21 days (group A) were compared to patients in whom secondary wound closure took more than 21 days or where skin grafting was indicated (group B). Demographic data and the primary parameters generated by HSI were documented. A Mann Whitney-U test was performed to compare the groups. A p-value below 0.05 was considered to be statistically significant. The data generated using HSI were combined to create the HSI burn index (BI). Using a logistic regression and receiver operating characteristics curve (ROC) sensitivity and specificity of the BI were calculated. The trial was officially registered on DRKS (registration number: DRKS00022843).

RESULTS

Overall, 59 patients with burn wounds were eligible for inclusion. Ten patients were excluded because of a poor data quality. Group A comprised 36 patients with a mean age of 41.5 years and a mean burnt body surface area of 2.7%. In comparison, 13 patients were allocated to group B because of the need for a skin graft (n = 10) or protracted secondary wound closure lasting more than 21 days. The mean age of these patients was 46.8 years. They had a mean affected body surface area of 4.0%. 24, 48, and 72 h after trauma the BI was 1.0 ± 0.28, 1.2 ± 0.29 and 1.55 ± 0.27 in group A and 0.78 ± 0.14, 1.05 ± 0.23 and 1.23 ± 0.27 in group B. At every time point significant differences were demonstrated between the groups. At 24 h, ROC analysis demonstrated BI threshold of 0.95 (sensitivity 0.61/specificity 1.0), on the second day of 1.17 (sensitivity 0.51/specificity 0.81) and on the third day of 1.27 (sensitivity 0.92/specificity 0.71).

CONCLUSION

Changes in microcirculation within the first 72 h after thermal trauma were reflected by an increasing BI in both groups. After 72 h, the BI is able to predict the need for a skin graft with a sensitivity of 92% and a specificity of 71%.

Applications of Artificial Intelligence Based on Medical Imaging in Glioma: Current State and Future Challenges

Glioma is one of the most fatal primary brain tumors, and it is well-known for its difficulty in diagnosis and management. Medical imaging techniques such as magnetic resonance imaging (MRI), positron emission tomography (PET), and spectral imaging can efficiently aid physicians in diagnosing, treating, and evaluating patients with gliomas. With the increasing clinical records and digital images, the application of artificial intelligence (AI) based on medical imaging has reduced the burden on physicians treating gliomas even further. This review will classify AI technologies and procedures used in medical imaging analysis. Additionally, we will discuss the applications of AI in glioma, including tumor segmentation and classification, prediction of genetic markers, and prediction of treatment response and prognosis, using MRI, PET, and spectral imaging. Despite the benefits of AI in clinical applications, several issues such as data management, incomprehension, safety, clinical efficacy evaluation, and ethical or legal considerations, remain to be solved. In the future, doctors and researchers should collaborate to solve these issues, with a particular emphasis on interdisciplinary teamwork.

Comparison of Hyperspectral Imaging and Microvascular Doppler for Perfusion Monitoring of Free Flaps in an In Vivo Rodent Model

To reduce microvascular free flap failure (MFF), monitoring is crucial for the early detection of malperfusion and allows timely salvage. Therefore, the aim of this study was to evaluate hyperspectral imaging (HSI) in comparison to micro-Doppler sonography (MDS) to monitor MFF perfusion in an in vivo rodent model. Bilateral groin flaps were raised on 20 Sprague−Dawley rats. The femoral artery was transected on the trial side and re-anastomosed. Flaps and anastomoses were assessed before, during, and after the period of ischemia every ten minutes for overall 60 min using HSI and MDS. The contralateral sides’ flaps served as controls. Tissue-oxygenation saturation (StO2), near-infrared perfusion index (NPI), hemoglobin (THI), and water distribution (TWI) were assessed by HSI, while blood flow was assessed by MDS. HSI correlates with the MDS signal in the case of sufficient and completely interrupted perfusion. HSI was able to validly and reproducibly detect tissue perfusion status using StO2 and NPI. After 40 min, flap perfusion decreased due to the general aggravation of hemodynamic circulatory situation, which resulted in a significant drop of StO2 (p < 0.005) and NPI (p < 0.005), whereas the Doppler signal remained unchanged. In accordance, HSI might be suitable to detect MFF general complications in an early stage and further decrease MFF failure rates, whereas MDS may only be used for direct complications at the anastomose site.

Hyperspectral image super-resolution based on the transfer of both spectra and multi-level features

Existing hyperspectral image (HSI) super-resolution methods fusing a high-resolution RGB image (HR-RGB) and a low-resolution HSI (LR-HSI) always rely on spatial degradation and handcrafted priors, which hinders their practicality. To address these problems, we propose a novel, to the best of our knowledge, method with two transfer models: a window-based linear mixing (W-LM) model and a feature transfer model. Specifically, W-LM initializes a high-resolution HSI (HR-HSI) by transferring the spectra from the LR-HSI to the HR-RGB. By using the proposed feature transfer model, the HR-RGB multi-level features extracted by a pre-trained convolutional neural network (CNN) are then transferred to the initialized HR-HSI. The proposed method fully exploits spectra of LR-HSI and multi-level features of HR-RGB and achieves super-resolution without requiring the spatial degradation model and any handcrafted priors. The experimental results for 32 × super-resolution on two public datasets and our real image set demonstrate the proposed method outperforms eight state-of-the-art existing methods.

Assessment of angle-dependent spectral distortion to develop accurate hyperspectral endoscopy

Hyperspectral endoscopy has shown its potential to improve disease diagnosis in gastrointestinal tracts. Recent approaches in developing hyperspectral endoscopy are mainly focusing on enhancing image speed and quality of spectral information under a clinical environment, but there are many issues in obtaining consistent spectral information due to complicated imaging conditions, including imaging angle, non-uniform illumination, working distance, and low reflected signal. We quantitatively investigated the effect of imaging angle on the distortion of spectral information by exploiting a bifurcated fiber, spectrometer, and tissue-mimicking phantom. Spectral distortion becomes severe as increasing the angle of the imaging fiber or shortening camera exposure time for fast image acquisition. Moreover, spectral ranges from 450 to 550 nm are more susceptible to the angle-dependent spectral distortion than longer spectral ranges. Therefore, imaging angles close to normal and longer target spectral ranges with enough detector exposure time could minimize spectral distortion in hyperspectral endoscopy. These findings will help implement clinical HSI endoscopy for the robust and accurate measurement of spectral information from patients in vivo.

A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging

Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.

[Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]

BACKGROUND

Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article.

OBJECTIVE

What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery?

MATERIAL AND METHODS

Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors.

RESULTS

In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91.

DISCUSSION

This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.

Coded Hyperspectral Image Reconstruction Using Deep External and Internal Learning

To solve the low spatial and/or temporal resolution problem which the conventional hyperspectral cameras often suffer from, coded hyperspectral imaging systems have attracted more attention recently. Recovering a hyperspectral image (HSI) from its corresponding coded image is an ill-posed inverse problem, and learning accurate prior of HSI is essential to solve this inverse problem. In this paper, we present an effective convolutional neural network (CNN) based method for coded HSI reconstruction, which learns the deep prior from the external dataset as well as the internal information of input coded image with spatial-spectral constraint. Specifically, we first develop a CNN-based channel attention reconstruction network to effectively exploit the spatial-spectral correlation of the HSI. Then, the reconstruction network is learned by leveraging an arbitrary external hyperspectral dataset to exploit the general spatial-spectral correlation under adversarial loss. Finally, we customize the network by internal learning with spatial-spectral constraint and total variation regularization for each coded image, which can make use of the internal imaging model to learn specific prior for current desirable image and effectively avoids overfitting. Experimental results using both synthetic data and real images show that our method outperforms the state-of-the-art methods on several popular coded hyperspectral imaging systems under both comprehensive quantitative metrics and perceptive quality.

Microscopy is better in color: development of a streamlined spectral light path for real-time multiplex fluorescence microscopy

Spectroscopic image data has provided molecular discrimination for numerous fields including: remote sensing, food safety and biomedical imaging. Despite the various technologies for acquiring spectral data, there remains a trade-off when acquiring data. Typically, spectral imaging either requires long acquisition times to collect an image stack with high spectral specificity or acquisition times are shortened at the expense of fewer spectral bands or reduced spatial sampling. Hence, new spectral imaging microscope platforms are needed to help mitigate these limitations. Fluorescence excitation-scanning spectral imaging is one such new technology, which allows more of the emitted signal to be detected than comparable emission-scanning spectral imaging systems. Here, we have developed a new optical geometry that provides spectral illumination for use in excitation-scanning spectral imaging microscope systems. This was accomplished using a wavelength-specific LED array to acquire spectral image data. Feasibility of the LED-based spectral illuminator was evaluated through simulation and benchtop testing and assessment of imaging performance when integrated with a widefield fluorescence microscope. Ray tracing simulations (TracePro) were used to determine optimal optical component selection and geometry. Spectral imaging feasibility was evaluated using a series of 6-label fluorescent slides. The LED-based system response was compared to a previously tested thin-film tunable filter (TFTF)-based system. Spectral unmixing successfully discriminated all fluorescent components in spectral image data acquired from both the LED and TFTF systems. Therefore, the LED-based spectral illuminator provided spectral image data sets with comparable information content so as to allow identification of each fluorescent component. These results provide proof-of-principle demonstration of the ability to combine output from many discrete wavelength LED sources using a double-mirror (Cassegrain style) optical configuration that can be further modified to allow for high speed, video-rate spectral image acquisition. Real-time spectral fluorescence microscopy would allow monitoring of rapid cell signaling processes (i.e., Ca²⁺ and other second messenger signaling) and has potential to be translated to clinical imaging platforms.

Hyperspectral Imaging (HSI) of Human Kidney Allografts

OBJECTIVE

Aim of our study was to test a noninvasive HSI technique as an intraoperative real time assessment tool for deceased donor kidney quality and function in human kidney allotransplantation.

SUMMARY OF BACKGROUND DATA

HSI is capable to deliver quantitative diagnostic information about tissue pathology, morphology, and composition, based on the spectral characteristics of the investigated tissue. Because tools for objective intraoperative graft viability and performance assessment are lacking, we applied this novel technique to human kidney transplantation.

METHODS

Hyperspectral images of distinct components of kidney allografts (parenchyma, ureter) were acquired 15 and 45 minutes after reperfusion and subsequently analyzed using specialized HSI acquisition software capable to compute oxygen saturation levels (StO2), near infrared perfusion indices (NIR), organ hemoglobin indices, and tissue water indices of explored tissues.

RESULTS

Seventeen kidney transplants were analyzed. Median recipient and donor age were 55 years. Cold ischemia time was 10.8 ± 4.1 hours and anastomosis time was 35 ± 7 minutes (mean ± standard deviation). Two patients (11.8%) developed delayed graft function (DGF). cold ischemia time was significantly longer (18.6 ± 1.6) in patients with DGF (P < 0.01). Kidneys with DGF furthermore displayed significant lower StO2 (P = 0.02) and NIR perfusion indices, 15 minutes after reperfusion (P < 0.01). Transplant ureters displayed a significant decrease of NIR perfusion with increased distance to the renal pelvis, identifying well and poor perfused segments.

CONCLUSION

Intraoperative HSI is feasible and meaningful to predict DGF in renal allografts. Furthermore, it can be utilized for image guided surgery, providing information about tissue oxygenation, perfusion, hemoglobin concentration, and water concentration, hence allowing intraoperative viability assessment of the kidney parenchyma and the ureter.

Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model

Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.

Prediction of the Quality of Thermally Sprayed Copper Coatings on Laser-Structured CFRP Surfaces Using Hyperspectral Imaging

With the progressive replacement of metallic parts by high-performance fiber-reinforced plastic (FRP) components, typical properties of metals are required to be placed on the material’s surface. A metallic coating applied to the FRP surface by thermal spraying, for instance, can fulfill these requirements, including electrical conductivity. In this work, laser pre-treatments are utilized for increasing the bond strength of metallic coatings. However, due to the high-precision material removal using pulsed laser radiation, the production-related heterogeneous fiber distribution in FRP leads to variations in the structuring result and consequently to different qualities of the subsequent coating. In this study, hyperspectral imaging (HSI) technologies in conjunction with deep learning were applied to carbon fiber-reinforced plastics (CFRP) structured by nanosecond pulsed laser. HSI-based prediction models could be developed, which allow for reliable prediction, with an accuracy of around 80%, of which laser-treated areas will successfully be coated and which will not. By using this objective and automatic evaluation, it is possible to avoid large amounts of rejects before further processing the parts and also to optimize the adhesion of coatings. Spatially resolved data enables local reworking during the laser process, making it feasible for the manufacturing process to achieve zero waste.

Measuring High Dynamic Range Spectral Reflectance of Artworks through an Image Capture Matrix Hyperspectral Camera

Commercial hyperspectral imaging systems typically use CCD or CMOS sensors. These types of sensors have a limited dynamic range and non-linear response. This means that when evaluating an artwork under uncontrolled lighting conditions and with light and dark areas in the same scene, hyperspectral images with underexposed or saturated areas would be obtained at low or high exposure times, respectively. To overcome this problem, this article presents a system for capturing hyperspectral images consisting of a matrix of twelve spectral filters placed in twelve cameras, which, after processing these images, makes it possible to obtain the high dynamic range image to measure the spectral reflectance of the work of art being evaluated. We show the developed system and describe all its components, calibration processes, and the algorithm implemented to obtain the high dynamic range spectral reflectance measurement. In order to validate the system, high dynamic range spectral reflectance measurements from Labsphere's Spectralon Reflectance Standards were performed and compared with the same reflectance measurements but using low dynamic range images. High dynamic range hyperspectral imaging improves the colorimetric accuracy and decreases the uncertainty of the spectral reflectance measurement based on low dynamic range imaging.

Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery

The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.

Visible and Near-Infrared hyperspectral imaging (HSI) can reliably quantify CD3 and CD45 positive inflammatory cells in myocarditis: Pilot study on formalin-fixed paraffin-embedded specimens from myocard obtained during autopsy

INTRODUCTION

To implement Hyperspectral Imaging (HSI) as a tool for quantifying inflammatory cells in tissue specimens by the example of myocarditis in a collective of forensic patients.

MATERIAL AND METHODS

44 consecutive patients with suspected myocardial inflammation at autopsy, diagnosed between 2013 and 2018 at the Institute of ForensicMedicine, Medical University of Innsbruck, were selected for this study. Using the IMEC SNAPSCAN camera, visible and near infrared hyperspectral images were collected from slides stained with CD3 and CD45 to assess quantity and spatial distribution of positive cells. Results were compared with visual assessment (VA) and conventional digital image analysis (DIA).

RESULTS

Finally, specimens of 40 patients were evaluated, of whom 36 patients (90%) suffered from myocarditis, two patients (5%) had suspected healing/healed myocarditis, and two did no have myocarditis (5%). The amount of CD3 and CD45 positive cells did not differ significantly between VA, HSI, and DIA (p^VA/HSI/DIA = 0.46 for CD3 and 0.81 for CD45). Coheńs Kappa showed a very high correlation between VA versus HSI, VA versus DIA, and HSI versus DIA for CD3 (Coheńs Kappa = 0.91, 1.00, and 0.91, respectively). For CD45 an almost as high correlation was seen for VA versus HSI and HSI versus DIA (Coheńs Kappa = 0.75 and 0.70) and VA versus DIA (Coheńs Kappa = 0.89).

CONCLUSION

HSI is a reliable and objective method to count inflammatory cells in tissue slides of suspected myocarditis. Implementation of HSI in digital pathology might further expand the possibility of a sophisticated method.

Screening chronic myeloid leukemia neutrophils using a novel 3-Dimensional Spectral Gradient Mapping algorithm on hyperspectral images

Background and objective Early diagnosis of chronic myeloid leukemia (CML) is important for effective treatment. The high spectral and spatial resolution of hyperspectral cellular or tissue images coupled with image analysis algorithms may provide avenues to detect and diagnose diseases early. Many algorithms have been used to analyze medical hyperspectral image data, each having their own strengths and short-comings. We present a novel 3-Dimensional Spectral Gradient Mapping (3-D SGM) method to analyze hyperspectral image cubes of CML versus healthy blood smears. Methods In the present study, we analyzed 13 hyperspectral image cubes of CML and healthy neutrophils. The 3-D SGM algorithm was compared to the conventional Windowed Spectral Angle Mapping (Windowed SAM) method. The 3-D SGM exploited the spectral information of the image cube together with the inter-band and inter-pixel data by extracting the 3-D gradient vector from each pixel. The Windowed SAM determined the similarity between the averaged window of a 2×2 training pixel group and the test pixel, in the multidimensional spectral angle. Results The specificity measure of 3-D SGM (97.7%) was superior to Windowed SAM (72.7%) at ruling out the presence of the disease, making it potentially ideal for screening patients. The positive likelihood ratio value of 3-D SGM (16.70) was superior in diagnosing the presence of the disease (i.e., positive test for CML) versus Windowed SAM (2.26). An accuracy value of 84.2% was achieved with 3-D SGM versus only 70.2% for Windowed SAM. Conclusion The new method is efficient and robust for analyzing hyperspectral images of CML versus healthy neutrophils. It has the potential to be developed into an inexpensive, minimally invasive method for screening CML, and could directly facilitate early diagnosis and treatment of the disease.

Hyperspectral Evaluation of the Human Liver During Major Resection

Objective

This study investigates the effects of PVE and vascular inflow control (VIC) on liver microperfusion and tissue oxygenation using hyperspectral imaging (HSI) technology.

Background

Mechanisms triggering future liver remnant (FLR) augmentation introduced by PVE have not been sufficiently studied in humans. Particularly, the arterial buffer response (ABR) of the liver might play a vital role.

Methods

Hyperspectral datacubes (TIVITA) acquired during 58 major liver resections were qualitatively and quantitatively analyzed for tissue oxygenation (StO2%), near-infrared (NIR) perfusion, organ-hemoglobin indices (OHI), and tissue-water indices (TWI). The primary study endpoint was measurement of hyperspectral differences in liver parenchyma subject to PVE and VIC before resection.

Results

HSI revealed parenchyma specific differences in StO2% with regard to the underlying disease (P < 0.001). Preoperative PVE (n = 23, 40%) lead to arterial hyperoxygenation and hyperperfusion of corresponding liver segments (StO2: 77.23% ± 11.93%, NIR: 0.46 ± 0.20[I]) when compared with the FLR (StO2: 66.13% ± 9.96%, NIR: 0.23 ± 0.12[I]; P < 0.001). In a case of insufficient PVE and the absence of FLR augmentation hyperspectral StO2 and NIR differences were absent. The hyperspectral assessment demonstrated increased liver tissue-oxygenation and perfusion in PVE-segments (n = 23 cases) and decreased total VIC in nonembolized FLR hemilivers (n = 35 cases; P < 0.001). Intraoperative HSI analysis of tumor tissue revealed marked tumor specific differences in StO2, NIR, OHI, and TWI (P < 0.001).

Conclusions

HSI allows intraoperative quantitative and qualitative assessment of microperfusion and StO2% of liver tissue. PVE lead to ABR-triggered tissue hyperoxygenation and cross-talk FLR augmentation. HSI furthermore facilitates intraoperative tumor tissue identification and enables image-guided liver surgery following VIC.

Hyperspectral evaluation of vasculature in induced peritonitis mouse models

Imaging of blood vessel structure in combination with functional information about blood oxygenation can be important in characterizing many different health conditions in which the growth of new vessels contributes to the overall condition. In this paper, we present a method for extracting comprehensive maps of the vasculature from hyperspectral images that include tissue and vascular oxygenation. We also show results from a preclinical study of peritonitis in mice. First, we analyze hyperspectral images using Beer-Lambert exponential attenuation law to obtain maps of hemoglobin species throughout the sample. We then use an automatic segmentation algorithm to extract blood vessels from the hemoglobin map and combine them into a vascular structure-oxygenation map. We apply this methodology to a series of hyperspectral images of the abdominal wall of mice with and without induced peritonitis. Peritonitis is an inflammation of peritoneum that leads, if untreated, to complications such as peritoneal sclerosis and even death. Characteristic inflammatory response can also be accompanied by changes in vasculature, such as neoangiogenesis. We demonstrate a potential application of the proposed segmentation and processing method by introducing an abnormal tissue fraction metric that quantifies the amount of tissue that deviates from the average values of healthy controls. It is shown that the proposed metric successfully discriminates between healthy control subjects and model subjects with induced peritonitis and has a high statistical significance.

Development and verification of the coaxial heterogeneous hyperspectral imaging system

A hyperspectral imaging system (HIS) is a helpful tool that acquires spatial and spectral information from a target. This study developed a coaxial heterogeneous HIS (CHHIS) to collect spectral images with wavelengths ranging from 400 to 1700 nm. In this system, a visible (VIS) spectrometer and a short-wave infrared (SWIR) spectrometer are combined with a coaxial optical path to share the same field of view. This structure reduces the complexity of spatial registration and maintains the scanning duration of two spectrometers as that of a single spectrometer. The spectrometers are also replaceable for extending the detecting spectral range of the system. The calibration methodologies, including spatial correction, spectral calibration, and reflectance calibration, were developed for this system. The signal-to-noise ratio of VIS and SWIR spectrometers in the CHHIS was up to 40 and 60 dB when the exposure time of the VIS and SWIR imaging sensors was 1000 and 10 ms, respectively. When the target distance was at 600 mm, the spatial error of VIS and SWIR images in the scanning direction was less than 1 pixel; these results proved that the system was stable.

Achieving dual-color imaging by dual-band perovskite photodetectors coupled with algorithms

Dual-color or multispectral imaging based on conventional optical imaging techniques is suffering from the bottleneck of complex manufacturing and time consumption caused by multiple imaging. Herein, we develop a dual-color computational imaging system combining a vertically stacked dual-channel dual-band perovskite photodetectors (PDs) and the advanced Fourier imaging algorithm. Significantly, our imaging system bypasses the complex fabrication process of high-density dual-band PD arrays and is enabled to capture two high-resolution spectral images at the same time. Based on the experiments and simulations, we confirm that the spectral overlap of dual-band PDs will cause detrimental effect for color identification, and optimizing the bandwidth spectrum is beneficial for achieving much better spectral imaging. Moreover, we have further improved the imaging quality by increasing the sampling rate and suppressing current fluctuations. We suggest that these results provide important interesting insights for the development of advanced imaging systems, including IR imaging, THz imaging, multispectral/hyperspectral imaging, etc.

Hyperspectral imaging for quantitative assessment of hepatic steatosis in human liver allografts

INTRODUCTION

In liver transplantation (LT), steatosis is commonly judged to be a risk factor for graft dysfunction, and quantitative assessment of hepatic steatosis remains crucial. Liver biopsy as the gold standard for evaluation of hepatic steatosis has certain drawbacks, i.e. invasiveness, and intra- and inter-observer variability. A non-invasive, quantitative modality could replace liver biopsy and eliminate these disadvantages, but has not yet been evaluated in human LT.

METHODS

We performed a pilot study to evaluate the feasibility and accuracy of hyperspectral imaging (HSI) in the assessment of hepatic steatosis of human liver allografts for transplantation. Thirteen deceased donor liver allografts were included in the study. The degree of steatosis was assessed by means of conventional liver biopsy as well as HSI, performed at the end of backtable preparation, during normothermic machine perfusion (NMP), and after reperfusion in the recipient.

RESULTS

Organ donors were 51 [30-83] years old, and 61.5% were male. Donor body mass index was 24.2 [16.5-38.0] kg/m2. The tissue lipid index (TLI) generated by HSI at the end of back-table preparation correlated significantly with the histopathologically assessed degree of overall hepatic steatosis (R2 = 0.9085, p<0.0001); this was based on a correlation of TLI and microvesicular steatosis (R2 = 0.8120; p<0.0001). There is also a linear relationship between the histopathologically assessed degree of overall steatosis and TLI during NMP (R2 = 0.5646; p = 0.0031) as well as TLI after reperfusion (R2 = 0.6562; p = 0.0008).

CONCLUSION

HSI may safely be applied for accurate assessment of hepatic steatosis in human liver grafts. Certainly, TLI needs further assessment and validation in larger sample sizes. This article is protected by copyright. All rights reserved.

In vivo multi spectral colonoscopy in mice

Multi- and hyperspectral endoscopy are possibilities to improve the endoscopic detection of neoplastic lesions in the colon and rectum during colonoscopy. However, most studies in this context are performed on histological samples/biopsies or ex vivo. This leads to the question if previous results can be transferred to an in vivo setting. Therefore, the current study evaluated the usefulness of multispectral endoscopy in identifying neoplastic lesions in the colon. The data set consists of 25 mice with colonic neoplastic lesions and the data analysis is performed by machine learning. Another question addressed was whether adding additional spatial features based on Gauss-Laguerre polynomials leads to an improved detection rate. As a result, detection of neoplastic lesions was achieved with an MCC of 0.47. Therefore, the classification accuracy of multispectral colonoscopy is comparable with hyperspectral colonoscopy in the same spectral range when additional spatial features are used. Moreover, this paper strongly supports the current path towards the application of multi/hyperspectral endoscopy in clinical settings and shows that the challenges from transferring results from ex vivo to in vivo endoscopy can be solved.

Repeatability and reproducibility of a hyperspectral imaging system for in vivo color evaluation

BACKGROUND

Color imaging is a tried and true method for the evaluation of cosmetic and dermatological effects, but it fails to capture all the information in a scene's spectral reflectance. For this reason, there has been in recent years increasing interest in the use of imaging spectrometers for clinical studies and product evaluation.

MATERIAL AND METHODS

We developed a novel HyperSpectral Imager (HSI) able to take in vivo full-face format images as a next generation instrument for skin color measurement and beyond. Here, we report part of the results of our first full-scale validation test of the HSI. We replicated a make-up foundation screening test by applying three products to a panel of 9 models and evaluated the product L^∗ , a^∗ , b^∗ , and ∆E effect immediately after application relative to the bare skin condition. We repeated this test twice in order to study the repeatability of the HSI as an evaluation instrument and during each test two different operators duplicated the data acquisition so we can assess the reproducibility of the measurements.

RESULTS

We find that the measurements from the HSI provide repeatability and reproducibility as good or better than those of our previous benchmark devices.

CONCLUSION

From these results, we conclude that not only is the HSI suitable for use in color evaluation studies, but also that it gives operational advantages over the previous generation of evaluation instruments, as it provides a spectral measurement combined with good spatial resolution. This allows for analysis of color over an area and post hoc selection of study regions and so opens new possibilities for studies of complex in vivo phenomena which neither non-imaging spectrometers nor conventional cameras can pursue. This study also raises points for future work concerning proper inclusion of instrument uncertainty in comparisons of results between instruments and handling of systematic uncertainties from analyses based on a single area.

Hyperspectral Imaging as a Tool for Viability Assessment During Normothermic Machine Perfusion of Human Livers: A Proof of Concept Pilot Study

Normothermic machine perfusion (NMP) allows for ex vivo viability and functional assessment prior to liver transplantation (LT). Hyperspectral imaging represents a suitable, non-invasive method to evaluate tissue morphology and organ perfusion during NMP. Liver allografts were subjected to NMP prior to LT. Serial image acquisition of oxygen saturation levels (StO2), organ hemoglobin (THI), near-infrared perfusion (NIR) and tissue water indices (TWI) through hyperspectral imaging was performed during static cold storage, at 1h, 6h, 12h and at the end of NMP. The readouts were correlated with perfusate parameters at equivalent time points. Twenty-one deceased donor livers were included in the study. Seven (33.0%) were discarded due to poor organ function during NMP. StO2 (p < 0.001), THI (p < 0.001) and NIR (p = 0.002) significantly augmented, from static cold storage (pre-NMP) to NMP end, while TWI dropped (p = 0.005) during the observational period. At 12-24h, a significantly higher hemoglobin concentration (THI) in the superficial tissue layers was seen in discarded, compared to transplanted livers (p = 0.036). Lactate values at 12h NMP correlated negatively with NIR perfusion index between 12 and 24h NMP and with the delta NIR perfusion index between 1 and 24h (rs = -0.883, p = 0.008 for both). Furthermore, NIR and TWI correlated with lactate clearance and pH. This study provides first evidence of feasibility of hyperspectral imaging as a potentially helpful contact-free organ viability assessment tool during liver NMP.

Hyperspectral Imaging for Assessment of Initial Graft Function in Human Kidney Transplantation

The aim of our study was to evaluate hyperspectral imaging (HSI) as a rapid, non-ionizing technique for the assessment of organ quality and the prediction of delayed graft function (DGF) in kidney transplantation after static cold storage (SCS, n = 20), as well as hypothermic machine perfusion (HMP, n = 18). HSI assessment of the kidney parenchyma was performed during organ preservation and at 10 and 30 min after reperfusion using the TIVITA^® Tissue System (Diaspective Vision GmbH, Am Salzhaff, Germany), calculating oxygen saturation (StO₂), near-infrared perfusion index (NIR), tissue haemoglobin index (THI), and tissue water index (TWI). Recipient and donor characteristics were comparable between organ preservation groups. Cold ischemic time was significantly longer in the HMP group (14.1 h [3.6–23.1] vs. 8.7h [2.2–17.0], p = 0.002). The overall presence of DGF was comparable between groups (HMP group n = 10 (55.6%), SCS group n = 10 (50.0%)). Prediction of DGF was possible in SCS and HMP kidneys; StO₂ at 10 (50.00 [17.75–76.25] vs. 63.17 [27.00–77.75]%, p = 0.0467) and 30 min (57.63 [18.25–78.25] vs. 65.38 [21.25–83.33]%, p = 0.0323) after reperfusion, as well as NIR at 10 (41.75 [1.0–58.00] vs. 48.63 [12.25–69.50], p = 0.0137) and 30 min (49.63 [8.50–66.75] vs. 55.80 [14.75–73.25], p = 0.0261) after reperfusion were significantly lower in DGF kidneys, independent of the organ preservation method. In conclusion, HSI is a reliable method for intraoperative assessment of renal microperfusion, applicable after organ preservation through SCS and HMP, and predicts the development of DGF.

Hyperspectral Wide-Field-Of-View Imaging to Study Dynamic Microcirculatory Changes During Hypoxia

OBJECTIVE

Hyperspectral imaging (HSI) provides a fast, reliable, and non-invasive way the study vascular microcirculation in animal models. Rapid hyperspectral imaging of large portions of the microcirculatory preparation is critical for understanding the function and regulation of vascular microcirculatory networks.

METHODS

This report presents the application of an off-the-shelf, benchtop, HSI linear scanning system to acquire larger field-of-view images of microcirculatory preparations. The HSI line detector was displaced perpendicular to the scanning direction to map larger areas, with a rate of displacement determined by the scanning rate and the exposure time. The collected image was analyzed to determine dynamic changes in the microcirculation.

RESULTS

The system records dynamic changes in microvascular hemoglobin (Hb) oxygen (O₂) saturation and vascular morphology during hypoxia and reoxygenation and has similar acquisition speeds to commonly referenced spectral-scanning HSI systems. Additionally, the HbO₂ saturations collected via HSI closely correlate with those collected by phosphorescence quenching microscopy.

CONCLUSION

The reported system enables dynamic functional microcirculation imaging for broad experimental and clinical applications.

3D printed auxetic heel pads for patients with diabetic mellitus

More than 422 million people worldwide suffered from diabetes mellitus (DM) in 2021. Diabetic foot is one the most critical complications resultant of DM. Foot ulceration and infection are frequently arisen, which are associated with changes in the mechanical properties of the plantar soft tissues, peripheral arterial disease, and sensory neuropathy. Diabetic insoles are currently the mainstay in reducing the risk of foot ulcers by reducing the magnitude of the pressure on the plantar Here, we propose a novel pressure relieving heel pad based on a circular auxetic re-entrant honeycomb structure by using three-dimensional (3D) printing technology to minimize the pressure on the heel, thus reducing the occurrence of foot ulcers. Finite element models (FEMs) are developed to evaluate the structural changes of the developed circular auxetic structure upon exertion of compressive forces. Moreover, the effects of the internal angle of the re-entrant structure on the peak contact force and the mean pressure acting on the heel as well as the contact area between the heel and the pads are investigated through a finite element analysis (FEA). Based on the result from the validated FEMs, the proposed heel pad with an auxetic structure demonstrates a distinct reduction in the peak contact force (∼10%) and the mean pressure (∼14%) in comparison to a conventional diabetic insole (PU foam). The characterized result of the designed circular auxetic structure not only provides new insights into diabetic foot protection, but also the design and development of various impact resistance products.

Discriminating healthy from tumor tissue in breast lumpectomy specimens using deep learning-based hyperspectral imaging

Achieving an adequate resection margin during breast-conserving surgery remains challenging due to the lack of intraoperative feedback. Here, we evaluated the use of hyperspectral imaging to discriminate healthy tissue from tumor tissue in lumpectomy specimens. We first used a dataset obtained on tissue slices to develop and evaluate three convolutional neural networks. Second, we fine-tuned the networks with lumpectomy data to predict the tissue percentages of the lumpectomy resection surface. A MCC of 0.92 was achieved on the tissue slices and an RMSE of 9% on the lumpectomy resection surface. This shows the potential of hyperspectral imaging to classify the resection margins of lumpectomy specimens.

Evaluation of hyperspectral imaging as a modern aid in clinical assessment of burn wounds of the upper extremity

The most common burn wound assessment continues to be the clinical inspection and the tactile examination, which are subjective and remain challenging even for experienced burn surgeons. Recently, hyperspectral imaging camera systems have been increasingly used to support the evaluation of burn wounds. The aim of our study was to determine if hyperspectral imaging analysis differentiates and objectifies the assessment of burn wounds in burns of the upper extremities. We included 97 superficial partial, deep partial dermal burns, and full thickness burns. Hyperspectral imaging analysis was performed for all burns using proprietary software. The software recorded parameters for tissue oxygenation (StO2), tissue hemoglobin index, and near-infrared perfusion. These values were compared with the recordings for healthy, non-burned skin. We found that hyperspectral imaging analysis effectively differentiates burn wounds and shows the ability to distinguish even superficial partial burns from deep partial burns in the near-infrared perfusion analysis feature. Although, it was not possible to differentiate burn wounds in all features. Currently, it is important to optimize the respective reference values of the individual burn degrees for an objectified assessment.

Intraoperative imaging in pathology-assisted surgery

The pathological assessment of surgical specimens during surgery can reduce the incidence of positive resection margins, which otherwise can result in additional surgeries or aggressive therapeutic regimens. To improve patient outcomes, intraoperative spectroscopic, fluorescence-based, structural, optoacoustic and radiological imaging techniques are being tested on freshly excised tissue. The specific clinical setting and tumour type largely determine whether endogenous or exogenous contrast is to be detected and whether the tumour specificity of the detected biomarker, image resolution, image-acquisition times or penetration depth are to be prioritized. In this Perspective, we describe current clinical standards for intraoperative tissue analysis and discuss how intraoperative imaging is being implemented. We also discuss potential implementations of intraoperative pathology-assisted surgery for clinical decision-making.

Time-lapse imaging using dual-color coded quantitative differential phase contrast microscopy

SIGNIFICANCE

Quantitative differential phase contrast (qDPC) microscopy enhances phase contrast by asymmetric illumination using partially coherent light and multiple intensity measurements. However, for live cell imaging, motion artifacts and image acquisition time are important issues. For live cell imaging, a large number of intensity measurements can limit the imaging quality and speed. The minimum number of intensity measurements in qDPC can greatly enhance performance for live imaging.

AIM

To obtain high-contrast, isotropic qDPC images with two intensity measurements and perform time-lapse imaging of biological samples.

APPROACH

Based on the color-coded design, a dual-color linear-gradient pupil is proposed to achieve isotropic phase contrast response with two intensity measurements. In our method, the purpose of designing a dual-color coded pupil is twofold: first, to obtain a linear amplitude gradient for asymmetric illumination, which is required to get a circular symmetry of transfer function, and second, to reduce the required number of frames for phase retrieval.

RESULTS

To demonstrate the imaging performance of our system, standard microlens arrays were used as samples. We performed time-lapse quantitative phase imaging of rat astrocytes under a low-oxygen environment. Detailed morphology and dynamic changes such as the apoptosis process and migration of cells were observed.

CONCLUSIONS

It is shown that dual-color linear-gradient pupils in qDPC can outperform half-circle and vortex pupils, and isotropic phase transfer function can be achieved with only two-axis measurements. The reduced number of frames helps in achieving faster imaging speed as compared to the typical qDPC system. The imaging performance of our system is evaluated by time-lapse imaging of rat astrocytes. Different morphological changes in cells during their life cycle were observed in terms of quantitative phase change values.

Combined indocyanine green and quantitative perfusion assessment with hyperspectral imaging during colorectal resections

Anastomotic insufficiencies still represent one of the most severe complications in colorectal surgery. Since tissue perfusion highly affects anastomotic healing, its objective assessment is an unmet clinical need. Indocyanine green-based fluorescence angiography (ICG-FA) and hyperspectral imaging (HSI) have received great interest in recent years but surgeons have to decide between both techniques. For the first time, two data processing pipelines capable of reconstructing an ICG-FA correlating signal from hyperspectral data were developed. Results were technically evaluated and compared to ground truth data obtained during colorectal resections. In 87% of 46 data sets, the reconstructed images resembled the ground truth data. The combined applicability of ICG-FA and HSI within one imaging system might provide supportive and complementary information about tissue vascularization, shorten surgery time, and reduce perioperative mortality.

Lasers in Live Cell Microscopy

Due to their unique properties—coherent radiation, diffraction limited focusing, low spectral bandwidth and in many cases short light pulses—lasers play an increasing role in live cell microscopy. Lasers are indispensable tools in 3D microscopy, e.g., confocal, light sheet or total internal reflection microscopy, as well as in super-resolution microscopy using wide-field or confocal methods. Further techniques, e.g., spectral imaging or fluorescence lifetime imaging (FLIM) often depend on the well-defined spectral or temporal properties of lasers. Furthermore, laser microbeams are used increasingly for optical tweezers or micromanipulation of cells. Three exemplary laser applications in live cell biology are outlined. They include fluorescence diagnosis, in particular in combination with Förster Resonance Energy Transfer (FRET), photodynamic therapy as well as laser-assisted optoporation, and demonstrate the potential of lasers in cell biology and—more generally—in biomedicine.

Optimized spectral filter design enables more accurate estimation of oxygen saturation in spectral imaging

Oxygen saturation (SO₂) in tissue is a crucially important physiological parameter with ubiquitous clinical utility in diagnosis, treatment, and monitoring, as well as widespread use as an invaluable preclinical research tool. Multispectral imaging can be used to visualize SO₂ non-invasively, non-destructively and without contact in real-time using narrow spectral filter sets, but typically, these spectral filter sets are poorly suited to a specific clinical task, application, or tissue type. In this work, we demonstrate the merit of optimizing spectral filter sets for more accurate estimation of SO₂. Using tissue modelling and simulated multispectral imaging, we demonstrate filter optimization reduces the root-mean-square-error (RMSE) in estimating SO₂ by up to 37% compared with evenly spaced filters. Moreover, we demonstrate up to a 79% decrease in RMSE for optimized filter sets compared with filter sets chosen to minimize mutual information. Wider adoption of this approach will result in more effective multispectral imaging systems that can address specific clinical needs and consequently, more widespread adoption of multispectral imaging technologies in disease diagnosis and treatment.

High-precision four-dimensional hyperspectral imager integrating fluorescence spectral detection and 3D surface shape measurement

A four-dimensional hyperspectral imager (FDHI) that combines fluorescence spectral detection and 3D surface morphology measurement is proposed. The FDHI consists of a hyperspectral line-scanner, a line structured light stereo vision system, and a line laser. The line laser is used as both the excitation light for the fluorescence and the scanning light line for the 3D profiling. At each scanning step, the system collects both fluorescent and 3D spatial data of the irradiated line region, which are fused to 4D data points based on a line mapping relationship between the datasets, and by scanning across the measurement object, a complete 4D dataset is obtained. The FDHI shows excellent performance with spatial and spectral resolution of 26.0 µm and 3 nm, respectively. The reported FDHI system and its applications provide a solution for 4D detection and analysis of fluorescent objects in meters measurement range, with advantage of high integration as two imaging modules sharing a same laser source.

Cancer detection from stained biopsies using high-speed spectral imaging

The escalating demand for diagnosing pathological biopsies requires the procedures to be expedited and automated. The existing imaging systems for measuring biopsies only measure color, and even though a lot of effort is invested in deep learning analysis, there are still serious challenges regarding the performance and validity of the data for the intended medical setting. We developed a system that rapidly acquires spectral images from biopsies, followed by spectral classification algorithms. The spectral information is remarkably more informative than the color information, and leads to very high accuracy in identifying cancer cells, as tested on tens of cancer cases. This was improved even more by using artificial intelligence algorithms that required a rather small training set, indicating the high level of information that exists in the spectral images. The most important spectral differences are observed in the nucleus and they are related to aneuploidy in tumor cells. Rapid spectral imaging measurement therefore can bridge the gap in the machine-aided diagnostics of whole biopsies, thus improving patient care, and expediting the treatment procedure.

Non-Local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration

Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) restoration, which includes the tasks of denoising, compressed HSI reconstruction and inpainting. Unfortunately, while its restoration performance benefits from more spectral bands, its runtime also substantially increases. In this paper, we claim that the HSI lies in a global spectral low-rank subspace, and the spectral subspaces of each full band patch group should lie in this global low-rank subspace. This motivates us to propose a unified paradigm combining the spatial and spectral properties for HSI restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity from the low-rank orthogonal basis exploration. An efficient alternating minimization algorithm with rank adaptation is developed. It is done by first solving a fidelity term-related problem for the update of a latent input image, and then learning a low-dimensional orthogonal basis and the related reduced image from the latent input image. Subsequently, non-local low-rank denoising is developed to refine the reduced image and orthogonal basis iteratively. Finally, the experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority with respect to state-of-the-art HSI restoration methods.

Optical biopsy mapping on endoscopic image mosaics with a marker-free probe

Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope.

Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours-A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks

Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. In this second part of a three-phase pilot study, we used a novel hand-held SICSURFIS Spectral Imager with an adaptable field of view and target-wise selectable wavelength channels to provide detailed spectral and spatial data for lesions on complex surfaces. The hyperspectral images (33 wavelengths, 477–891 nm) provided photometric data through individually controlled illumination modules, enabling convolutional networks to utilise spectral, spatial, and skin-surface models for the analyses. In total, 42 lesions were studied: 7 melanomas, 13 pigmented and 7 intradermal nevi, 10 basal cell carcinomas, and 5 squamous cell carcinomas. All lesions were excised for histological analyses. A pixel-wise analysis provided map-like images and classified pigmented lesions with a sensitivity of 87% and a specificity of 93%, and 79% and 91%, respectively, for non-pigmented lesions. A majority voting analysis, which provided the most probable lesion diagnosis, diagnosed 41 of 42 lesions correctly. This pilot study indicates that our non-invasive hyperspectral imaging system, which involves shape and depth data analysed by convolutional neural networks, is feasible for differentiating between malignant and benign pigmented and non-pigmented skin tumours, even on complex skin surfaces.