Intraoperative multispectral and hyperspectral label-free imaging: A systematic review of in vivo clinical studies

Advancing DIEP Flap Monitoring with Optical Imaging Techniques: A Narrative Review

OBJECTIVES

This review aims to explore recent advancements in optical imaging techniques for monitoring the viability of Deep Inferior Epigastric Perforator (DIEP) flap reconstruction. The objectives include highlighting the principles, applications, and clinical utility of optical imaging modalities such as near-infrared spectroscopy (NIRS), indocyanine green (ICG) fluorescence angiography, laser speckle contrast imaging (LSCI), hyperspectral imaging (HSI), dynamic infrared thermography (DIRT), and short-wave infrared thermography (SWIR) in assessing tissue perfusion and oxygenation. Additionally, this review aims to discuss the potential of these techniques in enhancing surgical outcomes by enabling timely intervention in cases of compromised flap perfusion.

MATERIALS AND METHODS

A comprehensive literature review was conducted to identify studies focusing on optical imaging techniques for monitoring DIEP flap viability. We searched PubMed, MEDLINE, and relevant databases, including Google Scholar, Web of Science, Scopus, PsycINFO, IEEE Xplore, and ProQuest Dissertations & Theses, among others, using specific keywords related to optical imaging, DIEP flap reconstruction, tissue perfusion, and surgical outcomes. This extensive search ensured we gathered comprehensive data for our analysis. Articles discussing the principles, applications, and clinical use of NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR in DIEP flap monitoring were selected for inclusion. Data regarding the techniques' effectiveness, advantages, limitations, and potential impact on surgical decision-making were extracted and synthesized.

RESULTS

Optical imaging modalities, including NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation in DIEP flap reconstruction. These techniques provide objective and quantitative data, enabling surgeons to monitor flap viability accurately. Studies have demonstrated the effectiveness of optical imaging in detecting compromised perfusion and facilitating timely intervention, thereby reducing the risk of flap complications such as partial or total loss. Furthermore, optical imaging modalities have shown promise in improving surgical outcomes by guiding intraoperative decision-making and optimizing patient care.

CONCLUSIONS

Recent advancements in optical imaging techniques present valuable tools for monitoring the viability of DIEP flap reconstruction. NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation, enabling accurate evaluation of flap viability. These modalities have the potential to enhance surgical outcomes by facilitating timely intervention in cases of compromised perfusion, thereby reducing the risk of flap complications. Incorporating optical imaging into clinical practice can provide surgeons with objective and quantitative data, assisting in informed decision-making for optimal patient care in DIEP flap reconstruction surgeries.

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Histological Hyperspectral Glioblastoma Dataset (HistologyHSI-GB)

Hyperspectral (HS) imaging (HSI) technology combines the main features of two existing technologies: imaging and spectroscopy. This allows to analyse simultaneously the morphological and chemical attributes of the objects captured by a HS camera. In recent years, the use of HSI provides valuable insights into the interaction between light and biological tissues, and makes it possible to detect patterns, cells, or biomarkers, thus, being able to identify diseases. This work presents the HistologyHSI-GB dataset, which contains 469 HS images from 13 patients diagnosed with brain tumours, specifically glioblastoma. The slides were stained with haematoxylin and eosin (H&E) and captured using a microscope at 20× power magnification. Skilled histopathologists diagnosed the slides and provided image-level annotations. The dataset was acquired using custom HSI instrumentation, consisting of a microscope equipped with an HS camera covering the spectral range from 400 to 1000 nm.

Optical Methods for Brain Tumor Detection: A Systematic Review

Background: In brain tumor surgery, maximal tumor resection is typically desired. This is complicated by infiltrative tumor cells which cannot be visually distinguished from healthy brain tissue. Optical methods are an emerging field that can potentially revolutionize brain tumor surgery through intraoperative differentiation between healthy and tumor tissues. Methods: This study aimed to systematically explore and summarize the existing literature on the use of Raman Spectroscopy (RS), Hyperspectral Imaging (HSI), Optical Coherence Tomography (OCT), and Diffuse Reflectance Spectroscopy (DRS) for brain tumor detection. MEDLINE, Embase, and Web of Science were searched for studies evaluating the accuracy of these systems for brain tumor detection. Outcome measures included accuracy, sensitivity, and specificity. Results: In total, 44 studies were included, covering a range of tumor types and technologies. Accuracy metrics in the studies ranged between 54 and 100% for RS, 69 and 99% for HSI, 82 and 99% for OCT, and 42 and 100% for DRS. Conclusions: This review provides insightful evidence on the use of optical methods in distinguishing tumor from healthy brain tissue.

Biophotonics-Intraoperative Guidance During Partial Nephrectomy: A Systematic Review and Meta-analysis

CONTEXT

Partial nephrectomy (PN) with intraoperative guidance by biophotonics has the potential to improve surgical outcomes due to higher precision. However, its value remains unclear since high-level evidence is lacking.

OBJECTIVE

To provide a comprehensive analysis of biophotonic techniques used for intraoperative real-time assistance during PN.

EVIDENCE ACQUISITION

We performed a comprehensive database search based on the PICO criteria, including studies published before October 2022. Two independent reviewers screened the titles and abstracts followed by full-text screening of eligible studies. For a quantitative analysis, a meta-analysis was conducted.

EVIDENCE SYNTHESIS

In total, 35 studies were identified for the qualitative analysis, including 27 studies on near-infrared fluorescence (NIRF) imaging using indocyanine green, four studies on hyperspectral imaging, two studies on folate-targeted molecular imaging, and one study each on optical coherence tomography and 5-aminolevulinic acid. The meta-analysis investigated seven studies on selective arterial clamping using NIRF. There was a significantly shorter warm ischemia time in the NIRF-PN group (mean difference [MD]: -2.9; 95% confidence interval [CI]: -5.6, -0.1; p = 0.04). No differences were noted regarding transfusions (odds ratio [OR]: 0.5; 95% CI: 0.2, 1.7; p = 0.27), positive surgical margins (OR: 0.7; 95% CI: 0.2, 2.0; p = 0.46), or major complications (OR: 0.4; 95% CI: 0.1, 1.2; p = 0.08). In the NIRF-PN group, functional results were favorable at short-term follow-up (MD of glomerular filtration rate decline: 7.6; 95% CI: 4.6, 10.5; p < 0.01), but leveled off at long-term follow-up (MD: 7.0; 95% CI: -2.8, 16.9; p = 0.16). Remarkably, these findings were not confirmed by the included randomized controlled trial.

CONCLUSIONS

Biophotonics comprises a heterogeneous group of imaging modalities that serve intraoperative decision-making and guidance. Implementation into clinical practice and cost effectiveness are the limitations that should be addressed by future research.

PATIENT SUMMARY

We reviewed the application of biophotonics during partial removal of the kidney in patients with kidney cancer. Our results suggest that these techniques support the surgeon in successfully performing the challenging steps of the procedure.

Rapid low-cost hyperspectral imaging system for quantitative assessment of infantile hemangioma

Hemangioma, the predominant benign tumor occurring in infancy, exhibits a wide range of prognoses and associated outcomes. The accurate determination of prognosis through noninvasive imaging modalities holds essential importance in enabling effective personalized treatment strategies and minimizing unnecessary surgical interventions for individual patients. The present study focuses on advancing the personalized prognosis of hemangioma by leveraging noninvasive optical sensing technologies by the development of a novel rapid hyperspectral sensor (image collection in 5 s, lateral resolution of 10 μm) that is capable of quantifying hemoglobin oxygenation and vascularization dynamics during the course of tumor evolution. We have developed a quantitative parameter for hemangioma assessment, that demonstrated agreement with the clinician's conclusion in 90% among all cases during clinical studies on six patients, who visited clinician from two to four times. The presented methodology has potential to be implemented as a supportive tool for accurate hemangioma diagnostics in clinics.

Multispectral indices for real-time and non-invasive tissue ischemia monitoring using snapshot cameras

An adequate supply of oxygen-rich blood is vital to maintain cell homeostasis, cellular metabolism, and overall tissue health. While classical methods of measuring tissue ischemia are often invasive, localized and require skin contact or contrast agents, spectral imaging shows promise as a non-invasive, wide field, and contrast-free approach. We evaluate three novel reflectance-based spectral indices from the 460 - 840 nm spectral range. With the aim of enabling real time visualization of tissue ischemia, information is extracted from only 2-3 spectral bands. Video-rate spectral data was acquired from arm occlusion experiments in 27 healthy volunteers. The performance of the indices was evaluated against binary Support Vector Machine (SVM) classification of healthy versus ischemic skin tissue, two other indices from literature, and tissue oxygenation estimated using spectral unmixing. Robustness was tested by evaluating these under various lighting conditions and on both the dorsal and palmar sides of the hand. A novel index with real-time capabilities using reflectance information only from 547 nm and 556 nm achieves an average classification accuracy of 88.48, compared to 92.65 using an SVM trained on all available wavelengths. Furthermore, the index has a higher accuracy compared to reference methods and its time dynamics compare well against the expected clinical responses. This holds promise for robust real-time detection of tissue ischemia, possibly contributing to improved patient care and clinical outcomes.

Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography

INTRODUCTION

Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA.

MATERIAL AND METHODS

The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works.

RESULTS

The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method.

CONCLUSIONS

Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.

Lightfield hyperspectral imaging in neuro-oncology surgery: an IDEAL 0 and 1 study

Introduction

Hyperspectral imaging (HSI) has shown promise in the field of intra-operative imaging and tissue differentiation as it carries the capability to provide real-time information invisible to the naked eye whilst remaining label free. Previous iterations of intra-operative HSI systems have shown limitations, either due to carrying a large footprint limiting ease of use within the confines of a neurosurgical theater environment, having a slow image acquisition time, or by compromising spatial/spectral resolution in favor of improvements to the surgical workflow. Lightfield hyperspectral imaging is a novel technique that has the potential to facilitate video rate image acquisition whilst maintaining a high spectral resolution. Our pre-clinical and first-in-human studies (IDEAL 0 and 1, respectively) demonstrate the necessary steps leading to the first in-vivo use of a real-time lightfield hyperspectral system in neuro-oncology surgery.

Methods

A lightfield hyperspectral camera (Cubert Ultris ×50) was integrated in a bespoke imaging system setup so that it could be safely adopted into the open neurosurgical workflow whilst maintaining sterility. Our system allowed the surgeon to capture in-vivo hyperspectral data (155 bands, 350-1,000 nm) at 1.5 Hz. Following successful implementation in a pre-clinical setup (IDEAL 0), our system was evaluated during brain tumor surgery in a single patient to remove a posterior fossa meningioma (IDEAL 1). Feedback from the theater team was analyzed and incorporated in a follow-up design aimed at implementing an IDEAL 2a study.

Results

Focusing on our IDEAL 1 study results, hyperspectral information was acquired from the cerebellum and associated meningioma with minimal disruption to the neurosurgical workflow. To the best of our knowledge, this is the first demonstration of HSI acquisition with 100+ spectral bands at a frame rate over 1Hz in surgery.

Discussion

This work demonstrated that a lightfield hyperspectral imaging system not only meets the design criteria and specifications outlined in an IDEAL-0 (pre-clinical) study, but also that it can translate into clinical practice as illustrated by a successful first in human study (IDEAL 1). This opens doors for further development and optimisation, given the increasing evidence that hyperspectral imaging can provide live, wide-field, and label-free intra-operative imaging and tissue differentiation.

Current and Future Advances in Surgical Therapy for Pituitary Adenoma

The vital physiological role of the pituitary gland, alongside its proximity to critical neurovascular structures, means that pituitary adenomas can cause significant morbidity or mortality. While enormous advancements have been made in the surgical care of pituitary adenomas, numerous challenges remain, such as treatment failure and recurrence. To meet these clinical challenges, there has been an enormous expansion of novel medical technologies (eg, endoscopy, advanced imaging, artificial intelligence). These innovations have the potential to benefit each step of the patient's journey, and ultimately, drive improved outcomes. Earlier and more accurate diagnosis addresses this in part. Analysis of novel patient data sets, such as automated facial analysis or natural language processing of medical records holds potential in achieving an earlier diagnosis. After diagnosis, treatment decision-making and planning will benefit from radiomics and multimodal machine learning models. Surgical safety and effectiveness will be transformed by smart simulation methods for trainees. Next-generation imaging techniques and augmented reality will enhance surgical planning and intraoperative navigation. Similarly, surgical abilities will be augmented by the future operative armamentarium, including advanced optical devices, smart instruments, and surgical robotics. Intraoperative support to surgical team members will benefit from a data science approach, utilizing machine learning analysis of operative videos to improve patient safety and orientate team members to a common workflow. Postoperatively, neural networks leveraging multimodal datasets will allow early detection of individuals at risk of complications and assist in the prediction of treatment failure, thus supporting patient-specific discharge and monitoring protocols. While these advancements in pituitary surgery hold promise to enhance the quality of care, clinicians must be the gatekeepers of the translation of such technologies, ensuring systematic assessment of risk and benefit prior to clinical implementation. In doing so, the synergy between these innovations can be leveraged to drive improved outcomes for patients of the future.

Intraoperative bowel perfusion assessment methods and their effects on anastomotic leak rates: meta-analysis

BACKGROUND

Anastomotic leak is one of the most feared complications of colorectal surgery, and probably linked to poor blood supply to the anastomotic site. Several technologies have been described for intraoperative assessment of bowel perfusion. This systematic review and meta-analysis aimed to evaluate the most frequently used bowel perfusion assessment modalities in elective colorectal procedures, and to assess their associated risk of anastomotic leak. Technologies included indocyanine green fluorescence angiography, diffuse reflectance spectroscopy, laser speckle contrast imaging, and hyperspectral imaging.

METHODS

The review was preregistered with PROSPERO (CRD42021297299). A comprehensive literature search was performed using Embase, MEDLINE, Cochrane Library, Scopus, and Web of Science. The final search was undertaken on 29 July 2022. Data were extracted by two reviewers and the MINORS criteria were applied to assess the risk of bias.

RESULTS

Some 66 eligible studies involving 11 560 participants were included. Indocyanine green fluorescence angiography was most used with 10 789 participants, followed by diffuse reflectance spectroscopy with 321, hyperspectral imaging with 265, and laser speckle contrast imaging with 185. In the meta-analysis, the total pooled effect of an intervention on anastomotic leak was 0.05 (95 per cent c.i. 0.04 to 0.07) in comparison with 0.10 (0.08 to 0.12) without. Use of indocyanine green fluorescence angiography, hyperspectral imaging, or laser speckle contrast imaging was associated with a significant reduction in anastomotic leak.

CONCLUSION

Bowel perfusion assessment reduced the incidence of anastomotic leak, with intraoperative indocyanine green fluorescence angiography, hyperspectral imaging, and laser speckle contrast imaging all demonstrating comparable results.

Synthetic white balancing for intra-operative hyperspectral imaging

Purpose

Hyperspectral imaging shows promise for surgical applications to non-invasively provide spatially resolved, spectral information. For calibration purposes, a white reference image of a highly reflective Lambertian surface should be obtained under the same imaging conditions. Standard white references are not sterilizable and so are unsuitable for surgical environments. We demonstrate the necessity for in situ white references and address this by proposing a novel, sterile, synthetic reference construction algorithm.

Approach

The use of references obtained at different distances and lighting conditions to the subject were examined. Spectral and color reconstructions were compared with standard measurements qualitatively and quantitatively, using Δ E and normalized RMSE, respectively. The algorithm forms a composite image from a video of a standard sterile ruler, whose imperfect reflectivity is compensated for. The reference is modeled as the product of independent spatial and spectral components, and a scalar factor accounting for gain, exposure, and light intensity. Evaluation of synthetic references against ideal but non-sterile references is performed using the same metrics alongside pixel-by-pixel errors. Finally, intraoperative integration is assessed though cadaveric experiments.

Results

Improper white balancing leads to increases in all quantitative and qualitative errors. Synthetic references achieve median pixel-by-pixel errors lower than 6.5% and produce similar reconstructions and errors to an ideal reference. The algorithm integrated well into surgical workflow, achieving median pixel-by-pixel errors of 4.77% while maintaining good spectral and color reconstruction.

Conclusions

We demonstrate the importance of in situ white referencing and present a novel synthetic referencing algorithm. This algorithm is suitable for surgery while maintaining the quality of classical data reconstruction.

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: application to surgical imaging

PURPOSE

Hyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes.

METHODS

We regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking.

RESULTS

Quantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis.

CONCLUSION

Our results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Effects of dimension reduction of hyperspectral images in skin gross pathology

BACKGROUND

Hyperspectral imaging (HSI) is an emerging modality for the gross pathology of the skin. Spectral signatures of HSI could discriminate malignant from benign tissue. Because of inherent redundancies in HSI and in order to facilitate the use of deep-learning models, dimension reduction is a common preprocessing step. The effects of dimension reduction choice, training scope, and number of retained dimensions have not been evaluated on skin HSI for segmentation tasks.

MATERIALS AND METHODS

An in-house dataset of HSI signatures from pigmented skin lesions was prepared and labeled with histology. Eleven different dimension reduction methods were used as preprocessing for tumor margin detection with support vector machines. Cluster-wise principal component analysis (ClusterPCA), a new variant of PCA, was proposed. The scope of application for dimension reduction was also investigated.

RESULTS

The components produced by ClusterPCA show good agreement with the expected optical properties of skin chromophores. Random forest importance performed best during classification. However, all methods suffered from low sensitivity and generalization.

CONCLUSION

Investigation of more complex reduction and segmentation schemes with emphasis on the nature of HSI and optical properties of the skin is necessary. Insights on dimension reduction for skin tissue could facilitate the development of HSI-based systems for cancer margin detection at gross level.

Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging

SIGNIFICANCE

Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems.

AIM

Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI.

APPROACH

MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods.

RESULTS

Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy.

CONCLUSIONS

The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.

Hyperspectral imaging for tumor segmentation on pigmented skin lesions

SIGNIFICANCE

Malignant skin tumors, which include melanoma and nonmelanoma skin cancers, are the most prevalent type of malignant tumor. Gross pathology of pigmented skin lesions (PSL) remains manual, time-consuming, and heavily dependent on the expertise of the medical personnel. Hyperspectral imaging (HSI) can assist in the detection of tumors and evaluate the status of tumor margins by their spectral signatures.

AIM

Tumor segmentation of medical HSI data is a research field. The goal of this study is to propose a framework for HSI-based tumor segmentation of PSL.

APPROACH

An HSI dataset of 28 PSL was prepared. Two frameworks for data preprocessing and tumor segmentation were proposed. Models based on machine learning and deep learning were used at the core of each framework.

RESULTS

Cross-validation performance showed that pixel-wise processing achieves higher segmentation performance, in terms of the Jaccard coefficient. Simultaneous use of spatio-spectral features produced more comprehensive tumor masks. A three-dimensional Xception-based network achieved performance similar to state-of-the-art networks while allowing for more detailed detection of the tumor border.

CONCLUSIONS

Good performance was achieved for melanocytic lesions, but margins were difficult to detect in some cases of basal cell carcinoma. The frameworks proposed in this study could be further improved for robustness against different pathologies and detailed delineation of tissue margins to facilitate computer-assisted diagnosis during gross pathology.

Comparison of image registration methods for combining laparoscopic video and spectral image data

Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy.

Protease-activated indocyanine green nanoprobes for intraoperative NIR fluorescence imaging of primary tumors

Tumor-targeted fluorescent probes in the near-infrared spectrum can provide invaluable information about the location and extent of primary and metastatic tumors during intraoperative procedures to ensure no residual tumors are left in the patient's body. Even though the first fluorescence-guided surgery was performed more than 50 years ago, it is still not accepted as a standard of care in part due to the lack of efficient and non-toxic targeted probes approved by regulatory agencies around the world. Herein, we report protease-activated cationic gelatin nanoparticles encapsulating indocyanine green (ICG) for the detection of primary breast tumors in murine models with high tumor-to-background ratios. Upon intravenous administration, these nanoprobes remain optically silent due to the energy resonance transfer among the bound ICG molecules. As the nanoprobes extravasate and are exposed to the acidic tumor microenvironment, their positive surface charges increase, facilitating cellular uptake. The internalized nanoprobes are activated upon proteolytic degradation of gelatin to allow high contrast between the tumor and normal tissue. Since both gelatin and ICG are FDA-approved for intravenous administration, this activatable nanoprobe can lead to quick clinical adoption and improve the treatment of patients undergoing image-guided cancer surgery.

Intraoperative tissue classification methods in orthopedic and neurological surgeries: A systematic review

Accurate tissue differentiation during orthopedic and neurological surgeries is critical, given that such surgeries involve operations on or in the vicinity of vital neurovascular structures and erroneous surgical maneuvers can lead to surgical complications. By now, the number of emerging technologies tackling the problem of intraoperative tissue classification methods is increasing. Therefore, this systematic review paper intends to give a general overview of existing technologies. The review was done based on the PRISMA principle and two databases: PubMed and IEEE Xplore. The screening process resulted in 60 full-text papers. The general characteristics of the methodology from extracted papers included data processing pipeline, machine learning methods if applicable, types of tissues that can be identified with them, phantom used to conduct the experiment, and evaluation results. This paper can be useful in identifying the problems in the current status of the state-of-the-art intraoperative tissue classification methods and designing new enhanced techniques.

[Artificial intelligence and hyperspectral imaging for image-guided assistance in minimally invasive surgery]

BACKGROUND

Intraoperative imaging assists surgeons during minimally invasive procedures. Hyperspectral imaging (HSI) is a noninvasive and noncontact optical technique with great diagnostic potential in medicine. The combination with artificial intelligence (AI) approaches to analyze HSI data is called intelligent HSI in this article.

OBJECTIVE

What are the medical applications and advantages of intelligent HSI for minimally invasive visceral surgery?

MATERIAL AND METHODS

Within various clinical studies HSI data from multiple in vivo tissue types and oncological resections were acquired using an HSI camera system. Different AI algorithms were evaluated for detection and discrimination of organs, risk structures and tumors.

RESULTS

In an experimental animal study 20 different organs could be differentiated with high precision (> 95%) using AI. In vivo, the parathyroid glands could be discriminated from surrounding tissue with an F1 score of 47% and sensitivity of 75%, and the bile duct with an F1 score of 79% and sensitivity of 90%. Furthermore, ex vivo tumor tissue could be successfully detected with an area under the receiver operating characteristic (ROC) curve (AUC) larger than 0.91.

DISCUSSION

This study demonstrates that intelligent HSI can automatically and accurately detect different tissue types. Despite great progress in the last decade intelligent HSI still has limitations. Thus, accurate AI algorithms that are easier to understand for the user and an extensive standardized and continuously growing database are needed. Further clinical studies should support the various medical applications and lead to the adoption of intelligent HSI in the clinical routine practice.

Hyperspectral and multispectral image processing for gross-level tumor detection in skin lesions: a systematic review

SIGNIFICANCE

Skin cancer is one of the most prevalent cancers worldwide. In the advent of medical digitization and telepathology, hyper/multispectral imaging (HMSI) allows for noninvasive, nonionizing tissue evaluation at a macroscopic level.

AIM

We aim to summarize proposed frameworks and recent trends in HMSI-based classification and segmentation of gross-level skin tissue.

APPROACH

A systematic review was performed, targeting HMSI-based systems for the classification and segmentation of skin lesions during gross pathology, including melanoma, pigmented lesions, and bruises. The review adhered to the 2020 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. For eligible reports published from 2010 to 2020, trends in HMSI acquisition, preprocessing, and analysis were identified.

RESULTS

HMSI-based frameworks for skin tissue classification and segmentation vary greatly. Most reports implemented simple image processing or machine learning, due to small training datasets. Methodologies were evaluated on heavily curated datasets, with the majority targeting melanoma detection. The choice of preprocessing scheme influenced the performance of the system. Some form of dimension reduction is commonly applied to avoid redundancies that are inherent in HMSI systems.

CONCLUSIONS

To use HMSI for tumor margin detection in practice, the focus of system evaluation should shift toward the explainability and robustness of the decision-making process.

Optical properties of human brain and tumour tissue: An ex vivo study spanning the visible range to beyond the second near-infrared window

Neuro-oncology surgery would benefit from detailed intraoperative tissue characterization provided by noncontact, contrast-agent-free, noninvasive optical imaging methods. In-depth knowledge of target tissue optical properties across a wide-wavelength spectrum could inform the design of optical imaging and computational methods to enable robust tissue analysis during surgery. We adapted a dual-beam integrating sphere to analyse small tissue samples and investigated ex vivo optical properties of five types of human brain tumour (meningioma, pituitary adenoma, schwannoma, low- and high-grade glioma) and nine different types of healthy brain tissue across a wavelength spectrum of 400 to 1800 nm. Fresh and frozen tissue samples were analysed. All tissue types demonstrated similar absorption spectra, but the reduced scattering coefficients of tumours show visible differences in the obtained optical spectrum compared to those of surrounding normal tissue. These results underline the potential of optical imaging technologies for intraoperative tissue characterization.

Optimized spectral filter design enables more accurate estimation of oxygen saturation in spectral imaging

Oxygen saturation (SO₂) in tissue is a crucially important physiological parameter with ubiquitous clinical utility in diagnosis, treatment, and monitoring, as well as widespread use as an invaluable preclinical research tool. Multispectral imaging can be used to visualize SO₂ non-invasively, non-destructively and without contact in real-time using narrow spectral filter sets, but typically, these spectral filter sets are poorly suited to a specific clinical task, application, or tissue type. In this work, we demonstrate the merit of optimizing spectral filter sets for more accurate estimation of SO₂. Using tissue modelling and simulated multispectral imaging, we demonstrate filter optimization reduces the root-mean-square-error (RMSE) in estimating SO₂ by up to 37% compared with evenly spaced filters. Moreover, we demonstrate up to a 79% decrease in RMSE for optimized filter sets compared with filter sets chosen to minimize mutual information. Wider adoption of this approach will result in more effective multispectral imaging systems that can address specific clinical needs and consequently, more widespread adoption of multispectral imaging technologies in disease diagnosis and treatment.

Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours-A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks

Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. In this second part of a three-phase pilot study, we used a novel hand-held SICSURFIS Spectral Imager with an adaptable field of view and target-wise selectable wavelength channels to provide detailed spectral and spatial data for lesions on complex surfaces. The hyperspectral images (33 wavelengths, 477–891 nm) provided photometric data through individually controlled illumination modules, enabling convolutional networks to utilise spectral, spatial, and skin-surface models for the analyses. In total, 42 lesions were studied: 7 melanomas, 13 pigmented and 7 intradermal nevi, 10 basal cell carcinomas, and 5 squamous cell carcinomas. All lesions were excised for histological analyses. A pixel-wise analysis provided map-like images and classified pigmented lesions with a sensitivity of 87% and a specificity of 93%, and 79% and 91%, respectively, for non-pigmented lesions. A majority voting analysis, which provided the most probable lesion diagnosis, diagnosed 41 of 42 lesions correctly. This pilot study indicates that our non-invasive hyperspectral imaging system, which involves shape and depth data analysed by convolutional neural networks, is feasible for differentiating between malignant and benign pigmented and non-pigmented skin tumours, even on complex skin surfaces.

Inspection of Line Defects in Transition Metal Dichalcogenides Using a Microscopic Hyperspectral Imaging Technique

The line defects of two-dimensional (2D) transition metal dichalcogenides (TMDs) play a vital role in determining their device performance. In this work, a microscopic hyperspectral imaging technique based on differential reflectance was introduced for the online inspection of line defects in TMDs. Upon comparison of the measurement results of imaging and spectra, the relationship between optical contrast and differential reflectance spectra was established. A light selection method was proposed to optimize the optical contrast of line defects. Via application of an image processing algorithm, an automatic detection of the line defects with a classification accuracy of 95% was achieved for WS₂, MoS₂, and MoSe₂. This work not only provides a microscopic hyperspectral imaging technique for detecting 2D material defects but also introduces a versatile design strategy for developing an advanced machine vision spectroscopic system.

Intraoperative reperfusion assessment of human pancreas allografts using hyperspectral imaging (HSI)

Background

The most common causes of early graft loss in pancreas transplantation are insufficient blood supply and leakage of the intestinal anastomosis. Therefore, it is critical to monitor graft perfusion and oxygenation during the early post-transplant period. The goal of our pilot study was to evaluate the utility of hyperspectral imaging (HSI) in monitoring the microcirculation of the graft and adequate perfusion of the intestinal anastomosis during pancreatic allotransplantation.

Methods

We imaged pancreatic grafts and intestinal anastomosis in real-time in three consecutive, simultaneous pancreas-kidney transplantations using the TIVITA^® HSI system. Further, the intraoperative oxygen saturation (StO₂), tissue perfusion (near-infrared perfusion index, NIR), organ hemoglobin index (OHI), and tissue water index (TWI) were measured 15 minutes after reperfusion by HSI.

Results

All pancreas grafts showed a high and homogeneous StO₂ (92.6%±10.45%). Intraoperative HSI analysis of the intestinal anastomosis displayed significant differences of StO₂ (graft duodenum 67.46%±5.60% vs. recipient jejunum: 75.93%±4.71%, P<0.001) and TWI {graft duodenum: 0.63±0.09 [I (Index)] vs. recipient jejunum: 0.72±0.09 [I], P<0.001}. NIR and OHI did not display remarkable differences {NIR duodenum: 0.68±0.06 [I] vs. NIR jejunum: 0.69±0.04 [I], P=0.747; OHI duodenum: 0.70±0.12 [I] vs. OHI jejunum: 0.68±0.13 [I], P=0.449}. All 3 patients had an uneventful postoperative course with one displaying a Banff 1a rejection which was responsive to steroid treatment.

Conclusions

Our study shows that contact-free HSI has potential utility as a novel tool for real-time monitoring of human pancreatic grafts after reperfusion, which could improve the outcome of pancreas transplantation. Further investigations are required to determine the predictive value of intraoperative HSI imaging.

LED-based Hyperspectral Endoscopic Imaging

Hyperspectral endoscopy can offer multiple advantages as compared to conventional endoscopy. Our goal is to design and develop a real-time hyperspectral endoscopic imaging system for the diagnosis of gastrointestinal (GI) tract cancers using a micro-LED array as an in-situ illumination source. The wavelengths of the system range from ultraviolet to visible and near infrared. To evaluate the use of the LED array for hyperspectral imaging, we designed a prototype system and conducted ex vivo experiments using normal and cancerous tissues of mice, chicken, and sheep. We compared the results of our LED-based approach with our reference hyperspectral camera system. The results confirm the similarity between the LED-based hyperspectral imaging system and the reference HSI camera. Our LED-based hyperspectral imaging system can be used not only as an endoscope but also as a laparoscopic or handheld devices for cancer detection and surgery.

Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

Hyperspectral imaging is one of the most promising techniques for intraoperative tissue characterisation. Snapshot mosaic cameras, which can capture hyperspectral data in a single exposure, have the potential to make a real-time hyperspectral imaging system for surgical decision-making possible. However, optimal exploitation of the captured data requires solving an ill-posed demosaicking problem and applying additional spectral corrections. In this work, we propose a supervised learning-based image demosaicking algorithm for snapshot hyperspectral images. Due to the lack of publicly available medical images acquired with snapshot mosaic cameras, a synthetic image generation approach is proposed to simulate snapshot images from existing medical image datasets captured by high-resolution, but slow, hyperspectral imaging devices. Image reconstruction is achieved using convolutional neural networks for hyperspectral image super-resolution, followed by spectral correction using a sensor-specific calibration matrix. The results are evaluated both quantitatively and qualitatively, showing clear improvements in image quality compared to a baseline demosaicking method using linear interpolation. Moreover, the fast processing time of 45 ms of our algorithm to obtain super-resolved RGB or oxygenation saturation maps per image for a state-of-the-art snapshot mosaic camera demonstrates the potential for its seamless integration into real-time surgical hyperspectral imaging applications.

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

Hyperspectral imaging (HSI) is a novel optical imaging modality, which has recently found diverse applications in the medical field. HSI is a hybrid imaging modality, combining a digital photographic camera with a spectrographic unit, and it allows for a contactless and non-destructive biochemical analysis of living tissue. HSI provides quantitative and qualitative information of the tissue composition at molecular level in a contrast-free manner, hence making it possible to objectively discriminate between different tissue types and between healthy and pathological tissue. Over the last two decades, HSI has been increasingly used in the medical field, and only recently it has found an application in the operating room. In the last few years, several research groups have used this imaging modality as an intraoperative guidance tool within different surgical disciplines. Despite its great potential, HSI still remains far from being routinely used in the daily surgical practice, since it is still largely unknown to most of the surgical community. The aim of this study is to provide clinical surgeons with an overview of the capabilities, current limitations, and future directions of HSI for intraoperative guidance.

Using physiological parameters measured by hyperspectral imaging to detect colorectal cancer

The accurate detection of malignant tissue during colorectal surgery impacts operation outcome. The non-invasive spectral imaging combined with machine learning (ML) methods showed to be promising for tumor identification. However, large spectral range implies large computing time. To reduce the number of features, ML methods (e.g. logistic regression and convolutional neuronal network CNN) were evaluated based on four physiological tissue parameters to automatically classify cancer and healthy mucosa in resected colon tissue. A ROC AUC of 0.81 was achieved with the CNN. This study shows that the use of only specific wavelengths bands can detect cancer.Clinical Relevance- These outcomes support the possibility to automatically classify colon tumor based on physiological parameters calculated using only specific wavelength bands. Hence, future image-guided colorectal surgeries can be performed with real-time multispectral imaging.

First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia

SIGNIFICANCE

The early detection of dysplasia in patients with Barrett's esophagus could improve outcomes by enabling curative intervention; however, dysplasia is often inconspicuous using conventional white-light endoscopy.

AIM

We sought to determine whether multispectral imaging (MSI) could be applied in endoscopy to improve detection of dysplasia in the upper gastrointestinal (GI) tract.

APPROACH

We used a commercial fiberscope to relay imaging data from within the upper GI tract to a snapshot MSI camera capable of collecting data from nine spectral bands. The system was deployed in a pilot clinical study of 20 patients (ClinicalTrials.gov NCT03388047) to capture 727 in vivo image cubes matched with gold-standard diagnosis from histopathology. We compared the performance of seven learning-based methods for data classification, including linear discriminant analysis, k-nearest neighbor classification, and a neural network.

RESULTS

Validation of our approach using a Macbeth color chart achieved an image-based classification accuracy of 96.5%. Although our patient cohort showed significant intra- and interpatient variance, we were able to resolve disease-specific contributions to the recorded MSI data. In classification, a combined principal component analysis and k-nearest-neighbor approach performed best, achieving accuracies of 95.8%, 90.7%, and 76.1%, respectively, for squamous, non-dysplastic Barrett's esophagus and neoplasia based on majority decisions per-image.

CONCLUSIONS

MSI shows promise for disease classification in Barrett's esophagus and merits further investigation as a tool in high-definition "chip-on-tip" endoscopes.

Hyperspectral Imaging and Machine Perfusion in Solid Organ Transplantation: Clinical Potentials of Combining Two Novel Technologies

Organ transplantation survival rates have continued to improve over the last decades, mostly due to reduction of mortality early after transplantation. The advancement of the field is facilitating a liberalization of the access to organ transplantation with more patients with higher risk profile being added to the waiting list. At the same time, the persisting organ shortage fosters strategies to rescue organs of marginal donors. In this regard, hypothermic and normothermic machine perfusion are recognized as one of the most important developments in the modern era. Owing to these developments, novel non-invasive tools for the assessment of organ quality are on the horizon. Hyperspectral imaging represents a potentially suitable method capable of evaluating tissue morphology and organ perfusion prior to transplantation. Considering the changing environment, we here discuss the hypothetical combination of organ machine perfusion and hyperspectral imaging as a prospective feasibility concept in organ transplantation.

Reflectance spectra analysis for mucous assessment

This review report represents an overview of research and development on medical hyperspectral imaging technology and its applications. Spectral imaging technology is attracting attention as a new imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. Considering the recent advances in imaging, this technology provides an opportunity for two-dimensional mapping of oxygen saturation (SatO₂) of blood with high accuracy, spatial spectral imaging, and its analysis and provides detection and diagnostic information about the tissue physiology and morphology. Multispectral imaging also provides information about tissue oxygenation, perfusion, and potential function during surgery. Analytical algorithm has been examined, and indication of accurate map of relative hemoglobin concentration and SatO₂ can be indicated with preferable resolution and frame rate. This technology is expected to provide promising biomedical information in practical use. Several studies suggested that blood flow and SatO₂ are associated with gastrointestinal disorders, particularly malignant tumor conditions. The use and analysis of spectroscopic images are expected to potentially play a role in the detection and diagnosis of these diseases.

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Bedside hyperspectral imaging indicates a microcirculatory sepsis pattern - an observational study

INTRODUCTION

Microcirculatory alterations are key mechanisms in sepsis pathophysiology leading to tissue hypoxia, edema formation, and organ dysfunction. Hyperspectral imaging (HSI) is an emerging imaging technology that uses tissue-light interactions to evaluate biochemical tissue characteristics including tissue oxygenation, hemoglobin content and water content. Currently, clinical data for HSI technologies in critical ill patients are still limited.

METHODS AND ANALYSIS

TIVITA® Tissue System was used to measure Tissue oxygenation (StO2), Tissue Hemoglobin Index (THI), Near Infrared Perfusion Index (NPI) and Tissue Water Index (TWI) in 25 healthy volunteers and 25 septic patients. HSI measurement sites were the palm, the fingertip, and a suprapatellar knee area. Septic patients were evaluated on admission to the ICU (E), 6 h afterwards (E+6) and three times a day (t3-t9) within a total observation period of 72 h. Primary outcome was the correlation of HSI results with daily SOFA-scores.

RESULTS

Serial HSI at the three measurement sites in healthy volunteers showed a low mean variance expressing high retest reliability. HSI at E demonstrated significantly lower StO2 and NPI as well as higher TWI at the palm and fingertip in septic patients compared to healthy volunteers. StO2 and TWI showed corresponding results at the suprapatellar knee area. In septic patients, palm and fingertip THI identified survivors (E-t4) and revealed predictivity for 28-day mortality (E). Fingertip StO2 and THI correlated to SOFA-score on day 2. TWI was consistently increased in relation to the TWI range of healthy controls during the observation time. Palm TWI correlated positively with SOFA scores on day 3.

DISCUSSION

HSI results in septic patients point to a distinctive microcirculatory pattern indicative of reduced skin oxygenation and perfusion quality combined with increased blood pooling and tissue water content. THI might possess risk-stratification properties and TWI could allow tissue edema evaluation in critically ill patients.

CONCLUSION

HSI technologies could open new perspectives in microcirculatory monitoring by visualizing oxygenation and perfusion quality combined with tissue water content in critically ill patients - a prerequisite for future tissue perfusion guided therapy concepts in intensive care medicine.

Development of a real-time spectral imaging system using in-site micro-LED-based illumination and high-speed micro-camera for endoscopic applications

We designed a compact, real-time LED-based endoscopic imaging system for the detection of various diseases including cancer. In gastrointestinal applications, conventional endoscopy cannot reliably differentiate tumor from normal tissue. Current hyperspectral imaging systems are too slow to be used for real-time endoscopic applications. We are investigating real-time spectral imaging for different tissue types. Our objective is to develop a catheter for real-time hyperspectral gastrointestinal endoscopy. The endoscope uses multiple wavelengths within UV, visible, and IR light spectra generated by a micro-LED array. We capture images with a monochrome micro camera, which is cost-effective and smaller than the current hyperspectral imagers. A wireless transceiver sends the captured images to a workstation for further processing, such as tumor detection. The spatial resolution of the system is defined by camera resolution and the distance to the object, while the number of LEDs in the multi-wavelength light source determines the spectral resolution. To investigate the properties and the limitations of our high-speed spectral imaging approach, we designed a prototype system. We conducted two experiments to measure the optimal forward voltages and lighting duration of the LEDs. These factors affect the maximum feasible imaging rate and resolution. The lighting duration of each LED can be shorter than 10 ms while producing an image with a high signal-to-noise ratio and no illumination interference. These results support the idea of using a high-speed camera and an LED-array for real-time hyperspectral endoscopic imaging.

[Integrity of swallowing apparatus-past, present, and future]

BACKGROUND

Swallowing is one of the most complex movement patterns. The integrity of the epithelial lining is essential.

OBJECTIVE

Which surgical techniques were developed at the beginnings of modern surgery and what methods are now available to maintain/reconstitute the integrity of the swallowing apparatus?

MATERIALS AND METHODS

This study comprises a selective literature search in early operation manuals and online archives, with incorporation of the authors' own experience.

RESULTS

Up until the 1950s, only very limited techniques were available to reconstruct the epithelial lining. Microvascular reanastomosed grafts were the game changer for reconstructive surgery, enabling reconstitution of the swallowing apparatus in primary surgery but also in challenging secondary interventions after insufficient or complicated primary therapy.

CONCLUSION

The need for anatomical and functional rehabilitation by reconstructive surgery is as pertinent as ever. Particularly in the oncological context, improved early detection and novel local treatment modalities could minimize treatment-associated damage to swallowing.

Hyperspectral characterization of re-epithelialization in an in vitro wound model

In vitro wound models are useful for research on wound re-epithelialization. Hyperspectral imaging represents a non-destructive alternative to histology analysis for detection of re-epithelialization. This study aims to characterize the main optical behavior of a wound model in order to enable development of detection algorithms. K-Means clustering and agglomerative analysis were used to group spatial regions based on the spectral behavior, and an inverse photon transport model was used to explain differences in optical properties. Six samples of the wound model were prepared from human tissue and followed over 22 days. Re-epithelialization occurred at a mean rate of 0.24 mm² /day after day 8 to 10. Suppression of wound spectral features was the main feature characterizing re-epithelialized and intact tissue. Modeling the photon transport through a diffuse layer placed on top of wound tissue properties reproduced the spectral behavior. The missing top layer represented by wounds is thus optically detectable using hyperspectral imaging.

Exploiting scale-invariance: a top layer targeted inverse model for hyperspectral images of wounds

Detection of re-epithelialization in wound healing is important, but challenging. Hyperspectral imaging can be used for non-destructive characterization, but efficient techniques are needed to extract and interpret the information. An inverse photon transport model suitable for characterization of re-epithelialization is validated and explored in this study. It exploits scale-invariance to enable fitting of the epidermal skin layer only. Monte Carlo simulations indicate that the fitted layer transmittance and reflectance spectra are unique, and that there exists an infinite number of coupled parameter solutions. The method is used to explain the optical behavior of and detect re-epithelialization in an in vitro wound model.

Multispectral Imaging for Automated Tissue Identification of Normal Human Surgical Specimens

OBJECTIVE

Safe surgery requires the accurate discrimination of tissue intraoperatively. We assess the feasibility of using multispectral imaging and deep learning to enhance surgical vision by automated identification of normal human head and neck tissues.

STUDY DESIGN

Construction and feasibility testing of novel multispectral imaging system for surgery.

SETTING

Academic university hospital.

SUBJECTS AND METHODS

Multispectral images of fresh-preserved human cadaveric tissues were captured with our adapted digital operating microscope. Eleven tissue types were sampled, each sequentially exposed to 6 lighting conditions. Two convolutional neural network machine learning models were developed to classify tissues based on multispectral and white-light color images (ARRInet-M and ARRInet-W, respectively). Blinded otolaryngology residents were asked to identify tissue specimens from white-light color images, and their performance was compared with that of the ARRInet models.

RESULTS

A novel multispectral imaging system was developed with minimal adaptation to an existing digital operating microscope. With 81.8% accuracy in tissue identification of full-size images, the multispectral ARRInet-M classifier outperformed the white-light-only ARRInet-W model (45.5%) and surgical residents (69.7%). Challenges with discrimination occurred with parotid vs fat and blood vessels vs nerve.

CONCLUSIONS

A deep learning model using multispectral imaging outperformed a similar model and surgical residents using traditional white-light imaging at the task of classifying normal human head and neck tissue ex vivo. These results suggest that multispectral imaging can enhance surgical vision and augment surgeons' ability to identify tissues during a procedure.

[Hyperspectral imaging for postoperative flap monitoring of pedicled flaps]

BACKGROUND

Since pedicle flaps were first described by the Indian physician Sushruta Samhita in the 6th century B. C., they have become an integral part of reconstructive surgery. As more and more research has been conducted into the underlying physical principles, flap monitoring has developed steadily in the last few decades. Hyperspectral Imaging (HSI) is a new quantitative measuring method for assessing the perfusion of the underlying tissue.

OBJECTIVE

This study aims to evaluate HSI as a monitoring method for pedicle flaps.

PATIENTS AND METHODS

In 16 patients who had undergone reconstructive surgery, oxygen saturation, haemoglobin and water concentration of the locoregional flap, necrotic flap areas as well as intact skin were measured on postoperative days 1 to 7. Subsequently, the data were statistically described and graphically illustrated.

RESULTS

HSI revealed increased haemoglobin concentration and decreased oxygen and water concentration in necrotic flap areas compared with the monitor island and healthy skin. The distribution of the values collected from the vital skin areas and the monitor island was almost identical.

CONCLUSION

HSI allows for safe, immediate, non-contact measurement of tissue perfusion of transferred tissue areas in patients after pedicle flap surgery. The use of HSI may improve postoperative flap monitoring.

Hyperspectral Imaging and the Retina: Worth the Wave?

Purpose

Hyperspectral imaging is gaining attention in the biomedical field because it generates additional spectral information to study physiological and clinical processes. Several technologies have been described; however an independent, systematic literature overview is lacking, especially in the field of ophthalmology. This investigation is the first to systematically overview scientific literature specifically regarding retinal hyperspectral imaging.

Methods

A systematic literature review was conducted, in accordance with PRISMA Statement 2009 criteria, in four bibliographic databases: Medline, Embase, Cochrane Database of Systematic Reviews, and Web of Science.

Results

Fifty-six articles were found that meet the review criteria. A range of techniques was reported: Fourier analysis, liquid crystal tunable filters, tunable laser sources, dual-slit monochromators, dispersive prisms and gratings, computed tomography, fiber optics, and Fabry-Perrot cavity filter covered complementary metal oxide semiconductor. We present a narrative synthesis and summary tables of findings of the included articles, because methodologic heterogeneity and diverse research topics prevented a meta-analysis being conducted.

Conclusions

Application in ophthalmology is still in its infancy. Most previous experiments have been performed in the field of retinal oximetry, providing valuable information in the diagnosis and monitoring of various ocular diseases. To date, none of these applications have graduated to clinical practice owing to the lack of sufficiently large validation studies.

Translational Relevance

Given the promising results that smaller studies show for hyperspectral imaging (e.g., in Alzheimer's disease), advanced research in larger validation studies is warranted to determine its true clinical potential.

Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range

SIGNIFICANCE

Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.

AIM

An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.

APPROACH

Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.

RESULTS

The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).

CONCLUSIONS

Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

Workflow and hardware for intraoperative hyperspectral data acquisition in neurosurgery

To prevent further brain tumour growth, malignant tissue should be removed as completely as possible in neurosurgical operations. Therefore, differentiation between tumour and brain tissue as well as detecting functional areas is very important. Hyperspectral imaging (HSI) can be used to get spatial information about brain tissue types and characteristics in a quasi-continuous reflection spectrum. In this paper, workflow and some aspects of an adapted hardware system for intraoperative hyperspectral data acquisition in neurosurgery are discussed. By comparing an intraoperative with a laboratory setup, the influences of the surgical microscope are made visible through the differences in illumination and a pixel- and wavelength-specific signal-to-noise ratio (SNR) calculation. Due to the significant differences in shape and wavelength-dependent intensity of light sources, it can be shown which kind of illumination is most suitable for the setups. Spectra between 550 and 1,000 nm are characterized of at least 40 dB SNR in laboratory and 25 dB in intraoperative setup in an area of the image relevant for evaluation. A first validation of the intraoperative hyperspectral imaging hardware setup shows that all system parts and intraoperatively recorded data can be evaluated. Exemplarily, a classification map was generated that allows visualization of measured properties of raw data. The results reveal that it is possible and beneficial to use HSI for wavelength-related intraoperative data acquisition in neurosurgery. There are still technical facts to optimize for raw data detection prior to adapting image processing algorithms to specify tissue quality and function.

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.

Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning

The performance of hyperspectral imaging (HSI) for tumor detection is investigated in ex-vivo specimens from the thyroid (N = 200) and salivary glands (N = 16) from 82 patients. Tissues were imaged with HSI in broadband reflectance and autofluorescence modes. For comparison, the tissues were imaged with two fluorescent dyes. Additionally, HSI was used to synthesize three-band RGB multiplex images to represent the human-eye response and Gaussian RGBs, which are referred to as HSI-synthesized RGB images. Using histological ground truths, deep learning algorithms were developed for tumor detection. For the classification of thyroid tumors, HSI-synthesized RGB images achieved the best performance with an AUC score of 0.90. In salivary glands, HSI had the best performance with 0.92 AUC score. This study demonstrates that HSI could aid surgeons and pathologists in detecting tumors of the thyroid and salivary glands.