Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

A proof of concept for microcirculation monitoring using machine learning based hyperspectral imaging in critically ill patients: a monocentric observational study

BACKGROUND

Impaired microcirculation is a cornerstone of sepsis development and leads to reduced tissue oxygenation, influenced by fluid and catecholamine administration during treatment. Hyperspectral imaging (HSI) is a non-invasive bedside technology for visualizing physicochemical tissue characteristics. Machine learning (ML) for skin HSI might offer an automated approach for bedside microcirculation assessment, providing an individualized tissue fingerprint of critically ill patients in intensive care. The study aimed to determine if machine learning could be utilized to automatically identify regions of interest (ROIs) in the hand, thereby distinguishing between healthy individuals and critically ill patients with sepsis using HSI.

METHODS

HSI raw data from 75 critically ill sepsis patients and from 30 healthy controls were recorded using TIVITA® Tissue System and analyzed using an automated ML approach. Additionally, patients were divided into two groups based on their SOFA scores for further subanalysis: less severely ill (SOFA ≤ 5) and severely ill (SOFA > 5). The analysis of the HSI raw data was fully-automated using MediaPipe for ROI detection (palm and fingertips) and feature extraction. HSI Features were statistically analyzed to highlight relevant wavelength combinations using Mann-Whitney-U test and Benjamini, Krieger, and Yekutieli (BKY) correction. In addition, Random Forest models were trained using bootstrapping, and feature importances were determined to gain insights regarding the wavelength importance for a model decision.

RESULTS

An automated pipeline for generating ROIs and HSI feature extraction was successfully established. HSI raw data analysis accurately distinguished healthy controls from sepsis patients. Wavelengths at the fingertips differed in the ranges of 575-695 nm and 840-1000 nm. For the palm, significant differences were observed in the range of 925-1000 nm. Feature importance plots indicated relevant information in the same wavelength ranges. Combining palm and fingertip analysis provided the highest reliability, with an AUC of 0.92 to distinguish between sepsis patients and healthy controls.

CONCLUSION

Based on this proof of concept, the integration of automated and standardized ROIs along with automated skin HSI analyzes, was able to differentiate between healthy individuals and patients with sepsis. This approach offers a reliable and objective assessment of skin microcirculation, facilitating the rapid identification of critically ill patients.

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Fluorescence excitation-scanning hyperspectral imaging with scalable 2D-3D deep learning framework for colorectal cancer detection

Colorectal cancer is one of the top contributors to cancer-related deaths in the United States, with over 100,000 estimated cases in 2020 and over 50,000 deaths. The most common screening technique is minimally invasive colonoscopy using either reflected white light endoscopy or narrow-band imaging. However, current imaging modalities have only moderate sensitivity and specificity for lesion detection. We have developed a novel fluorescence excitation-scanning hyperspectral imaging (HSI) approach to sample image and spectroscopic data simultaneously on microscope and endoscope platforms for enhanced diagnostic potential. Unfortunately, fluorescence excitation-scanning HSI datasets pose major challenges for data processing, interpretability, and classification due to their high dimensionality. Here, we present an end-to-end scalable Artificial Intelligence (AI) framework built for classification of excitation-scanning HSI microscopy data that provides accurate image classification and interpretability of the AI decision-making process. The developed AI framework is able to perform real-time HSI classification with different speed/classification performance trade-offs by tailoring the dimensionality of the dataset, supporting different dimensions of deep learning models, and varying the architecture of deep learning models. We have also incorporated tools to visualize the exact location of the lesion detected by the AI decision-making process and to provide heatmap-based pixel-by-pixel interpretability. In addition, our deep learning framework provides wavelength-dependent impact as a heatmap, which allows visualization of the contributions of HSI wavelength bands during the AI decision-making process. This framework is well-suited for HSI microscope and endoscope platforms, where real-time analysis and visualization of classification results are required by clinicians.

Histological Hyperspectral Glioblastoma Dataset (HistologyHSI-GB)

Hyperspectral (HS) imaging (HSI) technology combines the main features of two existing technologies: imaging and spectroscopy. This allows to analyse simultaneously the morphological and chemical attributes of the objects captured by a HS camera. In recent years, the use of HSI provides valuable insights into the interaction between light and biological tissues, and makes it possible to detect patterns, cells, or biomarkers, thus, being able to identify diseases. This work presents the HistologyHSI-GB dataset, which contains 469 HS images from 13 patients diagnosed with brain tumours, specifically glioblastoma. The slides were stained with haematoxylin and eosin (H&E) and captured using a microscope at 20× power magnification. Skilled histopathologists diagnosed the slides and provided image-level annotations. The dataset was acquired using custom HSI instrumentation, consisting of a microscope equipped with an HS camera covering the spectral range from 400 to 1000 nm.

Hyperspectral Imaging Detects Clitoral Vascular Issues in Gender-Affirming Surgery

The aim of this study was to assess the efficacy of hyperspectral imaging (HSI) as an intraoperative perfusion imaging modality during gender affirmation surgery (GAS). The hypothesis posited that HSI could quantify perfusion to the clitoral complex, thereby enabling the prediction of either uneventful wound healing or the occurrence of necrosis. In this non-randomised prospective clinical study, we enrolled 30 patients who underwent GAS in the form of vaginoplasty with the preparation of a clitoral complex from 2020 to 2024 and compared patients' characteristics as well as HSI data regarding clitoris necrosis. Individuals demonstrating uneventful wound healing pertaining to the clitoral complex were designated as Group A. Patients with complete necrosis of the neo-clitoris were assigned to Group B. Patient characteristics were collected and subsequently a comparative analysis carried out. No significant difference in patient characteristics was observed between the two groups. Necrosis occurred when both StO2 and NIR PI parameters fell below 40%. For the simultaneous occurrence of StO2 and NIR PI of 40% or less, a sensitivity of 92% and specificity of 72% was calculated. Intraoperatively, the onset of necrosis in the clitoral complex can be reliably predicted with the assistance of HSI.

[Indications and surgical strategy for bowel resection in mesenteric ischemia : Resection margins considering current guidelines and literature as well as the influence of new technical possibilities]

Acute mesenteric ischemia (AMI) is still a time-critical and life-threatening clinical picture. If exploration of the abdominal cavity is necessary during treatment, an intraoperative assessment of which segments of the intestines have a sufficient potential for recovery must be made. These decisions are mostly based on purely clinical parameters, which are subject to high level of uncertainty. This review article provides an overview of how this decision-making process and the determination of resection margins can be improved using technical aids, such as laser Doppler flowmetry (LDF), indocyanine green (ICG) fluorescence angiography or hyperspectral imaging (HSI). Furthermore, this article compiles guideline recommendations on the role of laparoscopy and the value of a planned second-look laparotomy. In addition, an overview of strategies for preventing short bowel syndrome is given and other aspects, such as the timing and technical aspects of placement of a preternatural anus and an anastomosis are highlighted.

Hyperspectral imaging and evaluation of surgical margins: where do we stand?

PURPOSE OF REVIEW

To highlight the recent literature on the use of hyperspectral imaging (HSI) for cancer margin evaluation ex vivo, for head and neck cancer pathology and in vivo during head and neck cancer surgery.

RECENT FINDINGS

HSI can be used ex vivo on unstained and stained tissue sections to analyze head and neck tissue and tumor cells in combination with machine learning approaches to analyze head and neck cancer cell characteristics and to discriminate the tumor border from normal tissue. Data on in vivo applications during head and neck cancer surgery are preliminary and limited. Even now an accuracy of 80% for tumor versus nonneoplastic tissue classification can be achieved for certain tasks, within the current in vivo settings.

SUMMARY

Significant progress has been made to introduce HSI for ex vivo head and neck cancer pathology evaluation and for an intraoperative use to define the tumor margins. To optimize the accuracy for in vivo use, larger HSI databases with annotations for head and neck cancer are needed.

Optimizing clinical O2 saturation mapping using hyperspectral imaging and diffuse reflectance spectroscopy in the context of epinephrine injection

Clinical determination of oxygen saturation (sO2) in patients is commonly performed via non-invasive optical techniques. However, reliance on a few wavelengths and some form of pre-determined calibration introduces limits to how these methods can be used. One example involves the assessment of sO2 after injection of local anesthetic using epinephrine, where some controversy exists around the time it takes for the epinephrine to have an effect. This is likely caused by a change in the tissue environment not accounted for by standard calibrated instruments and conventional analysis techniques. The present study aims to account for this changing environment by acquiring absorption spectra using hyperspectral imaging (HSI) and diffuse reflectance spectroscopy (DRS) before, during, and after the injection of local anesthesia containing epinephrine in human volunteers. We demonstrate the need to account for multiple absorbing species when applying linear spectral unmixing in order to obtain more clinically relevant sO2 values. In particular, we demonstrate how the inclusion of water absorption greatly affects the rate at which sO2 seemingly drops, which in turn sheds light on the current debate regarding the time required for local anesthesia with epinephrine to have an effect. In general, this work provides important insight into how spectral analysis methods need to be adapted to specific clinical scenarios to more accurately assess sO2.

Consensus conference statement on fluorescence-guided surgery (FGS) ESSO course on fluorescence-guided surgery

BACKGROUND

Fluorescence-guided surgery (FGS) has emerged as an innovative technique with promising applications in various surgical specialties. However, clinical implementation is hampered by limited availability of evidence-based reference work supporting the translation towards standard-of-care use in surgical practice. Therefore, we developed a consensus statement on current applications of FGS.

METHODS

During an international FGS course, participants anonymously voted on 36 statements. Consensus was defined as agreement ≥70% with participation grade of ≥80%. All participants of the questionnaire were stratified for user and handling experience within five domains of applicability (lymphatics & lymph node imaging; tissue perfusion; biliary anatomy and urinary tracts; tumor imaging in colorectal, HPB, and endocrine surgery, and quantification and (tumor-) targeted imaging). Results were pooled to determine consensus for each statement within the respective sections based on the degree of agreement.

RESULTS

In total 43/52 (81%) course participants were eligible as voting members for consensus, comprising the expert panel (n = 12) and trained users (n = 31). Consensus was achieved in 17 out of 36 (45%) statements with highest level of agreement for application of FGS in tissue perfusion and biliary/urinary tract visualization (71% and 67%, respectively) and lowest within the tumor imaging section (0%).

CONCLUSIONS

FGS is currently established for tissue perfusion and vital structure imaging. Lymphatics & lymph node imaging in breast cancer and melanoma are evolving, and tumor tissue imaging holds promise in early-phase trials. Quantification and (tumor-)targeted imaging are advancing toward clinical validation. Additional research is needed for tumor imaging due to a lack of consensus.

The microcirculation in perioperative medicine: a narrative review

The microcirculation describes the network of the smallest vessels in our cardiovascular system. On a microcirculatory level, oxygen delivery is determined by the flow of oxygen-carrying red blood cells in a given single capillary (capillary red blood cell flow) and the density of the capillary network in a given tissue volume (capillary vessel density). Handheld vital videomicroscopy enables visualisation of the capillary bed on the surface of organs and tissues but currently is only used for research. Measurements are generally possible on all organ surfaces but are most often performed in the sublingual area. In patients presenting for elective surgery, the sublingual microcirculation is usually intact and functional. Induction of general anaesthesia slightly decreases capillary red blood cell flow and increases capillary vessel density. During elective, even major, noncardiac surgery, the sublingual microcirculation is preserved and remains functional, presumably because elective noncardiac surgery is scheduled trauma and haemodynamic alterations are immediately treated by anaesthesiologists, usually restoring the macrocirculation before the microcirculation is substantially impaired. Additionally, surgery is regional trauma and thus likely causes regional, rather than systemic, impairment of the microcirculation. Whether or not the sublingual microcirculation is impaired after noncardiac surgery remains a subject of ongoing research. Similarly, it remains unclear if cardiac surgery, especially with cardiopulmonary bypass, impairs the sublingual microcirculation. The effects of therapeutic interventions specifically targeting the microcirculation remain to be elucidated and tested. Future research should focus on further improving microcirculation monitoring methods and investigating how regional microcirculation monitoring can inform clinical decision-making and treatment.

Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection

Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.

Bedside Hyperspectral Imaging and Organ Dysfunction Severity in Critically Ill COVID-19 Patients-A Prospective, Monocentric Observational Study

Hyperspectral imaging (HSI) is a non-invasive technology that provides information on biochemical tissue properties, including skin oxygenation and perfusion quality. Microcirculatory alterations are associated with organ dysfunction in septic COVID-19 patients. This prospective observational study investigated associations between skin HSI and organ dysfunction severity in critically ill COVID-19 patients. During the first seven days in the ICU, palmar HSI measurements were carried out with the TIVITA® tissue system. We report data from 52 critically ill COVID-19 patients, of whom 40 required extracorporeal membrane oxygenation (ECMO). HSI parameters for superficial tissue oxygenation (StO2) and oxygenation and perfusion quality (NPI) were persistently decreased. Hemoglobin tissue content (THI) increased, and tissue water content (TWI) was persistently elevated. Regression analysis showed strong indications for an association of NPI and weaker indications for associations of StO2, THI, and TWI with sequential organ failure assessment (SOFA) scoring. StO2 and NPI demonstrated negative associations with vasopressor support and lactate levels as well as positive associations with arterial oxygen saturation. These results suggest that skin HSI provides clinically relevant information, opening new perspectives for microcirculatory monitoring in critical care.

Detection and management of bile duct injury during cholecystectomy

INTRODUCTION

Cholecystectomy represents one of the most performed surgical procedures. Bile duct injuries (BDIs) are a dangerous complication of this intervention. With the advent of the laparoscopy, the rate of BDIs showed a growing trend that was partially justified by the learning curve of this technique.

EVIDENCE ACQUISITION

A literature search was conducted on Embase, Medline, and Cochrane databases to identify studies published up to October 2022 that analyzed the intraoperative detection and management of BDIs diagnosed during cholecystectomy.

EVIDENCE SYNTHESIS

According to the literature, approximately 25% of BDIs is diagnosed during the laparoscopic cholecystectomy. In the clinical suspicion of BDI, an intraoperative cholangiography is performed to confirm it. Complimentary technology, such as near-infrared cholangiography, can be also adopted. Intraoperative ultrasound represents a useful tool to furtherly define the biliary and the vascular anatomy. The proper classification of the type of BDI allows to identify the correct treatment. When a good expertise in hepato-pancreato-biliary surgery is available, a direct repair is performed with good outcomes both in case of simple and complex lesions. When the local resources are limited or there is a lack of dedicated surgical experience, patient referral to a reference center shows better outcomes. In particular, complex vasculo-biliary injuries require a highly specialized treatment. The key elements to transfer the patients are a good documentation of the injury, a proper drainage of the abdomen, and an antibiotic therapy.

CONCLUSIONS

BDI management requires a proper diagnostic process and prompt treatment to reduce the morbidity and mortality of this feared complication occurring during cholecystectomy.

Hyperspectral placenta dataset: Hyperspectral image acquisition, annotations, and processing of biological tissues in microsurgical training

The dataset consists of 101 hyperspectral images of four human placentas and six hyperspectral images of contrast dyes (i.e., indocyanine green and red and blue food colorant) that were captured in the range 515-900 nm, step = 5 nm. The hyperspectral images were manually annotated, delineating the key anatomical structures: arteries, veins, stroma, and the umbilical cord. Standard reference materials were used for flat-field correction. The dataset is instrumental for advancing machine-learning algorithms and automated classification of anatomical structures, particularly the classification of superficial and deep vessels and transparent tissue layers.

2023 WSES guidelines for the prevention, detection, and management of iatrogenic urinary tract injuries (IUTIs) during emergency digestive surgery

Iatrogenic urinary tract injury (IUTI) is a severe complication of emergency digestive surgery. It can lead to increased postoperative morbidity and mortality and have a long-term impact on the quality of life. The reported incidence of IUTIs varies greatly among the studies, ranging from 0.3 to 1.5%. Given the high volume of emergency digestive surgery performed worldwide, there is a need for well-defined and effective strategies to prevent and manage IUTIs. Currently, there is a lack of consensus regarding the prevention, detection, and management of IUTIs in the emergency setting. The present guidelines, promoted by the World Society of Emergency Surgery (WSES), were developed following a systematic review of the literature and an international expert panel discussion. The primary aim of these WSES guidelines is to provide evidence-based recommendations to support clinicians and surgeons in the prevention, detection, and management of IUTIs during emergency digestive surgery. The following key aspects were considered: (1) effectiveness of preventive interventions for IUTIs during emergency digestive surgery; (2) intra-operative detection of IUTIs and appropriate management strategies; (3) postoperative detection of IUTIs and appropriate management strategies and timing; and (4) effectiveness of antibiotic therapy (including type and duration) in case of IUTIs.

Hyperspectral Imaging in Brain Tumor Surgery-Evidence of Machine Learning-Based Performance

BACKGROUND

Hyperspectral imaging (HSI) has the potential to enhance surgical tissue detection and diagnostics. Definite utilization of intraoperative HSI guidance demands validated machine learning and public datasets that currently do not exist. Moreover, current imaging conventions are dispersed, and evidence-based paradigms for neurosurgical HSI have not been declared.

METHODS

We presented the rationale and a detailed clinical paradigm for establishing microneurosurgical HSI guidance. In addition, a systematic literature review was conducted to summarize the current indications and performance of neurosurgical HSI systems, with an emphasis on machine learning-based methods.

RESULTS

The published data comprised a few case series or case reports aiming to classify tissues during glioma operations. For a multitissue classification problem, the highest overall accuracy of 80% was obtained using deep learning. Our HSI system was capable of intraoperative data acquisition and visualization with minimal disturbance to glioma surgery.

CONCLUSIONS

In a limited number of publications, neurosurgical HSI has demonstrated unique capabilities in contrast to the established imaging techniques. Multidisciplinary work is required to establish communicable HSI standards and clinical impact. Our HSI paradigm endorses systematic intraoperative HSI data collection, which aims to facilitate the related standards, medical device regulations, and value-based medical imaging systems.

Hyperspectral imaging enables the differentiation of differentially inflated and perfused pulmonary tissue: a proof-of-concept study in pulmonary lobectomies for intersegmental plane mapping

OBJECTIVES

The identification of the intersegmental plane is a major interoperative challenges during pulmonary segmentectomies. The objective of this pilot study is to test the feasibility of lung perfusion assessment by Hyperspectral Imaging for identification of the intersegmental plane.

METHODS

A pilot study (clinicaltrials.org: NCT04784884) was conducted in patients with lung cancer. Measuring tissue oxygenation (StO₂; upper tissue perfusion), organ hemoglobin index (OHI), near-infrared index (NIR; deeper tissue perfusion) and tissue water index (TWI), the Hyperspectral Imaging measurements were carried out in inflated (P_vent) and deflated pulmonary lobes (P_nV) as well as in deflated pulmonary lobes with divided circulation (P_nVC) before dissection of the lobar bronchus.

RESULTS

A total of 341 measuring points were evaluated during pulmonary lobectomies. Pulmonary lobes showed a reduced StO2 (P_vent: 84.56% ± 3.92 vs. P_nV: 63.62% ± 11.62 vs. P_nVC: 39.20% ± 23.57; p<0.05) and NIR-perfusion (P_vent: 50.55 ± 5.62 vs. P_nV: 47.55 ± 3.38 vs. P_nVC: 27.60 ± 9.33; p<0.05). There were no differences of OHI and TWI between the three groups.

CONCLUSIONS

This pilot study demonstrates that HSI enables differentiation between different ventilated and perfused pulmonary tissue as a precondition for HSI segment mapping.

In Vitro Antibody Quantification with Hyperspectral Imaging in a Large Field of View for Clinical Applications

Hyperspectral imaging (HSI) is a non-invasive, contrast-free optical-based tool that has recently been applied in medical and basic research fields. The opportunity to use HSI to identify exogenous tumor markers in a large field of view (LFOV) could increase precision in oncological diagnosis and surgical treatment. In this study, the anti-high mobility group B1 (HMGB1) labeled with Alexa fluorophore (647 nm) was used as the target molecule. This is the proof-of-concept of HSI’s ability to quantify antibodies via an in vitro setting. A first test was performed to understand whether the relative absorbance provided by the HSI camera was dependent on volume at a 1:1 concentration. A serial dilution of 1:1, 10, 100, 1000, and 10,000 with phosphatase-buffered saline (PBS) was then used to test the sensitivity of the camera at the minimum and maximum volumes. For the analysis, images at 640 nm were extracted from the hypercubes according to peak signals matching the specificities of the antibody manufacturer. The results showed a positive correlation between relative absorbance and volume (r = 0.9709, p = 0.0013). The correlation between concentration and relative absorbance at min (1 µL) and max (20 µL) volume showed r = 0.9925, p < 0.0001, and r = 0.9992, p < 0.0001, respectively. These results demonstrate the HSI potential in quantifying HMGB1, hence deserving further studies in ex vivo and in vivo settings.

Intraoperative Determination of Bronchus Stump and Anastomosis Perfusion with Hyperspectral Imaging

BACKGROUND

The intraoperative evaluation of bronchus perfusion is limited. Hyperspectral Imaging (HSI) is a newly established intraoperative imaging technique that enables a non-invasive, real-time perfusion analysis. Therefore, the purpose of this study was to determine the intraoperative perfusion of bronchus stump and anastomosis during pulmonary resections with HSI.

METHODS

In this prospective, IDEAL Stage 2a study (Clinicaltrials.gov: NCT04784884) HSI measurements were carried out before bronchial dissection and after bronchial stump formation or bronchial anastomosis, respectively. Tissue oxygenation (StO₂; upper tissue perfusion), organ hemoglobin index (OHI), near-infrared index (NIR; deeper tissue perfusion) and tissue water index (TWI) were calculated.

RESULTS

Bronchus stumps showed a reduced NIR (77.82 ± 10.27 vs 68.01 ± 8.95; P = 0,02158) and OHI (48.60 ± 1.39 vs 38.15 ± 9.74; P = <.0001), although the perfusion of the upper tissue layers was equivalent before and after resection (67.42% ± 12.53 vs 65.91% ± 10.40). In the sleeve resection group, we found both a significant decrease in StO 2 and NIR between central bronchus and anastomosis region (StO₂: 65.09% ± 12.57 vs 49.45 ± 9.94; P = .044; NIR: 83.73 ± 10.92 vs 58.62 ± 3.01; P = .0063). Additionally, NIR was decreased in the re-anastomosed bronchus compared to central bronchus region (83.73 ± 10.92 vs 55.15 ± 17.56; P = .0029).

CONCLUSIONS

Although both bronchus stumps and anastomosis show an intraoperative reduction of tissue perfusion, there is no difference of tissue hemoglobin level in bronchus anastomosis.

Evaluation of Preprocessing Methods on Independent Medical Hyperspectral Databases to Improve Analysis

Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.

Intraoperative bowel perfusion quantification with hyperspectral imaging: a guidance tool for precision colorectal surgery

BACKGROUND

Poor anastomotic perfusion can cause anastomotic leaks (AL). Hyperspectral imaging (HSI), previously validated experimentally, provides accurate, real-time, contrast-free intestinal perfusion quantification. Clinical experience with HSI is limited. In this study, HSI was used to evaluate bowel perfusion intraoperatively.

METHODS

Fifty-two patients undergoing elective colorectal surgeries for neoplasia (n = 40) or diverticular disease (n = 12), were enrolled. Intestinal perfusion was assessed with HSI (TIVITA®, Diaspective Vision, Am Salzhaff, Germany). This device generates a perfusion heat map reflecting the tissue oxygen saturation (StO2) amount. Prior to anastomose creation, the clinical transection line (CTL) was highlighted on the proximal bowel and imaged with HSI. Upon StO2 heat map evaluation, the hyperspectral transection line (HTL) was identified. In case of CTL/HTL discrepancy > 5 mm, the bowel was always resected at the HTL. HSI outcomes were compared to the clinical ones.

RESULTS

AL occurred in one patient who underwent neoadjuvant radiochemotherapy and ultralow anterior resection for rectal cancer. HSI assessment was feasible in all patients, and StO2-values were significantly higher at proximal segments than distal ones. Twenty-six patients showed CTL/HTL discrepancy, and these patients had a lower mean StO2 (54.55 ± 21.30%) than patients without discrepancy (65.10 ± 21.30%, p = 0.000). Patients undergoing neoadjuvant radiochemotherapy showed a lower StO2 (51.41 ± 23.41%) than non-neoadjuvated patients (60.51 ± 24.98%, p = 0.010).

CONCLUSION

HSI is useful in detecting intraoperatively marginally perfused segments, for which the clinical appreciation is unreliable. Intestinal vascular supply is lower in patients undergoing neoadjuvant radiochemotherapy, and this novel finding together with the clinical impact of HSI perfusion quantification deserves further investigation in larger trials.

Automatic optical biopsy for colorectal cancer using hyperspectral imaging and artificial neural networks

BACKGROUND

Intraoperative identification of cancerous tissue is fundamental during oncological surgical or endoscopic procedures. This relies on visual assessment supported by histopathological evaluation, implying a longer operative time. Hyperspectral imaging (HSI), a contrast-free and contactless imaging technology, provides spatially resolved spectroscopic analysis, with the potential to differentiate tissue at a cellular level. However, HSI produces "big data", which is impossible to directly interpret by clinicians. We hypothesize that advanced machine learning algorithms (convolutional neural networks-CNNs) can accurately detect colorectal cancer in HSI data.

METHODS

In 34 patients undergoing colorectal resections for cancer, immediately after extraction, the specimen was opened, the tumor-bearing section was exposed and imaged using HSI. Cancer and normal mucosa were categorized from histopathology. A state-of-the-art CNN was developed to automatically detect regions of colorectal cancer in a hyperspectral image. Accuracy was validated with three levels of cross-validation (twofold, fivefold, and 15-fold).

RESULTS

32 patients had colorectal adenocarcinomas confirmed by histopathology (9 left, 11 right, 4 transverse colon, and 9 rectum). 6 patients had a local initial stage (T1-2) and 26 had a local advanced stage (T3-4). The cancer detection performance of the CNN using 15-fold cross-validation showed high sensitivity and specificity (87% and 90%, respectively) and a ROC-AUC score of 0.95 (considered outstanding). In the T1-2 group, the sensitivity and specificity were 89% and 90%, respectively, and in the T3-4 group, the sensitivity and specificity were 81% and 93%, respectively.

CONCLUSIONS

Automatic colorectal cancer detection on fresh specimens using HSI, using a properly trained CNN is feasible and accurate, even with small datasets, regardless of the local tumor extension. In the near future, this approach may become a useful intraoperative tool during oncological endoscopic and surgical procedures, and may result in precise and non-destructive optical biopsies to support objective and consistent tumor-free resection margins.

Comparison of image registration methods for combining laparoscopic video and spectral image data

Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy.

Staging of Skin Cancer Based on Hyperspectral Microscopic Imaging and Machine Learning

Skin cancer, a common type of cancer, is generally divided into basal cell carcinoma (BCC), squamous cell carcinoma (SCC) and malignant melanoma (MM). The incidence of skin cancer has continued to increase worldwide in recent years. Early detection can greatly reduce its morbidity and mortality. Hyperspectral microscopic imaging (HMI) technology can be used as a powerful tool for skin cancer diagnosis by reflecting the changes in the physical structure and microenvironment of the sample through the differences in the HMI data cube. Based on spectral data, this work studied the staging identification of SCC and the influence of the selected region of interest (ROI) on the staging results. In the SCC staging identification process, the optimal result corresponded to the standard normal variate transformation (SNV) for spectra preprocessing, the partial least squares (PLS) for dimensionality reduction, the hold-out method for dataset partition and the random forest (RF) model for staging identification, with the highest staging accuracy of 0.952 ± 0.014, and a kappa value of 0.928 ± 0.022. By comparing the staging results based on spectral characteristics from the nuclear compartments and peripheral regions, the spectral data of the nuclear compartments were found to contribute more to the accurate staging of SCC.

Design and Validation of a Custom-Made Laboratory Hyperspectral Imaging System for Biomedical Applications Using a Broadband LED Light Source

Hyperspectral imaging (HSI) is a promising optical modality that is already being used in numerous applications. Further expansion of the capabilities of HSI depends on the modularity and versatility of the systems, which would, inter alia, incorporate profilometry, fluorescence imaging, and Raman spectroscopy while following a rigorous calibration and verification protocols, thus offering new insights into the studied samples as well as verifiable, quantitative measurement results applicable to the development of quantitative metrics. Considering these objectives, we developed a custom-made laboratory HSI system geared toward biomedical applications. In this report, we describe the design, along with calibration, characterization, and verification protocols needed to establish such systems, with the overall goal of standardization. As an additional novelty, our HSI system uses a custom-built broadband LED-based light source for reflectance imaging, which is particularly important for biomedical applications due to the elimination of sample heating. Three examples illustrating the utility and advantages of the integrated system in biomedical applications are shown. Our attempt presents both the development of a custom-based laboratory HSI system with novel LED light source as well as a framework which may improve technological standards in HSI system design.

Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery

The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.

Video: Clinical evaluation of a laparoscopic hyperspectral imaging system

Background

Hyperspectral imaging (HSI) during surgical procedures is a new method for perfusion quantification and tissue discrimination. Its use has been limited to open surgery due to large camera sizes, missing color video, or long acquisition times. A hand-held, laparoscopic hyperspectral camera has been developed now to overcome those disadvantages and evaluated clinically for the first time.

Methods

In a clinical evaluation study, gastrointestinal resectates of ten cancer patients were investigated using the laparoscopic hyperspectral camera. Reference data from corresponding anatomical regions were acquired with a clinically approved HSI system. An image registration process was executed that allowed for pixel-wise comparisons of spectral data and parameter images (StO₂: oxygen saturation of tissue, NIR PI: near-infrared perfusion index, OHI: organ hemoglobin index, TWI: tissue water index) provided by both camera systems. The mean absolute error (MAE) and root mean square error (RMSE) served for the quantitative evaluations. Spearman’s rank correlation between factors related to the study design like the time of spectral white balancing and MAE, respectively RMSE, was calculated.

Results

The obtained mean MAEs between the TIVITA® Tissue and the laparoscopic hyperspectral system resulted in StO₂: 11% ± 7%, NIR PI: 14±3, OHI: 14± 5, and TWI: 10 ± 2. The mean RMSE between both systems was 0.1±0.03 from 500 to 750 nm and 0.15 ±0.06 from 750 to 1000 nm. Spearman’s rank correlation coefficients showed no significant correlation between MAE or RMSE and influencing factors related to the study design.

Conclusion

Qualitatively, parameter images of the laparoscopic system corresponded to those of the system for open surgery. Quantitative deviations were attributed to technical differences rather than the study design. Limitations of the presented study are addressed in current large-scale in vivo trials.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-022-09282-y.