Workflow and hardware for intraoperative hyperspectral data acquisition in neurosurgery

Hyperspectral and multispectral imaging in neurosurgery: a systematic literature review and meta-analysis

INTRODUCTION

The neuro-surgical community is witnessing a rising interest for surgical application of multispectral/hyperspectral imaging. Several potential technical applications of this optical imaging are reported, but the set-up is variable and so are the processing methods. We present a systematic review of the relevant literature on the topic.

MATERIALS AND METHODS

A literature search based on the PRISMA principles was performed on PubMed, SCOPUS, and Web of Science, using MESH terms and Boolean operators. Papers regarding intra-operative in-vivo application of multispectral and/or hyperspectral imaging in humans during neurosurgical procedures were included. Papers reporting technologies related to radiological applications were excluded. A meta-analysis on the performance metrics was also conducted.

RESULTS

Our search string retrieved 20 papers. The main applications of optical imaging during neurosurgery concern tumour detection and improvement of the extent of resection (15 papers) or visualization of perfusion changes during neuro-oncology or neuro-vascular surgery (5 papers). All the retrieved articles were pilot studies, proof of concepts, or case reports, with limited number of patients recruited. Sensitivity, specificity, and accuracy were promising in most of the reports, but the metanalysis showed heterogeneous approaches and results among studies.

CONCLUSIONS

The present review shows that several approaches are currently being tested to integrate hyperspectral imaging in neurosurgery, but most of the studies reported a limited pool of patients, with different approaches to data collection and analysis. Further studies on larger cohorts of patients are therefore desirable to fully explore the potential of this imaging technique.

Prediction of Biliary Complications After Human Liver Transplantation Using Hyperspectral Imaging and Convolutional Neural Networks: A Proof-of-concept Study

BACKGROUND

Biliary complications (BCs) negatively impact the outcome after liver transplantation. We herein tested whether hyperspectral imaging (HSI) generated data from bile ducts (BD) on reperfusion and machine learning techniques for data readout may serve as a novel approach for predicting BC.

METHODS

Tissue-specific data from 136 HSI liver images were integrated into a convolutional neural network (CNN). Fourteen patients undergoing liver transplantation after normothermic machine preservation served as a validation cohort. Assessment of oxygen saturation, organ hemoglobin, and tissue water levels through HSI was performed after completing the biliary anastomosis. Resected BD segments were analyzed by immunohistochemistry and real-time confocal microscopy.

RESULTS

Immunohistochemistry and real-time confocal microscopy revealed mild (grade I: 1%-40%) BD damage in 8 patients and moderate (grade II: 40%-80%) injury in 1 patient. Donor and recipient data alone had no predictive capacity toward BC. Deep learning-based analysis of HSI data resulted in >90% accuracy of automated detection of BD. The CNN-based analysis yielded a correct classification in 72% and 69% for BC/no BC. The combination of HSI with donor and recipient factors showed 94% accuracy in predicting BC.

CONCLUSIONS

Deep learning-based modeling using CNN of HSI-based tissue property data represents a noninvasive technique for predicting postoperative BC.

Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection

Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.

Hyperspectral placenta dataset: Hyperspectral image acquisition, annotations, and processing of biological tissues in microsurgical training

The dataset consists of 101 hyperspectral images of four human placentas and six hyperspectral images of contrast dyes (i.e., indocyanine green and red and blue food colorant) that were captured in the range 515-900 nm, step = 5 nm. The hyperspectral images were manually annotated, delineating the key anatomical structures: arteries, veins, stroma, and the umbilical cord. Standard reference materials were used for flat-field correction. The dataset is instrumental for advancing machine-learning algorithms and automated classification of anatomical structures, particularly the classification of superficial and deep vessels and transparent tissue layers.

Hyperspectral Imaging in Brain Tumor Surgery-Evidence of Machine Learning-Based Performance

BACKGROUND

Hyperspectral imaging (HSI) has the potential to enhance surgical tissue detection and diagnostics. Definite utilization of intraoperative HSI guidance demands validated machine learning and public datasets that currently do not exist. Moreover, current imaging conventions are dispersed, and evidence-based paradigms for neurosurgical HSI have not been declared.

METHODS

We presented the rationale and a detailed clinical paradigm for establishing microneurosurgical HSI guidance. In addition, a systematic literature review was conducted to summarize the current indications and performance of neurosurgical HSI systems, with an emphasis on machine learning-based methods.

RESULTS

The published data comprised a few case series or case reports aiming to classify tissues during glioma operations. For a multitissue classification problem, the highest overall accuracy of 80% was obtained using deep learning. Our HSI system was capable of intraoperative data acquisition and visualization with minimal disturbance to glioma surgery.

CONCLUSIONS

In a limited number of publications, neurosurgical HSI has demonstrated unique capabilities in contrast to the established imaging techniques. Multidisciplinary work is required to establish communicable HSI standards and clinical impact. Our HSI paradigm endorses systematic intraoperative HSI data collection, which aims to facilitate the related standards, medical device regulations, and value-based medical imaging systems.

Cervical cell classification with deep-learning algorithms

Cervical cancer is a serious threat to the lives and health of women. The accurate analysis of cervical cell smear images is an important diagnostic basis for cancer identification. However, pathological data are often complex and difficult to analyze accurately because pathology images contain a wide variety of cells. To improve the recognition accuracy of cervical cell smear images, we propose a novel deep-learning model based on the improved Faster R-CNN, shallow feature enhancement networks, and generative adversarial networks. First, we used a global average pooling layer to enhance the robustness of the data feature transformation. Second, we designed a shallow feature enhancement network to improve the localization and recognition of weak cells. Finally, we established a data augmentation network to improve the detection capability of the model. The experimental results demonstrate that our proposed methods are superior to CenterNet, YOLOv5, and Faster R-CNN algorithms in some aspects, such as shorter time consumption, higher recognition precision, and stronger adaptive ability. Its maximum accuracy is 99.81%, and the overall mean average precision is 89.4% for the SIPaKMeD and Herlev datasets. Our method provides a useful reference for cervical cell smear image analysis. The missed diagnosis rate and false diagnosis rate are relatively high for cervical cell smear images of different pathologies and stages. Therefore, our algorithms need to be further improved to achieve a better balance. We will use a hyperspectral microscope to obtain more spectral data of cervical cells and input them into deep-learning models for data processing and classification research. First, we sent training samples of cervical cells into our proposed deep-learning model. Then, we used the proposed model to train eight types of cervical cells. Finally, we utilized the trained classifier to test the untrained samples and obtained the classification results. Fig 1. Deep-learning cervical cell classification framework.

Hyperspectral Imaging as a Tool for Viability Assessment During Normothermic Machine Perfusion of Human Livers: A Proof of Concept Pilot Study

Normothermic machine perfusion (NMP) allows for ex vivo viability and functional assessment prior to liver transplantation (LT). Hyperspectral imaging represents a suitable, non-invasive method to evaluate tissue morphology and organ perfusion during NMP. Liver allografts were subjected to NMP prior to LT. Serial image acquisition of oxygen saturation levels (StO2), organ hemoglobin (THI), near-infrared perfusion (NIR) and tissue water indices (TWI) through hyperspectral imaging was performed during static cold storage, at 1h, 6h, 12h and at the end of NMP. The readouts were correlated with perfusate parameters at equivalent time points. Twenty-one deceased donor livers were included in the study. Seven (33.0%) were discarded due to poor organ function during NMP. StO2 (p < 0.001), THI (p < 0.001) and NIR (p = 0.002) significantly augmented, from static cold storage (pre-NMP) to NMP end, while TWI dropped (p = 0.005) during the observational period. At 12-24h, a significantly higher hemoglobin concentration (THI) in the superficial tissue layers was seen in discarded, compared to transplanted livers (p = 0.036). Lactate values at 12h NMP correlated negatively with NIR perfusion index between 12 and 24h NMP and with the delta NIR perfusion index between 1 and 24h (rs = -0.883, p = 0.008 for both). Furthermore, NIR and TWI correlated with lactate clearance and pH. This study provides first evidence of feasibility of hyperspectral imaging as a potentially helpful contact-free organ viability assessment tool during liver NMP.

Hyperspectral Imaging during Normothermic Machine Perfusion—A Functional Classification of Ex Vivo Kidneys Based on Convolutional Neural Networks

Facing an ongoing organ shortage in transplant medicine, strategies to increase the use of organs from marginal donors by objective organ assessment are being fostered. In this context, normothermic machine perfusion provides a platform for ex vivo organ evaluation during preservation. Consequently, analytical tools are emerging to determine organ quality. In this study, hyperspectral imaging (HSI) in the wavelength range of 550–995 nm was applied. Classification of 26 kidneys based on HSI was established using KidneyResNet, a convolutional neural network (CNN) based on the ResNet-18 architecture, to predict inulin clearance behavior. HSI preprocessing steps were implemented, including automated region of interest (ROI) selection, before executing the KidneyResNet algorithm. Training parameters and augmentation methods were investigated concerning their influence on the prediction. When classifying individual ROIs, the optimized KidneyResNet model achieved 84% and 62% accuracy in the validation and test set, respectively. With a majority decision on all ROIs of a kidney, the accuracy increased to 96% (validation set) and 100% (test set). These results demonstrate the feasibility of HSI in combination with KidneyResNet for non-invasive prediction of ex vivo kidney function. This knowledge of preoperative renal quality may support the organ acceptance decision.

Hyperspectral Imaging and Machine Perfusion in Solid Organ Transplantation: Clinical Potentials of Combining Two Novel Technologies

Organ transplantation survival rates have continued to improve over the last decades, mostly due to reduction of mortality early after transplantation. The advancement of the field is facilitating a liberalization of the access to organ transplantation with more patients with higher risk profile being added to the waiting list. At the same time, the persisting organ shortage fosters strategies to rescue organs of marginal donors. In this regard, hypothermic and normothermic machine perfusion are recognized as one of the most important developments in the modern era. Owing to these developments, novel non-invasive tools for the assessment of organ quality are on the horizon. Hyperspectral imaging represents a potentially suitable method capable of evaluating tissue morphology and organ perfusion prior to transplantation. Considering the changing environment, we here discuss the hypothetical combination of organ machine perfusion and hyperspectral imaging as a prospective feasibility concept in organ transplantation.

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Comparison of different spectral cameras for image-guided organ transplantation

SIGNIFICANCE

Hyperspectral and multispectral imaging (HMSI) in medical applications provides information about the physiology, morphology, and composition of tissues and organs. The use of these technologies enables the evaluation of biological objects and can potentially be applied as an objective assessment tool for medical professionals.

AIM

Our study investigates HMSI systems for their usability in medical applications.

APPROACH

Four HMSI systems (one hyperspectral pushbroom camera and three multispectral snapshot cameras) were examined and a spectrometer was used as a reference system, which was initially validated with a standardized color chart. The spectral accuracy of the cameras reproducing chemical properties of different biological objects (porcine blood, physiological porcine tissue, and pathological porcine tissue) was analyzed using the Pearson correlation coefficient.

RESULTS

All the HMSI cameras examined were able to provide the characteristic spectral properties of blood and tissues. A pushbroom camera and two snapshot systems achieve Pearson coefficients of at least 0.97 compared to the ground truth, indicating a very high positive correlation. Only one snapshot camera performs moderately to high positive correlation (0.59 to 0.85).

CONCLUSION

The knowledge of the suitability of HMSI cameras for accurate measurement of chemical properties of biological objects offers a good opportunity for the selection of the optimal imaging tool for specific medical applications, such as organ transplantation.

Is Hyperspectral Imaging Suitable for Assessing Collateral Circulation Prior Radial Forearm Free Flap Harvesting? Comparison of Hyperspectral Imaging and Conventional Allen's Test

(1) Background: This cross-sectional study aims to compare a new and non-invasive approach using hyperspectral imaging (HSI) with the conventional modified Allen’s test (MAT) for the assessment of collateral perfusion prior to radial forearm free flap harvest in healthy adults. (2) HSI of the right hand of 114 patients was recorded. Here, three recordings were carried out: (I) basic status (perfusion), (II) after occlusion of ulnar and radial artery (occlusion) and (III) after releasing the ulnar artery (reperfusion). At all recordings, tissue oxygenation/superficial perfusion (StO₂ (0–100%); 0–1 mm depth), tissue hemoglobin index (THI (0–100)) and near infrared perfusion index/deep perfusion (NIR (0–100); 0–4 mm depth) were assessed. A modified Allen’s test (control) was conducted and compared with the HSI-results. (3) Results: Statistically significant differences between perfusion (I) and artery occlusion (II) and between artery occlusion (II) and reperfusion (III) could be observed within the population with a non-pathological MAT (each <0.001). Significant correlations were observed for the difference between perfusion and reperfusion in THI and the height of the MAT (p < 0.05). Within the population with a MAT >8 s, an impairment in reperfusion was shown (each p < 0.05) and the difference between perfusion and reperfusion exhibited a strong correlation to the height of the MAT (each p < 0.01). (4) Conclusions: The results indicate a reliable differentiation between perfusion and occlusion by HSI. Therefore, HSI could be a useful tool for verification of the correct performance of the MAT as well as to confirm the final diagnosis, as it provides an objective, reproducible method whose results strongly correlate with those obtained by MAT. What is more, it can be easily applied by non-medical personnel.

Algorithm for Mapping Kidney Tissue Water Content during Normothermic Machine Perfusion Using Hyperspectral Imaging

The preservation of kidneys using normothermic machine perfusion (NMP) prior to transplantation has the potential for predictive evaluation of organ quality. Investigations concerning the quantitative assessment of physiological tissue parameters and their dependence on organ function lack in this context. In this study, hyperspectral imaging (HSI) in the wavelength range of 500–995 nm was conducted for the determination of tissue water content (TWC) in kidneys. The quantitative relationship between spectral data and the reference TWC values was established by partial least squares regression (PLSR). Different preprocessing methods were applied to investigate their influence on predicting the TWC of kidneys. In the full wavelength range, the best models for absorbance and reflectance spectra provided Rp2 values of 0.968 and 0.963, as well as root-mean-square error of prediction (RMSEP) values of 2.016 and 2.155, respectively. Considering an optimal wavelength range (800–980 nm), the best model based on reflectance spectra (Rp2 value of 0.941, RMSEP value of 3.202). Finally, the visualization of TWC distribution in all pixels of kidneys’ HSI image was implemented. The results show the feasibility of HSI for a non-invasively and accurate TWC prediction in kidneys, which could be used in the future to assess the quality of kidneys during the preservation period.