A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

Development of a visible to 1600 nm hyperspectral imaging rigid-scope system using supercontinuum light and an acousto-optic tunable filter

In this study, we developed a rigid-scope system that can perform hyperspectral imaging (HSI) between visible and 1600 nm wavelengths using a supercontinuum light source and an acousto-optic tunable filter to emit specific wavelengths. The system optical performance was verified, and the classification ability was investigated. Consequently, it was demonstrated that HSI (490-1600 nm) could be performed. In addition, seven different targets could be classified by the neural network with an accuracy of 99.6%, recall of 93.7%, and specificity of 99.1% when the wavelength range of over 1000 nm (OTN) was extracted from HSI data as train data.

Meta-Attention Network Based Spectral Reconstruction with Snapshot Near-Infrared Metasurface

Near-infrared (NIR) spectral information is important for detecting and analyzing material compositions. However, snapshot NIR spectral imaging systems still pose significant challenges owing to the lack of high-performance NIR filters and bulky setups, preventing effective encoding and integration with mobile devices. This study introduces a snapshot spectral imaging system that employs a compact NIR metasurface featuring 25 distinct C4 symmetry structures. Benefitting from the sufficient spectral variety and low correlation coefficient among these structures, center-wavelength accuracy of 0.05 nm and full width at half maximum accuracy of 0.13 nm are realized. The system maintains good performance within an incident angle of 1°. A novel meta-attention network prior iterative denoising reconstruction (MAN-IDR) algorithm is developed to achieve high-quality NIR spectral imaging. By leveraging the designed metasurface and MAN-IDR, the NIR spectral images, exhibiting precise textures, minimal artifacts in the spatial dimension, and little crosstalk between spectral channels, are reconstructed from a single grayscale recording image. The proposed NIR metasurface and MAN-IDR hold great promise for further integration with smartphones and drones, guaranteeing the adoption of NIR spectral imaging in real-world scenarios such as aerospace, health diagnostics, and machine vision.

Targeted multispectral filter array design for the optimization of endoscopic cancer detection in the gastrointestinal tract

Significance

Color differences between healthy and diseased tissue in the gastrointestinal (GI) tract are detected visually by clinicians during white light endoscopy; however, the earliest signs of cancer are often just a slightly different shade of pink compared to healthy tissue making it hard to detect. Improving contrast in endoscopy is important for early detection of disease in the GI tract during routine screening and surveillance.

Aim

We aim to target alternative colors for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic "chip-on-tip" configuration.

Approach

Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the GI tract. The toolbox was first extended to use additional classification models (k -nearest neighbor, support vector machine, and spectral angle mapper). Using input spectral data from published clinical trials examining the esophagus and colon, we optimized the design of MSFAs with three to nine different bands.

Results

We examined the variation of the spectral and spatial classification accuracies as a function of the number of bands. The MSFA configurations tested showed good classification accuracies when compared to the full hyperspectral data available from the clinical spectra used in these studies.

Conclusion

The ability to retain good classification accuracies with a reduced number of spectral bands could enable the future deployment of multispectral imaging in an endoscopic chip-on-tip configuration using simplified MSFA hardware. Further studies using an expanded clinical dataset are needed to confirm these findings.

Endoscopic Hyperspectral Imaging System to Discriminate Tissue Characteristics in Tissue Phantom and Orthotopic Mouse Pancreatic Tumor Model

In this study, we developed an endoscopic hyperspectral imaging (eHSI) system and evaluated its performance in analyzing tissues within tissue phantoms and orthotopic mouse pancreatic tumor models. Our custom-built eHSI system incorporated a liquid crystal tunable filter. To assess its tissue discrimination capabilities, we acquired images of tissue phantoms, distinguishing between fat and muscle regions. The system underwent supervised training using labeled samples, and this classification model was then applied to other tissue phantom images for evaluation. In the tissue phantom experiment, the eHSI effectively differentiated muscle from fat and background tissues. The precision scores regarding fat tissue classification were 98.3% for the support vector machine, 97.7% for the neural network, and 96.0% with a light gradient-boosting machine algorithm, respectively. Furthermore, we applied the eHSI system to identify tumors within an orthotopic mouse pancreatic tumor model. The F-score of each pancreatic tumor-bearing model reached 73.1% for the KPC tumor model and 63.1% for the Pan02 tumor models. The refined imaging conditions and optimization of the fine-tuning of classification algorithms enhance the versatility and diagnostic efficacy of eHSI in biomedical applications.

Versatile spaceborne photonics with chalcogenide phase-change materials

Recent growth in space systems has seen increasing capabilities packed into smaller and lighter Earth observation and deep space mission spacecraft. Phase-change materials (PCMs) are nonvolatile, reconfigurable, fast-switching, and have recently shown a high degree of space radiation tolerance, thereby making them an attractive materials platform for spaceborne photonics applications. They promise robust, lightweight, and energy-efficient reconfigurable optical systems whose functions can be dynamically defined on-demand and on-orbit to deliver enhanced science or mission support in harsh environments on lean power budgets. This comment aims to discuss the recent advances in rapidly growing PCM research and its potential to transition from conventional terrestrial optoelectronics materials platforms to versatile spaceborne photonic materials platforms for current and next-generation space and science missions. Materials International Space Station Experiment-14 (MISSE-14) mission-flown PCMs outside of the International Space Station (ISS) and key results and NASA examples are highlighted to provide strong evidence of the applicability of spaceborne photonics.

Artificial Intelligence-Triaged 3-Dimensional Pathology to Improve Detection of Esophageal Neoplasia While Reducing Pathologist Workloads

Early detection of esophageal neoplasia via evaluation of endoscopic surveillance biopsies is the key to maximizing survival for patients with Barrett's esophagus, but it is hampered by the sampling limitations of conventional slide-based histopathology. Comprehensive evaluation of whole biopsies with 3-dimensional (3D) pathology may improve early detection of malignancies, but large 3D pathology data sets are tedious for pathologists to analyze. Here, we present a deep learning-based method to automatically identify the most critical 2-dimensional (2D) image sections within 3D pathology data sets for pathologists to review. Our method first generates a 3D heatmap of neoplastic risk for each biopsy, then classifies all 2D image sections within the 3D data set in order of neoplastic risk. In a clinical validation study, we diagnose esophageal biopsies with artificial intelligence-triaged 3D pathology (3 images per biopsy) vs standard slide-based histopathology (16 images per biopsy) and show that our method improves detection sensitivity while reducing pathologist workloads.

Snapshot hyperspectral imaging of intracellular lasers

Intracellular lasers are emerging as powerful biosensors for multiplexed tracking and precision sensing of cells and their microenvironment. This sensing capacity is enabled by quantifying their narrow-linewidth emission spectra, which is presently challenging to do at high speeds. In this work, we demonstrate rapid snapshot hyperspectral imaging of intracellular lasers. Using integral field mapping with a microlens array and a diffraction grating, we obtain images of the spatial and spectral intensity distribution from a single camera acquisition. We demonstrate widefield hyperspectral imaging over a 3 × 3 mm2 field of view and volumetric imaging over 250 × 250 × 800 µm3 (XYZ) volumes with a lateral (XY) resolution of 5 µm, axial (Z) resolution of 10 µm, and a spectral resolution of less than 0.8 nm. We evaluate the performance and outline the challenges and strengths of snapshot methods in the context of characterizing the emission from intracellular lasers. This method offers new opportunities for a diverse range of applications, including high-throughput and long-term biosensing with intracellular lasers.

Intersecting of circular apertures to measure integer and fractional topological charge of vortex beams

Diffraction patterns of optical vortex beams (VBs) by differently shaped apertures are used to determine their topological charge (TC). In this paper, we show by simulations and experiments that diffraction of a Laguerre-Gaussian (LG) beam by intersecting circular apertures can be used to reveal the TC. The presented aperture structure has the advantage of the measurement of fractional TC in addition to the integer, sensitivity to the sign of TC, and low sensitivity to adjusting apertures. Accordingly, in addition to the integer TC up to 8, the fractional TC is measured with a step of 0.1 by two intersecting circular apertures (TICA). By examining a wide range of similarity criteria between the diffraction pattern of the fractional TC and the pattern of the lower integer TC, three metrics for measuring the fractional TC are found. Furthermore, the determination of integer TC up to 6 for three intersecting circular apertures (THICA) is demonstrated.

The Application of Machine Perfusion as an Enhanced ex vivo Model for Optical Imaging

Optical imaging techniques such as spectral imaging show promise for the assessment of tissue health during surgery; however, the validation and translation of such techniques into clinical practise is limited by the lack of representative tissue models. In this paper, we demonstrate the application of an organ perfusion machine as an ex vivo tissue model for optical imaging. Three porcine livers are perfused at stepped blood oxygen saturations. Over the duration of each perfusion, spectral data of the tissue are captured via diffuse optical spectroscopy and multispectral imaging. These data are synchronised with blood oxygen saturation measurements recorded by the perfusion machine. Shifts in the optical properties of the tissue are demonstrated over the duration of the each perfusion, as the tissue becomes reperfused and as the oxygen saturation is varied.

Recognition of liver tumors by predicted hyperspectral features based on patient's Computed Tomography radiomics features

BACKGROUND

Primary liver tumors have posed a serious threat to human life and health, and their early diagnosis is urgent. Therefore, enhancing the accuracy of non-invasive early detection of liver tumors is imperative.

METHODS

Firstly, image enhancement was applied to augment the dataset, resulting in a total of 464 samples after employing seven data augmentation methods. Subsequently, the XGBoost model was utilized to construct and learn the mapping relationship between Computed Tomography (CT) and corresponding hyperspectral imaging (HSI) data. This model enables the prediction of HSI features corresponding to CT features, thereby enriching CT with more comprehensive hyperspectral information.

RESULTS

Four classifiers were employed to discern the presence of tumors in patients. The results demonstrated exceptional performance, with a classification accuracy exceeding 90%.

CONCLUSIONS

This study proposes an artificial intelligence-based methodology that utilizes early CT radiomics features to predict HSI features. Subsequently, the results are utilized for non-invasive tumor prediction and early screening, thereby enhancing the accuracy of non-invasive liver tumor detection.

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral trade-off. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from a red-green-blue (RGB) image without complete hyperspectral measurements. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral trade-off, offering simple hardware requirements and potential applications of various machine learning techniques.

Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Significance

Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.

Aim

With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.

Approach

We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.

Results

There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.

Conclusions

Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.

Mosaic-free compound eye camera based on multidirectional photodetectors and single-pixel imaging

Compound-eye wide field-of-view (FOV) imaging generally faces the disadvantages of a complex system, low resolution, and complicated image mosaic. Single-pixel imaging has proven to very beneficial in building a high-resolution and simple wide-FOV camera, but its ability to overcome the problem of image mosaics still needs to be demonstrated. In this Letter, we propose a novel, to the best of our knowledge, kind of artificial compound eye based on multidirectional photodetectors (PDs) and demonstrate theoretically and experimentally that mosaics are unnecessary in multidirectional PD-based single-pixel imaging. In addition, we show experimentally that only nine multidirectional PDs are needed to obtain wide-angle images in a hemisphere to realize wide-FOV mosaic-free imaging. This work greatly simplifies the concept of compound-eye cameras and is very enlightening for detector design in wide-FOV single-pixel imaging, plausibly leading to the development of single-pixel endoscopic imaging.

Evaluation of Preprocessing Methods on Independent Medical Hyperspectral Databases to Improve Analysis

Currently, one of the most common causes of death worldwide is cancer. The development of innovative methods to support the early and accurate detection of cancers is required to increase the recovery rate of patients. Several studies have shown that medical Hyperspectral Imaging (HSI) combined with artificial intelligence algorithms is a powerful tool for cancer detection. Various preprocessing methods are commonly applied to hyperspectral data to improve the performance of the algorithms. However, there is currently no standard for these methods, and no studies have compared them so far in the medical field. In this work, we evaluated different combinations of preprocessing steps, including spatial and spectral smoothing, Min-Max scaling, Standard Normal Variate normalization, and a median spatial smoothing technique, with the goal of improving tumor detection in three different HSI databases concerning colorectal, esophagogastric, and brain cancers. Two machine learning and deep learning models were used to perform the pixel-wise classification. The results showed that the choice of preprocessing method affects the performance of tumor identification. The method that showed slightly better results with respect to identifing colorectal tumors was Median Filter preprocessing (0.94 of area under the curve). On the other hand, esophagogastric and brain tumors were more accurately identified using Min-Max scaling preprocessing (0.93 and 0.92 of area under the curve, respectively). However, it is observed that the Median Filter method smooths sharp spectral features, resulting in high variability in the classification performance. Therefore, based on these results, obtained with different databases acquired by different HSI instrumentation, the most relevant preprocessing technique identified in this work is Min-Max scaling.

Comparison of image registration methods for combining laparoscopic video and spectral image data

Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy.

Detection of seed purity of hybrid wheat using reflectance and transmittance hyperspectral imaging technology

Chemical hybridization and genic male sterility systems are two main methods of hybrid wheat production; however, complete sterility of female wheat plants cannot be guaranteed owing to the influence of the growth stage and weather. Consequently, hybrid wheat seeds are inevitably mixed with few parent seeds, especially female seeds. Therefore, seed purity is a key factor in the popularization of hybrid wheat. However, traditional seed purity detection and variety identification methods are time-consuming, laborious, and destructive. Therefore, to establish a non-destructive classification method for hybrid and female parent seeds, three hybrid wheat varieties (Jingmai 9, Jingmai 11, and Jingmai 183) and their parent seeds were sampled. The transmittance and reflectance spectra of all seeds were collected via hyperspectral imaging technology, and a classification model was established using partial least squares-discriminant analysis (PLS-DA) combined with various preprocessing methods. The transmittance spectrum significantly improved the classification of hybrids and female parents compared to that obtained using reflectance spectrum. Specifically, using transmittance spectrum combined with a characteristic wavelength-screening algorithm, the Detrend-CARS-PLS-DA model was established, and the accuracy rates in the testing sets of Jingmai 9, Jingmai 11, and Jingmai 183 were 95.69%, 98.25%, and 97.25%, respectively. In conclusion, transmittance hyperspectral imaging combined with a machine learning algorithm can effectively distinguish female parent seeds from hybrid seeds. These results provide a reference for rapid seed purity detection in the hybrid production process. Owing to the non-destructive and rapid nature of hyperspectral imaging, the detection of hybrid wheat seed purity can be improved by online sorting in the future.

Noninvasive hemoglobin sensing and imaging: optical tools for disease diagnosis

SIGNIFICANCE

Measurement and imaging of hemoglobin oxygenation are used extensively in the detection and diagnosis of disease; however, the applied instruments vary widely in their depth of imaging, spatiotemporal resolution, sensitivity, accuracy, complexity, physical size, and cost. The wide variation in available instrumentation can make it challenging for end users to select the appropriate tools for their application and to understand the relative limitations of different methods.

AIM

We aim to provide a systematic overview of the field of hemoglobin imaging and sensing.

APPROACH

We reviewed the sensing and imaging methods used to analyze hemoglobin oxygenation, including pulse oximetry, spectral reflectance imaging, diffuse optical imaging, spectroscopic optical coherence tomography, photoacoustic imaging, and diffuse correlation spectroscopy.

RESULTS

We compared and contrasted the ability of different methods to determine hemoglobin biomarkers such as oxygenation while considering factors that influence their practical application.

CONCLUSIONS

We highlight key limitations in the current state-of-the-art and make suggestions for routes to advance the clinical use and interpretation of hemoglobin oxygenation information.

An achromatic metafiber for focusing and imaging across the entire telecommunication range

Dispersion engineering is essential to the performance of most modern optical systems including fiber-optic devices. Even though the chromatic dispersion of a meter-scale single-mode fiber used for endoscopic applications is negligible, optical lenses located on the fiber end face for optical focusing and imaging suffer from strong chromatic aberration. Here we present the design and nanoprinting of a 3D achromatic diffractive metalens on the end face of a single-mode fiber, capable of performing achromatic and polarization-insensitive focusing across the entire near-infrared telecommunication wavelength band ranging from 1.25 to 1.65 µm. This represents the whole single-mode domain of commercially used fibers. The unlocked height degree of freedom in a 3D nanopillar meta-atom largely increases the upper bound of the time-bandwidth product of an achromatic metalens up to 21.34, leading to a wide group delay modulation range spanning from -8 to 14 fs. Furthermore, we demonstrate the use of our compact and flexible achromatic metafiber for fiber-optic confocal imaging, capable of creating in-focus sharp images under broadband light illumination. These results may unleash the full potential of fiber meta-optics for widespread applications including hyperspectral endoscopic imaging, femtosecond laser-assisted treatment, deep tissue imaging, wavelength-multiplexing fiber-optic communications, fiber sensing, and fiber lasers.

Assessment of angle-dependent spectral distortion to develop accurate hyperspectral endoscopy

Hyperspectral endoscopy has shown its potential to improve disease diagnosis in gastrointestinal tracts. Recent approaches in developing hyperspectral endoscopy are mainly focusing on enhancing image speed and quality of spectral information under a clinical environment, but there are many issues in obtaining consistent spectral information due to complicated imaging conditions, including imaging angle, non-uniform illumination, working distance, and low reflected signal. We quantitatively investigated the effect of imaging angle on the distortion of spectral information by exploiting a bifurcated fiber, spectrometer, and tissue-mimicking phantom. Spectral distortion becomes severe as increasing the angle of the imaging fiber or shortening camera exposure time for fast image acquisition. Moreover, spectral ranges from 450 to 550 nm are more susceptible to the angle-dependent spectral distortion than longer spectral ranges. Therefore, imaging angles close to normal and longer target spectral ranges with enough detector exposure time could minimize spectral distortion in hyperspectral endoscopy. These findings will help implement clinical HSI endoscopy for the robust and accurate measurement of spectral information from patients in vivo.

A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging

Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.

Endoscopic Imaging Technology Today

One of the most applied imaging methods in medicine is endoscopy. A highly specialized image modality has been developed since the first modern endoscope, the “Lichtleiter” of Bozzini was introduced in the early 19th century. Multiple medical disciplines use endoscopy for diagnostics or to visualize and support therapeutic procedures. Therefore, the shapes, functionalities, handling concepts, and the integrated and surrounding technology of endoscopic systems were adapted to meet these dedicated medical application requirements. This survey gives an overview of modern endoscopic technology’s state of the art. Therefore, the portfolio of several manufacturers with commercially available products on the market was screened and summarized. Additionally, some trends for upcoming developments were collected.

Criteria for the design of tissue-mimicking phantoms for the standardization of biophotonic instrumentation

A lack of accepted standards and standardized phantoms suitable for the technical validation of biophotonic instrumentation hinders the reliability and reproducibility of its experimental outputs. In this Perspective, we discuss general criteria for the design of tissue-mimicking biophotonic phantoms, and use these criteria and state-of-the-art developments to critically review the literature on phantom materials and on the fabrication of phantoms. By focusing on representative examples of standardization in diffuse optical imaging and spectroscopy, fluorescence-guided surgery and photoacoustic imaging, we identify unmet needs in the development of phantoms and a set of criteria (leveraging characterization, collaboration, communication and commitment) for the standardization of biophotonic instrumentation.

Optical biopsy mapping on endoscopic image mosaics with a marker-free probe

Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope.

Chemical Design of Activatable Photoacoustic Probes for Precise Biomedical Applications

Photoacoustic (PA) imaging technology, a three-dimensional hybrid imaging modality that integrates the advantage of optical and acoustic imaging, has great application prospects in molecular imaging due to its high imaging depth and resolution. To endow PA imaging with the ability for real-time molecular visualization and precise biomedical diagnosis, numerous activatable molecular PA probes which can specifically alter their PA intensities upon reacting with the targets or biological events of interest have been developed. This review highlights the recent developments of activatable PA probes for precise biomedical applications including molecular detection of the biotargets and imaging of the biological events. First, the generation mechanism of PA signals will be given, followed by a brief introduction to contrast agents used for PA probe design. Then we will particularly summarize the general design principles for the alteration of PA signals and activatable strategies for developing precise PA probes. Furthermore, we will give a detailed discussion of activatable PA probes in molecular detection and biomedical imaging applications in living systems. At last, the current challenges and outlooks of future PA probes will be discussed. We hope that this review will stimulate new ideas to explore the potentials of activatable PA probes for precise biomedical applications in the future.

360º optoacoustic capsule endoscopy at 50 Hz for esophageal imaging

Gastrointestinal (GI) endoscopy is a common medical diagnostic procedure used for esophageal cancer detection. Current emerging capsule optoacoustic endoscopes, however, suffer from low pulse repetition rates and slow scanning units limit attainable imaging frame rates. Consequently, motion artifacts result in inaccurate spatial mapping and misinterpretation of data. To overcome these limitations, we report a 360^º, 50 Hz frame rate, distal scanning capsule optoacoustic endoscope. The translational capability of the instrument for human GI tract imaging was characterized with an Archimedean spiral phantom consisting of twelve 100 µm sutures, a stainless steel mesh with a pitch of 3 mm and an ex vivo pig esophagus sample. We estimated an imaging penetration depth of ~0.84 mm in vivo by immersing the mesh phantom in intralipid solution to simulate light scattering in human esophageal tissue and validated our findings ex vivo using pig esophagus. This proof-of-concept study demonstrates the translational potential of the proposed video-rate endoscope for human GI tract imaging.

Hyperspectral Imaging for Clinical Applications

Measuring morphological and biochemical features of tissue is crucial for disease diagnosis and surgical guidance, providing clinically significant information related to pathophysiology. Hyperspectral imaging (HSI) techniques obtain both spatial and spectral features of tissue without labeling molecules such as fluorescent dyes, which provides rich information for improved disease diagnosis and treatment. Recent advances in HSI systems have demonstrated its potential for clinical applications, especially in disease diagnosis and image-guided surgery. This review summarizes the basic principle of HSI and optical systems, deep-learning-based image analysis, and clinical applications of HSI to provide insight into this rapidly growing field of research. In addition, the challenges facing the clinical implementation of HSI techniques are discussed.

Classical Dichotomy of Macrophages and Alternative Activation Models Proposed with Technological Progress

Macrophages are important immune cells that participate in the regulation of inflammation in implant dentistry, and their activation/polarization state is considered to be the basis for their functions. The classic dichotomy activation model is commonly accepted, however, due to the discovery of macrophage heterogeneity and more functional and iconic exploration at different technologies; some studies have discovered the shortcomings of the dichotomy model and have put forward the concept of alternative activation models through the application of advanced technologies such as cytometry by time-of-flight (CyTOF), single-cell RNA-seq (scRNA-seq), and hyperspectral image (HSI). These alternative models have great potential to help macrophages divide phenotypes and functional genes.

Biomedical Applications of Translational Optical Imaging: From Molecules to Humans

Light is a powerful investigational tool in biomedicine, at all levels of structural organization. Its multitude of features (intensity, wavelength, polarization, interference, coherence, timing, non-linear absorption, and even interactions with itself) able to create contrast, and thus images that detail the makeup and functioning of the living state can and should be combined for maximum effect, especially if one seeks simultaneously high spatiotemporal resolution and discrimination ability within a living organism. The resulting high relevance should be directed towards a better understanding, detection of abnormalities, and ultimately cogent, precise, and effective intervention. The new optical methods and their combinations needed to address modern surgery in the operating room of the future, and major diseases such as cancer and neurodegeneration are reviewed here, with emphasis on our own work and highlighting selected applications focusing on quantitation, early detection, treatment assessment, and clinical relevance, and more generally matching the quality of the optical detection approach to the complexity of the disease. This should provide guidance for future advanced theranostics, emphasizing a tighter coupling-spatially and temporally-between detection, diagnosis, and treatment, in the hope that technologic sophistication such as that of a Mars rover can be translationally deployed in the clinic, for saving and improving lives.

Using physiological parameters measured by hyperspectral imaging to detect colorectal cancer

The accurate detection of malignant tissue during colorectal surgery impacts operation outcome. The non-invasive spectral imaging combined with machine learning (ML) methods showed to be promising for tumor identification. However, large spectral range implies large computing time. To reduce the number of features, ML methods (e.g. logistic regression and convolutional neuronal network CNN) were evaluated based on four physiological tissue parameters to automatically classify cancer and healthy mucosa in resected colon tissue. A ROC AUC of 0.81 was achieved with the CNN. This study shows that the use of only specific wavelengths bands can detect cancer.Clinical Relevance- These outcomes support the possibility to automatically classify colon tumor based on physiological parameters calculated using only specific wavelength bands. Hence, future image-guided colorectal surgeries can be performed with real-time multispectral imaging.

Advances in optical gastrointestinal endoscopy: a technical review

Optical endoscopy is the primary diagnostic and therapeutic tool for management of gastrointestinal (GI) malignancies. Most GI neoplasms arise from precancerous lesions; thus, technical innovations to improve detection and diagnosis of precancerous lesions and early cancers play a pivotal role in improving outcomes. Over the last few decades, the field of GI endoscopy has witnessed enormous and focused efforts to develop and translate accurate, user-friendly, and minimally invasive optical imaging modalities. From a technical point of view, a wide range of novel optical techniques is now available to probe different aspects of light-tissue interaction at macroscopic and microscopic scales, complementing white light endoscopy. Most of these new modalities have been successfully validated and translated to routine clinical practice. Herein, we provide a technical review of the current status of existing and promising new optical endoscopic imaging technologies for GI cancer screening and surveillance. We summarize the underlying principles of light-tissue interaction, the imaging performance at different scales, and highlight what is known about clinical applicability and effectiveness. Furthermore, we discuss recent discovery and translation of novel molecular probes that have shown promise to augment endoscopists' ability to diagnose GI lesions with high specificity. We also review and discuss the role and potential clinical integration of artificial intelligence-based algorithms to provide decision support in real time. Finally, we provide perspectives on future technology development and its potential to transform endoscopic GI cancer detection and diagnosis.

Microenvironmental modulation of the developing tumour: an immune-stromal dialogue

Successful establishment of a tumour relies on a cascade of interactions between cancer cells and stromal cells within an evolving microenvironment. Both immune and nonimmune cellular components are key factors in this process, and the individual players may change their role from tumour elimination to tumour promotion as the microenvironment develops. While the tumour-stroma crosstalk present in an established tumour is well-studied, aspects in the early tumour or premalignant microenvironment have received less attention. This is in part due to the challenges in studying this process in the clinic or in mouse models. Here, we review the key anti- and pro-tumour factors in the early microenvironment and discuss how understanding this process may be exploited in the clinic.

Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.

First experience in clinical application of hyperspectral endoscopy for evaluation of colonic polyps

Early detection and resection of adenomatous polyps prevents their progression to colorectal cancer (CRC), significantly improving patient outcomes. Polyps are typically identified and removed during white-light colonoscopy. Unfortunately, the rate of interval cancers that arise between CRC screening events remains high, linked to poor visualization of polyps during screening and incomplete polyp removal. Here, we sought to evaluate the potential of a hyperspectral endoscope (HySE) to enhance polyp discrimination for detection and resection. We designed, built and tested a new compact HySE in a proof-of-concept clinical study. We successfully collected spectra from three tissue types in seven patients undergoing routine colonoscopy screening. The acquired spectral data from normal tissue and polyps, both pre- and post- resection, were subjected to quantitative analysis using spectral angle mapping and machine learning, which discriminated the data by tissue type, meriting further investigation of HySE as a clinical tool.

Hyperspectral Mapping for the Detection of SARS-CoV-2 Using Nanomolecular Probes with Yoctomole Sensitivity

Efficient monitoring of SARS-CoV-2 outbreak requires the use of a sensitive and rapid diagnostic test. Although SARS-CoV-2 RNA can be detected by RT-qPCR, the molecular-level quantification of the viral load is still challenging, time-consuming, and labor-intensive. Here, we report an ultrasensitive hyperspectral sensor (HyperSENSE) based on hafnium nanoparticles (HfNPs) for specific detection of COVID-19 causative virus, SARS-CoV-2. Density functional theoretical calculations reveal that HfNPs exhibit higher changes in their absorption wavelength and light scattering when bound to their target SARS-CoV-2 RNA sequence relative to the gold nanoparticles. The assay has a turnaround time of a few seconds and has a limit of detection in the yoctomolar range, which is 1 000 000-fold times higher than the currently available COVID-19 tests. We demonstrated in ∼100 COVID-19 clinical samples that the assay is highly sensitive and has a specificity of 100%. We also show that HyperSENSE can rapidly detect other viruses such as influenza A H1N1. The outstanding sensitivity indicates the potential of the current biosensor in detecting the prevailing presymptomatic and asymptomatic COVID-19 cases. Thus, integrating hyperspectral imaging with nanomaterials establishes a diagnostic platform for ultrasensitive detection of COVID-19 that can potentially be applied to any emerging infectious pathogen.

Deep Learning Analysis of In Vivo Hyperspectral Images for Automated Intraoperative Nerve Detection

Nerves are critical structures that may be difficult to recognize during surgery. Inadvertent nerve injuries can have catastrophic consequences for the patient and lead to life-long pain and a reduced quality of life. Hyperspectral imaging (HSI) is a non-invasive technique combining photography with spectroscopy, allowing non-invasive intraoperative biological tissue property quantification. We show, for the first time, that HSI combined with deep learning allows nerves and other tissue types to be automatically recognized in in vivo hyperspectral images. An animal model was used, and eight anesthetized pigs underwent neck midline incisions, exposing several structures (nerve, artery, vein, muscle, fat, skin). State-of-the-art machine learning models were trained to recognize these tissue types in HSI data. The best model was a convolutional neural network (CNN), achieving an overall average sensitivity of 0.91 and a specificity of 1.0, validated with leave-one-patient-out cross-validation. For the nerve, the CNN achieved an average sensitivity of 0.76 and a specificity of 0.99. In conclusion, HSI combined with a CNN model is suitable for in vivo nerve recognition.

Reflectance spectra analysis for mucous assessment

This review report represents an overview of research and development on medical hyperspectral imaging technology and its applications. Spectral imaging technology is attracting attention as a new imaging modality for medical applications, especially in disease diagnosis and image-guided surgery. Considering the recent advances in imaging, this technology provides an opportunity for two-dimensional mapping of oxygen saturation (SatO₂) of blood with high accuracy, spatial spectral imaging, and its analysis and provides detection and diagnostic information about the tissue physiology and morphology. Multispectral imaging also provides information about tissue oxygenation, perfusion, and potential function during surgery. Analytical algorithm has been examined, and indication of accurate map of relative hemoglobin concentration and SatO₂ can be indicated with preferable resolution and frame rate. This technology is expected to provide promising biomedical information in practical use. Several studies suggested that blood flow and SatO₂ are associated with gastrointestinal disorders, particularly malignant tumor conditions. The use and analysis of spectroscopic images are expected to potentially play a role in the detection and diagnosis of these diseases.

Intraoperative hyperspectral label-free imaging: from system design to first-in-patient translation

Despite advances in intraoperative surgical imaging, reliable discrimination of critical tissue during surgery remains challenging. As a result, decisions with potentially life-changing consequences for patients are still based on the surgeon's subjective visual assessment. Hyperspectral imaging (HSI) provides a promising solution for objective intraoperative tissue characterisation, with the advantages of being non-contact, non-ionising and non-invasive. However, while its potential to aid surgical decision-making has been investigated for a range of applications, to date no real-time intraoperative HSI (iHSI) system has been presented that follows critical design considerations to ensure a satisfactory integration into the surgical workflow. By establishing functional and technical requirements of an intraoperative system for surgery, we present an iHSI system design that allows for real-time wide-field HSI and responsive surgical guidance in a highly constrained operating theatre. Two systems exploiting state-of-the-art industrial HSI cameras, respectively using linescan and snapshot imaging technology, were designed and investigated by performing assessments against established design criteria and ex vivo tissue experiments. Finally, we report the use of our real-time iHSI system in a clinical feasibility case study as part of a spinal fusion surgery. Our results demonstrate seamless integration into existing surgical workflows.

Over 1000 nm Near-Infrared Multispectral Imaging System for Laparoscopic In Vivo Imaging

In this study, a laparoscopic imaging device and a light source able to select wavelengths by bandpass filters were developed to perform multispectral imaging (MSI) using over 1000 nm near-infrared (OTN-NIR) on regions under a laparoscope. Subsequently, MSI (wavelengths: 1000–1400 nm) was performed using the built device on nine live mice before and after tumor implantation. The normal and tumor pixels captured within the mice were used as teaching data sets, and the tumor-implanted mice data were classified using a neural network applied following a leave-one-out cross-validation procedure. The system provided a specificity of 89.5%, a sensitivity of 53.5%, and an accuracy of 87.8% for subcutaneous tumor discrimination. Aggregated true-positive (TP) pixels were confirmed in all tumor-implanted mice, which indicated that the laparoscopic OTN-NIR MSI could potentially be applied in vivo for classifying target lesions such as cancer in deep tissues.

Skin Complications of Diabetes Mellitus Revealed by Polarized Hyperspectral Imaging and Machine Learning

Aging and diabetes lead to protein glycation and cause dysfunction of collagen-containing tissues. The accompanying structural and functional changes of collagen significantly contribute to the development of various pathological malformations affecting the skin, blood vessels, and nerves, causing a number of complications, increasing disability risks and threat to life. In fact, no methods of non-invasive assessment of glycation and associated metabolic processes in biotissues or prediction of possible skin complications, e.g., ulcers, currently exist for endocrinologists and clinical diagnosis. In this publication, utilizing emerging photonics-based technology, innovative solutions in machine learning, and definitive physiological characteristics, we introduce a diagnostic approach capable of evaluating the skin complications of diabetes mellitus at the very earlier stage. The results of the feasibility studies, as well as the actual tests on patients with diabetes and healthy volunteers, clearly show the ability of the approach to differentiate diabetic and control groups. Furthermore, the developed in-house polarization-based hyperspectral imaging technique accomplished with the implementation of the artificial neural network provides new horizons in the study and diagnosis of age-related diseases.

Pose estimation of a markerless fiber bundle for endoscopic optical biopsy

Purpose: We present a markerless vision-based method for on-the-fly three-dimensional (3D) pose estimation of a fiberscope instrument to target pathologic areas in the endoscopic view during exploration. Approach: A 2.5-mm-diameter fiberscope is inserted through the endoscope's operating channel and connected to an additional camera to perform complementary observation of a targeted area such as a multimodal magnifier. The 3D pose of the fiberscope is estimated frame-by-frame by maximizing the similarity between its silhouette (automatically detected in the endoscopic view using a deep learning neural network) and a cylindrical shape bound to a kinematic model reduced to three degrees-of-freedom. An alignment of the cylinder axis, based on Plücker coordinates from the straight edges detected in the image, makes convergence faster and more reliable. Results: The performance on simulations has been validated with a virtual trajectory mimicking endoscopic exploration and on real images of a chessboard pattern acquired with different endoscopic configurations. The experiments demonstrated a good accuracy and robustness of the proposed algorithm with errors of 0.33 ± 0.68    mm in distance position and 0.32 ± 0.11   deg in axis orientation for the 3D pose estimation, which reveals its superiority over previous approaches. This allows multimodal image registration with sufficient accuracy of < 3    pixels . Conclusion: Our pose estimation pipeline was executed on simulations and patterns; the results demonstrate the robustness of our method and the potential of fiber-optical instrument image-based tracking for pose estimation and multimodal registration. It can be fully implemented in software and therefore easily integrated into a routine clinical environment.

Feedforward Artificial Neural Network-Based Colorectal Cancer Detection Using Hyperspectral Imaging: A Step towards Automatic Optical Biopsy

Simple Summary

Detection of colorectal carcinoma is performed visually by investigators and is confirmed pathologically. With hyperspectral imaging, an expanded spectral range of optical information is now available for analysis. The acquired recordings were analyzed with a neural network, and it was possible to differentiate tumor from healthy mucosa in colorectal carcinoma by automatic classification with high reliability. Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. This is a step towards optical biopsy.

Abstract

Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.

Photoacoustic endoscopy: A progress review

Endoscopy has been widely used in biomedical imaging and integrated with various optical and acoustic imaging modalities. Photoacoustic imaging (PAI), one of the fastest growing biomedical imaging modalities, is a noninvasive and nonionizing method that owns rich optical contrast, deep acoustic penetration depth, multiscale and multiparametric imaging capability. Hence, it is preferred to miniaturize the volume of PAI and develop an emerged endoscopic imaging modality referred to as photoacoustic endoscopy (PAE). It has been developed for more than one decade since the first report of PAE. Unfortunately, until now, there is no mature photoacoustic endoscopic technique recognized in clinic due to various technical limitations. To address this concern, recent development of new scanning mechanisms, adoption of novel optical/acoustic devices, utilization of superior computation methods and exploration of multimodality strategies have significantly promoted the progress of PAE toward clinic. In this review, we comprehensively reviewed recent progresses in single- and multimodality PAE with new physics, mechanisms and strategies to achieve practical devices for potential applicable scenarios including esophageal, gastrointestinal, urogenital and intravascular imaging. We ended this review with challenges and prospects for future development of PAE.

Snapshot multidimensional photography through active optical mapping

Multidimensional photography can capture optical fields beyond the capability of conventional image sensors that measure only two-dimensional (2D) spatial distribution of light. By mapping a high-dimensional datacube of incident light onto a 2D image sensor, multidimensional photography resolves the scene along with other information dimensions, such as wavelength and time. However, the application of current multidimensional imagers is fundamentally restricted by their static optical architectures and measurement schemes—the mapping relation between the light datacube voxels and image sensor pixels is fixed. To overcome this limitation, we propose tunable multidimensional photography through active optical mapping. A high-resolution spatial light modulator, referred to as an active optical mapper, permutes and maps the light datacube voxels onto sensor pixels in an arbitrary and programmed manner. The resultant system can readily adapt the acquisition scheme to the scene, thereby maximising the measurement flexibility. Through active optical mapping, we demonstrate our approach in two niche implementations: hyperspectral imaging and ultrafast imaging.

Multidimensional photography has traditionally been restricted by their static optical architectures and measurement schemes. Here, the authors present a tunable multidimensional photography approach employing active optical mapping, which allows them to adapt the acquisition schemes to the scene.

In vivo detection of murine glioblastoma through Raman and reflectance fiber-probe spectroscopies

Significance: Glioblastoma (GBM) is the most common and aggressive malignant brain tumor in adults. With a worldwide incidence rate of 2 to 3 per 100,000 people, it accounts for more than 60% of all brain cancers; currently, its 5-year survival rate is < 5 % . GBM treatment relies mainly on surgical resection. In this framework, multimodal optical spectroscopy could provide a fast and label-free tool for improving tumor detection and guiding the removal of diseased tissues. Aim: Discriminating healthy brain from GBM tissues in an animal model through the combination of Raman and reflectance spectroscopies. Approach: EGFP-GL261 cells were injected into the brains of eight laboratory mice for inducing murine GBM in these animals. A multimodal optical fiber probe combining fluorescence, Raman, and reflectance spectroscopy was used to localize in vivo healthy and tumor brain areas and to collect their spectral information. Results: Tumor areas were localized through the detection of EGFP fluorescence emission. Then, Raman and reflectance spectra were collected from healthy and tumor tissues, and later analyzed through principal component analysis and linear discriminant analysis in order to develop a classification algorithm. Raman and reflectance spectra resulted in 92% and 93% classification accuracy, respectively. Combining together these techniques allowed improving the discrimination between healthy and tumor tissues up to 97%. Conclusions: These preliminary results demonstrate the potential of multimodal fiber-probe spectroscopy for in vivo label-free detection and delineation of brain tumors, and thus represent an additional, encouraging step toward clinical translation and deployment of fiber-probe spectroscopy.

Hyperspectral evaluation of hepatic oxygenation in a model of total vs. arterial liver ischaemia

Liver ischaemia reperfusion injury (IRI) is a dreaded pathophysiological complication which may lead to an impaired liver function. The level of oxygen hypoperfusion affects the level of cellular damage during the reperfusion phase. Consequently, intraoperative localisation and quantification of oxygen impairment would help in the early detection of liver ischaemia. To date, there is no real-time, non-invasive, and intraoperative tool which can compute an organ oxygenation map, quantify and discriminate different types of vascular occlusions intraoperatively. Hyperspectral imaging (HSI) is a non-invasive optical methodology which can quantify tissue oxygenation and which has recently been applied to the medical field. A hyperspectral camera detects the relative reflectance of a tissue in the range of 500 to 1000 nm, allowing the quantification of organic compounds such as oxygenated and deoxygenated haemoglobin at different depths. Here, we show the first comparative study of liver oxygenation by means of HSI quantification in a model of total vascular inflow occlusion (VIO) vs. hepatic artery occlusion (HAO), correlating optical properties with capillary lactate and histopathological evaluation. We found that liver HSI could discriminate between VIO and HAO. These results were confirmed via cross-validation of HSI which detected and quantified intestinal congestion in VIO. A significant correlation between the near-infrared spectra and capillary lactate was found (r = − 0.8645, p = 0.0003 VIO, r = − 0.7113, p = 0.0120 HAO). Finally, a statistically significant negative correlation was found between the histology score and the near-infrared parameter index (NIR) (r = − 0.88, p = 0.004). We infer that HSI, by predicting capillary lactates and the histopathological score, would be a suitable non-invasive tool for intraoperative liver perfusion assessment.

Polarizer-Free AOTF-Based SWIR Hyperspectral Imaging for Biomedical Applications

Optical biomedical imaging in short wave infrared (SWIR) range within 0.9–1.7 μm is a rapidly developing technique. For this reason, there is an increasing interest in cost-effective and robust hardware for hyperspectral imaging data acquisition in this range. Tunable-filter-based solutions are of particular interest as they provide image processing flexibility and effectiveness in terms of collected data volume. Acousto-optical tunable filters (AOTFs) provide a unique set of features necessary for high-quality SWIR hyperspectral imaging. In this paper, we discuss a polarizer-free configuration of an imaging AOTF that provides a compact and easy-to-integrate design of the whole imager. We have carried out image quality analysis of this system, assembled it and validated its efficiency through multiple experiments. The developed system can be helpful in many hyperspectral applications including biomedical analyses.

Integrated photoacoustic and hyperspectral dual-modality microscopy for co-imaging of melanoma and cutaneous squamous cell carcinoma in vivo

Skin carcinoma such as melanoma (MM) and cutaneous squamous cell carcinoma (cSCC) are considered as the highest mortality and the most aggressive skin cancers in dermatology. In view that early diagnosis and treatment can greatly improve the survival rate and life quality of the patients, developing noninvasive and effective evaluation methods is of great significance for the detection and identification of early stage cutaneous cancers. In this article, we propose a hybrid photoacoustic and hyperspectral dual-modality microscopy to evaluate and differentiate skin carcinoma by structural and multiphysiological parameters. The proposed system's imaging abilities are verified by mimic phantoms and normal mice experiments. Furthermore, in vivo characterization and evaluation results of MM and cSCC mice are obtained successfully, which prove this novel method could be used as a reliable and useful method for skin cancer detection in early stages.

Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range

SIGNIFICANCE

Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.

AIM

An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.

APPROACH

Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.

RESULTS

The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).

CONCLUSIONS

Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.