Comparison of image registration methods for combining laparoscopic video and spectral image data

Comparative Analysis of Color Space and Channel, Detector, and Descriptor for Feature-Based Image Registration

The current study aimed to quantify the value of color spaces and channels as a potential superior replacement for standard grayscale images, as well as the relative performance of open-source detectors and descriptors for general feature-based image registration purposes, based on a large benchmark dataset. The public dataset UDIS-D, with 1106 diverse image pairs, was selected. In total, 21 color spaces or channels including RGB, XYZ, Y'CrCb, HLS, L*a*b* and their corresponding channels in addition to grayscale, nine feature detectors including AKAZE, BRISK, CSE, FAST, HL, KAZE, ORB, SIFT, and TBMR, and 11 feature descriptors including AKAZE, BB, BRIEF, BRISK, DAISY, FREAK, KAZE, LATCH, ORB, SIFT, and VGG were evaluated according to reprojection error (RE), root mean square error (RMSE), structural similarity index measure (SSIM), registration failure rate, and feature number, based on 1,950,984 image registrations. No meaningful benefits from color space or channel were observed, although XYZ, RGB color space and L* color channel were able to outperform grayscale by a very minor margin. Per the dataset, the best-performing color space or channel, detector, and descriptor were XYZ/RGB, SIFT/FAST, and AKAZE. The most robust color space or channel, detector, and descriptor were L*a*b*, TBMR, and VGG. The color channel, detector, and descriptor with the most initial detector features and final homography features were Z/L*, FAST, and KAZE. In terms of the best overall unfailing combinations, XYZ/RGB+SIFT/FAST+VGG/SIFT seemed to provide the highest image registration quality, while Z+FAST+VGG provided the most image features.

Registration of hyperspectral images and mass spectrometry data for the correlation of tissue optical spectra and molecular profiles

Hyperspectral imaging (HSI) is a label-free imaging modality that is emerging for non-invasive detection of various diseases including cancers. HSI provides high-resolution spatial images where each pixel has a spectral curve with numerous wavelength bands from the visible to infrared ranges. The rich spatial and spectral information can be used to discriminate various types of tissues and pathophysiological conditions. However, it can be difficult to explain spectral data with respect to the underline cellular and molecular mechanism. In this study, we developed an approach that registers hyperspectral images and mass spectrometry (MS) data where MS provides tissue molecular profiles. Human prostate tissues that were obtained after prostatectomy were used in the experiments. The whole prostate was first sliced every six mm. A customized hyperspectral surgical microscope was used to acquire HSI data from the sliced tissue. For MS data analysis, the sliced tissue of the prostate was divided into 51 small regions and then processed separately for each region. The immediately adjacent tissue was sliced and processed histologically for H&E staining. The MS molecular profiles were correlated with the hyperspectral images in this study.

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

Significance

Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.

Aim

To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.

Approach

The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.

Results

The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps.

Conclusions

High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.

The Safe Values of Quantitative Perfusion Parameters of ICG Angiography Based on Tissue Oxygenation of Hyperspectral Imaging for Laparoscopic Colorectal Surgery: A Prospective Observational Study

BACKGROUND

Safe values for quantitative perfusion parameters of indocyanine green (ICG) angiography have not been fully defined, and interpretation remains at the surgeon's discretion. This prospective observational study aimed to establish the safe values for the quantitative perfusion parameters by comparing tissue oxygenation levels from HSI images in laparoscopic colorectal surgery.

METHODS

ICG angiography was performed using a laparoscopic near-infrared (NIR) camera system with ICG diluted in 10 mL of distilled water. For quantitative perfusion parameters, the changes in fluorescence intensity with perfusion times were analyzed to plot a time-fluorescence intensity graph. To assess real-time tissue oxygen saturation (StO2) in the colon, the TIVITA® Tissue System was utilized for hyperspectral imaging (HSI) acquisition. The StO2 levels were compared with the quantitative perfusion parameters derived from ICG angiography at corresponding points to define the safe range of ICG parameters reflecting good tissue oxygenation.

RESULTS

In the regression analysis, T1/2MAX, TMAX, slope, and NIR perfusion index were correlated with tissue oxygen saturation. Using this regression model, the cutoff values of quantitative perfusion parameters were calculated as T1/2MAX ≤ 10 s, TMAX ≤ 30 s, slope ≥ 5, and NIR perfusion index ≥50, which best reflected colon StO2 higher than 60%. Diagnostic values were analyzed to predict colon StO2 of 60% or more, and the ICG perfusion parameters T1/2MAX, TMAX, and perfusion TR showed high sensitivity values of 97% or more, indicating their ability to correctly identify cases with acceptable StO2.

CONCLUSION

The safe values for quantitative perfusion parameters derived from ICG angiography were T1/2MAX ≤ 10 s and TMAX ≤ 30 s, which were associated with colon tissue oxygenation levels higher than 60% in the laparoscopic colorectal surgery.