Optimized spectral filter design enables more accurate estimation of oxygen saturation in spectral imaging

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

Significance

Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.

Aim

To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.

Approach

The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.

Results

The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156    μ m . The effective frame rate was 20 fps.

Conclusions

High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.

Enhancing intraoperative tumor delineation with multispectral short-wave infrared fluorescence imaging and machine learning

SIGNIFICANCE

Fluorescence-guided surgery (FGS) provides specific real-time visualization of tumors, but intensity-based measurement of fluorescence is prone to errors. Multispectral imaging (MSI) in the short-wave infrared (SWIR) has the potential to improve tumor delineation by enabling machine-learning classification of pixels based on their spectral characteristics.

AIM

Determine whether MSI can be applied to FGS and combined with machine learning to provide a robust method for tumor visualization.

APPROACH

A multispectral SWIR fluorescence imaging device capable of collecting data from six spectral filters was constructed and deployed on neuroblastoma (NB) subcutaneous xenografts ( n = 6 ) after the injection of a NB-specific NIR-I fluorescent probe (Dinutuximab-IRDye800). We constructed image cubes representing fluorescence collected from ∼ 850 to 1450 nm and compared the performance of seven learning-based methods for pixel-by-pixel classification, including linear discriminant analysis, k -nearest neighbor classification, and a neural network.

RESULTS

The spectra of tumor and non-tumor tissue were subtly different and conserved between individuals. In classification, a combine principal component analysis and k -nearest-neighbor approach with area under curve normalization performed best, achieving 97.5% per-pixel classification accuracy (97.1%, 93.5%, and 99.2% for tumor, non-tumor tissue and background, respectively).

CONCLUSIONS

The development of dozens of new imaging agents provides a timely opportunity for multispectral SWIR imaging to revolutionize next-generation FGS.

SpeCamX: mobile app that turns unmodified smartphones into multispectral imagers

We present the development of SpeCamX, a mobile application that enables an unmodified smartphone into a multispectral imager. Multispectral imaging provides detailed spectral information about objects or scenes, but its accessibility has been limited due to its specialized requirements for the device. SpeCamX overcomes this limitation by utilizing the RGB photographs captured by smartphones and converting them into multispectral images spanning a range of 420 to 680 nm without a need for internal modifications or external attachments. The app also includes plugin functions for extracting medical information from the resulting multispectral data cube. In a clinical study, SpeCamX was used to implement an augmented smartphone bilirubinometer, predicting blood bilirubin levels (BBL) with superior performance in accuracy, efficiency and stability compared to default smartphone cameras. This innovative technology democratizes multispectral imaging, making it accessible to a wider audience and opening new possibilities for both medical and non-medical applications.

Genetic optimization of mid-infrared filters for a machine learning chemical classifier

Miniaturized mid-infrared spectrometers present opportunities for applications that range from health monitoring to agriculture. One approach combines arrays of spectral filters with infrared photodetectors, called filter-array detector-array (FADA) microspectrometers. A paper recently reported a FADA microspectrometer in tandem with machine learning for chemical identification. In that work, a FADA microspectrometer with 20 filters was assembled and tested. The filters were band-pass, or band-stop designs that evenly spanned the microspectrometer's operating wavelength range. However, given that a machine learning classifier can be trained on an arbitrary filter basis, it is not apparent that evenly spaced filters are optimal. Here, through simulations with noise, we use a genetic algorithm to optimize six bandpass filters to best identify liquid and gaseous chemicals. We report that the classifiers trained with the optimized filter sets outperform those trained with evenly spaced filter sets and those handpicked to target the absorption bands of the chemicals investigated.