Machine learning approach for rapid and accurate estimation of optical properties using spatial frequency domain imaging

Learnable real-time inference of molecular composition from diffuse spectroscopy of brain tissue

Significance

Diffuse optical modalities such as broadband near-infrared spectroscopy (bNIRS) and hyperspectral imaging (HSI) represent a promising alternative for low-cost, non-invasive, and fast monitoring of living tissue. Particularly, the possibility of extracting the molecular composition of the tissue from the optical spectra deems the spectroscopy techniques as a unique diagnostic tool.

Aim

No established method exists to streamline the inference of the biochemical composition from the optical spectrum for real-time applications such as surgical monitoring. We analyze a machine learning technique for inference of changes in the molecular composition of brain tissue.

Approach

We propose modifications to the existing learnable methodology based on the Beer-Lambert law. We evaluate the method's applicability to linear and nonlinear formulations of this physical law. The approach is tested on data obtained from the bNIRS- and HSI-based monitoring of brain tissue.

Results

The results demonstrate that the proposed method enables real-time molecular composition inference while maintaining the accuracy of traditional methods. Preliminary findings show that Beer-Lambert law-based spectral unmixing allows contrasting brain anatomy semantics such as the vessel tree and tumor area.

Conclusion

We present a data-driven technique for inferring molecular composition change from diffuse spectroscopy of brain tissue, potentially enabling intra-operative monitoring.

Reconstruction of Optical Properties in Turbid Media: Omitting the Need of the Collimated Transmission for an Integrating Sphere Setup

Currently, the most reliable approach to reconstruct optical properties, namely absorption coefficient, reduced scattering coefficient, scattering coefficient and asymmetry factor, of turbid media is through inverse Monte Carlo simulation. To determine these optical properties, three measurements are required: total transmission, total reflection and collimated transmission. However, the accurate determination of the collimated transmission is very difficult. To overcome the difficulty of measuring the collimated transmission, it is proposed to measure the total transmission and total reflection of the same sample with two different thicknesses instead. To prove this alternative solution, machine learning is used to prove that the repeated measurement for two different thicknesses carries all the necessary information. As a result, all four optical properties can be measured with high accuracy, particularly for interpolation problems where the test data fall within the range of the training data. For extrapolation problems, high accuracy can be achieved in the determination of at least the absorption coefficient, reduced scattering coefficient and scattering coefficient. Hence, these results allow that in the future, an easier and therefore more precise reconstruction of the optical properties is possible, potentially even with inverse Monte Carlo simulations as the current standard.

Tutorial on methods for estimation of optical absorption and scattering properties of tissue

Significance

The estimation of tissue optical properties using diffuse optics has found a range of applications in disease detection, therapy monitoring, and general health care. Biomarkers derived from the estimated optical absorption and scattering coefficients can reflect the underlying progression of many biological processes in tissues.

Aim

Complex light-tissue interactions make it challenging to disentangle the absorption and scattering coefficients, so dedicated measurement systems are required. We aim to help readers understand the measurement principles and practical considerations needed when choosing between different estimation methods based on diffuse optics.

Approach

The estimation methods can be categorized as: steady state, time domain, time frequency domain (FD), spatial domain, and spatial FD. The experimental measurements are coupled with models of light-tissue interactions, which enable inverse solutions for the absorption and scattering coefficients from the measured tissue reflectance and/or transmittance.

Results

The estimation of tissue optical properties has been applied to characterize a variety of ex vivo and in vivo tissues, as well as tissue-mimicking phantoms. Choosing a specific estimation method for a certain application has to trade-off its advantages and limitations.

Conclusion

Optical absorption and scattering property estimation is an increasingly important and accessible approach for medical diagnosis and health monitoring.

Designing a use-error robust machine learning model for quantitative analysis of diffuse reflectance spectra

Significance

Machine learning (ML)-enabled diffuse reflectance spectroscopy (DRS) is increasingly used as an alternative to the computation-intensive inverse Monte Carlo (MCI) simulation to predict tissue's optical properties, including the absorption coefficient, μ a and reduced scattering coefficient, μ s ' .

Aim

We aim to develop a use-error-robust ML algorithm for optical property prediction from DRS spectra.

Approach

We developed a wavelength-independent regressor (WIR) to predict optical properties from DRS data. For validation, we generated 1520 simulated DRS spectra with the forward Monte Carlo model, where μ a = 0.44 to 2.45    cm - 1 , and μ s ' = 6.53 to 9.58    cm - 1 . We introduced common use-errors, such as wavelength miscalibrations and intensity fluctuations. Finally, we collected 882 experimental DRS images from 170 tissue-mimicking phantoms and compared performances of the WIR model, a dense neural network, and the MCI model.

Results

When compounding all use-errors on simulated data, the WIR model best balanced accuracy and speed, yielding errors of 1.75% for μ a and 1.53% for μ s ' , compared to the MCI's 50.9% for μ a and 24.6% for μ s ' . Regarding experimental data, WIR model had mean errors of 13.2% and 6.1% for μ a and μ s ' , respectively. The errors for MCI were about eight times higher.

Conclusions

The WIR model presents reliable use-error-robust optical property predictions from DRS data.

Neural network-based inverse model for diffuse reflectance spectroscopy

In diffuse reflectance spectroscopy, the retrieval of the optical properties of a target requires the inversion of a measured reflectance spectrum. This is typically achieved through the use of forward models such as diffusion theory or Monte Carlo simulations, which are iteratively applied to optimize the solution for the optical parameters. In this paper, we propose a novel neural network-based approach for solving this inverse problem, and validate its performance using experimentally measured diffuse reflectance data from a previously reported phantom study. Our inverse model was developed from a neural network forward model that was pre-trained with data from Monte Carlo simulations. The neural network forward model then creates a lookup table to invert the diffuse reflectance to the optical coefficients. We describe the construction of the neural network-based inverse model and test its ability to accurately retrieve optical properties from experimentally acquired diffuse reflectance data in liquid optical phantoms. Our results indicate that the developed neural network-based model achieves comparable accuracy to traditional Monte Carlo-based inverse model while offering improved speed and flexibility, potentially providing an alternative for developing faster clinical diagnosis tools. This study highlights the potential of neural networks in solving inverse problems in diffuse reflectance spectroscopy.

Physics-guided neural network for tissue optical properties estimation

Finding the optical properties of tissue is essential for various biomedical diagnostic/therapeutic applications such as monitoring of blood oxygenation, tissue metabolism, skin imaging, photodynamic therapy, low-level laser therapy, and photo-thermal therapy. Hence, the research for more accurate and versatile optical properties estimation techniques has always been a primary interest of researchers, especially in the field of bioimaging and bio-optics. In the past, most of the prediction methods were based on physics-based models such as the pronounced diffusion approximation method. In more recent years, with the advancement and growing popularity of machine learning techniques, most of the prediction methods are data-driven. While both methods have been proven to be useful, each of them suffers from several shortcomings that could be complemented by their counterparts. Thus, there is a need to bring the two domains together to obtain superior prediction accuracy and generalizability. In this work, we proposed a physics-guided neural network (PGNN) for tissue optical properties regression which integrates physics prior and constraint into the artificial neural network (ANN) model. With this method, we have demonstrated superior generalizability of PGNN compared to its pure ANN counterpart. The prediction accuracy and generalizability of the network were evaluated on single-layered tissue samples simulated with Monte Carlo simulation. Two different test datasets, the in-domain test dataset and out-domain dataset were used to evaluate in-domain generalizability and out-domain generalizability, respectively. The physics-guided neural network (PGNN) showed superior generalizability for both in-domain and out-domain prediction compared to pure ANN.

Model for the diffuse reflectance in spatial frequency domain imaging

Significance

In spatial frequency domain imaging (SDFI), tissue is illuminated with sinusoidal intensity patterns at different spatial frequencies. For low spatial frequencies, the reflectance is diffuse and a model derived by Cuccia et al. (doi 10.1117/1.3088140) is commonly used to extract optical properties. An improved model resulting in more accurate optical property extraction could lead to improved diagnostic algorithms.

Aim

To develop a model that improves optical property extraction for the diffuse reflectance in SFDI compared to the model of Cuccia et al.

Approach

We derive two analytical models for the diffuse reflectance, starting from the theoretical radial reflectance R ( ρ ) for a pencil-beam illumination under the partial current boundary condition (PCBC) and the extended boundary condition (EBC). We compare both models and the model of Cuccia et al. to Monte Carlo simulations.

Results

The model based on the PCBC resulted in the lowest errors, improving median relative errors compared to the model of Cuccia et al. by 45% for the reflectance, 10% for the reduced scattering coefficient and 64% for the absorption coefficient.

Conclusions

For the diffuse reflectance in SFDI, the model based on the PCBC provides more accurate results than the currently used model by Cuccia et al.

OpticalBERT and OpticalTable-SQA: Text- and Table-Based Language Models for the Optical-Materials Domain

Text mining in the optical-materials domain is becoming increasingly important as the number of scientific publications in this area grows rapidly. Language models such as Bidirectional Encoder Representations from Transformers (BERT) have opened up a new era and brought a significant boost to state-of-the-art natural-language-processing (NLP) tasks. In this paper, we present two "materials-aware" text-based language models for optical research, OpticalBERT and OpticalPureBERT, which are trained on a large corpus of scientific literature in the optical-materials domain. These two models outperform BERT and previous state-of-the-art models in a variety of text-mining tasks about optical materials. We also release the first "materials-aware" table-based language model, OpticalTable-SQA. This is a querying facility that solicits answers to questions about optical materials using tabular information that pertains to this scientific domain. The OpticalTable-SQA model was realized by fine-tuning the Tapas-SQA model using a manually annotated OpticalTableQA data set which was curated specifically for this work. While preserving its sequential question-answering performance on general tables, the OpticalTable-SQA model significantly outperforms Tapas-SQA on optical-materials-related tables. All models and data sets are available to the optical-materials-science community.

Quantification of bowel ischaemia using real-time multispectral Single Snapshot Imaging of Optical Properties (SSOP)

BACKGROUND

Single snapshot imaging of optical properties (SSOP) is a relatively new non-invasive, real-time, contrast-free optical imaging technology, which allows for the real-time quantitative assessment of physiological properties, including tissue oxygenation (StO2). This study evaluates the accuracy of multispectral SSOP in quantifying bowel ischaemia in a preclinical experimental model.

METHODS

In six pigs, an ischaemic bowel segment was created by dividing the arcade branches. Five regions of interest (ROIs) were identified on the bowel loop, as follows: ROI 1: central ischaemic; ROI 2: left marginal; ROI 3: left vascularised; ROI 4: right marginal; and ROI 5: right vascularised. The Trident imaging system, specifically developed for real-time tissue oxygenation imaging using SSOP, was used to image before (T0) and after ischaemia induction. Capillary and systemic lactates were measured at each time point (T0, T15, T30, T45, T60), as well as StO2 values acquired by means of SSOP (SSOP-StO2).

RESULTS

The mean value of SSOP-StO2 in ROI 1 was 30.08 ± 6.963 and was significantly lower when compared to marginal ROIs (ROI 2 + ROI 4: 45.67 ± 10.02 p =  < 0.0001), and to vascularised ROIs (ROI 3 + ROI 5: 48.08 ± 7.083 p =  < 0.0001). SSOP-StO2 was significantly correlated with normalised lactates r = - 0.5892 p < 0.0001 and with histology r =- 0.6251 p = 0.0002.

CONCLUSION

Multispectral SSOP allows for a contrast-free accurate assessment of small bowel perfusion identifying physiological tissue oxygenation as confirmed with perfusion biomarkers.

Extracting Tissue Optical Properties and Detecting Bruised Tissue in Pears Quickly and Accurately Based on Spatial Frequency Domain Imaging and Machine Learning

Recently, Spatial Frequency Domain Imaging (SFDI) has gradually become an alternative method to extract tissue optical properties (OPs), as it provides a wide-field, no-contact acquisition. SFDI extracts OPs by least-square fitting (LSF) based on the diffuse approximation equation, but there are shortcomings in the speed and accuracy of extracting OPs. This study proposed a Long Short-term Memory Regressor (LSTMR) solution to extract tissue OPs. This method allows for fast and accurate extraction of tissue OPs. Firstly, the imaging system was developed, which is more compact and portable than conventional SFDI systems. Next, numerical simulation was performed using the Monte Carlo forward model to obtain the dataset, and then the mapping model was established using the dataset. Finally, the model was applied to detect the bruised tissue of 'crown' pears. The results show that the mean absolute errors of the absorption coefficient and the reduced scattering coefficient are no more than 0.32% and 0.21%, and the bruised tissue of 'crown' pears can be highlighted by the change of OPs. Compared with the LSF, the speed of extracting tissue OPs is improved by two orders of magnitude, and the accuracy is greatly improved. The study contributes to the rapid and accurate extraction of tissue OPs based on SFDI and has great potential in food safety assessment.

Rapid extraction of skin physiological parameters from hyperspectral images using machine learning

Noninvasive assessment of skin structure using hyperspectral images has been intensively studied in recent years. Due to the high computational cost of the classical methods, such as the inverse Monte Carlo (IMC), much research has been done with the aim of using machine learning (ML) methods to reduce the time required for estimating parameters. This study aims to evaluate the accuracy and the estimation speed of the ML methods for this purpose and compare them to the traditionally used inverse adding-doubling (IAD) algorithm. We trained three models – an artificial neural network (ANN), a 1D convolutional neural network (CNN), and a random forests (RF) model – to predict seven skin parameters. The models were trained on simulated data computed using the adding-doubling algorithm. To improve predictive performance, we introduced a stacked dynamic weighting (SDW) model combining the predictions of all three individually trained models. SDW model was trained by using only a handful of real-world spectra on top of the ANN, CNN and RF models that were trained using simulated data. Models were evaluated based on the estimated parameters’ mean absolute error (MAE), considering the surface inclination angle and comparing skin spectra with spectra fitted by the IAD algorithm. On simulated data, the lowest MAE was achieved by the RF model (0.0030), while the SDW model achieved the lowest MAE on in vivo measured spectra (0.0113). The shortest time to estimate parameters for a single spectrum was 93.05 μs. Results suggest that ML algorithms can produce accurate estimates of human skin optical parameters in near real-time.

Training generative adversarial networks for optical property mapping using synthetic image data

We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.

A database of refractive indices and dielectric constants auto-generated using ChemDataExtractor

The ability to auto-generate databases of optical properties holds great potential for advancing optical research, especially with regards to the data-driven discovery of optical materials. An optical property database of refractive indices and dielectric constants is presented, which comprises a total of 49,076 refractive index and 60,804 dielectric constant data records on 11,054 unique chemicals. The database was auto-generated using the state-of-the-art natural language processing software, ChemDataExtractor, using a corpus of 388,461 scientific papers. The data repository offers a representative overview of the information on linear optical properties that resides in scientific papers from the past 30 years. Public availability of these data will enable a quick search for the optical property of certain materials. The large size of this repository will accelerate data-driven research on the design and prediction of optical materials and their properties. To the best of our knowledge, this is the first auto-generated database of optical properties from a large number of scientific papers. We provide a web interface to aid the use of our database.

Measurement(s)	refractive indices • dielectric constants
Technology Type(s)	natural language processing

Toward the development of a polarimetric tool to diagnose the fibrotic human ventricular myocardium

SIGNIFICANCE

Optical polarimetry is an emerging modality that effectively quantifies the bulk optical properties that correlate with the anisotropic structural properties of cardiac tissues. We demonstrate the application of a polarimetric tool for characterizing healthy and fibrotic human myocardial tissues efficiently with a high degree of accuracy.

AIM

The study was aimed to characterize the myocardial tissues from the left ventricle and right ventricle of N  =  7 control and N  =  10 diseased subjects. The diseased subjects were composed of two groups: N  =  7 with rheumatic heart disease (RHD) and N  =  3 with myxomatous valve (MV) disease.

APPROACH

A portable, affordable, and accurate linear polarization-based diagnostic tool is developed to measure the degree of linear polarization (DOLP) of the myocardial tissues while working at a wavelength of 850 nm.

RESULTS

The sensitivity, specificity, and accuracy of the polarimetric tool in distinguishing the control group from the RHD group were found to be 73.33%, 76.92%, and 75%, respectively, and from the MV group were 91.6%, 62.5%, and 80%, respectively, which demonstrates the efficacy of the polarimetric tool to distinguish the healthy myocardial tissues from diseased tissues.

CONCLUSIONS

We have successfully developed a polarimetric tool that can aid cardiologists in characterizing the myocardial tissues in conjunction with endomyocardial biopsy. This work should be followed up with experiments on a large cohort of control and diseased subjects. We intend to create and develop a probe to quantify the DOLP of in vivo heart tissue during surgery.

Multispectral snapshot imaging of skin microcirculatory hemoglobin oxygen saturation using artificial neural networks trained on in vivo data

SIGNIFICANCE

Developing algorithms for estimating blood oxygenation from snapshot multispectral imaging (MSI) data is challenging due to the complexity of sensor characteristics and photon transport modeling in tissue. We circumvent this using a method where artificial neural networks (ANNs) are trained on in vivo MSI data with target values from a point-measuring reference method.

AIM

To develop and evaluate a methodology where a snapshot filter mosaic camera is utilized for imaging skin hemoglobin oxygen saturation (SO2), using ANNs.

APPROACH

MSI data were acquired during occlusion provocations. ANNs were trained to estimate SO2 with MSI data as input, targeting data from a validated probe-based reference system. Performance of ANNs with different properties and training data sets was compared.

RESULTS

The method enables spatially resolved estimation of skin tissue SO2. Results are comparable to those acquired using a Monte-Carlo-based approach when relevant training data are used.

CONCLUSIONS

Training an ANN on in vivo MSI data covering a wide range of target values acquired during an occlusion protocol enable real-time estimation of SO2 maps. Data from the probe-based reference system can be used as target despite differences in sampling depth and measurement position.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Spatial frequency domain imaging technology based on Fourier single-pixel imaging

SIGNIFICANCE

Optical properties (absorption coefficient and scattering coefficient) of tissue are the most critical parameters for disease diagnosis-based optical method. In recent years, researchers proposed spatial frequency domain imaging (SFDI) to quantitatively map tissue optical properties in a broad field of contactless imaging. To solve the limitations in wavebands unsuitable for silicon-based sensor technology, a compressed sensing (CS) algorithm is used to reproduce the original signal by a single-pixel detectors. Currently, the existing single-pixel SFDI method mainly uses a random sampling policy to extract and recover signals in the acquisition stage. However, these methods are memory-hungry and time-consuming, and they cannot generate discernible results under low sampling rate. Explorations on high performance and efficiency single-pixel SFDI are of great significance for clinical application.

AIM

Fourier single-pixel imaging can reconstruct signals with less time and space costs and has fewer reconstruction errors. We focus on an SFDI algorithm based on Fourier single-pixel imaging and propose our Fourier single-pixel image-based spatial frequency domain imaging method (FSI-SFDI).

APPROACH

First, we use Fourier single-pixel imaging algorithm to collect and compress signals and SFDI algorithm to generate optical parameters. Given the basis that the main energy of general image signals is concentrated in the range of low frequency of Fourier frequency domain, our FSI-SFDI uses a circular-sampling scheme to sample data points in the low-frequency region. Then, we reconstruct the image details from these points by optimization-based inverse-FFT method.

RESULTS

Our algorithm is tested on simulated data. Results show that the root mean square error (RMSE) of optical parameters is lower than 5% when the data reduction is 92%, and it can generate discernible optical parameter image with low sampling rate. We can observe that our FSI-SFDI primarily recovers the optical properties while keeping the RMSE under the upper bound of 4.5% when we use an image with 512  ×  512 resolution as the example for calculation and analysis. Not only that but also our algorithm consumes less space and time for an image with 256  ×  256 resolution, the signal reconstruction takes only 1.65 ms, and requires less RAM memory. Compared to CS-SFDI method, our FSI-SFDI can reduce the required number of measurements through optimizing algorithm.

CONCLUSIONS

Moreover, FSI-SFDI is capable of recovering high-quality resolvable images with lower sampling rate, higher-resolution images with less memory and time consumed than previous CS-SFDI method, which is very promising for clinical data collection and medical analysis.

In vivo two-dimensional quantitative imaging of skin and cutaneous microcirculation with perturbative spatial frequency domain imaging (p-SFDI)

We introduce perturbative spatial frequency domain imaging (p-SFDI) for fast two-dimensional (2D) mapping of the optical properties and physiological characteristics of skin and cutaneous microcirculation using spatially modulated visible light. Compared to the traditional methods for recovering 2D maps through a pixel-by-pixel inversion, p-SFDI significantly shortens parameter retrieval time, largely avoids the random fitting errors caused by measurement noise, and enhances the image reconstruction quality. The efficacy of p-SFDI is demonstrated by in vivo imaging forearm of one healthy subject, recovering the 2D spatial distribution of cutaneous hemoglobin concentration, oxygen saturation, scattering properties, the melanin content, and the epidermal thickness over a large field of view. Furthermore, the temporal and spatial variations in physiological parameters under the forearm reactive hyperemia protocol are revealed, showing its applications in monitoring temporal and spatial dynamics.

Imaging sub-diffuse optical properties of cancerous and normal skin tissue using machine learning-aided spatial frequency domain imaging

SIGNIFICANCE

Sub-diffuse optical properties may serve as useful cancer biomarkers, and wide-field heatmaps of these properties could aid physicians in identifying cancerous tissue. Sub-diffuse spatial frequency domain imaging (sd-SFDI) can reveal such wide-field maps, but the current time cost of experimentally validated methods for rendering these heatmaps precludes this technology from potential real-time applications.

AIM

Our study renders heatmaps of sub-diffuse optical properties from experimental sd-SFDI images in real time and reports these properties for cancerous and normal skin tissue subtypes.

APPROACH

A phase function sampling method was used to simulate sd-SFDI spectra over a wide range of optical properties. A machine learning model trained on these simulations and tested on tissue phantoms was used to render sub-diffuse optical property heatmaps from sd-SFDI images of cancerous and normal skin tissue.

RESULTS

The model accurately rendered heatmaps from experimental sd-SFDI images in real time. In addition, heatmaps of a small number of tissue samples are presented to inform hypotheses on sub-diffuse optical property differences across skin tissue subtypes.

CONCLUSION

These results bring the overall process of sd-SFDI a fundamental step closer to real-time speeds and set a foundation for future real-time medical applications of sd-SFDI such as image guided surgery.

Machine learning for real-time optical property recovery in interstitial photodynamic therapy: a stimulation-based study

With the continued development of non-toxic photosensitizer drugs, interstitial photodynamic therapy (iPDT) is showing more favorable outcomes in recent clinical trials. IPDT planning is crucial to further increase the treatment efficacy. However, it remains a major challenge to generate a high-quality, patient-specific plan due to uncertainty in tissue optical properties (OPs), µ a and µ s . These parameters govern how light propagates inside tissues, and any deviation from the planning-assumed values during treatment could significantly affect the treatment outcome. In this work, we increase the robustness of iPDT against OP variations by using machine learning models to recover the patient-specific OPs from light dosimetry measurements and then re-optimizing the diffusers' optical powers to adapt to these OPs in real time. Simulations on virtual brain tumor models show that reoptimizing the power allocation with the recovered OPs significantly reduces uncertainty in the predicted light dosimetry for all tissues involved.

Diffuse reflectance and machine learning techniques to differentiate colorectal cancer ex vivo

In this study, we used machine learning techniques to reconstruct the wavelength dependence of the absorption coefficient of human normal and pathological colorectal mucosa tissues. Using only diffuse reflectance spectra from the ex vivo mucosa tissues as input to algorithms, several approaches were tried before obtaining good matching between the generated absorption coefficients and the ones previously calculated for the mucosa tissues from invasive experimental spectral measurements. Considering the optimized match for the results generated with the multilayer perceptron regression method, we were able to identify differentiated accumulation of lipofuscin in the absorption coefficient spectra of both mucosa tissues as we have done before with the corresponding results calculated directly from invasive measurements. Considering the random forest regressor algorithm, the estimated absorption coefficient spectra almost matched the ones previously calculated. By subtracting the absorption of lipofuscin from these spectra, we obtained similar hemoglobin ratios at 410/550 nm: 18.9-fold/9.3-fold for the healthy mucosa and 46.6-fold/24.2-fold for the pathological mucosa, while from direct calculations, those ratios were 19.7-fold/10.1-fold for the healthy mucosa and 33.1-fold/17.3-fold for the pathological mucosa. The higher values obtained in this study indicate a higher blood content in the pathological samples used to measure the diffuse reflectance spectra. In light of such accuracy and sensibility to the presence of hidden absorbers, with a different accumulation between healthy and pathological tissues, good perspectives become available to develop minimally invasive spectroscopy methods for in vivo early detection and monitoring of colorectal cancer.

Machine learning estimation of tissue optical properties

Dynamic, in vivo measurement of the optical properties of biological tissues is still an elusive and critically important problem. Here we develop a technique for inverting a Monte Carlo simulation to extract tissue optical properties from the statistical moments of the spatio-temporal response of the tissue by training a 5-layer fully connected neural network. We demonstrate the accuracy of the method across a very wide parameter space on a single homogeneous layer tissue model and demonstrate that the method is insensitive to parameter selection of the neural network model itself. Finally, we propose an experimental setup capable of measuring the required information in real time in an in vivo environment and demonstrate proof-of-concept level experimental results.

Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning

SIGNIFICANCE

Spatial frequency-domain imaging (SFDI) is a powerful technique for mapping tissue oxygen saturation over a wide field of view. However, current SFDI methods either require a sequence of several images with different illumination patterns or, in the case of single-snapshot optical properties (SSOP), introduce artifacts and sacrifice accuracy.

AIM

We introduce OxyGAN, a data-driven, content-aware method to estimate tissue oxygenation directly from single structured-light images.

APPROACH

OxyGAN is an end-to-end approach that uses supervised generative adversarial networks. Conventional SFDI is used to obtain ground truth tissue oxygenation maps for ex vivo human esophagi, in vivo hands and feet, and an in vivo pig colon sample under 659- and 851-nm sinusoidal illumination. We benchmark OxyGAN by comparing it with SSOP and a two-step hybrid technique that uses a previously developed deep learning model to predict optical properties followed by a physical model to calculate tissue oxygenation.

RESULTS

When tested on human feet, cross-validated OxyGAN maps tissue oxygenation with an accuracy of 96.5%. When applied to sample types not included in the training set, such as human hands and pig colon, OxyGAN achieves a 93% accuracy, demonstrating robustness to various tissue types. On average, OxyGAN outperforms SSOP and a hybrid model in estimating tissue oxygenation by 24.9% and 24.7%, respectively. Finally, we optimize OxyGAN inference so that oxygenation maps are computed ∼10 times faster than previous work, enabling video-rate, 25-Hz imaging.

CONCLUSIONS

Due to its rapid acquisition and processing speed, OxyGAN has the potential to enable real-time, high-fidelity tissue oxygenation mapping that may be useful for many clinical applications.

Real-time, wide-field and high-quality single snapshot imaging of optical properties with profile correction using deep learning

The development of real-time, wide-field and quantitative diffuse optical imaging methods to visualize functional and structural biomarkers of living tissues is a pressing need for numerous clinical applications including image-guided surgery. In this context, Spatial Frequency Domain Imaging (SFDI) is an attractive method allowing for the fast estimation of optical properties using the Single Snapshot of Optical Properties (SSOP) approach. Herein, we present a novel implementation of SSOP based on a combination of deep learning network at the filtering stage and Graphics Processing Units (GPU) capable of simultaneous high visual quality image reconstruction, surface profile correction and accurate optical property (OP) extraction in real-time across large fields of view. In the most optimal implementation, the presented methodology demonstrates megapixel profile-corrected OP imaging with results comparable to that of profile-corrected SFDI, with a processing time of 18 ms and errors relative to SFDI method less than 10% in both profilometry and profile-corrected OPs. This novel processing framework lays the foundation for real-time multispectral quantitative diffuse optical imaging for surgical guidance and healthcare applications. All code and data used for this work is publicly available at www.healthphotonics.org under the resources tab.

Wavelength weightings in machine learning for ovine joint tissue differentiation using diffuse reflectance spectroscopy (DRS)

Objective: To investigate the DRS of ovine joint tissue to determine the optimal optical wavelengths for tissue differentiation and relate these wavelengths to the biomolecular composition of tissues. In this study, we combine machine learning with DRS for tissue classification and then look further at the weighting matrix of the classifier to further understand the key differentiating features. Methods: Supervised machine learning was used to analyse DRS data. After normalising the data, dimension reduction was achieved through multiclass Fisher's linear discriminant analysis (Multiclass FLDA) and classified with linear discriminant analysis (LDA). The classifier was first run with all the tissue types and the wavelength range 190 nm - 1081 nm. We analysed the weighting matrix of the classifier and then ran the classifier again, the first time using the ten highest weighted wavelengths and the second using only the single highest. Our method was applied to a dataset containing ovine joint tissue including cartilage, cortical and subchondral bone, fat, ligament, meniscus, and muscle. Results: It achieved a classification accuracy of 100% using the wavelength 190 nm - 1081 nm (2048 attributes) with an accuracy of 90% being present for 10 attributes with the exception of those with comparable compositions such as ligament and meniscus. An accuracy greater than 70% was achieved using a single wavelength, with the same exceptions. Conclusion: Multiclass FLDA combined with LDA is a viable technique for tissue identification from DRS data. The majority of differentiating features existed within the wavelength ranges 370-470 and 800-1010 nm. Focusing on key spectral regions means that a spectrometer with a narrower range can potentially be used, with less computational power needed for subsequent analysis.

Real-time skin chromophore estimation from hyperspectral images using a neural network

BACKGROUND

Hyperspectral imaging for in vivo human skin study has shown great potential by providing non-invasive measurement from which information usually invisible to the human eye can be revealed. In particular, maps of skin parameters including oxygen rate, blood volume fraction, and melanin concentration can be estimated from a hyperspectral image by using an optical model and an optimization algorithm. These applications, relying on hyperspectral images acquired with a high-resolution camera especially dedicated to skin measurement, have yielded promising results. However, the data analysis process is relatively expensive in terms of computation cost, with calculation of full-face skin property maps requiring up to 5 hours for 3-megapixels hyperspectral images. Such a computation time prevents punctual previewing and quality assessment of the maps immediately after acquisition.

METHODS

To address this issue, we have implemented a neural network that models the optimization-based analysis algorithm. This neural network has been trained on a set of hyperspectral images, acquired from 204 patients and their corresponding skin parameter maps, which were calculated by optimization.

RESULTS

The neural network is able to generate skin parameter maps that are visually very faithful to the reference maps much more quickly than the optimization-based algorithm, with computation times as short as 2 seconds for a 3-megapixel image representing a full face and 0.5 seconds for a 1-megapixel image representing a smaller area of skin. The average deviation calculated on selected areas shows the network's promising generalization ability, even on wide-field full-face images.

CONCLUSION

Currently, the network is adequate for preview purposes, providing relatively accurate results in a few seconds.

GANPOP: Generative Adversarial Network Prediction of Optical Properties From Single Snapshot Wide-Field Images

We present a deep learning framework for wide-field, content-aware estimation of absorption and scattering coefficients of tissues, called Generative Adversarial Network Prediction of Optical Properties (GANPOP). Spatial frequency domain imaging is used to obtain ground-truth optical properties at 660 nm from in vivo human hands and feet, freshly resected human esophagectomy samples, and homogeneous tissue phantoms. Images of objects with either flat-field or structured illumination are paired with registered optical property maps and are used to train conditional generative adversarial networks that estimate optical properties from a single input image. We benchmark this approach by comparing GANPOP to a single-snapshot optical property (SSOP) technique, using a normalized mean absolute error (NMAE) metric. In human gastrointestinal specimens, GANPOP with a single structured-light input image estimates the reduced scattering and absorption coefficients with 60% higher accuracy than SSOP while GANPOP with a single flat-field illumination image achieves similar accuracy to SSOP. When applied to both in vivo and ex vivo swine tissues, a GANPOP model trained solely on structured-illumination images of human specimens and phantoms estimates optical properties with approximately 46% improvement over SSOP, indicating adaptability to new, unseen tissue types. Given a training set that appropriately spans the target domain, GANPOP has the potential to enable rapid and accurate wide-field measurements of optical properties.

Real-time optical properties and oxygenation imaging using custom parallel processing in the spatial frequency domain

The development of real-time, wide-field and quantitative diffuse optical imaging methods is becoming increasingly popular for biological and medical applications. Recent developments introduced a novel approach for real-time multispectral acquisition in the spatial frequency domain using spatio-temporal modulation of light. Using this method, optical properties maps (absorption and reduced scattering) could be obtained for two wavelengths (665 nm and 860 nm). These maps, in turn, are used to deduce oxygen saturation levels in tissues. However, while the acquisition was performed in real-time, processing was performed post-acquisition and was not in real-time. In the present article, we present CPU and GPU processing implementations for this method with special emphasis on processing time. The obtained results show that the proposed custom direct method using a General Purpose Graphic Processing Unit (GPGPU) and C CUDA (Compute Unified Device Architecture) implementation enables 1.6 milliseconds processing time for a 1 Mega-pixel image with a maximum average error of 0.1% in extracting optical properties.

Special Section Guest Editorial: Special Section on Spatial Frequency Domain Imaging

This guest editorial introduces the Special Section on Spatial Frequency Domain Imaging.

Spatial frequency domain imaging in 2019: principles, applications, and perspectives

Spatial frequency domain imaging (SFDI) has witnessed very rapid growth over the last decade, owing to its unique capabilities for imaging optical properties and chromophores over a large field-of-view and in a rapid manner. We provide a comprehensive review of the principles of this imaging method as of 2019, review the modeling of light propagation in this domain, describe acquisition methods, provide an understanding of the various implementations and their practical limitations, and finally review applications that have been published in the literature. Importantly, we also introduce a group effort by several key actors in the field for the dissemination of SFDI, including publications, advice in hardware and implementations, and processing code, all freely available online.

Spatial frequency domain imaging in 2019: principles, applications, and perspectives

Spatial frequency domain imaging (SFDI) has witnessed very rapid growth over the last decade, owing to its unique capabilities for imaging optical properties and chromophores over a large field-of-view and in a rapid manner. We provide a comprehensive review of the principles of this imaging method as of 2019, review the modeling of light propagation in this domain, describe acquisition methods, provide an understanding of the various implementations and their practical limitations, and finally review applications that have been published in the literature. Importantly, we also introduce a group effort by several key actors in the field for the dissemination of SFDI, including publications, advice in hardware and implementations, and processing code, all freely available online.

Burn wound classification model using spatial frequency-domain imaging and machine learning

Accurate assessment of burn severity is critical for wound care and the course of treatment. Delays in classification translate to delays in burn management, increasing the risk of scarring and infection. To this end, numerous imaging techniques have been used to examine tissue properties to infer burn severity. Spatial frequency-domain imaging (SFDI) has also been used to characterize burns based on the relationships between histologic observations and changes in tissue properties. Recently, machine learning has been used to classify burns by combining optical features from multispectral or hyperspectral imaging. Rather than employ models of light propagation to deduce tissue optical properties, we investigated the feasibility of using SFDI reflectance data at multiple spatial frequencies, with a support vector machine (SVM) classifier, to predict severity in a porcine model of graded burns. Calibrated reflectance images were collected using SFDI at eight wavelengths (471 to 851 nm) and five spatial frequencies (0 to 0.2  mm  -  1). Three models were built from subsets of this initial dataset. The first subset included data taken at all wavelengths with the planar (0  mm  -  1) spatial frequency, the second comprised data at all wavelengths and spatial frequencies, and the third used all collected data at values relative to unburned tissue. These data subsets were used to train and test cubic SVM models, and compared against burn status 28 days after injury. Model accuracy was established through leave-one-out cross-validation testing. The model based on images obtained at all wavelengths and spatial frequencies predicted burn severity at 24 h with 92.5% accuracy. The model composed of all values relative to unburned skin was 94.4% accurate. By comparison, the model that employed only planar illumination was 88.8% accurate. This investigation suggests that the combination of SFDI with machine learning has potential for accurately predicting burn severity.

Single snapshot imaging of optical properties using a single-pixel camera: a simulation study

We present the effects of using a single-pixel camera approach to extract optical properties with the single-snapshot spatial frequency-domain imaging method. We acquired images of a human hand for spatial frequencies ranging from 0.1 to 0.4  mm  -  1 with increasing compression ratios using adaptive basis scan wavelet prediction strategy. In summary, our findings indicate that the extracted optical properties remained usable up to 99% of compression rate at a spatial frequency of 0.2  mm  -  1 with errors of 5% in reduced scattering and 10% in absorption.

Real-time, wide-field, and quantitative oxygenation imaging using spatiotemporal modulation of light

Quantitative diffuse optical imaging has the potential to provide valuable functional information about tissue status, such as oxygen saturation or blood content to healthcare practitioners in real time. However, significant technical challenges have so far prevented such tools from being deployed in the clinic. Toward achieving this goal, prior research introduced methods based on spatial frequency domain imaging (SFDI) that allow real-time (within milliseconds) wide-field imaging of optical properties but at a single wavelength. However, for this technology to be useful to clinicians, images must be displayed in terms of metrics related to the physiological state of the tissue, hence interpretable to guide decision-making. For this purpose, recent developments introduced multispectral SFDI methods for rapid imaging of oxygenation parameters up to 16 frames per seconds (fps). We introduce real-time, wide-field, and quantitative blood parameters imaging using spatiotemporal modulation of light. Using this method, we are able to quantitatively obtain optical properties at 100 fps at two wavelengths (665 and 860 nm), and therefore oxygenation, oxyhemoglobin, and deoxyhemoglobin, using a single camera with, at most, 4.2% error in comparison with standard SFDI acquisitions.