Efficient estimation of subdiffusive optical parameters in real time from spatially resolved reflectance by artificial neural networks

Application of transfer learning for rapid calibration of spatially resolved diffuse reflectance probes for extraction of tissue optical properties

Significance

Treatment planning for light-based therapies including photodynamic therapy requires tissue optical property knowledge. This is recoverable with spatially resolved diffuse reflectance spectroscopy (DRS) but requires precise source-detector separation (SDS) determination and time-consuming simulations.

Aim

An artificial neural network (ANN) to map from DRS at multiple SDS to optical properties was created. This trained ANN was adapted to fiber-optic probes with varying SDS using transfer learning (TL).

Approach

An ANN mapping from measurements to Monte Carlo simulation to optical properties was created with one fiber-optic probe. A second probe with different SDS was used for TL algorithm creation. Data from a third were used to test this algorithm.

Results

The initial ANN recovered absorber concentration with RMSE = 0.29    μ M (7.5% mean error) and μ s ' at 665 nm (μ s , 665 ' ) with RMSE = 0.77    cm - 1 (2.5% mean error). For probe 2, TL significantly improved absorber concentration (0.38 versus 1.67    μ M RMSE, p = 0.0005 ) and μ ' s , 665 (0.71 versus 1.8    cm - 1 RMSE, p = 0.0005 ) recovery. A third probe also showed improved absorber (0.7 versus 4.1    μ M RMSE, p < 0.0001 ) and μ s , 665 ' (1.68 versus 2.08    cm - 1 RMSE, p = 0.2 ) recovery.

Conclusions

TL-based probe-to-probe calibration can rapidly adapt an ANN created for one probe to similar target probes, enabling accurate optical property recovery with the target probe.

Application of Transfer Learning for Rapid Calibration of Spatially-resolved Diffuse Reflectance Probes for Extraction of Tissue Optical Properties

Significance

Treatment planning for light-based therapies including photodynamic therapy requires tissue optical property knowledge. These are recoverable with spatially-resolved diffuse reflectance spectroscopy (DRS), but requires precise source-detector separation (SDS) determination and time-consuming simulations.

Aim

An artificial neural network (ANN) to map from DRS at short SDS to optical properties was created. This trained ANN was adapted to fiber-optic probes with varying SDS using transfer learning.

Approach

An ANN mapping from measurements to Monte Carlo simulation to optical properties was created with one fiber-optic probe. A second probe with different SDS was used for transfer learning algorithm creation. Data from a third were used to test this algorithm.

Results

The initial ANN recovered absorber concentration with RMSE=0.29 µM (7.5% mean error) and µ s ' at 665 nm (µ s,665 ' ) with RMSE=0.77 cm -1 (2.5% mean error). For probe-2, transfer learning significantly improved absorber concentration (0.38 vs. 1.67 µM, p=0.0005) and µ s,665 ' (0.71 vs. 1.8 cm -1 , p=0.0005) recovery. A third probe also showed improved absorber (0.7 vs. 4.1 µM, p<0.0001) and µ s,665 ' (1.68 vs. 2.08 cm -1 , p=0.2) recovery.

Conclusions

A data-driven approach to optical property extraction can be used to rapidly calibrate new fiber-optic probes with varying SDS, with as few as three calibration spectra.

Neural network-based inverse model for diffuse reflectance spectroscopy

In diffuse reflectance spectroscopy, the retrieval of the optical properties of a target requires the inversion of a measured reflectance spectrum. This is typically achieved through the use of forward models such as diffusion theory or Monte Carlo simulations, which are iteratively applied to optimize the solution for the optical parameters. In this paper, we propose a novel neural network-based approach for solving this inverse problem, and validate its performance using experimentally measured diffuse reflectance data from a previously reported phantom study. Our inverse model was developed from a neural network forward model that was pre-trained with data from Monte Carlo simulations. The neural network forward model then creates a lookup table to invert the diffuse reflectance to the optical coefficients. We describe the construction of the neural network-based inverse model and test its ability to accurately retrieve optical properties from experimentally acquired diffuse reflectance data in liquid optical phantoms. Our results indicate that the developed neural network-based model achieves comparable accuracy to traditional Monte Carlo-based inverse model while offering improved speed and flexibility, potentially providing an alternative for developing faster clinical diagnosis tools. This study highlights the potential of neural networks in solving inverse problems in diffuse reflectance spectroscopy.

Neural network-based optimization of sub-diffuse reflectance spectroscopy for improved parameter prediction and efficient data collection

In this study, a general and systematical investigation of sub-diffuse reflectance spectroscopy is implemented. A Gegenbauer-kernel phase function-based Monte Carlo is adopted to describe photon transport more efficiently. To improve the computational efficiency and accuracy, two neural network algorithms, namely, back propagation neural network and radial basis function neural network are utilized to predict the absorption coefficient μ a , reduced scattering coefficient μ s ' and sub-diffusive quantifier γ , simultaneously, at multiple source-detector separations (SDS). The predicted results show that the three parameters can be predicated accurately by selecting five SDSs or above. Based on the simulation results, a four wavelength (520, 650, 785 and 830 nm) measurement system using five SDSs is designed by adopting phase-lock-in technique. Furtherly, the trained neural-network models are utilized to extract optical properties from the phantom and in vivo experimental data. The results verify the feasibility and effectiveness of our proposed system and methods in mucosal disease diagnosis.

Real-time and accurate estimation ex vivo of four basic optical properties from thin tissue based on a cascade forward neural network

Double integrating sphere measurements obtained from thin ex vivo tissues provides more spectral information and hence allows full estimation of all basic optical properties (OPs) theoretically. However, the ill-conditioned nature of the OP determination increases excessively with the reduction in tissue thickness. Therefore, it is crucial to develop a model for thin ex vivo tissues that is robust to noise. Herein, we present a deep learning solution to precisely extract four basic OPs in real-time from thin ex vivo tissues, leveraging a dedicated cascade forward neural network (CFNN) for each OP with an additional introduced input of the refractive index of the cuvette holder. The results show that the CFNN-based model enables accurate and fast evaluation of OPs, as well as robustness to noise. Our proposed method overcomes the highly ill-conditioned restriction of OP evaluation and can distinguish the effects of slight changes in measurable quantities without any a priori knowledge.

Ultracompact Deep Neural Network for Ultrafast Optical Property Extraction in Spatial Frequency Domain Imaging (SFDI)

Spatial frequency domain imaging (SFDI) is a powerful, label-free imaging technique capable of the wide-field quantitative mapping of tissue optical properties and, subsequently, chromophore concentrations. While SFDI hardware acquisition methods have advanced towards video-rate, the inverse problem (i.e., the mapping of acquired diffuse reflectance to optical properties) has remained a bottleneck for real-time data processing and visualization. Deep learning methods are adept at fitting nonlinear patterns, and may be ideal for rapidly solving the SFDI inverse problem. While current deep neural networks (DNN) are growing increasingly larger and more complex (e.g., with millions of parameters or more), our study shows that it can also be beneficial to move in the other direction, i.e., make DNNs that are smaller and simpler. Here, we propose an ultracompact, two-layer, fully connected DNN structure (each layer with four and two neurons, respectively) for ultrafast optical property extractions, which is 30×–600× faster than current methods with a similar or improved accuracy, allowing for an inversion time of 5.5 ms for 696 × 520 pixels. We further demonstrated the proposed inverse model in numerical simulations, and comprehensive phantom characterization, as well as offering in vivo measurements of dynamic physiological processes. We further demonstrated that the computation time could achieve another 200× improvement with a GPU device. This deep learning structure will help to enable fast and accurate real-time SFDI measurements, which are crucial for pre-clinical, clinical, and industrial applications.

Phase function estimation from a diffuse optical image via deep learning

Objective. The phase function is a key element of a light propagation model for Monte Carlo (MC) simulation, which is usually fitted with an analytic function with associated parameters. In recent years, machine learning methods were reported to estimate the parameters of the phase function of a particular form such as the Henyey–Greenstein phase function but, to our knowledge, no studies have been performed to determine the form of the phase function. Approach. Here we design a convolutional neural network (CNN) to estimate the phase function from a diffuse optical image without any explicit assumption on the form of the phase function. Specifically, we use a Gaussian mixture model (GMM) as an example to represent the phase function generally and learn the model parameters accurately. The GMM is selected because it provides the analytic expression of phase function to facilitate deflection angle sampling in MC simulation, and does not significantly increase the number of free parameters. Main Results. Our proposed method is validated on MC-simulated reflectance images of typical biological tissues using the Henyey–Greenstein phase function with different anisotropy factors. The mean squared error of the phase function is 0.01 and the relative error of the anisotropy factor is 3.28%. Significance. We propose the first data-driven CNN-based inverse MC model to estimate the form of scattering phase function. The effects of field of view and spatial resolution are analyzed and the findings provide guidelines for optimizing the experimental protocol in practical applications.

MCDataset: a public reference dataset of Monte Carlo simulated quantities for multilayered and voxelated tissues computed by massively parallel PyXOpto Python package

SIGNIFICANCE

Current open-source Monte Carlo (MC) method implementations for light propagation modeling are many times tedious to build and require third-party licensed software that can often discourage prospective researchers in the biomedical optics community from fully utilizing the light propagation tools. Furthermore, the same drawback also limits rigorous cross-validation of physical quantities estimated by various MC codes.

AIM

Proposal of an open-source tool for light propagation modeling and an easily accessible dataset to encourage fruitful communications amongst researchers and pave the way to a more consistent comparison between the available implementations of the MC method.

APPROACH

The PyXOpto implementation of the MC method for multilayered and voxelated tissues based on the Python programming language and PyOpenCL extension enables massively parallel computation on numerous OpenCL-enabled devices. The proposed implementation is used to compute a large dataset of reflectance, transmittance, energy deposition, and sampling volume for various source, detector, and tissue configurations.

RESULTS

The proposed PyXOpto agrees well with the original MC implementation. However, further validation reveals a noticeable bias introduced by the random number generator used in the original MC implementation.

CONCLUSIONS

Establishing a common dataset is highly important for the validation of existing and development of MC codes for light propagation in turbid media.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Analysis of diffuse reflectance spectroscopy by means of Bayesian inference and separation of the parameters for scattering strength and spectral dependence of the scattering

For many applications in tissue optics, knowledge of the scattering coefficient or at least the reduced scattering coefficient is essential. In addition, its spectral dependence is also an important feature, as this provides information about scatterer sizes. While the characterization of the spectral dependence should be a simple fit, experimental results show strong fluctuations even for the same tissue type. For in-vivo measurements, this problem is even greater. Therefore, it is the aim of this study to analyze the instabilities of the scattering characterizations and find a solution for it by means of Bayesian inference. In this study, this behavior is investigated using the example of diffuse reflectance spectroscopy. It can be shown that the currently used fitting functions are unstable for the fitting as both parameters for characterizing the reduced scattering coefficient describe the spectral dependence as well as the scattering strength. However, by a simple coordinate transform, a stable mathematical description of the scattering is derived. By the fact that a posteriori probability of the reduced scattering coefficient narrows down significantly with the Bayesian inference, the new fitting function is verified.

Imaging sub-diffuse optical properties of cancerous and normal skin tissue using machine learning-aided spatial frequency domain imaging

SIGNIFICANCE

Sub-diffuse optical properties may serve as useful cancer biomarkers, and wide-field heatmaps of these properties could aid physicians in identifying cancerous tissue. Sub-diffuse spatial frequency domain imaging (sd-SFDI) can reveal such wide-field maps, but the current time cost of experimentally validated methods for rendering these heatmaps precludes this technology from potential real-time applications.

AIM

Our study renders heatmaps of sub-diffuse optical properties from experimental sd-SFDI images in real time and reports these properties for cancerous and normal skin tissue subtypes.

APPROACH

A phase function sampling method was used to simulate sd-SFDI spectra over a wide range of optical properties. A machine learning model trained on these simulations and tested on tissue phantoms was used to render sub-diffuse optical property heatmaps from sd-SFDI images of cancerous and normal skin tissue.

RESULTS

The model accurately rendered heatmaps from experimental sd-SFDI images in real time. In addition, heatmaps of a small number of tissue samples are presented to inform hypotheses on sub-diffuse optical property differences across skin tissue subtypes.

CONCLUSION

These results bring the overall process of sd-SFDI a fundamental step closer to real-time speeds and set a foundation for future real-time medical applications of sd-SFDI such as image guided surgery.

Direct mapping from diffuse reflectance to chromophore concentrations in multi-fx spatial frequency domain imaging (SFDI) with a deep residual network (DRN)

Spatial frequency domain imaging (SFDI) is an emerging technology that enables label-free, non-contact, and wide-field mapping of tissue chromophore contents, such as oxy- and deoxy-hemoglobin concentrations. It has been shown that the use of more than two spatial frequencies (multi-fx ) can vastly improve measurement accuracy and reduce chromophore estimation uncertainties, but real-time multi-fx SFDI for chromophore monitoring has been limited in practice due to the slow speed of available chromophore inversion algorithms. Existing inversion algorithms have to first convert the multi-fx diffuse reflectance to optical absorptions, and then solve a set of linear equations to estimate chromophore concentrations. In this work, we present a deep learning framework, noted as a deep residual network (DRN), that is able to directly map from diffuse reflectance to chromophore concentrations. The proposed DRN is over 10x faster than the state-of-the-art method for chromophore inversion and enables 25x improvement on the frame rate for in vivo real-time oxygenation mapping. The proposed deep learning model will help enable real-time and highly accurate chromophore monitoring with multi-fx SFDI.

Optical properties of PlatSil SiliGlass tissue-mimicking phantoms

In this work, we revise the preparation procedure and conduct an in depth characterization of optical properties for the recently proposed silicone-based tissue-mimicking optical phantoms in the spectral range from 475 to 925 nm. The optical properties are characterized in terms of refractive index and its temperature dependence, absorption and reduced scattering coefficients and scattering phase function related quantifiers. The scattering phase function and related quantifiers of the optical phantoms are first assessed within the framework of the Mie theory by using the measured refractive index of SiliGlass and size distribution of the hollow silica spherical particles that serve as scatterers. A set of purely absorbing optical phantoms in cuvettes is used to evaluate the linearity of the absorption coefficient with respect to the concentration of black pigment that serves as the absorber. Finally, the optical properties in terms of the absorption and reduced scattering coefficients and the subdiffusive scattering phase function quantifier γ are estimated for a subset of phantoms from spatially resolved reflectance using deep learning aided inverse models. To this end, an optical fiber probe with six linearly arranged optical fibers is used to collect the backscattered light at small and large distances from the source fiber. The underlying light propagation modeling is based on the stochastic Monte Carlo method that accounts for all the details of the optical fiber probe.

Fast and accurate Monte Carlo simulations of subdiffusive spatially resolved reflectance for a realistic optical fiber probe tip model aided by a deep neural network

In this work, we introduce a framework for efficient and accurate Monte Carlo (MC) simulations of spatially resolved reflectance (SRR) acquired by optical fiber probes that account for all the details of the probe tip including reflectivity of the stainless steel and the properties of the epoxy fill and optical fibers. While using full details of the probe tip is essential for accurate MC simulations of SRR, the break-down of the radial symmetry in the detection scheme leads to about two orders of magnitude longer simulation times. The introduced framework mitigates this performance degradation, by an efficient reflectance regression model that maps SRR obtained by fast MC simulations based on a simplified probe tip model to SRR simulated using the full details of the probe tip. We show that a small number of SRR samples is sufficient to determine the parameters of the regression model. Finally, we use the regression model to simulate SRR for a stainless steel optical probe with six linearly placed fibers and experimentally validate the framework through the use of inverse models for estimation of absorption and reduced scattering coefficients and subdiffusive scattering phase function quantifiers.

Multidiameter single-fiber reflectance spectroscopy of heavily pigmented skin: modeling the inhomogeneous distribution of melanin

When analyzing multidiameter single-fiber reflectance (MDSFR) spectra, the inhomogeneous distribution of melanin pigments in skin tissue is usually not accounted for. Especially in heavily pigmented skins, this can result in bad fits and biased estimation of tissue optical properties. A model is introduced to account for the inhomogeneous distribution of melanin pigments in skin tissue. In vivo visible MDSFR measurements were performed on heavily pigmented skin of type IV to VI. Skin tissue optical properties and related physiological properties were extracted from the measured spectra using the introduced model. The absorption of melanin pigments described by the introduced model demonstrates a good correlation with the co-localized measurement of the well-known melanin index.

Fast and precise image generation of blood vessels embedded in skin

A software for fast rendering the visual appearance of a blood vessel located in human skin was developed based on a numerical solution of the radiative transfer equation. The user can specify geometrical properties, such as the depth and the diameter of the vessel, and physiological properties, such as the oxygen saturation of the vessel or the blood concentration in the skin. From these data, the spatially and spectrally resolved reflectance from the skin containing the blood vessel is calculated via Monte Carlo simulations, by which a two-dimensional image is generated. The short calculation time of about a second is achieved by precalculating and storing the spatially resolved reflectance for a variety of combinations of the optical and geometrical properties. This concept gives the user the opportunity to rapidly explore the influence of the physiological and geometrical properties of the investigated blood vessel on its visual appearance. The correctness of the lookup table was validated by comparison with independent Monte Carlo simulations. Rendering examples of different blood vessels in human skins are given. The current version of the software can be downloaded at https://www.ilm-ulm.de/software.

Deep learning model for ultrafast multifrequency optical property extractions for spatial frequency domain imaging

Spatial frequency domain imaging (SFDI) is emerging as an important new method in biomedical imaging due to its ability to provide label-free, wide-field tissue optical property maps. Most prior SFDI studies have utilized two spatial frequencies (2-f_x) for optical property extractions. The use of more than two frequencies (multi-f_x) can vastly improve the accuracy and reduce uncertainties in optical property estimates for some tissue types, but it has been limited in practice due to the slow speed of available inversion algorithms. We present a deep learning solution that eliminates this bottleneck by solving the multi-f_x inverse problem 300× to 100,000× faster, with equivalent or improved accuracy compared to competing methods. The proposed deep learning inverse model will help to enable real-time and highly accurate tissue measurements with SFDI.