Real-time, wide-field and high-quality single snapshot imaging of optical properties with profile correction using deep learning

SE-FSCNet: full-scale connection network for single-shot phase demodulation

The accuracy of phase demodulation has significant impact on the accuracy of fringe projection 3D measurement. Currently, researches based on deep learning methods for extracting wrapped phase mostly use U-Net as the subject of network. The connection method between its hierarchies has certain shortcomings in global information transmission, which hinders the improvement of wrapped phase prediction accuracy. We propose a single-shot phase demodulation method for fringe projection based on a novel full-scale connection network SE-FSCNet. The encoder and decoder of the SE-FSCNet have the same number of hierarchies but are not completely symmetrical. At the decoder a full-scale connection method and feature fusion module are designed so that SE-FSCNet has better abilities of feature transmission and utilization compared with U-Net. A channel attention module based on squeeze and excitation is also introduced to assign appropriate weights to features with different scales, which has been proved by the ablation study. The experiments conducted on the test set have demonstrated that the SE-FSCNet can achieve higher precision than the traditional Fourier transform method and the U-Net in phase demodulation.

Ultra-miniature dual-wavelength spatial frequency domain imaging for micro-endoscopy

Significance

There is a need for a cost-effective, quantitative imaging tool that can be deployed endoscopically to better detect early stage gastrointestinal cancers. Spatial frequency domain imaging (SFDI) is a low-cost imaging technique that produces near-real time, quantitative maps of absorption and reduced scattering coefficients, but most implementations are bulky and suitable only for use outside the body.

Aim

We aim to develop an ultra-miniature SFDI system comprising an optical fiber array (diameter 0.125 mm) and a micro camera (1 × 1    mm package) to displace conventionally bulky components, in particular, the projector.

Approach

First, we fabricated a prototype with an outer diameter of 3 mm, although the individual component dimensions could permit future packaging to a < 1.5    mm diameter. We developed a phase-tracking algorithm to rapidly extract images with fringe projections at three equispaced phase shifts to perform SFDI demodulation.

Results

To validate the performance, we first demonstrate comparable recovery of quantitative optical properties between our ultra-miniature system and a conventional bench-top SFDI system with an agreement of 15% and 6% for absorption and reduced scattering, respectively. Next, we demonstrate imaging of absorption and reduced scattering of tissue-mimicking phantoms providing enhanced contrast between simulated tissue types (healthy and tumour), done simultaneously at wavelengths of 515 and 660 nm. Using a support vector machine classifier, we estimate that sensitivity and specificity values of > 90 % are feasible for detecting simulated squamous cell carcinoma.

Conclusions

This device shows promise as a cost-effective, quantitative imaging tool to detect variations in optical absorption and scattering as indicators of cancer.

Relevance and utility of the in-vivo and ex-vivo optical properties of the skin reported in the literature: a review [Invited]

Imaging non-invasively into the human body is currently limited by cost (MRI and CT scan), image resolution (ultrasound), exposure to ionising radiation (CT scan and X-ray), and the requirement for exogenous contrast agents (CT scan and PET scan). Optical imaging has the potential to overcome all these issues but is currently limited by imaging depth due to the scattering and absorption properties of human tissue. Skin is the first barrier encountered by light when imaging non-invasively, and therefore a clear understanding of the way that light interacts with skin is required for progress on optical medical imaging to be made. Here we present a thorough review of the optical properties of human skin measured in-vivo and compare these to the previously collated ex-vivo measurements. Both in-vivo and ex-vivo published data show high inter- and intra-publication variability making definitive answers regarding optical properties at given wavelengths challenging. Overall, variability is highest for ex-vivo absorption measurements with differences of up to 77-fold compared with 9.6-fold for the in-vivo absorption case. The impact of this variation on optical penetration depth and transport mean free path is presented and potential causes of these inconsistencies are discussed. We propose a set of experimental controls and reporting requirements for future measurements. We conclude that a robust in-vivo dataset, measured across a broad spectrum of wavelengths, is required for the development of future technologies that significantly increase the depth of optical imaging.

Designing and simulating realistic spatial frequency domain imaging systems using open-source 3D rendering software

Spatial frequency domain imaging (SFDI) is a low-cost imaging technique that maps absorption and reduced scattering coefficients, offering improved contrast for important tissue structures such as tumours. Practical SFDI systems must cope with various imaging geometries including imaging planar samples ex vivo, imaging inside tubular lumen in vivo e.g. for endoscopy, and measuring tumours or polyps of varying morphology. There is a need for a design and simulation tool to accelerate design of new SFDI systems and simulate realistic performance under these scenarios. We present such a system implemented using open-source 3D design and ray-tracing software Blender that simulates media with realistic absorption and scattering in a wide range of geometries. By using Blender's Cycles ray-tracing engine, our system simulates effects such as varying lighting, refractive index changes, non-normal incidence, specular reflections and shadows, enabling realistic evaluation of new designs. We first demonstrate quantitative agreement between Monte-Carlo simulated absorption and reduced scattering coefficients with those simulated from our Blender system, achieving 16 % discrepancy in absorption coefficient and 18 % in reduced scattering coefficient. However, we then show that using an empirically derived look-up table the errors reduce to 1 % and 0.7 % respectively. Next, we simulate SFDI mapping of absorption, scattering and shape for simulated tumour spheroids, demonstrating enhanced contrast. Finally we demonstrate SFDI mapping inside a tubular lumen, which highlighted a important design insight: custom look-up tables must be generated for different longitudinal sections of the lumen. With this approach we achieved 2 % absorption error and 2 % scattering error. We anticipate our simulation system will aid in the design of novel SFDI systems for key biomedical applications.

Quantification of bowel ischaemia using real-time multispectral Single Snapshot Imaging of Optical Properties (SSOP)

BACKGROUND

Single snapshot imaging of optical properties (SSOP) is a relatively new non-invasive, real-time, contrast-free optical imaging technology, which allows for the real-time quantitative assessment of physiological properties, including tissue oxygenation (StO2). This study evaluates the accuracy of multispectral SSOP in quantifying bowel ischaemia in a preclinical experimental model.

METHODS

In six pigs, an ischaemic bowel segment was created by dividing the arcade branches. Five regions of interest (ROIs) were identified on the bowel loop, as follows: ROI 1: central ischaemic; ROI 2: left marginal; ROI 3: left vascularised; ROI 4: right marginal; and ROI 5: right vascularised. The Trident imaging system, specifically developed for real-time tissue oxygenation imaging using SSOP, was used to image before (T0) and after ischaemia induction. Capillary and systemic lactates were measured at each time point (T0, T15, T30, T45, T60), as well as StO2 values acquired by means of SSOP (SSOP-StO2).

RESULTS

The mean value of SSOP-StO2 in ROI 1 was 30.08 ± 6.963 and was significantly lower when compared to marginal ROIs (ROI 2 + ROI 4: 45.67 ± 10.02 p =  < 0.0001), and to vascularised ROIs (ROI 3 + ROI 5: 48.08 ± 7.083 p =  < 0.0001). SSOP-StO2 was significantly correlated with normalised lactates r = - 0.5892 p < 0.0001 and with histology r =- 0.6251 p = 0.0002.

CONCLUSION

Multispectral SSOP allows for a contrast-free accurate assessment of small bowel perfusion identifying physiological tissue oxygenation as confirmed with perfusion biomarkers.

Measurement of tissue optical properties in a wide spectral range: a review [Invited]

A distinctive feature of this review is a critical analysis of methods and results of measurements of the optical properties of tissues in a wide spectral range from deep UV to terahertz waves. Much attention is paid to measurements of the refractive index of biological tissues and liquids, the knowledge of which is necessary for the effective application of many methods of optical imaging and diagnostics. The optical parameters of healthy and pathological tissues are presented, and the reasons for their differences are discussed, which is important for the discrimination of pathologies and the demarcation of their boundaries. When considering the interaction of terahertz radiation with tissues, the concept of an effective medium is discussed, and relaxation models of the effective optical properties of tissues are presented. Attention is drawn to the manifestation of the scattering properties of tissues in the THz range and the problems of measuring the optical properties of tissues in this range are discussed. In conclusion, a method for the dynamic analysis of the optical properties of tissues under optical clearing using an application of immersion agents is presented. The main mechanisms and technologies of optical clearing, as well as examples of the successful application for differentiation of healthy and pathological tissues, are analyzed.

Training generative adversarial networks for optical property mapping using synthetic image data

We demonstrate the training of a generative adversarial network (GAN) for the prediction of optical property maps (scattering and absorption) using spatial frequency domain imaging (SFDI) image data sets that are generated synthetically with a free open-source 3D modelling and rendering software, Blender. The flexibility of Blender is exploited to simulate 5 models with real-life relevance to clinical SFDI of diseased tissue: flat samples containing a single material, flat samples containing 2 materials, flat samples containing 3 materials, flat samples with spheroidal tumours and cylindrical samples with spheroidal tumours. The last case is particularly relevant as it represents wide-field imaging inside a tubular organ e.g. the gastro-intestinal tract. In all 5 scenarios we show the GAN provides an accurate reconstruction of the optical properties from single SFDI images with a mean normalised error ranging from 1.0-1.2% for absorption and 1.1%-1.2% for scattering, resulting in visually improved contrast for tumour spheroid structures. This compares favourably with the ∼10% absorption error and ∼10% scattering error achieved using GANs on experimental SFDI data. Next, we perform a bi-directional cross-validation of our synthetically-trained GAN, retrained with 90% synthetic and 10% experimental data to encourage domain transfer, with a GAN trained fully on experimental data and observe visually accurate results with an error of 6.3%-10.3% for absorption and 6.6%-11.9% for scattering. Our synthetically trained GAN is therefore highly relevant to real experimental samples but provides the significant added benefits of large training datasets, perfect ground-truths and the ability to test realistic imaging geometries, e.g. inside cylinders, for which no conventional single-shot demodulation algorithms exist. In the future, we expect that the application of techniques such as domain adaptation or training on hybrid real-synthetic datasets will create a powerful tool for fast, accurate production of optical property maps for real clinical imaging systems.

Deep learning-based method for non-uniform motion-induced error reduction in dynamic microscopic 3D shape measurement

The non-uniform motion-induced error reduction in dynamic fringe projection profilometry is complex and challenging. Recently, deep learning (DL) has been successfully applied to many complex optical problems with strong nonlinearity and exhibits excellent performance. Inspired by this, a deep learning-based method is developed for non-uniform motion-induced error reduction by taking advantage of the powerful ability of nonlinear fitting. First, a specially designed dataset of motion-induced error reduction is generated for network training by incorporating complex nonlinearity. Then, the corresponding DL-based architecture is proposed and it contains two parts: in the first part, a fringe compensation module is developed as network pre-processing to reduce the phase error caused by fringe discontinuity; in the second part, a deep neural network is employed to extract the high-level features of error distribution and establish a pixel-wise hidden nonlinear mapping between the phase with motion-induced error and the ideal one. Both simulations and real experiments demonstrate the feasibility of the proposed method in dynamic macroscopic measurement.

Multi-laboratory performance assessment of diffuse optics instruments: the BitMap exercise

SIGNIFICANCE

Multi-laboratory initiatives are essential in performance assessment and standardization-crucial for bringing biophotonics to mature clinical use-to establish protocols and develop reference tissue phantoms that all will allow universal instrument comparison.

AIM

The largest multi-laboratory comparison of performance assessment in near-infrared diffuse optics is presented, involving 28 instruments and 12 institutions on a total of eight experiments based on three consolidated protocols (BIP, MEDPHOT, and NEUROPT) as implemented on three kits of tissue phantoms. A total of 20 synthetic indicators were extracted from the dataset, some of them defined here anew.

APPROACH

The exercise stems from the Innovative Training Network BitMap funded by the European Commission and expanded to include other European laboratories. A large variety of diffuse optics instruments were considered, based on different approaches (time domain/frequency domain/continuous wave), at various stages of maturity and designed for different applications (e.g., oximetry, spectroscopy, and imaging).

RESULTS

This study highlights a substantial difference in hardware performances (e.g., nine decades in responsivity, four decades in dark count rate, and one decade in temporal resolution). Agreement in the estimates of homogeneous optical properties was within 12% of the median value for half of the systems, with a temporal stability of <5  %   over 1 h, and day-to-day reproducibility of <3  %  . Other tests encompassed linearity, crosstalk, uncertainty, and detection of optical inhomogeneities.

CONCLUSIONS

This extensive multi-laboratory exercise provides a detailed assessment of near-infrared Diffuse optical instruments and can be used for reference grading. The dataset-available soon in an open data repository-can be evaluated in multiple ways, for instance, to compare different analysis tools or study the impact of hardware implementations.

Ultracompact Deep Neural Network for Ultrafast Optical Property Extraction in Spatial Frequency Domain Imaging (SFDI)

Spatial frequency domain imaging (SFDI) is a powerful, label-free imaging technique capable of the wide-field quantitative mapping of tissue optical properties and, subsequently, chromophore concentrations. While SFDI hardware acquisition methods have advanced towards video-rate, the inverse problem (i.e., the mapping of acquired diffuse reflectance to optical properties) has remained a bottleneck for real-time data processing and visualization. Deep learning methods are adept at fitting nonlinear patterns, and may be ideal for rapidly solving the SFDI inverse problem. While current deep neural networks (DNN) are growing increasingly larger and more complex (e.g., with millions of parameters or more), our study shows that it can also be beneficial to move in the other direction, i.e., make DNNs that are smaller and simpler. Here, we propose an ultracompact, two-layer, fully connected DNN structure (each layer with four and two neurons, respectively) for ultrafast optical property extractions, which is 30×–600× faster than current methods with a similar or improved accuracy, allowing for an inversion time of 5.5 ms for 696 × 520 pixels. We further demonstrated the proposed inverse model in numerical simulations, and comprehensive phantom characterization, as well as offering in vivo measurements of dynamic physiological processes. We further demonstrated that the computation time could achieve another 200× improvement with a GPU device. This deep learning structure will help to enable fast and accurate real-time SFDI measurements, which are crucial for pre-clinical, clinical, and industrial applications.

Prospects of Structural Similarity Index for Medical Image Analysis

An image quality matrix provides a significant principle for objectively observing an image based on an alteration between the original and distorted images. During the past two decades, a novel universal image quality assessment has been developed with the ability of adaptation with human visual perception for measuring the difference of a degraded image from the reference image, namely a structural similarity index. Structural similarity has since been widely used in various sectors, including medical image evaluation. Although numerous studies have reported the use of structural similarity as an evaluation strategy for computer-based medical images, reviews on the prospects of using structural similarity for medical imaging applications have been rare. This paper presents previous studies implementing structural similarity in analyzing medical images from various imaging modalities. In addition, this review describes structural similarity from the perspective of a family’s historical background, as well as progress made from the original to the recent structural similarity, and its strengths and drawbacks. Additionally, potential research directions in applying such similarities related to medical image analyses are described. This review will be beneficial in guiding researchers toward the discovery of potential medical image examination methods that can be improved through structural similarity index.

Rapid Quantification of Tissue Perfusion Properties with a Two-Stage Look-Up Table

Tissue perfusion properties reveal crucial information pertinent to clinical diagnosis and treatment. Multispectral spatial frequency domain imaging (SFDI) is an emerging imaging technique that has been widely used to quantify tissue perfusion properties. However, slow processing speed limits its usefulness in real-time imaging applications. In this study, we present a two-stage look-up table (LUT) approach that accurately and rapidly quantifies optical (absorption and reduced scattering maps) and perfusion (total hemoglobin and oxygen saturation maps) properties using stage-1 and stage-2 LUTs, respectively, based on reflectance images at 660 and 850 nm. The two-stage LUT can be implemented on both CPU and GPU computing platforms. Quantifying tissue perfusion properties using the simulated diffuse reflectance images, we achieved a quantification speed of 266, 174, and 74 frames per second for three image sizes 512 × 512, 1024 × 1024, and 2048 × 2048 pixels, respectively. Quantification of tissue perfusion properties was highly accurate with only 3.5% and 2.5% error for total hemoglobin and oxygen saturation quantification, respectively. The two-stage LUT has the potential to be integrated with dual-sensor cameras to enable real-time quantification of tissue hemodynamics.

3D k-space reflectance fluorescence tomography via deep learning

We report on the potential to perform image reconstruction in 3D k-space reflectance fluorescence tomography (FT) using deep learning (DL). Herein, we adopt a modified AUTOMAP architecture and develop a training methodology that leverages an open-source Monte-Carlo-based simulator to generate a large dataset. Using an enhanced EMNIST (EEMNIST) dataset as an embedded contrast function allows us to train the network efficiently. The optical strategy utilizes k-space illumination in a reflectance configuration to probe tissue in the mesoscopic regime with high sensitivity and resolution. The proposed DL model training and validation is performed with both in silico data and a phantom experiment. Overall, our results indicate that the approach can correctly reconstruct both single and multiple fluorescent embedding(s) in a 3D volume. Furthermore, the presented technique is shown to outperform the traditional approaches [least-squares (LSQ) and total-variation minimization (TVAL)], especially at higher depths. We, therefore, expect the proposed computational technique to have future implications in preclinical studies.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Halftone spatial frequency domain imaging enables kilohertz high-speed label-free non-contact quantitative mapping of optical properties for strongly turbid media

The ability to quantify optical properties (i.e., absorption and scattering) of strongly turbid media has major implications on the characterization of biological tissues, fluid fields, and many others. However, there are few methods that can provide wide-field quantification of optical properties, and none is able to perform quantitative optical property imaging with high-speed (e.g., kilohertz) capabilities. Here we develop a new imaging modality termed halftone spatial frequency domain imaging (halftone-SFDI), which is approximately two orders of magnitude faster than the state-of-the-art, and provides kilohertz high-speed, label-free, non-contact, wide-field quantification for the optical properties of strongly turbid media. This method utilizes halftone binary patterned illumination to target the spatial frequency response of turbid media, which is then mapped to optical properties using model-based analysis. We validate the halftone-SFDI on an array of phantoms with a wide range of optical properties as well as in vivo human tissue. We demonstrate with an in vivo rat brain cortex imaging study, and show that halftone-SFDI can longitudinally monitor the absolute concentration as well as spatial distribution of functional chromophores in tissue. We also show that halftone-SFDI can spatially map dual-wavelength optical properties of a highly dynamic flow field at kilohertz speed. Together, these results highlight the potential of halftone-SFDI to enable new capabilities in fundamental research and translational studies including brain science and fluid dynamics.

Single Snapshot Imaging of Optical Properties (SSOP) for Perfusion Assessment during Gastric Conduit Creation for Esophagectomy: An Experimental Study on Pigs

Simple Summary

Anastomotic leak is the most dangerous complication occurring after esophagectomy and its relationship with inadequate visceral perfusion is widely recognized. Currently, the adequate perfusion of the gastric conduit is intraoperatively assessed by surgeons using subjective indicators (e.g., serosal color or pulsatile flow of vessels). During the last decades, several innovative optical techniques based on the interaction of light with tissue have been developed to monitor perfusion in esophagogastric surgery. However, these innovative approaches are characterized by a lack of video rate and reproducibility. They also provide operator-dependent results and lengthen the surgical workflow. Single Snapshot imaging of Optical Properties (SSOP) is an optical technique, which can overcome such limitations, providing quantitative information on the optical properties of biological tissues over a large field of view. It is the first study to demonstrate the accuracy of SSOP in the quantification of serosal StO₂% in a porcine gastric conduit model.

Abstract

Anastomotic leakage (AL) is a serious complication occurring after esophagectomy. The current knowledge suggests that inadequate intraoperative perfusion in the anastomotic site contributes to an increase in the AL rate. Presently, clinical estimation undertaken by surgeons is not accurate and new technology is necessary to improve the intraoperative assessment of tissue oxygenation. In the present study, we demonstrate the application of a novel optical technology, namely Single Snapshot imaging of Optical Properties (SSOP), used to quantify StO₂% in an open surgery experimental gastric conduit (GC) model. After the creation of a gastric conduit, local StO₂% was measured with a preclinical SSOP system for 60 min in the antrum (ROI-A), corpus (ROI-C), and fundus (ROI-F). The removed region (ROI-R) acted as ischemic control. ROI-R had statistically significant lower StO₂% when compared to all other ROIs at T15, T30, T45, and T60 (p < 0.0001). Local capillary lactates (LCLs) and StO₂% correlation was statistically significant (R = −0.8439, 95% CI −0.9367 to −0.6407, p < 0.0001). Finally, SSOP could discriminate resected from perfused regions and ROI-A from ROI-F (the future anastomotic site). In conclusion, SSOP could well be a suitable technology to assess intraoperative perfusion of GC, providing consistent StO₂% quantification and ROIs discrimination.

Rapid quantification of tissue perfusion properties with a two-stage look-up table: a simulation study.. [DOI: 10.1101/2021.11.04.467306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Tissue perfusion properties reveal crucial information pertinent to clinical diagnosis and treatment. Multispectral spatial frequency domain imaging (SFDI) is an emerging imaging technique that has been widely used to quantify tissue perfusion properties. However, slow processing speed limits its usefulness in real-time imaging applications. In this study, we present a two-stage look-up table (LUT) approach that accurately and rapidly quantifies optical (absorption and reduced scattering maps) and perfusion (total hemoglobin and oxygen saturation maps) properties using stage-1 and stage-2 LUTs, respectively, based on reflectance images at 660nm and 850nm. The two-stage LUT can be implemented on both CPU and GPU computing platforms. Quantifying tissue perfusion properties using the simulated diffuse reflectance images, we achieved a quantification speed of 266, 174, and 74 frames per second for three image sizes 512×512, 1024×1024, and 2048×2048 pixels, respectively. Quantification of tissue perfusion properties was highly accurate with only 3.5% and 2.5% error for total hemoglobin and oxygen saturation quantification, respectively. The two-stage LUT has the potential to be adopted in existing SFDI applications to enable real-time imaging capability of tissue hemodynamics.

In vivo two-dimensional quantitative imaging of skin and cutaneous microcirculation with perturbative spatial frequency domain imaging (p-SFDI)

We introduce perturbative spatial frequency domain imaging (p-SFDI) for fast two-dimensional (2D) mapping of the optical properties and physiological characteristics of skin and cutaneous microcirculation using spatially modulated visible light. Compared to the traditional methods for recovering 2D maps through a pixel-by-pixel inversion, p-SFDI significantly shortens parameter retrieval time, largely avoids the random fitting errors caused by measurement noise, and enhances the image reconstruction quality. The efficacy of p-SFDI is demonstrated by in vivo imaging forearm of one healthy subject, recovering the 2D spatial distribution of cutaneous hemoglobin concentration, oxygen saturation, scattering properties, the melanin content, and the epidermal thickness over a large field of view. Furthermore, the temporal and spatial variations in physiological parameters under the forearm reactive hyperemia protocol are revealed, showing its applications in monitoring temporal and spatial dynamics.

Imaging sub-diffuse optical properties of cancerous and normal skin tissue using machine learning-aided spatial frequency domain imaging

SIGNIFICANCE

Sub-diffuse optical properties may serve as useful cancer biomarkers, and wide-field heatmaps of these properties could aid physicians in identifying cancerous tissue. Sub-diffuse spatial frequency domain imaging (sd-SFDI) can reveal such wide-field maps, but the current time cost of experimentally validated methods for rendering these heatmaps precludes this technology from potential real-time applications.

AIM

Our study renders heatmaps of sub-diffuse optical properties from experimental sd-SFDI images in real time and reports these properties for cancerous and normal skin tissue subtypes.

APPROACH

A phase function sampling method was used to simulate sd-SFDI spectra over a wide range of optical properties. A machine learning model trained on these simulations and tested on tissue phantoms was used to render sub-diffuse optical property heatmaps from sd-SFDI images of cancerous and normal skin tissue.

RESULTS

The model accurately rendered heatmaps from experimental sd-SFDI images in real time. In addition, heatmaps of a small number of tissue samples are presented to inform hypotheses on sub-diffuse optical property differences across skin tissue subtypes.

CONCLUSION

These results bring the overall process of sd-SFDI a fundamental step closer to real-time speeds and set a foundation for future real-time medical applications of sd-SFDI such as image guided surgery.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Intravital mesoscopic fluorescence molecular tomography allows non-invasive in vivo monitoring and quantification of breast cancer growth dynamics

Preclinical breast tumor models are an invaluable tool to systematically study tumor progression and treatment response, yet methods to non-invasively monitor the involved molecular and mechanistic properties under physiologically relevant conditions are limited. Here we present an intravital mesoscopic fluorescence molecular tomography (henceforth IFT) approach that is capable of tracking fluorescently labeled tumor cells in a quantitative manner inside the mammary gland of living mice. Our mesoscopic approach is entirely non-invasive and thus permits prolonged observational periods of several months. The relatively high sensitivity and spatial resolution further enable inferring the overall number of oncogene-expressing tumor cells as well as their tumor volume over the entire cycle from early tumor growth to residual disease following the treatment phase. Our IFT approach is a promising method for studying tumor growth dynamics in a quantitative and longitudinal fashion in-vivo.

Profile-based intensity and frequency corrections for single-snapshot spatial frequency domain imaging

We have proposed the profile-based intensity and frequency corrections for single-snapshot spatial frequency domain (SFD) imaging to mitigate surface profile effects on the measured intensity and spatial frequency in extracting the optical properties. In the scheme, the spatially modulated frequency of the projected sinusoidal pattern is adaptively adjusted according to the sample surface profile, reducing distortions of the modulation amplitude in the single-snapshot demodulation and errors in the optical property extraction. The profile effects on both the measured intensities of light incident onto and reflected from the sample are then compensated using Minnaert's correction to obtain the true diffuse reflectance of the sample. We have validated the method by phantom experiments using a highly sensitive SFD imaging system based on the single-pixel photon-counting detection and assessed error reductions in extracting the absorption and reduced scattering coefficients by an average of 40% and 10%, respectively. Further, an in vivo topography experiment of the opisthenar vessels has demonstrated its clinical feasibility.

Speckle illumination SFDI for projector-free optical property mapping

Spatial frequency domain imaging can map tissue scattering and absorption properties over a wide field of view, making it useful for clinical applications such as wound assessment and surgical guidance. This technique has previously required the projection of fully characterized illumination patterns. Here, we show that random and unknown speckle illumination can be used to sample the modulation transfer function of tissues at known spatial frequencies, allowing the quantitative mapping of optical properties with simple laser diode illumination. We compute low- and high-spatial frequency response parameters from the local power spectral density for each pixel and use a lookup table to accurately estimate absorption and scattering coefficients in tissue phantoms, in vivo human hand, and ex vivo swine esophagus. Because speckle patterns can be generated over a large depth of field and field of view with simple coherent illumination, this approach may enable optical property mapping in new form-factors and applications, including endoscopy.

Direct mapping from diffuse reflectance to chromophore concentrations in multi-fx spatial frequency domain imaging (SFDI) with a deep residual network (DRN)

Spatial frequency domain imaging (SFDI) is an emerging technology that enables label-free, non-contact, and wide-field mapping of tissue chromophore contents, such as oxy- and deoxy-hemoglobin concentrations. It has been shown that the use of more than two spatial frequencies (multi-fx ) can vastly improve measurement accuracy and reduce chromophore estimation uncertainties, but real-time multi-fx SFDI for chromophore monitoring has been limited in practice due to the slow speed of available chromophore inversion algorithms. Existing inversion algorithms have to first convert the multi-fx diffuse reflectance to optical absorptions, and then solve a set of linear equations to estimate chromophore concentrations. In this work, we present a deep learning framework, noted as a deep residual network (DRN), that is able to directly map from diffuse reflectance to chromophore concentrations. The proposed DRN is over 10x faster than the state-of-the-art method for chromophore inversion and enables 25x improvement on the frame rate for in vivo real-time oxygenation mapping. The proposed deep learning model will help enable real-time and highly accurate chromophore monitoring with multi-fx SFDI.