Review of structured light in diffuse optical imaging

Simultaneous reconstruction of 3D fluorescence distribution and object surface using structured light illumination and dual-camera detection

Fluorescence molecular tomography (FMT) serves as a noninvasive modality for visualizing volumetric fluorescence distribution within biological tissues, thereby proving to be an invaluable imaging tool for preclinical animal studies. The conventional FMT relies upon a point-by-point raster scan strategy, enhancing the dataset for subsequent reconstruction but concurrently elongating the data acquisition process. The resultant diminished temporal resolution has persistently posed a bottleneck, constraining its utility in dynamic imaging studies. We introduce a novel system capable of simultaneous FMT and surface extraction, which is attributed to the implementation of a rapid line scanning approach and dual-camera detection. The system performance was characterized through phantom experiments, while the influence of scanning line density on reconstruction outcomes has been systematically investigated via both simulation and experiments. In a proof-of-concept study, our approach successfully captures a moving fluorescence bolus in three dimensions with an elevated frame rate of approximately 2.5 seconds per frame, employing an optimized scan interval of 5 mm. The notable enhancement in the spatio-temporal resolution of FMT holds the potential to broaden its applications in dynamic imaging tasks, such as surgical navigation.

Single-pixel imaging through non-homogeneous turbid media with adaptive illumination

The presence of scattering media limits the quality of images obtained by optical systems. Single-pixel imaging techniques based on structured illumination are highly tolerant to the presence of scattering between the object and the sensor, but very sensitive when the scattering medium is between the light source and the object. This makes it difficult to develop single-pixel imaging techniques for the case of objects immersed in scattering media. We present what we believe to be a new system for imaging objects through inhomogeneous scattering media in an epi-illumination configuration. It works in an adaptive way by combining diffuse optical imaging (DOI) and single pixel imaging (SPI) techniques in two stages. First, the turbid media is characterized by projecting light patterns with an LED array and applying DOI techniques. Second, the LED array is programmed to project light only through the less scattering areas of the media, while simultaneously using a digital micromirror device (DMD) to project light patterns onto the target using Hadamard basis coding functions. With this adaptive technique, we are able to obtain images of targets through two different scattering media with better quality than using conventional illumination. We also show that the system works with fluorescent targets.

Wide-field illumination diffuse optical tomography within a framework of single-pixel time-domain spatial frequency domain imaging

We present a wide-field illumination time-domain (TD) diffusion optical tomography (DOT) for three-dimensional (3-D) reconstruction within a shallow region under the illuminated surface of the turbid medium. The methodological foundation is laid on the single-pixel spatial frequency domain (SFD) imaging that facilitates the adoption of the well-established time-correlated single-photon counting (TCSPC)-based TD detection and generalized pulse spectrum techniques (GPST)-based reconstruction. To ameliorate the defects of the conventional diffusion equation (DE) in the forward modeling of TD-SFD-DOT, mainly the low accuracy in the near-field region and in profiling early-photon migration, we propose a modified model employing the time-dependent δ-P1 approximation and verify its improved accuracy in comparison with both the Monte Carlo and DE-based ones. For a simplified inversion process, a modified GPST approach is extended to TD-SFD-DOT that enables the effective separation of the absorption and scattering coefficients using a steady-state equivalent strategy. Furthermore, we set up a single-pixel TD-SFD-DOT system that employs the TCSPC-based TD detection in the SFD imaging framework. For assessments of the reconstruction approach and the system performance, phantom experiments are performed for a series of scenarios. The results show the effectiveness of the proposed methodology for rapid 3-D reconstruction of the absorption and scattering coefficients within a depth range of about 5 mean free pathlengths.

Development of a glass-based imaging phantom to model the optical properties of human tissue

The fabrication of a stable, reproducible optical imaging phantom is critical to the assessment and optimization of optical imaging systems. We demonstrate the use of an alternative material, glass, for the development of tissue-mimicking phantoms. The glass matrix was doped with nickel ions to approximate the absorption of hemoglobin. Scattering levels representative of human tissue were induced in the glass matrix through controlled crystallization at elevated temperatures. We show that this type of glass is a viable material for creating tissue-mimicking optical phantoms by providing controlled levels of scattering and absorption with excellent optical homogeneity, long-term stability and reproducibility.

Noninvasive Visualization of Amyloid-Beta Deposits in Alzheimer's Amyloidosis Mice via Fluorescence Molecular Tomography Using Contrast Agent

Alzheimer's disease is pathologically featured by the accumulation of amyloid-beta (Aβ) plaque and neurofibrillary tangles. Compared to small animal positron emission tomography, optical imaging features nonionizing radiation, low cost, and logistic convenience. Optical detection of Aβ deposits is typically implemented by 2D macroscopic imaging and various microscopic techniques assisted with Aβ-targeted contrast agents. Here, we introduce fluorescence molecular tomography (FMT), a macroscopic 3D fluorescence imaging technique, convenient for in vivo longitudinal monitoring of the animal brain without the involvement of cranial window opening operation. This chapter aims to provide the protocols for FMT in vivo imaging of Aβ deposits in the brain of rodent model of Alzheimer's disease. The materials, stepwise method, notes, limitations of FMT, and emerging opportunities for FMT techniques are presented.

Deep learning-based fusion of widefield diffuse optical tomography and micro-CT structural priors for accurate 3D reconstructions

Widefield illumination and detection strategies leveraging structured light have enabled fast and robust probing of tissue properties over large surface areas and volumes. However, when applied to diffuse optical tomography (DOT) applications, they still require a time-consuming and expert-centric solving of an ill-posed inverse problem. Deep learning (DL) models have been recently proposed to facilitate this challenging step. Herein, we expand on a previously reported deep neural network (DNN) -based architecture (modified AUTOMAP - ModAM) for accurate and fast reconstructions of the absorption coefficient in 3D DOT based on a structured light illumination and detection scheme. Furthermore, we evaluate the improved performances when incorporating a micro-CT structural prior in the DNN-based workflow, named Z-AUTOMAP. This Z-AUTOMAP significantly improves the widefield imaging process's spatial resolution, especially in the transverse direction. The reported DL-based strategies are validated both in silico and in experimental phantom studies using spectral micro-CT priors. Overall, this is the first successful demonstration of micro-CT and DOT fusion using deep learning, greatly enhancing the prospect of rapid data-integration strategies, often demanded in challenging pre-clinical scenarios.

Periodic-net: an end-to-end data driven framework for diffuse optical imaging of breast cancer from noisy boundary data

SIGNIFICANCE

The machine learning (ML) approach plays a critical role in assessing biomedical imaging processes especially optical imaging (OI) including segmentation, classification, and reconstruction, intending to achieve higher accuracy efficiently.

AIM

This research aims to develop an end-to-end deep learning framework for diffuse optical imaging (DOI) with multiple datasets to detect breast cancer and reconstruct its optical properties in the early stages.

APPROACH

The proposed Periodic-net is a nondestructive deep learning (DL) algorithm for the reconstruction and evaluation of inhomogeneities in an inverse model with high accuracy, while boundary measurements are calculated by solving a forward problem with sources/detectors arranged uniformly around a circular domain in various combinations, including 16 × 15 , 20 × 19 , and 36 × 35 boundary measurement setups.

RESULTS

The results of image reconstruction on numerical and phantom datasets demonstrate that the proposed network provides higher-quality images with a greater amount of small details, superior immunity to noise, and sharper edges with a reduction in image artifacts than other state-of-the-art competitors.

CONCLUSIONS

The network is highly effective at the simultaneous reconstruction of optical properties, i.e., absorption and reduced scattering coefficients, by optimizing the imaging time without degrading inclusions localization and image quality.

Spatial frequency domain Mueller matrix imaging

SIGNIFICANCE

Mueller matrix polarimetry (MMP) and spatial frequency domain imaging (SFDI) are wide-field optical imaging modalities that differentiate tissue primarily by structure alignment and photon transport coefficient, respectively. Because these effects can be related, combining MMP and SFDI may enhance tissue differentiation beyond the capability of each modality alone.

AIM

An instrument was developed to combine MMP and SFDI with the goal of testing whether it enhances contrast of features in reflection mode.

APPROACH

The instrument was constructed using liquid crystal elements for polarization control, a digital light processing projector for generating sinusoidal illumination patterns, and a digital camera for imaging. A theoretical analysis shows that the SFD Mueller matrix is complex-valued and does not follow the same behavior as a regular Mueller matrix. Images were acquired from an anisotropic tissue phantom, an optical fiber bundle, and cerebellum, thalamus, and cerebrum tissues.

RESULTS

The measurement results suggest that singly scattered, few scattered, and diffusely scattered photon paths can be distinguished in some of the samples investigated. The combined imaging modality yields additional spatial frequency phase information, which highlights paths having only a few scattering events.

CONCLUSIONS

The combination of MMP and SFDI offers contrast mechanisms inaccessible by each modality used alone.

Computational macroscopic lifetime imaging and concentration unmixing of autofluorescence

Single-pixel computational imaging can leverage highly sensitive detectors that concurrently acquire data across spectral and temporal domains. For molecular imaging, such methodology enables to collect rich intensity and lifetime multiplexed fluorescence datasets. Herein we report on the application of a single-pixel structured light-based platform for macroscopic imaging of tissue autofluorescence. The super-continuum visible excitation and hyperspectral single-pixel detection allow for parallel characterization of autofluorescence intensity and lifetime. Furthermore, we exploit a deep learning based data processing pipeline, to perform autofluorescence unmixing while yielding the autofluorophores' concentrations. The full scheme (setup and processing) is validated in silico and in vitro with clinically relevant autofluorophores flavin adenine dinucleotide, riboflavin, and protoporphyrin. The presented results demonstrate the potential of the methodology for macroscopically quantifying the intensity and lifetime of autofluorophores, with higher specificity for cases of mixed emissions, which are ubiquitous in autofluorescence and multiplexed in vivo imaging.

Coherent enhancement of optical remission in diffusive media

Remitted waves are used for sensing and imaging in diverse diffusive media from the Earth's crust to the human brain. Separating the source and detector increases the penetration depth of light, but the signal strength decreases rapidly, leading to a poor signal-to-noise ratio. Here, we show, experimentally and numerically, that wavefront shaping a laser beam incident on a diffusive sample enables an enhancement of remission by an order of magnitude at depths of up to 10 transport mean free paths. We develop a theoretical model which predicts the maximal remission enhancement. Our analysis reveals a significant improvement in the sensitivity of remitted waves to local changes of absorption deep inside diffusive media. This work illustrates the potential of coherent wavefront control for noninvasive diffuse wave imaging applications, such as diffuse optical tomography and functional near-infrared spectroscopy.

Motion Deblurring for Single-Pixel Spatial Frequency Domain Imaging

The single-pixel imaging technique is applied to spatial frequency domain imaging (SFDI) to bring significant performance advantages in band extension and sensitivity enhancement. However, the large number of samplings required can cause severe quality degradations in the measured image when imaging a moving target. This work presents a novel method of motion deblurring for single-pixel SFDI. In this method, the Fourier coefficients of the reflected image are measured by the Fourier single-pixel imaging technique. On this basis, a motion-degradation-model-based compensation, which is derived by the phase-shift and frequency-shift properties of Fourier transform, is adopted to eliminate the effects of target displacements on the measurements. The target displacements required in the method are obtained using a fast motion estimation approach. A series of numerical and experimental validations show that the proposed method can effectively deblur the moving targets and accordingly improves the accuracy of the extracted optical properties, rendering it a potentially powerful way of broadening the clinical application of single-pixel SFDI.

Radiance backscattered by a strongly scattering medium in the high spatial frequency limit

We study the radiative transfer of a spatially modulated plane wave incident on a half-space composed of a uniformly scattering and absorbing medium. For spatial frequencies that are large compared to the scattering coefficient, we find that first-order scattering governs the leading behavior of the radiance backscattered by the medium. The first-order scattering approximation reveals a specific curve on the backscattered hemisphere where the radiance is concentrated. Along this curve, the radiance assumes a particularly simple expression that is directly proportional to the phase function. These results are inherent to the radiative transfer equation at large spatial frequency and do not have a strong dependence on any particular optical property. Consequently, these results provide the means by which spatial frequency domain imaging technologies can directly measure the phase function of a sample. Numerical simulations using the discrete ordinate method along with the source integration interpolation method validate these theoretical findings.

3D k-space reflectance fluorescence tomography via deep learning

We report on the potential to perform image reconstruction in 3D k-space reflectance fluorescence tomography (FT) using deep learning (DL). Herein, we adopt a modified AUTOMAP architecture and develop a training methodology that leverages an open-source Monte-Carlo-based simulator to generate a large dataset. Using an enhanced EMNIST (EEMNIST) dataset as an embedded contrast function allows us to train the network efficiently. The optical strategy utilizes k-space illumination in a reflectance configuration to probe tissue in the mesoscopic regime with high sensitivity and resolution. The proposed DL model training and validation is performed with both in silico data and a phantom experiment. Overall, our results indicate that the approach can correctly reconstruct both single and multiple fluorescent embedding(s) in a 3D volume. Furthermore, the presented technique is shown to outperform the traditional approaches [least-squares (LSQ) and total-variation minimization (TVAL)], especially at higher depths. We, therefore, expect the proposed computational technique to have future implications in preclinical studies.

Visualization of reflectance, transmittance, and application amount distribution of the cosmetic foundation layer on skin

We developed a method for visualization of makeup finishing with structured lighting. By analyzing images with a sequence of projection patterns, reflectance and transmittance of the cosmetic foundation (FD) layer were extracted as spatial maps using the difference between the light spread of bare skin and made-up skin. The spatial maps reflect conditions and distribution of applied FD under real situations. By calibrating the relationship between optical properties and the amount of FD applied, the application amount distribution was also estimated. Additionally, we proposed approximation formulae to estimate the above values without images of bare skin. These formulae provide good agreement with the original formula for reflectance.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Compressed sensing in fluorescence microscopy

Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.

Macroscopic Fluorescence Lifetime Imaging for Monitoring of Drug-Target Engagement

Precision medicine promises to improve therapeutic efficacy while reducing adverse effects, especially in oncology. However, despite great progresses in recent years, precision medicine for cancer treatment is not always part of routine care. Indeed, the ability to specifically tailor therapies to distinct patient profiles requires still significant improvements in targeted therapy development as well as decreases in drug treatment failures. In this regard, preclinical animal research is fundamental to advance our understanding of tumor biology, and diagnostic and therapeutic response. Most importantly, the ability to measure drug-target engagement accurately in live and intact animals is critical in guiding the development and optimization of targeted therapy. However, a major limitation of preclinical molecular imaging modalities is their lack of capability to directly and quantitatively discriminate between drug accumulation and drug-target engagement at the pathological site. Recently, we have developed Macroscopic Fluorescence Lifetime Imaging (MFLI) as a unique feature of optical imaging to quantitate in vivo drug-target engagement. MFLI quantitatively reports on nanoscale interactions via lifetime-sensing of Förster Resonance Energy Transfer (FRET) in live, intact animals. Hence, MFLI FRET acts as a direct reporter of receptor dimerization and target engagement via the measurement of the fraction of labeled-donor entity undergoing binding to its respective receptor. MFLI is expected to greatly impact preclinical imaging and also adjacent fields such as image-guided surgery and drug development.

Developing diagnostic assessment of breast lumpectomy tissues using radiomic and optical signatures

High positive margin rates in oncologic breast-conserving surgery are a pressing clinical problem. Volumetric X-ray scanning is emerging as a powerful ex vivo specimen imaging technique for analyzing resection margins, but X-rays lack contrast between non-malignant and malignant fibrous tissues. In this study, combined micro-CT and wide-field optical image radiomics were developed to classify malignancy of breast cancer tissues, demonstrating that X-ray/optical radiomics improve malignancy classification. Ninety-two standardized features were extracted from co-registered micro-CT and optical spatial frequency domain imaging samples extracted from 54 breast tumors exhibiting seven tissue subtypes confirmed by microscopic histological analysis. Multimodal feature sets improved classification performance versus micro-CT alone when adipose samples were included (AUC = 0.88 vs. 0.90; p-value = 3.65e-11) and excluded, focusing the classification task on exclusively non-malignant fibrous versus malignant tissues (AUC = 0.78 vs. 0.85; p-value = 9.33e-14). Extending the radiomics approach to high-dimensional optical data-termed "optomics" in this study-offers a promising optical image analysis technique for cancer detection. Radiomic feature data and classification source code are publicly available.

Single-pixel imaging: An overview of different methods to be used for 3D space reconstruction in harsh environments

Different imaging solutions have been proposed over the last few decades, aimed at three-dimensional (3D) space reconstruction and obstacle detection, either based on stereo-vision principles using active pixel sensors operating in the visible part of the spectra or based on active Near Infra-Red (NIR) illumination applying the time-of-flight principle, to mention just a few. If extremely low quantum efficiencies for NIR active illumination yielded by silicon-based detector solutions are considered together with the huge photon noise levels produced by the background illumination accompanied by Rayleigh scattering effects taking place in outdoor applications, the operating limitations of these systems under harsh weather conditions, especially if relatively low-power active illumination is used, are evident. If longer wavelengths for active illumination are applied to overcome these issues, indium gallium arsenide (InGaAs)-based photodetectors become the technology of choice, and for low-cost solutions, using a single InGaAs photodetector or an InGaAs line-sensor becomes a promising choice. In this case, the principles of Single-Pixel Imaging (SPI) and compressive sensing acquire a paramount importance. Thus, in this paper, we review and compare the different SPI developments reported. We cover a variety of SPI system architectures, modulation methods, pattern generation and reconstruction algorithms, embedded system approaches, and 2D/3D image reconstruction methods. In addition, we introduce a Near Infra-Red Single-Pixel Imaging (NIR-SPI) sensor aimed at detecting static and dynamic objects under outdoor conditions for unmanned aerial vehicle applications.

Compressed sensing time-resolved spectrometer for quantification of light absorbers in turbid media

Time-resolved (TR) spectroscopy is well-suited to address the challenges of quantifying light absorbers in highly scattering media such as living tissue; however, current TR spectrometers are either based on expensive array detectors or rely on wavelength scanning. Here, we introduce a TR spectrometer architecture based on compressed sensing (CS) and time-correlated single-photon counting. Using both CS and basis scanning, we demonstrate that-in homogeneous and two-layer tissue-mimicking phantoms made of Intralipid and Indocyanine Green-the CS method agrees with or outperforms uncompressed approaches. Further, we illustrate the superior depth sensitivity of TR spectroscopy and highlight the potential of the device to quantify absorption changes in deeper (>1 cm) tissue layers.

Recent Progress on Molecular Photoacoustic Imaging with Carbon-Based Nanocomposites

For biomedical imaging, the interest in noninvasive imaging methods is ever increasing. Among many modalities, photoacoustic imaging (PAI), which is a combination of optical and ultrasound imaging techniques, has received attention because of its unique advantages such as high spatial resolution, deep penetration, and safety. Incorporation of exogenous imaging agents further amplifies the effective value of PAI, since they can deliver other specified functions in addition to imaging. For these agents, carbon-based materials can show a large specific surface area and interesting optoelectronic properties, which increase their effectiveness and have proved their potential in providing a theragnostic platform (diagnosis + therapy) that is essential for clinical use. In this review, we introduce the current state of the PAI modality, address recent progress on PAI imaging that takes advantage of carbon-based agents, and offer a future perspective on advanced PAI systems using carbon-based agents.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Biosafety, Functionalities, and Applications of Biomedical Micro/nanomotors

Due to their unique ability to actively move, micro/nanomotors offer the possibility of breaking through the limitations of traditional passive drug delivery systems for the treatment of many diseases, and have attracted the increasing attention of researchers. However, at present, the realization of many advantages of micro/nanomotors in disease treatment in vivo is still in its infancy, because of the complexity and particularity of diseases in different parts of human body. In this Minireview, we first focus on the biosafety and functionality of micro/nanomotors as a biomedical treatment system. Then, we address the treatment difficulties of various diseases in vivo (such as ophthalmic disease, orthopedic disease, gastrointestinal disease, cardiovascular disease, and cancer), and then review the research progress of biomedical micro/nanomotors in the past 20 years, Finally, we propose the challenges in this field and possible future development directions.

Spatial properties and propagation dynamics of apodized Hermite-Gauss beams

Hermite-Gauss beams (HGBs), which have been presented as exact solutions to the paraxial wave equation in Cartesian coordinates, are natural resonating modes in stable laser resonators. In this work, we demonstrate that the apodized Hermite-Gauss beam (ApHGB), by a decaying exponential function exp⁡(γx) (γ is the decaying or apodization parameter), is equivalent, under certain circumstances, to the apodized Airy beam (ApAiB). Based on the asymptotic treatment of the Hermite polynomials, we provide analytical expressions to describe their propagation dynamics in free space, then we investigate their similarity as a function of the apodization parameter. Moreover, by combining symmetric ApHGBs, we build dual and quad ApHGBs. The obtained numerical simulations confirm our analytical predictions. We believe that the ApHGB could be used as an alternative to the Airy beam in many applications.

Quo Vadis, Nanoparticle-Enabled In Vivo Fluorescence Imaging?

The exciting advancements that we are currently witnessing in terms of novel materials and synthesis approaches are leading to the development of colloidal nanoparticles (NPs) with increasingly greater tunable properties. We have now reached a point where it is possible to synthesize colloidal NPs with functionalities tailored to specific societal demands. The impact of this new wave of colloidal NPs has been especially important in the field of biomedicine. In that vein, luminescent NPs with improved brightness and near-infrared working capabilities have turned out to be optimal optical probes that are capable of fast and high-resolution in vivo imaging. However, luminescent NPs have thus far only reached a limited portion of their potential. Although we believe that the best is yet to come, the future might not be as bright as some of us think (and have hoped!). In particular, translation of NP-based fluorescence imaging from preclinical studies to clinics is not straightforward. In this Perspective, we provide a critical assessment and highlight promising research avenues based on the latest advances in the fields of luminescent NPs and imaging technologies. The disillusioned outlook we proffer herein might sound pessimistic at first, but we consider it necessary to avoid pursuing "pipe dreams" and redirect the efforts toward achievable-yet ambitious-goals.

Decay behavior and optical parameter identification for spatial-frequency domain imaging by the radiative transport equation

The decay behavior of specific intensity is studied for spatial-frequency domain imaging (SFDI). It is shown using the radiative transport equation that the decay is given by a superposition of different decay modes, and the decay rates of these modes are determined by spatial frequencies and Case's eigenvalues. This explains why SFDI can focus on shallow regions. The fact that light with nonzero spatial frequency rapidly decays makes it possible to exclusively extract optical properties of the top layer of a layered medium. We determine optical properties of the top layer of a solid phantom. This measurement is verified with different layered media of numerical phantoms.

High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo

Single pixel imaging frameworks facilitate the acquisition of high-dimensional optical data in biological applications with photon starved conditions. However, they are still limited to slow acquisition times and low pixel resolution. Herein, we propose a convolutional neural network for fluorescence lifetime imaging with compressed sensing at high compression (NetFLICS-CR), which enables in vivo applications at enhanced resolution, acquisition and processing speeds, without the need for experimental training datasets. NetFLICS-CR produces intensity and lifetime reconstructions at 128 × 128 pixel resolution over 16 spectral channels while using only up to 1% of the required measurements, therefore reducing acquisition times from ∼2.5 hours at 50% compression to ∼3 minutes at 99% compression. Its potential is demonstrated in silico, in vitro and for mice in vivo through the monitoring of receptor-ligand interactions in liver and bladder and further imaging of intracellular delivery of the clinical drug Trastuzumab to HER2-positive breast tumor xenografts. The data acquisition time and resolution improvement through NetFLICS-CR, facilitate the translation of single pixel macroscopic flurorescence lifetime imaging (SP-MFLI) for in vivo monitoring of lifetime properties and drug uptake.

Real-time, wide-field and high-quality single snapshot imaging of optical properties with profile correction using deep learning

The development of real-time, wide-field and quantitative diffuse optical imaging methods to visualize functional and structural biomarkers of living tissues is a pressing need for numerous clinical applications including image-guided surgery. In this context, Spatial Frequency Domain Imaging (SFDI) is an attractive method allowing for the fast estimation of optical properties using the Single Snapshot of Optical Properties (SSOP) approach. Herein, we present a novel implementation of SSOP based on a combination of deep learning network at the filtering stage and Graphics Processing Units (GPU) capable of simultaneous high visual quality image reconstruction, surface profile correction and accurate optical property (OP) extraction in real-time across large fields of view. In the most optimal implementation, the presented methodology demonstrates megapixel profile-corrected OP imaging with results comparable to that of profile-corrected SFDI, with a processing time of 18 ms and errors relative to SFDI method less than 10% in both profilometry and profile-corrected OPs. This novel processing framework lays the foundation for real-time multispectral quantitative diffuse optical imaging for surgical guidance and healthcare applications. All code and data used for this work is publicly available at www.healthphotonics.org under the resources tab.

Spatial frequency domain imaging for the longitudinal monitoring of vascularization during mouse femoral graft healing

Allograft is the current gold standard for treating critical-sized bone defects. However, allograft healing is usually compromised partially due to poor host-mediated vascularization. In the efforts towards developing new methods to enhance allograft healing, a non-terminal technique for monitoring the vascularization is needed in pre-clinical mouse models. In this study, we developed a non-invasive instrument based on spatial frequency domain imaging (SFDI) for longitudinal monitoring of the mouse femoral graft healing. SFDI technique provided total hemoglobin concentration (THC) and oxygen saturation (StO2) of the graft and the surrounding soft tissues. SFDI measurements were performed from 1 day before to 44 days after graft transplantation. Autograft, another type of bone graft with higher vascularization potential was also measured as a comparison to allograft. For both grafts, the overall temporal changes of the measured THC agreed with the physiological expectations of vascularization timeline during bone healing. A significantly greater increase in THC was observed in the autograft group compared to the allograft group, which agreed with the expectation that allografts have more compromised vascularization.

Quadrant Fourier transform and its application in decoding OAM signals

We present a new, to the best of our knowledge, concept of using quadrant Fourier transforms (QFTs) formed by microlens arrays (MLAs) to decode complex optical signals based on the optical intensity collected per quadrant area after the MLAs. From a computational optics viewpoint, we show the most promising use of the QFT in low-cost and passive decoding of laser signals carrying optical angular momenta (OAM) that are prevalent in research frontiers of optical communications, computation, and imaging. There are numerous ways of creating, adding, and combining OAM states in optical waves, while decoding or demultiplexing approaches often turn out to be complicated or expensive. The simple OAM decoder formed by a pair of identical MLAs, which are concatenated in the focal plane and transversely offset by half-pitch length, can accomplish the imaging task with four pixels per cell. By sorting the gradient curls of the optical wave into local quadrant cells, the decoder analyzes the intensity reallocation that is proportional to the gradients and computes the gradient curls accordingly. The low-cost, compactness, and simplicity of the proposed OAM sensor will further promote OAM-based applications, as well as many other applications that exploit the spatial complexity of optical signals.

Macroscopic fluorescence lifetime topography enhanced via spatial frequency domain imaging

We report on a macroscopic fluorescence lifetime imaging (MFLI) topography computational framework based around machine learning with the main goal of retrieving the depth of fluorescent inclusions deeply seated in bio-tissues. This approach leverages the depth-resolved information inherent to time-resolved fluorescence data sets coupled with the retrieval of in situ optical properties as obtained via spatial frequency domain imaging (SFDI). Specifically, a Siamese network architecture is proposed with optical properties (OPs) and time-resolved fluorescence decays as input followed by simultaneous retrieval of lifetime maps and depth profiles. We validate our approach using comprehensive in silico data sets as well as with a phantom experiment. Overall, our results demonstrate that our approach can retrieve the depth of fluorescence inclusions, especially when coupled with optical properties estimation, with high accuracy. We expect the presented computational approach to find great utility in applications such as optical-guided surgery.

Emerging nanotherapeutics for antithrombotic treatment

Thrombus causes insufficient blood flow and ischemia damages to brain and heart, leading to life-threatening cardio-cerebrovascular diseases. Development of efficient antithrombotic strategies has long been a high priority, owing to the high morbidity and mortality of thrombotic diseases. With the rapid development of biomedical nanotechnology in diagnosis and treatment of thrombotic disorder, remarkable progresses have been made in antithrombotic nanomedicines in recent years. Herein, we outline the recent advances in this field at the intersection of thrombus theranostics and biomedical nanotechnology. First, thrombus diagnosis techniques based on biomedical nanotechnology are presented. Then, emerging antithrombotic nanotherapeutics are overviewed, including thrombus-targeting strategies, thrombus stimuli-responsive nanosystems and phase transition-driven nanotherapeutics. Furthermore, multifunctional nanosystems for combination theranostics of thrombotic diseases are discussed. Finally, the design considerations, advantages and challenges of these biomedical nanotechnology-driven therapeutics in clinical translation are highlighted.

Accelerating Monte Carlo modeling of structured-light-based diffuse optical imaging via "photon sharing"

The increasing use of spatially modulated imaging and single-pixel detection techniques demands computationally efficient methods for light transport modeling. Herein, we report an easy-to-implement yet significantly more efficient Monte Carlo (MC) method for simultaneously simulating spatially modulated illumination and detection patterns accurately in 3D complex domains. We have implemented this accelerated algorithm, named "photon sharing," in our open-source MC simulators, reporting 13.6× and 5.5× speedups in mesh- and voxel-based MC benchmarks, respectively. In addition, the proposed algorithm is readily used to accelerate the solving of inverse problems in spatially modulated imaging systems by building Jacobians of all illumination-detection pattern pairs concurrently, resulting in a 12.4-fold speed improvement.

Wide-field optical spectroscopy system integrating reflectance and spatial frequency domain imaging to measure attenuation-corrected intrinsic tissue fluorescence in radical prostatectomy specimens

The development of a multimodal optical imaging system is presented that integrates endogenous fluorescence and diffuse reflectance spectroscopy with single-wavelength spatial frequency domain imaging (SFDI) and surface profilometry. The system images specimens at visible wavelengths with a spatial resolution of 70 µm, a field of view of 25 cm² and a depth of field of ∼1.5 cm. The results of phantom experiments are presented demonstrating the system retrieves absorption and reduced scattering coefficient maps using SFDI with <6% reconstruction errors. A phase-shifting profilometry technique is implemented and the resulting 3-D surface used to compute a geometric correction ensuring optical properties reconstruction errors are maintained to <6% in curved media with height variations <20 mm. Combining SFDI-computed optical properties with data from diffuse reflectance spectra is shown to correct fluorescence using a model based on light transport in tissue theory. The system is used to image a human prostate, demonstrating its ability to distinguish prostatic tissue (anterior stroma, hyperplasia, peripheral zone) from extra-prostatic tissue (urethra, ejaculatory ducts, peri-prostatic tissue). These techniques could be integrated in robotic-assisted surgical systems to enhance information provided to surgeons and improve procedural accuracy by minimizing the risk of damage to extra-prostatic tissue during radical prostatectomy procedures and eventually detect residual cancer.

Investigation of the quantification of hemoglobin and cytochrome-c-oxidase in the exposed cortex with near-infrared hyperspectral imaging: a simulation study

SIGNIFICANCE

We present a Monte Carlo (MC) computational framework that simulates near-infrared (NIR) hyperspectral imaging (HSI) aimed at assisting quantification of the in vivo hemodynamic and metabolic states of the exposed cerebral cortex in small animal experiments. This can be done by targeting the NIR spectral signatures of oxygenated (HbO2) and deoxygenated (HHb) hemoglobin for hemodynamics as well as the oxidative state of cytochrome-c-oxidase (oxCCO) for measuring tissue metabolism.

AIM

The aim of this work is to investigate the performances of HSI for this specific application as well as to assess key factors for the future design and operation of a benchtop system.

APPROACH

The MC framework, based on Mesh-based Monte Carlo (MMC), reproduces a section of the exposed cortex of a mouse from an in vivo image and replicates hyperspectral illumination and detection at multiple NIR wavelengths (up to 121).

RESULTS

The results demonstrate: (1) the fitness of the MC framework to correctly simulate hyperspectral data acquisition; (2) the capability of HSI to reconstruct spatial changes in the concentrations of HbO2, HHb, and oxCCO during a simulated hypoxic condition; (3) that eight optimally selected wavelengths between 780 and 900 nm provide minimal differences in the accuracy of the hyperspectral results, compared to the "gold standard" of 121 wavelengths; and (4) the possibility to mitigate partial pathlength effects in the reconstructed data and to enhance quantification of the hemodynamic and metabolic responses.

CONCLUSIONS

The MC framework is proved to be a flexible and useful tool for simulating HSI also for different applications and targets.

JP, Austin W, Tabassum SM, Aguénounon E, Tilbury K, Saager RB, Gioux S, Roblyer D. OpenSFDI: an open-source guide for constructing a spatial frequency domain imaging system

Significance: Spatial frequency domain imaging (SFDI) is a diffuse optical measurement technique that can quantify tissue optical absorption (μ_a) and reduced scattering (<inline-formula>μs'</inline-formula>) on a pixel-by-pixel basis. Measurements of μ_a at different wavelengths enable the extraction of molar concentrations of tissue chromophores over a wide field, providing a noncontact and label-free means to assess tissue viability, oxygenation, microarchitecture, and molecular content. We present here openSFDI: an open-source guide for building a low-cost, small-footprint, three-wavelength SFDI system capable of quantifying μ_a and <inline-formula>μs'</inline-formula> as well as oxyhemoglobin and deoxyhemoglobin concentrations in biological tissue. The companion website provides a complete parts list along with detailed instructions for assembling the openSFDI system.<p> Aim: We describe the design of openSFDI and report on the accuracy and precision of optical property extractions for three different systems fabricated according to the instructions on the openSFDI website.</p> <p> Approach: Accuracy was assessed by measuring nine tissue-simulating optical phantoms with a physiologically relevant range of μ_a and <inline-formula>μs'</inline-formula> with the openSFDI systems and a commercial SFDI device. Precision was assessed by repeatedly measuring the same phantom over 1 h.</p> <p> Results: The openSFDI systems had an error of 0  ±  6  %   in μ_a and -2  ±  3  %   in <inline-formula>μs'</inline-formula>, compared to a commercial SFDI system. Bland-Altman analysis revealed the limits of agreement between the two systems to be   ±  0.004  mm^-  1 for μ_a and -0.06 to 0.1  mm^-  1 for <inline-formula>μs'</inline-formula>. The openSFDI system had low drift with an average standard deviation of 0.0007  mm^-  1 and 0.05  mm^-  1 in μ_a and <inline-formula>μs'</inline-formula>, respectively.</p>,<p> Conclusion: The openSFDI provides a customizable hardware platform for research groups seeking to utilize SFDI for quantitative diffuse optical imaging.</p>

System configuration optimization for mesoscopic fluorescence molecular tomography

Tissue engineering applications demand 3D, non-invasive, and longitudinal assessment of bioprinted constructs. Current emphasis is on developing tissue constructs mimicking in vivo conditions; however, these are increasingly challenging to image as they are typically a few millimeters thick and turbid, limiting the usefulness of classical fluorescence microscopic techniques. For such applications, we developed a Mesoscopic Fluorescence Molecular Tomography methodology that collects high information content data to enable high-resolution tomographic reconstruction of fluorescence biomarkers at millimeters depths. This imaging approach is based on an inverse problem; hence, its imaging performances are dependent on critical technical considerations including optode sampling, forward model design and inverse solver parameters. Herein, we investigate the impact of the optical system configuration parameters, including detector layout, number of detectors, combination of detector and source numbers, and scanning mode with uncoupled or coupled source and detector array, on the 3D imaging performances. Our results establish that an MFMT system with a 2D detection chain implemented in a de-scanned mode provides the optimal imaging reconstruction performances.

Quantifying skin photodamage with spatial frequency domain imaging: statistical results

We investigated the change in optical properties and vascular parameters to characterize skin tissue from mild photodamage to actinic keratosis (AK) with comparison to a published photodamage scale. Multi-wavelength spatial frequency domain imaging (SFDI) measurements were performed on the dorsal forearms of 55 adult subjects with various amounts of photodamage. Dermatologists rated the levels of photodamage based upon the photographs in blinded fashion to allow comparison with SFDI data. For characterization of statistical data, we used artificial neural networks. Our results indicate that optical and vascular parameters can be used to quantify photodamage and can discriminate between the stages as low, medium, and high grades, with the best performance of ∼70%, ∼76% and 80% for characterization of low- medium- and high-grade lesions, respectively. Ultimately, clinicians can use this noninvasive approach for risk assessment and frequent monitoring of high-risk populations.

Structured light imaging for breast-conserving surgery, part II: texture analysis and classification

Subdiffuse spatial frequency domain imaging (sd-SFDI) data of 42 freshly excised, bread-loafed tumor resections from breast-conserving surgery (BCS) were evaluated using texture analysis and a machine learning framework for tissue classification. Resections contained 56 regions of interest (RoIs) determined by expert histopathological analysis. RoIs were coregistered with sd-SFDI data and sampled into ∼4  ×  4  mm2 subimage samples of confirmed and homogeneous histological categories. Sd-SFDI reflectance textures were analyzed using gray-level co-occurrence matrix pixel statistics, image primitives, and power spectral density curve parameters. Texture metrics exhibited statistical significance (p-value  <  0.05) between three benign and three malignant tissue subtypes. Pairs of benign and malignant subtypes underwent texture-based, binary classification with correlation-based feature selection. Classification performance was evaluated using fivefold cross-validation and feature grid searching. Classification using subdiffuse, monochromatic reflectance (illumination spatial frequency of fx  =  1.37  mm  −  1, optical wavelength of λ  =  490  nm) achieved accuracies ranging from 0.55 (95% CI: 0.41 to 0.69) to 0.95 (95% CI: 0.90 to 1.00) depending on the benign–malignant diagnosis pair. Texture analysis of sd-SFDI data maintains the spatial context within images, is free of light transport model assumptions, and may provide an alternative, computationally efficient approach for wide field-of-view (cm2) BCS tumor margin assessment relative to pixel-based optical scatter or color properties alone.

Real-time optical properties and oxygenation imaging using custom parallel processing in the spatial frequency domain

The development of real-time, wide-field and quantitative diffuse optical imaging methods is becoming increasingly popular for biological and medical applications. Recent developments introduced a novel approach for real-time multispectral acquisition in the spatial frequency domain using spatio-temporal modulation of light. Using this method, optical properties maps (absorption and reduced scattering) could be obtained for two wavelengths (665 nm and 860 nm). These maps, in turn, are used to deduce oxygen saturation levels in tissues. However, while the acquisition was performed in real-time, processing was performed post-acquisition and was not in real-time. In the present article, we present CPU and GPU processing implementations for this method with special emphasis on processing time. The obtained results show that the proposed custom direct method using a General Purpose Graphic Processing Unit (GPGPU) and C CUDA (Compute Unified Device Architecture) implementation enables 1.6 milliseconds processing time for a 1 Mega-pixel image with a maximum average error of 0.1% in extracting optical properties.

Special Section Guest Editorial: Special Section on Spatial Frequency Domain Imaging

This guest editorial introduces the Special Section on Spatial Frequency Domain Imaging.

Quantitative Wide-Field Imaging Techniques for Fluorescence Guided Neurosurgery

Fluorescence guided surgery (FGS) has fueled the development of novel technologies aimed at maximizing the utility of fluorescence imaging to help clinicians diagnose and in certain cases treat diseases across a breadth of disciplines such as dermatology, gynecology, oncology, ophthalmology, and neurosurgery. In neurosurgery, the goal of FGS technologies is to provide the neurosurgeon with additional information which can serve as a visual aid to better identify tumor tissue and associated margins. Yet, current clinical FGS technologies are qualitative in nature, limiting the ability to make accurate, reliable, and repeatable measurements. To this end, developments in fluorescence quantification are needed to overcome current limitations of FGS. Here we present an overview of the recent developments in quantitative fluorescence guidance technologies and conclude with the most recent developments aimed at wide-field quantitative fluorescence imaging approaches in neurosurgery.

Structured illumination Mueller matrix imaging

We perform Mueller matrix imaging (MMI) of diffusely scattering phantoms under sinusoidal irradiance of varying spatial frequency. Quantitative polarimetric sensing via MMI completely characterizes a sample's polarimetric properties, while structured illumination (SI) allows for the control of photon path length. Intralipid phantoms were measured with varying absorption and with varying depth to demonstrate photon path length control for Mueller matrix elements. We observe unpolarized intensity, linear polarization, and circular polarization to depend upon spatial frequency differently. Finally, we measured an ex vivo chicken skin sample over a bright and dark substrate to further demonstrate the sensitivity of SI-MMI to depth.

Custom Scanning Hyperspectral Imaging System for Biomedical Applications: Modeling, Benchmarking, and Specifications

Prototyping hyperspectral imaging devices in current biomedical optics research requires taking into consideration various issues regarding optics, imaging, and instrumentation. In summary, an ideal imaging system should only be limited by exposure time, but there will be technological limitations (e.g., actuator delay and backlash, network delays, or embedded CPU speed) that should be considered, modeled, and optimized. This can be achieved by constructing a multiparametric model for the imaging system in question. The article describes a rotating-mirror scanning hyperspectral imaging device, its multiparametric model, as well as design and calibration protocols used to achieve its optimal performance. The main objective of the manuscript is to describe the device and review this imaging modality, while showcasing technical caveats, models and benchmarks, in an attempt to simplify and standardize specifications, as well as to incentivize prototyping similar future designs.

Single snapshot imaging of optical properties using a single-pixel camera: a simulation study

We present the effects of using a single-pixel camera approach to extract optical properties with the single-snapshot spatial frequency-domain imaging method. We acquired images of a human hand for spatial frequencies ranging from 0.1 to 0.4  mm  -  1 with increasing compression ratios using adaptive basis scan wavelet prediction strategy. In summary, our findings indicate that the extracted optical properties remained usable up to 99% of compression rate at a spatial frequency of 0.2  mm  -  1 with errors of 5% in reduced scattering and 10% in absorption.

Real-time, wide-field, and quantitative oxygenation imaging using spatiotemporal modulation of light

Quantitative diffuse optical imaging has the potential to provide valuable functional information about tissue status, such as oxygen saturation or blood content to healthcare practitioners in real time. However, significant technical challenges have so far prevented such tools from being deployed in the clinic. Toward achieving this goal, prior research introduced methods based on spatial frequency domain imaging (SFDI) that allow real-time (within milliseconds) wide-field imaging of optical properties but at a single wavelength. However, for this technology to be useful to clinicians, images must be displayed in terms of metrics related to the physiological state of the tissue, hence interpretable to guide decision-making. For this purpose, recent developments introduced multispectral SFDI methods for rapid imaging of oxygenation parameters up to 16 frames per seconds (fps). We introduce real-time, wide-field, and quantitative blood parameters imaging using spatiotemporal modulation of light. Using this method, we are able to quantitatively obtain optical properties at 100 fps at two wavelengths (665 and 860 nm), and therefore oxygenation, oxyhemoglobin, and deoxyhemoglobin, using a single camera with, at most, 4.2% error in comparison with standard SFDI acquisitions.

Correcting Cherenkov light attenuation in tissue using spatial frequency domain imaging for quantitative surface dosimetry during whole breast radiation therapy

Imaging Cherenkov emission during radiotherapy permits real-time visualization of external beam delivery on superficial tissue. This signal is linear with absorbed dose in homogeneous media, indicating potential for quantitative dosimetry. In humans, the inherent heterogeneity of tissue optical properties (primarily from blood and skin pigment) distorts the linearity between detected Cherenkov signal and absorbed dose. We examine the potential to correct for superficial vasculature using spatial frequency domain imaging (SFDI) to map tissue optical properties for large fields of view. In phantoms, applying intensity corrections to simulate blood vessels improves Cherenkov image (CI) negative contrast by 24% for a vessel 1.9-mm-in diameter. In human trials, SFDI and CI are acquired for women undergoing whole breast radiotherapy. Applied corrections reduce heterogeneity due to vasculature within the sampling limits of the SFDI from a 22% difference as compared to the treatment plan, down to 6% in one region and from 14% down to 4% in another region. The optimal use for this combined imaging system approach is to correct for small heterogeneities such as superficial blood vessels or for interpatient variations in blood/melanin content such that the corrected CI more closely represents the surface dose delivered.

Direct approach to compute Jacobians for diffuse optical tomography using perturbation Monte Carlo-based photon "replay"

Perturbation Monte Carlo (pMC) has been previously proposed to rapidly recompute optical measurements when small perturbations of optical properties are considered, but it was largely restricted to changes associated with prior tissue segments or regions-of-interest. In this work, we expand pMC to compute spatially and temporally resolved sensitivity profiles, i.e. the Jacobians, for diffuse optical tomography (DOT) applications. By recording the pseudo random number generator (PRNG) seeds of each detected photon, we are able to "replay" all detected photons to directly create the 3D sensitivity profiles for both absorption and scattering coefficients. We validate the replay-based Jacobians against the traditional adjoint Monte Carlo (aMC) method, and demonstrate the feasibility of using this approach for efficient 3D image reconstructions using in vitro hyperspectral wide-field DOT measurements. The strengths and limitations of the replay approach regarding its computational efficiency and accuracy are discussed, in comparison with aMC, for point-detector systems as well as wide-field pattern-based and hyperspectral imaging systems. The replay approach has been implemented in both of our open-source MC simulators - MCX and MMC (http://mcx.space).