Convolutional neural network-based approach to estimate bulk optical properties in diffuse optical tomography

Deep orthogonal multi-wavelength fusion for tomogram-free diagnosis in diffuse optical imaging

Novel portable diffuse optical tomography (DOT) devices for breast cancer lesions hold great promise for non-invasive, non-ionizing breast cancer screening. Critical to this capability is not just the identification of lesions but rather the complex problem of discriminating between malignant and benign lesions. To accurately reconstruct the highly heterogeneous tissue of a cancer lesion in healthy breast tissue using DOT, multiple wavelengths can be leveraged to maximize signal penetration while minimizing sensitivity to noise. However, these wavelength responses can overlap, capture common information, and correlate, potentially confounding reconstruction and downstream end tasks. We show that an orthogonal fusion loss regularizes multi-wavelength DOT leading to improved reconstruction and accuracy of end-to-end discrimination of malignant versus benign lesions. We further show that our raw-to-task model significantly reduces computational complexity without sacrificing accuracy, making it ideal for real-time throughput, desired in medical settings where handheld devices have severely restricted power budgets. Furthermore, our results indicate that image reconstruction is not necessary for unbiased classification of lesions with a balanced accuracy of 77% and 66% on the synthetic dataset and clinical dataset, respectively, using the raw-to-task model. Code is available at https://github.com/sfu-mial/FuseNet.

Enhanced model iteration algorithm with graph neural network for diffuse optical tomography

Diffuse optical tomography (DOT) employs near-infrared light to reveal the optical parameters of biological tissues. Due to the strong scattering of photons in tissues and the limited surface measurements, DOT reconstruction is severely ill-posed. The Levenberg-Marquardt (LM) is a popular iteration method for DOT, however, it is computationally expensive and its reconstruction accuracy needs improvement. In this study, we propose a neural model based iteration algorithm which combines the graph neural network with Levenberg-Marquardt (GNNLM), which utilizes a graph data structure to represent the finite element mesh. In order to verify the performance of the graph neural network, two GNN variants, namely graph convolutional neural network (GCN) and graph attention neural network (GAT) were employed in the experiments. The results showed that GCNLM performs best in the simulation experiments within the training data distribution. However, GATLM exhibits superior performance in the simulation experiments outside the training data distribution and real experiments with breast-like phantoms. It demonstrated that the GATLM trained with simulation data can generalize well to situations outside the training data distribution without transfer training. This offers the possibility to provide more accurate absorption coefficient distributions in clinical practice.

Automated pipeline for breast cancer diagnosis using US assisted diffuse optical tomography

Ultrasound (US)-guided diffuse optical tomography (DOT) is a portable and non-invasive imaging modality for breast cancer diagnosis and treatment response monitoring. However, DOT data pre-processing and imaging reconstruction often require labor intensive manual processing which hampers real-time diagnosis. In this study, we aim at providing an automated US-assisted DOT pre-processing, imaging and diagnosis pipeline to achieve near real-time diagnosis. We have developed an automated DOT pre-processing method including motion detection, mismatch classification using deep-learning approach, and outlier removal. US-lesion information needed for DOT reconstruction was extracted by a semi-automated lesion segmentation approach combined with a US reading algorithm. A deep learning model was used to evaluate the quality of the reconstructed DOT images and a two-step deep-learning model developed earlier is implemented to provide final diagnosis based on US imaging features and DOT measurements and imaging results. The presented US-assisted DOT pipeline accurately processed the DOT measurements and reconstruction and reduced the procedure time to 2 to 3 minutes while maintained a comparable classification result with manually processed dataset.

Two-stage classification strategy for breast cancer diagnosis using ultrasound-guided diffuse optical tomography and deep learning

Significance

Ultrasound (US)-guided diffuse optical tomography (DOT) has demonstrated great potential for breast cancer diagnosis in which real-time or near real-time diagnosis with high accuracy is desired.

Aim

We aim to use US-guided DOT to achieve an automated, fast, and accurate classification of breast lesions.

Approach

We propose a two-stage classification strategy with deep learning. In the first stage, US images and histograms created from DOT perturbation measurements are combined to predict benign lesions. Then the non-benign suspicious lesions are passed through to the second stage, which combine US image features, DOT histogram features, and 3D DOT reconstructed images for final diagnosis.

Results

The first stage alone identified 73.0% of benign cases without image reconstruction. In distinguishing between benign and malignant breast lesions in patient data, the two-stage classification approach achieved an area under the receiver operating characteristic curve of 0.946, outperforming the diagnoses of all single-modality models and of a single-stage classification model that combines all US images, DOT histogram, and imaging features.

Conclusions

The proposed two-stage classification strategy achieves better classification accuracy than single-modality-only models and a single-stage classification model that combines all features. It can potentially distinguish breast cancers from benign lesions in near real-time.

Physics-guided neural network for tissue optical properties estimation

Finding the optical properties of tissue is essential for various biomedical diagnostic/therapeutic applications such as monitoring of blood oxygenation, tissue metabolism, skin imaging, photodynamic therapy, low-level laser therapy, and photo-thermal therapy. Hence, the research for more accurate and versatile optical properties estimation techniques has always been a primary interest of researchers, especially in the field of bioimaging and bio-optics. In the past, most of the prediction methods were based on physics-based models such as the pronounced diffusion approximation method. In more recent years, with the advancement and growing popularity of machine learning techniques, most of the prediction methods are data-driven. While both methods have been proven to be useful, each of them suffers from several shortcomings that could be complemented by their counterparts. Thus, there is a need to bring the two domains together to obtain superior prediction accuracy and generalizability. In this work, we proposed a physics-guided neural network (PGNN) for tissue optical properties regression which integrates physics prior and constraint into the artificial neural network (ANN) model. With this method, we have demonstrated superior generalizability of PGNN compared to its pure ANN counterpart. The prediction accuracy and generalizability of the network were evaluated on single-layered tissue samples simulated with Monte Carlo simulation. Two different test datasets, the in-domain test dataset and out-domain dataset were used to evaluate in-domain generalizability and out-domain generalizability, respectively. The physics-guided neural network (PGNN) showed superior generalizability for both in-domain and out-domain prediction compared to pure ANN.

High-Speed Time-Domain Diffuse Optical Tomography with a Sensitivity Equation-based Neural Network

Steady progress in time-domain diffuse optical tomography (TD-DOT) technology is allowing for the first time the design of low-cost, compact, and high-performance systems, thus promising more widespread clinical TD-DOT use, such as for recording brain tissue hemodynamics. TD-DOT is known to provide more accurate values of optical properties and physiological parameters compared to its frequency-domain or steady-state counterparts. However, achieving high temporal resolution is still difficult, as solving the inverse problem is computationally demanding, leading to relatively long reconstruction times. The runtime is further compromised by processes that involve 'nontrivial' empirical tuning of reconstruction parameters, which increases complexity and inefficiency. To address these challenges, we present a new reconstruction algorithm that combines a deep-learning approach with our previously introduced sensitivity-equation-based, non-iterative sparse optical reconstruction (SENSOR) code. The new algorithm (called SENSOR-NET) unfolds the iterations of SENSOR into a deep neural network. In this way, we achieve high-resolution sparse reconstruction using only learned parameters, thus eliminating the need to tune parameters prior to reconstruction empirically. Furthermore, once trained, the reconstruction time is not dependent on the number of sources or wavelengths used. We validate our method with numerical and experimental data and show that accurate reconstructions with 1 mm spatial resolution can be obtained in under 20 milliseconds regardless of the number of sources used in the setup. This opens the door for real-time brain monitoring and other high-speed DOT applications.

Fusion deep learning approach combining diffuse optical tomography and ultrasound for improving breast cancer classification

Diffuse optical tomography (DOT) is a promising technique that provides functional information related to tumor angiogenesis. However, reconstructing the DOT function map of a breast lesion is an ill-posed and underdetermined inverse process. A co-registered ultrasound (US) system that provides structural information about the breast lesion can improve the localization and accuracy of DOT reconstruction. Additionally, the well-known US characteristics of benign and malignant breast lesions can further improve cancer diagnosis based on DOT alone. Inspired by a fusion model deep learning approach, we combined US features extracted by a modified VGG-11 network with images reconstructed from a DOT deep learning auto-encoder-based model to form a new neural network for breast cancer diagnosis. The combined neural network model was trained with simulation data and fine-tuned with clinical data: it achieved an AUC of 0.931 (95% CI: 0.919-0.943), superior to those achieved using US images alone (0.860) or DOT images alone (0.842).

Machine Learning Diffuse Optical Tomography Using Extreme Gradient Boosting and Genetic Programming

Diffuse optical tomography (DOT) is a non-invasive method for detecting breast cancer; however, it struggles to produce high-quality images due to the complexity of scattered light and the limitations of traditional image reconstruction algorithms. These algorithms can be affected by boundary conditions and have a low imaging accuracy, a shallow imaging depth, a long computation time, and a high signal-to-noise ratio. However, machine learning can potentially improve the performance of DOT by being better equipped to solve inverse problems, perform regression, classify medical images, and reconstruct biomedical images. In this study, we utilized a machine learning model called "XGBoost" to detect tumors in inhomogeneous breasts and applied a post-processing technique based on genetic programming to improve accuracy. The proposed algorithm was tested using simulated DOT measurements from complex inhomogeneous breasts and evaluated using the cosine similarity metrics and root mean square error loss. The results showed that the use of XGBoost and genetic programming in DOT could lead to more accurate and non-invasive detection of tumors in inhomogeneous breasts compared to traditional methods, with the reconstructed breasts having an average cosine similarity of more than 0.97 ± 0.07 and average root mean square error of around 0.1270 ± 0.0031 compared to the ground truth.

Unrolled-DOT: an interpretable deep network for diffuse optical tomography

SIGNIFICANCE

Imaging through scattering media is critical in many biomedical imaging applications, such as breast tumor detection and functional neuroimaging. Time-of-flight diffuse optical tomography (ToF-DOT) is one of the most promising methods for high-resolution imaging through scattering media. ToF-DOT and many traditional DOT methods require an image reconstruction algorithm. Unfortunately, this algorithm often requires long computational runtimes and may produce lower quality reconstructions in the presence of model mismatch or improper hyperparameter tuning.

AIM

We used a data-driven unrolled network as our ToF-DOT inverse solver. The unrolled network is faster than traditional inverse solvers and achieves higher reconstruction quality by accounting for model mismatch.

APPROACH

Our model "Unrolled-DOT" uses the learned iterative shrinkage thresholding algorithm. In addition, we incorporate a refinement U-Net and Visual Geometry Group (VGG) perceptual loss to further increase the reconstruction quality. We trained and tested our model on simulated and real-world data and benchmarked against physics-based and learning-based inverse solvers.

RESULTS

In experiments on real-world data, Unrolled-DOT outperformed learning-based algorithms and achieved over 10× reduction in runtime and mean-squared error, compared to traditional physics-based solvers.

CONCLUSION

We demonstrated a learning-based ToF-DOT inverse solver that achieves state-of-the-art performance in speed and reconstruction quality, which can aid in future applications for noninvasive biomedical imaging.

Periodic-net: an end-to-end data driven framework for diffuse optical imaging of breast cancer from noisy boundary data

SIGNIFICANCE

The machine learning (ML) approach plays a critical role in assessing biomedical imaging processes especially optical imaging (OI) including segmentation, classification, and reconstruction, intending to achieve higher accuracy efficiently.

AIM

This research aims to develop an end-to-end deep learning framework for diffuse optical imaging (DOI) with multiple datasets to detect breast cancer and reconstruct its optical properties in the early stages.

APPROACH

The proposed Periodic-net is a nondestructive deep learning (DL) algorithm for the reconstruction and evaluation of inhomogeneities in an inverse model with high accuracy, while boundary measurements are calculated by solving a forward problem with sources/detectors arranged uniformly around a circular domain in various combinations, including 16 × 15 , 20 × 19 , and 36 × 35 boundary measurement setups.

RESULTS

The results of image reconstruction on numerical and phantom datasets demonstrate that the proposed network provides higher-quality images with a greater amount of small details, superior immunity to noise, and sharper edges with a reduction in image artifacts than other state-of-the-art competitors.

CONCLUSIONS

The network is highly effective at the simultaneous reconstruction of optical properties, i.e., absorption and reduced scattering coefficients, by optimizing the imaging time without degrading inclusions localization and image quality.

Reconstructing a Deblurred 3D Structure in a Turbid Medium from a Single Blurred 2D Image—For Near-Infrared Transillumination Imaging of a Human Body

To provide another modality for three-dimensional (3D) medical imaging, new techniques were developed to reconstruct a 3D structure in a turbid medium from a single blurred 2D image obtained using near-infrared transillumination imaging. One technique uses 1D information of a curvilinear absorber, or the intensity profile across the absorber image. Profiles in different conditions are calculated by convolution with the depth-dependent point spread function (PSF) of the transillumination image. In databanks, profiles are stored as lookup tables to connect the contrast and spread of the profile to the absorber depth. One-to-one correspondence from the contrast and spread to the absorber depth and thickness were newly found. Another technique uses 2D information of the transillumination image of a volumetric absorber. A blurred 2D image is deconvolved with the depth-dependent PSF, thereby producing many images with points of focus on different parts. The depth of the image part can be estimated by searching the deconvolved images for the image part in the best focus. To suppress difficulties of high-spatial-frequency noise, we applied a noise-robust focus stacking method. Experimentation verified the feasibility of the proposed techniques, and suggested their applicability to curvilinear and volumetric absorbers such as blood vessel networks and cancerous lesions in tissues.

A Model-Based Iterative Learning Approach for Diffuse Optical Tomography

Diffuse optical tomography (DOT) utilises near-infrared light for imaging spatially distributed optical parameters, typically the absorption and scattering coefficients. The image reconstruction problem of DOT is an ill-posed inverse problem, due to the non-linear light propagation in tissues and limited boundary measurements. The ill-posedness means that the image reconstruction is sensitive to measurement and modelling errors. The Bayesian approach for the inverse problem of DOT offers the possibility of incorporating prior information about the unknowns, rendering the problem less ill-posed. It also allows marginalisation of modelling errors utilising the so-called Bayesian approximation error method. A more recent trend in image reconstruction techniques is the use of deep learning, which has shown promising results in various applications from image processing to tomographic reconstructions. In this work, we study the non-linear DOT inverse problem of estimating the (absolute) absorption and scattering coefficients utilising a 'model-based' learning approach, essentially intertwining learned components with the model equations of DOT. The proposed approach was validated with 2D simulations and 3D experimental data. We demonstrated improved absorption and scattering estimates for targets with a mix of smooth and sharp image features, implying that the proposed approach could learn image features that are difficult to model using standard Gaussian priors. Furthermore, it was shown that the approach can be utilised in compensating for modelling errors due to coarse discretisation enabling computationally efficient solutions. Overall, the approach provided improved computation times compared to a standard Gauss-Newton iteration.

Regression-based neural network for improving image reconstruction in diffuse optical tomography

Diffuse optical tomography (DOT) is a non-invasive imaging technique utilizing multi-scattered light at visible and infrared wavelengths to detect anomalies in tissues. However, the DOT image reconstruction is based on solving the inverse problem, which requires massive calculations and time. In this article, for the first time, to the best of our knowledge, a simple, regression-based cascaded feed-forward deep learning neural network is derived to solve the inverse problem of DOT in compressed breast geometry. The predicted data is subsequently utilized to visualize the breast tissues and their anomalies. The dataset in this study is created using a Monte-Carlo algorithm, which simulates the light propagation in the compressed breast placed inside a parallel plate source-detector geometry (forward process). The simulated DL-DOT system's performance is evaluated using the Pearson correlation coefficient (R) and the Mean squared error (MSE) metrics. Although a comparatively smaller dataset (50 nos.) is used, our simulation results show that the developed feed-forward network algorithm to solve the inverse problem delivers an increment of ∼30% over the analytical solution approach, in terms of R. Furthermore, the proposed network's MSE outperforms that of the analytical solution's MSE by a large margin revealing the robustness of the network and the adaptability of the system for potential applications in medical settings.

Deep-learning based image reconstruction for MRI-guided near-infrared spectral tomography

Non-invasive near-infrared spectral tomography (NIRST) can incorporate the structural information provided by simultaneous magnetic resonance imaging (MRI), and this has significantly improved the images obtained of tissue function. However, the process of MRI guidance in NIRST has been time consuming because of the needs for tissue-type segmentation and forward diffuse modeling of light propagation. To overcome these problems, a reconstruction algorithm for MRI-guided NIRST based on deep learning is proposed and validated by simulation and real patient imaging data for breast cancer characterization. In this approach, diffused optical signals and MRI images were both used as the input to the neural network, and simultaneously recovered the concentrations of oxy-hemoglobin, deoxy-hemoglobin, and water via end-to-end training by using 20,000 sets of computer-generated simulation phantoms. The simulation phantom studies showed that the quality of the reconstructed images was improved, compared to that obtained by other existing reconstruction methods. Reconstructed patient images show that the well-trained neural network with only simulation data sets can be directly used for differentiating malignant from benign breast tumors.

Tutorial on the Use of Deep Learning in Diffuse Optical Tomography

Diffuse optical tomography using deep learning is an emerging technology that has found impressive medical diagnostic applications. However, creating an optical imaging system that uses visible and near-infrared (NIR) light is not straightforward due to photon absorption and multi-scattering by tissues. The high distortion levels caused due to these effects make the image reconstruction incredibly challenging. To overcome these challenges, various techniques have been proposed in the past, with varying success. One of the most successful techniques is the application of deep learning algorithms in diffuse optical tomography. This article discusses the current state-of-the-art diffuse optical tomography systems and comprehensively reviews the deep learning algorithms used in image reconstruction. This article attempts to provide researchers with the necessary background and tools to implement deep learning methods to solve diffuse optical tomography.

Deep learning-based method to accurately estimate breast tissue optical properties in the presence of the chest wall

SIGNIFICANCE

In general, image reconstruction methods used in diffuse optical tomography (DOT) are based on diffusion approximation, and they consider the breast tissue as a homogenous, semi-infinite medium. However, the semi-infinite medium assumption used in DOT reconstruction is not valid when the chest wall is underneath the breast tissue.

AIM

We aim to reduce the chest wall's effect on the estimated average optical properties of breast tissue and obtain accurate forward model for DOT reconstruction.

APPROACH

We propose a deep learning-based neural network approach where a convolution neural network (CNN) is trained to simultaneously obtain accurate optical property values for both the breast tissue and the chest wall.

RESULTS

The CNN model shows great promise in reducing errors in estimating the optical properties of the breast tissue in the presence of a shallow chest wall. For patient data, the CNN model predicted the breast tissue optical absorption coefficient, which was independent of chest wall depth.

CONCLUSIONS

Our proposed method can be readily used in DOT and diffuse spectroscopy measurements to improve the accuracy of estimated tissue optical properties.

Three-dimensional imaging through turbid media using deep learning: NIR transillumination imaging of animal bodies

Using near-infrared (NIR) light with 700-1200 nm wavelength, transillumination images of small animals and thin parts of a human body such as a hand or foot can be obtained. They are two-dimensional (2D) images of internal absorbing structures in a turbid medium. A three-dimensional (3D) see-through image is obtainable if one can identify the depth of each part of the structure in the 2D image. Nevertheless, the obtained transillumination images are blurred severely because of the strong scattering in the turbid medium. Moreover, ascertaining the structure depth from a 2D transillumination image is difficult. To overcome these shortcomings, we have developed a new technique using deep learning principles. A fully convolutional network (FCN) was trained with 5,000 training pairs of clear and blurred images. Also, a convolutional neural network (CNN) was trained with 42,000 training pairs of blurred images and corresponding depths in a turbid medium. Numerous training images were provided by the convolution with a point spread function derived from diffusion approximation to the radiative transport equation. The validity of the proposed technique was confirmed through simulation. Experiments demonstrated its applicability. This technique can provide a new tool for the NIR imaging of animal bodies and biometric authentication of a human body.

Data-specific mask-guided image reconstruction for diffuse optical tomography

Conventional approaches in diffuse optical tomography (DOT) image reconstruction often address the ill-posed inverse problem via regularization with a constant penalty parameter, which uniformly smooths out the solution. In this study, we present a data-specific mask-guided scheme that incorporates a prior mask constraint into the image reconstruction framework. The prior mask was created from the DOT data itself by exploiting the multi-measurement vector formulation. We accordingly propose two methods to integrate the prior mask into the reconstruction process. First, as a soft prior by exploiting a spatially varying regularization. Second, as a hard prior by imposing a region-of-interest-limited reconstruction. Furthermore, the latter method iterates between discrete and continuous steps to update the mask and optical parameters, respectively. The proposed methods showed enhanced optical contrast accuracy, improved spatial resolution, and reduced noise level in DOT reconstructed images compared with the conventional approaches such as the modified Levenberg-Marquardt approach and the l₁-regularization based sparse recovery approach.

Development of digital breast tomosynthesis and diffuse optical tomography fusion imaging for breast cancer detection

Diffuse optical tomography (DOT) non-invasively measures the functional characteristics of breast lesions using near infrared light to probe tissue optical properties. This study aimed to evaluate a new digital breast tomosynthesis (DBT)/DOT fusion imaging technique and obtain preliminary data for breast cancer detection. Twenty-eight women were prospectively enrolled and underwent both DBT and DOT examinations. DBT/DOT fusion imaging was created after acquisition of both examinations. Two breast radiologists analyzed DBT and DOT images independently, and then finally evaluated the fusion images. The diagnostic performance of each reading session was compared and interobserver agreement was assessed. The technical success rate was 96.4%, with one failure due to an error during DOT data storage. Among the 27 women finally included in the analysis, 13 had breast cancer. The areas under the receiver operating characteristic curve (AUCs) for DBT were 0.783 and 0.854 for readers 1 and 2, respectively. DOT showed comparable diagnostic performance to DBT for both readers. The AUCs were significantly improved (P = 0.004) when the DBT/DOT fusion images were used. Interobserver agreements were highest for the DBT/DOT fusion images. In conclusion, this study suggests that DBT/DOT fusion imaging technique appears to be a promising tool for breast cancer diagnosis.