Compositional-prior-guided image reconstruction algorithm for multi-modality imaging

FDU-Net: Deep Learning-Based Three-Dimensional Diffuse Optical Image Reconstruction

Near-infrared diffuse optical tomography (DOT) is a promising functional modality for breast cancer imaging; however, the clinical translation of DOT is hampered by technical limitations. Specifically, conventional finite element method (FEM)-based optical image reconstruction approaches are time-consuming and ineffective in recovering full lesion contrast. To address this, we developed a deep learning-based reconstruction model (FDU-Net) comprised of a Fully connected subnet, followed by a convolutional encoder-Decoder subnet, and a U-Net for fast, end-to-end 3D DOT image reconstruction. The FDU-Net was trained on digital phantoms that include randomly located singular spherical inclusions of various sizes and contrasts. Reconstruction performance was evaluated in 400 simulated cases with realistic noise profiles for the FDU-Net and conventional FEM approaches. Our results show that the overall quality of images reconstructed by FDU-Net is significantly improved compared to FEM-based methods and a previously proposed deep-learning network. Importantly, once trained, FDU-Net demonstrates substantially better capability to recover true inclusion contrast and location without using any inclusion information during reconstruction. The model was also generalizable to multi-focal and irregularly shaped inclusions unseen during training. Finally, FDU-Net, trained on simulated data, could successfully reconstruct a breast tumor from a real patient measurement. Overall, our deep learning-based approach demonstrates marked superiority over the conventional DOT image reconstruction methods while also offering over four orders of magnitude acceleration in computational time. Once adapted to the clinical breast imaging workflow, FDU-Net has the potential to provide real-time accurate lesion characterization by DOT to assist the clinical diagnosis and management of breast cancer.

Method to improve the localization accuracy and contrast recovery of lesions in separately acquired X-ray and diffuse optical tomographic breast imaging

Near-infrared diffuse optical tomography (DOT) has the potential to improve the accuracy of breast cancer diagnosis and aid in monitoring the response of breast tumors to chemotherapy by providing hemoglobin-based functional imaging. The use of structural lesion priors derived from clinical breast imaging methods, such as mammography, can improve recovery of tumor optical contrast; however, accurate lesion prior placement is essential to take full advantage of prior-guided DOT image reconstruction. Simultaneous optical and anatomical imaging may not always be possible or desired, which can make the accurate registration of the lesion prior challenging. In this paper, we present a three-step lesion prior scanning approach to facilitate improved accuracy in lesion localization based on the optical contrast quantified by the total hemoglobin concentration (HbT) for non-simultaneous multimodal DOT and digital breast tomosynthesis (DBT) imaging. In three challenging breast cancer patient cases, where no clear optical contrast was present initially, we have demonstrated consistent improvement in the recovered HbT lesion contrast by utilizing this method.

Deep-learning based image reconstruction for MRI-guided near-infrared spectral tomography

Non-invasive near-infrared spectral tomography (NIRST) can incorporate the structural information provided by simultaneous magnetic resonance imaging (MRI), and this has significantly improved the images obtained of tissue function. However, the process of MRI guidance in NIRST has been time consuming because of the needs for tissue-type segmentation and forward diffuse modeling of light propagation. To overcome these problems, a reconstruction algorithm for MRI-guided NIRST based on deep learning is proposed and validated by simulation and real patient imaging data for breast cancer characterization. In this approach, diffused optical signals and MRI images were both used as the input to the neural network, and simultaneously recovered the concentrations of oxy-hemoglobin, deoxy-hemoglobin, and water via end-to-end training by using 20,000 sets of computer-generated simulation phantoms. The simulation phantom studies showed that the quality of the reconstructed images was improved, compared to that obtained by other existing reconstruction methods. Reconstructed patient images show that the well-trained neural network with only simulation data sets can be directly used for differentiating malignant from benign breast tumors.

Data-specific mask-guided image reconstruction for diffuse optical tomography

Conventional approaches in diffuse optical tomography (DOT) image reconstruction often address the ill-posed inverse problem via regularization with a constant penalty parameter, which uniformly smooths out the solution. In this study, we present a data-specific mask-guided scheme that incorporates a prior mask constraint into the image reconstruction framework. The prior mask was created from the DOT data itself by exploiting the multi-measurement vector formulation. We accordingly propose two methods to integrate the prior mask into the reconstruction process. First, as a soft prior by exploiting a spatially varying regularization. Second, as a hard prior by imposing a region-of-interest-limited reconstruction. Furthermore, the latter method iterates between discrete and continuous steps to update the mask and optical parameters, respectively. The proposed methods showed enhanced optical contrast accuracy, improved spatial resolution, and reduced noise level in DOT reconstructed images compared with the conventional approaches such as the modified Levenberg-Marquardt approach and the l₁-regularization based sparse recovery approach.

Improved ptychographic inspection of EUV reticles via inclusion of prior information

The development of actinic mask metrology tools represents one of the major challenges to be addressed on the roadmap of extreme ultraviolet (EUV) lithography. Technological advancements in EUV lithography result in the possibility to print increasingly fine and highly resolved structures on a silicon wafer; however, the presence of fine-scale defects, interspersed in the printable mask layout, may lead to defective wafer prints. Hence, the development of actinic methods for review of potential defect sites becomes paramount. Here, we report on a ptychographic algorithm that makes use of prior information about the object to be retrieved, generated by means of rigorous computations, to improve the detectability of defects whose dimensions are of the order of the wavelength. The comprehensive study demonstrates that the inclusion of prior information as a regularizer in the ptychographic optimization problem results in a higher reconstruction quality and an improved robustness to noise with respect to the standard ptychographic iterative engine (PIE). We show that the proposed method decreases the number of scan positions necessary to retrieve a high-quality image and relaxes requirements in terms of signal-to-noise ratio (SNR). The results are further compared with state-of-the-art total variation-based ptychographic imaging.

Impact of errors in experimental parameters on reconstructed breast images using diffuse optical tomography

Near-infrared diffuse optical tomography (NIR-DOT) is an emerging technology that offers hemoglobin based, functional imaging tumor biomarkers for breast cancer management. The most promising clinical translation opportunities are in the differential diagnosis of malignant vs. benign lesions, and in early response assessment and guidance for neoadjuvant chemotherapy. Accurate quantification of the tissue oxy- and deoxy-hemoglobin concentration across the field of view, as well as repeatability during longitudinal imaging in the context of therapy guidance, are essential for the successful translation of NIR-DOT to clinical practice. The ill-posed and ill-condition nature of the DOT inverse problem makes this technique particularly susceptible to model errors that may occur, for example, when the experimental conditions do not fully match the assumptions built into the image reconstruction process. To evaluate the susceptibility of DOT images to experimental errors that might be encountered in practice for a parallel-plate NIR-DOT system, we simulated 7 different types of errors, each with a range of magnitudes. We generated simulated data by using digital breast phantoms derived from five actual mammograms of healthy female volunteers, to which we added a 1-cm tumor. After applying each of the experimental error types and magnitudes to the simulated measurements, we reconstructed optical images with and without structural prior guidance and assessed the overall error in the total hemoglobin concentrations (HbT) and in the HbT contrast between the lesion and surrounding area vs. the best-case scenarios. It is found that slight in-plane probe misalignment and plate rotation did not result in large quantification errors. However, any out-of-plane probe tilting could result in significant deterioration in lesion contrast. Among the error types investigated in this work, optical images were the least likely to be impacted by breast shape inaccuracies but suffered the largest deterioration due to cross-talk between signal channels. However, errors in optical images could be effectively controlled when experimental parameters were properly estimated during data acquisition and accounted for in the image processing procedure. Finally, optical images recovered using structural priors were, in general, less susceptible to experimental errors; however, lesion contrasts were more sensitive to errors when tumor locations were used as a priori info. Findings in this simulation study can provide guidelines for system design and operation in optical breast imaging studies.

Multiobjective guided priors improve the accuracy of near-infrared spectral tomography for breast imaging

An image reconstruction regularization approach for magnetic resonance imaging-guided near-infrared spectral tomography has been developed to improve quantification of total hemoglobin (HbT) and water. By combining prior information from dynamic contrast enhanced (DCE) and diffusion weighted (DW) MR images, the absolute bias errors of HbT and water in the tumor were reduced by 22% and 18%, 21% and 6%, and 10% and 11%, compared to that in the no-prior, DCE- or DW-guided reconstructed images in three-dimensional simulations, respectively. In addition, the apparent contrast values of HbT and water were increased in patient image reconstruction from 1.4 and 1.4 (DCE) or 1.8 and 1.4 (DW) to 4.6 and 1.6.

Direct regularization from co-registered anatomical images for MRI-guided near-infrared spectral tomographic image reconstruction

Combining anatomical information from high resolution imaging modalities to guide near-infrared spectral tomography (NIRST) is an efficient strategy for improving the quality of the reconstructed spectral images. A new approach for incorporating image information directly into the inversion matrix regularization was examined using Direct Regularization from Images (DRI), which encodes the gray-scale data into the NIRST image reconstruction problem. This process has the benefit of eliminating user intervention such as image segmentation of distinct regions. Specifically, the Dynamic Contrast Enhanced Magnetic Resonance (DCE-MR) image intensity value differences within the anatomical image were used to implement an exponentially-weighted regularization function between the image pixels. The algorithm was validated using simulated reconstructions with noise, and the results showed that spatial resolution and robustness of the reconstructed images were significantly improved by appropriate choice of the regularization weight parameters. The proposed approach was also tested on in vivo breast data acquired in a recent clinical trial combining NIRST / MRI for cancer tumor characterization. Relative to the standard "no priors" diffuse recovery, the contrast of the tumor to the normal surrounding tissue increased from 2.4 to 3.6, and the difference between the tumor size segmented from DCE-MR images and reconstructed optical images decreased from 18% to 6%, while there was an overall decrease in surface artifacts.

Characterizing breast lesions through robust multimodal data fusion using independent diffuse optical and x-ray breast imaging

To enable tissue function-based tumor diagnosis over the large number of existing digital mammography systems worldwide, we propose a cost-effective and robust approach to incorporate tomographic optical tissue characterization with separately acquired digital mammograms. Using a flexible contour-based registration algorithm, we were able to incorporate an independently measured two-dimensional x-ray mammogram as structural priors in a joint optical/x-ray image reconstruction, resulting in improved spatial details in the optical images and robust optical property estimation. We validated this approach with a retrospective clinical study of 67 patients, including 30 malignant and 37 benign cases, and demonstrated that the proposed approach can help to distinguish malignant from solid benign lesions and fibroglandular tissues, with a performance comparable to the approach using spatially coregistered optical/x-ray measurements.

Characterization of structural-prior guided optical tomography using realistic breast models derived from dual-energy x-ray mammography

Multi-spectral near-infrared diffuse optical tomography (DOT) is capable of providing functional tissue assessment that can complement structural mammographic images for more comprehensive breast cancer diagnosis. To take full advantage of the readily available sub-millimeter resolution structural information in a multi-modal imaging setting, an efficient x-ray/optical joint image reconstruction model has been proposed previously to utilize anatomical information from a mammogram as a structural prior. In this work, we develop a complex digital breast phantom (available at http://openjd.sf.net/digibreast) based on direct measurements of fibroglandular tissue volume fractions using dual-energy mammographic imaging of a human breast. We also extend our prior-guided reconstruction algorithm to facilitate the recovery of breast tumors, and perform a series of simulation-based studies to systematically evaluate the impact of lesion sizes and contrasts, tissue background, mesh resolution, inaccurate priors, and regularization parameters, on the recovery of breast tumors using multi-modal DOT/x-ray measurements. Our studies reveal that the optical property estimation error can be reduced by half by utilizing structural priors; the minimum detectable tumor size can also be reduced by half when prior knowledge regarding the tumor location is provided. Moreover, our algorithm is shown to be robust to false priors on tumor location.

Shape-parameterized diffuse optical tomography holds promise for sensitivity enhancement of fluorescence molecular tomography

A fundamental approach to enhancing the sensitivity of the fluorescence molecular tomography (FMT) is to incorporate diffuse optical tomography (DOT) to modify the light propagation modeling. However, the traditional voxel-based DOT has been involving a severely ill-posed inverse problem and cannot retrieve the optical property distributions with the acceptable quantitative accuracy and spatial resolution. Although, with the aid of an anatomical imaging modality, the structural-prior-based DOT method with either the hard- or soft-prior scheme holds promise for in vivo acquiring the optical background of tissues, the low robustness of the hard-prior scheme to the segmentation error and inferior performance of the soft-prior one in the quantitative accuracy limit its further application. We propose in this paper a shape-parameterized DOT method for not only effectively determining the regional optical properties but potentially achieving reasonable structural amelioration, lending itself to FMT for comparably improved recovery of fluorescence distribution.

Regularization functional semi-automated incorporation of anatomical prior information in image-guided fluorescence tomography

The use of anatomical priors in fluorescence tomography is known to improve image quality and accuracy significantly. However, the use of prior information is often implemented by incorporating user segmented structural images into the optical reconstruction algorithm, a process requiring significant time and expertise. We propose an automated implementation which encodes the gray-scale prior image directly into the regularization term, eliminating the need for direct prior image segmentation, which is extendable to any spatially defined prior data. The proposed method is supported by in vivo studies.

A review of breast tomosynthesis. Part II. Image reconstruction, processing and analysis, and advanced applications

Many important post-acquisition aspects of breast tomosynthesis imaging can impact its clinical performance. Chief among them is the reconstruction algorithm that generates the representation of the three-dimensional breast volume from the acquired projections. But even after reconstruction, additional processes, such as artifact reduction algorithms, computer aided detection and diagnosis, among others, can also impact the performance of breast tomosynthesis in the clinical realm. In this two part paper, a review of breast tomosynthesis research is performed, with an emphasis on its medical physics aspects. In the companion paper, the first part of this review, the research performed relevant to the image acquisition process is examined. This second part will review the research on the post-acquisition aspects, including reconstruction, image processing, and analysis, as well as the advanced applications being investigated for breast tomosynthesis.

Low-cost three-dimensional imaging system combining fluorescence and ultrasound

In this paper, we present a dual-modality imaging system combining three-dimensional (3D) continuous-wave transillumination fluorescence tomography with 3D ultrasound (US) imaging. We validated the system with two phantoms, one containing fluorescent inclusions (Cy5.5) at different depths, and another varying-thickness semicylindrical phantom. Using raster scanning, the combined fluorescence/US system was used to collect the boundary fluorescent emission in the X-Y plane, as well as recovered the 3D surface and position of the inclusions from US signals. US images were segmented to provide soft priors for the fluorescence image reconstruction. Phantom results demonstrated that with priors derived from the US images, the fluorescent reconstruction quality was significantly improved. As further evaluation, we show pilot in vivo results using an Apo-E mouse to assess the feasibility and performance of this system in animal studies. Limitations and potential to be used in artherosclerosis studies are then discussed.

Towards the next generation of near-infrared spectroscopy

Although near-infrared spectroscopy (NIRS) was originally designed for clinical monitoring of tissue oxygenation, it has also been developing into a useful tool for neuroimaging studies (functional NIRS). Over the past 30 years, technology has developed and NIRS has found a wide range of applications. However, the accuracy and reliability of NIRS have not yet been widely accepted, mainly because of the difficulties in selective and quantitative detection of signals arising in cerebral tissue, which subject the use of NIRS to a number of practical restrictions. This review summarizes the strengths and advantages of NIRS over other neuroimaging modalities and demonstrates specific examples. The issues of selective quantitative measurement of cerebral haemoglobin during brain activation are also discussed, together with the problems of applying the methods of functional magnetic resonance imaging data analysis to NIRS data analysis. Finally, near-infrared optical tomography--the next generation of NIRS--is described as a potential technique to overcome the limitations of NIRS.

Investigating the impact of spatial priors on the performance of model-based IVUS elastography

This paper describes methods that provide pre-requisite information for computing circumferential stress in modulus elastograms recovered from vascular tissue-information that could help cardiologists detect life-threatening plaques and predict their propensity to rupture. The modulus recovery process is an ill-posed problem; therefore, additional information is needed to provide useful elastograms. In this work, prior geometrical information was used to impose hard or soft constraints on the reconstruction process. We conducted simulation and phantom studies to evaluate and compare modulus elastograms computed with soft and hard constraints versus those computed without any prior information. The results revealed that (1) the contrast-to-noise ratio of modulus elastograms achieved using the soft prior and hard prior reconstruction methods exceeded those computed without any prior information; (2) the soft prior and hard prior reconstruction methods could tolerate up to 8% measurement noise, and (3) the performance of soft and hard prior modulus elastograms degraded when incomplete spatial priors were employed. This work demonstrates that including spatial priors in the reconstruction process should improve the performance of model-based elastography, and the soft prior approach should enhance the robustness of the reconstruction process to errors in the geometrical information.