(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Bimodal PET/MRI generative reconstruction based on VAE architectures

Objective.In this study, we explore positron emission tomography (PET)/magnetic resonance imaging (MRI) joint reconstruction within a deep learning framework, introducing a novel synergistic method.Approach.We propose a new approach based on a variational autoencoder (VAE) constraint combined with the alternating direction method of multipliers (ADMM) optimization technique. We explore three VAE architectures, joint VAE, product of experts-VAE and multimodal JS divergence (MMJSD), to determine the optimal latent representation for the two modalities. We then trained and evaluated the architectures on a brain PET/MRI dataset.Main results.We showed that our approach takes advantage of each modality sharing information to each other, which results in improved peak signal-to-noise ratio and structural similarity as compared with traditional reconstruction, particularly for short acquisition times. We find that the one particular architecture, MMJSD, is the most effective for our methodology.Significance.The proposed method outperforms conventional approaches especially in noisy and undersampled conditions by making use of the two modalities together to compensate for the missing information.

Enhancing ultrasonic attenuation images through multi-frequency coupling with total nuclear variation

Quantitative ultrasound is a non-invasive image modality that numerically characterizes tissues for medical diagnosis using acoustical parameters, such as the attenuation coefficient slope. A previous study introduced the total variation spectral log difference (TVSLD) method, which denoises spectral log ratios on a single-channel basis without inter-channel coupling. Therefore, this work proposes a multi-frequency joint framework by coupling information across frequency channels exploiting structural similarities among the spectral ratios to increase the quality of the attenuation images. A modification based on the total nuclear variation (TNV) was considered. Metrics were compared to the TVSLD method with simulated and experimental phantoms and two samples of fibroadenoma in vivo breast tissue. The TNV demonstrated superior performance, yielding enhanced attenuation coefficient slope maps with fewer artifacts at boundaries and a stable error. In terms of the contrast-to-noise ratio enhancement, the TNV approach obtained an average percentage improvement of 34% in simulation, 38% in the experimental phantom, and 89% in two in vivo breast tissue samples compared to TVSLD, showing potential to enhance visual clarity and depiction of attenuation images.

Derivative-Free Iterative One-Step Reconstruction for Multispectral CT

Image reconstruction in multispectral computed tomography (MSCT) requires solving a challenging nonlinear inverse problem, commonly tackled via iterative optimization algorithms. Existing methods necessitate computing the derivative of the forward map and potentially its regularized inverse. In this work, we present a simple yet highly effective algorithm for MSCT image reconstruction, utilizing iterative update mechanisms that leverage the full forward model in the forward step and a derivative-free adjoint problem. Our approach demonstrates both fast convergence and superior performance compared to existing algorithms, making it an interesting candidate for future work. We also discuss further generalizations of our method and its combination with additional regularization and other data discrepancy terms.

Systematic Review on Learning-based Spectral CT

Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.

A list-mode multi-energy window low-count SPECT reconstruction method for isotopes with multiple emission peaks

BACKGROUND

Single-photon emission computed tomography (SPECT) provides a mechanism to perform absorbed-dose quantification tasks for [Formula: see text]-particle radiopharmaceutical therapies ([Formula: see text]-RPTs). However, quantitative SPECT for [Formula: see text]-RPT is challenging due to the low number of detected counts, the complex emission spectrum, and other image-degrading artifacts. Towards addressing these challenges, we propose a low-count quantitative SPECT reconstruction method for isotopes with multiple emission peaks.

METHODS

Given the low-count setting, it is important that the reconstruction method extracts the maximal possible information from each detected photon. Processing data over multiple energy windows and in list-mode (LM) format provide mechanisms to achieve that objective. Towards this goal, we propose a list-mode multi energy window (LM-MEW) ordered-subsets expectation-maximization-based SPECT reconstruction method that uses data from multiple energy windows in LM format and include the energy attribute of each detected photon. For computational efficiency, we developed a multi-GPU-based implementation of this method. The method was evaluated using 2-D SPECT simulation studies in a single-scatter setting conducted in the context of imaging [[Formula: see text]Ra]RaCl[Formula: see text], an FDA-approved RPT for metastatic prostate cancer.

RESULTS

The proposed method yielded improved performance on the task of estimating activity uptake within known regions of interest in comparison to approaches that use a single energy window or use binned data. The improved performance was observed in terms of both accuracy and precision and for different sizes of the region of interest.

CONCLUSIONS

Results of our studies show that the use of multiple energy windows and processing data in LM format with the proposed LM-MEW method led to improved quantification performance in low-count SPECT of isotopes with multiple emission peaks. These results motivate further development and validation of the LM-MEW method for such imaging applications, including for [Formula: see text]-RPT SPECT.

Triple modality image reconstruction of PET data using SPECT, PET, CT information increases lesion uptake in images of patients treated with radioembolization with [Formula: see text] micro-spheres

PURPOSE

Nuclear medicine imaging modalities like computed tomography (CT), single photon emission CT (SPECT) and positron emission tomography (PET) are employed in the field of theranostics to estimate and plan the dose delivered to tumors and the surrounding tissues and to monitor the effect of the therapy. However, therapeutic radionuclides often provide poor images, which translate to inaccurate treatment planning and inadequate monitoring images. Multimodality information can be exploited in the reconstruction to enhance image quality. Triple modality PET/SPECT/CT scanners are particularly useful in this context due to the easier registration process between images. In this study, we propose to include PET, SPECT and CT information in the reconstruction of PET data. The method is applied to Yttrium-90 ([Formula: see text]Y) data.

METHODS

Data from a NEMA phantom filled with [Formula: see text]Y were used for validation. PET, SPECT and CT data from 10 patients treated with Selective Internal Radiation Therapy (SIRT) were used. Different combinations of prior images using the Hybrid kernelized expectation maximization were investigated in terms of VOI activity and noise suppression.

RESULTS

Our results show that triple modality PET reconstruction provides significantly higher uptake when compared to the method used as standard in the hospital and OSEM. In particular, using CT-guided SPECT images, as guiding information in the PET reconstruction significantly increases uptake quantification on tumoral lesions.

CONCLUSION

This work proposes the first triple modality reconstruction method and demonstrates up to 69% lesion uptake increase over standard methods with SIRT [Formula: see text]Y patient data. Promising results are expected for other radionuclide combination used in theranostic applications using PET and SPECT.

Evaluation of a pipeline for simulation, reconstruction, and classification in ultrasound-aided diffuse optical tomography of breast tumors

SIGNIFICANCE

Diffuse optical tomography is an ill-posed problem. Combination with ultrasound can improve the results of diffuse optical tomography applied to the diagnosis of breast cancer and allow for classification of lesions.

AIM

To provide a simulation pipeline for the assessment of reconstruction and classification methods for diffuse optical tomography with concurrent ultrasound information.

APPROACH

A set of breast digital phantoms with benign and malignant lesions was simulated building on the software VICTRE. Acoustic and optical properties were assigned to the phantoms for the generation of B-mode images and optical data. A reconstruction algorithm based on a two-region nonlinear fitting and incorporating the ultrasound information was tested. Machine learning classification methods were applied to the reconstructed values to discriminate lesions into benign and malignant after reconstruction.

RESULTS

The approach allowed us to generate realistic US and optical data and to test a two-region reconstruction method for a large number of realistic simulations. When information is extracted from ultrasound images, at least 75% of lesions are correctly classified. With ideal two-region separation, the accuracy is higher than 80%.

CONCLUSIONS

A pipeline for the generation of realistic ultrasound and diffuse optics data was implemented. Machine learning methods applied to a optical reconstruction with a nonlinear optical model and morphological information permit to discriminate malignant lesions from benign ones.

Synergistic tomographic image reconstruction: part 2

This special issue is the second part of a themed issue that focuses on synergistic tomographic image reconstruction and includes a range of contributions in multiple disciplines and application areas. The primary subject of study lies within inverse problems which are tackled with various methods including statistical and computational approaches. This volume covers algorithms and methods for a wide range of imaging techniques such as spectral X-ray computed tomography (CT), positron emission tomography combined with CT or magnetic resonance imaging, bioluminescence imaging and fluorescence-mediated imaging as well as diffuse optical tomography combined with ultrasound. Some of the articles demonstrate their utility on real-world challenges, either medical applications (e.g. motion compensation for imaging patients) or applications in material sciences (e.g. material decomposition and characterization). One of the desired outcomes of the special issues is to bring together different scientific communities which do not usually interact as they do not share the same platforms such as journals and conferences. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

Enhanced diffuse optical tomographic reconstruction using concurrent ultrasound information

Multimodal imaging is an active branch of research as it has the potential to improve common medical imaging techniques. Diffuse optical tomography (DOT) is an example of a low resolution, functional imaging modality that typically has very low resolution due to the ill-posedness of its underlying inverse problem. Combining the functional information of DOT with a high resolution structural imaging modality has been studied widely. In particular, the combination of DOT with ultrasound (US) could serve as a useful tool for clinicians for the formulation of accurate diagnosis of breast lesions. In this paper, we propose a novel method for US-guided DOT reconstruction using a portable time-domain measurement system. B-mode US imaging is used to retrieve morphological information on the probed tissues by means of a semi-automatical segmentation procedure based on active contour fitting. A two-dimensional to three-dimensional extrapolation procedure, based on the concept of distance transform, is then applied to generate a three-dimensional edge-weighting prior for the regularization of DOT. The reconstruction procedure has been tested on experimental data obtained on specifically designed dual-modality silicon phantoms. Results show a substantial quantification improvement upon the application of the implemented technique. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

Synergistic tomographic image reconstruction: part 1

This special issue focuses on synergistic tomographic image reconstruction in a range of contributions in multiple disciplines and various application areas. The topic of image reconstruction covers substantial inverse problems (Mathematics) which are tackled with various methods including statistical approaches (e.g. Bayesian methods, Monte Carlo) and computational approaches (e.g. machine learning, computational modelling, simulations). The issue is separated in two volumes. This volume focuses mainly on algorithms and methods. Some of the articles will demonstrate their utility on real-world challenges, either medical applications (e.g. cardiovascular diseases, proton therapy planning) or applications in material sciences (e.g. material decomposition and characterization). One of the desired outcomes of the special issue is to bring together different scientific communities which do not usually interact as they do not share the same platforms (such as journals and conferences). This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.