Deep learning-based image reconstruction for TOF PET with DIRECT data partitioning format

The effects of back-projection variants in BPF-like TOF PET reconstruction using CNN filtration - Based on simulated and clinical brain data

BACKGROUND

The back-projection strategies such as confidence weighting (CW) and most likely annihilation position (MLAP) have been adopted into back-projection-and-filtering-like (BPF-like) deep reconstruction model and shown great potential on fast and accurate PET reconstruction. Although the two methods degenerate to an identical model at the time resolution of 0 ps, they represent two distinct approaches at the realistic time resolutions of current commercial systems. There is a lack of a systematic and fair assessment on these differences.

PURPOSE

This work aims to analyze the impact of back-projection variants on CNN-based PET image reconstruction to find the most effective back-projection model, and ultimately contribute to accurate PET reconstruction.

METHODS

Different back-projection strategies (CW and MLAP) and different angular view processing methods (view-summed and view-grouped) were considered, leading to the comparison of four back-projection variants integrated with the same CNN filtration model. Meanwhile, we investigated two strategies of physical effect compensation, either introducing pre-corrected data as the input or adding a channel of attenuation map to the CNN model. After training models separately on Monte-Carlo-simulated BrainWeb phantoms with full dose (events = 3×107), we tested them on both simulated phantoms and clinical brain scans with two dosage levels. For the performance assessment, peak signal-to-noise ratio (PSNR) and root mean square error (RMSE) were used to evaluate the pixel-wise error, structural similarity index (SSIM) to evaluate the structural similarity, and contrast recovery coefficient (CRC) in manually selected ROI to compare the region recovery.

RESULTS

Compared to two MLAP-based histo-image reconstruction models, two CW-based back-projected image methods produced clearer, sharper, and more detailed images, from both simulated and clinical data. For angular view processing methods, view-grouped histo-image improved image quality, while view-grouped cwbp-image showed no advantage except for contrast recovery. Quantitative analysis on simulated data demonstrated that the view-summed cwbp-image model achieved the best PSNR, RMSE, SSIM, while the 8-view cwbp-image model achieved the best CRC in lesions and the white matter. Additionally, the multi-channel input model including the back-projection image and attenuation map was proved to be the most efficient and simplest method for compensating for physical effects for brain data. Applying Gaussian blur to the histo-image yielded images with limited improvement. All above results hold for both the half-dose and the full-dose cases.

CONCLUSION

For brain imaging, the evaluation based on metrics PSNR, RMSE, SSIM, and CRC indicates that the view-summed CW-based back-projection variant is the most effective input for the BPF-like reconstruction model using CNN filtration, which can involve the attenuation map through an additional channel to effectively compensate for physical effects.

ReconU-Net: a direct PET image reconstruction using U-Net architecture with back projection-induced skip connection

Objective.This study aims to introduce a novel back projection-induced U-Net-shaped architecture, called ReconU-Net, based on the original U-Net architecture for deep learning-based direct positron emission tomography (PET) image reconstruction. Additionally, our objective is to visualize the behavior of direct PET image reconstruction by comparing the proposed ReconU-Net architecture with the original U-Net architecture and existing DeepPET encoder-decoder architecture without skip connections.Approach. The proposed ReconU-Net architecture uniquely integrates the physical model of the back projection operation into the skip connection. This distinctive feature facilitates the effective transfer of intrinsic spatial information from the input sinogram to the reconstructed image via an embedded physical model. The proposed ReconU-Net was trained using Monte Carlo simulation data from the Brainweb phantom and tested on both simulated and real Hoffman brain phantom data.Main results. The proposed ReconU-Net method provided better reconstructed image in terms of the peak signal-to-noise ratio and contrast recovery coefficient than the original U-Net and DeepPET methods. Further analysis shows that the proposed ReconU-Net architecture has the ability to transfer features of multiple resolutions, especially non-abstract high-resolution information, through skip connections. Unlike the U-Net and DeepPET methods, the proposed ReconU-Net successfully reconstructed the real Hoffman brain phantom, despite limited training on simulated data.Significance. The proposed ReconU-Net can improve the fidelity of direct PET image reconstruction, even with small training datasets, by leveraging the synergistic relationship between data-driven modeling and the physics model of the imaging process.

Deep learning-based PET image denoising and reconstruction: a review

This review focuses on positron emission tomography (PET) imaging algorithms and traces the evolution of PET image reconstruction methods. First, we provide an overview of conventional PET image reconstruction methods from filtered backprojection through to recent iterative PET image reconstruction algorithms, and then review deep learning methods for PET data up to the latest innovations within three main categories. The first category involves post-processing methods for PET image denoising. The second category comprises direct image reconstruction methods that learn mappings from sinograms to the reconstructed images in an end-to-end manner. The third category comprises iterative reconstruction methods that combine conventional iterative image reconstruction with neural-network enhancement. We discuss future perspectives on PET imaging and deep learning technology.

AI for PET image reconstruction

Image reconstruction for positron emission tomography (PET) has been developed over many decades, with advances coming from improved modelling of the data statistics and improved modelling of the imaging physics. However, high noise and limited spatial resolution have remained issues in PET imaging, and state-of-the-art PET reconstruction has started to exploit other medical imaging modalities (such as MRI) to assist in noise reduction and enhancement of PET's spatial resolution. Nonetheless, there is an ongoing drive towards not only improving image quality, but also reducing the injected radiation dose and reducing scanning times. While the arrival of new PET scanners (such as total body PET) is helping, there is always a need to improve reconstructed image quality due to the time and count limited imaging conditions. Artificial intelligence (AI) methods are now at the frontier of research for PET image reconstruction. While AI can learn the imaging physics as well as the noise in the data (when given sufficient examples), one of the most common uses of AI arises from exploiting databases of high-quality reference examples, to provide advanced noise compensation and resolution recovery. There are three main AI reconstruction approaches: (i) direct data-driven AI methods which rely on supervised learning from reference data, (ii) iterative (unrolled) methods which combine our physics and statistical models with AI learning from data, and (iii) methods which exploit AI with our known models, but crucially can offer benefits even in the absence of any example training data whatsoever. This article reviews these methods, considering opportunities and challenges of AI for PET reconstruction.

Fully 3D implementation of the end-to-end deep image prior-based PET image reconstruction using block iterative algorithm

Objective. Deep image prior (DIP) has recently attracted attention owing to its unsupervised positron emission tomography (PET) image reconstruction method, which does not require any prior training dataset. In this paper, we present the first attempt to implement an end-to-end DIP-based fully 3D PET image reconstruction method that incorporates a forward-projection model into a loss function.Approach. A practical implementation of a fully 3D PET image reconstruction could not be performed at present because of a graphics processing unit memory limitation. Consequently, we modify the DIP optimization to a block iteration and sequential learning of an ordered sequence of block sinograms. Furthermore, the relative difference penalty (RDP) term is added to the loss function to enhance the quantitative accuracy of the PET image.Main results. We evaluated our proposed method using Monte Carlo simulation with [18F]FDG PET data of a human brain and a preclinical study on monkey-brain [18F]FDG PET data. The proposed method was compared with the maximum-likelihood expectation maximization (EM), maximuma posterioriEM with RDP, and hybrid DIP-based PET reconstruction methods. The simulation results showed that, compared with other algorithms, the proposed method improved the PET image quality by reducing statistical noise and better preserved the contrast of brain structures and inserted tumors. In the preclinical experiment, finer structures and better contrast recovery were obtained with the proposed method.Significance.The results indicated that the proposed method could produce high-quality images without a prior training dataset. Thus, the proposed method could be a key enabling technology for the straightforward and practical implementation of end-to-end DIP-based fully 3D PET image reconstruction.

Deep learning-based image reconstruction and post-processing methods in positron emission tomography for low-dose imaging and resolution enhancement

Image processing plays a crucial role in maximising diagnostic quality of positron emission tomography (PET) images. Recently, deep learning methods developed across many fields have shown tremendous potential when applied to medical image enhancement, resulting in a rich and rapidly advancing literature surrounding this subject. This review encapsulates methods for integrating deep learning into PET image reconstruction and post-processing for low-dose imaging and resolution enhancement. A brief introduction to conventional image processing techniques in PET is firstly presented. We then review methods which integrate deep learning into the image reconstruction framework as either deep learning-based regularisation or as a fully data-driven mapping from measured signal to images. Deep learning-based post-processing methods for low-dose imaging, temporal resolution enhancement and spatial resolution enhancement are also reviewed. Finally, the challenges associated with applying deep learning to enhance PET images in the clinical setting are discussed and future research directions to address these challenges are presented.

A back‐projection‐and‐filtering‐like (BPF‐like) reconstruction method with the deep learning filtration from listmode data in TOF‐PET

PURPOSE

The time-of-flight (TOF) information improves signal-to-noise ratio (SNR) for positron emission tomography (PET) imaging. Existing analytical algorithms for TOF PET usually follow a filtered back-projection process on reconstructing images from the sinogram data. This work aims to develop a back-projection-and-filtering-like (BPF-like) algorithm that reconstructs the TOF PET image directly from listmode data rapidly.

METHODS

We extended the 2D conventional non-TOF PET projection model to a TOF case, where projection data are represented as line integrals weighted by the one-dimensional TOF kernel along the projection direction. After deriving the central slice theorem and the TOF back-projection of listmode data, we designed a deep learning network with a modified U-net architecture to perform the spatial filtration (reconstruction filter). The proposed BP-Net method was validated via Monte Carlo simulations of TOF PET listmode data with three different time resolutions for two types of activity phantoms. The network was only trained on the simulated full-dose XCAT dataset and then evaluated on XCAT and Jaszczak data with different time resolutions and dose levels.

RESULTS

Reconstructed images show that when compared with the conventional BPF algorithm and the MLEM algorithm proposed for TOF PET, the proposed BP-Net method obtains better image quality in terms of peak signal-to-noise ratio, relative mean square error, and structure similarity index; besides, the reconstruction speed of the BP-Net is 1.75 times faster than BPF and 29.05 times faster than MLEM using 15 iterations. The results also indicate that the performance of the BP-Net degrades with worse time resolutions and lower tracer doses, but degrades less than BPF or MLEM reconstructions.

CONCLUSION

In this work, we developed an analytical-like reconstruction in the form of BPF with the reconstruction filtering operation performed via a deep network. The method runs even faster than the conventional BPF algorithm and provides accurate reconstructions from listmode data in TOF-PET, free of rebinning data to a sinogram.