Respiratory deformation registration in 4D-CT/cone beam CT using deep learning

Motion compensated cone-beam CT reconstruction using ana priorimotion model from CT simulation: a pilot study

Objective. To combat the motion artifacts present in traditional 4D-CBCT reconstruction, an iterative technique known as the motion-compensated simultaneous algebraic reconstruction technique (MC-SART) was previously developed. MC-SART employs a 4D-CBCT reconstruction to obtain an initial model, which suffers from a lack of sufficient projections in each bin. The purpose of this study is to demonstrate the feasibility of introducing a motion model acquired during CT simulation to MC-SART, coined model-based CBCT (MB-CBCT).Approach. For each of 5 patients, we acquired 5DCTs during simulation and pre-treatment CBCTs with a simultaneous breathing surrogate. We cross-calibrated the 5DCT and CBCT breathing waveforms by matching the diaphragms and employed the 5DCT motion model parameters for MC-SART. We introduced the Amplitude Reassignment Motion Modeling technique, which measures the ability of the model to control diaphragm sharpness by reassigning projection amplitudes with varying resolution. We evaluated the sharpness of tumors and compared them between MB-CBCT and 4D-CBCT. We quantified sharpness by fitting an error function across anatomical boundaries. Furthermore, we compared our MB-CBCT approach to the traditional MC-SART approach. We evaluated MB-CBCT's robustness over time by reconstructing multiple fractions for each patient and measuring consistency in tumor centroid locations between 4D-CBCT and MB-CBCT.Main results. We found that the diaphragm sharpness rose consistently with increasing amplitude resolution for 4/5 patients. We observed consistently high image quality across multiple fractions, and observed stable tumor centroids with an average 0.74 ± 0.31 mm difference between the 4D-CBCT and MB-CBCT. Overall, vast improvements over 3D-CBCT and 4D-CBCT were demonstrated by our MB-CBCT technique in terms of both diaphragm sharpness and overall image quality.Significance. This work is an important extension of the MC-SART technique. We demonstrated the ability ofa priori5DCT models to provide motion compensation for CBCT reconstruction. We showed improvements in image quality over both 4D-CBCT and the traditional MC-SART approach.

Deep learning-based lung image registration: A review

Lung image registration can effectively describe the relative motion of lung tissues, thereby helping to solve series problems in clinical applications. Since the lungs are soft and fairly passive organs, they are influenced by respiration and heartbeat, resulting in discontinuity of lung motion and large deformation of anatomic features. This poses great challenges for accurate registration of lung image and its applications. The recent application of deep learning (DL) methods in the field of medical image registration has brought promising results. However, a versatile registration framework has not yet emerged due to diverse challenges of registration for different regions of interest (ROI). DL-based image registration methods used for other ROI cannot achieve satisfactory results in lungs. In addition, there are few review articles available on DL-based lung image registration. In this review, the development of conventional methods for lung image registration is briefly described and a more comprehensive survey of DL-based methods for lung image registration is illustrated. The DL-based methods are classified according to different supervision types, including fully-supervised, weakly-supervised and unsupervised. The contributions of researchers in addressing various challenges are described, as well as the limitations of these approaches. This review also presents a comprehensive statistical analysis of the cited papers in terms of evaluation metrics and loss functions. In addition, publicly available datasets for lung image registration are also summarized. Finally, the remaining challenges and potential trends in DL-based lung image registration are discussed.

An overview of artificial intelligence in medical physics and radiation oncology

Artificial intelligence (AI) is developing rapidly and has found widespread applications in medicine, especially radiotherapy. This paper provides a brief overview of AI applications in radiotherapy, and highlights the research directions of AI that can potentially make significant impacts and relevant ongoing research works in these directions. Challenging issues related to the clinical applications of AI, such as robustness and interpretability of AI models, are also discussed. The future research directions of AI in the field of medical physics and radiotherapy are highlighted.

Evaluation of a respiratory motion-corrected image reconstruction algorithm in 2-[18F]FDG and [68Ga]Ga-DOTA-NOC PET/CT: impacts on image quality and tumor quantification

Background

Respiratory motions may cause artifacts on positron emission tomography (PET) images that degrade image quality and quantification accuracy. This study aimed to evaluate the effect of a respiratory motion-corrected image reconstruction (MCIR) algorithm on image quality and tumor quantification compared with nongated/nonmotion-corrected reconstruction.

Methods

We used a phantom consisting of 5 motion spheres immersed in a chamber driven by a motor. The spheres and the background chamber were filled with 18F solution at a sphere-to-background ratio of 5:1. We enrolled 42 and 16 patients undergoing 2-deoxy-2-[¹⁸F]fluoro-D-glucose {2-[¹⁸F]FDG} and ⁶⁸Ga-labeled [1,4,7,10-tetraazacyclododecane-1,4,7,10-tetraacetic acid]-1-Nal3-octreotide {[⁶⁸Ga]Ga-DOTA-NOC} PET/computed tomography (CT) from whom 74 and 30 lesions were segmented, respectively. Three reconstructions were performed: data-driven gating-based motion correction (DDGMC), external vital signal module-based motion correction (VSMMC), and noncorrection reconstruction. The standardized uptake values (SUVs) and the volume of the spheres and the lesions were measured and compared among the 3 reconstruction groups. The image noise in the liver was measured, and the visual image quality of motion artifacts was scored by radiologists in the patient study.

Results

In the phantom study, the spheres' SUVs increased by 26-36%, and the volumes decreased by 35-38% in DDGMC and VSMMC compared with the noncorrection group. In the 2-[¹⁸F]FDG PET patient study, the lesions' SUVs had a median increase of 10.87-12.65% while the volumes had a median decrease of 14.88-15.18% in DDGMC and VSMMC compared with those of noncorrection. In the [⁶⁸Ga]Ga-DOTA-NOC PET patient study, the lesions' SUVs increased by 14.23-15.45%, and the volumes decreased by 19.11-20.94% in DDGMC and VSMMC. The image noise in the liver was equal between the DDGMC, VSMMC, and noncorrection groups. Radiologists found improved image quality in more than 45% of the cases in DDGMC and VSMMC compared with the noncorrection group. There was no statistically significant difference in SUVs, volumes, or visual image quality scores between DDGMC and VSMMC.

Conclusions

MCIR improves tumor quantification accuracy and visual image quality by reducing respiratory motion artifacts without compromised image noise performance or elongated acquisition time in 2-[¹⁸F]FDG and [⁶⁸Ga]Ga-DOTA-NOC PET/CT tumor imaging. The performance of DDG-driven MCIR is as good as that of the external device-driven solution.

DragNet: Learning-based deformable registration for realistic cardiac MR sequence generation from a single frame

Deformable image registration (DIR) can be used to track cardiac motion. Conventional DIR algorithms aim to establish a dense and non-linear correspondence between independent pairs of images. They are, nevertheless, computationally intensive and do not consider temporal dependencies to regulate the estimated motion in a cardiac cycle. In this paper, leveraging deep learning methods, we formulate a novel hierarchical probabilistic model, termed DragNet, for fast and reliable spatio-temporal registration in cine cardiac magnetic resonance (CMR) images and for generating synthetic heart motion sequences. DragNet is a variational inference framework, which takes an image from the sequence in combination with the hidden states of a recurrent neural network (RNN) as inputs to an inference network per time step. As part of this framework, we condition the prior probability of the latent variables on the hidden states of the RNN utilised to capture temporal dependencies. We further condition the posterior of the motion field on a latent variable from hierarchy and features from the moving image. Subsequently, the RNN updates the hidden state variables based on the feature maps of the fixed image and the latent variables. Different from traditional methods, DragNet performs registration on unseen sequences in a forward pass, which significantly expedites the registration process. Besides, DragNet enables generating a large number of realistic synthetic image sequences given only one frame, where the corresponding deformations are also retrieved. The probabilistic framework allows for computing spatio-temporal uncertainties in the estimated motion fields. Our results show that DragNet performance is comparable with state-of-the-art methods in terms of registration accuracy, with the advantage of offering analytical pixel-wise motion uncertainty estimation across a cardiac cycle and being a motion generator. We will make our code publicly available.

A review of deep learning-based deformable medical image registration

The alignment of images through deformable image registration is vital to clinical applications (e.g., atlas creation, image fusion, and tumor targeting in image-guided navigation systems) and is still a challenging problem. Recent progress in the field of deep learning has significantly advanced the performance of medical image registration. In this review, we present a comprehensive survey on deep learning-based deformable medical image registration methods. These methods are classified into five categories: Deep Iterative Methods, Supervised Methods, Unsupervised Methods, Weakly Supervised Methods, and Latest Methods. A detailed review of each category is provided with discussions about contributions, tasks, and inadequacies. We also provide statistical analysis for the selected papers from the point of view of image modality, the region of interest (ROI), evaluation metrics, and method categories. In addition, we summarize 33 publicly available datasets that are used for benchmarking the registration algorithms. Finally, the remaining challenges, future directions, and potential trends are discussed in our review.

Unsupervised computed tomography and cone-beam computed tomography image registration using a dual attention network

Background

The registration of computed tomography (CT) and cone-beam computed tomography (CBCT) plays a key role in image-guided radiotherapy (IGRT). However, the large intensity variation between CT and CBCT images limits the registration performance and its clinical application in IGRT. In this study, a learning-based unsupervised approach was developed to address this issue and accurately register CT and CBCT images by predicting the deformation field.

Methods

A dual attention module was used to handle the large intensity variation between CT and CBCT images. Specifically, a scale-aware position attention block (SP-BLOCK) and a scale-aware channel attention block (SC-BLOCK) were employed to integrate contextual information from the image space and channel dimensions. The SP-BLOCK enhances the correlation of similar features by weighting and aggregating multi-scale features at different positions, while the SC-BLOCK handles the multiple features of all channels to selectively emphasize dependencies between channel maps.

Results

The proposed method was compared with existing mainstream methods on the 4D-LUNG data set. Compared to other mainstream methods, it achieved the highest structural similarity (SSIM) and dice similarity coefficient (DICE) scores of 86.34% and 89.74%, respectively, and the lowest target registration error (TRE) of 2.07 mm.

Conclusions

The proposed method can register CT and CBCT images with high accuracy without the needs of manual labeling. It provides an effective way for high-accuracy patient positioning and target localization in IGRT.

A review of deep learning-based three-dimensional medical image registration methods

Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.