Respiratory motion models: A review

Beat Pilot Tone (BPT): Simultaneous MRI and RF motion sensing at arbitrary frequencies

PURPOSE

To introduce a simple system exploitation with the potential to turn MRI scanners into general-purpose radiofrequency (RF) motion monitoring systems.

METHODS

Inspired by Pilot Tone (PT), this work proposes Beat Pilot Tone (BPT), in which two or more RF tones at arbitrary frequencies are transmitted continuously during the scan. These tones create motion-modulated standing wave patterns that are sensed by the receiver coil array, incidentally mixed by intermodulation in the receiver chain, and digitized simultaneously with the MRI data. BPT can operate at almost any frequency as long as the intermodulation products lie within the bandwidth of the receivers. BPT's mechanism is explained in electromagnetic simulations and validated experimentally.

RESULTS

Phantom and volunteer experiments over a range of transmit frequencies suggest that BPT may offer frequency-dependent sensitivity to motion. Using a semi-flexible anterior receiver array, BPT appears to sense cardiac-induced body vibrations at microwave frequencies (≥ $$ \ge $$ 1.2 GHz). At lower frequencies, it exhibits a similar cardiac signal shape to PT, likely due to blood volume changes. Other volunteer experiments with respiratory, bulk, and head motion show that BPT can achieve greater sensitivity to motion than PT and greater separability between motion types. Basic multiple-input multiple-output (4 × 22 $$ 4\times 22 $$ MIMO) operation with simultaneous PT and BPT in head motion is demonstrated using two transmit antennas and a 22-channel head-neck coil.

CONCLUSION

BPT may offer a rich source of motion information that is frequency-dependent, simultaneous, and complementary to PT and the MRI exam.

Stability analysis of patient-specific 4DCT- and 4DCBCT-based correspondence models

BACKGROUND

Surrogate-based motion compensation in stereotactic body radiation therapy (SBRT) strongly relies on a constant relationship between an external breathing signal and the internal tumor motion over the course of treatment, that is, a stable patient-specific correspondence model.

PURPOSE

This study aims to develop methods for analyzing the stability of correspondence models by integrating planning 4DCT and pretreatment 4D cone-beam computed tomography (4DCBCT) data and assessing the relation to patient-specific clinical parameters.

METHODS

For correspondence modeling, a regression-based approach is applied, correlating patient-specific internal motion (vector fields computed by deformable image registration) and external breathing signals (recorded by Varian's RPM and RGSC system). To analyze correspondence model stability, two complementary methods are proposed. (1) Target volume-based analysis: 4DCBCT-based correspondence models predict clinical target volumes (GTV and internal target volume [ITV]) within the planning 4DCT, which are evaluated by overlap and distance measures (Dice similarity coefficient [DSC]/average symmetric surface distance [ASSD]). (2) System matrix-based analysis: 4DCBCT-based regression models are compared to 4DCT-based models using mean squared difference (MSD) and principal component analysis of the system matrices. Stability analysis results are correlated with clinical parameters. Both methods are applied to a dataset of 214 pretreatment 4DCBCT scans (Varian TrueBeam) from a cohort of 46 lung tumor patients treated with ITV-based SBRT (planning 4DCTs acquired with Siemens AS Open and SOMATOM go.OPEN Pro CT scanners).

RESULTS

Consistent results across the two complementary analysis approaches (Spearman correlation coefficient of0.6 / 0.7 $0.6/ 0.7$ between system matrix-based MSD and GTV-based DSC/ASSD) were observed. Analysis showed that stability was not predominant, with 114/214 fraction-wise models not surpassing a threshold ofD S C > 0.7 $DSC > 0.7$ for the GTV, and only 14/46 patients demonstrating aD S C > 0.7 $DSC > 0.7$ in all fractions. Model stability did not degrade over the course of treatment. The mean GTV-based DSC is0.59 ± 0.26 $0.59\pm 0.26$ (mean ASSD of2.83 ± 3.37 $2.83\pm 3.37$ ) and the respective ITV-based DSC is0.69 ± 0.20 $0.69\pm 0.20$ (mean ASSD of2.35 ± 1.81 $2.35\pm 1.81$ ). The clinical parameters showed a strong correlation between smaller tumor motion ranges and increased stability.

CONCLUSIONS

The proposed methods identify patients with unstable correspondence models prior to each treatment fraction, serving as direct indicators for the necessity of replanning and adaptive treatment approaches to account for internal-external motion variations throughout the course of treatment.

Cardio-respiratory motion compensation for coronary roadmapping in fluoroscopic imaging

BACKGROUND

Inferring the shape and position of coronary artery poses challenges when using fluoroscopic image guidance during percutaneous coronary intervention (PCI) procedure. Although angiography enables coronary artery visualization, the use of injected contrast agent raises concerns about radiation exposure and the risk of contrast-induced nephropathy. To address these issues, dynamic coronary roadmapping overlaid on fluoroscopic images can provide coronary visual feedback without contrast injection.

PURPOSE

This paper proposes a novel cardio-respiratory motion compensation method that utilizes cardiac state synchronization and catheter motion estimation to achieve coronary roadmapping in fluoroscopic images.

METHODS

For more accurate cardiac state synchronization, video frame interpolation is applied to increase the frame rate of the original limited angiographic images, resulting in higher framerate and more adequate roadmaps. The proposed method also incorporates a multi-length cross-correlation based adaptive electrocardiogram (ECG) matching to address irregular cardiac motion situation. Furthermore, a shape-constrained path searching method is proposed to extract catheter structure from both fluoroscopic and angiographic image. Then catheter motion is estimated using a cascaded matching approach with an outlier removal strategy, leading to a final corrected roadmap.

RESULTS

Evaluation of the proposed method on clinical x-ray images demonstrates its effectiveness, achieving a 92.8% F1 score for catheter extraction on 589 fluoroscopic and angiographic images. Additionally, the method achieves a 5.6-pixel distance error of the coronary roadmap on 164 intraoperative fluoroscopic images.

CONCLUSIONS

Overall, the proposed method achieves accurate coronary roadmapping in fluoroscopic images and shows potential to overlay accurate coronary roadmap on fluoroscopic image in assisting PCI.

Liver respiratory-induced motion estimation using abdominal surface displacement as a surrogate: robotic phantom and clinical validation with varied correspondence models

PURPOSE

This work presents the implementation of an RGB-D camera as a surrogate signal for liver respiratory-induced motion estimation. This study aims to validate the feasibility of RGB-D cameras as a surrogate in a human subject experiment and to compare the performance of different correspondence models.

METHODS

The proposed approach uses an RGB-D camera to compute an abdominal surface reconstruction and estimate the liver respiratory-induced motion. Two sets of validation experiments were conducted, first, using a robotic liver phantom and, secondly, performing a clinical study with human subjects. In the clinical study, three correspondence models were created changing the conditions of the learning-based model.

RESULTS

The motion model for the robotic liver phantom displayed an error below 3 mm with a coefficient of determination above 90% for the different directions of motion. The clinical study presented errors of 4.5, 2.5, and 2.9 mm for the three different motion models with a coefficient of determination above 80% for all three cases.

CONCLUSION

RGB-D cameras are a promising method to accurately estimate the liver respiratory-induced motion. The internal motion can be estimated in a non-contact, noninvasive and flexible approach. Additionally, three training conditions for the correspondence model are studied to potentially mitigate intra- and inter-fraction motion.

R-MFE-TCN: A correlation prediction model between body surface and tumor during respiratory movement

BACKGROUND

2D CT image-guided radiofrequency ablation (RFA) is an exciting minimally invasive treatment that can destroy liver tumors without removing them. However, CT images can only provide limited static information, and the tumor will move with the patient's respiratory movement. Therefore, how to accurately locate tumors under free conditions is an urgent problem to be solved at present.

PURPOSE

The purpose of this study is to propose a respiratory correlation prediction model for mixed reality surgical assistance system, Riemannian and Multivariate Feature Enhanced Temporal Convolutional Network (R-MFE-TCN), and to achieve accurate respiratory correlation prediction.

METHODS

The model adopts a respiration-oriented Riemannian information enhancement strategy to expand the diversity of the dataset. A new Multivariate Feature Enhancement module (MFE) is proposed to retain respiratory data information, so that the network can fully explore the correlation of internal and external data information, the dual-channel is used to retain multivariate respiratory feature, and the Multi-headed Self-attention obtains respiratory peak-to-valley value periodic information. This information significantly improves the prediction performance of the network. At the same time, the PSO algorithm is used for hyperparameter optimization. In the experiment, a total of seven patients' internal and external respiratory motion trajectories were obtained from the dataset, and the first six patients were selected as the training set. The respiratory signal collection frequency was 21 Hz.

RESULTS

A large number of experiments on the dataset prove the good performance of this method, which improves the prediction accuracy while also having strong robustness. This method can reduce the delay deviation under long window prediction and achieve good performance. In the case of 400 ms, the average RMSE and MAE are 0.0453  and 0.0361 mm, respectively, which is better than other research methods.

CONCLUSION

The R-MFE-TCN can be extended to respiratory correlation prediction in different clinical situations, meeting the accuracy requirements for respiratory delay prediction in surgical assistance.

Embedding-Alignment Fusion-Based Graph Convolution Network With Mixed Learning Strategy for 4D Medical Image Reconstruction

In recent years, 4D medical image involving structural and motion information of tissue has attracted increasing attention. The key to the 4D image reconstruction is to stack the 2D slices based on matching the aligned motion states. In this study, the distribution of the 2D slices with the different motion states is modeled as a manifold graph, and the reconstruction is turned to be the graph alignment. An embedding-alignment fusion-based graph convolution network (GCN) with a mixed-learning strategy is proposed to align the graphs. Herein, the embedding and alignment processes of graphs interact with each other to realize a precise alignment with retaining the manifold distribution. The mixed strategy of self- and semi-supervised learning makes the alignment sparse to avoid the mismatching caused by outliers in the graph. In the experiment, the proposed 4D reconstruction approach is validated on the different modalities including Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Ultrasound (US). We evaluate the reconstruction accuracy and compare it with those of state-of-the-art methods. The experiment results demonstrate that our approach can reconstruct a more accurate 4D image.

CT respiratory motion synthesis using joint supervised and adversarial learning

Objective.Four-dimensional computed tomography (4DCT) imaging consists in reconstructing a CT acquisition into multiple phases to track internal organ and tumor motion. It is commonly used in radiotherapy treatment planning to establish planning target volumes. However, 4DCT increases protocol complexity, may not align with patient breathing during treatment, and lead to higher radiation delivery.Approach.In this study, we propose a deep synthesis method to generate pseudo respiratory CT phases from static images for motion-aware treatment planning. The model produces patient-specific deformation vector fields (DVFs) by conditioning synthesis on external patient surface-based estimation, mimicking respiratory monitoring devices. A key methodological contribution is to encourage DVF realism through supervised DVF training while using an adversarial term jointly not only on the warped image but also on the magnitude of the DVF itself. This way, we avoid excessive smoothness typically obtained through deep unsupervised learning, and encourage correlations with the respiratory amplitude.Main results.Performance is evaluated using real 4DCT acquisitions with smaller tumor volumes than previously reported. Results demonstrate for the first time that the generated pseudo-respiratory CT phases can capture organ and tumor motion with similar accuracy to repeated 4DCT scans of the same patient. Mean inter-scans tumor center-of-mass distances and Dice similarity coefficients were 1.97 mm and 0.63, respectively, for real 4DCT phases and 2.35 mm and 0.71 for synthetic phases, and compares favorably to a state-of-the-art technique (RMSim).Significance.This study presents a deep image synthesis method that addresses the limitations of conventional 4DCT by generating pseudo-respiratory CT phases from static images. Although further studies are needed to assess the dosimetric impact of the proposed method, this approach has the potential to reduce radiation exposure in radiotherapy treatment planning while maintaining accurate motion representation. Our training and testing code can be found athttps://github.com/cyiheng/Dynagan.

Motion compensated cone-beam CT reconstruction using ana priorimotion model from CT simulation: a pilot study

Objective. To combat the motion artifacts present in traditional 4D-CBCT reconstruction, an iterative technique known as the motion-compensated simultaneous algebraic reconstruction technique (MC-SART) was previously developed. MC-SART employs a 4D-CBCT reconstruction to obtain an initial model, which suffers from a lack of sufficient projections in each bin. The purpose of this study is to demonstrate the feasibility of introducing a motion model acquired during CT simulation to MC-SART, coined model-based CBCT (MB-CBCT).Approach. For each of 5 patients, we acquired 5DCTs during simulation and pre-treatment CBCTs with a simultaneous breathing surrogate. We cross-calibrated the 5DCT and CBCT breathing waveforms by matching the diaphragms and employed the 5DCT motion model parameters for MC-SART. We introduced the Amplitude Reassignment Motion Modeling technique, which measures the ability of the model to control diaphragm sharpness by reassigning projection amplitudes with varying resolution. We evaluated the sharpness of tumors and compared them between MB-CBCT and 4D-CBCT. We quantified sharpness by fitting an error function across anatomical boundaries. Furthermore, we compared our MB-CBCT approach to the traditional MC-SART approach. We evaluated MB-CBCT's robustness over time by reconstructing multiple fractions for each patient and measuring consistency in tumor centroid locations between 4D-CBCT and MB-CBCT.Main results. We found that the diaphragm sharpness rose consistently with increasing amplitude resolution for 4/5 patients. We observed consistently high image quality across multiple fractions, and observed stable tumor centroids with an average 0.74 ± 0.31 mm difference between the 4D-CBCT and MB-CBCT. Overall, vast improvements over 3D-CBCT and 4D-CBCT were demonstrated by our MB-CBCT technique in terms of both diaphragm sharpness and overall image quality.Significance. This work is an important extension of the MC-SART technique. We demonstrated the ability ofa priori5DCT models to provide motion compensation for CBCT reconstruction. We showed improvements in image quality over both 4D-CBCT and the traditional MC-SART approach.

Respiratory motion modelling for MR-guided lung cancer radiotherapy: model development and geometric accuracy evaluation

Objective.Respiratory motion of lung tumours and adjacent structures is challenging for radiotherapy. Online MR-imaging cannot currently provide real-time volumetric information of the moving patient anatomy, therefore limiting precise dose delivery, delivered dose reconstruction, and downstream adaptation methods.Approach.We tailor a respiratory motion modelling framework towards an MR-Linac workflow to estimate the time-resolved 4D motion from real-time data. We develop a multi-slice acquisition scheme which acquires thick, overlapping 2D motion-slices in different locations and orientations, interleaved with 2D surrogate-slices from a fixed location. The framework fits a motion model directly to the input data without the need for sorting or binning to account for inter- and intra-cycle variation of the breathing motion. The framework alternates between model fitting and motion-compensated super-resolution image reconstruction to recover a high-quality motion-free image and a motion model. The fitted model can then estimate the 4D motion from 2D surrogate-slices. The framework is applied to four simulated anthropomorphic datasets and evaluated against known ground truth anatomy and motion. Clinical applicability is demonstrated by applying our framework to eight datasets acquired on an MR-Linac from four lung cancer patients.Main results.The framework accurately reconstructs high-quality motion-compensated 3D images with 2 mm3isotropic voxels. For the simulated case with the largest target motion, the motion model achieved a mean deformation field error of 1.13 mm. For the patient cases residual error registrations estimate the model error to be 1.07 mm (1.64 mm), 0.91 mm (1.32 mm), and 0.88 mm (1.33 mm) in superior-inferior, anterior-posterior, and left-right directions respectively for the building (application) data.Significance.The motion modelling framework estimates the patient motion with high accuracy and accurately reconstructs the anatomy. The image acquisition scheme can be flexibly integrated into an MR-Linac workflow whilst maintaining the capability of online motion-management strategies based on cine imaging such as target tracking and/or gating.

Fiberbots: Robotic fibers for high-precision minimally invasive surgery

Precise manipulation of flexible surgical tools is crucial in minimally invasive surgical procedures, necessitating a miniature and flexible robotic probe that can precisely direct the surgical instruments. In this work, we developed a polymer-based robotic fiber with a thermal actuation mechanism by local heating along the sides of a single fiber. The fiber robot was fabricated by highly scalable fiber drawing technology using common low-cost materials. This low-profile (below 2 millimeters in diameter) robotic fiber exhibits remarkable motion precision (below 50 micrometers) and repeatability. We developed control algorithms coupling the robot with endoscopic instruments, demonstrating high-resolution in situ molecular and morphological tissue mapping. We assess its practicality and safety during in vivo laparoscopic surgery on a porcine model. High-precision motion of the fiber robot delivered endoscopically facilitates the effective use of cellular-level intraoperative tissue identification and ablation technologies, potentially enabling precise removal of cancer in challenging surgical sites.

Surrogate-driven respiratory motion model for projection-resolved motion estimation and motion compensated cone-beam CT reconstruction from unsorted projection data

Objective.As the most common solution to motion artefact for cone-beam CT (CBCT) in radiotherapy, 4DCBCT suffers from long acquisition time and phase sorting error. This issue could be addressed if the motion at each projection could be known, which is a severely ill-posed problem. This study aims to obtain the motion at each time point and motion-free image simultaneously from unsorted projection data of a standard 3DCBCT scan.Approach.Respiration surrogate signals were extracted by the Intensity Analysis method. A general framework was then deployed to fit a surrogate-driven motion model that characterized the relation between the motion and surrogate signals at each time point. Motion model fitting and motion compensated reconstruction were alternatively and iteratively performed. Stochastic subset gradient based method was used to significantly reduce the computation time. The performance of our method was comprehensively evaluated through digital phantom simulation and also validated on clinical scans from six patients.Results.For digital phantom experiments, motion models fitted with ground-truth or extracted surrogate signals both achieved a much lower motion estimation error and higher image quality, compared with non motion-compensated results.For the public SPARE Challenge datasets, more clear lung tissues and less blurry diaphragm could be seen in the motion compensated reconstruction, comparable to the benchmark 4DCBCT images but with a higher temporal resolution. Similar results were observed for two real clinical 3DCBCT scans.Significance.The motion compensated reconstructions and motion models produced by our method will have direct clinical benefit by providing more accurate estimates of the delivered dose and ultimately facilitating more accurate radiotherapy treatments for lung cancer patients.

A Respiratory Motion Prediction Method Based on LSTM-AE with Attention Mechanism for Spine Surgery

Respiratory motion-induced vertebral movements can adversely impact intraoperative spine surgery, resulting in inaccurate positional information of the target region and unexpected damage during the operation. In this paper, we propose a novel deep learning architecture for respiratory motion prediction, which can adapt to different patients. The proposed method utilizes an LSTM-AE with attention mechanism network that can be trained using few-shot datasets during operation. To ensure real-time performance, a dimension reduction method based on the respiration-induced physical movement of spine vertebral bodies is introduced. The experiment collected data from prone-positioned patients under general anaesthesia to validate the prediction accuracy and time efficiency of the LSTM-AE-based motion prediction method. The experimental results demonstrate that the presented method (RMSE: 4.39%) outperforms other methods in terms of accuracy within a learning time of 2 min. The maximum predictive errors under the latency of 333 ms with respect to the x, y, and z axes of the optical camera system were 0.13, 0.07, and 0.10 mm, respectively, within a motion range of 2 mm.

A Patient-Specific correspondence model to track tumor location in thorax during radiation therapy

PURPOSE

We present a patient-specific model to estimate tumor location in the thorax during radiation therapy using chest surface displacement as the surrogate signal.

METHODS

Two types of data are used for model construction: Four-dimensional computed tomography (4D-CT) images of the patient and the displacement of two points on the patient's skin on the thoracic area. Principal component analysis is used to fit the correspondence model. This model incorporates the recorded surrogate signals during radiation delivery as input and delivers the 3D trajectory of the tumor as output. We evaluated the accuracy of the proposed model on a respiratory phantom and five lung cancer patients.

RESULTS

For the respiratory phantom, the location of the center of the sphere during treatment was calculated in three directions: Left-Right (LR), Anterior-Posterior (AP) and, Superior-Inferior (SI). The error of localization was less than 1 mm in the LR and AP directions and less than 2 mm in the SI direction. The location of the tumor center for two of the patients, and the location of the apex of the diaphragm for the other three, was calculated in three directions. For all patients, the localization error in the LR and AP directions was less than 1.1 mm for two fractions and the maximum localization error in the SI direction was 6.4 mm.

CONCLUSIONS

This work presents a feasibility study of utilizing surface displacement data to locate the tumor in the thorax during radiation treatment. Future work will validate the model on a larger patient population.

Continuous time-resolved estimated synthetic 4D-CTs for dose reconstruction of lung tumor treatments at a 0.35 T MR-linac

Objective.To experimentally validate a method to create continuous time-resolved estimated synthetic 4D-computed tomography datasets (tresCTs) based on orthogonal cine MRI data for lung cancer treatments at a magnetic resonance imaging (MRI) guided linear accelerator (MR-linac).Approach.A breathing porcine lung phantom was scanned at a CT scanner and 0.35 T MR-linac. Orthogonal cine MRI series (sagittal/coronal orientation) at 7.3 Hz, intersecting tumor-mimicking gelatin nodules, were deformably registered to mid-exhale 3D-CT and 3D-MRI datasets. The time-resolved deformation vector fields were extrapolated to 3D and applied to a reference synthetic 3D-CT image (sCTref), while accounting for breathing phase-dependent lung density variations, to create 82 s long tresCTs at 3.65 Hz. Ten tresCTs were created for ten tracked nodules with different motion patterns in two lungs. For each dataset, a treatment plan was created on the mid-exhale phase of a measured ground truth (GT) respiratory-correlated 4D-CT dataset with the tracked nodule as gross tumor volume (GTV). Each plan was recalculated on the GT 4D-CT, randomly sampled tresCT, and static sCTrefimages. Dose distributions for corresponding breathing phases were compared in gamma (2%/2 mm) and dose-volume histogram (DVH) parameter analyses.Main results.The mean gamma pass rate between all tresCT and GT 4D-CT dose distributions was 98.6%. The mean absolute relative deviations of the tresCT with respect to GT DVH parameters were 1.9%, 1.0%, and 1.4% for the GTVD98%,D50%, andD2%, respectively, 1.0% for the remaining nodulesD50%, and 1.5% for the lungV20Gy. The gamma pass rate for the tresCTs was significantly larger (p< 0.01), and the GTVD50%deviations with respect to the GT were significantly smaller (p< 0.01) than for the sCTref.Significance.The results suggest that tresCTs could be valuable for time-resolved reconstruction and intrafractional accumulation of the dose to the GTV for lung cancer patients treated at MR-linacs in the future.

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities

BACKGROUND

Machine learning (ML) is considered an important technology for future data analysis in health care.

METHODS

The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers.

RESULTS AND CONCLUSION

In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future.

KEY POINTS

· ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET..

Unsupervised deep learning framework for data-driven gating in positron emission tomography

BACKGROUND

Physiological motion, such as respiratory motion, has become a limiting factor in the spatial resolution of positron emission tomography (PET) imaging as the resolution of PET detectors continue to improve. Motion-induced misregistration between PET and CT images can also cause attenuation correction artifacts. Respiratory gating can be used to freeze the motion and to reduce motion induced artifacts.

PURPOSE

In this study, we propose a robust data-driven approach using an unsupervised deep clustering network that employs an autoencoder (AE) to extract latent features for respiratory gating.

METHODS

We first divide list-mode PET data into short-time frames. The short-time frame images are reconstructed without attenuation, scatter, or randoms correction to avoid attenuation mismatch artifacts and to reduce image reconstruction time. The deep AE is then trained using reconstructed short-time frame images to extract latent features for respiratory gating. No additional data are required for the AE training. K-means clustering is subsequently used to perform respiratory gating based on the latent features extracted by the deep AE. The effectiveness of our proposed Deep Clustering method was evaluated using physical phantom and real patient datasets. The performance was compared against phase gating based on an external signal (External) and image based principal component analysis (PCA) with K-means clustering (Image PCA).

RESULTS

The proposed method produced gated images with higher contrast and sharper myocardium boundaries than those obtained using the External gating method and Image PCA. Quantitatively, the gated images generated by the proposed Deep Clustering method showed larger center of mass (COM) displacement and higher lesion contrast than those obtained using the other two methods.

CONCLUSIONS

The effectiveness of our proposed method was validated using physical phantom and real patient data. The results showed our proposed framework could provide superior gating than the conventional External method and Image PCA.

Elastic Registration Algorithm Based on Three-dimensional Pulmonary MRI in Quantitative Assessment of Severity of Idiopathic Pulmonary Fibrosis

PURPOSE

To quantitatively analyze lung elasticity in idiopathic pulmonary fibrosis (IPF) using elastic registration based on 3-dimensional pulmonary magnetic resonance imaging (3D-PMRI) and to assess its' correlations with the severity of IPF patients.

MATERIAL AND METHODS

Thirty male patients with IPF (mean age: 62±6 y) and 30 age-matched male healthy controls (mean age: 62±6 y) were prospectively enrolled. 3D-PMRI was acquired with a 3-dimensional ultrashort echo time sequence in end-inspiration and end-expiration. MR images were registered from end-inspiration to end-expiration with the elastic registration algorithm. Jacobian determinants were calculated from deformation fields on color maps. The log means of the Jacobian determinants (Jac-mean) and Dice similarity coefficient were used to describe lung elasticity between 2 groups. Then, the correlation of lung elasticity with dyspnea Medical Research Council (MRC) score, exercise tolerance, health-related quality of life, lung function, and the extent of pulmonary fibrosis on chest computed tomography were analyzed.

RESULTS

The Jac-mean of IPF patients (-0.19, [IQR: -0.22, -0.15]) decreased (absolute value), compared with healthy controls (-0.28, [IQR: -0.31, -0.24], P<0.001). The lung elasticity in IPF patients with dyspnea MRC≥3 (Jac-mean: -0.15; Dice: 0.06) was significantly lower than MRC 1 (Jac-mean: -0.22, P=0.001; Dice: 0.10, P=0.001) and MRC 2 (Jac-mean: -0.21, P=0.007; Dice: 0.09, P<0.001). In addition, the Jac-mean negatively correlated with forced vital capacity % (r=-0.487, P<0.001), forced expiratory volume 1% (r=-0.413, P=0.004), TLC% (r=-0.488, P<0.001), diffusing capacity of the lungs for carbon monoxide % predicted (r=-0.555, P<0.001), 6-minute walk distance (r=-0.441, P=0.030) and positively correlated with respiratory symptoms (r=0.430, P=0.042). Meanwhile, the Dice similarity coefficient positively correlated with forced vital capacity % (r=0.577, P=0.004), forced expiratory volume 1% (r=0.526, P=0.012), diffusing capacity of the lungs for carbon monoxide % predicted (r=0.435, P=0.048), 6-minute walk distance (r=0.473, P=0.016), final peripheral oxygen saturation (r=0.534, P=0.004), the extent of fibrosis on chest computed tomography (r=-0.421, P=0.021) and negatively correlated with activity (r=-0.431, P=0.048).

CONCLUSION

Lung elasticity decreased in IPF patients and correlated with dyspnea, exercise tolerance, health-related quality of life, lung function, and the extent of pulmonary fibrosis. The lung elasticity based on elastic registration of 3D-PMRI may be a new nonradiation imaging biomarker for quantitative evaluation of the severity of IPF.

Motion-compensated low-rank reconstruction for simultaneous structural and functional UTE lung MRI

PURPOSE

Three-dimensional UTE MRI has shown the ability to provide simultaneous structural and functional lung imaging, but it is limited by respiratory motion and relatively low lung parenchyma SNR. The purpose of this paper is to improve this imaging by using a respiratory phase-resolved reconstruction approach, named motion-compensated low-rank reconstruction (MoCoLoR), which directly incorporates motion compensation into a low-rank constrained reconstruction model for highly efficient use of the acquired data.

THEORY AND METHODS

The MoCoLoR reconstruction is formulated as an optimization problem that includes a low-rank constraint using estimated motion fields to reduce the rank, optimizing over both the motion fields and reconstructed images. The proposed reconstruction along with XD and motion state-weighted motion-compensation (MostMoCo) methods were applied to 18 lung MRI scans of pediatric and young adult patients. The data sets were acquired under free-breathing and without sedation with 3D radial UTE sequences in approximately 5 min. After reconstruction, they went through ventilation analyses. Performance across reconstruction regularization and motion-state parameters were also investigated.

RESULTS

The in vivo experiments results showed that MoCoLoR made efficient use of the data, provided higher apparent SNR compared with state-of-the-art XD reconstruction and MostMoCo reconstructions, and yielded high-quality respiratory phase-resolved images for ventilation mapping. The method was effective across the range of patients scanned.

CONCLUSION

The motion-compensated low-rank regularized reconstruction approach makes efficient use of acquired data and can improve simultaneous structural and functional lung imaging with 3D-UTE MRI. It enables the scanning of pediatric patients under free-breathing and without sedation.

Analysis of emerging trends and hot spots in respiratory biomechanics from 2003 to 2022 based on CiteSpace

Introduction: With the global prevalence of coronavirus disease 2019 (COVID-19), an increasing number of people are experiencing respiratory discomfort. Respiratory biomechanics can monitor breathing patterns and respiratory movements and it is easier to prevent, diagnose, treat or rehabilitate. However, there is still a lack of global knowledge structure in the field of respiratory biomechanics. With the help of CiteSpace software, we aim to help researchers identify potential collaborators and collaborating institutions, hotspots and research frontiers in respiratory biomechanics. Methods: Articles on respiratory biomechanics from 2003 to 2022 were retrieved from the Web of Science Core Collection by using a specific strategy, resulting a total of 2,850 publications. We used CiteSpace 6.1.R6 to analyze the year of publication, journal/journals cited, country, institution, author/authors cited, references, keywords and research trends. Co-citation maps were created to visually observe research hot spots and knowledge structures. Results and discussion: The number of annual publications gradually increased over the past 20 years. Medical Physics published the most articles and had the most citations in this study. The United States was the most influential country, with the highest number and centrality of publications. The most productive and influential institution was Harvard University in the United States. Keall PJ was the most productive author and MCCLELLAND JR was the most cited authors The article by Keall PJ (2006) article (cocitation counts: 55) and the article by McClelland JR (2013) were the most representative and symbolic references, with the highest cocitation number and centrality, respectively. The top keywords were "radiotherapy", "volume", and "ventilation". The top Frontier keywords were "organ motion," "deep inspiration," and "deep learning". The keywords were clustered to form seven labels. Currently, the main area of research in respiratory biomechanics is respiratory motion related to imaging techniques. Future research may focus on respiratory assistance techniques and respiratory detection techniques. At the same time, in the future, we will pay attention to personalized medicine and precision medicine, so that people can monitor their health status anytime and anywhere.

X-ray source arrays for volumetric imaging during radiotherapy treatment

This work presents a novel hardware configuration for radiotherapy systems to enable fast 3D X-ray imaging before and during treatment delivery. Standard external beam radiotherapy linear accelerators (linacs) have a single X-ray source and detector located at ± 90° from the treatment beam respectively. The entire system can be rotated around the patient acquiring multiple 2D X-ray images to create a 3D cone-beam Computed Tomography (CBCT) image before treatment delivery to ensure the tumour and surrounding organs align with the treatment plan. Scanning with a single source is slow relative to patient respiration or breath holds and cannot be performed during treatment delivery, limiting treatment delivery accuracy in the presence of patient motion and excluding some patients from concentrated treatment plans that would be otherwise expected to have improved outcomes. This simulation study investigated whether recent advances in carbon nanotube (CNT) field emission source arrays, high frame rate (60 Hz) flat panel detectors and compressed sensing reconstruction algorithms could circumvent imaging limitations of current linacs. We investigated a novel hardware configuration incorporating source arrays and high frame rate detectors into an otherwise standard linac. We investigated four potential pre-treatment scan protocols that could be achieved in a 17 s breath hold or 2-10 1 s breath holds. Finally, we demonstrated for the first time volumetric X-ray imaging during treatment delivery by using source arrays, high frame rate detectors and compressed sensing. Image quality was assessed quantitatively over the CBCT geometric field of view as well as across each axis through the tumour centroid. Our results demonstrate that source array imaging enables larger volumes to be imaged with acquisitions as short as 1 s albeit with reduced image quality arising from lower photon flux and shorter imaging arcs.

Gaussian Processes for real-time 3D motion and uncertainty estimation during MR-guided radiotherapy

Respiratory motion during radiotherapy causes uncertainty in the tumor's location, which is typically addressed by an increased radiation area and a decreased dose. As a result, the treatments' efficacy is reduced. The recently proposed hybrid MR-linac scanner holds the promise to efficiently deal with such respiratory motion through real-time adaptive MR-guided radiotherapy (MRgRT). For MRgRT, motion-fields should be estimated from MR-data and the radiotherapy plan should be adapted in real-time according to the estimated motion-fields. All of this should be performed with a total latency of maximally 200 ms, including data acquisition and reconstruction. A measure of confidence in such estimated motion-fields is highly desirable, for instance to ensure the patient's safety in case of unexpected and undesirable motion. In this work, we propose a framework based on Gaussian Processes to infer 3D motion-fields and uncertainty maps in real-time from only three readouts of MR-data. We demonstrated an inference frame rate up to 69 Hz including data acquisition and reconstruction, thereby exploiting the limited amount of required MR-data. Additionally, we designed a rejection criterion based on the motion-field uncertainty maps to demonstrate the framework's potential for quality assurance. The framework was validated in silico and in vivo on healthy volunteer data (n=5) acquired using an MR-linac, thereby taking into account different breathing patterns and controlled bulk motion. Results indicate end-point-errors with a 75th percentile below 1 mm in silico, and a correct detection of erroneous motion estimates with the rejection criterion. Altogether, the results show the potential of the framework for application in real-time MR-guided radiotherapy with an MR-linac.

A novel external/internal tumor tracking approach to compensate for respiratory motion baseline drifts

Objective.Real-time respiratory tumor tracking as implemented in a robotic treatment unit is based on continuous optical measurement of the position of external markers and a correlation model between them and internal target positions, which are established with X-ray imaging of the tumor, or fiducials placed in or around the tumor. Correlation models are created with fifteen simultaneously measured external/internal marker position pairs divided over the respiratory cycle. Every 45-150 s, the correlation model is updated by replacing the three first acquired data pairs with three new pairs. Tracking simulations for >120.000 computer-generated respiratory tracks demonstrated that this tracking approach resulted in relevant inaccuracies in internal target position predictions, especially in case of presence of respiratory motion baseline drifts.Approach.To better cope with drifts, we introduced a novel correlation model with an explicit time dependence, and we proposed to replace the currently applied linear-motion tracking (LMT) by mixed-model tracking (MMT). In MMT, the linear correlation model is extended with an explicit time dependence in case of a detected baseline drift. MMT prediction accuracies were then established for the same >120.000 computer-generated patients as used for LMT.Main results.For 150 s update intervals, MMT outperformed LMT in internal target position prediction accuracy for 93.7 ∣ 97.2% of patients with 0.25 ∣ 0.5 mm min^-1linear respiratory motion baseline drifts with similar numbers of X-ray images and similar treatment times. For the upper 25% of patients, mean 3D internal target position prediction errors reduced by 0.7 ∣ 1.8 mm, while near maximum reductions (upper 10% of patients) were 0.9 ∣ 2.0 mm.Significance.For equal numbers of acquired X-ray images, MMT greatly improved tracking accuracy compared to LMT, especially in the presence of baseline drifts. Even with almost 50% less acquired X-ray images, MMT still outperformed LMT in internal target position prediction accuracy.

Ultra-wide band radar for prospective respiratory motion correction in the liver

Objective.T1 mapping of the liver is time consuming and can be challenging due to respiratory motion. Here we present a prospective slice tracking approach, which utilizes an external ultra-wide band radar signal and allows for efficient T1 mapping during free-breathing.Approach.The fast radar signal is calibrated to an MR-based motion signal to create a motion model. This motion model provides motion estimates, which are used to carry out slice tracking for any subsequent clinical scan. This approach was evaluated in simulations, phantom experiments andin vivoscans.Main results.Radar-based slice tracking was implemented on an MR system with a total latency of 77 ms. Moving phantom experiments showed accurate motion prediction with an error of 0.12 mm in anterior-posterior and 0.81 mm in head-feet direction. The model error remained stable for up to two hours.In vivoexperiments showed visible image improvement with a motion model error three times smaller than with a respiratory bellow. For T1 mapping during free-breathing the proposed approach provided similar results compared to reference T1 mapping during a breathhold.Significance.The proposed radar-based approach achieves accurate slice tracking and enables efficient T1 mapping of the liver during free-breathing. This motion correction approach is independent from scanning parameters and could also be used for applications like MR guided radiotherapy or MR Elastography.

RMSim: controlled respiratory motion simulation on static patient scans

Objective.This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR.Approach.We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation.Main results.We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim's potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data.Significance.The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released athttps://github.com/nadeemlab/SeqX2Y.

Breathing-Adapted Imaging Techniques for Rapid 4-Dimensional Lung Tomosynthesis

Purpose

This article presents enhancements to a 4-dimensional (4D) lung digital tomosynthesis (DTS) model introduced in a 2018 paper. That model was proposed as an adjunct to 4D computed tomography (4DCT) to improve tumor localization through artifact reduction achieved by imaging the entire lung in all projections, reducing the projection collection time duration for each phase compared with 4DCT, and requiring only a single-breath cycle to capture all phases. This is applicable to SABR treatment planning. Enhancements comprise customized patient 4D-DTS x-ray scanning parameters.

Methods and Materials

Imaging parameters derived with the 4D-DTS model were arc duration, frames per second, pulse duration, and tube current normalized to single-chest radiographic milliampere-seconds (mA/mAs_AEC). Optimized phase-specific DTS projections imaging parameters were derived for volunteer respiration-tracking surrogate waveforms and for sinusoidal waveforms. These parameters are temporally matched to the respiratory surrogate waveform and presented as continuous data plots during a period of 20 seconds. Comparison is made between surrogate excursions during a single-phase CT and 4D-DTS reconstructions.

Results

4D-DTS imaging techniques were customized to volunteer respiratory waveforms and sinusoidal waveforms. Technique settings at the highest velocity portions of the volunteer waveforms were arc duration 0.066 seconds, frame rate 921 Hz, pulse duration 1.076 ms, and normalized tube current 76.2 s^-1. Technique settings at the highest velocity portions of the sinusoidal waveforms were arc duration 0.029 seconds, frame rate 2074 Hz, pulse duration 0.472 ms, and normalized tube current 173.6 s^-1. Sinusoidal surrogate excursion distance at the highest velocity portion of the waveform during a CT rotation of 0.5 seconds ranged from 2.68 to 21.09 mm, all greater than the limiting excursion distance chosen in the 4D-DTS model.

Conclusions

4D-DTS image technique settings can be customized to individual patient breathing patterns so that captured range of motion satisfies an operator-selected value.

Patient-specific Cardio-respiratory Motion Prediction in X-ray Angiography using LSTM Networks

Objective.To develop a novel patient-specific cardio-respiratory motion prediction approach for X-ray angiography time series based on a simple long short-term memory (LSTM) model.Approach.The cardio-respiratory motion behavior in an X-ray image sequence was represented as a sequence of 2D affine transformation matrices, which provide the displacement information of contrasted moving objects (arteries and medical devices) in a sequence. The displacement information includes translation, rotation, shearing, and scaling in 2D. A many-to-many LSTM model was developed to predict 2D transformation parameters in matrix form for future frames based on previously generated images. The method was developed with 64 simulated phantom datasets (pediatric and adult patients) using a realistic cardio-respiratory motion simulator (XCAT) and was validated using 10 different patient X-ray angiography sequences.Main results.Using this method we achieved less than 1 mm prediction error for complex cardio-respiratory motion prediction. The following mean prediction error values were recorded over all the simulated sequences: 0.39 mm (for both motions), 0.33 mm (for only cardiac motion), and 0.47 mm (for only respiratory motion). The mean prediction error for the patient dataset was 0.58 mm.Significance.This study paves the road for a patient-specific cardio-respiratory motion prediction model, which might improve navigation guidance during cardiac interventions.

Limitations of phase-sorting based pencil beam scanned 4D proton dose calculations under irregular motion

Objective.4D dose calculation (4DDC) for pencil beam scanned (PBS) proton therapy is typically based on phase-sorting of individual pencil beams onto phases of a single breathing cycle 4DCT. Understanding the dosimetric limitations and uncertainties of this approach is essential, especially for the realistic treatment scenario with irregular free breathing motion.Approach.For three liver and three lung cancer patient CTs, the deformable multi-cycle motion from 4DMRIs was used to generate six synthetic 4DCT(MRI)s, providing irregular motion (11/15 cycles for liver/lung; tumor amplitudes ∼4-18 mm). 4DDCs for two-field plans were performed, with the temporal resolution of the pencil beam delivery (4-200 ms) or with 8 phases per breathing cycle (500-1000 ms). For the phase-sorting approach, the tumor center motion was used to determine the phase assignment of each spot. The dose was calculated either using the full free breathing motion or individually repeating each single cycle. Additionally, the use of an irregular surrogate signal prior to 4DDC on a repeated cycle was simulated. The CTV volume with absolute dose differences >5% (V_dosediff>5%) and differences in CTVV_95%andD_5%-D_95%compared to the free breathing scenario were evaluated.Main results.Compared to 4DDC considering the full free breathing motion with finer spot-wise temporal resolution, 4DDC based on a repeated single 4DCT resulted inV_dosediff>5%of on average 34%, which resulted in an overestimation ofV_95%up to 24%. However, surrogate based phase-sorting prior to 4DDC on a single cycle 4DCT, reduced the averageV_dosediff>5%to 16% (overestimationV_95%up to 19%). The 4DDC results were greatly influenced by the choice of reference cycle (V_dosediff>5%up to 55%) and differences due to temporal resolution were much smaller (V_dosediff>5%up to 10%).Significance.It is important to properly consider motion irregularity in 4D dosimetric evaluations of PBS proton treatments, as 4DDC based on a single 4DCT can lead to an underestimation of motion effects.

NuTracker: a coordinate-based neural network representation of lung motion for intrafraction tumor tracking with various surrogates in radiotherapy

Objective. Tracking tumors and surrounding tissues in real-time is critical for reducing errors and uncertainties during radiotherapy. Existing methods are either limited by the linear representation or scale poorly with the volume resolution. To address both issues, we propose a novel coordinate-based neural network representation of lung motion to predict the instantaneous 3D volume at arbitrary spatial resolution from various surrogates: patient surface, fiducial marker, and single kV projection.Approach. The proposed model, namely NuTracker, decomposes the 4DCT into a template volume and dense displacement fields (DDFs), and uses two coordinate neural networks to predict them from spatial coordinates and surrogate states. The predicted template is spatially warped with the predicted DDF to produce the deformed volume for a given surrogate state. The nonlinear coordinate networks enable representing complex motion at infinite resolution. The decomposition allows imposing different regularizations on the spatial and temporal domains. The meta-learning and multi-task learning are used to train NuTracker across patients and tasks, so that commonalities and differences can be exploited. NuTracker was evaluated on seven patients implanted with markers using a leave-one-phase-out procedure.Main results. The 3D marker localization error is 0.66 mm on average and <1 mm at 95th-percentile, which is about 26% and 32% improvement over the predominant linear methods. The tumor coverage and image quality are improved by 5.7% and 11% in terms of dice and PSNR. The difference in the localization error for different surrogates is small and is not statistically significant. Cross-population learning and multi-task learning contribute to performance. The model tolerates surrogate drift to a certain extent.Significance. NuTracker can provide accurate estimation for entire tumor volume based on various surrogates at infinite resolution. It is of great potential to apply the coordinate network to other imaging modalities, e.g. 4DCBCT and other tasks, e.g. 4D dose calculation.

The future of MRI in radiation therapy: Challenges and opportunities for the MR community

Radiation therapy is a major component of cancer treatment pathways worldwide. The main aim of this treatment is to achieve tumor control through the delivery of ionizing radiation while preserving healthy tissues for minimal radiation toxicity. Because radiation therapy relies on accurate localization of the target and surrounding tissues, imaging plays a crucial role throughout the treatment chain. In the treatment planning phase, radiological images are essential for defining target volumes and organs-at-risk, as well as providing elemental composition (e.g., electron density) information for radiation dose calculations. At treatment, onboard imaging informs patient setup and could be used to guide radiation dose placement for sites affected by motion. Imaging is also an important tool for treatment response assessment and treatment plan adaptation. MRI, with its excellent soft tissue contrast and capacity to probe functional tissue properties, holds great untapped potential for transforming treatment paradigms in radiation therapy. The MR in Radiation Therapy ISMRM Study Group was established to provide a forum within the MR community to discuss the unmet needs and fuel opportunities for further advancement of MRI for radiation therapy applications. During the summer of 2021, the study group organized its first virtual workshop, attended by a diverse international group of clinicians, scientists, and clinical physicists, to explore our predictions for the future of MRI in radiation therapy for the next 25 years. This article reviews the main findings from the event and considers the opportunities and challenges of reaching our vision for the future in this expanding field.

Thorax Dynamic Modeling and Biomechanical Analysis of Chest Breathing in Supine Lying Position

During respiration, the expansion and contraction of the chest and abdomen are coupled with each other, presenting a complex torso movement pattern. A finite element (FE) model of chest breathing based on the HUMOS2 human body model was developed. One-dimensional muscle units with active contraction functions were incorporated into the model based on Hill's active muscle model so as to generate muscle contraction forces that can change over time. The model was validated by comparing it to the surface displacement of the chest and abdomen during respiration. Then, the mechanism of the coupled motion of the chest and abdomen was analyzed. The analyses revealed that since the abdominal wall muscles are connected to the lower edge of the rib cage through tendons, the movement of the rib cage may cause the abdominal wall muscles to be stretched in both horizontal and vertical in a supine position. The anteroposterior and the right-left diameters of the chest will increase at inspiration, while the right-left diameter of the abdomen will decrease even though the anteroposterior diameter of the abdomen increases. The external intercostal muscles at different regions had different effects on the motion of the ribs during respiration. In particular, the external intercostal muscles at the lateral region had a larger effect on pump handle movement than bucket handle movement, and the external intercostal muscles at the dorsal region had a greater influence on bucket handle movement than pump handle movement.

Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy

BACKGROUND AND OBJECTIVE

During lung cancer radiotherapy, the position of infrared reflective objects on the chest can be recorded to estimate the tumor location. However, radiotherapy systems have a latency inherent to robot control limitations that impedes the radiation delivery precision. Prediction with online learning of recurrent neural networks (RNN) allows for adaptation to non-stationary respiratory signals, but classical methods such as real-time recurrent learning (RTRL) and truncated backpropagation through time are respectively slow and biased. This study investigates the capabilities of unbiased online recurrent optimization (UORO) to forecast respiratory motion and enhance safety in lung radiotherapy.

METHODS

We used nine observation records of the three-dimensional (3D) position of three external markers on the chest and abdomen of healthy individuals breathing during intervals from 73s to 222s. The sampling frequency was 10Hz, and the amplitudes of the recorded trajectories range from 6mm to 40mm in the superior-inferior direction. We forecast the 3D location of each marker simultaneously with a horizon value (the time interval in advance for which the prediction is made) between 0.1s and 2.0s, using an RNN trained with UORO. We compare its performance with an RNN trained with RTRL, least mean squares (LMS), and offline linear regression. We provide closed-form expressions for quantities involved in the gradient loss calculation in UORO, thereby making its implementation efficient. Training and cross-validation were performed during the first minute of each sequence.

RESULTS

On average over the horizon values considered and the nine sequences, UORO achieves the lowest root-mean-square (RMS) error and maximum error among the compared algorithms. These errors are respectively equal to 1.3mm and 8.8mm, and the prediction time per time step was lower than 2.8ms (Dell Intel core i9-9900K 3.60 GHz). Linear regression has the lowest RMS error for the horizon values 0.1s and 0.2s, followed by LMS for horizon values between 0.3s and 0.5s, and UORO for horizon values greater than 0.6s.

CONCLUSIONS

UORO can accurately predict the 3D position of external markers for intermediate to high response times with an acceptable time performance. This will help limit unwanted damage to healthy tissues caused by radiotherapy.

A personalized image-guided intervention system for peripheral lung cancer on patient-specific respiratory motion model

PURPOSE

Due to respiratory motion, precise tracking of lung nodule movement is a persistent challenge for guiding percutaneous lung biopsy during image-guided intervention. We developed an automated image-guided system incorporating effective and robust tracking algorithms to address this challenge. Accurate lung motion prediction and personalized image-guided intervention are the key technological contributions of this work.

METHODS

A patient-specific respiratory motion model is developed to predict pulmonary movements of individual patients. It is based on the relation between the artificial 4D CT and corresponding positions tracked by position sensors attached on the chest using an electromagnetic (EM) tracking system. The 4D CT image of the thorax during breathing is calculated through deformable registration of two 3D CT scans acquired at inspiratory and expiratory breath-hold. The robustness and accuracy of the image-guided intervention system were assessed on a static thorax phantom under different clinical parametric combinations.

RESULTS

Real 4D CT images of ten patients were used to evaluate the accuracy of the respiratory motion model. The mean error of the model in different breathing phases was 1.59 ± 0.66 mm. Using a static thorax phantom, we achieved an average targeting accuracy of 3.18 ± 1.2 mm across 50 independent tests with different intervention parameters. The positive results demonstrate the robustness and accuracy of our system for personalized lung cancer intervention.

CONCLUSIONS

The proposed system integrates a patient-specific respiratory motion compensation model to reduce the effect of respiratory motion during percutaneous lung biopsy and help interventional radiologists target the lesion efficiently. Our preclinical studies indicate that the image-guided system has the ability to accurately predict and track lung nodules of individual patients and has the potential for use in the diagnosis and treatment of early stage lung cancer.

Motion modeling from 4D MR images of liver simulating phantom

Background and purpose

A novel method of retrospective liver modeling was developed based on four‐dimensional magnetic resonance (4D‐MR) images. The 4D‐MR images will be utilized in generation of the subject‐specific deformable liver model to be used in radiotherapy planning (RTP). The purpose of this study was to test and validate the developed 4D‐magnetic resonance imaging (MRI) method with extensive phantom tests. We also aimed to build a motion model with image registration methods from liver simulating phantom images.

Materials and methods

A deformable phantom was constructed by combining deformable tissue‐equivalent material and a programmable 4D CIRS‐platform. The phantom was imaged in 1.5 T MRI scanner with T2‐weighted 4D SSFSE and T1‐weighted Ax dual‐echo Dixon SPGR sequences, and in computed tomography (CT). In addition, geometric distortion of the 4D sequence was measured with a GRADE phantom. The motion model was developed; the phases of the 4D‐MRI were used as surrogate data, and displacement vector fields (DVF's) were used as a motion measurement. The motion model and the developed 4D‐MRI method were evaluated and validated with extensive tests.

Result

The 4D‐MRI method enabled an accuracy of 2 mm using our deformable phantom compared to the 4D‐CT. Results showed a mean accuracy of <2 mm between coordinates and DVF's measured from the 4D images. Three‐dimensional geometric accuracy results with the GRADE phantom were: 0.9‐mm mean and 2.5 mm maximum distortion within a 100 mm distance, and 2.2 mm mean, 5.2 mm maximum distortion within a 150 mm distance from the isocenter.

Conclusions

The 4D‐MRI method was validated with phantom tests as a necessary step before patient studies. The subject‐specific motion model was generated and will be utilized in the generation of the deformable liver model of patients to be used in RTP.

Time-resolved MRI for off-line treatment robustness evaluation in carbon-ion radiotherapy of pancreatic cancer

PURPOSE

In this study, we investigate the use of magnetic resonance imaging (MRI) for the clinical evaluation of gating treatment robustness in carbon-ion radiotherapy (CIRT) of pancreatic cancer. Indeed, MRI allows radiation-free repeated scans and fast dynamic sequences for time-resolved (TR) imaging (cine-MRI), providing information on inter- and intra-fraction cycle-to-cycle variations of respiratory motion. MRI can therefore support treatment planning and verification, overcoming the limitations of the current clinical standard, that is, four-dimensional computed tomography (4DCT), which describes an "average" breathing cycle neglecting breathing motion variability.

METHODS

We integrated a technique to generate a virtual CT (vCT) from 3D MRI with a method for 3D reconstruction from 2D cine-MRI, to produce TR vCTs for dose recalculations. For eight patients, the method allowed evaluating inter-fraction variations at end-exhale and intra-fraction cycle-to-cycle variability within the gating window in terms of tumor displacement and dose to the target and organs at risk.

RESULTS

The median inter-fraction tumor motion was in the range 3.33-12.16 mm, but the target coverage was robust (-0.4% median D95% variation). Concerning cycle-to-cycle variations, the gating technique was effective in limiting tumor displacement (1.35 mm median gating motion) and corresponding dose variations (-3.9% median D95% variation). The larger exposure of organs at risk (duodenum and stomach) was caused by inter-fraction motion, whereas intra-fraction cycle-to-cycle dose variations were limited.

CONCLUSIONS

This study proposed a method for the generation of TR vCTs from MRI, which enabled an off-line evaluation of gating treatment robustness and suggested its feasibility to support treatment planning of pancreatic tumors in CIRT.

MR-assisted PET respiratory motion correction using deep-learning based short-scan motion fields

PURPOSE

We evaluated the impact of PET respiratory motion correction (MoCo) in a phantom and patients. Moreover, we proposed and examined a PET MoCo approach using motion vector fields (MVFs) from a deep-learning reconstructed short MRI scan.

METHODS

The evaluation of PET MoCo was performed in a respiratory motion phantom study with varying lesion sizes and tumor to background ratios (TBRs) using a static scan as the ground truth. MRI-based MVFs were derived from either 2000 spokes (MoCo₂₀₀₀ , 5-6 min acquisition time) using a Fourier transform reconstruction or 200 spokes (MoCo_P2P200 , 30-40 s acquisition time) using a deep-learning Phase2Phase (P2P) reconstruction and then incorporated into PET MoCo reconstruction. For six patients with hepatic lesions, the performance of PET MoCo was evaluated using quantitative metrics (SUV_max , SUV_peak , SUV_mean , lesion volume) and a blinded radiological review on lesion conspicuity.

RESULTS

MRI-assisted PET MoCo methods provided similar results to static scans across most lesions with varying TBRs in the phantom. Both MoCo₂₀₀₀ and MoCo_P2P200 PET images had significantly higher SUV_max , SUV_peak , SUV_mean and significantly lower lesion volume than non-motion-corrected (non-MoCo) PET images. There was no statistical difference between MoCo₂₀₀₀ and MoCo_P2P200 PET images for SUV_max , SUV_peak , SUV_mean or lesion volume. Both radiological reviewers found that MoCo₂₀₀₀ and MoCo_P2P200 PET significantly improved lesion conspicuity.

CONCLUSION

An MRI-assisted PET MoCo method was evaluated using the ground truth in a phantom study. In patients with hepatic lesions, PET MoCo images improved quantitative and qualitative metrics based on only 30-40 s of MRI motion modeling data.

Dosimetry impact of gating latency in cine magnetic resonance image guided breath-hold pancreatic cancer radiotherapy

Objective.We investigated dosimetry effect of gating latency in cine magnetic resonance image (cine MRI) guided breath-hold pancreatic cancer radiotherapy.Approach.The gating latency was calculated based on cine MRI obtained from 17 patients who received MRI guided radiotherapy. Because of the cine MRI-related latency, beam overshoot occurs when beam remains on while the tracking target already moves out of the target boundary. The number of beam on/off events was calculated from the cine MRI data. We generated both IMRT and VMAT plans for all 17 patients using 33 Gy prescription, and created motion plans by applying isocenter shift that corresponds to motion-induced tumor displacement. The GTV and PTV coverage and dose to nearby critical structures were compared between the motion and original plan to evaluate the dosimetry change caused by cine MRI latency.Main results.The time ratio of cine MRI imaging latency over the treatment duration is 6.6 ± 3.1%, the mean and median percentage of beam-on events <4 s are 67.0 ± 14.3% and 66.6%. When a gating boundary of 4 mm and a target-out threshold of 5% is used, there is no significant difference for GTV V33Gy between the motion and original plan (p = 0.861 and 0.397 for IMRT and VMAT planning techniques, respectively). However, the PTV V33Gy and stomach Dmax for the motion plans are significantly lower; duodenum V12.5 Gy and V18Gy are significantly higher when compared with the original plans, for both IMRT and VMAT planning techniques.Significance.The cine MRI gating latency can significantly decrease the dose delivered to the PTV, and increase the dose to the nearby critical structures. However, no significant difference is observed for the GTV coverage. The dosimetry impact can be mitigated by implementing additional beam-on control techniques which reduces unnecessary beam on events and/or by using faster cine MRI sequences which reduces the latency period.

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities

BACKGROUND

 Machine learning (ML) is considered an important technology for future data analysis in health care.

METHODS

 The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers.

RESULTS AND CONCLUSION

 In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future.

KEY POINTS

  · ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET..

CITATION FORMAT

· Küstner T, Hepp T, Seith F. Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities. Fortschr Röntgenstr 2022; DOI: 10.1055/a-1718-4128.

Registration of IRT and visible light images in neurosurgery: analysis and comparison of automatic intensity-based registration approaches

PURPOSE

The purpose of this study is to analyze and compare six automatic intensity-based registration methods for intraoperative infrared thermography (IRT) and visible light imaging (VIS/RGB). The practical requirement is to get a good performance of Euclidean distance between manually set landmarks in reference and target images as well as to achieve a high structural similarity index metric (SSIM) and peak signal-to-noise ratio (PSNR) with respect to the reference image.

METHODS

In this study, preprocessing is applied to bring both image types to a similar intensity. Similarity transformation is employed to align roughly IRT and visible light images. Two optimizers and two measures are used in this process. Thereafter, due to locally different displacement of the brain surface through respiration and heartbeat, two non-rigid transformations are applied, and finally, a bicubic interpolation is carried out to compensate for the resulting estimated transformation. Performance was assessed using eleven image datasets. The registration accuracy of the different computational approaches was assessed based on SSIM and PSNR. Additionally, five concise landmarks for each dataset were selected manually in reference and target images and the Euclidean distance between the corresponding landmarks was compared.

RESULTS

The results are showing that the combination of normalized intensity, mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration results in improved accuracy and performance as compared to all other methods tested here. Furthermore, the obtained results led to [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] registrations for datasets 1, 2, 5, 7, and 8 with respect to the second best result by calculating the mean Euclidean distance of five landmarks.

CONCLUSIONS

We conclude that the mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration can achieve better accuracy and performance to those other methods mentioned here for automatic registration of IRT and visible light images in neurosurgery.

Real-Time Non-Rigid 3D Respiratory Motion Estimation for MR-Guided Radiotherapy Using MR-MOTUS

The MR-Linac is a combination of an MR-scanner and radiotherapy linear accelerator (Linac) which holds the promise to increase the precision of radiotherapy treatments with MR-guided radiotherapy by monitoring motion during radiotherapy with MRI, and adjusting the radiotherapy plan accordingly. Optimal MR-guidance for respiratory motion during radiotherapy requires MR-based 3D motion estimation with a latency of 200-500 ms. Currently this is still challenging since typical methods rely on MR-images, and are therefore limited by the 3D MR-imaging latency. In this work, we present a method to perform non-rigid 3D respiratory motion estimation with 170 ms latency, including both acquisition and reconstruction. The proposed method called real-time low-rank MR-MOTUS reconstructs motion-fields directly from k -space data, and leverages an explicit low-rank decomposition of motion-fields to split the large scale 3D+t motion-field reconstruction problem posed in our previous work into two parts: (I) a medium-scale offline preparation phase and (II) a small-scale online inference phase which exploits the results of the offline phase for real-time computations. The method was validated on free-breathing data of five volunteers, acquired with a 1.5T Elekta Unity MR-Linac. Results show that the reconstructed 3D motion-field are anatomically plausible, highly correlated with a self-navigation motion surrogate ( R=0.975 ±0.0110 ), and can be reconstructed with a total latency of 170 ms that is sufficient for real-time MR-guided abdominal radiotherapy.

Motion-compensated fat-water imaging for 3D cardiac MRI at ultra-high fields

PURPOSE

Respiratory motion-compensated (MC) 3D cardiac fat-water imaging at 7T.

METHODS

Free-breathing bipolar 3D triple-echo gradient-recalled-echo (GRE) data with radial phase-encoding (RPE) trajectory were acquired in 11 healthy volunteers (7M\4F, 21-35 years, mean: 30 years) with a wide range of body mass index (BMI; 19.9-34.0 kg/m² ) and volunteer tailored B 1 + shimming. The bipolar-corrected triple-echo GRE-RPE data were binned into different respiratory phases (self-navigation) and were used for the estimation of non-rigid motion vector fields (MF) and respiratory resolved (RR) maps of the main magnetic field deviations (ΔB₀ ). RR ΔB₀ maps and MC ΔB₀ maps were compared to a reference respiratory phase to assess respiration-induced changes. Subsequently, cardiac binned fat-water images were obtained using a model-based, respiratory motion-corrected image reconstruction.

RESULTS

The 3D cardiac fat-water imaging at 7T was successfully demonstrated. Local respiration-induced frequency shifts in MC ΔB₀ maps are small compared to the chemical shifts used in the multi-peak model. Compared to the reference exhale ΔB₀ map these changes are in the order of 10 Hz on average. Cardiac binned MC fat-water reconstruction reduced respiration induced blurring in the fat-water images, and flow artifacts are reduced in the end-diastolic fat-water separated images.

CONCLUSION

This work demonstrates the feasibility of 3D fat-water imaging at UHF for the entire human heart despite spatial and temporal B 1 + and B₀ variations, as well as respiratory and cardiac motion.

PET respiratory motion correction: quo vadis?

Positron emission tomography (PET) respiratory motion correction has been a subject of great interest for the last twenty years, prompted mainly by the development of multimodality imaging devices such as PET/computed tomography (CT) and PET/magnetic resonance imaging (MRI). PET respiratory motion correction involves a number of steps including acquisition synchronization, motion estimation and finally motion correction. The synchronization steps include the use of different external device systems or data driven approaches which have been gaining ground over the last few years. Patient specific or generic motion models using the respiratory synchronized datasets can be subsequently derived and used for correction either in the image space or within the image reconstruction process. Similar overall approaches can be considered and have been proposed for both PET/CT and PET/MRI devices. Certain variations in the case of PET/MRI include the use of MRI specific sequences for the registration of respiratory motion information. The proposed review includes a comprehensive coverage of all these areas of development in field of PET respiratory motion for different multimodality imaging devices and approaches in terms of synchronization, estimation and subsequent motion correction. Finally, a section on perspectives including the potential clinical usage of these approaches is included.

Ultrasound-based sensors for respiratory motion assessment in multimodality PET imaging

Breathing motion can displace internal organs by up to several cm; as such, it is a primary factor limiting image quality in medical imaging. Motion can also complicate matters when trying to fuse images from different modalities, acquired at different locations and/or on different days. Currently available devices for monitoring breathing motion often do so indirectly, by detecting changes in the outline of the torso rather than the internal motion itself, and these devices are often fixed to floors, ceilings or walls, and thus cannot accompany patients from one location to another. We have developed small ultrasound-based sensors, referred to as 'organ configuration motion' (OCM) sensors, that attach to the skin and provide rich motion-sensitive information. In the present work we tested the ability of OCM sensors to enable respiratory gating during in vivo PET imaging. A motion phantom involving an FDG solution was assembled, and two cancer patients scheduled for a clinical PET/CT exam were recruited for this study. OCM signals were used to help reconstruct phantom and in vivo data into time series of motion-resolved images. As expected, the motion-resolved images captured the underlying motion. In Patient #1, a single large lesion proved to be mostly stationary through the breathing cycle. However, in Patient #2, several small lesions were mobile during breathing, and our proposed new approach captured their breathing-related displacements. In summary, a relatively inexpensive hardware solution was developed here for respiration monitoring. Because the proposed sensors attach to the skin, as opposed to walls or ceilings, they can accompany patients from one procedure to the next, potentially allowing data gathered in different places and at different times to be combined and compared in ways that account for breathing motion.

LAPNet: Non-Rigid Registration Derived in k-Space for Magnetic Resonance Imaging

Physiological motion, such as cardiac and respiratory motion, during Magnetic Resonance (MR) image acquisition can cause image artifacts. Motion correction techniques have been proposed to compensate for these types of motion during thoracic scans, relying on accurate motion estimation from undersampled motion-resolved reconstruction. A particular interest and challenge lie in the derivation of reliable non-rigid motion fields from the undersampled motion-resolved data. Motion estimation is usually formulated in image space via diffusion, parametric-spline, or optical flow methods. However, image-based registration can be impaired by remaining aliasing artifacts due to the undersampled motion-resolved reconstruction. In this work, we describe a formalism to perform non-rigid registration directly in the sampled Fourier space, i.e. k-space. We propose a deep-learning based approach to perform fast and accurate non-rigid registration from the undersampled k-space data. The basic working principle originates from the Local All-Pass (LAP) technique, a recently introduced optical flow-based registration. The proposed LAPNet is compared against traditional and deep learning image-based registrations and tested on fully-sampled and highly-accelerated (with two undersampling strategies) 3D respiratory motion-resolved MR images in a cohort of 40 patients with suspected liver or lung metastases and 25 healthy subjects. The proposed LAPNet provided consistent and superior performance to image-based approaches throughout different sampling trajectories and acceleration factors.

Deep learning-based motion tracking using ultrasound images

PURPOSE

Ultrasound (US) imaging is an established imaging modality capable of offering video-rate volumetric images without ionizing radiation. It has the potential for intra-fraction motion tracking in radiation therapy. In this study, a deep learning-based method has been developed to tackle the challenges in motion tracking using US imaging.

METHODS

We present a Markov-like network, which is implemented via generative adversarial networks, to extract features from sequential US frames (one tracked frame followed by untracked frames) and thereby estimate a set of deformation vector fields (DVFs) through the registration of the tracked frame and the untracked frames. The positions of the landmarks in the untracked frames are finally determined by shifting landmarks in the tracked frame according to the estimated DVFs. The performance of the proposed method was evaluated on the testing dataset by calculating the tracking error (TE) between the predicted and ground truth landmarks on each frame.

RESULTS

The proposed method was evaluated using the MICCAI CLUST 2015 dataset which was collected using seven US scanners with eight types of transducers and the Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) dataset which was acquired using GE Vivid E95 ultrasound scanners. The CLUST dataset contains 63 2D and 22 3D US image sequences respectively from 42 and 18 subjects, and the CAMUS dataset includes 2D US images from 450 patients. On CLUST dataset, our proposed method achieved a mean tracking error of 0.70 ± 0.38 mm for the 2D sequences and 1.71 ± 0.84 mm for the 3D sequences for those public available annotations. And on CAMUS dataset, a mean tracking error of 0.54 ± 1.24 mm for the landmarks in the left atrium was achieved.

CONCLUSIONS

A novel motion tracking algorithm using US images based on modern deep learning techniques has been demonstrated in this study. The proposed method can offer millimeter-level tumor motion prediction in real time, which has the potential to be adopted into routine tumor motion management in radiation therapy.

Motion in nuclear cardiology imaging: types, artifacts, detection and correction techniques

In this paper, the authors review the field of motion detection and correction in nuclear cardiology with single photon emission computed tomography (SPECT) and positron emission tomography (PET) imaging systems. We start with a brief overview of nuclear cardiology applications and description of SPECT and PET imaging systems, then explaining the different types of motion and their related artefacts. Moreover, we classify and describe various techniques for motion detection and correction, discussing their potential advantages including reference to metrics and tasks, particularly towards improvements in image quality and diagnostic performance. In addition, we emphasize limitations encountered in different motion detection and correction methods that may challenge routine clinical applications and diagnostic performance.

Multi-channel respiratory signal detection system for 4D-CT in radiotherapy by measuring the back pressure

This study proposes a novel respiratory signal detection system for 4D-CT in radiotherapy by measuring back pressure changes at multiple positions on CT couch. The 12-channel pressure sensor is fixed on CT couch to obtain patient's back pressure signal. The 12-channel signal is transmitted to a PC at a sampling rate of 50 Hz after a signal conditioning circuit and an analog-digital converter. The amplitude of pressure changes is characterized to select the optimal channel. This system is validated by comparing with the respiratory signal collected synchronously with a real-time position management (RPM) system on 10 healthy volunteers. The correlation coefficient between the signals is 0.82 ± 0.09 (standard deviation) and the time shift is 0.32 ± 0.15 second. We conclude that the back pressure signal acquired by the proposed system has the potential to replace the clinical RPM system for respiratory signal detection in 4D-CT data acquisition.

Intrafractional motion models based on principal components in Magnetic Resonance guided prostate radiotherapy

Background and purpose

Devices that combine an MR-scanner with a Linac for radiotherapy, referred to as MR-Linac systems, introduce the possibility to acquire high resolution images prior and during treatment. Hence, there is a possibility to acquire individualised learning sets for motion models for each fraction and the construction of intrafractional motion models. We investigated the feasibility for a principal component analysis (PCA) based, intrafractional motion model of the male pelvic region.

Materials and methods

4D-scans of nine healthy male volunteers were utilized, FOV covering the entire pelvic region including prostate, bladder and rectum with manual segmentation of each organ at each time frame. Deformable image registration with an optical flow algorithm was performed for each subject with the first time frame as reference. PCA was performed on a subset of the resulting displacement vector fields to construct individualised motion models evaluated on the remaining fields.

Results

The registration algorithm produced accurate registration result, in general DICE overlap >0.95 across all time frames. Cumulative variance of the eigen values from the PCA showed that 50% or more of the motion is explained in the first component for all subjects. However, the size and direction for the components differed between subjects. Adding more than two components did not improve the accuracy significantly and the model was able to explain motion down to about 1 mm.

Conclusions

An individualised intrafractional male pelvic motion model is feasible. Geometric accuracy was about 1 mm based on 1–2 principal components.

Population-based 3D respiratory motion modelling from convolutional autoencoders for 2D ultrasound-guided radiotherapy

Radiotherapy is a widely used treatment modality for various types of cancers. A challenge for precise delivery of radiation to the treatment site is the management of internal motion caused by the patient's breathing, especially around abdominal organs such as the liver. Current image-guided radiation therapy (IGRT) solutions rely on ionising imaging modalities such as X-ray or CBCT, which do not allow real-time target tracking. Ultrasound imaging (US) on the other hand is relatively inexpensive, portable and non-ionising. Although 2D US can be acquired at a sufficient temporal frequency, it doesn't allow for target tracking in multiple planes, while 3D US acquisitions are not adapted for real-time. In this work, a novel deep learning-based motion modelling framework is presented for ultrasound IGRT. Our solution includes an image similarity-based rigid alignment module combined with a deep deformable motion model. Leveraging the representational capabilities of convolutional autoencoders, our deformable motion model associates complex 3D deformations with 2D surrogate US images through a common learned low dimensional representation. The model is trained on a variety of deformations and anatomies which enables it to generate the 3D motion experienced by the liver of a previously unseen subject. During inference, our framework only requires two pre-treatment 3D volumes of the liver at extreme breathing phases and a live 2D surrogate image representing the current state of the organ. In this study, the presented model is evaluated on a 3D+t US data set of 20 volunteers based on image similarity as well as anatomical target tracking performance. We report results that surpass comparable methodologies in both metric categories with a mean tracking error of 3.5±2.4 mm, demonstrating the potential of this technique for IGRT.

Probabilistic 4D predictive model from in-room surrogates using conditional generative networks for image-guided radiotherapy

Shape and location organ variability induced by respiration constitutes one of the main challenges during dose delivery in radiotherapy. Providing up-to-date volumetric information during treatment can improve tumor tracking, thereby increasing treatment efficiency and reducing damage to healthy tissue. We propose a novel probabilistic model to address the problem of volumetric estimation with scalable predictive horizon from image-based surrogates during radiotherapy treatments, thus enabling out-of-plane tracking of targets. This problem is formulated as a conditional learning task, where the predictive variables are the 2D surrogate images and a pre-operative static 3D volume. The model learns a distribution of realistic motion fields over a population dataset. Simultaneously, a seq-2-seq inspired temporal mechanism acts over the surrogate images yielding extrapolated-in-time representations. The phase-specific motion distributions are associated with the predicted temporal representations, allowing the recovery of dense organ deformation in multiple times. Due to its generative nature, this model enables uncertainty estimations by sampling the latent space multiple times. Furthermore, it can be readily personalized to a new subject via fine-tuning, and does not require inter-subject correspondences. The proposed model was evaluated on free-breathing 4D MRI and ultrasound datasets from 25 healthy volunteers, as well as on 11 cancer patients. A navigator-based data augmentation strategy was used during the slice reordering process to increase model robustness against inter-cycle variability. The patient data was used as a hold-out test set. Our approach yields volumetric prediction from image surrogates with a mean error of 1.67 ± 1.68 mm and 2.17 ± 0.82 mm in unseen cases of the patient MRI and US datasets, respectively. Moreover, model personalization yields a mean landmark error of 1.4 ± 1.1 mm compared to ground truth annotations in the volunteer MRI dataset, with statistically significant improvements over state-of-the-art.

Generalized low-rank nonrigid motion-corrected reconstruction for MR fingerprinting

PURPOSE

Develop a novel low-rank motion-corrected (LRMC) reconstruction for nonrigid motion-corrected MR fingerprinting (MRF).

METHODS

Generalized motion-corrected (MC) reconstructions have been developed for steady-state imaging. Here we extend this framework to enable nonrigid MC for transient imaging applications with varying contrast, such as MRF. This is achieved by integrating low-rank dictionary-based compression into the generalized MC model to reconstruct MC singular images, reducing motion artifacts in the resulting parametric maps. The proposed LRMC reconstruction was applied for cardiac motion correction in 2D myocardial MRF (T₁ and T₂ ) with extended cardiac acquisition window (~450 ms) and for respiratory MC in free-breathing 3D myocardial and 3D liver MRF. Experiments were performed in phantom and 22 healthy subjects. The proposed approach was compared with reference spin echo (phantom) and with 2D electrocardiogram-triggered/breath-hold MOLLI and T₂ gradient-and-spin echo conventional maps (in vivo 2D and 3D myocardial MRF).

RESULTS

Phantom results were in general agreement with reference spin-echo measurements, presenting relative errors of approximately 5.4% and 5.5% for T₁ and short T₂ (<100 ms), respectively. The proposed LRMC MRF reduced residual blurring artifacts with respect to no MC for cardiac or respiratory motion in all cases (2D and 3D myocardial, 3D abdominal). In 2D myocardial MRF, left-ventricle T₁ values were 1150 ± 41 ms for LRMC MRF and 1010 ± 56 ms for MOLLI; T₂ values were 43.8 ± 2.3 ms for LRMC MRF and 49.5 ± 4.5 ms for T₂ gradient and spin echo. Corresponding measurements for 3D myocardial MRF were 1085 ± 30 ms and 1062 ± 29 ms for T₁ , and 43.5 ± 1.9 ms and 51.7 ± 1.7 ms for T₂ . For 3D liver, LRMC MRF measured liver T₁ at 565 ± 44 ms and liver T₂ at 35.4 ± 2.4 ms.

CONCLUSION

The proposed LRMC reconstruction enabled generalized (nonrigid) MC for 2D and 3D MRF, both for cardiac and respiratory motion. The proposed approach reduced motion artifacts in the MRF maps with respect to no motion compensation and achieved good agreement with reference measurements.