Joint synthesis and registration network for deformable MR-CBCT image registration for neurosurgical guidance

Vessel-targeted compensation of deformable motion in interventional cone-beam CT

The present standard of care for unresectable liver cancer is transarterial chemoembolization (TACE), which involves using chemotherapeutic particles to selectively embolize the arteries supplying hepatic tumors. Accurate volumetric identification of intricate fine vascularity is crucial for selective embolization. Three-dimensional imaging, particularly cone-beam CT (CBCT), aids in visualization and targeting of small vessels in such highly variable anatomy, but long image acquisition time results in intra-scan patient motion, which distorts vascular structures and tissue boundaries. To improve clarity of vascular anatomy and intra-procedural utility, this work proposes a targeted motion estimation and compensation framework that removes the need for any prior information or external tracking and for user interaction. Motion estimation is performed in two stages: (i) a target identification stage that segments arteries and catheters in the projection domain using a multi-view convolutional neural network to construct a coarse 3D vascular mask; and (ii) a targeted motion estimation stage that iteratively solves for the time-varying motion field via optimization of a vessel-enhancing objective function computed over the target vascular mask. The vessel-enhancing objective is derived through eigenvalues of the local image Hessian to emphasize bright tubular structures. Motion compensation is achieved via spatial transformer operators that apply time-dependent deformations to partial angle reconstructions, allowing efficient minimization via gradient backpropagation. The framework was trained and evaluated in anatomically realistic simulated motion-corrupted CBCTs mimicking TACE of hepatic tumors, at intermediate (3.0 mm) and large (6.0 mm) motion magnitudes. Motion compensation substantially improved median vascular DICE score (from 0.30 to 0.59 for large motion), image SSIM (from 0.77 to 0.93 for large motion), and vessel sharpness (0.189 mm-1 to 0.233 mm-1 for large motion) in simulated cases. Motion compensation also demonstrated increased vessel sharpness (0.188 mm-1 before to 0.205 mm-1 after) and reconstructed vessel length (median increased from 37.37 to 41.00 mm) on a clinical interventional CBCT. The proposed anatomy-aware motion compensation framework presented a promising approach for improving the utility of CBCT for intra-procedural vascular imaging, facilitating selective embolization procedures.

A unified generation-registration framework for improved MR-based CT synthesis in proton therapy

BACKGROUND

The use of magnetic resonance (MR) imaging for proton therapy treatment planning is gaining attention as a highly effective method for guidance. At the core of this approach is the generation of computed tomography (CT) images from MR scans. However, the critical issue in this process is accurately aligning the MR and CT images, a task that becomes particularly challenging in frequently moving body areas, such as the head-and-neck. Misalignments in these images can result in blurred synthetic CT (sCT) images, adversely affecting the precision and effectiveness of the treatment planning.

PURPOSE

This study introduces a novel network that cohesively unifies image generation and registration processes to enhance the quality and anatomical fidelity of sCTs derived from better-aligned MR images.

METHODS

The approach synergizes a generation network (G) with a deformable registration network (R), optimizing them jointly in MR-to-CT synthesis. This goal is achieved by alternately minimizing the discrepancies between the generated/registered CT images and their corresponding reference CT counterparts. The generation network employs a UNet architecture, while the registration network leverages an implicit neural representation (INR) of the displacement vector fields (DVFs). We validated this method on a dataset comprising 60 head-and-neck patients, reserving 12 cases for holdout testing.

RESULTS

Compared to the baseline Pix2Pix method with MAE 124.95 ± $\pm$ 30.74 HU, the proposed technique demonstrated 80.98 ± $\pm$ 7.55 HU. The unified translation-registration network produced sharper and more anatomically congruent outputs, showing superior efficacy in converting MR images to sCTs. Additionally, from a dosimetric perspective, the plan recalculated on the resulting sCTs resulted in a remarkably reduced discrepancy to the reference proton plans.

CONCLUSIONS

This study conclusively demonstrates that a holistic MR-based CT synthesis approach, integrating both image-to-image translation and deformable registration, significantly improves the precision and quality of sCT generation, particularly for the challenging body area with varied anatomic changes between corresponding MR and CT.

Deformable registration of preoperative MR and intraoperative long-length tomosynthesis images for guidance of spine surgery via image synthesis

PURPOSE

Improved integration and use of preoperative imaging during surgery hold significant potential for enhancing treatment planning and instrument guidance through surgical navigation. Despite its prevalent use in diagnostic settings, MR imaging is rarely used for navigation in spine surgery. This study aims to leverage MR imaging for intraoperative visualization of spine anatomy, particularly in cases where CT imaging is unavailable or when minimizing radiation exposure is essential, such as in pediatric surgery.

METHODS

This work presents a method for deformable 3D-2D registration of preoperative MR images with a novel intraoperative long-length tomosynthesis imaging modality (viz., Long-Film [LF]). A conditional generative adversarial network is used to translate MR images to an intermediate bone image suitable for registration, followed by a model-based 3D-2D registration algorithm to deformably map the synthesized images to LF images. The algorithm's performance was evaluated on cadaveric specimens with implanted markers and controlled deformation, and in clinical images of patients undergoing spine surgery as part of a large-scale clinical study on LF imaging.

RESULTS

The proposed method yielded a median 2D projection distance error of 2.0 mm (interquartile range [IQR]: 1.1-3.3 mm) and a 3D target registration error of 1.5 mm (IQR: 0.8-2.1 mm) in cadaver studies. Notably, the multi-scale approach exhibited significantly higher accuracy compared to rigid solutions and effectively managed the challenges posed by piecewise rigid spine deformation. The robustness and consistency of the method were evaluated on clinical images, yielding no outliers on vertebrae without surgical instrumentation and 3% outliers on vertebrae with instrumentation.

CONCLUSIONS

This work constitutes the first reported approach for deformable MR to LF registration based on deep image synthesis. The proposed framework provides access to the preoperative annotations and planning information during surgery and enables surgical navigation within the context of MR images and/or dual-plane LF images.

Structure-aware independently trained multi-scale registration network for cardiac images

Image registration is a primary task in various medical image analysis applications. However, cardiac image registration is difficult due to the large non-rigid deformation of the heart and the complex anatomical structure. This paper proposes a structure-aware independently trained multi-scale registration network (SIMReg) to address this challenge. Using image pairs of different resolutions, independently train each registration network to extract image features of large deformation image pairs at different resolutions. In the testing stage, the large deformation registration is decomposed into a multi-scale registration process, and the deformation fields of different resolutions are fused by a step-by-step deformation method, thus solving the difficulty of directly processing large deformation. Meanwhile, the targeted introduction of MIND (modality independent neighborhood descriptor) structural features to guide network training enhances the registration of cardiac structural contours and improves the registration effect of local details. Experiments were carried out on the open cardiac dataset ACDC (automated cardiac diagnosis challenge), and the average Dice value of the experimental results of the proposed method was 0.833. Comparative experiments showed that the proposed SIMReg could better solve the problem of heart image registration and achieve a better registration effect on cardiac images.

L2NLF: a novel linear-to-nonlinear framework for multi-modal medical image registration

In recent years, deep learning has ushered in significant development in medical image registration, and the method of non-rigid registration using deep neural networks to generate a deformation field has higher accuracy. However, unlike monomodal medical image registration, multimodal medical image registration is a more complex and challenging task. This paper proposes a new linear-to-nonlinear framework (L2NLF) for multimodal medical image registration. The first linear stage is essentially image conversion, which can reduce the difference between two images without changing the authenticity of medical images, thus transforming multimodal registration into monomodal registration. The second nonlinear stage is essentially unsupervised deformable registration based on the deep neural network. In this paper, a brand-new registration network, CrossMorph, is designed, a deep neural network similar to the U-net structure. As the backbone of the encoder, the volume CrossFormer block can better extract local and global information. Booster promotes the reduction of more deep features and shallow features. The qualitative and quantitative experimental results on T1 and T2 data of 240 patients' brains show that L2NLF can achieve excellent registration effect in the image conversion part with very low computation, and it will not change the authenticity of the converted image at all. Compared with the current state-of-the-art registration method, CrossMorph can effectively reduce average surface distance, improve dice score, and improve the deformation field's smoothness. The proposed methods have potential value in clinical application.

GAN-based metal artifacts region inpainting in brain MRI imaging with reflective registration

BACKGROUND AND OBJECTIVE

Metallic magnetic resonance imaging (MRI) implants can introduce magnetic field distortions, resulting in image distortion, such as bulk shifts and signal-loss artifacts. Metal Artifacts Region Inpainting Network (MARINet), using the symmetry of brain MRI images, has been developed to generate normal MRI images in the image domain and improve image quality.

METHODS

T1-weighted MRI images containing or located near the teeth of 100 patients were collected. A total of 9000 slices were obtained after data augmentation. Then, MARINet based on U-Net with a dual-path encoder was employed to inpaint the artifacts in MRI images. The input of MARINet contains the original image and the flipped registered image, with partial convolution used concurrently. Subsequently, we compared PConv with partial convolution, and GConv with gated convolution, SDEdit using a diffusion model for inpainting the artifact region of MRI images. The mean absolute error (MAE) and peak signal-to-noise ratio (PSNR) for the mask were used to compare the results of these methods. In addition, the artifact masks of clinical MRI images were inpainted by physicians.

RESULTS

MARINet could directly and effectively inpaint the incomplete MRI images generated by masks in the image domain. For the test results of PConv, GConv, SDEdit, and MARINet, the masked MAEs were 0.1938, 0.1904, 0.1876, and 0.1834, respectively, and the masked PSNRs were 17.39, 17.40, 17.49, and 17.60 dB, respectively. The visualization results also suggest that the network can recover the tissue texture, alveolar shape, and tooth contour. Additionally, for clinical artifact MRI images, MARINet completed the artifact region inpainting task more effectively when compared with other models.

CONCLUSIONS

By leveraging the quasi-symmetry of brain MRI images, MARINet can directly and effectively inpaint the metal artifacts in MRI images in the image domain, restoring the tooth contour and detail, thereby enhancing the image quality.

CT-CBCT deformable registration using weakly-supervised artifact-suppression transfer learning network

Objective.Computed tomography-cone-beam computed tomography (CT-CBCT) deformable registration has great potential in adaptive radiotherapy. It plays an important role in tumor tracking, secondary planning, accurate irradiation, and the protection of at-risk organs. Neural networks have been improving CT-CBCT deformable registration, and almost all registration algorithms based on neural networks rely on the gray values of both CT and CBCT. The gray value is a key factor in the loss function, parameter training, and final efficacy of the registration. Unfortunately, the scattering artifacts in CBCT affect the gray values of different pixels inconsistently. Therefore, the direct registration of the original CT-CBCT introduces artifact superposition loss.Approach. In this study, a histogram analysis method for the gray values was used. Based on an analysis of the gray value distribution characteristics of different regions in CT and CBCT, the degree of superposition of the artifact in the region of disinterest was found to be much higher than that in the region of interest. Moreover, the former was the main reason for artifact superposition loss. Consequently, a new weakly supervised two-stage transfer-learning network based on artifact suppression was proposed. The first stage was a pre-training network designed to suppress artifacts contained in the region of disinterest. The second stage was a convolutional neural network that registered the suppressed CBCT and CT.Main Results. Through a comparative test of the thoracic CT-CBCT deformable registration, whose data were collected from the Elekta XVI system, the rationality and accuracy after artifact suppression were confirmed to be significantly improved compared with the other algorithms without artifact suppression.Significance. This study proposed and verified a new deformable registration method with multi-stage neural networks, which can effectively suppress artifacts and further improve registration by incorporating a pre-training technique and an attention mechanism.