TranSMS: Transformers for Super-Resolution Calibration in Magnetic Particle Imaging

RETNet: Resolution enhancement Transformer network for magnetic particle imaging based on X-space

Magnetic Particle Imaging (MPI) can visualize the concentration distribution of superparamagnetic iron-oxide nanoparticles (SPIONs) in tissues with the advantages of high sensitivity and high temporal resolution. However, the low spatial resolution of MPI limits its application. Increasing the gradient strength of the selection field can improve the resolution of MPI, but also increase power consumption and noise. A feasible and cost-effective method to address this limitation is to reconstruct high gradient (HG) image from low gradient (LG) image using algorithms. Deep learning has been a powerful tool for improving the resolution of medical imaging techniques. In this study, we propose a Resolution Enhancement Transformer Network (RETNet) for reconstructing HG image with high-resolution from LG image with low-resolution as input, avoiding high power consumption and high noise in the system with HG field. RETNet leverages a shallow feature extractor to capture shallow features, a cross-scale-Transformer (CST) to focus on textural features, a residual-swin-Transformer (RST) to focus on structural features, and an image reconstruction module to aggregate these three types of features and reconstruct the HG image. Textural and structural features extracted can ensure the integrity of the details and the realization of high definition in the reconstructed image. Ablation experiments demonstrate the significant contribution of these two modules to reconstruct the HG image. Comparative experiments, including experiments at noise-free and multiple noise levels, confirm the high robustness of RETNet. Simulation, phantom, and in vivo experiments consistently demonstrate that RETNet outperforms competing methods and effectively improves the resolution of MPI.

O-PRESS: Boosting OCT axial resolution with Prior guidance, Recurrence, and Equivariant Self-Supervision

Optical coherence tomography (OCT) is a noninvasive technology that enables real-time imaging of tissue microanatomies. The axial resolution of OCT is intrinsically constrained by the spectral bandwidth of the employed light source while maintaining a fixed center wavelength for a specific application. Physically extending this bandwidth faces strong limitations and requires a substantial cost. We present a novel computational approach, called as O-PRESS, for boosting the axial resolution of OCT with Prior guidance, a Recurrent mechanism, and Equivariant Self-Supervision. Diverging from conventional deconvolution methods that rely on physical models or data-driven techniques, our method seamlessly integrates OCT modeling and deep learning, enabling us to achieve real-time axial-resolution enhancement exclusively from measurements without a need for paired images. Our approach solves two primary tasks of resolution enhancement and noise reduction with one treatment. Both tasks are executed in a self-supervised manner, with equivariance imaging and free space priors guiding their respective processes. Experimental evaluations, encompassing both quantitative metrics and visual assessments, consistently verify the efficacy and superiority of our approach, which exhibits performance on par with fully supervised methods. Importantly, the robustness of our model is affirmed, showcasing its dual capability to enhance axial resolution while concurrently improving the signal-to-noise ratio.

Transformer for low concentration image denoising in magnetic particle imaging

Objective.Magnetic particle imaging (MPI) is an emerging tracer-basedin vivoimaging technology. The use of MPI at low superparamagnetic iron oxide nanoparticle concentrations has the potential to be a promising area of clinical application due to the inherent safety for humans. However, low tracer concentrations reduce the signal-to-noise ratio of the magnetization signal, leading to severe noise artifacts in the reconstructed MPI images. Hardware improvements have high complexity, while traditional methods lack robustness to different noise levels, making it difficult to improve the quality of low concentration MPI images.Approach.Here, we propose a novel deep learning method for MPI image denoising and quality enhancing based on a sparse lightweight transformer model. The proposed residual-local transformer structure reduces model complexity to avoid overfitting, in which an information retention block facilitates feature extraction capabilities for the image details. Besides, we design a noisy concentration dataset to train our model. Then, we evaluate our method with both simulated and real MPI image data.Main results.Simulation experiment results show that our method can achieve the best performance compared with the existing deep learning methods for MPI image denoising. More importantly, our method is effectively performed on the real MPI image of samples with an Fe concentration down to 67μgFeml-1.Significance.Our method provides great potential for obtaining high quality MPI images at low concentrations.

MPIGAN: An end-to-end deep based generative framework for high-resolution magnetic particle imaging reconstruction

BACKGROUND

Magnetic particle imaging (MPI) is a recently developed, non-invasive in vivo imaging technique to map the spatial distribution of superparamagnetic iron oxide nanoparticles (SPIONs) in animal tissues with high sensitivity and speed. It is a challenge to reconstruct images directly from the received signals of MPI device due to the complex physical behavior of the nanoparticles. System matrix and X-space are two commonly used MPI reconstruction methods, where the former is extremely time-consuming and the latter usually produces blurry images.

PURPOSE

Currently, we proposed an end-to-end machine learning framework to reconstruct high-resolution MPI images from 1-D voltage signals directly and efficiently.

METHODS

The proposed framework, which we termed "MPIGAN", was trained on a large MPI simulation dataset containing 291 597 pairs of high-resolution 2-D phantom images and each image's corresponding voltage signals, so that it was able to accurately capture the nonlinear relationship between the spatial distribution of SPIONs and the received voltage signal, and realized high-resolution MPI image reconstruction.

RESULTS

Experiment results showed that, MPIGAN exhibited remarkable abilities in high-resolution MPI image reconstruction. MPIGAN outperformed the traditional methods of system matrix and X-space in recovering the fine-scale structure of magnetic nanoparticles' spatial distribution and achieving enhanced reconstruction performance in both visual effects and quantitative assessments. Moreover, even when the received signals were severely contaminated with noise, MPIGAN could still generate high-quality MPI images.

CONCLUSION

Our study provides a promising AI solution for end-to-end, efficient, and high-resolution magnetic particle imaging reconstruction.

Accurate Concentration Recovery for Quantitative Magnetic Particle Imaging Reconstruction via Nonconvex Regularization

Magnetic particle imaging (MPI) uses nonlinear response signals to noninvasively detect magnetic nanoparticles in space, and its quantitative properties hold promise for future precise quantitative treatments. In reconstruction, the system matrix based method necessitates suitable regularization terms, such as Tikhonov or non-negative fused lasso (NFL) regularization, to stabilize the solution. While NFL regularization offers clearer edge information than Tikhonov regularization, it carries a biased estimate of the l1 penalty, leading to an underestimation of the reconstructed concentration and adversely affecting the quantitative properties. In this paper, a new nonconvex regularization method including min-max concave (MC) and total variation (TV) regularization is proposed. This method utilized MC penalty to provide nearly unbiased sparse constraints and adds the TV penalty to provide a uniform intensity distribution of images. By combining the alternating direction multiplication method (ADMM) and the two-step parameter selection method, a more accurate quantitative MPI reconstruction was realized. The performance of the proposed method was verified on the simulation data, the Open-MPI dataset, and measured data from a homemade MPI scanner. The results indicate that the proposed method achieves better image quality while maintaining the quantitative properties, thus overcoming the drawback of intensity underestimation by the NFL method while providing edge information. In particular, for the measured data, the proposed method reduced the relative error in the intensity of the reconstruction results from 28% to 8%.

Dual-space high-frequency learning for transformer-based MRI super-resolution

BACKGROUND AND OBJECTIVE

Magnetic resonance imaging (MRI) can provide rich and detailed high-contrast information of soft tissues, while the scanning of MRI is time-consuming. To accelerate MR imaging, a variety of Transformer-based single image super-resolution methods are proposed in recent years, achieving promising results thanks to their superior capability of capturing long-range dependencies. Nevertheless, most existing works prioritize the design of transformer attention blocks to capture global information. The local high-frequency details, which are pivotal to faithful MRI restoration, are unfortunately neglected.

METHODS

In this work, we propose a high-frequency enhanced learning scheme to effectively improve the awareness of high frequency information in current Transformer-based MRI single image super-resolution methods. Specifically, we present two entirely plug-and-play modules designed to equip Transformer-based networks with the ability to recover high-frequency details from dual spaces: 1) in the feature space, we design a high-frequency block (Hi-Fe block) paralleled with Transformer-based attention layers to extract rich high-frequency features; while 2) in the image intensity space, we tailor a high-frequency amplification module (HFA) to further refine the high-frequency details. By fully exploiting the merits of the two modules, our framework can recover abundant and diverse high-frequency information, rendering faithful MRI super-resolved results with fine details.

RESULTS

We integrated our modules with six Transformer-based models and conducted experiments across three datasets. The results indicate that our plug-and-play modules can enhance the super-resolution performance of all foundational models to varying degrees, surpassing the capabilities of existing state-of-the-art single image super-resolution networks.

CONCLUSION

Comprehensive comparison of super-resolution images and high-frequency maps from various methods, clearly demonstrating that our module possesses the capability to restore high-frequency information, showing huge potential in clinical practice for accelerated MRI reconstruction.

A systematic 3-D magnetic particle imaging simulation model for quantitative analysis of reconstruction image quality

BACKGROUND AND OBJECTIVE

Magnetic particle imaging (MPI) is an emerging imaging technology in medical tomography that utilizes the nonlinear magnetization response of superparamagnetic iron oxide (SPIO) particles to determine the in vivo spatial distribution of nanoparticle contrast agents. The reconstruction image quality of MPI is determined by the characteristics of magnetic particles, the setting of the MPI scanner parameters, and the hardware interference of MPI systems. We explore a feasible method to systematically and quickly analyze the impact of these factors on MPI reconstruction image quality.

METHODS

We propose a systematic 3-D MPI simulation model. The MPI simulation model has the capability of quickly producing the simulated reconstruction images of a scanned phantom, and quantitative analysis of MPI reconstruction image quality can be achieved by comparing the differences between the input image and output image. These factors are mainly classified as imaging parameters and interference parameters in our model. In order to reduce the computational time of the simulation model, we introduce GPU parallel programming to accelerate the processing of large complex matrix data. For ease of use, we also construct a reliable, high-performance, and open-source 3-D MPI simulation software tool based on our model. The efficiency of our model is evaluated by using OpenMPIData. To demonstrate the capabilities of our model, we conduct simulation experiments using parameters consistent with a real MPI scanner for improving MPI image quality.

RESULTS

The experimental results show that our simulation model can systematically and quickly evaluate the impact of imaging parameters and interference parameters on MPI reconstruction image quality.

CONCLUSIONS

We developed an easy-to-use and open-source 3-D MPI simulation software tool based on our simulation model incorporating all the stages of MPI formation, from signal acquisition to image reconstruction. In the future, our simulation model has potential guiding significance to practical MPI images.

One model to unite them all: Personalized federated learning of multi-contrast MRI synthesis

Curation of large, diverse MRI datasets via multi-institutional collaborations can help improve learning of generalizable synthesis models that reliably translate source- onto target-contrast images. To facilitate collaborations, federated learning (FL) adopts decentralized model training while mitigating privacy concerns by avoiding sharing of imaging data. However, conventional FL methods can be impaired by the inherent heterogeneity in the data distribution, with domain shifts evident within and across imaging sites. Here we introduce the first personalized FL method for MRI Synthesis (pFLSynth) that improves reliability against data heterogeneity via model specialization to individual sites and synthesis tasks (i.e., source-target contrasts). To do this, pFLSynth leverages an adversarial model equipped with novel personalization blocks that control the statistics of generated feature maps across the spatial/channel dimensions, given latent variables specific to sites and tasks. To further promote communication efficiency and site specialization, partial network aggregation is employed over later generator stages while earlier generator stages and the discriminator are trained locally. As such, pFLSynth enables multi-task training of multi-site synthesis models with high generalization performance across sites and tasks. Comprehensive experiments demonstrate the superior performance and reliability of pFLSynth in MRI synthesis against prior federated methods.

SCANeXt: Enhancing 3D medical image segmentation with dual attention network and depth-wise convolution

Existing approaches to 3D medical image segmentation can be generally categorized into convolution-based or transformer-based methods. While convolutional neural networks (CNNs) demonstrate proficiency in extracting local features, they encounter challenges in capturing global representations. In contrast, the consecutive self-attention modules present in vision transformers excel at capturing long-range dependencies and achieving an expanded receptive field. In this paper, we propose a novel approach, termed SCANeXt, for 3D medical image segmentation. Our method combines the strengths of dual attention (Spatial and Channel Attention) and ConvNeXt to enhance representation learning for 3D medical images. In particular, we propose a novel self-attention mechanism crafted to encompass spatial and channel relationships throughout the entire feature dimension. To further extract multiscale features, we introduce a depth-wise convolution block inspired by ConvNeXt after the dual attention block. Extensive evaluations on three benchmark datasets, namely Synapse, BraTS, and ACDC, demonstrate the effectiveness of our proposed method in terms of accuracy. Our SCANeXt model achieves a state-of-the-art result with a Dice Similarity Score of 95.18% on the ACDC dataset, significantly outperforming current methods.

Fusion Networks of CNN and Transformer with Channel Attention for Accurate Tumor Imaging in Magnetic Particle Imaging

BACKGROUND

Magnetic Particle Imaging (MPI) is an emerging molecular imaging technique. However, since X-space reconstruction ignores system properties, it can lead to blurring of the reconstructed image, posing challenges for accurate quantification. To address this issue, we propose the use of deep learning to remove the blurry artifacts; (2) Methods: Our network architecture consists of a combination of Convolutional Neural Network (CNN) and Transformer. The CNN utilizes convolutional layers to automatically extract pixel-level local features and reduces the size of feature maps through pooling layers, effectively capturing local information within the images. The Transformer module is responsible for extracting contextual features from the images and efficiently capturing long-range dependencies, enabling a more effective modeling of global features in the images. By combining the features extracted by both CNN and Transformer, we capture both global and local features simultaneously, thereby improving the quality of reconstructed images; (3) Results: Experimental results demonstrate that the network effectively removes blurry artifacts from the images, and it exhibits high accuracy in precise tumor quantification. The proposed method shows superior performance over the state-of-the-art methods; (4) Conclusions: This bears significant implications for the image quality improvement and clinical application of MPI technology.

Unsupervised Medical Image Translation With Adversarial Diffusion Models

Imputation of missing images via source-to-target modality translation can improve diversity in medical imaging protocols. A pervasive approach for synthesizing target images involves one-shot mapping through generative adversarial networks (GAN). Yet, GAN models that implicitly characterize the image distribution can suffer from limited sample fidelity. Here, we propose a novel method based on adversarial diffusion modeling, SynDiff, for improved performance in medical image translation. To capture a direct correlate of the image distribution, SynDiff leverages a conditional diffusion process that progressively maps noise and source images onto the target image. For fast and accurate image sampling during inference, large diffusion steps are taken with adversarial projections in the reverse diffusion direction. To enable training on unpaired datasets, a cycle-consistent architecture is devised with coupled diffusive and non-diffusive modules that bilaterally translate between two modalities. Extensive assessments are reported on the utility of SynDiff against competing GAN and diffusion models in multi-contrast MRI and MRI-CT translation. Our demonstrations indicate that SynDiff offers quantitatively and qualitatively superior performance against competing baselines.

Progressive Pretraining Network for 3D System Matrix Calibration in Magnetic Particle Imaging

Magnetic particle imaging (MPI) is an emerging technique for determining magnetic nanoparticle distributions in biological tissues. Although system-matrix (SM)-based image reconstruction offers higher image quality than the X-space-based approach, the SM calibration measurement is time-consuming. Additionally, the SM should be recalibrated if the tracer's characteristics or the magnetic field environment change, and repeated SM measurement further increase the required labor and time. Therefore, fast SM calibration is essential for MPI. Existing calibration methods commonly treat each row of the SM as independent of the others, but the rows are inherently related through the coil channel and frequency index. As these two elements can be regarded as additional multimodal information, we leverage the transformer architecture with a self-attention mechanism to encode them. Although the transformer has shown superiority in multimodal fusion learning across several fields, its high complexity may lead to overfitting when labeled data are scarce. Compared with labeled SM (i.e., full size), low-resolution SM data can be easily obtained, and fully using such data may alleviate overfitting. Accordingly, we propose a pseudo-label-based progressive pretraining strategy to leverage unlabeled data. Our method outperforms existing calibration methods on a public real-world OpenMPI dataset and simulation dataset. Moreover, our method improves the resolution of two in-house MPI scanners without requiring full-size SM measurements. Ablation studies confirm the contributions of modeling SM inter-row relations and the proposed pretraining strategy.

Magnetic particle imaging deblurring with dual contrastive learning and adversarial framework

Magnetic particle imaging (MPI) is an emerging medical imaging technique that has high sensitivity, contrast, and excellent depth penetration. In MPI, x-space is a reconstruction method that transforms the measured voltages into particle concentrations. The reconstructed native image can be modeled as a convolution of the magnetic particle concentration with a point-spread function (PSF). The PSF is one of the important parameters in deconvolution. However, accurately measuring or modeling the PSF in the hardware used for deconvolution is challenging due to the various environment and magnetic particle relaxation. The inaccurate PSF estimation may lead to the loss of the content structure of the MPI image, especially in low gradient fields. In this study, we developed a Dual Adversarial Network (DAN) with patch-wise contrastive constraint to deblur the MPI image. This method can overcome the limitations of unpaired data in data acquisition scenarios and remove the blur around the boundary more effectively than the common deconvolution method. We evaluated the performance of the proposed DAN model on simulated and real data. Experimental results confirmed that our model performs favorably against the deconvolution method that is mainly used for deblurring the MPI image and other GAN-based deep learning models.

A transformer-based hierarchical registration framework for multimodality deformable image registration

Deformable image registration (DIR) between daily and reference images is fundamentally important for adaptive radiotherapy. In the last decade, deep learning-based image registration methods have been developed with faster computation time and improved robustness compared to traditional methods. However, the registration performance is often degraded in extra-cranial sites with large volume containing multiple anatomic regions, such as Computed Tomography (CT)/Magnetic Resonance (MR) images used in head and neck (HN) radiotherapy. In this study, we developed a hierarchical deformable image registration (DIR) framework, Patch-based Registration Network (Patch-RegNet), to improve the accuracy and speed of CT-MR and MR-MR registration for head-and-neck MR-Linac treatments. Patch-RegNet includes three steps: a whole volume global registration, a patch-based local registration, and a patch-based deformable registration. Following a whole-volume rigid registration, the input images were divided into overlapping patches. Then a patch-based rigid registration was applied to achieve accurate local alignment for subsequent DIR. We developed a ViT-Morph model, a combination of a convolutional neural network (CNN) and the Vision Transformer (ViT), for the patch-based DIR. A modality independent neighborhood descriptor was adopted in our model as the similarity metric to account for both inter-modality and intra-modality registration. The CT-MR and MR-MR DIR models were trained with 242 CT-MR and 213 MR-MR image pairs from 36 patients, respectively, and both tested with 24 image pairs (CT-MR and MR-MR) from 6 other patients. The registration performance was evaluated with 7 manually contoured organs (brainstem, spinal cord, mandible, left/right parotids, left/right submandibular glands) by comparing with the traditional registration methods in Monaco treatment planning system and the popular deep learning-based DIR framework, Voxelmorph. Evaluation results show that our method outperformed VoxelMorph by 6 % for CT-MR registration, and 4 % for MR-MR registration based on DSC measurements. Our hierarchical registration framework has been demonstrated achieving significantly improved DIR accuracy of both CT-MR and MR-MR registration for head-and-neck MR-guided adaptive radiotherapy.

Super-resolution of magnetic systems using deep learning

We construct a deep neural network to enhance the resolution of spin structure images formed by spontaneous symmetry breaking in the magnetic systems. Through the deep neural network, an image is expanded to a super-resolution image and reduced to the original image size to be fitted with the input feed image. The network does not require ground truth images in the training process. Therefore, it can be applied when low-resolution images are provided as training datasets, while high-resolution images are not obtainable due to the intrinsic limitation of microscope techniques. To show the usefulness of the network, we train the network with two types of simulated magnetic structure images; one is from self-organized maze patterns made of chiral magnetic structures, and the other is from magnetic domains separated by walls that are topological defects of the system. The network successfully generates high-resolution images highly correlated with the exact solutions in both cases. To investigate the effectiveness and the differences between datasets, we study the network's noise tolerance and compare the networks' reliabilities. The network is applied with experimental data obtained by magneto-optical Kerr effect microscopy and spin-polarized low-energy electron microscopy.

Self-supervised Signal Denoising for Magnetic Particle Imaging

Magnetic particle imaging (MPI) is a medical imaging technology with high resolution and high sensitivity, which tracks the distribution of superparamagnetic iron oxide nanoparticles (SPIONs) in the nonlinear response to dynamic excitation at a field-free region. However, various noises distort the signals resulting in a decline in imaging quality. Traditional threshold-based methods cannot remove dynamic noise in MPI signals. Therefore, a self-supervised denoising method is proposed to denoise MPI signals in this study. The approach adopted U-net as the backbone and modified the network for MPI signals. The network is trained using two periods of noisy signals and the shape prior knowledge of the MPI signals is introduced for promoting the convergence of the self-supervised net. The experiments show that the learning-based method can still denoising the MPI signal without labeling data and eventually improve image quality, and our approach can achieve the best performance compared with other self-supervised methods in MPI signal denoising.

Adaptive diffusion priors for accelerated MRI reconstruction

Deep MRI reconstruction is commonly performed with conditional models that de-alias undersampled acquisitions to recover images consistent with fully-sampled data. Since conditional models are trained with knowledge of the imaging operator, they can show poor generalization across variable operators. Unconditional models instead learn generative image priors decoupled from the operator to improve reliability against domain shifts related to the imaging operator. Recent diffusion models are particularly promising given their high sample fidelity. Nevertheless, inference with a static image prior can perform suboptimally. Here we propose the first adaptive diffusion prior for MRI reconstruction, AdaDiff, to improve performance and reliability against domain shifts. AdaDiff leverages an efficient diffusion prior trained via adversarial mapping over large reverse diffusion steps. A two-phase reconstruction is executed following training: a rapid-diffusion phase that produces an initial reconstruction with the trained prior, and an adaptation phase that further refines the result by updating the prior to minimize data-consistency loss. Demonstrations on multi-contrast brain MRI clearly indicate that AdaDiff outperforms competing conditional and unconditional methods under domain shifts, and achieves superior or on par within-domain performance.

Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI

The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this challenge, we propose a hierarchical perception adversarial learning framework (HP-ALF). HP-ALF can perceive the image information in the hierarchical mechanism: image-level perception and patch-level perception. The former can reduce the visual perception difference in the entire image, and thus achieve aliasing artifact removal. The latter can reduce this difference in the regions of the image, and thus recover fine details. Specifically, HP-ALF achieves the hierarchical mechanism by utilizing multilevel perspective discrimination. This discrimination can provide the information from two perspectives (overall and regional) for adversarial learning. It also utilizes a global and local coherent discriminator to provide structure information to the generator during training. In addition, HP-ALF contains a context-aware learning block to effectively exploit the slice information between individual images for better reconstruction performance. The experiments validated on three datasets demonstrate the effectiveness of HP-ALF and its superiority to the comparative methods.

BolT: Fused window transformers for fMRI time series analysis

Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time scales. Here, we present BolT, a blood-oxygen-level-dependent transformer model, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Encoding is performed on temporally-overlapped windows within the time series to capture local representations. To integrate information temporally, cross-window attention is computed between base tokens in each window and fringe tokens from neighboring windows. To gradually transition from local to global representations, the extent of window overlap and thereby number of fringe tokens are progressively increased across the cascade. Finally, a novel cross-window regularization is employed to align high-level classification features across the time series. Comprehensive experiments on large-scale public datasets demonstrate the superior performance of BolT against state-of-the-art methods. Furthermore, explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings in the literature.

Deep learning based MRI reconstruction with transformer

Magnetic resonance imaging (MRI) has become one of the most powerful imaging techniques in medical diagnosis, yet the prolonged scanning time becomes a bottleneck for application. Reconstruction methods based on compress sensing (CS) have made progress in reducing this cost by acquiring fewer points in the k-space. Traditional CS methods impose restrictions from different sparse domains to regularize the optimization that always requires balancing time with accuracy. Neural network techniques enable learning a better prior from sample pairs and generating the results in an analytic way. In this paper, we propose a deep learning based reconstruction method to restore high-quality MRI images from undersampled k-space data in an end-to-end style. Unlike prior literature adopting convolutional neural networks (CNN), advanced Swin Transformer is used as the backbone of our work, which proved to be powerful in extracting deep features of the image. In addition, we combined the k-space consistency in the output and further improved the quality. We compared our models with several reconstruction methods and variants, and the experiment results proved that our model achieves the best results in samples at low sampling rates. The source code of KTMR could be acquired at https://github.com/BITwzl/KTMR.

Extrapolation of System Matrices in Magnetic Particle Imaging

Magnetic particle imaging exploits the non-linear magnetization of superparamagnetic iron-oxide particles to generate a tomographic image in a defined field-of-view. For reconstruction of the particle distribution, a time-consuming calibration step is required, in which system matrices get measured using a robot. To achieve artifact-free images, system matrices need to cover not only the field-of-view but also a larger area around it. Especially for large measurements- inevitable for future clinical application- this leads to long calibration time and high consumption of persistent memory. In this work, we analyze the signal in the outer part of the system matrix and motivate the usage of extrapolation methods to computationally expand the system matrix after restricting the calibration to the field-of-view. We propose a suitable extrapolation method and show its applicability on measured 2D and 3D data. In doing so, we achieve a considerable reduction of calibration time and consumption of persistent memory while preserving an artifact-free result.

Cross-domain knowledge transfer based parallel-cascaded multi-scale attention network for limited view reconstruction in projection magnetic particle imaging

Projection magnetic particle imaging (MPI) can significantly improve the temporal resolution of three-dimensional (3D) imaging compared to that using traditional point by point scanning. However, the dense view of projections required for tomographic reconstruction limits the scope of temporal resolution optimization. The solution to this problem in computed tomography (CT) is using limited view projections (sparse view or limited angle) for reconstruction, which can be divided into: completing the limited view sinogram and image post-processing for streaking artifacts caused by insufficient projections. Benefiting from large-scale CT datasets, both categories of deep learning-based methods have achieved tremendous progress; yet, there is a data scarcity limitation in MPI. We propose a cross-domain knowledge transfer learning strategy that can transfer the prior knowledge of the limited view learned by the model in CT to MPI, which can help reduce the network requirements for real MPI data. In addition, the size of the imaging target affects the scale of the streaking artifacts caused by insufficient projections. Therefore, we propose a parallel-cascaded multi-scale attention module that allows the network to adaptively identify streaking artifacts at different scales. The proposed method was evaluated on real phantom and in vivo mouse data, and it significantly outperformed several advanced limited view methods. The streaking artifacts caused by an insufficient number of projections can be overcome using the proposed method.

Anisotropic edge-preserving network for resolution enhancement in unidirectional Cartesian magnetic particle imaging

Objective. Magnetic particle imaging (MPI) is a novel imaging modality. It is crucial to acquire accurate localization of the superparamagnetic iron oxide nanoparticles distributions in MPI. However, the spatial resolution of unidirectional Cartesian trajectory MPI exhibits anisotropy, which blurs the boundaries of MPI images and makes precise localization difficult. In this paper, we propose an anisotropic edge-preserving network (AEP-net) to alleviate the anisotropic resolution of MPI.Methods. AEP-net resolve the resolution anisotropy by constructing an asymmertic convolution. To recover the edge information, we design the uncertainty region module. In addition, we evaluated the performance of the proposed AEP-net model by using simulations and experimental data.Results. The results show that the AEP-net model alleviates the anisotropy of the unidirectional Cartesian trajectory and preserves edge details in the MPI image. By comparing the visualization results and the metrics, we demonstrate that our method is superior to other methods.Significance. The proposed method produces accurate visualization in unidirectional Cartesian devices and promotes accurate quantization, which promote the biomedical applications using MPI.

System matrix recovery based on deep image prior in magnetic particle imaging

Objective.Magnetic particle imaging (MPI) is an emerging tomography imaging technique with high specificity and temporal-spatial resolution. MPI reconstruction based on the system matrix (SM) is an important research content in MPI. However, SM is usually obtained by measuring the response of an MPI scanner at all positions in the field of view. This process is very time-consuming, and the scanner will overheat in a long period of continuous operation, which is easy to generate thermal noise and affects MPI imaging performance.Approach.In this study, we propose a deep image prior-based method that prominently decreases the time of SM calibration. It is an unsupervised method that utilizes the neural network structure itself to recover a high-resolution SM from a downsampled SM without the need to train the network using a large amount of training data.Main results.Experiments on the Open MPI data show that the time of SM calibration can be greatly reduced with only slight degradation of image quality.Significance.This study provides a novel method for obtaining SM in MPI, which shows the potential to achieve SM recovery at a high downsampling rate. It is expected that this study will increase the practicability of MPI in biomedical applications and promote the development of MPI in the future.