Deep learning for undersampled MRI reconstruction

Imaging error reduction in radial cine-MRI with deep learning-based intra-frame motion compensation

Objective.Radial cine-MRI allows for sliding window reconstruction at nearly arbitrary frame rate, promising high-speed imaging for intra-fractional motion monitoring in magnetic resonance guided radiotherapy. However, motion within the reconstruction window may determine the location of the reconstructed target to deviate from the true real-time position (target positioning errors), particularly in cases of fast breathing or for anatomical structures affected by the heartbeat. In this work, we present a proof-of-concept study aiming to enhance radial cine-MR imaging by implementing deep-learning-based intra-frame motion compensation techniques.Approach.A novel network (TransSin-UNet) was proposed to continuously estimate the final-position image of the target, corresponding to end of the frame acquisition. Within the radial k-space reconstruction window, the spatial-temporal dependencies among the sinogram representation of the spokes were modeled by a transformer encoder subnetwork, followed by a UNet subnetwork operating in the spatial domain for pixel-level fine-tuning. By simulating motion-dependent radial sampling with (tiny) golden angles, we generated datasets from 25 4D digital anthropomorphic lung cancer phantoms. The network was then trained and extensively evaluated across datasets characterized by varying azimuthal radial profile increments.Main Results.The method required additional 4.8 ms per frame over the conventional approach involving direct image reconstruction with motion-corrupted spokes. TransSin-UNet outperformed architectures relying solely on transformer encoders or UNets across all the comparative evaluations, leading to a noticeable enhancement in image quality and target positioning accuracy. The normalized root mean-squared error decreased by 50% from the initial value of 0.188 on average, whereas the mean Dice similarity coefficient of the gross tumor volume increased from 85.1% to 96.2% in the investigated cases. Furthermore, the final-positions of anatomical structures undergoing substantial intra-frame deformations were precisely derived.Significance.The proposed approach enables an effective intra-frame motion compensation, offering an opportunity to reduce errors in radial cine-MR imaging for real-time motion management.

A review of deep learning-based reconstruction methods for accelerated MRI using spatiotemporal and multi-contrast redundancies

Accelerated magnetic resonance imaging (MRI) has played an essential role in reducing data acquisition time for MRI. Acceleration can be achieved by acquiring fewer data points in k-space, which results in various artifacts in the image domain. Conventional reconstruction methods have resolved the artifacts by utilizing multi-coil information, but with limited robustness. Recently, numerous deep learning-based reconstruction methods have been developed, enabling outstanding reconstruction performances with higher acceleration. Advances in hardware and developments of specialized network architectures have produced such achievements. Besides, MRI signals contain various redundant information including multi-coil redundancy, multi-contrast redundancy, and spatiotemporal redundancy. Utilization of the redundant information combined with deep learning approaches allow not only higher acceleration, but also well-preserved details in the reconstructed images. Consequently, this review paper introduces the basic concepts of deep learning and conventional accelerated MRI reconstruction methods, followed by review of recent deep learning-based reconstruction methods that exploit various redundancies. Lastly, the paper concludes by discussing the challenges, limitations, and potential directions of future developments.

Assessing deep learning reconstruction for faster prostate MRI: visual vs. diagnostic performance metrics

OBJECTIVE

Deep learning (DL) MRI reconstruction enables fast scan acquisition with good visual quality, but the diagnostic impact is often not assessed because of large reader study requirements. This study used existing diagnostic DL to assess the diagnostic quality of reconstructed images.

MATERIALS AND METHODS

A retrospective multisite study of 1535 patients assessed biparametric prostate MRI between 2016 and 2020. Likely clinically significant prostate cancer (csPCa) lesions (PI-RADS ≥ 4) were delineated by expert radiologists. T2-weighted scans were retrospectively undersampled, simulating accelerated protocols. DL reconstruction (DLRecon) and diagnostic DL detection (DLDetect) were developed. The effect on the partial area under (pAUC), the Free-Response Operating Characteristic (FROC) curve, and the structural similarity (SSIM) were compared as metrics for diagnostic and visual quality, respectively. DLDetect was validated with a reader concordance analysis. Statistical analysis included Wilcoxon, permutation, and Cohen's kappa tests for visual quality, diagnostic performance, and reader concordance.

RESULTS

DLRecon improved visual quality at 4- and 8-fold (R4, R8) subsampling rates, with SSIM (range: -1 to 1) improved to 0.78 ± 0.02 (p < 0.001) and 0.67 ± 0.03 (p < 0.001) from 0.68 ± 0.03 and 0.51 ± 0.03, respectively. However, diagnostic performance at R4 showed a pAUC FROC of 1.33 (CI 1.28-1.39) for DL and 1.29 (CI 1.23-1.35) for naive reconstructions, both significantly lower than fully sampled pAUC of 1.58 (DL: p = 0.024, naïve: p = 0.02). Similar trends were noted for R8.

CONCLUSION

DL reconstruction produces visually appealing images but may reduce diagnostic accuracy. Incorporating diagnostic AI into the assessment framework offers a clinically relevant metric essential for adopting reconstruction models into clinical practice.

CLINICAL RELEVANCE STATEMENT

In clinical settings, caution is warranted when using DL reconstruction for MRI scans. While it recovered visual quality, it failed to match the prostate cancer detection rates observed in scans not subjected to acceleration and DL reconstruction.

A hybrid deep image prior and compressed sensing reconstruction method for highly accelerated 3D coronary magnetic resonance angiography

Introduction

High-resolution whole-heart coronary magnetic resonance angiography (CMRA) often suffers from unreasonably long scan times, rendering imaging acceleration highly desirable. Traditional reconstruction methods used in CMRA rely on either hand-crafted priors or supervised learning models. Although the latter often yield superior reconstruction quality, they require a large amount of training data and memory resources, and may encounter generalization issues when dealing with out-of-distribution datasets.

Methods

To address these challenges, we introduce an unsupervised reconstruction method that combines deep image prior (DIP) with compressed sensing (CS) to accelerate 3D CMRA. This method incorporates a slice-by-slice DIP reconstruction and 3D total variation (TV) regularization, enabling high-quality reconstruction under a significant acceleration while enforcing continuity in the slice direction. We evaluated our method by comparing it to iterative SENSE, CS-TV, CS-wavelet, and other DIP-based variants, using both retrospectively and prospectively undersampled datasets.

Results

The results demonstrate the superiority of our 3D DIP-CS approach, which improved the reconstruction accuracy relative to the other approaches across both datasets. Ablation studies further reveal the benefits of combining DIP with 3D TV regularization, which leads to significant improvements of image quality over pure DIP-based methods. Evaluation of vessel sharpness and image quality scores shows that DIP-CS improves the quality of reformatted coronary arteries.

Discussion

The proposed method enables scan-specific reconstruction of high-quality 3D CMRA from a five-minute acquisition, without relying on fully-sampled training data or placing a heavy burden on memory resources.

Enhancing global sensitivity and uncertainty quantification in medical image reconstruction with Monte Carlo arbitrary-masked mamba

Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has shown superiority in learning visual representation, which combines the advantages of linear scalability and global sensitivity. In this study, we introduce MambaMIR, an Arbitrary-Masked Mamba-based model with wavelet decomposition for joint medical image reconstruction and uncertainty estimation. A novel Arbitrary Scan Masking (ASM) mechanism "masks out" redundant information to introduce randomness for further uncertainty estimation. Compared to the commonly used Monte Carlo (MC) dropout, our proposed MC-ASM provides an uncertainty map without the need for hyperparameter tuning and mitigates the performance drop typically observed when applying dropout to low-level tasks. For further texture preservation and better perceptual quality, we employ the wavelet transformation into MambaMIR and explore its variant based on the Generative Adversarial Network, namely MambaMIR-GAN. Comprehensive experiments have been conducted for multiple representative medical image reconstruction tasks, demonstrating that the proposed MambaMIR and MambaMIR-GAN outperform other baseline and state-of-the-art methods in different reconstruction tasks, where MambaMIR achieves the best reconstruction fidelity and MambaMIR-GAN has the best perceptual quality. In addition, our MC-ASM provides uncertainty maps as an additional tool for clinicians, while mitigating the typical performance drop caused by the commonly used dropout.

MLMFNet: A multi-level modality fusion network for multi-modal accelerated MRI reconstruction

Magnetic resonance imaging produces detailed anatomical and physiological images of the human body that can be used in the clinical diagnosis and treatment of diseases. However, MRI suffers its comparatively longer acquisition time than other imaging methods and is thus vulnerable to motion artifacts, which ultimately lead to likely failed or even wrong diagnosis. In order to perform faster reconstruction, deep learning-based methods along with traditional strategies such as parallel imaging and compressed sensing come into play in recent years in this field. Meanwhile, in order to better analyze the diseases, it is also often necessary to acquire images in the same region of interest under different modalities, which yield images with different contrast levels. However, most of these aforementioned methods tend to use single-modal images for reconstruction, neglecting the correlation and redundancy information embedded in MR images acquired with different modalities. While there are works on multi-modal reconstruction, the information is yet to be efficiently explored. In this paper, we propose an end-to-end neural network called MLMFNet, which helps the reconstruction of the target modality by using information from the auxiliary modality across feature channels and layers. Specifically, this is highlighted by three components: (I) An encoder based on UNet with a single-stream strategy that fuses auxiliary and target modalities; (II) a decoder that tends to multi-level features from all layers of the encoder, and (III) a channel attention module. Quantitative and qualitative analyses are performed on a public brain dataset and knee brain dataset, which show that the proposed method achieves satisfying results in MRI reconstruction within the multi-modal context, and also demonstrate its effectiveness and potential to be used in clinical practice.

Feature Fusion for Multi-Coil Compressed MR Image Reconstruction

Magnetic resonance imaging (MRI) occupies a pivotal position within contemporary diagnostic imaging modalities, offering non-invasive and radiation-free scanning. Despite its significance, MRI's principal limitation is the protracted data acquisition time, which hampers broader practical application. Promising deep learning (DL) methods for undersampled magnetic resonance (MR) image reconstruction outperform the traditional approaches in terms of speed and image quality. However, the intricate inter-coil correlations have been insufficiently addressed, leading to an underexploitation of the rich information inherent in multi-coil acquisitions. In this article, we proposed a method called "Multi-coil Feature Fusion Variation Network" (MFFVN), which introduces an encoder to extract the feature from multi-coil MR image directly and explicitly, followed by a feature fusion operation. Coil reshaping enables the 2D network to achieve satisfactory reconstruction results, while avoiding the introduction of a significant number of parameters and preserving inter-coil information. Compared with VN, MFFVN yields an improvement in the average PSNR and SSIM of the test set, registering enhancements of 0.2622 dB and 0.0021 dB respectively. This uplift can be attributed to the integration of feature extraction and fusion stages into the network's architecture, thereby effectively leveraging and combining the multi-coil information for enhanced image reconstruction quality. The proposed method outperforms the state-of-the-art methods on fastMRI dataset of multi-coil brains under a fourfold acceleration factor without incurring substantial computation overhead.

Joint k-ω Space Image Reconstruction and Data Fitting for Chemical Exchange Saturation Transfer Magnetic Resonance Imaging

Chemical exchange saturation transfer (CEST) magnetic resonance imaging (MRI) is a novel MRI technology to image certain compounds at extremely low concentrations. Long acquisition time to measure signals at a set of offset frequencies of the Z-spectra and to repeat measurements to reduce noise pose significant challenges to its applications. This study explores correlations of CEST MR images along the spatial and Z-spectral dimensions to improve MR image quality and robustness of magnetization transfer ratio (MTR) asymmetry estimation via a joint k-ω reconstruction model. The model was formulated as an optimization problem with respect to MR images at all frequencies ω, while incorporating regularizations along the spatial and spectral dimensions. The solution was subject to a self-consistency condition that the Z-spectrum of each pixel follows a multi-peak data fitting model corresponding to different CEST pools. The optimization problem was solved using the alternating direction method of multipliers. The proposed joint reconstruction method was evaluated on a simulated CEST MRI phantom and semi-experimentally on choline and iopamidol phantoms with added Gaussian noise of various levels. Results demonstrated that the joint reconstruction method was more tolerable to noise and reduction in number of offset frequencies by improving signal-to-noise ratio (SNR) of the reconstructed images and reducing uncertainty in MTR asymmetry estimation. In the choline and iopamidol phantom cases with 10.5% noise in the measurement data, our method achieved an averaged SNR of 31.0 dB and 32.2 dB compared to the SNR of 24.7 dB and 24.4 dB in the conventional reconstruction approach. It reduced uncertainty of the MTR asymmetry estimation over all regions of interest by 54.4% and 43.7%, from 1.71 and 2.38 to 0.78 and 1.71, respectively.

Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI

OBJECTIVE

This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics.

MATERIALS AND METHODS

To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters.

RESULTS

ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA.

DISCUSSION

These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.

Constrained alternating minimization for parameter mapping (CAMP)

OBJECTIVE

To improve image quality in highly accelerated parameter mapping by incorporating a linear constraint that relates consecutive images.

APPROACH

In multi-echo T1 or T2 mapping, scan time is often shortened by acquiring undersampled but complementary measures of k-space at each TE or TI. However, residual undersampling artifacts from the individual images can then degrade the quality of the final parameter maps. In this work, a new reconstruction method, dubbed Constrained Alternating Minimization for Parameter mapping (CAMP), is introduced. This method simultaneously extracts T2 or T1* maps in addition to an image for each TE or TI from accelerated datasets, leveraging the constraints of the decay to improve the reconstructed image quality. The model enforces exponential decay through a linear constraint, resulting in a biconvex objective function that lends itself to alternating minimization. The method was tested in four in vivo volunteer experiments and validated in phantom studies and healthy subjects, using T2 and T1 mapping, with accelerations of up to 12.

MAIN RESULTS

CAMP is demonstrated for accelerated radial and Cartesian acquisitions in T2 and T1 mapping. The method is even applied to generate an entire T2 weighted image series from a single TSE dataset, despite the blockwise k-space sampling at each echo time. Experimental undersampled phantom and in vivo results processed with CAMP exhibit reduced artifacts without introducing bias.

SIGNIFICANCE

For a wide array of applications, CAMP linearizes the model cost function without sacrificing model accuracy so that the well-conditioned and highly efficient reconstruction algorithm improves the image quality of accelerated parameter maps.

Development of a whole spinal MRI-based tumor burden scoring method in participants with multiple myeloma: a pilot study of prognostic significance

The aim of the study was to develop a new whole spinal MRI-based tumor burden scoring method in participants with newly diagnosed multiple myeloma (MM) and to explore its prognostic significance. We prospectively recruited participants with newly diagnosed MM; performed whole spinal MRI (sagittal FSE T1WI, sagittal IDEAL T2WI, and axial FLAIR T2WI) on them; and collected their clinical data, early treatment response, progression-free survival (PFS), and overall survival (OS). We developed a new tumor burden scoring method according to the extent of bone marrow infiltration in five MRI patterns. All participants were divided into good response and poor response groups after four treatment cycles. Univariate, multivariate analyses, and ROC were used to determine the performance of independent predictors. Thresholds for PFS and OS were calculated using X-tile, and their prognostic significance were assessed by Kaplan-Meier. The Kruskal-Wallis H test was used to compare the differences of tumor burden score between the revised International Staging System (R-ISS) stages. The new tumor burden scoring method was used in 62 participants (median score, 12; range, 0-18). The tumor burden score (OR 1.266, p = 0.002) was an independent predictor of poor response and the AUC was 0.838. Higher tumor burden scores were associated with shorter PFS (p = 0.002) and OS (p = 0.011). The tumor burden score was higher in R-ISS-III than in R-ISS-I and R-ISS-II (p = 0.016 and p = 0.006, respectively). The tumor burden score was an excellent predictor of prognosis and may serve as a supplemental marker for R-ISS.

Improving deep PROPELLER MRI via synthetic blade augmentation and enhanced generalization

In PROPELLER MRI, obtaining sufficient high-quality blade data remains a challenge, so the efficiency and generalization of deep learning-based reconstruction models are deteriorated. Due to narrow rotated and translated blades acquired in PROPELLER, the technique of data augmentation that is used for deep learning-based Cartesian MRI reconstruction cannot be directly applied. To address the issue, this paper introduces a novel approach for the generation of synthetic PROPELLER blades, and it is subsequently employed in data augmentation for undersampled blades reconstruction. The principal aim of this study is to address the challenges of reconstructing undersampled blades to enhance both image quality and computational efficiency. Evaluation metrics including PSNR, NMSE, and SSIM indicate superior performance of the model trained with augmented data compared to non-augmented counterparts. The synthetic blade augmentation significantly enhances the model's generalization capability and enables robust performance across varying imaging conditions. Furthermore, the study demonstrates the feasibility of utilizing synthetic blades exclusively in the training phase, suggesting a reduced dependency on real PROPELLER blades. This innovation in synthetic blade generation and data augmentation technique contributes to enhanced image quality and improved generalization capability of the associated deep learning model for PROPELLER MRI reconstruction.

Joint reconstruction and segmentation in undersampled 3D knee MRI combining shape knowledge and deep learning

Objective.Task-adapted image reconstruction methods using end-to-end trainable neural networks (NNs) have been proposed to optimize reconstruction for subsequent processing tasks, such as segmentation. However, their training typically requires considerable hardware resources and thus, only relatively simple building blocks, e.g. U-Nets, are typically used, which, albeit powerful, do not integrate model-specific knowledge.Approach.In this work, we extend an end-to-end trainable task-adapted image reconstruction method for a clinically realistic reconstruction and segmentation problem of bone and cartilage in 3D knee MRI by incorporating statistical shape models (SSMs). The SSMs model the prior information and help to regularize the segmentation maps as a final post-processing step. We compare the proposed method to a simultaneous multitask learning approach for image reconstruction and segmentation (MTL) and to a complex SSMs-informed segmentation pipeline (SIS).Main results.Our experiments show that the combination of joint end-to-end training and SSMs to further regularize the segmentation maps obtained by MTL highly improves the results, especially in terms of mean and maximal surface errors. In particular, we achieve the segmentation quality of SIS and, at the same time, a substantial model reduction that yields a five-fold decimation in model parameters and a computational speedup of an order of magnitude.Significance.Remarkably, even for undersampling factors of up toR= 8, the obtained segmentation maps are of comparable quality to those obtained by SIS from ground-truth images.

On retrospective k-space subsampling schemes for deep MRI reconstruction

Acquiring fully-sampled MRI k-space data is time-consuming, and collecting accelerated data can reduce the acquisition time. Employing 2D Cartesian-rectilinear subsampling schemes is a conventional approach for accelerated acquisitions; however, this often results in imprecise reconstructions, even with the use of Deep Learning (DL), especially at high acceleration factors. Non-rectilinear or non-Cartesian trajectories can be implemented in MRI scanners as alternative subsampling options. This work investigates the impact of the k-space subsampling scheme on the quality of reconstructed accelerated MRI measurements produced by trained DL models. The Recurrent Variational Network (RecurrentVarNet) was used as the DL-based MRI-reconstruction architecture. Cartesian, fully-sampled multi-coil k-space measurements from three datasets were retrospectively subsampled with different accelerations using eight distinct subsampling schemes: four Cartesian-rectilinear, two Cartesian non-rectilinear, and two non-Cartesian. Experiments were conducted in two frameworks: scheme-specific, where a distinct model was trained and evaluated for each dataset-subsampling scheme pair, and multi-scheme, where for each dataset a single model was trained on data randomly subsampled by any of the eight schemes and evaluated on data subsampled by all schemes. In both frameworks, RecurrentVarNets trained and evaluated on non-rectilinearly subsampled data demonstrated superior performance, particularly for high accelerations. In the multi-scheme setting, reconstruction performance on rectilinearly subsampled data improved when compared to the scheme-specific experiments. Our findings demonstrate the potential for using DL-based methods, trained on non-rectilinearly subsampled measurements, to optimize scan time and image quality.

Deep learning-based diffusion tensor cardiac magnetic resonance reconstruction: a comparison study

In vivo cardiac diffusion tensor imaging (cDTI) is a promising Magnetic Resonance Imaging (MRI) technique for evaluating the microstructure of myocardial tissue in living hearts, providing insights into cardiac function and enabling the development of innovative therapeutic strategies. However, the integration of cDTI into routine clinical practice poses challenging due to the technical obstacles involved in the acquisition, such as low signal-to-noise ratio and prolonged scanning times. In this study, we investigated and implemented three different types of deep learning-based MRI reconstruction models for cDTI reconstruction. We evaluated the performance of these models based on the reconstruction quality assessment, the diffusion tensor parameter assessment as well as the computational cost assessment. Our results indicate that the models discussed in this study can be applied for clinical use at an acceleration factor (AF) of × 2 and × 4 , with the D5C5 model showing superior fidelity for reconstruction and the SwinMR model providing higher perceptual scores. There is no statistical difference from the reference for all diffusion tensor parameters at AF × 2 or most DT parameters at AF × 4 , and the quality of most diffusion tensor parameter maps is visually acceptable. SwinMR is recommended as the optimal approach for reconstruction at AF × 2 and AF × 4 . However, we believe that the models discussed in this study are not yet ready for clinical use at a higher AF. At AF × 8 , the performance of all models discussed remains limited, with only half of the diffusion tensor parameters being recovered to a level with no statistical difference from the reference. Some diffusion tensor parameter maps even provide wrong and misleading information.

MA-RECON: Mask-aware deep-neural-network for robust fast MRI k-space interpolation

BACKGROUND AND OBJECTIVE

High-quality reconstruction of MRI images from under-sampled 'k-space' data, which is in the Fourier domain, is crucial for shortening MRI acquisition times and ensuring superior temporal resolution. Over recent years, a wealth of deep neural network (DNN) methods have emerged, aiming to tackle the complex, ill-posed inverse problem linked to this process. However, their instability against variations in the acquisition process and anatomical distribution exposes a deficiency in the generalization of relevant physical models within these DNN architectures. The goal of our work is to enhance the generalization capabilities of DNN methods for k-space interpolation by introducing 'MA-RECON', an innovative mask-aware DNN architecture and associated training method.

METHODS

Unlike preceding approaches, our 'MA-RECON' architecture encodes not only the observed data but also the under-sampling mask within the model structure. It implements a tailored training approach that leverages data generated with a variety of under-sampling masks to stimulate the model's generalization of the under-sampled MRI reconstruction problem. Therefore, effectively represents the associated inverse problem, akin to the classical compressed sensing approach.

RESULTS

The benefits of our MA-RECON approach were affirmed through rigorous testing with the widely accessible fastMRI dataset. Compared to standard DNN methods and DNNs trained with under-sampling mask augmentation, our approach demonstrated superior generalization capabilities. This resulted in a considerable improvement in robustness against variations in both the acquisition process and anatomical distribution, especially in regions with pathology.

CONCLUSION

In conclusion, our mask-aware strategy holds promise for enhancing the generalization capacity and robustness of DNN-based methodologies for MRI reconstruction from undersampled k-space data. Code is available in the following link: https://github.com/nitzanavidan/PD_Recon.

Ultrafast Brain MRI with Deep Learning Reconstruction for Suspected Acute Ischemic Stroke

Background Deep learning (DL)-accelerated MRI can substantially reduce examination times. However, studies prospectively evaluating the diagnostic performance of DL-accelerated MRI reconstructions in acute suspected stroke are lacking. Purpose To investigate the interchangeability of DL-accelerated MRI with conventional MRI in patients with suspected acute ischemic stroke at 1.5 T. Materials and Methods In this prospective study, 211 participants with suspected acute stroke underwent clinically indicated MRI at 1.5 T between June 2022 and March 2023. For each participant, conventional MRI (including T1-weighted, T2-weighted, T2*-weighted, T2 fluid-attenuated inversion-recovery, and diffusion-weighted imaging; 14 minutes 18 seconds) and DL-accelerated MRI (same sequences; 3 minutes 4 seconds) were performed. The primary end point was the interchangeability between conventional and DL-accelerated MRI for acute ischemic infarction detection. Secondary end points were interchangeability regarding the affected vascular territory and clinically relevant secondary findings (eg, microbleeds, neoplasm). Three readers evaluated the overall occurrence of acute ischemic stroke, affected vascular territory, clinically relevant secondary findings, overall image quality, and diagnostic confidence. For acute ischemic lesions, size and signal intensities were assessed. The margin for interchangeability was chosen as 5%. For interrater agreement analysis and interrater reliability analysis, multirater Fleiss κ and the intraclass correlation coefficient, respectively, was determined. Results The study sample consisted of 211 participants (mean age, 65 years ± 16 [SD]); 123 male and 88 female). Acute ischemic stroke was confirmed in 79 participants. Interchangeability was demonstrated for all primary and secondary end points. No individual equivalence indexes (IEIs) exceeded the interchangeability margin of 5% (IEI, -0.002 [90% CI: -0.007, 0.004]). Almost perfect interrater agreement was observed (P > .91). DL-accelerated MRI provided higher overall image quality (P < .001) and diagnostic confidence (P < .001). The signal properties of acute ischemic infarctions were similar in both techniques and demonstrated good to excellent interrater reliability (intraclass correlation coefficient, ≥0.8). Conclusion Despite being four times faster, DL-accelerated brain MRI was interchangeable with conventional MRI for acute ischemic lesion detection. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Haller in this issue.

New clinical opportunities of low-field MRI: heart, lung, body, and musculoskeletal

Contemporary whole-body low-field MRI scanners (< 1 T) present new and exciting opportunities for improved body imaging. The fundamental reason is that the reduced off-resonance and reduced SAR provide substantially increased flexibility in the design of MRI pulse sequences. Promising body applications include lung parenchyma imaging, imaging adjacent to metallic implants, cardiac imaging, and dynamic imaging in general. The lower cost of such systems may make MRI favorable for screening high-risk populations and population health research, and the more open configurations allowed may prove favorable for obese subjects and for pregnant women. This article summarizes promising body applications for contemporary whole-body low-field MRI systems, with a focus on new platforms developed within the past 5 years. This is an active area of research, and one can expect many improvements as MRI physicists fully explore the landscape of pulse sequences that are feasible, and as clinicians apply these to patient populations.

McSTRA: A multi-branch cascaded swin transformer for point spread function-guided robust MRI reconstruction

Deep learning MRI reconstruction methods are often based on Convolutional neural network (CNN) models; however, they are limited in capturing global correlations among image features due to the intrinsic locality of the convolution operation. Conversely, the recent vision transformer models (ViT) are capable of capturing global correlations by applying self-attention operations on image patches. Nevertheless, the existing transformer models for MRI reconstruction rarely leverage the physics of MRI. In this paper, we propose a novel physics-based transformer model titled, the Multi-branch Cascaded Swin Transformers (McSTRA) for robust MRI reconstruction. McSTRA combines several interconnected MRI physics-related concepts with the Swin transformers: it exploits global MRI features via the shifted window self-attention mechanism; it extracts MRI features belonging to different spectral components via a multi-branch setup; it iterates between intermediate de-aliasing and data consistency via a cascaded network with intermediate loss computations; furthermore, we propose a point spread function-guided positional embedding generation mechanism for the Swin transformers which exploit the spread of the aliasing artifacts for effective reconstruction. With the combination of all these components, McSTRA outperforms the state-of-the-art methods while demonstrating robustness in adversarial conditions such as higher accelerations, noisy data, different undersampling protocols, out-of-distribution data, and abnormalities in anatomy.

Advances in medical image analysis with vision Transformers: A comprehensive review

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.

Technical Note: Novel imaging method to obtain gray matter-attenuated inversion recovery image using low-field magnetic resonance imaging systems

INTRODUCTION

The double inversion recovery (DIR) technique suppresses two types of tissue signals with different T1 values by applying two inversion recovery (IR) pulses with different inversion times (TI). In contrast, the double tissue suppression with multi-echo acquisition and single TI combining HIRE (DOMUST-HIRE) method, is a technique enabling the white-matter-attenuated inversion recovery (WAIR) images by setting one inversion time (TI) in a sequence based on the multi-echo method and subtracting the second echo image from the first echo image. Here, we propose a new sequence that can provide the gray-matter-attenuated inversion recovery image based on the DOMUST-HIRE method.

METHODS

In this small clinical study, we performed determination of optimal TI and physical evaluation by imaging a subject's head with T1WI and our proposed method for GAIR images.

RESULTS

Our proposed method could increase the contrast ratio and the contrast-to-noise ratio between white matter (WM) and gray matter (GM), whereas the signal-to-noise ratio WM and GM decreased than with T1WI method.

CONCLUSIONS

Our proposed method can be used to suppress GM and CSF signals.

IMPLICATIONS FOR PRACTICE

The use of our proposed method in low-field MRI systems could provide GAIR image.

Intrafraction Motion Management With MR-Guided Radiation Therapy

High quality radiation therapy requires highly accurate and precise dose delivery. MR-guided radiotherapy (MRgRT), integrating an MRI scanner with a linear accelerator, offers excellent quality images in the treatment room without subjecting patient to ionizing radiation. MRgRT therefore provides a powerful tool for intrafraction motion management. This paper summarizes different sources of intrafraction motion for different disease sites and describes the MR imaging techniques available to visualize and quantify intrafraction motion. It provides an overview of MR guided motion management strategies and of the current technical capabilities of the commercially available MRgRT systems. It describes how these motion management capabilities are currently being used in clinical studies, protocols and provides a future outlook.

Two-and-a-half order score-based model for solving 3D ill-posed inverse problems

Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are crucial technologies in the field of medical imaging. Score-based models demonstrated effectiveness in addressing different inverse problems encountered in the field of CT and MRI, such as sparse-view CT and fast MRI reconstruction. However, these models face challenges in achieving accurate three dimensional (3D) volumetric reconstruction. The existing score-based models predominantly concentrate on reconstructing two-dimensional (2D) data distributions, resulting in inconsistencies between adjacent slices in the reconstructed 3D volumetric images. To overcome this limitation, we propose a novel two-and-a-half order score-based model (TOSM). During the training phase, our TOSM learns data distributions in 2D space, simplifying the training process compared to working directly on 3D volumes. However, during the reconstruction phase, the TOSM utilizes complementary scores along three directions (sagittal, coronal, and transaxial) to achieve a more precise reconstruction. The development of TOSM is built on robust theoretical principles, ensuring its reliability and efficacy. Through extensive experimentation on large-scale sparse-view CT and fast MRI datasets, our method achieved state-of-the-art (SOTA) results in solving 3D ill-posed inverse problems, averaging a 1.56 dB peak signal-to-noise ratio (PSNR) improvement over existing sparse-view CT reconstruction methods across 29 views and 0.87 dB PSNR improvement over existing fast MRI reconstruction methods with × 2 acceleration. In summary, TOSM significantly addresses the issue of inconsistency in 3D ill-posed problems by modeling the distribution of 3D data rather than 2D distribution which has achieved remarkable results in both CT and MRI reconstruction tasks.

Multi-contrast high-field quality image synthesis for portable low-field MRI using generative adversarial networks and paired data

Introduction

Portable low-field strength (64mT) MRI scanners promise to increase access to neuroimaging for clinical and research purposes, however these devices produce lower quality images compared to high-field scanners. In this study, we developed and evaluated a deep learning architecture to generate high-field quality brain images from low-field inputs using a paired dataset of multiple sclerosis (MS) patients scanned at 64mT and 3T.

Methods

A total of 49 MS patients were scanned on portable 64mT and standard 3T scanners at Penn (n=25) or the National Institutes of Health (NIH, n=24) with T1-weighted, T2-weighted and FLAIR acquisitions. Using this paired data, we developed a generative adversarial network (GAN) architecture for low- to high-field image translation (LowGAN). We then evaluated synthesized images with respect to image quality, brain morphometry, and white matter lesions.

Results

Synthetic high-field images demonstrated visually superior quality compared to low-field inputs and significantly higher normalized cross-correlation (NCC) to actual high-field images for T1 (p=0.001) and FLAIR (p<0.001) contrasts. LowGAN generally outperformed the current state-of-the-art for low-field volumetrics. For example, thalamic, lateral ventricle, and total cortical volumes in LowGAN outputs did not differ significantly from 3T measurements. Synthetic outputs preserved MS lesions and captured a known inverse relationship between total lesion volume and thalamic volume.

Conclusions

LowGAN generates synthetic high-field images with comparable visual and quantitative quality to actual high-field scans. Enhancing portable MRI image quality could add value and boost clinician confidence, enabling wider adoption of this technology.

Parallel-stream fusion of scan-specific and scan-general priors for learning deep MRI reconstruction in low-data regimes

Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan times. Reconstruction methods can alleviate this limitation by recovering clinically usable images from accelerated acquisitions. In particular, learning-based methods promise performance leaps by employing deep neural networks as data-driven priors. A powerful approach uses scan-specific (SS) priors that leverage information regarding the underlying physical signal model for reconstruction. SS priors are learned on each individual test scan without the need for a training dataset, albeit they suffer from computationally burdening inference with nonlinear networks. An alternative approach uses scan-general (SG) priors that instead leverage information regarding the latent features of MRI images for reconstruction. SG priors are frozen at test time for efficiency, albeit they require learning from a large training dataset. Here, we introduce a novel parallel-stream fusion model (PSFNet) that synergistically fuses SS and SG priors for performant MRI reconstruction in low-data regimes, while maintaining competitive inference times to SG methods. PSFNet implements its SG prior based on a nonlinear network, yet it forms its SS prior based on a linear network to maintain efficiency. A pervasive framework for combining multiple priors in MRI reconstruction is algorithmic unrolling that uses serially alternated projections, causing error propagation under low-data regimes. To alleviate error propagation, PSFNet combines its SS and SG priors via a novel parallel-stream architecture with learnable fusion parameters. Demonstrations are performed on multi-coil brain MRI for varying amounts of training data. PSFNet outperforms SG methods in low-data regimes, and surpasses SS methods with few tens of training samples. On average across tasks, PSFNet achieves 3.1 dB higher PSNR, 2.8% higher SSIM, and 0.3 × lower RMSE than baselines. Furthermore, in both supervised and unsupervised setups, PSFNet requires an order of magnitude lower samples compared to SG methods, and enables an order of magnitude faster inference compared to SS methods. Thus, the proposed model improves deep MRI reconstruction with elevated learning and computational efficiency.

Low-rank tensor assisted K-space generative model for parallel imaging reconstruction

Although recent deep learning methods, especially generative models, have shown good performance in magnetic resonance imaging, there is still much room for improvement. Considering that the sample number and internal dimension in score-based generative model have key influence on estimating the gradients of data distribution, we present a new idea for parallel imaging reconstruction, named low-rank tensor assisted k-space generative model (LR-KGM). It means that we transform low-rank information into high-dimensional prior information for learning. More specifically, the multi-channel data is constructed into a large Hankel matrix to reduce the number of training samples, which is subsequently collapsed into a tensor for the stage of prior learning. In the testing phase, the low-rank rotation strategy is utilized to impose low-rank constraints on the output tensors of the generative network. Furthermore, we alternate the reconstruction between traditional generative iterations and low-rank high-dimensional tensor iterations. Experimental comparisons with the state-of-the-arts demonstrated that the proposed LR-KGM method achieved better performance.

Data-efficient Bayesian learning for radial dynamic MR reconstruction

BACKGROUND

Cardiac MRI has become the gold-standard imaging technique for assessing cardiovascular morphology and function. In spite of this, its slow data acquisition process presents imaging challenges due to the motion from heartbeats, respiration, and blood flow. In recent studies, deep learning (DL) algorithms have shown promising results for the task of image reconstruction. However, there have been instances where they have introduced artifacts that may be misinterpreted as pathologies or may obscure the detection of pathologies. Therefore, it is important to obtain a metric, such as the uncertainty of the network output, that identifies such artifacts. However, this can be quite challenging for large-scale image reconstruction problems such as dynamic multi-coil non-Cartesian MRI.

PURPOSE

To efficiently quantify uncertainties of a physics-informed DL-based image reconstruction method for a large-scale accelerated 2D multi-coil dynamic radial MRI reconstruction problem, and demonstrate the benefits of physics-informed DL over model-agnostic DL in reducing uncertainties while at the same time improving image quality.

METHODS

We extended a recently proposed physics-informed 2D U-Net that learns spatio-temporal slices (named XT-YT U-Net), and employed it for the task of uncertainty quantification (UQ) by using Monte Carlo dropout and a Gaussian negative log-likelihood loss function. Our data comprised 2D dynamic MR images acquired with a radial balanced steady-state free precession sequence. The XT-YT U-Net, which allows for training with a limited amount of data, was trained and validated on a dataset of 15 healthy volunteers, and further tested on data from four patients. An extensive comparison between physics-informed and model-agnostic neural networks (NNs) concerning the obtained image quality and uncertainty estimates was performed. Further, we employed calibration plots to assess the quality of the UQ.

RESULTS

The inclusion of the MR-physics model of data acquisition as a building block in the NN architecture led to higher image quality (NRMSE:- 33 ± 8.2 % $-33 \pm 8.2 \%$ , PSNR:6.3 ± 1.3 % $6.3 \pm 1.3 \%$ , and SSIM:1.9 ± 0.96 % $1.9 \pm 0.96 \%$ ), lower uncertainties (- 46 ± 8.7 % $-46 \pm 8.7 \%$ ), and, based on the calibration plots, an improved UQ compared to its model-agnostic counterpart. Furthermore, the UQ information can be used to differentiate between anatomical structures (e.g., coronary arteries, ventricle boundaries) and artifacts.

CONCLUSIONS

Using an XT-YT U-Net, we were able to quantify uncertainties of a physics-informed NN for a high-dimensional and computationally demanding 2D multi-coil dynamic MR imaging problem. In addition to improving the image quality, embedding the acquisition model in the network architecture decreased the reconstruction uncertainties as well as quantitatively improved the UQ. The UQ provides additional information to assess the performance of different network approaches.

A Systematic Analysis of Additive Manufacturing Techniques in the Bioengineering of In Vitro Cardiovascular Models

Additive Manufacturing is noted for ease of product customization and short production run cost-effectiveness. As our global population approaches 8 billion, additive manufacturing has a future in maintaining and improving average human life expectancy for the same reasons that it has advantaged general manufacturing. In recent years, additive manufacturing has been applied to tissue engineering, regenerative medicine, and drug delivery. Additive Manufacturing combined with tissue engineering and biocompatibility studies offers future opportunities for various complex cardiovascular implants and surgeries. This paper is a comprehensive overview of current technological advancements in additive manufacturing with potential for cardiovascular application. The current limitations and prospects of the technology for cardiovascular applications are explored and evaluated.

Improving MR image quality with a multi-task model, using convolutional losses

PURPOSE

During the acquisition of MRI data, patient-, sequence-, or hardware-related factors can introduce artefacts that degrade image quality. Four of the most significant tasks for improving MRI image quality have been bias field correction, super-resolution, motion-, and noise correction. Machine learning has achieved outstanding results in improving MR image quality for these tasks individually, yet multi-task methods are rarely explored.

METHODS

In this study, we developed a model to simultaneously correct for all four aforementioned artefacts using multi-task learning. Two different datasets were collected, one consisting of brain scans while the other pelvic scans, which were used to train separate models, implementing their corresponding artefact augmentations. Additionally, we explored a novel loss function that does not only aim to reconstruct the individual pixel values, but also the image gradients, to produce sharper, more realistic results. The difference between the evaluated methods was tested for significance using a Friedman test of equivalence followed by a Nemenyi post-hoc test.

RESULTS

Our proposed model generally outperformed other commonly-used correction methods for individual artefacts, consistently achieving equal or superior results in at least one of the evaluation metrics. For images with multiple simultaneous artefacts, we show that the performance of using a combination of models, trained to correct individual artefacts depends heavily on the order that they were applied. This is not an issue for our proposed multi-task model. The model trained using our novel convolutional loss function always outperformed the model trained with a mean squared error loss, when evaluated using Visual Information Fidelity, a quality metric connected to perceptual quality.

CONCLUSION

We trained two models for multi-task MRI artefact correction of brain, and pelvic scans. We used a novel loss function that significantly improves the image quality of the outputs over using mean squared error. The approach performs well on real world data, and it provides insight into which artefacts it detects and corrects for. Our proposed model and source code were made publicly available.

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities

BACKGROUND

Machine learning (ML) is considered an important technology for future data analysis in health care.

METHODS

The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers.

RESULTS AND CONCLUSION

In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future.

KEY POINTS

· ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET..

Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives

Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.

Model-based Deep Learning Reconstruction Using a Folded Image Training Strategy for Abdominal 3D T1-weighted Imaging

PURPOSE

To evaluate the feasibility of folded image training strategy (FITS) and the quality of images reconstructed using the improved model-based deep learning (iMoDL) network trained with FITS (FITS-iMoDL) for abdominal MR imaging.

METHODS

This retrospective study included abdominal 3D T1-weighted images of 122 patients. In the experimental analyses, peak SNR (PSNR) and structure similarity index (SSIM) of images reconstructed with FITS-iMoDL were compared with those with the following reconstruction methods: conventional model-based deep learning (conv-MoDL), MoDL trained with FITS (FITS-MoDL), total variation regularized compressed sensing (CS), and parallel imaging (CG-SENSE). In the clinical analysis, SNR and image contrast were measured on the reference, FITS-iMoDL, and CS images. Three radiologists evaluated the image quality using a 5-point scale to determine the mean opinion score (MOS).

RESULTS

The PSNR of FITS-iMoDL was significantly higher than that of FITS-MoDL, conv-MoDL, CS, and CG-SENSE (P < 0.001). The SSIM of FITS-iMoDL was significantly higher than those of the others (P < 0.001), except for FITS-MoDL (P = 0.056). In the clinical analysis, the SNR of FITS-iMoDL was significantly higher than that of the reference and CS (P < 0.0001). Image contrast was equivalent within an equivalence margin of 10% among these three image sets (P < 0.0001). MOS was significantly improved in FITS-iMoDL (P < 0.001) compared with CS images in terms of liver edge and vessels conspicuity, lesion depiction, artifacts, blurring, and overall image quality.

CONCLUSION

The proposed method, FITS-iMoDL, allowed a deeper MoDL reconstruction network without increasing memory consumption and improved image quality on abdominal 3D T1-weighted imaging compared with CS images.

CS-MRI Reconstruction Using an Improved GAN with Dilated Residual Networks and Channel Attention Mechanism

Compressed sensing (CS) MRI has shown great potential in enhancing time efficiency. Deep learning techniques, specifically generative adversarial networks (GANs), have emerged as potent tools for speedy CS-MRI reconstruction. Yet, as the complexity of deep learning reconstruction models increases, this can lead to prolonged reconstruction time and challenges in achieving convergence. In this study, we present a novel GAN-based model that delivers superior performance without the model complexity escalating. Our generator module, built on the U-net architecture, incorporates dilated residual (DR) networks, thus expanding the network's receptive field without increasing parameters or computational load. At every step of the downsampling path, this revamped generator module includes a DR network, with the dilation rates adjusted according to the depth of the network layer. Moreover, we have introduced a channel attention mechanism (CAM) to distinguish between channels and reduce background noise, thereby focusing on key information. This mechanism adeptly combines global maximum and average pooling approaches to refine channel attention. We conducted comprehensive experiments with the designed model using public domain MRI datasets of the human brain. Ablation studies affirmed the efficacy of the modified modules within the network. Incorporating DR networks and CAM elevated the peak signal-to-noise ratios (PSNR) of the reconstructed images by about 1.2 and 0.8 dB, respectively, on average, even at 10× CS acceleration. Compared to other relevant models, our proposed model exhibits exceptional performance, achieving not only excellent stability but also outperforming most of the compared networks in terms of PSNR and SSIM. When compared with U-net, DR-CAM-GAN's average gains in SSIM and PSNR were 14% and 15%, respectively. Its MSE was reduced by a factor that ranged from two to seven. The model presents a promising pathway for enhancing the efficiency and quality of CS-MRI reconstruction.

Sinogram upsampling using Primal-Dual UNet for undersampled CT and radial MRI reconstruction

Computed tomography (CT) and magnetic resonance imaging (MRI) are two widely used clinical imaging modalities for non-invasive diagnosis. However, both of these modalities come with certain problems. CT uses harmful ionising radiation, and MRI suffers from slow acquisition speed. Both problems can be tackled by undersampling, such as sparse sampling. However, such undersampled data leads to lower resolution and introduces artefacts. Several techniques, including deep learning based methods, have been proposed to reconstruct such data. However, the undersampled reconstruction problem for these two modalities was always considered as two different problems and tackled separately by different research works. This paper proposes a unified solution for both sparse CT and undersampled radial MRI reconstruction, achieved by applying Fourier transform-based pre-processing on the radial MRI and then finally reconstructing both modalities using sinogram upsampling combined with filtered back-projection. The Primal-Dual network is a deep learning based method for reconstructing sparsely-sampled CT data. This paper introduces Primal-Dual UNet, which improves the Primal-Dual network in terms of accuracy and reconstruction speed. The proposed method resulted in an average SSIM of 0.932±0.021 while performing sparse CT reconstruction for fan-beam geometry with a sparsity level of 16, achieving a statistically significant improvement over the previous model, which resulted in 0.919±0.016. Furthermore, the proposed model resulted in 0.903±0.019 and 0.957±0.023 average SSIM while reconstructing undersampled brain and abdominal MRI data with an acceleration factor of 16, respectively - statistically significant improvements over the original model, which resulted in 0.867±0.025 and 0.949±0.025. Finally, this paper shows that the proposed network not only improves the overall image quality, but also improves the image quality for the regions-of-interest: liver, kidneys, and spleen; as well as generalises better than the baselines in presence the of a needle.

Faster Elbow MRI with Deep Learning Reconstruction-Assessment of Image Quality, Diagnostic Confidence, and Anatomy Visualization Compared to Standard Imaging

OBJECTIVE

The objective of this study was to evaluate a deep learning (DL) reconstruction for turbo spin echo (TSE) sequences of the elbow regarding image quality and visualization of anatomy.

MATERIALS AND METHODS

Between October 2020 and June 2021, seventeen participants (eight patients, nine healthy subjects; mean age: 43 ± 16 (20-70) years, eight men) were prospectively included in this study. Each patient underwent two examinations: standard MRI, including TSE sequences reconstructed with a generalized autocalibrating partial parallel acquisition reconstruction (TSESTD), and prospectively undersampled TSE sequences reconstructed with a DL reconstruction (TSEDL). Two radiologists evaluated the images concerning image quality, noise, edge sharpness, artifacts, diagnostic confidence, and delineation of anatomical structures using a 5-point Likert scale, and rated the images concerning the detection of common pathologies.

RESULTS

Image quality was significantly improved in TSEDL (mean 4.35, IQR 4-5) compared to TSESTD (mean 3.76, IQR 3-4, p = 0.008). Moreover, TSEDL showed decreased noise (mean 4.29, IQR 3.5-5) compared to TSESTD (mean 3.35, IQR 3-4, p = 0.004). Ratings for delineation of anatomical structures, artifacts, edge sharpness, and diagnostic confidence did not differ significantly between TSEDL and TSESTD (p > 0.05). Inter-reader agreement was substantial to almost perfect (κ = 0.628-0.904). No difference was found concerning the detection of pathologies between the readers and between TSEDL and TSESTD. Using DL, the acquisition time could be reduced by more than 35% compared to TSESTD.

CONCLUSION

TSEDL provided improved image quality and decreased noise while receiving equal ratings for edge sharpness, artifacts, delineation of anatomical structures, diagnostic confidence, and detection of pathologies compared to TSESTD. Providing more than a 35% reduction of acquisition time, TSEDL may be clinically relevant for elbow imaging due to increased patient comfort and higher patient throughput.

Mitigating Under-Sampling Artifacts in 3D Photoacoustic Imaging Using Res-UNet Based on Digital Breast Phantom

In recent years, photoacoustic (PA) imaging has rapidly grown as a non-invasive screening technique for breast cancer detection using three-dimensional (3D) hemispherical arrays due to their large field of view. However, the development of breast imaging systems is hindered by a lack of patients and ground truth samples, as well as under-sampling problems caused by high costs. Most research related to solving these problems in the PA field were based on 2D transducer arrays or simple regular shape phantoms for 3D transducer arrays or images from other modalities. Therefore, we demonstrate an effective method for removing under-sampling artifacts based on deep neural network (DNN) to reconstruct high-quality PA images using numerical digital breast simulations. We constructed 3D digital breast phantoms based on human anatomical structures and physical properties, which were then subjected to 3D Monte-Carlo and K-wave acoustic simulations to mimic acoustic propagation for hemispherical transducer arrays. Finally, we applied a 3D delay-and-sum reconstruction algorithm and a Res-UNet network to achieve higher resolution on sparsely-sampled data. Our results indicate that when using a 757 nm laser with uniform intensity distribution illuminated on a numerical digital breast, the imaging depth can reach 3 cm with 0.25 mm spatial resolution. In addition, the proposed DNN can significantly enhance image quality by up to 78.4%, as measured by MS-SSIM, and reduce background artifacts by up to 19.0%, as measured by PSNR, even at an under-sampling ratio of 10%. The post-processing time for these improvements is only 0.6 s. This paper suggests a new 3D real time DNN method addressing the sparse sampling problem based on numerical digital breast simulations, this approach can also be applied to clinical data and accelerate the development of 3D photoacoustic hemispherical transducer arrays for early breast cancer diagnosis.

Diffusion models in medical imaging: A comprehensive survey

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical imaging. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including image-to-image translation, reconstruction, registration, classification, segmentation, denoising, 2/3D generation, anomaly detection, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub.1 We aim to update the relevant latest papers within it regularly.

Transformers in medical imaging: A survey

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as de facto operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, restoration, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging.

CSA: A Channel-Separated Attention Module for Enhancing MRI Reconstruction

Channel attention mechanisms have been proven to effectively enhance network performance in various visual tasks, including the Magnetic Resonance Imaging (MRI) reconstruction task. Channel attention mechanisms typically involve channel dimensionality reduction and cross-channel interaction operations to achieve complexity reduction and generate more effective weights of channels. However, the operations may negatively impact MRI reconstruction performance since it was found that there is no discernible correlation between adjacent channels and the low information value in some feature maps. Therefore, we proposed the Channel-Separated Attention (CSA) module tailored for MRI reconstruction networks. Each layer of the CSA module avoids compressing channels, thereby allowing for lossless information transmission. Additionally, we employed the Hadamard product to realize that each channel's importance weight was generated solely based on itself, avoiding cross-channel interaction and reducing the computational complexity. We replaced the original channel attention module with the CSA module in an advanced MRI reconstruction network and noticed that CSA module achieved superior reconstruction performance with fewer parameters. Furthermore, we conducted comparative experiments with state-of-the-art channel attention modules on an identical network backbone, CSA module achieved competitive reconstruction outcomes with only approximately 1.036% parameters of the Squeeze-and-Excitation (SE) module. Overall, the CSA module makes an optimal trade-off between complexity and reconstruction quality to efficiently and effectively enhance MRI reconstruction. The code is available at https://github.com/smd1997/CSA-Net.

Adaptive diffusion priors for accelerated MRI reconstruction

Deep MRI reconstruction is commonly performed with conditional models that de-alias undersampled acquisitions to recover images consistent with fully-sampled data. Since conditional models are trained with knowledge of the imaging operator, they can show poor generalization across variable operators. Unconditional models instead learn generative image priors decoupled from the operator to improve reliability against domain shifts related to the imaging operator. Recent diffusion models are particularly promising given their high sample fidelity. Nevertheless, inference with a static image prior can perform suboptimally. Here we propose the first adaptive diffusion prior for MRI reconstruction, AdaDiff, to improve performance and reliability against domain shifts. AdaDiff leverages an efficient diffusion prior trained via adversarial mapping over large reverse diffusion steps. A two-phase reconstruction is executed following training: a rapid-diffusion phase that produces an initial reconstruction with the trained prior, and an adaptation phase that further refines the result by updating the prior to minimize data-consistency loss. Demonstrations on multi-contrast brain MRI clearly indicate that AdaDiff outperforms competing conditional and unconditional methods under domain shifts, and achieves superior or on par within-domain performance.

Ngo L, J. Manning W, Nezafat R. Accelerated Cardiac MRI Cine with Use of Resolution Enhancement Generative Adversarial Inline Neural Network

Background Cardiac cine can benefit from deep learning-based image reconstruction to reduce scan time and/or increase spatial and temporal resolution. Purpose To develop and evaluate a deep learning model that can be combined with parallel imaging or compressed sensing (CS). Materials and Methods The deep learning model was built on the enhanced super-resolution generative adversarial inline neural network, trained with use of retrospectively identified cine images and evaluated in participants prospectively enrolled from September 2021 to September 2022. The model was applied to breath-hold electrocardiography (ECG)-gated segmented and free-breathing real-time cine images collected with reduced spatial resolution with use of generalized autocalibrating partially parallel acquisitions (GRAPPA) or CS. The deep learning model subsequently restored spatial resolution. For comparison, GRAPPA-accelerated cine images were collected. Diagnostic quality and artifacts were evaluated by two readers with use of Likert scales and compared with use of Wilcoxon signed-rank tests. Agreement for left ventricle (LV) function, volume, and strain was assessed with Bland-Altman analysis. Results The deep learning model was trained on 1616 patients (mean age ± SD, 56 years ± 16; 920 men) and evaluated in 181 individuals, 126 patients (mean age, 57 years ± 16; 77 men) and 55 healthy subjects (mean age, 27 years ± 10; 15 men). In breath-hold ECG-gated segmented cine and free-breathing real-time cine, the deep learning model and GRAPPA showed similar diagnostic quality scores (2.9 vs 2.9, P = .41, deep learning vs GRAPPA) and artifact score (4.4 vs 4.3, P = .55, deep learning vs GRAPPA). Deep learning acquired more sections per breath-hold than GRAPPA (3.1 vs one section, P < .001). In free-breathing real-time cine, the deep learning showed a similar diagnostic quality score (2.9 vs 2.9, P = .21, deep learning vs GRAPPA) and lower artifact score (3.9 vs 4.3, P < .001, deep learning vs GRAPPA). For both sequences, the deep learning model showed excellent agreement for LV parameters, with near-zero mean differences and narrow limits of agreement compared with GRAPPA. Conclusion Deep learning-accelerated cardiac cine showed similarly accurate quantification of cardiac function, volume, and strain to a standardized parallel imaging method. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Vannier and Wang in this issue.

M4Raw: A multi-contrast, multi-repetition, multi-channel MRI k-space dataset for low-field MRI research

Recently, low-field magnetic resonance imaging (MRI) has gained renewed interest to promote MRI accessibility and affordability worldwide. The presented M4Raw dataset aims to facilitate methodology development and reproducible research in this field. The dataset comprises multi-channel brain k-space data collected from 183 healthy volunteers using a 0.3 Tesla whole-body MRI system, and includes T1-weighted, T2-weighted, and fluid attenuated inversion recovery (FLAIR) images with in-plane resolution of ~1.2 mm and through-plane resolution of 5 mm. Importantly, each contrast contains multiple repetitions, which can be used individually or to form multi-repetition averaged images. After excluding motion-corrupted data, the partitioned training and validation subsets contain 1024 and 240 volumes, respectively. To demonstrate the potential utility of this dataset, we trained deep learning models for image denoising and parallel imaging tasks and compared their performance with traditional reconstruction methods. This M4Raw dataset will be valuable for the development of advanced data-driven methods specifically for low-field MRI. It can also serve as a benchmark dataset for general MRI reconstruction algorithms.

Adaptive model-based Magnetic Resonance

PURPOSE

Conventional sequences are static in nature, fixing measurement parameters in advance in anticipation of a wide range of expected tissue parameter values. We set out to design and benchmark a new, personalized approach-termed adaptive MR-in which incoming subject data is used to update and fine-tune the pulse sequence parameters in real time.

METHODS

We implemented an adaptive, real-time multi-echo (MTE) experiment for estimating T₂ s. Our approach combined a Bayesian framework with model-based reconstruction. It maintained and continuously updated a prior distribution of the desired tissue parameters, including T₂ , which was used to guide the selection of sequence parameters in real time.

RESULTS

Computer simulations predicted accelerations between 1.7- and 3.3-fold for adaptive multi-echo sequences relative to static ones. These predictions were corroborated in phantom experiments. In healthy volunteers, our adaptive framework accelerated the measurement of T₂ for n-acetyl-aspartate by a factor of 2.5.

CONCLUSION

Adaptive pulse sequences that alter their excitations in real time could provide substantial reductions in acquisition times. Given the generality of our proposed framework, our results motivate further research into other adaptive model-based approaches to MRI and MRS.

Improving accelerated MRI by deep learning with sparsified complex data

PURPOSE

To obtain high-quality accelerated MR images with complex-valued reconstruction from undersampled k-space data.

METHODS

The MRI scans from human subjects were retrospectively undersampled with a regular pattern using skipped phase encoding, leading to ghosts in zero-filling reconstruction. A complex difference transform along the phase-encoding direction was applied in image domain to yield sparsified complex-valued edge maps. These sparse edge maps were used to train a complex-valued U-type convolutional neural network (SCU-Net) for deghosting. A k-space inverse filtering was performed on the predicted deghosted complex edge maps from SCU-Net to obtain final complex images. The SCU-Net was compared with other algorithms including zero-filling, GRAPPA, RAKI, finite difference complex U-type convolutional neural network (FDCU-Net), and CU-Net, both qualitatively and quantitatively, using such metrics as structural similarity index, peak SNR, and normalized mean square error.

RESULTS

The SCU-Net was found to be effective in deghosting aliased edge maps even at high acceleration factors. High-quality complex images were obtained by performing an inverse filtering on deghosted edge maps. The SCU-Net compared favorably with other algorithms.

CONCLUSION

Using sparsified complex data, SCU-Net offers higher reconstruction quality for regularly undersampled k-space data. The proposed method is especially useful for phase-sensitive MRI applications.

Fast Multi-Contrast MRI Acquisition by Optimal Sampling of Information Complementary to Pre-Acquired MRI Contrast

Recent studies on multi-contrast MRI reconstruction have demonstrated the potential of further accelerating MRI acquisition by exploiting correlation between contrasts. Most of the state-of-the-art approaches have achieved improvement through the development of network architectures for fixed under-sampling patterns, without considering inter-contrast correlation in the under-sampling pattern design. On the other hand, sampling pattern learning methods have shown better reconstruction performance than those with fixed under-sampling patterns. However, most under-sampling pattern learning algorithms are designed for single contrast MRI without exploiting complementary information between contrasts. To this end, we propose a framework to optimize the under-sampling pattern of a target MRI contrast which complements the acquired fully-sampled reference contrast. Specifically, a novel image synthesis network is introduced to extract the redundant information contained in the reference contrast, which is exploited in the subsequent joint pattern optimization and reconstruction network. We have demonstrated superior performance of our learned under-sampling patterns on both public and in-house datasets, compared to the commonly used under-sampling patterns and state-of-the-art methods that jointly optimize the reconstruction network and the under-sampling patterns, up to 8-fold under-sampling factor.

Location-Dependent Spatiotemporal Antialiasing in Photoacoustic Computed Tomography

Photoacoustic computed tomography (PACT) images optical absorption contrast by detecting ultrasonic waves induced by optical energy deposition in materials such as biological tissues. An ultrasonic transducer array or its scanning equivalent is used to detect ultrasonic waves. The spatial distribution of the transducer elements must satisfy the spatial Nyquist criterion; otherwise, spatial aliasing occurs and causes artifacts in reconstructed images. The spatial Nyquist criterion poses different requirements on the transducer elements' distributions for different locations in the image domain, which has not been studied previously. In this research, we elaborate on the location dependency through spatiotemporal analysis and propose a location-dependent spatiotemporal antialiasing method. By applying this method to PACT in full-ring array geometry, we effectively mitigate aliasing artifacts with minimal effects on image resolution in both numerical simulations and in vivo experiments.

Multi-weight respecification of scan-specific learning for parallel imaging

Parallel imaging is widely used in magnetic resonance imaging as an acceleration technology. Traditional linear reconstruction methods in parallel imaging often suffer from noise amplification. Recently, a non-linear robust artificial-neural-network for k-space interpolation (RAKI) exhibits superior noise resilience over other linear methods. However, RAKI performs poorly at high acceleration rates and needs a large number of autocalibration signals as the training samples. In order to tackle these issues, we propose a multi-weight method that implements multiple weighting matrices on the under-sampled data, named MW-RAKI. Enforcing multiple weighted matrices on the measurements can effectively reduce the influence of noise and increase the data constraints. Furthermore, we incorporate the strategy of multiple weighting matrixes into a residual version of RAKI, and form MW-rRAKI. Experimental comparisons with the alternative methods demonstrated noticeably better reconstruction performances, particularly at high acceleration rates. With only 12.5% of the k-space data is available, the PSNR of MW-RAKI and MW-rRAKI is improved by about 3 dB and 4 dB compared to RAKI and rRAKI, respectively.

Tile-based microscopic image processing for malaria screening using a deep learning approach

BACKGROUND

Manual microscopic examination remains the golden standard for malaria diagnosis. But it is laborious, and pathologists with experience are needed for accurate diagnosis. The need for computer-aided diagnosis methods is driven by the enormous workload and difficulties associated with manual microscopy based examination. While the importance of computer-aided diagnosis is increasing at an enormous pace, fostered by the advancement of deep learning algorithms, there are still challenges in detecting small objects such as malaria parasites in microscopic images of blood films. The state-of-the-art (SOTA) deep learning-based object detection models are inefficient in detecting small objects accurately because they are underrepresented on benchmark datasets. The performance of these models is affected by the loss of detailed spatial information due to in-network feature map downscaling. This is due to the fact that the SOTA models cannot directly process high-resolution images due to their low-resolution network input layer.

METHODS

In this study, an efficient and robust tile-based image processing method is proposed to enhance the performance of malaria parasites detection SOTA models. Three variants of YOLOV4-based object detectors are adopted considering their detection accuracy and speed. These models were trained using tiles generated from 1780 high-resolution P. falciparum-infected thick smear microscopic images. The tiling of high-resolution images improves the performance of the object detection models. The detection accuracy and the generalization capability of these models have been evaluated using three datasets acquired from different regions.

RESULTS

The best-performing model using the proposed tile-based approach outperforms the baseline method significantly (Recall, [95.3%] vs [57%] and Average Precision, [87.1%] vs [76%]). Furthermore, the proposed method has outperformed the existing approaches that used different machine learning techniques evaluated on similar datasets.

CONCLUSIONS

The experimental results show that the proposed method significantly improves P. falciparum detection from thick smear microscopic images while maintaining real-time detection speed. Furthermore, the proposed method has the potential to assist and reduce the workload of laboratory technicians in malaria-endemic remote areas of developing countries where there is a critical skill gap and a shortage of experts.