Image reconstruction by domain-transform manifold learning

High-performance rapid MR parameter mapping using model-based deep adversarial learning

PURPOSE

To develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping.

METHODS

The proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T2 mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method.

RESULTS

The proposed adversarial learning approach achieved accurate T2 mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T2 estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning.

CONCLUSION

The proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.

Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data

PURPOSE

To develop a strategy for training a physics-guided MRI reconstruction neural network without a database of fully sampled data sets.

METHODS

Self-supervised learning via data undersampling (SSDU) for physics-guided deep learning reconstruction partitions available measurements into two disjoint sets, one of which is used in the data consistency (DC) units in the unrolled network and the other is used to define the loss for training. The proposed training without fully sampled data is compared with fully supervised training with ground-truth data, as well as conventional compressed-sensing and parallel imaging methods using the publicly available fastMRI knee database. The same physics-guided neural network is used for both proposed SSDU and supervised training. The SSDU training is also applied to prospectively two-fold accelerated high-resolution brain data sets at different acceleration rates, and compared with parallel imaging.

RESULTS

Results on five different knee sequences at an acceleration rate of 4 shows that the proposed self-supervised approach performs closely with supervised learning, while significantly outperforming conventional compressed-sensing and parallel imaging, as characterized by quantitative metrics and a clinical reader study. The results on prospectively subsampled brain data sets, in which supervised learning cannot be used due to lack of ground-truth reference, show that the proposed self-supervised approach successfully performs reconstruction at high acceleration rates (4, 6, and 8). Image readings indicate improved visual reconstruction quality with the proposed approach compared with parallel imaging at acquisition acceleration.

CONCLUSION

The proposed SSDU approach allows training of physics-guided deep learning MRI reconstruction without fully sampled data, while achieving comparable results with supervised deep learning MRI trained on fully sampled data.

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge

PURPOSE

To advance research in the field of machine learning for MR image reconstruction with an open challenge.

METHODS

We provided participants with a dataset of raw k-space data from 1,594 consecutive clinical exams of the knee. The goal of the challenge was to reconstruct images from these data. In order to strike a balance between realistic data and a shallow learning curve for those not already familiar with MR image reconstruction, we ran multiple tracks for multi-coil and single-coil data. We performed a two-stage evaluation based on quantitative image metrics followed by evaluation by a panel of radiologists. The challenge ran from June to December of 2019.

RESULTS

We received a total of 33 challenge submissions. All participants chose to submit results from supervised machine learning approaches.

CONCLUSIONS

The challenge led to new developments in machine learning for image reconstruction, provided insight into the current state of the art in the field, and highlighted remaining hurdles for clinical adoption.

A Novel Neural Model With Lateral Interaction for Learning Tasks

We propose a novel neural model with lateral interaction for learning tasks. The model consists of two functional fields: an elementary field to extract features and a high-level field to store and recognize patterns. Each field is composed of some neurons with lateral interaction, and the neurons in different fields are connected by the rules of synaptic plasticity. The model is established on the current research of cognition and neuroscience, making it more transparent and biologically explainable. Our proposed model is applied to data classification and clustering. The corresponding algorithms share similar processes without requiring any parameter tuning and optimization processes. Numerical experiments validate that the proposed model is feasible in different learning tasks and superior to some state-of-the-art methods, especially in small sample learning, one-shot learning, and clustering.

Ultrasound transmission tomography image reconstruction with a fully convolutional neural network

Image reconstruction of ultrasound computed tomography based on the wave equation is able to show much more structural details than simpler ray-based image reconstruction methods. However, to invert the wave-based forward model is computationally demanding. To address this problem, we develop an efficient fully learned image reconstruction method based on a convolutional neural network. The image is reconstructed via one forward propagation of the network given input sensor data, which is much faster than the reconstruction using conventional iterative optimization methods. To transform the ultrasound measured data in the sensor domain into the reconstructed image in the image domain, we apply multiple down-scaling and up-scaling convolutional units to efficiently increase the number of hidden layers with a large receptive and projective field that can cover all elements in inputs and outputs, respectively. For dataset generation, a paraxial approximation forward model is used to simulate ultrasound measurement data. The neural network is trained with a dataset derived from natural images in ImageNet and tested with a dataset derived from medical images in OA-Breast Phantom dataset. Test results show the superior efficiency of the proposed neural network to other reconstruction algorithms including popular neural networks. When compared with conventional iterative optimization algorithms, our neural network can reconstruct a 110 × 86 image more than 20 times faster on a CPU and 1000 times faster on a GPU with comparable image quality and is also more robust to noise.

SARA-GAN: Self-Attention and Relative Average Discriminator Based Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction

Research on undersampled magnetic resonance image (MRI) reconstruction can increase the speed of MRI imaging and reduce patient suffering. In this paper, an undersampled MRI reconstruction method based on Generative Adversarial Networks with the Self-Attention mechanism and the Relative Average discriminator (SARA-GAN) is proposed. In our SARA-GAN, the relative average discriminator theory is applied to make full use of the prior knowledge, in which half of the input data of the discriminator is true and half is fake. At the same time, a self-attention mechanism is incorporated into the high-layer of the generator to build long-range dependence of the image, which can overcome the problem of limited convolution kernel size. Besides, spectral normalization is employed to stabilize the training process. Compared with three widely used GAN-based MRI reconstruction methods, i.e., DAGAN, DAWGAN, and DAWGAN-GP, the proposed method can obtain a higher peak signal-to-noise ratio (PSNR) and structural similarity index measure(SSIM), and the details of the reconstructed image are more abundant and more realistic for further clinical scrutinization and diagnostic tasks.

Robust water-fat separation based on deep learning model exploring multi-echo nature of mGRE

PURPOSE

To design a new deep learning network for fast and accurate water-fat separation by exploring the correlations between multiple echoes in multi-echo gradient-recalled echo (mGRE) sequence and evaluate the generalization capabilities of the network for different echo times, field inhomogeneities, and imaging regions.

METHODS

A new multi-echo bidirectional convolutional residual network (MEBCRN) was designed to separate water and fat images in a fast and accurate manner for the mGRE data. This new MEBCRN network contains 2 main modules, the first 1 is the feature extraction module, which learns the correlations between consecutive echoes, and the other one is the water-fat separation module that processes the feature information extracted from the feature extraction module. The multi-layer feature fusion (MLFF) mechanism and residual structure were adopted in the water-fat separation module to increase separation accuracy and robustness. Moreover, we trained the network using in vivo abdomen images and tested it on the abdomen, knee, and wrist images.

RESULTS

The results showed that the proposed network could separate water and fat images accurately. The comparison of the proposed network and other deep learning methods shows the advantage in both quantitative metrics and robustness for different TEs, field inhomogeneities, and images acquired for various imaging regions.

CONCLUSION

The proposed network could learn the correlations between consecutive echoes and separate water and fat images effectively. The deep learning method has certain generalization capabilities for TEs and field inhomogeneity. Although the network was trained only in vivo abdomen images, it could be applied for different imaging regions.

Triple-D network for efficient undersampled magnetic resonance images reconstruction

Compressed sensing (CS) theory can help accelerate magnetic resonance imaging (MRI) by sampling partial k-space measurements. However, conventional optimization-based CS-MRI methods are often time-consuming and are based on fixed transform or shallow image dictionaries, which limits modeling capabilities. Recently, deep learning models have been used to solve the CS-MRI problem. However, recent researches have focused on modeling in image domain, and the potential of k-space modeling capability has not been utilized seriously. In this paper, we propose a deep model called Dual Domain Dense network (Triple-D network), which consisted of some k-space and image domain sub-network. These sub-networks are connected with dense connections, which can utilize feature maps at different levels to enhance performance. To further promote model capabilities, we use two strategies: multi-supervision strategies, which can avoid loss of supervision information; channel-wise attention layer (CA layer), which can adaptively adjust the weight of the feature map. Experimental results show that the proposed Triple-D network provides promising performance in CS-MRI, and it can effectively work on different sampling trajectories and noisy settings.

Dynamic MRI reconstruction with end-to-end motion-guided network

Temporal correlation in dynamic magnetic resonance imaging (MRI), such as cardiac MRI, is informative and important to understand motion mechanisms of body regions. Modeling such information into the MRI reconstruction process produces temporally coherent image sequence and reduces imaging artifacts and blurring. However, existing deep learning based approaches neglect motion information during the reconstruction procedure, while traditional motion-guided methods are hindered by heuristic parameter tuning and long inference time. We propose a novel dynamic MRI reconstruction approach called MODRN and an end-to-end improved version called MODRN(e2e), both of which enhance the reconstruction quality by infusing motion information into the modeling process with deep neural networks. The central idea is to decompose the motion-guided optimization problem of dynamic MRI reconstruction into three components: Dynamic Reconstruction Network, Motion Estimation and Motion Compensation. Extensive experiments have demonstrated the effectiveness of our proposed approach compared to other state-of-the-art approaches.

Artificial Intelligence for Optimization and Interpretation of PET/CT and PET/MR Images

Artificial intelligence (AI) has recently attracted much attention for its potential use in healthcare applications. The use of AI to improve and extract more information out of medical images, given their parallels with natural images and the immense progress in the area of computer vision, has been at the forefront of these advances. This is due to a convergence of factors, including the increasing numbers of scans performed, the availability of open source AI tools, and decreases in the costs of hardware required to implement these technologies. In this article, we review the progress in the use of AI toward optimizing PET/CT and PET/MRI studies. These two methods, which combine molecular information with structural and (in the case of MRI) functional imaging, are extremely valuable for a wide range of clinical indications. They are also tremendously data-rich modalities and as such are highly amenable to data-driven technologies such as AI. The first half of the article will focus on methods to improve PET reconstruction and image quality, which has multiple benefits including faster image acquisition, image reconstruction, and lower or even "zero" radiation dose imaging. It will also address the value of AI-driven methods to perform MR-based attenuation correction. The second half will address how some of these advances can be used to perform to optimize diagnosis from the acquired images, with examples given for whole-body oncology, cardiology, and neurology indications. Overall, it is likely that the use of AI will markedly improve both the quality and safety of PET/CT and PET/MRI as well as enhance our ability to interpret the scans and follow lesions over time. This will hopefully lead to expanded clinical use cases for these valuable technologies leading to better patient care.

Appearance Learning for Image-Based Motion Estimation in Tomography

In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Geometric information within this process is usually depending on the system setting only, i.e., the scanner position or readout direction. Patient motion therefore corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object. To this end, we train a siamese triplet network to predict the reprojection error (RPE) for the complete acquisition as well as an approximate distribution of the RPE along the single views from the reconstructed volume in a multi-task learning approach. The RPE measures the motion-induced geometric deviations independent of the object based on virtual marker positions, which are available during training. We train our network using 27 patients and deploy a 21-4-2 split for training, validation and testing. In average, we achieve a residual mean RPE of 0.013mm with an inter-patient standard deviation of 0.022mm. This is twice the accuracy compared to previously published results. In a motion estimation benchmark the proposed approach achieves superior results in comparison with two state-of-the-art measures in nine out of twelve experiments. The clinical applicability of the proposed method is demonstrated on a motion-affected clinical dataset.

Known Operator Learning Enables Constrained Projection Geometry Conversion: Parallel to Cone-Beam for Hybrid MR/X-Ray Imaging

X-ray imaging is a wide-spread real-time imaging technique. Magnetic Resonance Imaging (MRI) offers a multitude of contrasts that offer improved guidance to interventionalists. As such simultaneous real-time acquisition and overlay would be highly favorable for image-guided interventions, e.g., in stroke therapy. One major obstacle in this setting is the fundamentally different acquisition geometry. MRI k -space sampling is associated with parallel projection geometry, while the X-ray acquisition results in perspective distorted projections. The classical rebinning methods to overcome this limitation inherently suffers from a loss of resolution. To counter this problem, we present a novel rebinning algorithm for parallel to cone-beam conversion. We derive a rebinning formula that is then used to find an appropriate deep neural network architecture. Following the known operator learning paradigm, the novel algorithm is mapped to a neural network with differentiable projection operators enabling data-driven learning of the remaining unknown operators. The evaluation aims in two directions: First, we give a profound analysis of the different hypotheses to the unknown operator and investigate the influence of numerical training data. Second, we evaluate the performance of the proposed method against the classical rebinning approach. We demonstrate that the derived network achieves better results than the baseline method and that such operators can be trained with simulated data without losing their generality making them applicable to real data without the need for retraining or transfer learning.

Distortion correction of single-shot EPI enabled by deep-learning

PURPOSE

A distortion correction method for single-shot EPI was proposed. Point-spread-function encoded EPI (PSF-EPI) images were used as the references to correct traditional EPI images based on deep neural network.

THEORY AND METHODS

The PSF-EPI method can obtain distortion-free echo planar images. In this study, a 2D U-net based network was trained to achieve the distortion correction of single-shot EPI (SS-EPI) images, using PSF-EPI images as targets in the training stage. Anatomical T2W-TSE images were also fed into the network to improve the quality of the results. The applications in diffusion-weighted images were used as examples in this work. The network was trained on data acquired on healthy volunteers and tested on data of both healthy volunteers and patients. The corrected EPI images from the proposed method were also compared with those from field-mapping and top-up based distortion correction methods.

RESULTS

Experimental results showed that the proposed method can correct for EPI distortions better than both the field-mapping and top-up based methods, and the results were close to the distortion-free images from PSF-EPI. Additionally, inclusion of T2W-TSE images helped improve distortion correction of the SS-EPI images without contaminating the output noticeably. The experiments with patients and different MRI platforms demonstrated the generalization feasibility of the proposed method preliminarily.

CONCLUSION

Through the correction of diffusion-weighted images, the proposed deep-learning based method was demonstrated to have the feasibility to correct for the distortion of EPI images.

Rapid Knee MRI Acquisition and Analysis Techniques for Imaging Osteoarthritis

Osteoarthritis (OA) of the knee is a major source of disability that has no known treatment or cure. Morphological and compositional MRI is commonly used for assessing the bone and soft tissues in the knee to enhance the understanding of OA pathophysiology. However, it is challenging to extend these imaging methods and their subsequent analysis techniques to study large population cohorts due to slow and inefficient imaging acquisition and postprocessing tools. This can create a bottleneck in assessing early OA changes and evaluating the responses of novel therapeutics. The purpose of this review article is to highlight recent developments in tools for enhancing the efficiency of knee MRI methods useful to study OA. Advances in efficient MRI data acquisition and reconstruction tools for morphological and compositional imaging, efficient automated image analysis tools, and hardware improvements to further drive efficient imaging are discussed in this review. For each topic, we discuss the current challenges as well as potential future opportunities to alleviate these challenges. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY STAGE: 3.

High signal-to-noise ratio reconstruction of low bit-depth optical coherence tomography using deep learning

SIGNIFICANCE

Reducing the bit depth is an effective approach to lower the cost of an optical coherence tomography (OCT) imaging device and increase the transmission efficiency in data acquisition and telemedicine. However, a low bit depth will lead to the degradation of the detection sensitivity, thus reducing the signal-to-noise ratio (SNR) of OCT images.

AIM

We propose using deep learning to reconstruct high SNR OCT images from low bit-depth acquisition.

APPROACH

The feasibility of our approach is evaluated by applying this approach to the quantized 3- to 8-bit data from native 12-bit interference fringes. We employ a pixel-to-pixel generative adversarial network (pix2pixGAN) architecture in the low-to-high bit-depth OCT image transition.

RESULTS

Extensively, qualitative and quantitative results show our method could significantly improve the SNR of the low bit-depth OCT images. The adopted pix2pixGAN is superior to other possible deep learning and compressed sensing solutions.

CONCLUSIONS

Our work demonstrates that the proper integration of OCT and deep learning could benefit the development of healthcare in low-resource settings.

Deep-Learning Image Reconstruction for Real-Time Photoacoustic System

Recent advances in photoacoustic (PA) imaging have enabled detailed images of microvascular structure and quantitative measurement of blood oxygenation or perfusion. Standard reconstruction methods for PA imaging are based on solving an inverse problem using appropriate signal and system models. For handheld scanners, however, the ill-posed conditions of limited detection view and bandwidth yield low image contrast and severe structure loss in most instances. In this paper, we propose a practical reconstruction method based on a deep convolutional neural network (CNN) to overcome those problems. It is designed for real-time clinical applications and trained by large-scale synthetic data mimicking typical microvessel networks. Experimental results using synthetic and real datasets confirm that the deep-learning approach provides superior reconstructions compared to conventional methods.

Rapid single scan ramped hybrid-encoding for bicomponent T2* mapping in a human knee joint: A feasibility study

The purpose of this study is to determine the feasibility of using a single scan ramped hybrid-encoding (RHE) method for rapid bicomponent T2* analysis of the human knee joint. The proposed method utilizes RHE to acquire ultrashort echo time (UTE) and subsequent gradient echo images at 16 different echo times ranging between 40 μs and 30 ms in a single scan. In the proposed RHE technique, UTE imaging was followed by acquisition of 14 gradient recalled echo images, where an additional UTE image was obtained within the first readout by oversampling single point imaging (SPI) encoding. The single scan RHE method with a 9-minute scan time was performed on human cadaveric knee joints from six donors and in vivo knee joints from four healthy volunteers at 3 T. A bicomponent signal model was used to characterize the short T2* and long T2* water components. Mean bicomponent T2* parameters for patellar tendon, anterior cruciate ligament (ACL), posterior cruciate ligament (PCL) and meniscus were calculated. In the experimental results, the RHE technique provided bicomponent T2* parameter estimations of tendon, ACL, PCL and meniscus, which were similar to previously reported values in the literature. In conclusion, the proposed single scan RHE technique provides rapid bicomponent T2* analysis of the human knee joint with a total scan time of less than 9 minutes.

Is Artificial Intelligence the New Friend for Radiologists? A Review Article

Artificial intelligence (AI) is a path-breaking advancement for many industries, including the health care sector. The expeditious development of information technology and data processing has led to the formation of recent tools known as artificial intelligence. Radiology has been a portal for medical technological advancements, and AI will likely be no dissimilar. Radiology is the platform for many technological advances in the medical field; AI can undoubtedly impact every step of a radiologist's workflow. AI can simplify every activity like ordering and scheduling, protocoling and acquisition, image interpretation, reporting, communication, and billing. AI has eminent potential to augment efficiency and accuracy throughout radiology, but it also possesses inherent drawbacks and biases. We collected studies that were published in the past five years using PubMed as our database. We chose studies that were relevant to artificial intelligence in radiology. We mainly focused on the overview of AI in radiology, components included in the functioning of AI, AI assisting in the radiologists' workflow, ethical aspects of AI, challenges, and biases that AI experiencing together with some clinical applications of AI. Of all 33 studies, we found 15 articles discussed the overview and components of AI, five articles about AI affecting radiologist's workflow, five articles related to challenges and biases in AI, two articles discussed ethical aspects of AI, and six articles about practical implications of AI. We found out that the application of AI could make time-dependent tasks that can be performed effortlessly, permitting radiologists more time and opportunities to engage in patient care via increased time for consultation and development in imaging and extracting useful data from those images. AI could only be an aid to radiologists but will not replace a radiologist. Radiologists who use AI to their benefit, rather than to avoid it out of fear, might supersede those radiologists who do not. Substantial research should be done regarding the practical implications of AI algorithms for residents curriculum and the benefits of AI in radiology.

Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon's time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.

Potentials and caveats of AI in hybrid imaging

State-of-the-art patient management frequently mandates the investigation of both anatomy and physiology of the patients. Hybrid imaging modalities such as the PET/MRI, PET/CT and SPECT/CT have the ability to provide both structural and functional information of the investigated tissues in a single examination. With the introduction of such advanced hardware fusion, new problems arise such as the exceedingly large amount of multi-modality data that requires novel approaches of how to extract a maximum of clinical information from large sets of multi-dimensional imaging data. Artificial intelligence (AI) has emerged as one of the leading technologies that has shown promise in facilitating highly integrative analysis of multi-parametric data. Specifically, the usefulness of AI algorithms in the medical imaging field has been heavily investigated in the realms of (1) image acquisition and reconstruction, (2) post-processing and (3) data mining and modelling. Here, we aim to provide an overview of the challenges encountered in hybrid imaging and discuss how AI algorithms can facilitate potential solutions. In addition, we highlight the pitfalls and challenges in using advanced AI algorithms in the context of hybrid imaging and provide suggestions for building robust AI solutions that enable reproducible and transparent research.

Using Deep Learning to Accelerate Knee MRI at 3 T: Results of an Interchangeability Study

OBJECTIVE. Deep learning (DL) image reconstruction has the potential to disrupt the current state of MRI by significantly decreasing the time required for MRI examinations. Our goal was to use DL to accelerate MRI to allow a 5-minute comprehensive examination of the knee without compromising image quality or diagnostic accuracy. MATERIALS AND METHODS. A DL model for image reconstruction using a variational network was optimized. The model was trained using dedicated multisequence training, in which a single reconstruction model was trained with data from multiple sequences with different contrast and orientations. After training, data from 108 patients were retrospectively undersampled in a manner that would correspond with a net 3.49-fold acceleration of fully sampled data acquisition and a 1.88-fold acceleration compared with our standard twofold accelerated parallel acquisition. An interchangeability study was performed, in which the ability of six readers to detect internal derangement of the knee was compared for clinical and DL-accelerated images. RESULTS. We found a high degree of interchangeability between standard and DL-accelerated images. In particular, results showed that interchanging the sequences would produce discordant clinical opinions no more than 4% of the time for any feature evaluated. Moreover, the accelerated sequence was judged by all six readers to have better quality than the clinical sequence. CONCLUSION. An optimized DL model allowed acceleration of knee images that performed interchangeably with standard images for detection of internal derangement of the knee. Importantly, readers preferred the quality of accelerated images to that of standard clinical images.

Diverse Applications of Artificial Intelligence in Neuroradiology

Recent advances in artificial intelligence (AI) and deep learning (DL) hold promise to augment neuroimaging diagnosis for patients with brain tumors and stroke. Here, the authors review the diverse landscape of emerging neuroimaging applications of AI, including workflow optimization, lesion segmentation, and precision education. Given the many modalities used in diagnosing neurologic diseases, AI may be deployed to integrate across modalities (MR imaging, computed tomography, PET, electroencephalography, clinical and laboratory findings), facilitate crosstalk among specialists, and potentially improve diagnosis in patients with trauma, multiple sclerosis, epilepsy, and neurodegeneration. Together, there are myriad applications of AI for neuroradiology."

Gene correlation network analysis to identify regulatory factors in sepsis

BACKGROUND AND OBJECTIVES

Sepsis is a leading cause of mortality and morbidity in the intensive care unit. Regulatory mechanisms underlying the disease progression and prognosis are largely unknown. The study aimed to identify master regulators of mortality-related modules, providing potential therapeutic target for further translational experiments.

METHODS

The dataset GSE65682 from the Gene Expression Omnibus (GEO) database was utilized for bioinformatic analysis. Consensus weighted gene co-expression netwoek analysis (WGCNA) was performed to identify modules of sepsis. The module most significantly associated with mortality were further analyzed for the identification of master regulators of transcription factors and miRNA.

RESULTS

A total number of 682 subjects with various causes of sepsis were included for consensus WGCNA analysis, which identified 27 modules. The network was well preserved among different causes of sepsis. Two modules designated as black and light yellow module were found to be associated with mortality outcome. Key regulators of the black and light yellow modules were the transcription factor CEBPB (normalized enrichment score = 5.53) and ETV6 (NES = 6), respectively. The top 5 miRNA regulated the most number of genes were hsa-miR-335-5p (n = 59), hsa-miR-26b-5p (n = 57), hsa-miR-16-5p (n = 44), hsa-miR-17-5p (n = 42), and hsa-miR-124-3p (n = 38). Clustering analysis in 2-dimension space derived from manifold learning identified two subclasses of sepsis, which showed significant association with survival in Cox proportional hazard model (p = 0.018).

CONCLUSIONS

The present study showed that the black and light-yellow modules were significantly associated with mortality outcome. Master regulators of the module included transcription factor CEBPB and ETV6. miRNA-target interactions identified significantly enriched miRNA.

Effectiveness of Deep Learning Algorithms to Determine Laterality in Radiographs

Develop a highly accurate deep learning model to reliably classify radiographs by laterality. Digital Imaging and Communications in Medicine (DICOM) data for nine body parts was extracted retrospectively. Laterality was determined directly if encoded properly or inferred using other elements. Curation confirmed categorization and identified inaccurate labels due to human error. Augmentation enriched training data to semi-equilibrate classes. Classification and object detection models were developed on a dedicated workstation and tested on novel images. Receiver operating characteristic (ROC) curves, sensitivity, specificity, and accuracy were calculated. Study-level accuracy was determined and both were compared to human performance. An ensemble model was tested for the rigorous use-case of automatically classifying exams retrospectively. The final classification model identified novel images with an ROC area under the curve (AUC) of 0.999, improving on previous work and comparable to human performance. A similar ROC curve was observed for per-study analysis with AUC of 0.999. The object detection model classified images with accuracy of 99% or greater at both image and study level. Confidence scores allow adjustment of sensitivity and specificity as needed; the ensemble model designed for the highly specific use-case of automatically classifying exams was comparable and arguably better than human performance demonstrating 99% accuracy with 1% of exams unchanged and no incorrect classification. Deep learning models can classify radiographs by laterality with high accuracy and may be applied in a variety of settings that could improve patient safety and radiologist satisfaction. Rigorous use-cases requiring high specificity are achievable.

Multiple Slice k-space Deep Learning for Magnetic Resonance Imaging Reconstruction

Magnetic resonance imaging (MRI) has been one of the most powerful and valuable imaging methods for medical diagnosis and staging of disease. Due to the long scan time of MRI acquisition, k-space under-samplings is required during the acquisition processing. Thus, MRI reconstruction, which transfers undersampled k-space data to high-quality magnetic resonance imaging, becomes an important and meaningful task. There have been many explorations on k-space interpolation for MRI reconstruction. However, most of these methods ignore the strong correlation between target slice and its adjacent slices. Inspired by this, we propose a fully data-driven deep learning algorithm for k-space interpolation, utilizing the correlation information between the target slice and its neighboring slices. A novel network is proposed, which models the inter-dependencies between different slices. In addition, the network is easily implemented and expended. Experiments show that our methods consistently surpass existing image-domain and k-space-domain magnetic resonance imaging reconstructing methods.

J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction

Modern MRI schemes, which rely on compressed sensing or deep learning algorithms to recover MRI data from undersampled multichannel Fourier measurements, are widely used to reduce the scan time. The image quality of these approaches is heavily dependent on the sampling pattern. We introduce a continuous strategy to optimize the sampling pattern and the network parameters jointly. We use a multichannel forward model, consisting of a non-uniform Fourier transform with continuously defined sampling locations, to realize the data consistency block within a model-based deep learning image reconstruction scheme. This approach facilitates the joint and continuous optimization of the sampling pattern and the CNN parameters to improve image quality. We observe that the joint optimization of the sampling patterns and the reconstruction module significantly improves the performance of most deep learning reconstruction algorithms. The source code is available at https://github.com/hkaggarwal/J-MoDL.

Non-contrast coronary magnetic resonance angiography: current frontiers and future horizons

Coronary magnetic resonance angiography (coronary MRA) is advantageous in its ability to assess coronary artery morphology and function without ionizing radiation or contrast media. However, technical limitations including reduced spatial resolution, long acquisition times, and low signal-to-noise ratios prevent it from clinical routine utilization. Nonetheless, each of these limitations can be specifically addressed by a combination of novel technologies including super-resolution imaging, compressed sensing, and deep-learning reconstruction. In this paper, we first review the current clinical use and motivations for non-contrast coronary MRA, discuss currently available coronary MRA techniques, and highlight current technical developments that hold unique potential to optimize coronary MRA image acquisition and post-processing. In the final section, we examine the various research-based coronary MRA methods and metrics that can be leveraged to assess coronary stenosis severity, physiological function, and atherosclerotic plaque characterization. We specifically discuss how such technologies may contribute to the clinical translation of coronary MRA into a robust modality for routine clinical use.

Volumetric imaging of myelin in vivo using 3D inversion recovery-prepared ultrashort echo time cones magnetic resonance imaging

Direct myelin imaging is promising for characterization of multiple sclerosis (MS) brains at diagnosis and in response to therapy. In this study, a 3D inversion recovery-prepared ultrashort echo time cones (IR-UTE-Cones) sequence was used for both morphological and quantitative imaging of myelin on a clinical 3 T scanner. Myelin powder phantoms with different myelin concentrations were imaged with the 3D UTE-Cones sequence and it showed a strong correlation between concentrations and UTE-Cones signals, demonstrating the ability of the UTE-Cones sequence to directly image myelin in the brain. Quantitative myelin imaging with multi-echo IR-UTE-Cones sequences show similar T₂ * values for a D₂ O-exchanged myelin phantom (T₂ * = 0.33 ± 0.04 ms), ex vivo brain specimens (T₂ * = 0.20 ± 0.04 ms) and in vivo healthy volunteers (T₂ * = 0.254 ± 0.023 ms), further confirming the feasibility of 3D IR-UTE-Cones sequences for direct myelin imaging in vivo. In ex vivo MS brain study, signal loss is observed in MS lesions, which was confirmed with histology. For the in vivo study, the lesions in MS patients also show myelin signal loss using the proposed direct myelin imaging method, demonstrating the clinical potential for MS diagnosis. Furthermore, the measured IR-UTE-Cones signal intensities show a significant difference between normal-appearing white matter in MS patients and normal white matter in volunteers, which cannot be found in clinical used T₂ -FLAIR sequences. Thus, the proposed 3D IR-UTE-Cones sequence showed clinical potential for MS diagnosis with the capability of direct myelin detection of the whole brain.

Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy

This brief review summarizes the major applications of artificial intelligence (AI), in particular deep learning approaches, in molecular imaging and radiation therapy research. To this end, the applications of artificial intelligence in five generic fields of molecular imaging and radiation therapy, including PET instrumentation design, PET image reconstruction quantification and segmentation, image denoising (low-dose imaging), radiation dosimetry and computer-aided diagnosis, and outcome prediction are discussed. This review sets out to cover briefly the fundamental concepts of AI and deep learning followed by a presentation of seminal achievements and the challenges facing their adoption in clinical setting.

Deriving new soft tissue contrasts from conventional MR images using deep learning

Versatile soft tissue contrast in magnetic resonance imaging is a unique advantage of the imaging modality. However, the versatility is not fully exploited. In this study, we propose a deep learning-based strategy to derive more soft tissue contrasts from conventional MR images obtained in standard clinical MRI. Two types of experiments are performed. First, MR images corresponding to different pulse sequences are predicted from one or more images already acquired. As an example, we predict T_1ρ weighted knee image from T₂ weighted image and/or T₁ weighted image. Furthermore, we estimate images corresponding to alternative imaging parameter values. In a representative case, variable flip angle images are predicted from a single T₁ weighted image, whose accuracy is further validated in quantitative T₁ map subsequently derived. To accomplish these tasks, images are retrospectively collected from 56 subjects, and self-attention convolutional neural network models are trained using 1104 knee images from 46 subjects and tested using 240 images from 10 other subjects. High accuracy has been achieved in resultant qualitative images as well as quantitative T₁ maps. The proposed deep learning method can be broadly applied to obtain more versatile soft tissue contrasts without additional scans or used to normalize MR data that were inconsistently acquired for quantitative analysis.

Artificial intelligence in stroke imaging: Current and future perspectives

Artificial intelligence (AI) is a fast-growing research area in computer science that aims to mimic cognitive processes through a number of techniques. Supervised machine learning, a subfield of AI, includes methods that can identify patterns in high-dimensional data using labeled 'ground truth' data and apply these learnt patterns to analyze, interpret, or make predictions on new datasets. Supervised machine learning has become a significant area of interest within the medical community. Radiology and neuroradiology in particular are especially well suited for application of machine learning due to the vast amount of data that is generated. One devastating disease for which neuroimaging plays a significant role in the clinical management is stroke. Within this context, AI techniques can play pivotal roles for image-based diagnosis and management of stroke. This overview focuses on the recent advances of artificial intelligence methods - particularly supervised machine learning and deep learning - with respect to workflow, image acquisition and reconstruction, and image interpretation in patients with acute stroke, while also discussing potential pitfalls and future applications.

Domain Transform Network for Photoacoustic Tomography from Limited-view and Sparsely Sampled Data

Medical image reconstruction methods based on deep learning have recently demonstrated powerful performance in photoacoustic tomography (PAT) from limited-view and sparse data. However, because most of these methods must utilize conventional linear reconstruction methods to implement signal-to-image transformations, their performance is restricted. In this paper, we propose a novel deep learning reconstruction approach that integrates appropriate data pre-processing and training strategies. The Feature Projection Network (FPnet) presented herein is designed to learn this signal-to-image transformation through data-driven learning rather than through direct use of linear reconstruction. To further improve reconstruction results, our method integrates an image post-processing network (U-net). Experiments show that the proposed method can achieve high reconstruction quality from limited-view data with sparse measurements. When employing GPU acceleration, this method can achieve a reconstruction speed of 15 frames per second.

An Attribute-Weighted Bayes Classifier Based on Asymmetric Correlation Coefficient

In this research, an attribute-weighted one-dependence Bayes estimation algorithm based on the asymmetric correlation coefficient is proposed. The asymmetric correlation coefficients Tau_y and Lambda_y, respectively, are used to calculate the correlation between parent attributes and category labels, then the result of calculation is regarded as weight to the parent attribute. The algorithm is applied to eight types of different datasets including binary classification and multiple classification from the UCI database. By comparing the time complexity and classification accuracy, experimental results show that the algorithm can significantly improve the classification performance with less prediction error. In addition, several baseline methods such as KNN, ANN, logistic regression and SVM are used for comparison with the proposed method.

Joint multi-contrast variational network reconstruction (jVN) with application to rapid 2D and 3D imaging

PURPOSE

To improve the image quality of highly accelerated multi-channel MRI data by learning a joint variational network that reconstructs multiple clinical contrasts jointly.

METHODS

Data from our multi-contrast acquisition were embedded into the variational network architecture where shared anatomical information is exchanged by mixing the input contrasts. Complementary k-space sampling across imaging contrasts and Bunch-Phase/Wave-Encoding were used for data acquisition to improve the reconstruction at high accelerations. At 3T, our joint variational network approach across T1w, T2w and T2-FLAIR-weighted brain scans was tested for retrospective under-sampling at R = 6 (2D) and R = 4 × 4 (3D) acceleration. Prospective acceleration was also performed for 3D data where the combined acquisition time for whole brain coverage at 1 mm isotropic resolution across three contrasts was less than 3 min.

RESULTS

Across all test datasets, our joint multi-contrast network better preserved fine anatomical details with reduced image-blurring when compared to the corresponding single-contrast reconstructions. Improvement in image quality was also obtained through complementary k-space sampling and Bunch-Phase/Wave-Encoding where the synergistic combination yielded the overall best performance as evidenced by exemplary slices and quantitative error metrics.

CONCLUSION

By leveraging shared anatomical structures across the jointly reconstructed scans, our joint multi-contrast approach learnt more efficient regularizers, which helped to retain natural image appearance and avoid over-smoothing. When synergistically combined with advanced encoding techniques, the performance was further improved, enabling up to R = 16-fold acceleration with good image quality. This should help pave the way to very rapid high-resolution brain exams.

Neural Architecture Search for compressed sensing Magnetic Resonance image reconstruction

Recent works have demonstrated that deep learning (DL) based compressed sensing (CS) implementation can accelerate Magnetic Resonance (MR) Imaging by reconstructing MR images from sub-sampled k-space data. However, network architectures adopted in previous methods are all designed by handcraft. Neural Architecture Search (NAS) algorithms can automatically build neural network architectures which have outperformed human designed ones in several vision tasks. Inspired by this, here we proposed a novel and efficient network for the MR image reconstruction problem via NAS instead of manual attempts. Particularly, a specific cell structure, which was integrated into the model-driven MR reconstruction pipeline, was automatically searched from a flexible pre-defined operation search space in a differentiable manner. Experimental results show that our searched network can produce better reconstruction results compared to previous state-of-the-art methods in terms of PSNR and SSIM with 4∼6 times fewer computation resources. Extensive experiments were conducted to analyze how hyper-parameters affect reconstruction performance and the searched structures. The generalizability of the searched architecture was also evaluated on different organ MR datasets. Our proposed method can reach a better trade-off between computation cost and reconstruction performance for MR reconstruction problem with good generalizability and offer insights to design neural networks for other medical image applications. The evaluation code will be available at https://github.com/yjump/NAS-for-CSMRI.

Artificial intelligence in radiation oncology

Artificial intelligence (AI) has the potential to fundamentally alter the way medicine is practised. AI platforms excel in recognizing complex patterns in medical data and provide a quantitative, rather than purely qualitative, assessment of clinical conditions. Accordingly, AI could have particularly transformative applications in radiation oncology given the multifaceted and highly technical nature of this field of medicine with a heavy reliance on digital data processing and computer software. Indeed, AI has the potential to improve the accuracy, precision, efficiency and overall quality of radiation therapy for patients with cancer. In this Perspective, we first provide a general description of AI methods, followed by a high-level overview of the radiation therapy workflow with discussion of the implications that AI is likely to have on each step of this process. Finally, we describe the challenges associated with the clinical development and implementation of AI platforms in radiation oncology and provide our perspective on how these platforms might change the roles of radiotherapy medical professionals.

Deep-learning-based image quality enhancement of compressed sensing magnetic resonance imaging of vessel wall: comparison of self-supervised and unsupervised approaches

While high-resolution proton density-weighted magnetic resonance imaging (MRI) of intracranial vessel walls is significant for a precise diagnosis of intracranial artery disease, its long acquisition time is a clinical burden. Compressed sensing MRI is a prospective technology with acceleration factors that could potentially reduce the scan time. However, high acceleration factors result in degraded image quality. Although recent advances in deep-learning-based image restoration algorithms can alleviate this problem, clinical image pairs used in deep learning training typically do not align pixel-wise. Therefore, in this study, two different deep-learning-based denoising algorithms-self-supervised learning and unsupervised learning-are proposed; these algorithms are applicable to clinical datasets that are not aligned pixel-wise. The two approaches are compared quantitatively and qualitatively. Both methods produced promising results in terms of image denoising and visual grading. While the image noise and signal-to-noise ratio of self-supervised learning were superior to those of unsupervised learning, unsupervised learning was preferable over self-supervised learning in terms of radiomic feature reproducibility.

CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions

Cardiac CINE magnetic resonance imaging is the gold-standard for the assessment of cardiac function. Imaging accelerations have shown to enable 3D CINE with left ventricular (LV) coverage in a single breath-hold. However, 3D imaging remains limited to anisotropic resolution and long reconstruction times. Recently deep learning has shown promising results for computationally efficient reconstructions of highly accelerated 2D CINE imaging. In this work, we propose a novel 4D (3D + time) deep learning-based reconstruction network, termed 4D CINENet, for prospectively undersampled 3D Cartesian CINE imaging. CINENet is based on (3 + 1)D complex-valued spatio-temporal convolutions and multi-coil data processing. We trained and evaluated the proposed CINENet on in-house acquired 3D CINE data of 20 healthy subjects and 15 patients with suspected cardiovascular disease. The proposed CINENet network outperforms iterative reconstructions in visual image quality and contrast (+ 67% improvement). We found good agreement in LV function (bias ± 95% confidence) in terms of end-systolic volume (0 ± 3.3 ml), end-diastolic volume (− 0.4 ± 2.0 ml) and ejection fraction (0.1 ± 3.2%) compared to clinical gold-standard 2D CINE, enabling single breath-hold isotropic 3D CINE in less than 10 s scan and ~ 5 s reconstruction time.

Myelin Imaging in Human Brain Using a Short Repetition Time Adiabatic Inversion Recovery Prepared Ultrashort Echo Time (STAIR-UTE) MRI Sequence in Multiple Sclerosis

Background Water signal contamination is a major challenge for direct ultrashort echo time (UTE) imaging of myelin in vivo because water contributes most of the signals detected in white matter. Purpose To validate a new short repetition time (TR) adiabatic inversion recovery (STAIR) prepared UTE (STAIR-UTE) sequence designed to suppress water signals and to allow imaging of ultrashort T2 protons of myelin in white matter using a clinical 3-T scanner. Materials and Methods In this prospective study, an optimization framework was used to obtain the optimal inversion time for nulling water signals using STAIR-UTE imaging at different TRs. Numeric simulation and phantom studies were performed. Healthy volunteers and participants with multiple sclerosis (MS) underwent MRI between November 2018 and October 2019 to compare STAIR-UTE and a clinical T2-weighted fluid-attenuated inversion recovery sequence for assessment of MS lesions. UTE measures of myelin were also performed to allow comparison of signals in lesions and with those in normal-appearing white matter (NAWM) in patients with MS and in normal white matter (NWM) in healthy volunteers. Results Simulation and phantom studies both suggest that the proposed STAIR-UTE technique can effectively suppress long T2 tissues with a broad range of T1s. Ten healthy volunteers (mean age, 33 years ± 8 [standard deviation]; six women) and 10 patients with MS (mean age, 51 years ± 16; seven women) were evaluated. The three-dimensional STAIR-UTE sequence effectively suppressed water components in white matter and selectively imaged myelin, which had a measured T2* value of 0.21 msec ± 0.04 in the volunteer study. A much lower mean UTE measure of myelin proton density was found in MS lesions (3.8 mol/L ± 1.5), and a slightly lower mean UTE measure was found in NAWM (7.2 mol/L ± 0.8) compared with that in NWM (8.0 mol/L ± 0.8) in the healthy volunteers (P < .001 for both comparisons). Conclusion The short repetition time adiabatic inversion recovery-prepared ultrashort echo time sequence provided efficient water signal suppression for volumetric imaging of myelin in the brain and showed excellent myelin signal contrast as well as marked ultrashort echo time signal reduction in multiple sclerosis lesions and a smaller reduction in normal-appearing white matter compared with normal white matter in volunteers. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Messina and Port in this issue.

Deep learning-based image reconstruction and motion estimation from undersampled radial k-space for real-time MRI-guided radiotherapy

To enable magnetic resonance imaging (MRI)-guided radiotherapy with real-time adaptation, motion must be quickly estimated with low latency. The motion estimate is used to adapt the radiation beam to the current anatomy, yielding a more conformal dose distribution. As the MR acquisition is the largest component of latency, deep learning (DL) may reduce the total latency by enabling much higher undersampling factors compared to conventional reconstruction and motion estimation methods. The benefit of DL on image reconstruction and motion estimation was investigated for obtaining accurate deformation vector fields (DVFs) with high temporal resolution and minimal latency. 2D cine MRI acquired at 1.5 T from 135 abdominal cancer patients were retrospectively included in this study. Undersampled radial golden angle acquisitions were retrospectively simulated. DVFs were computed using different combinations of conventional- and DL-based methods for image reconstruction and motion estimation, allowing a comparison of four approaches to achieve real-time motion estimation. The four approaches were evaluated based on the end-point-error and root-mean-square error compared to a ground-truth optical flow estimate on fully-sampled images, the structural similarity (SSIM) after registration and time necessary to acquire k-space, reconstruct an image and estimate motion. The lowest DVF error and highest SSIM were obtained using conventional methods up to [Formula: see text]. For undersampling factors [Formula: see text], the lowest DVF error and highest SSIM were obtained using conventional image reconstruction and DL-based motion estimation. We have found that, with this combination, accurate DVFs can be obtained up to [Formula: see text] with an average root-mean-square error up to 1 millimeter and an SSIM greater than 0.8 after registration, taking 60 milliseconds. High-quality 2D DVFs from highly undersampled k-space can be obtained with a high temporal resolution with conventional image reconstruction and a deep learning-based motion estimation approach for real-time adaptive MRI-guided radiotherapy.

Artifact removal using a hybrid-domain convolutional neural network for limited-angle computed tomography imaging

The suppression of streak artifacts in computed tomography with a limited-angle configuration is challenging. Conventional analytical algorithms, such as filtered backprojection (FBP), are not successful due to incomplete projection data. Moreover, model-based iterative total variation algorithms effectively reduce small streaks but do not work well at eliminating large streaks. In contrast, FBP mapping networks and deep-learning-based postprocessing networks are outstanding at removing large streak artifacts; however, these methods perform processing in separate domains, and the advantages of multiple deep learning algorithms operating in different domains have not been simultaneously explored. In this paper, we present a hybrid-domain convolutional neural network (hdNet) for the reduction of streak artifacts in limited-angle computed tomography. The network consists of three components: the first component is a convolutional neural network operating in the sinogram domain, the second is a domain transformation operation, and the last is a convolutional neural network operating in the CT image domain. After training the network, we can obtain artifact-suppressed CT images directly from the sinogram domain. Verification results based on numerical, experimental and clinical data confirm that the proposed method can significantly reduce serious artifacts.

Enhancing the X-Ray Differential Phase Contrast Image Quality With Deep Learning Technique

OBJECTIVE

The purpose of this work is to investigate the feasibility of using deep convolutional neural network (CNN) to improve the image quality of a grating-based X-ray differential phase contrast imaging (XPCI) system.

METHODS

In this work, a novel deep CNN based phase signal extraction and image noise suppression algorithm (named as XP-NET) is developed. The numerical phase phantom, the ex vivo biological specimen and the ACR breast phantom are evaluated via the numerical simulations and experimental studies, separately. Moreover, images are also evaluated under different low radiation levels to verify its dose reduction capability.

RESULTS

Compared with the conventional analytical method, the novel XP-NET algorithm is able to reduce the bias of large DPC signals and hence increasing the DPC signal accuracy by more than 15%. Additionally, the XP-NET is able to reduce DPC image noise by about 50% for low dose DPC imaging tasks.

CONCLUSION

This proposed novel end-to-end supervised XP-NET has a great potential to improve the DPC signal accuracy, reduce image noise, and preserve object details.

SIGNIFICANCE

We demonstrate that the deep CNN technique provides a promising approach to improve the grating-based XPCI performance and its dose efficiency in future biomedical applications.