Image reconstruction by domain-transform manifold learning

Data-driven synthetic MRI FLAIR artifact correction via deep neural network

BACKGROUND

FLAIR (fluid attenuated inversion recovery) imaging via synthetic MRI methods leads to artifacts in the brain, which can cause diagnostic limitations. The main sources of the artifacts are attributed to the partial volume effect and flow, which are difficult to correct by analytical modeling. In this study, a deep learning (DL)-based synthetic FLAIR method was developed, which does not require analytical modeling of the signal.

PURPOSE

To correct artifacts in synthetic FLAIR using a DL method.

STUDY TYPE

Retrospective.

SUBJECTS

A total of 80 subjects with clinical indications (60.6 ± 16.7 years, 38 males, 42 females) were divided into three groups: a training set (56 subjects, 62.1 ± 14.8 years, 25 males, 31 females), a validation set (1 subject, 62 years, male), and the testing set (23 subjects, 57.3 ± 20.4 years, 13 males, 10 females).

FIELD STRENGTH/SEQUENCE

3 T MRI using a multiple-dynamic multiple-echo acquisition (MDME) sequence for synthetic MRI and a conventional FLAIR sequence.

ASSESSMENT

Normalized root mean square (NRMSE) and structural similarity (SSIM) were computed for uncorrected synthetic FLAIR and DL-corrected FLAIR. In addition, three neuroradiologists scored the three FLAIR datasets blindly, evaluating image quality and artifacts for sulci/periventricular and intraventricular/cistern space regions.

STATISTICAL TESTS

Pairwise Student's t-tests and a Wilcoxon test were performed.

RESULTS

For quantitative assessment, NRMSE improved from 4.2% to 2.9% (P < 0.0001) and SSIM improved from 0.85 to 0.93 (P < 0.0001). Additionally, NRMSE values significantly improved from 1.58% to 1.26% (P < 0.001), 3.1% to 1.5% (P < 0.0001), and 2.7% to 1.4% (P < 0.0001) in white matter, gray matter, and cerebral spinal fluid (CSF) regions, respectively, when using DL-corrected FLAIR. For qualitative assessment, DL correction achieved improved overall quality, fewer artifacts in sulci and periventricular regions, and in intraventricular and cistern space regions.

DATA CONCLUSION

The DL approach provides a promising method to correct artifacts in synthetic FLAIR.

LEVEL OF EVIDENCE

4 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2019;50:1413-1423.

MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping

PURPOSE

To develop and evaluate a novel deep learning-based image reconstruction approach called MANTIS (Model-Augmented Neural neTwork with Incoherent k-space Sampling) for efficient MR parameter mapping.

METHODS

MANTIS combines end-to-end convolutional neural network (CNN) mapping, incoherent k-space undersampling, and a physical model as a synergistic framework. The CNN mapping directly converts a series of undersampled images straight into MR parameter maps using supervised training. Signal model fidelity is enforced by adding a pathway between the undersampled k-space and estimated parameter maps to ensure that the parameter maps produced synthesized k-space consistent with the acquired undersampling measurements. The MANTIS framework was evaluated on the T₂ mapping of the knee at different acceleration rates and was compared with 2 other CNN mapping methods and conventional sparsity-based iterative reconstruction approaches. Global quantitative assessment and regional T₂ analysis for the cartilage and meniscus were performed to demonstrate the reconstruction performance of MANTIS.

RESULTS

MANTIS achieved high-quality T₂ mapping at both moderate (R = 5) and high (R = 8) acceleration rates. Compared to conventional reconstruction approaches that exploited image sparsity, MANTIS yielded lower errors (normalized root mean square error of 6.1% for R = 5 and 7.1% for R = 8) and higher similarity (structural similarity index of 86.2% at R = 5 and 82.1% at R = 8) to the reference in the T₂ estimation. MANTIS also achieved superior performance compared to direct CNN mapping and a 2-step CNN method.

CONCLUSION

The MANTIS framework, with a combination of end-to-end CNN mapping, signal model-augmented data consistency, and incoherent k-space sampling, is a promising approach for efficient and robust estimation of quantitative MR parameters.

Artificial intelligence in cancer imaging: Clinical challenges and applications

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

Noncontrast MR angiography: An update

Both computed tomography (CT) angiography (CTA) and contrast-enhanced MR angiography (CEMRA) have proven to be useful and accurate cross-sectional imaging modalities over a wide range of vascular territories and vascular disorders. A key advantage of MRA is that, unlike CTA, it can be performed without the administration of a contrast agent. In this review article we consider the motivations for using noncontrast MRA, potential contrast mechanisms, imaging techniques, advantages, and drawbacks with respect to CTA and CEMRA, and the level of evidence for using the various MRA techniques. In addition, we explore new developments that promise to expand the reliability and range of clinical applications for noncontrast MRA, along with functional MRA capabilities not available with CTA or CEMRA. Level of Evidence: 1 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2019;49:355-373.

High-performance medicine: the convergence of human and artificial intelligence

The use of artificial intelligence, and the deep-learning subtype in particular, has been enabled by the use of labeled big data, along with markedly enhanced computing power and cloud storage, across all sectors. In medicine, this is beginning to have an impact at three levels: for clinicians, predominantly via rapid, accurate image interpretation; for health systems, by improving workflow and the potential for reducing medical errors; and for patients, by enabling them to process their own data to promote health. The current limitations, including bias, privacy and security, and lack of transparency, along with the future directions of these applications will be discussed in this article. Over time, marked improvements in accuracy, productivity, and workflow will likely be actualized, but whether that will be used to improve the patient-doctor relationship or facilitate its erosion remains to be seen.

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Improvement of image quality at CT and MRI using deep learning

Deep learning has been developed by computer scientists. Here, we discuss techniques for improving the image quality of diagnostic computed tomography and magnetic resonance imaging with the aid of deep learning. We categorize the techniques for improving the image quality as "noise and artifact reduction", "super resolution" and "image acquisition and reconstruction". For each category, we present and outline the features of some studies.

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.

Real-time coherent diffraction inversion using deep generative networks

Phase retrieval, or the process of recovering phase information in reciprocal space to reconstruct images from measured intensity alone, is the underlying basis to a variety of imaging applications including coherent diffraction imaging (CDI). Typical phase retrieval algorithms are iterative in nature, and hence, are time-consuming and computationally expensive, making real-time imaging a challenge. Furthermore, iterative phase retrieval algorithms struggle to converge to the correct solution especially in the presence of strong phase structures. In this work, we demonstrate the training and testing of CDI NN, a pair of deep deconvolutional networks trained to predict structure and phase in real space of a 2D object from its corresponding far-field diffraction intensities alone. Once trained, CDI NN can invert a diffraction pattern to an image within a few milliseconds of compute time on a standard desktop machine, opening the door to real-time imaging.

Rapid compositional mapping of knee cartilage with compressed sensing MRI

More than a decade after the introduction of compressed sensing (CS) in MRI, researchers are still working on ways to translate it into different research and clinical applications. The greatest advantage of CS in MRI is the reduced amount of k-space data needed to reconstruct images, which can be exploited to reduce scan time or to improve spatial resolution and volumetric coverage. Efficient data acquisition using CS is extremely important for compositional mapping of the musculoskeletal system in general and knee cartilage mapping techniques in particular. High-resolution quantitative information about tissue biochemical composition could be obtained in just a few minutes using CS MRI. However, in order to make this goal a reality, some issues still need to be addressed. In this article we review the current state of the art of CS methods for rapid compositional mapping of knee cartilage. Specifically, data acquisition strategies, image reconstruction algorithms, and data fitting models are discussed. Different CS studies for T2 and T1ρ mapping of knee cartilage are reviewed, with illustrative results. Future directions, opportunities, and challenges of rapid compositional mapping techniques are also discussed. Level of Evidence: 4 Technical Efficacy: Stage 6 J. Magn. Reson. Imaging 2018;47:1185-1198.

A framework for constraining image SNR loss due to MR raw data compression

INTRODUCTION

Computationally intensive image reconstruction algorithms can be used online during MRI exams by streaming data to remote high-performance computers. However, data acquisition rates often exceed the bandwidth of the available network resources creating a bottleneck. Data compression is, therefore, desired to ensure fast data transmission.

METHODS

The added noise variance due to compression was determined through statistical analysis for two compression libraries (one custom and one generic) that were implemented in this framework. Limiting the compression error variance relative to the measured thermal noise allowed for image signal-to-noise ratio loss to be explicitly constrained.

RESULTS

Achievable compression ratios are dependent on image SNR, user-defined SNR loss tolerance, and acquisition type. However, a 1% reduction in SNR yields approximately four to ninefold compression ratios across MRI acquisition strategies. For free-breathing cine data reconstructed in the cloud, the streaming bandwidth was reduced from 37 to 6.1 MB/s, alleviating the network transmission bottleneck.

CONCLUSION

Our framework enabled data compression for online reconstructions and allowed SNR loss to be constrained based on a user-defined SNR tolerance. This practical tool will enable real-time data streaming and greater than fourfold faster cloud upload times.

Deep neural network predicts emotional responses of the human brain from functional magnetic resonance imaging

An artificial neural network with multiple hidden layers (known as a deep neural network, or DNN) was employed as a predictive model (DNN_p) for the first time to predict emotional responses using whole-brain functional magnetic resonance imaging (fMRI) data from individual subjects. During fMRI data acquisition, 10 healthy participants listened to 80 International Affective Digital Sound stimuli and rated their own emotions generated by each sound stimulus in terms of the arousal, dominance, and valence dimensions. The whole-brain spatial patterns from a general linear model (i.e., beta-valued maps) for each sound stimulus and the emotional response ratings were used as the input and output for the DNN_P, respectively. Based on a nested five-fold cross-validation scheme, the paired input and output data were divided into training (three-fold), validation (one-fold), and test (one-fold) data. The DNN_P was trained and optimized using the training and validation data and was tested using the test data. The Pearson's correlation coefficients between the rated and predicted emotional responses from our DNN_P model with weight sparsity optimization (mean ± standard error 0.52 ± 0.02 for arousal, 0.51 ± 0.03 for dominance, and 0.51 ± 0.03 for valence, with an input denoising level of 0.3 and a mini-batch size of 1) were significantly greater than those of DNN models with conventional regularization schemes including elastic net regularization (0.15 ± 0.05, 0.15 ± 0.06, and 0.21 ± 0.04 for arousal, dominance, and valence, respectively), those of shallow models including logistic regression (0.11 ± 0.04, 0.10 ± 0.05, and 0.17 ± 0.04 for arousal, dominance, and valence, respectively; average of logistic regression and sparse logistic regression), and those of support vector machine-based predictive models (SVM_ps; 0.12 ± 0.06, 0.06 ± 0.06, and 0.10 ± 0.06 for arousal, dominance, and valence, respectively; average of linear and non-linear SVM_ps). This difference was confirmed to be significant with a Bonferroni-corrected p-value of less than 0.001 from a one-way analysis of variance (ANOVA) and subsequent paired t-test. The weights of the trained DNN_Ps were interpreted and input patterns that maximized or minimized the output of the DNN_Ps (i.e., the emotional responses) were estimated. Based on a binary classification of each emotion category (e.g., high arousal vs. low arousal), the error rates for the DNN_P (31.2% ± 1.3% for arousal, 29.0% ± 1.7% for dominance, and 28.6% ± 3.0% for valence) were significantly lower than those for the linear SVM_P (44.7% ± 2.0%, 50.7% ± 1.7%, and 47.4% ± 1.9% for arousal, dominance, and valence, respectively) and the non-linear SVM_P (48.8% ± 2.3%, 52.2% ± 1.9%, and 46.4% ± 1.3% for arousal, dominance, and valence, respectively), as confirmed by the Bonferroni-corrected p < 0.001 from the one-way ANOVA. Our study demonstrates that the DNN_p model is able to reveal neuronal circuitry associated with human emotional processing - including structures in the limbic and paralimbic areas, which include the amygdala, prefrontal areas, anterior cingulate cortex, insula, and caudate. Our DNN_p model was also able to use activation patterns in these structures to predict and classify emotional responses to stimuli.

Deep Learning in Neuroradiology

Deep learning is a form of machine learning using a convolutional neural network architecture that shows tremendous promise for imaging applications. It is increasingly being adapted from its original demonstration in computer vision applications to medical imaging. Because of the high volume and wealth of multimodal imaging information acquired in typical studies, neuroradiology is poised to be an early adopter of deep learning. Compelling deep learning research applications have been demonstrated, and their use is likely to grow rapidly. This review article describes the reasons, outlines the basic methods used to train and test deep learning models, and presents a brief overview of current and potential clinical applications with an emphasis on how they are likely to change future neuroradiology practice. Facility with these methods among neuroimaging researchers and clinicians will be important to channel and harness the vast potential of this new method.

The potential for machine learning algorithms to improve and reduce the cost of 3-dimensional printing for surgical planning

INTRODUCTION

3D-printed anatomical models play an important role in medical and research settings. The recent successes of 3D anatomical models in healthcare have led many institutions to adopt the technology. However, there remain several issues that must be addressed before it can become more wide-spread. Of importance are the problems of cost and time of manufacturing. Machine learning (ML) could be utilized to solve these issues by streamlining the 3D modeling process through rapid medical image segmentation and improved patient selection and image acquisition. The current challenges, potential solutions, and future directions for ML and 3D anatomical modeling in healthcare are discussed.

AREAS COVERED

This review covers research articles in the field of machine learning as related to 3D anatomical modeling. Topics discussed include automated image segmentation, cost reduction, and related time constraints.

EXPERT COMMENTARY

ML-based segmentation of medical images could potentially improve the process of 3D anatomical modeling. However, until more research is done to validate these technologies in clinical practice, their impact on patient outcomes will remain unknown. We have the necessary computational tools to tackle the problems discussed. The difficulty now lies in our ability to collect sufficient data.

Single-shot ultrafast optical imaging

Single-shot ultrafast optical imaging can capture two-dimensional transient scenes in the optical spectral range at ≥100 million frames per second. This rapidly evolving field surpasses conventional pump-probe methods by possessing the real-time imaging capability, which is indispensable for recording non-repeatable and difficult-to-reproduce events and for understanding physical, chemical, and biological mechanisms. In this mini-review, we survey comprehensively the state-of-the-art single-shot ultrafast optical imaging. Based on the illumination requirement, we categorized the field into active-detection and passive-detection domains. Depending on the specific image acquisition and reconstruction strategies, these two categories are further divided into a total of six sub-categories. Under each sub-category, we describe operating principles, present representative cutting-edge techniques with a particular emphasis on their methodology and applications, and discuss their advantages and challenges. Finally, we envision prospects of technical advancement in this field.

Large T1 contrast enhancement using superparamagnetic nanoparticles in ultra-low field MRI

Superparamagnetic iron oxide nanoparticles (SPIONs) are widely investigated and utilized as magnetic resonance imaging (MRI) contrast and therapy agents due to their large magnetic moments. Local field inhomogeneities caused by these high magnetic moments are used to generate T₂ contrast in clinical high-field MRI, resulting in signal loss (darker contrast). Here we present strong T₁ contrast enhancement (brighter contrast) from SPIONs (diameters from 11 nm to 22 nm) as observed in the ultra-low field (ULF) MRI at 0.13 mT. We have achieved a high longitudinal relaxivity for 18 nm SPION solutions, r₁ = 615 s⁻¹ mM⁻¹, which is two orders of magnitude larger than typical commercial Gd-based T₁ contrast agents operating at high fields (1.5 T and 3 T). The significantly enhanced r₁ value at ultra-low fields is attributed to the coupling of proton spins with SPION magnetic fluctuations (Brownian and Néel) associated with a low frequency peak in the imaginary part of AC susceptibility (χ”). SPION-based T₁-weighted ULF MRI has the advantages of enhanced signal, shorter imaging times, and iron-oxide-based nontoxic biocompatible agents. This approach shows promise to become a functional imaging technique, similar to PET, where low spatial resolution is compensated for by important functional information.

Artificial intelligence in radiology

Artificial intelligence (AI) algorithms, particularly deep learning, have demonstrated remarkable progress in image-recognition tasks. Methods ranging from convolutional neural networks to variational autoencoders have found myriad applications in the medical image analysis field, propelling it forward at a rapid pace. Historically, in radiology practice, trained physicians visually assessed medical images for the detection, characterization and monitoring of diseases. AI methods excel at automatically recognizing complex patterns in imaging data and providing quantitative, rather than qualitative, assessments of radiographic characteristics. In this Opinion article, we establish a general understanding of AI methods, particularly those pertaining to image-based tasks. We explore how these methods could impact multiple facets of radiology, with a general focus on applications in oncology, and demonstrate ways in which these methods are advancing the field. Finally, we discuss the challenges facing clinical implementation and provide our perspective on how the domain could be advanced.

Opening the black box of neural networks: methods for interpreting neural network models in clinical applications

Artificial neural networks (ANNs) are powerful tools for data analysis and are particularly suitable for modeling relationships between variables for best prediction of an outcome. While these models can be used to answer many important research questions, their utility has been critically limited because the interpretation of the "black box" model is difficult. Clinical investigators usually employ ANN models to predict the clinical outcomes or to make a diagnosis; the model however is difficult to interpret for clinicians. To address this important shortcoming of neural network modeling methods, we describe several methods to help subject-matter audiences (e.g., clinicians, medical policy makers) understand neural network models. Garson's algorithm describes the relative magnitude of the importance of a descriptor (predictor) in its connection with outcome variables by dissecting the model weights. The Lek's profile method explores the relationship of the outcome variable and a predictor of interest, while holding other predictors at constant values (e.g., minimum, 20th quartile, maximum). While Lek's profile was developed specifically for neural networks, partial dependence plot is a more generic version that visualize the relationship between an outcome and one or two predictors. Finally, the local interpretable model-agnostic explanations (LIME) method can show the predictions of any classification or regression, by approximating it locally with an interpretable model. R code for the implementations of these methods is shown by using example data fitted with a standard, feed-forward neural network model. We offer codes and step-by-step description on how to use these tools to facilitate better understanding of ANN.

Magnetic Resonance Imaging technology-bridging the gap between noninvasive human imaging and optical microscopy

Technological advances in Magnetic Resonance Imaging (MRI) have provided substantial gains in the sensitivity and specificity of functional neuroimaging. Mounting evidence demonstrates that the hemodynamic changes utilized in functional MRI can be far more spatially and thus neuronally specific than previously believed. This has motivated a push toward novel, high-resolution MR imaging strategies that can match this biological resolution limit while recording from the entire human brain. Although sensitivity increases are a necessary component, new MR encoding technologies are required to convert improved sensitivity into higher resolution. These new sampling strategies improve image acquisition efficiency and enable increased image encoding in the time-frame needed to follow hemodynamic changes associated with brain activation.

Deep learning enables reduced gadolinium dose for contrast-enhanced brain MRI

BACKGROUND

There are concerns over gadolinium deposition from gadolinium-based contrast agents (GBCA) administration.

PURPOSE

To reduce gadolinium dose in contrast-enhanced brain MRI using a deep learning method.

STUDY TYPE

Retrospective, crossover.

POPULATION

Sixty patients receiving clinically indicated contrast-enhanced brain MRI.

SEQUENCE

3D T₁ -weighted inversion-recovery prepped fast-spoiled-gradient-echo (IR-FSPGR) imaging was acquired at both 1.5T and 3T. In 60 brain MRI exams, the IR-FSPGR sequence was obtained under three conditions: precontrast, postcontrast images with 10% low-dose (0.01mmol/kg) and 100% full-dose (0.1 mmol/kg) of gadobenate dimeglumine. We trained a deep learning model using the first 10 cases (with mixed indications) to approximate full-dose images from the precontrast and low-dose images. Synthesized full-dose images were created using the trained model in two test sets: 20 patients with mixed indications and 30 patients with glioma.

ASSESSMENT

For both test sets, low-dose, true full-dose, and the synthesized full-dose postcontrast image sets were compared quantitatively using peak-signal-to-noise-ratios (PSNR) and structural-similarity-index (SSIM). For the test set comprised of 20 patients with mixed indications, two neuroradiologists scored blindly and independently for the three postcontrast image sets, evaluating image quality, motion-artifact suppression, and contrast enhancement compared with precontrast images.

STATISTICAL ANALYSIS

Results were assessed using paired t-tests and noninferiority tests.

RESULTS

The proposed deep learning method yielded significant (n = 50, P < 0.001) improvements over the low-dose images (>5 dB PSNR gains and >11.0% SSIM). Ratings on image quality (n = 20, P = 0.003) and contrast enhancement (n = 20, P < 0.001) were significantly increased. Compared to true full-dose images, the synthesized full-dose images have a slight but not significant reduction in image quality (n = 20, P = 0.083) and contrast enhancement (n = 20, P = 0.068). Slightly better (n = 20, P = 0.039) motion-artifact suppression was noted in the synthesized images. The noninferiority test rejects the inferiority of the synthesized to true full-dose images for image quality (95% CI: -14-9%), artifacts suppression (95% CI: -5-20%), and contrast enhancement (95% CI: -13-6%).

DATA CONCLUSION

With the proposed deep learning method, gadolinium dose can be reduced 10-fold while preserving contrast information and avoiding significant image quality degradation.

LEVEL OF EVIDENCE

3 Technical Efficacy: Stage 5 J. MAGN. RESON. IMAGING 2018;48:330-340.