Incorporation of a spectral model in a convolutional neural network for accelerated spectral fitting

Recurrent neural network-aided processing of incomplete free induction decays in 1H-MRS of the brain

In the case of limited sampling windows or truncation of free induction decays (FIDs) for artifact removal in proton magnetic resonance spectroscopy (1H-MRS) and spectroscopic imaging (1H-MRSI), metabolite quantification needs to be performed on incomplete FIDs. Given that FIDs are naturally time-domain sequential data, we investigated the potential of recurrent neural network (RNN)-types of neural networks (NNs) in the processing of incomplete human brain FIDs with or without FID restoration prior to quantitative analysis at 3.0T. First, we employed an RNN encoder-decoder and developed it to restore incomplete FIDs (rRNN) with different amounts of sampled data. The quantification of metabolites from the rRNN-restored FIDs was achieved by using LCModel. Second, we modified the RNN encoder-decoder and developed it to convert incomplete brain FIDs into incomplete metabolite-only FIDs without restoration, followed by linear regression using a metabolite basis set for quantitative analysis (cRNN). In consideration of the practical benefit of the FID restoration with respect to pure zero-filling, development and analysis of the NNs were focused particularly on the incomplete FIDs with only the first 64 data points retained. All NNs were trained on simulated data and tested mainly on in vivo data acquired from healthy volunteers (n = 27). Strong correlations were obtained between the NN-derived and ground truth metabolite content (LCModel-derived content on fully sampled FIDs) for myo-inositol, total choline, and total creatine (normalized to total N-acetylaspartate) on the in vivo data using both rRNN (R = 0.83-0.94; p ≤ 0.05) and cRNN (R = 0.86-0.91; p ≤ 0.05). RNN-types of NNs have potential in the quantification of the major brain metabolites from the FIDs with substantially reduced sampled data points. For the metabolites with low to medium SNR, the performance of the NNs needs to be further improved, for which development of more elaborate and advanced simulation techniques would be of help, but remains challenging.

TensorFit: A torch-based tool for ultrafast metabolite fitting of large MRSI data sets

PURPOSE

To introduce a tool (TensorFit) for ultrafast and robust metabolite fitting of MRSI data based on Torch's auto-differentiation and optimization framework.

METHODS

TensorFit was implemented in Python based on Torch's auto-differentiation to fit individual metabolites in MRS spectra. The underlying time domain and/or frequency domain fitting model is based on a linear combination of metabolite spectroscopic response. The computational time efficiency and accuracy of TensorFit were tested on simulated and in vivo MRS data and compared against TDFDFit and QUEST.

RESULTS

TensorFit demonstrates a significant improvement in computation speed, achieving a 165-times acceleration compared with TDFDFit and 115 times against QUEST. TensorFit showed smaller percentual errors on simulated data compared with TDFDFit and QUEST. When tested on in vivo data, it performed similarly to TDFDFit with a 2% better fit in terms of mean squared error while obtaining a 169-fold speedup.

CONCLUSION

TensorFit enables fast and robust metabolite fitting in large MRSI data sets compared with conventional metabolite fitting methods. This tool could boost the clinical applicability of large 3D MRSI by enabling the fitting of large MRSI data sets within computation times acceptable in a clinical environment.

Comparison of convolutional-neural-networks-based method and LCModel on the quantification of in vivo magnetic resonance spectroscopy

BACKGROUND

Quantification of metabolites concentrations in institutional unit (IU) is important for inter-subject and long-term comparisons in the applications of magnetic resonance spectroscopy (MRS). Recently, deep learning (DL) algorithms have found a variety of applications on the process of MRS data. A quantification strategy compatible to DL base MRS spectral processing method is, therefore, useful.

MATERIALS AND METHODS

This study aims to investigate whether metabolite concentrations quantified using a convolutional neural network (CNN) based method, coupled with a scaling procedure that normalizes spectral signals for CNN input and linear regression, can effectively reflect variations in metabolite concentrations in IU across different brain regions with varying signal-to-noise ratios (SNR) and linewidths (LW). An error index based on standard error (SE) is proposed to indicate the confidence levels associated with metabolite predictions. In vivo MRS spectra were acquired from three brain regions of 43 subjects using a 3T system.

RESULTS

The metabolite concentrations in IU of five major metabolites, quantified using CNN and LCModel, exhibit similar ranges with Pearson's correlation coefficients ranging from 0.24 to 0.78. The SE of the metabolites shows a positive correlation with Cramer-Rao lower bound (CRLB) (r=0.46) and  absolute CRLB (r=0.81), calculated by multiplying CRLBs with the quantified metabolite content.

CONCLUSION

In conclusion, the CNN based method with the proposed scaling procedures can be employed to quantify in vivo MRS spectra and derive metabolites concentrations in IU. The SE can be used as error index, indicating predicted uncertainties for metabolites and sharing information similar to the absolute CRLB.

Multi-Parametric Molecular Imaging of the Brain Using Optimized Multi-TE Subspace MRSI

OBJECTIVE

To develop a novel multi-TE MR spectroscopic imaging (MRSI) approach to enable label-free, simultaneous, high-resolution mapping of several molecules and their biophysical parameters in the brain.

METHODS

The proposed method uniquely integrated an augmented molecular-component-specific subspace model for multi-TE 1H-MRSI signals, an estimation-theoretic experiment optimization (nonuniform TE selection) for molecule separation and parameter estimation, a physics-driven subspace learning strategy for spatiospectral reconstruction and molecular quantification, and a new accelerated multi-TE MRSI acquisition for generating high-resolution data in clinically relevant times. Numerical studies, phantom and in vivo experiments were conducted to validate the optimized experiment design and demonstrate the imaging capability offered by the proposed method.

RESULTS

The proposed TE optimization improved estimation of metabolites, neurotransmitters and their T2's over conventional TE choices, e.g., reducing variances of neurotransmitter concentration by  ∼  40% and metabolite T2 by  ∼  60%. Simultaneous metabolite and neurotransmitter mapping of the brain can be achieved at a nominal resolution of 3.4 × 3.4 × 6.4 mm 3. High-resolution, 3D metabolite T2 mapping was made possible for the first time. The translational potential of the proposed method was demonstrated by mapping biochemical abnormality in a post-traumatic epilepsy (PTE) patient.

CONCLUSION

The feasibility for high-resolution mapping of metabolites/neurotransmitters and metabolite T2's within clinically relevant time was demonstrated. We expect our method to offer richer information for revealing and understanding metabolic alterations in neurological diseases.

SIGNIFICANCE

A novel multi-TE MRSI approach was presented that enhanced the technological capability of multi-parametric molecular imaging of the brain. The proposed method presents new technology development and application opportunities for providing richer molecular level information to uncover and comprehend metabolic changes relevant in various neurological applications.

Water removal in MR spectroscopic imaging with Casorati singular value decomposition

PURPOSE

Water removal is one of the computational bottlenecks in the processing of high-resolution MRSI data. The purpose of this work is to propose an approach to reduce the computing time required for water removal in large MRS data.

METHODS

In this work, we describe a singular value decomposition-based approach that uses the partial position-time separability and the time-domain linear predictability of MRSI data to reduce the computational time required for water removal. Our approach arranges MRS signals in a Casorati matrix form, applies low-rank approximations utilizing singular value decomposition, removes residual water from the most prominent left-singular vectors, and finally reconstructs the water-free matrix using the processed left-singular vectors.

RESULTS

We have demonstrated the effectiveness of our proposed algorithm for water removal using both simulated and in vivo data. The proposed algorithm encompasses a pip-installable tool ( https://pypi.org/project/CSVD/), available on GitHub ( https://github.com/amirshamaei/CSVD), empowering researchers to use it in future studies. Additionally, to further promote transparency and reproducibility, we provide comprehensive code for result replication.

CONCLUSIONS

The findings of this study suggest that the proposed method is a promising alternative to existing water removal methods due to its low processing time and good performance in removing water signals.

CloudBrain-MRS: An intelligent cloud computing platform for in vivo magnetic resonance spectroscopy preprocessing, quantification, and analysis

Magnetic resonance spectroscopy (MRS) is an important clinical imaging method for diagnosis of diseases. MRS spectrum is used to observe the signal intensity of metabolites or further infer their concentrations. Although the magnetic resonance vendors commonly provide basic functions of spectrum plots and metabolite quantification, the spread of clinical research of MRS is still limited due to the lack of easy-to-use processing software or platform. To address this issue, we have developed CloudBrain-MRS, a cloud-based online platform that provides powerful hardware and advanced algorithms. The platform can be accessed simply through a web browser, without the need of any program installation on the user side. CloudBrain-MRS also integrates the classic LCModel and advanced artificial intelligence algorithms and supports batch preprocessing, quantification, and analysis of MRS data from different vendors. Additionally, the platform offers useful functions: (1) Automatically statistical analysis to find biomarkers for diseases; (2) Consistency verification between the classic and artificial intelligence quantification algorithms; (3) Colorful three-dimensional visualization for easy observation of individual metabolite spectrum. Last, data of both healthy subjects and patients with mild cognitive impairment are used to demonstrate the functions of the platform. To the best of our knowledge, this is the first cloud computing platform for in vivo MRS with artificial intelligence processing. We have shared our cloud platform at MRSHub, providing at least two years of free access and service. If you are interested, please visit https://mrshub.org/software_all/#CloudBrain-MRS or https://csrc.xmu.edu.cn/CloudBrain.html.

Denoising single MR spectra by deep learning: Miracle or mirage?

PURPOSE

The inherently poor SNR of MRS measurements presents a significant hurdle to its clinical application. Denoising by machine or deep learning (DL) was proposed as a remedy. It is investigated whether such denoising leads to lower estimate uncertainties or whether it essentially reduces noise in signal-free areas only.

METHODS

Noise removal based on supervised DL with U-nets was implemented using simulated 1 H MR spectra of human brain in two approaches: (1) via time-frequency domain spectrograms and (2) using 1D spectra as input. Quality of denoising was evaluated in three ways: (1) by an adapted fit quality score, (2) by traditional model fitting, and (3) by quantification via neural networks.

RESULTS

Visually appealing spectra were obtained; hinting that denoising is well-suited for MRS. However, an adapted denoising score showed that noise removal is inhomogeneous and more efficient for signal-free areas. This was confirmed by quantitative analysis of traditional fit results as well as DL quantitation following DL denoising. DL denoising, although apparently successful as judged by mean squared errors, led to substantially biased estimates in both implementations.

CONCLUSION

The implemented DL-based denoising techniques may be useful for display purposes, but do not help quantitative evaluations, confirming expectations based on estimation theory: Cramér Rao lower bounds defined by the original data and the appropriate fitting model cannot be circumvented in an unbiased way for single data sets, unless additional prior knowledge can be incurred in the form of parameter restrictions/relations or applicable substates.

Quantification of spatially localized MRS by a novel deep learning approach without spectral fitting

PURPOSE

To propose a novel end-to-end deep learning model to quantify absolute metabolite concentrations from in vivo J-point resolved spectroscopy (JPRESS) without using spectral fitting.

METHODS

A novel encoder-decoder-style neural network was created, which was trained to predict metabolite concentrations and individual component signals concurrently from 3T JPRESS data in the time domain. The training data set contained 100 000 samples created by spin-density simulations using experimentally used RF pulses. Concentrations, phase, frequencies, linewidths, and T2 relaxation times in the training data set were varied over a large range with uniform distributions. Random synthesized noise and extraneous signals were added to the data set. Two thousand validation samples were created similarly to the training data set but with mean concentrations close to in vivo values. An in vivo test was conducted with 20 samples acquired from the human brain.

RESULTS

Both validation and in vivo test results showed that the proposed model successfully predicted metabolite concentrations as well as individual metabolite signals without involving spectral fitting, while extraneous peaks or unregistered signals were filtered out. Compared with the short-TE spectral fitting by LCModel, the proposed method had the advantage that the undesired correlations between the estimated concentrations and noise levels and between metabolites were eliminated or substantially reduced.

CONCLUSION

The proposed method provides a working deep learning model that directly maps in vivo JPRESS data to metabolite concentrations. Because spectral fitting is not used, the trained model does not depend on the assumptions associated with parameter tuning when applied to in vivo data.

Application of a 1H Brain MRS Benchmark Dataset to Deep Learning for Out-of-Voxel Artifacts

Neural networks are potentially valuable for many of the challenges associated with MRS data. The purpose of this manuscript is to describe the AGNOSTIC dataset, which contains 259,200 synthetic 1H MRS examples for training and testing neural networks. AGNOSTIC was created using 270 basis sets that were simulated across 18 field strengths and 15 echo times. The synthetic examples were produced to resemble in vivo brain data with combinations of metabolite, macromolecule, residual water signals, and noise. To demonstrate the utility, we apply AGNOSTIC to train two Convolutional Neural Networks (CNNs) to address out-of-voxel (OOV) echoes. A Detection Network was trained to identify the point-wise presence of OOV echoes, providing proof of concept for real-time detection. A Prediction Network was trained to reconstruct OOV echoes, allowing subtraction during post-processing. Complex OOV signals were mixed into 85% of synthetic examples to train two separate CNNs for the detection and prediction of OOV signals. AGNOSTIC is available through Dryad and all Python 3 code is available through GitHub. The Detection network was shown to perform well, identifying 95% of OOV echoes. Traditional modeling of these detected OOV signals was evaluated and may prove to be an effective method during linear-combination modeling. The Prediction Network greatly reduces OOV echoes within FIDs and achieved a median log10 normed-MSE of -1.79, an improvement of almost two orders of magnitude.

A review of machine learning applications for the proton MR spectroscopy workflow

This literature review presents a comprehensive overview of machine learning (ML) applications in proton MR spectroscopy (MRS). As the use of ML techniques in MRS continues to grow, this review aims to provide the MRS community with a structured overview of the state-of-the-art methods. Specifically, we examine and summarize studies published between 2017 and 2023 from major journals in the MR field. We categorize these studies based on a typical MRS workflow, including data acquisition, processing, analysis, and artificial data generation. Our review reveals that ML in MRS is still in its early stages, with a primary focus on processing and analysis techniques, and less attention given to data acquisition. We also found that many studies use similar model architectures, with little comparison to alternative architectures. Additionally, the generation of artificial data is a crucial topic, with no consistent method for its generation. Furthermore, many studies demonstrate that artificial data suffers from generalization issues when tested on in vivo data. We also conclude that risks related to ML models should be addressed, particularly for clinical applications. Therefore, output uncertainty measures and model biases are critical to investigate. Nonetheless, the rapid development of ML in MRS and the promising results from the reviewed studies justify further research in this field.

Quantification of MR spectra by deep learning in an idealized setting: Investigation of forms of input, network architectures, optimization by ensembles of networks, and training bias

PURPOSE

The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution.

METHODS

Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF).

RESULTS

Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value.

CONCLUSION

MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

Physics-informed deep learning approach to quantification of human brain metabolites from magnetic resonance spectroscopy data

PURPOSE

While the recommended analysis method for magnetic resonance spectroscopy data is linear combination model (LCM) fitting, the supervised deep learning (DL) approach for quantification of MR spectroscopy (MRS) and MR spectroscopic imaging (MRSI) data recently showed encouraging results; however, supervised learning requires ground truth fitted spectra, which is not practical. Moreover, this work investigates the feasibility and efficiency of the LCM-based self-supervised DL method for the analysis of MRS data.

METHOD

We present a novel DL-based method for the quantification of relative metabolite concentrations, using quantum-mechanics simulated metabolite responses and neural networks. We trained, validated, and evaluated the proposed networks with simulated and publicly accessible in-vivo human brain MRS data and compared the performance with traditional methods. A novel adaptive macromolecule fitting algorithm is included. We investigated the performance of the proposed methods in a Monte Carlo (MC) study.

RESULT

The validation using low-SNR simulated data demonstrated that the proposed methods could perform quantification comparably to other methods. The applicability of the proposed method for the quantification of in-vivo MRS data was demonstrated. Our proposed networks have the potential to reduce computation time significantly.

CONCLUSION

The proposed model-constrained deep neural networks trained in a self-supervised manner can offer fast and efficient quantification of MRS and MRSI data. Our proposed method has the potential to facilitate clinical practice by enabling faster processing of large datasets such as high-resolution MRSI datasets, which may have thousands of spectra.

Joint spectral quantification of MR spectroscopic imaging using linear tangent space alignment-based manifold learning

PURPOSE

To develop a manifold learning-based method that leverages the intrinsic low-dimensional structure of MR Spectroscopic Imaging (MRSI) signals for joint spectral quantification.

METHODS

A linear tangent space alignment (LTSA) model was proposed to represent MRSI signals. In the proposed model, the signals of each metabolite were represented using a subspace model and the local coordinates of the subspaces were aligned to the global coordinates of the underlying low-dimensional manifold via linear transform. With the basis functions of the subspaces predetermined via quantum mechanics simulations, the global coordinates and the matrices for the local-to-global coordinate alignment were estimated by fitting the proposed LTSA model to noisy MRSI data with a spatial smoothness constraint on the global coordinates and a sparsity constraint on the matrices.

RESULTS

The performance of the proposed method was validated using numerical simulation data and in vivo proton-MRSI experimental data acquired on healthy volunteers at 3T. The results of the proposed method were compared with the QUEST method and the subspace-based method. In all the compared cases, the proposed method achieved superior performance over the QUEST and the subspace-based methods both qualitatively in terms of noise and artifacts in the estimated metabolite concentration maps, and quantitatively in terms of spectral quantification accuracy measured by normalized root mean square errors.

CONCLUSION

Joint spectral quantification using linear tangent space alignment-based manifold learning improves the accuracy of MRSI spectral quantification.

High-Dimensional MR Spatiospectral Imaging by Integrating Physics-Based Modeling and Data-Driven Machine Learning: Current progress and future directions

Magnetic resonance spectroscopic imaging (MRSI) offers a unique molecular window into the physiological and pathological processes in the human body. However, the applications of MRSI have been limited by a number of long-standing technical challenges due to high dimensionality and low signal-to-noise ratio (SNR). Recent technological developments integrating physics-based modeling and data-driven machine learning that exploit unique physical and mathematical properties of MRSI signals have demonstrated impressive performance in addressing these challenges for rapid, high-resolution, quantitative MRSI. This paper provides a systematic review of these progresses in the context of MRSI physics and offers perspectives on promising future directions.

Model-informed unsupervised deep learning approaches to frequency and phase correction of MRS signals

PURPOSE

A supervised deep learning (DL) approach for frequency and phase correction (FPC) of MRS data recently showed encouraging results, but obtaining transients with labels for supervised learning is challenging. This work investigates the feasibility and efficiency of unsupervised deep learning-based FPC.

METHODS

Two novel deep learning-based FPC methods (deep learning-based Cr referencing and deep learning-based spectral registration), which use a priori physics domain knowledge, are presented. The proposed networks were trained, validated, and evaluated using simulated, phantom, and publicly accessible in vivo MEGA-edited MRS data. The performance of our proposed FPC methods was compared with other generally used FPC methods, in terms of precision and time efficiency. A new measure was proposed in this study to evaluate the FPC method performance. The ability of each of our methods to carry out FPC at varying SNR levels was evaluated. A Monte Carlo study was carried out to investigate the performance of our proposed methods.

RESULTS

The validation using low-SNR manipulated simulated data demonstrated that the proposed methods could perform FPC comparably with other methods. The evaluation showed that the deep learning-based spectral registration over a limited frequency range method achieved the highest performance in phantom data. The applicability of the proposed method for FPC of GABA-edited in vivo MRS data was demonstrated. Our proposed networks have the potential to reduce computation time significantly.

CONCLUSIONS

The proposed physics-informed deep neural networks trained in an unsupervised manner with complex data can offer efficient FPC of large MRS data in a shorter time.

Molecular MRI-Based Monitoring of Cancer Immunotherapy Treatment Response

Immunotherapy constitutes a paradigm shift in cancer treatment. Its FDA approval for several indications has yielded improved prognosis for cases where traditional therapy has shown limited efficiency. However, many patients still fail to benefit from this treatment modality, and the exact mechanisms responsible for tumor response are unknown. Noninvasive treatment monitoring is crucial for longitudinal tumor characterization and the early detection of non-responders. While various medical imaging techniques can provide a morphological picture of the lesion and its surrounding tissue, a molecular-oriented imaging approach holds the key to unraveling biological effects that occur much earlier in the immunotherapy timeline. Magnetic resonance imaging (MRI) is a highly versatile imaging modality, where the image contrast can be tailored to emphasize a particular biophysical property of interest using advanced engineering of the imaging pipeline. In this review, recent advances in molecular-MRI based cancer immunotherapy monitoring are described. Next, the presentation of the underlying physics, computational, and biological features are complemented by a critical analysis of the results obtained in preclinical and clinical studies. Finally, emerging artificial intelligence (AI)-based strategies to further distill, quantify, and interpret the image-based molecular MRI information are discussed in terms of perspectives for the future.

Convolutional neural networks to predict brain tumor grades and Alzheimer’s disease with MR spectroscopic imaging data

Purpose

To evaluate the value of convolutional neural network (CNN) in the diagnosis of human brain tumor or Alzheimer’s disease by MR spectroscopic imaging (MRSI) and to compare its Matthews correlation coefficient (MCC) score against that of other machine learning methods and previous evaluation of the same data. We address two challenges: 1) limited number of cases in MRSI datasets and 2) interpretability of results in the form of relevant spectral regions.

Methods

A shallow CNN with only one hidden layer and an ad-hoc loss function was constructed involving two branches for processing spectral and image features of a brain voxel respectively. Each branch consists of a single convolutional hidden layer. The output of the two convolutional layers is merged and fed to a classification layer that outputs class predictions for the given brain voxel.

Results

Our CNN method separated glioma grades 3 and 4 and identified Alzheimer’s disease patients using MRSI and complementary MRI data with high MCC score (Area Under the Curve were 0.87 and 0.91 respectively). The results demonstrated superior effectiveness over other popular methods as Partial Least Squares or Support Vector Machines. Also, our method automatically identified the spectral regions most important in the diagnosis process and we show that these are in good agreement with existing biomarkers from the literature.

Conclusion

Shallow CNNs models integrating image and spectral features improved quantitative and exploration and diagnosis of brain diseases for research and clinical purposes. Software is available at https://bitbucket.org/TeslaH2O/cnn_mrsi.

Developments in proton MR spectroscopic imaging of prostate cancer

In this paper, we review the developments of 1H-MR spectroscopic imaging (MRSI) methods designed to investigate prostate cancer, covering key aspects such as specific hardware, dedicated pulse sequences for data acquisition and data processing and quantification techniques. Emphasis is given to recent advancements in MRSI methodologies, as well as future developments, which can lead to overcome difficulties associated with commonly employed MRSI approaches applied in clinical routine. This includes the replacement of standard PRESS sequences for volume selection, which we identified as inadequate for clinical applications, by sLASER sequences and implementation of 1H MRSI without water signal suppression. These may enable a new evaluation of the complementary role and significance of MRSI in prostate cancer management.

Improved signal-to-noise performance of MultiNet GRAPPA 1 H FID MRSI reconstruction with semi-synthetic calibration data

PURPOSE

To further develop MultiNet GRAPPA, a neural-network-based reconstruction, for lower SNR proton MRSI (¹ H MRSI) data using adapted undersampling schemes and improved training sets.

METHODS

¹ H FID-MRSI data and an anatomical image for GRAPPA reconstruction were acquired in two slices in the human brain (n = 6) at 7T. MRSI data were retrospectively undersampled for a 4×, 6×, and 7× acceleration rate. Signal-to-noise, relative error (RE) between accelerated and fully sampled metabolic maps, RMS of the lipid artifacts, and fitting reliability were compared across acceleration rates, to the fully sampled data, and with different kinds and amounts of training images.

RESULTS

Training with semi-synthetic images resulted in higher SNR and lower lipid RMS relative to training with acquired images from one or several subjects. SNR increased with the number of semi-synthetic training images and the 4× accelerated data retains ∼30% more SNR than other accelerated data. Spectra reconstructed with 20 semi-synthetic averages retained ∼100% more SNR and had ∼5% lower lipid RMS than those reconstructed with the center k-space points of one image as was originally proposed for very high SNR MRSI data and had higher fitting reliability. The metabolite RE was lowest when training with 20-semi-synthetic training images and highest when training with the center k-space points of one image.

CONCLUSION

MultiNet GRAPPA is feasible with lower SNR ¹ H MRSI data if 20-semi-synthetic training images are used at a 4× acceleration rate. This acceleration rate provided the best trade-off between scan time and spectral SNR.

Bayesian deep learning-based 1 H-MRS of the brain: Metabolite quantification with uncertainty estimation using Monte Carlo dropout

PURPOSE

To develop a Bayesian convolutional neural network (BCNN) with Monte Carlo dropout sampling for metabolite quantification with simultaneous uncertainty estimation in deep learning-based proton MRS of the brain.

METHODS

Human brain spectra were simulated using basis spectra for 17 metabolites and macromolecules (N = 100 000) at 3.0 Tesla. In addition, actual in vivo spectra (N = 5) were modified by adjusting SNR and linewidth with increasing severity of spectral degradation (N = 50). A BCNN was trained on the simulated spectra to generate a noise-free, line-narrowed, macromolecule signal-removed, metabolite-only spectrum from a typical human brain spectrum. At inference, each input spectrum was Monte Carlo dropout sampled (50 times), and the resulting mean spectrum and variance spectrum were used for metabolite quantification and uncertainty estimation, respectively.

RESULTS

Using the simulated spectra, the mean absolute percent errors of the BCNN-predicted metabolite content were < 10% for Cr, Glu, Gln, mI, NAA, and Tau (< 5% for Glu, NAA, and mI). For all metabolites, the correlations (r's) between the ground-truth error and BCNN-predicted uncertainty ranged 0.72-0.94 (0.83 ± 0.06; p < 0.001). Using the modified in vivo spectra, the extent of variation in the estimated metabolite content against the increasing severity of spectral degradation tended to be smaller with BCNN than with linear combination of model spectra (LCModel). Overall, the variation in metabolite content tended to be more highly correlated with the uncertainty from BCNN than with the Cramér-Rao lower-bounds from LCModel (0.938 ± 0.019 vs. 0.881 ± 0.057 [p = 0.115]).

CONCLUSION

The BCNN with Monte Carlo dropout sampling may be used in deep learning-based MRS for the estimation of uncertainty in the machine-predicted metabolite content, which is important in the clinical application of deep learning-based MRS.

Learning-based motion artifact removal networks for quantitative R 2 ∗ mapping

PURPOSE

To introduce two novel learning-based motion artifact removal networks (LEARN) for the estimation of quantitative motion- and B 0 -inhomogeneity-corrected R 2 ∗ maps from motion-corrupted multi-Gradient-Recalled Echo (mGRE) MRI data.

METHODS

We train two convolutional neural networks (CNNs) to correct motion artifacts for high-quality estimation of quantitative B 0 -inhomogeneity-corrected R 2 ∗ maps from mGRE sequences. The first CNN, LEARN-IMG, performs motion correction on complex mGRE images, to enable the subsequent computation of high-quality motion-free quantitative R 2 ∗ (and any other mGRE-enabled) maps using the standard voxel-wise analysis or machine learning-based analysis. The second CNN, LEARN-BIO, is trained to directly generate motion- and B 0 -inhomogeneity-corrected quantitative R 2 ∗ maps from motion-corrupted magnitude-only mGRE images by taking advantage of the biophysical model describing the mGRE signal decay.

RESULTS

We show that both CNNs trained on synthetic MR images are capable of suppressing motion artifacts while preserving details in the predicted quantitative R 2 ∗ maps. Significant reduction of motion artifacts on experimental in vivo motion-corrupted data has also been achieved by using our trained models.

CONCLUSION

Both LEARN-IMG and LEARN-BIO can enable the computation of high-quality motion- and B 0 -inhomogeneity-corrected R 2 ∗ maps. LEARN-IMG performs motion correction on mGRE images and relies on the subsequent analysis for the estimation of R 2 ∗ maps, while LEARN-BIO directly performs motion- and B 0 -inhomogeneity-corrected R 2 ∗ estimation. Both LEARN-IMG and LEARN-BIO jointly process all the available gradient echoes, which enables them to exploit spatial patterns available in the data. The high computational speed of LEARN-BIO is an advantage that can lead to a broader clinical application.

High-Resolution Magnetic Resonance Spectroscopic Imaging using a Multi-Encoder Attention U-Net with Structural and Adversarial Loss

Common to most medical imaging techniques, the spatial resolution of Magnetic Resonance Spectroscopic Imaging (MRSI) is ultimately limited by the achievable SNR. This work presents a deep learning method for 1H-MRSI spatial resolution enhancement, based on the observation that multi-parametric MRI images provide relevant spatial priors for MRSI enhancement. A Multi-encoder Attention U-Net (MAU-Net) architecture was constructed to process a MRSI metabolic map and three different MRI modalities through separate encoding paths. Spatial attention modules were incorporated to automatically learn spatial weights that highlight salient features for each MRI modality. MAU-Net was trained based on in vivo brain imaging data from patients with high-grade gliomas, using a combined loss function consisting of pixel, structural and adversarial loss. Experimental results showed that the proposed method is able to reconstruct high-quality metabolic maps with a high-resolution of 64×64 from a low-resolution of 16 × 16, with better performance compared to several baseline methods.

Deep Learning and Its Application to Function Approximation for MR in Medicine: An Overview

This article presents an overview of deep learning (DL) and its applications to function approximation for MR in medicine. The aim of this article is to help readers develop various applications of DL. DL has made a large impact on the literature of many medical sciences, including MR. However, its technical details are not easily understandable for non-experts of machine learning (ML).The first part of this article presents an overview of DL and its related technologies, such as artificial intelligence (AI) and ML. AI is explained as a function that can receive many inputs and produce many outputs. ML is a process of fitting the function to training data. DL is a kind of ML, which uses a composite of many functions to approximate the function of interest. This composite function is called a deep neural network (DNN), and the functions composited into a DNN are called layers. This first part also covers the underlying technologies required for DL, such as loss functions, optimization, initialization, linear layers, non-linearities, normalization, recurrent neural networks, regularization, data augmentation, residual connections, autoencoders, generative adversarial networks, model and data sizes, and complex-valued neural networks.The second part of this article presents an overview of the applications of DL in MR and explains how functions represented as DNNs are applied to various applications, such as RF pulse, pulse sequence, reconstruction, motion correction, spectroscopy, parameter mapping, image synthesis, and segmentation.

Hyperpolarized MRI, functional MRI, MR spectroscopy and CEST to provide metabolic information in vivo

Access to metabolic information in vivo using magnetic resonance (MR) technologies has generally been the niche of MR spectroscopy (MRS) and spectroscopic imaging (MRSI). Metabolic fluxes can be studied using the infusion of substrates labeled with magnetic isotopes, with the use of hyperpolarization especially powerful. Unfortunately, these promising methods are not yet accepted clinically, where fast, simple, and reliable measurement and diagnosis are key. Recent advances in functional MRI and chemical exchange saturation transfer (CEST) MRI allow the use of water imaging to study oxygen metabolism and tissue metabolite levels. These, together with the use of novel data analysis approaches such as machine learning for all of these metabolic MR approaches, are increasing the likelihood of their clinical translation.

3D whole-brain metabolite imaging to improve characterization of low-to-intermediate grade gliomas

PURPOSE

MRI is the standard imaging modality used for diagnosis, treatment planning, and post-treatment management of gliomas. Contrast-enhanced T1-weighted (CE-T1w) MRI is used to plan biopsy and radiation for grade IV gliomas but is less effective for grade II and III gliomas (i.e., low-to-intermediate grade gliomas) which may have minimal or no enhancement. Magnetic resonance spectroscopic imaging (MRSI) is an advanced MRI technique that has been shown, to improve diagnostic yield of biopsy and target delineation for grade IV glioma. The purpose of this study is to determine if MRSI can improve characterization and tissue sampling of low-to-intermediate grade gliomas.

METHODS

Prospective grade II and grade III glioma patients were enrolled to undergo whole brain high-resolution MRSI prior to tissue sampling. Choline/N-acetyl-aspartate (Cho/NAA) maps were overlaid on anatomic imaging and imported into stereotactic biopsy software. Patients were treated with standard-of-care surgery and radiation. Volumes of spectroscopically abnormal tissue were generated and compared with anatomic imaging and areas of enhancing recurrence on follow-up imaging.

RESULTS

Ten patients had pathologic diagnosis of grade II (n = 4) or grade III (n = 6) with a median follow-up of 27.3 months. Five patients had recurrence, and regions of recurrence were found to overlap with metabolically abnormal regions on MRSI at the time of diagnosis.

CONCLUSION

MRSI in low-to-intermediate grade glioma patients is predictive of areas of subsequent recurrence. Larger studies are needed to determine if MRSI can be used to guide surgical and radiation treatment planning in these patients.

Advanced magnetic resonance spectroscopic neuroimaging: Experts' consensus recommendations

Magnetic resonance spectroscopic imaging (MRSI) offers considerable promise for monitoring metabolic alterations associated with disease or injury; however, to date, these methods have not had a significant impact on clinical care, and their use remains largely confined to the research community and a limited number of clinical sites. The MRSI methods currently implemented on clinical MRI instruments have remained essentially unchanged for two decades, with only incremental improvements in sequence implementation. During this time, a number of technological developments have taken place that have already greatly benefited the quality of MRSI measurements within the research community and which promise to bring advanced MRSI studies to the point where the technique becomes a true imaging modality, while making the traditional review of individual spectra a secondary requirement. Furthermore, the increasing use of biomedical MR spectroscopy studies has indicated clinical areas where advanced MRSI methods can provide valuable information for clinical care. In light of this rapidly changing technological environment and growing understanding of the value of MRSI studies for biomedical studies, this article presents a consensus from a group of experts in the field that reviews the state-of-the-art for clinical proton MRSI studies of the human brain, recommends minimal standards for further development of vendor-provided MRSI implementations, and identifies areas which need further technical development.

Deep learning can accelerate and quantify simulated localized correlated spectroscopy

Nuclear magnetic resonance spectroscopy (MRS) allows for the determination of atomic structures and concentrations of different chemicals in a biochemical sample of interest. MRS is used in vivo clinically to aid in the diagnosis of several pathologies that affect metabolic pathways in the body. Typically, this experiment produces a one dimensional (1D) 1H spectrum containing several peaks that are well associated with biochemicals, or metabolites. However, since many of these peaks overlap, distinguishing chemicals with similar atomic structures becomes much more challenging. One technique capable of overcoming this issue is the localized correlated spectroscopy (L-COSY) experiment, which acquires a second spectral dimension and spreads overlapping signal across this second dimension. Unfortunately, the acquisition of a two dimensional (2D) spectroscopy experiment is extremely time consuming. Furthermore, quantitation of a 2D spectrum is more complex. Recently, artificial intelligence has emerged in the field of medicine as a powerful force capable of diagnosing disease, aiding in treatment, and even predicting treatment outcome. In this study, we utilize deep learning to: (1) accelerate the L-COSY experiment and (2) quantify L-COSY spectra. All training and testing samples were produced using simulated metabolite spectra for chemicals found in the human body. We demonstrate that our deep learning model greatly outperforms compressed sensing based reconstruction of L-COSY spectra at higher acceleration factors. Specifically, at four-fold acceleration, our method has less than 5% normalized mean squared error, whereas compressed sensing yields 20% normalized mean squared error. We also show that at low SNR (25% noise compared to maximum signal), our deep learning model has less than 8% normalized mean squared error for quantitation of L-COSY spectra. These pilot simulation results appear promising and may help improve the efficiency and accuracy of L-COSY experiments in the future.

Separation of Metabolites and Macromolecules for Short-TE 1H-MRSI Using Learned Component-Specific Representations

Short-echo-time (TE) proton magnetic resonance spectroscopic imaging (MRSI) allows for simultaneously mapping a number of molecules in the brain, and has been recognized as an important tool for studying in vivo biochemistry in various neuroscience and disease applications. However, separation of the metabolite and macromolecule (MM) signals present in the short-TE data with significant spectral overlaps remains a major technical challenge. This work introduces a new approach to solve this problem by integrating imaging physics and representation learning. Specifically, a mixed unsupervised and supervised learning-based strategy was developed to learn the metabolite and MM-specific low-dimensional representations using deep autoencoders. A constrained reconstruction formulation is proposed to integrate the MRSI spatiospectral encoding model and the learned representations as effective constraints for signal separation. An efficient algorithm was developed to solve the resulting optimization problem with provable convergence. Simulation and experimental results have been obtained to demonstrate the component-specific representation power of the learned models and the capability of the proposed method in separating metabolite and MM signals for practical short-TE [Formula: see text]-MRSI data.

Unsupervised anomaly detection using generative adversarial networks in 1H-MRS of the brain

The applicability of generative adversarial networks (GANs) capable of unsupervised anomaly detection (AnoGAN) was investigated in the management of quality of ¹H-MRS human brain spectra at 3.0 T. The AnoGAN was trained in an unsupervised manner solely on simulated normal brain spectra and used for filtering out abnormal spectra with a broad range of abnormalities, which were simulated by including abnormal ranges of SNR, linewidth and metabolite concentrations and spectral artifacts such as ghost, residual water, and lipid. The AnoGAN was able to filter out those spectra with SNR less than ~11-12 dB with an accuracy of ~80% or higher (assuming a normal SNR range to be 15-18 dB). It also detected with an accuracy of ~80% or higher those spectra, in which NAA levels were reduced by ~25-30% or more from the lower bound and elevated by ~20-30% or more from the upper bound of the normal concentration range (7.5-17 mmol/L), while the concentrations of the rest of the metabolites were all within the normal ranges. Despite the fact that those spectra contaminated with ghost, residual water or lipid have never been involved in the training or optimization of the AnoGAN, they were correctly classified as abnormal regardless of the types of the artifacts, depending solely on their intensity. Although the current version of our AnoGAN requires further technical improvement particularly for the detection of linewidth-associated abnormality and validation on in vivo data, our unsupervised deep learning-based approach could be an option in addition to those previously reported supervised deep learning-based approaches in the binary classification of spectral quality with an extended abnormal spectra regime.

Deep learning using a biophysical model for robust and accelerated reconstruction of quantitative, artifact-free and denoised R 2 * images

PURPOSE

To introduce a novel deep learning method for Robust and Accelerated Reconstruction (RoAR) of quantitative and B0-inhomogeneity-corrected R 2 * maps from multi-gradient recalled echo (mGRE) MRI data.

METHODS

RoAR trains a convolutional neural network (CNN) to generate quantitative R 2 ∗ maps free from field inhomogeneity artifacts by adopting a self-supervised learning strategy given (a) mGRE magnitude images, (b) the biophysical model describing mGRE signal decay, and (c) preliminary-evaluated F-function accounting for contribution of macroscopic B0 field inhomogeneities. Importantly, no ground-truth R 2 * images are required and F-function is only needed during RoAR training but not application.

RESULTS

We show that RoAR preserves all features of R 2 * maps while offering significant improvements over existing methods in computation speed (seconds vs. hours) and reduced sensitivity to noise. Even for data with SNR = 5 RoAR produced R 2 * maps with accuracy of 22% while voxel-wise analysis accuracy was 47%. For SNR = 10 the RoAR accuracy increased to 17% vs. 24% for direct voxel-wise analysis.

CONCLUSIONS

RoAR is trained to recognize the macroscopic magnetic field inhomogeneities directly from the input magnitude-only mGRE data and eliminate their effect on R 2 ∗ measurements. RoAR training is based on the biophysical model and does not require ground-truth R 2 * maps. Since RoAR utilizes signal information not just from individual voxels but also accounts for spatial patterns of the signals in the images, it reduces the sensitivity of R 2 * maps to the noise in the data. These features plus high computational speed provide significant benefits for the potential usage of RoAR in clinical settings.

Precision Digital Oncology: Emerging Role of Radiomics-based Biomarkers and Artificial Intelligence for Advanced Imaging and Characterization of Brain Tumors

Advances in computerized image analysis and the use of artificial intelligence-based approaches for image-based analysis and construction of prediction algorithms represent a new era for noninvasive biomarker discovery. In recent literature, it has become apparent that radiologic images can serve as mineable databases that contain large amounts of quantitative features with potential clinical significance. Extraction and analysis of these quantitative features is commonly referred to as texture or radiomic analysis. Numerous studies have demonstrated applications for texture and radiomic characterization methods for assessing brain tumors to improve noninvasive predictions of tumor histologic characteristics, molecular profile, distinction of treatment-related changes, and prediction of patient survival. In this review, the current use and future potential of texture or radiomic-based approaches with machine learning for brain tumor image analysis and prediction algorithm construction will be discussed. This technology has the potential to advance the value of diagnostic imaging by extracting currently unused information on medical scans that enables more precise, personalized therapy; however, significant barriers must be overcome if this technology is to be successfully implemented on a wide scale for routine use in the clinical setting. Keywords: Adults and Pediatrics, Brain/Brain Stem, CNS, Computer Aided Diagnosis (CAD), Computer Applications-General (Informatics), Image Postprocessing, Informatics, Neural Networks, Neuro-Oncology, Oncology, Treatment Effects, Tumor Response Supplemental material is available for this article. © RSNA, 2020.

Deep learning-based target metabolite isolation and big data-driven measurement uncertainty estimation in proton magnetic resonance spectroscopy of the brain

PURPOSE

The aim of this study was to develop a method for metabolite quantification with simultaneous measurement uncertainty estimation in deep learning-based proton magnetic resonance spectroscopy (¹ H-MRS).

METHODS

The reliability of metabolite quantification depends on signal-to-noise ratio (SNR), linewidth, and degree of spectral overlap (DSO), and therefore knowledge about these factors may be utilized in measurement uncertainty estimation in deep learning-based ¹ H-MRS. While SNR and linewidth are typically estimated from a representative singlet, DSO needs to be estimated metabolite-specifically. We developed convolutional neural networks (CNNs) capable of isolating target metabolite signal on simulated rat brain spectra at 9.4T, such that, in addition to metabolite content, the signal-to-background ratio (SBR) as a quantitative metric of DSO can be estimated directly from CNN-output for each metabolite. The CNN-predicted SBR was adjusted according to its pre-defined relationship to the ground-truth SBR by exploiting the big spectral data (N = 80 000), and used for measurement uncertainty estimation together with the SNR and linewidth from the CNN-input spectrum. The proposed method was tested first on the simulated spectra in comparison with LCModel and jMRUI and further on in vivo spectra.

RESULTS

The proposed method outperformed LCModel and jMRUI in both quantitative accuracy and measurement uncertainty estimation. Using in vivo data, the metabolite concentrations from the proposed method were close to the reported ranges with the measurement uncertainty of glutamine, glutamate, myo-inositol, N-acetylaspartate, and Tau less than 10%.

CONCLUSION

The proposed method may be used for metabolite quantification with measurement uncertainty estimation in rat brain at 9.4T by exploiting the spectral isolation capability of the CNNs and the availability of big spectral data.

Constrained Magnetic Resonance Spectroscopic Imaging by Learning Nonlinear Low-Dimensional Models

Magnetic resonance spectroscopic imaging (MRSI) is a powerful molecular imaging modality but has very limited speed, resolution, and SNR tradeoffs. Construction of a low-dimensional model to effectively reduce the dimensionality of the imaging problem has recently shown great promise in improving these tradeoffs. This paper presents a new approach to model and reconstruct the spectroscopic signals by learning a nonlinear low-dimensional representation of the general MR spectra. Specifically, we trained a deep neural network to capture the low-dimensional manifold, where the high-dimensional spectroscopic signals reside. A regularization formulation is proposed to effectively integrate the learned model and physics-based data acquisition model for MRSI reconstruction with the capability to incorporate additional spatiospectral constraints. An efficient numerical algorithm was developed to solve the associated optimization problem involving back-propagating the trained network. Simulation and experimental results were obtained to demonstrate the representation power of the learned model and the ability of the proposed formulation in producing SNR-enhancing reconstruction from the practical MRSI data.