Super-Resolution 1H Magnetic Resonance Spectroscopic Imaging Utilizing Deep Learning

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Magnetic Resonance Spectroscopy for Cervical Cancer: Review and Potential Prognostic Applications

This review article investigates the utilization of MRS in the setting of cervical cancer. A variety of different techniques have been used in this space including single-voxel techniques such as point-resolved spectroscopy (PRESS) and stimulated echo acquisition mode spectroscopy (STEAM). Furthermore, the experimental parameters for these acquisitions including field strength, repetition times (TR), and echo times (TE) vary greatly. This study critically examines eleven MRS studies that focus on cervical cancer. Out of the eleven studies, ten studies utilized PRESS acquisition, while the remaining study used STEAM acquisition. These studies generally showed that the choline signal is altered in cervical cancer (4/11 studies), the lipid signal is generally increased in cervical cancer or the lipid distribution is changed (5/11 studies), and that diffusion-weighted imaging (DWI) can quantitatively detect lower apparent diffusion coefficient (ADC) values in cervical cancer (2/11 studies). Two studies also investigated the role of MRS for monitoring treatment response and demonstrated mixed results regarding choline signal, and one of these studies showed increased lipid signal for non-responders. There are several new MRS technologies that have yet to be implemented for cervical cancer including advanced spectroscopic imaging and artificial intelligence, and those technologies are also discussed in the article.

Super-resolution techniques for biomedical applications and challenges

Super-resolution (SR) techniques have revolutionized the field of biomedical applications by detailing the structures at resolutions beyond the limits of imaging or measuring tools. These techniques have been applied in various biomedical applications, including microscopy, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, electroencephalogram (EEG), ultrasound, etc. SR methods are categorized into two main types: traditional non-learning-based methods and modern learning-based approaches. In both applications, SR methodologies have been effectively utilized on biomedical images, enhancing the visualization of complex biological structures. Additionally, these methods have been employed on biomedical data, leading to improvements in computational precision and efficiency for biomedical simulations. The use of SR techniques has resulted in more detailed and accurate analyses in diagnostics and research, essential for early disease detection and treatment planning. However, challenges such as computational demands, data interpretation complexities, and the lack of unified high-quality data persist. The article emphasizes these issues, underscoring the need for ongoing development in SR technologies to further improve biomedical research and patient care outcomes.

Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI

OBJECTIVE

This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics.

MATERIALS AND METHODS

To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters.

RESULTS

ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA.

DISCUSSION

These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.

Denoising single MR spectra by deep learning: Miracle or mirage?

PURPOSE

The inherently poor SNR of MRS measurements presents a significant hurdle to its clinical application. Denoising by machine or deep learning (DL) was proposed as a remedy. It is investigated whether such denoising leads to lower estimate uncertainties or whether it essentially reduces noise in signal-free areas only.

METHODS

Noise removal based on supervised DL with U-nets was implemented using simulated 1 H MR spectra of human brain in two approaches: (1) via time-frequency domain spectrograms and (2) using 1D spectra as input. Quality of denoising was evaluated in three ways: (1) by an adapted fit quality score, (2) by traditional model fitting, and (3) by quantification via neural networks.

RESULTS

Visually appealing spectra were obtained; hinting that denoising is well-suited for MRS. However, an adapted denoising score showed that noise removal is inhomogeneous and more efficient for signal-free areas. This was confirmed by quantitative analysis of traditional fit results as well as DL quantitation following DL denoising. DL denoising, although apparently successful as judged by mean squared errors, led to substantially biased estimates in both implementations.

CONCLUSION

The implemented DL-based denoising techniques may be useful for display purposes, but do not help quantitative evaluations, confirming expectations based on estimation theory: Cramér Rao lower bounds defined by the original data and the appropriate fitting model cannot be circumvented in an unbiased way for single data sets, unless additional prior knowledge can be incurred in the form of parameter restrictions/relations or applicable substates.

Super-resolution of magnetic systems using deep learning

We construct a deep neural network to enhance the resolution of spin structure images formed by spontaneous symmetry breaking in the magnetic systems. Through the deep neural network, an image is expanded to a super-resolution image and reduced to the original image size to be fitted with the input feed image. The network does not require ground truth images in the training process. Therefore, it can be applied when low-resolution images are provided as training datasets, while high-resolution images are not obtainable due to the intrinsic limitation of microscope techniques. To show the usefulness of the network, we train the network with two types of simulated magnetic structure images; one is from self-organized maze patterns made of chiral magnetic structures, and the other is from magnetic domains separated by walls that are topological defects of the system. The network successfully generates high-resolution images highly correlated with the exact solutions in both cases. To investigate the effectiveness and the differences between datasets, we study the network's noise tolerance and compare the networks' reliabilities. The network is applied with experimental data obtained by magneto-optical Kerr effect microscopy and spin-polarized low-energy electron microscopy.

A review of machine learning applications for the proton MR spectroscopy workflow

This literature review presents a comprehensive overview of machine learning (ML) applications in proton MR spectroscopy (MRS). As the use of ML techniques in MRS continues to grow, this review aims to provide the MRS community with a structured overview of the state-of-the-art methods. Specifically, we examine and summarize studies published between 2017 and 2023 from major journals in the MR field. We categorize these studies based on a typical MRS workflow, including data acquisition, processing, analysis, and artificial data generation. Our review reveals that ML in MRS is still in its early stages, with a primary focus on processing and analysis techniques, and less attention given to data acquisition. We also found that many studies use similar model architectures, with little comparison to alternative architectures. Additionally, the generation of artificial data is a crucial topic, with no consistent method for its generation. Furthermore, many studies demonstrate that artificial data suffers from generalization issues when tested on in vivo data. We also conclude that risks related to ML models should be addressed, particularly for clinical applications. Therefore, output uncertainty measures and model biases are critical to investigate. Nonetheless, the rapid development of ML in MRS and the promising results from the reviewed studies justify further research in this field.

Noninvasive 3-Dimensional 1 H-Magnetic Resonance Spectroscopic Imaging of Human Brain Glucose and Neurotransmitter Metabolism Using Deuterium Labeling at 3T : Feasibility and Interscanner Reproducibility

OBJECTIVES

Noninvasive, affordable, and reliable mapping of brain glucose metabolism is of critical interest for clinical research and routine application as metabolic impairment is linked to numerous pathologies, for example, cancer, dementia, and depression. A novel approach to map glucose metabolism noninvasively in the human brain has been presented recently on ultrahigh-field magnetic resonance (MR) scanners (≥7T) using indirect detection of deuterium-labeled glucose and downstream metabolites such as glutamate, glutamine, and lactate. The aim of this study was to demonstrate the feasibility to noninvasively detect deuterium-labeled downstream glucose metabolites indirectly in the human brain via 3-dimensional (3D) proton ( 1 H) MR spectroscopic imaging on a clinical 3T MR scanner without additional hardware.

MATERIALS AND METHODS

This prospective, institutional review board-approved study was performed in 7 healthy volunteers (mean age, 31 ± 4 years, 5 men/2 women) after obtaining written informed consent. After overnight fasting and oral deuterium-labeled glucose administration, 3D metabolic maps were acquired every ∼4 minutes with ∼0.24 mL isotropic spatial resolution using real-time motion-, shim-, and frequency-corrected echo-less 3D 1 H-MR spectroscopic Imaging on a clinical routine 3T MR system. To test the interscanner reproducibility of the method, subjects were remeasured on a similar 3T MR system. Time courses were analyzed using linear regression and nonparametric statistical tests. Deuterium-labeled glucose and downstream metabolites were detected indirectly via their respective signal decrease in dynamic 1 H MR spectra due to exchange of labeled and unlabeled molecules.

RESULTS

Sixty-five minutes after deuterium-labeled glucose administration, glutamate + glutamine (Glx) signal intensities decreased in gray/white matter (GM/WM) by -1.63 ± 0.3/-1.0 ± 0.3 mM (-13% ± 3%, P = 0.02/-11% ± 3%, P = 0.02), respectively. A moderate to strong negative correlation between Glx and time was observed in GM/WM ( r = -0.64, P < 0.001/ r = -0.54, P < 0.001), with 60% ± 18% ( P = 0.02) steeper slopes in GM versus WM, indicating faster metabolic activity. Other nonlabeled metabolites showed no significant changes. Excellent intrasubject repeatability was observed across scanners for static results at the beginning of the measurement (coefficient of variation 4% ± 4%), whereas differences were observed in individual Glx dynamics, presumably owing to physiological variation of glucose metabolism.

CONCLUSION

Our approach translates deuterium metabolic imaging to widely available clinical routine MR scanners without specialized hardware, offering a safe, affordable, and versatile (other substances than glucose can be labeled) approach for noninvasive imaging of glucose and neurotransmitter metabolism in the human brain.

A2OURSR: Adaptive adjustment based real MRI super-resolution via opinion-unaware measurements

High-quality and high-resolution magnetic resonance (MR) images can provide more details for diagnosis and analyses. Recently, MR images guided neurosurgery has become an emerging technique in clinics. Unlike other medical imaging techniques, it is impossible to achieve both real-time imaging and high image quality in MR imaging. The real-time performance is closely related to the nuclear magnetic equipment itself as well as the collection strategy of the k space data. Optimizing the imaging time cost via the corresponding algorithm is harder than enhancing image quality. Further, in reconstructing low-resolution and noise-rich MR images, getting relatively high-definition and resolution MR images as references are difficult or impossible. In addition, the existing methods are restricted in learning the controllable functions under the supervision of known degradation types and levels. As a result, severely bad results are inevitable when the modeling assumptions are far apart from the actual situation. To address these problems, we propose a novel adaptive adjustment method based on real MR images via opinion-unaware measurements for real super-resolution (A²OURSR). It can estimate the degree of blur and noise from the test image itself using two scores. These two scores can be considered pseudo labels to train the adaptive adjustable degradation estimation module. Then, the outputs of the above model are used as the inputs of the conditional network to tweak the generated results. Thus, the results can be automatically adjusted via the whole dynamic model. Extensive experimental results show that the proposed A²OURSR is superior to state-of-the-art methods on benchmarks quantitatively and visually.

Physics-informed deep learning approach to quantification of human brain metabolites from magnetic resonance spectroscopy data

PURPOSE

While the recommended analysis method for magnetic resonance spectroscopy data is linear combination model (LCM) fitting, the supervised deep learning (DL) approach for quantification of MR spectroscopy (MRS) and MR spectroscopic imaging (MRSI) data recently showed encouraging results; however, supervised learning requires ground truth fitted spectra, which is not practical. Moreover, this work investigates the feasibility and efficiency of the LCM-based self-supervised DL method for the analysis of MRS data.

METHOD

We present a novel DL-based method for the quantification of relative metabolite concentrations, using quantum-mechanics simulated metabolite responses and neural networks. We trained, validated, and evaluated the proposed networks with simulated and publicly accessible in-vivo human brain MRS data and compared the performance with traditional methods. A novel adaptive macromolecule fitting algorithm is included. We investigated the performance of the proposed methods in a Monte Carlo (MC) study.

RESULT

The validation using low-SNR simulated data demonstrated that the proposed methods could perform quantification comparably to other methods. The applicability of the proposed method for the quantification of in-vivo MRS data was demonstrated. Our proposed networks have the potential to reduce computation time significantly.

CONCLUSION

The proposed model-constrained deep neural networks trained in a self-supervised manner can offer fast and efficient quantification of MRS and MRSI data. Our proposed method has the potential to facilitate clinical practice by enabling faster processing of large datasets such as high-resolution MRSI datasets, which may have thousands of spectra.

High-Dimensional MR Spatiospectral Imaging by Integrating Physics-Based Modeling and Data-Driven Machine Learning: Current progress and future directions

Magnetic resonance spectroscopic imaging (MRSI) offers a unique molecular window into the physiological and pathological processes in the human body. However, the applications of MRSI have been limited by a number of long-standing technical challenges due to high dimensionality and low signal-to-noise ratio (SNR). Recent technological developments integrating physics-based modeling and data-driven machine learning that exploit unique physical and mathematical properties of MRSI signals have demonstrated impressive performance in addressing these challenges for rapid, high-resolution, quantitative MRSI. This paper provides a systematic review of these progresses in the context of MRSI physics and offers perspectives on promising future directions.

Model-informed unsupervised deep learning approaches to frequency and phase correction of MRS signals

PURPOSE

A supervised deep learning (DL) approach for frequency and phase correction (FPC) of MRS data recently showed encouraging results, but obtaining transients with labels for supervised learning is challenging. This work investigates the feasibility and efficiency of unsupervised deep learning-based FPC.

METHODS

Two novel deep learning-based FPC methods (deep learning-based Cr referencing and deep learning-based spectral registration), which use a priori physics domain knowledge, are presented. The proposed networks were trained, validated, and evaluated using simulated, phantom, and publicly accessible in vivo MEGA-edited MRS data. The performance of our proposed FPC methods was compared with other generally used FPC methods, in terms of precision and time efficiency. A new measure was proposed in this study to evaluate the FPC method performance. The ability of each of our methods to carry out FPC at varying SNR levels was evaluated. A Monte Carlo study was carried out to investigate the performance of our proposed methods.

RESULTS

The validation using low-SNR manipulated simulated data demonstrated that the proposed methods could perform FPC comparably with other methods. The evaluation showed that the deep learning-based spectral registration over a limited frequency range method achieved the highest performance in phantom data. The applicability of the proposed method for FPC of GABA-edited in vivo MRS data was demonstrated. Our proposed networks have the potential to reduce computation time significantly.

CONCLUSIONS

The proposed physics-informed deep neural networks trained in an unsupervised manner with complex data can offer efficient FPC of large MRS data in a shorter time.

Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges

Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.

Convolutional neural networks to predict brain tumor grades and Alzheimer’s disease with MR spectroscopic imaging data

Purpose

To evaluate the value of convolutional neural network (CNN) in the diagnosis of human brain tumor or Alzheimer’s disease by MR spectroscopic imaging (MRSI) and to compare its Matthews correlation coefficient (MCC) score against that of other machine learning methods and previous evaluation of the same data. We address two challenges: 1) limited number of cases in MRSI datasets and 2) interpretability of results in the form of relevant spectral regions.

Methods

A shallow CNN with only one hidden layer and an ad-hoc loss function was constructed involving two branches for processing spectral and image features of a brain voxel respectively. Each branch consists of a single convolutional hidden layer. The output of the two convolutional layers is merged and fed to a classification layer that outputs class predictions for the given brain voxel.

Results

Our CNN method separated glioma grades 3 and 4 and identified Alzheimer’s disease patients using MRSI and complementary MRI data with high MCC score (Area Under the Curve were 0.87 and 0.91 respectively). The results demonstrated superior effectiveness over other popular methods as Partial Least Squares or Support Vector Machines. Also, our method automatically identified the spectral regions most important in the diagnosis process and we show that these are in good agreement with existing biomarkers from the literature.

Conclusion

Shallow CNNs models integrating image and spectral features improved quantitative and exploration and diagnosis of brain diseases for research and clinical purposes. Software is available at https://bitbucket.org/TeslaH2O/cnn_mrsi.

Developments in proton MR spectroscopic imaging of prostate cancer

In this paper, we review the developments of 1H-MR spectroscopic imaging (MRSI) methods designed to investigate prostate cancer, covering key aspects such as specific hardware, dedicated pulse sequences for data acquisition and data processing and quantification techniques. Emphasis is given to recent advancements in MRSI methodologies, as well as future developments, which can lead to overcome difficulties associated with commonly employed MRSI approaches applied in clinical routine. This includes the replacement of standard PRESS sequences for volume selection, which we identified as inadequate for clinical applications, by sLASER sequences and implementation of 1H MRSI without water signal suppression. These may enable a new evaluation of the complementary role and significance of MRSI in prostate cancer management.

In vivo brain MR spectroscopy in gliomas: clinical and pre-clinical chances

Purpose

Gliomas, the most common primary brain tumours, have recently been re-classified incorporating molecular aspects with important clinical, prognostic, and predictive implications. Concurrently, the reprogramming of metabolism, altering intracellular and extracellular metabolites affecting gene expression, differentiation, and the tumour microenvironment, is increasingly being studied, and alterations in metabolic pathways are becoming hallmarks of cancer. Magnetic resonance spectroscopy (MRS) is a complementary, non-invasive technique capable of quantifying multiple metabolites. The aim of this review focuses on the methodology and analysis techniques in proton MRS (1H MRS), including a brief look at X-nuclei MRS, and on its perspectives for diagnostic and prognostic biomarkers in gliomas in both clinical practice and preclinical research.

Methods

PubMed literature research was performed cross-linking the following key words: glioma, MRS, brain, in-vivo, human, animal model, clinical, pre-clinical, techniques, sequences, 1H, X-nuclei, Artificial Intelligence (AI), hyperpolarization.

Results

We selected clinical works (n = 51), preclinical studies (n = 35) and AI MRS application papers (n = 15) published within the last two decades. The methodological papers (n = 62) were taken into account since the technique first description.

Conclusions

Given the development of treatments targeting specific cancer metabolic pathways, MRS could play a key role in allowing non-invasive assessment for patient diagnosis and stratification, predicting and monitoring treatment responses and prognosis. The characterization of gliomas through MRS will benefit of a wide synergy among scientists and clinicians of different specialties within the context of new translational competences. Head coils, MRI hardware and post-processing analysis progress, advances in research, experts’ consensus recommendations and specific professionalizing programs will make the technique increasingly trustworthy, responsive, accessible.

Assessment of hepatic fatty acids during non-alcoholic steatohepatitis progression using magnetic resonance spectroscopy

INTRODUCTION AND OBJECTIVES

Non-alcoholic fatty liver disease (NAFLD) encompasses a spectrum of liver abnormalities including steatosis, steatohepatitis, fibrosis, and cirrhosis. Liver biopsy remains the gold standard method to determine the disease stage in NAFLD but is an invasive and risky procedure. Studies have previously reported that changes in intrahepatic fatty acids (FA) composition are related to the progression of NAFLD, mainly in its early stages. The aim of this study was to characterize the liver FA composition in mice fed a Choline-deficient L-amino-defined (CDAA) diet at different stages of NAFLD using magnetic resonance spectroscopy (MRS).

METHODS

We used in-vivo MRS to perform a longitudinal characterization of hepatic FA changes in NAFLD mice for 10 weeks. We validated our findings with ex-vivo MRS, gas chromatography-mass spectrometry and histology.

RESULTS

In-vivo and ex-vivo results showed that livers from CDAA-fed mice exhibit a significant increase in liver FA content as well as a change in FA composition compared with control mice. After 4 weeks of CDAA diet, a decrease in polyunsaturated and an increase in monounsaturated FA were observed. These changes were associated with the appearance of early stages of steatohepatitis, confirmed by histology (NAFLD Activity Score (NAS) = 4.5). After 10 weeks of CDAA-diet, the liver FA composition remained stable while the NAS increased further to 6 showing a combination of early and late stages of steatohepatitis.

CONCLUSION

Our results suggest that monitoring lipid composition in addition to total water/fat with MRS may yield additional insights that can be translated for non-invasive stratification of high-risk NAFLD patients.

Deep learning super-resolution magnetic resonance spectroscopic imaging of brain metabolism and mutant isocitrate dehydrogenase glioma

BACKGROUND

Magnetic resonance spectroscopic imaging (MRSI) can be used in glioma patients to map the metabolic alterations associated with IDH1,2 mutations that are central criteria for glioma diagnosis. The aim of this study was to achieve super-resolution (SR) MRSI using deep learning to image tumor metabolism in patients with mutant IDH glioma.

METHODS

We developed a deep learning method based on generative adversarial network (GAN) using Unet as generator network to upsample MRSI by a factor of 4. Neural networks were trained on simulated metabolic images from 75 glioma patients. The performance of deep neuronal networks was evaluated on MRSI data measured in 20 glioma patients and 10 healthy controls at 3T with a whole-brain 3D MRSI protocol optimized for detection of d-2-hydroxyglutarate (2HG). To further enhance structural details of metabolic maps we used prior information from high-resolution anatomical MR imaging. SR MRSI was compared to ground truth by Mann-Whitney U-test of peak signal-to-noise ratio (PSNR), structure similarity index measure (SSIM), feature-based similarity index measure (FSIM), and mean opinion score (MOS).

RESULTS

Deep learning SR improved PSNR by 17%, SSIM by 5%, FSIM by 7%, and MOS by 30% compared to conventional interpolation methods. In mutant IDH glioma patients proposed method provided the highest resolution for 2HG maps to clearly delineate tumor margins and tumor heterogeneity.

CONCLUSIONS

Our results indicate that proposed deep learning methods are effective in enhancing spatial resolution of metabolite maps. Patient results suggest that this may have great clinical potential for image guided precision oncology therapy.

Inter-subject stability and regional concentration estimates of 3D-FID-MRSI in the human brain at 7 T

PURPOSE

Recently, a 3D-concentric ring trajectory (CRT)-based free induction decay (FID)-MRSI sequence was introduced for fast high-resolution metabolic imaging at 7 T. This technique provides metabolic ratio maps of almost the entire brain within clinically feasible scan times, but its robustness has not yet been thoroughly investigated. Therefore, we have assessed quantitative concentration estimates and their variability in healthy volunteers using this approach.

METHODS

We acquired whole-brain 3D-CRT-FID-MRSI at 7 T in 15 min with 3.4 mm nominal isometric resolution in 24 volunteers (12 male, 12 female, mean age 27 ± 6 years). Concentration estimate maps were calculated for 15 metabolites using internal water referencing and evaluated in 55 different regions of interest (ROIs) in the brain. Data quality, mean metabolite concentrations, and their inter-subject coefficients of variation (CVs) were compared for all ROIs.

RESULTS

Of 24 datasets, one was excluded due to motion artifacts. The concentrations of total choline, total creatine, glutamate, myo-inositol, and N-acetylaspartate in 44 regions were estimated within quality thresholds. Inter-subject CVs (mean over 44 ROIs/minimum/maximum) were 9%/5%/19% for total choline, 10%/6%/20% for total creatine, 11%/7%/24% for glutamate, 10%/6%/19% for myo-inositol, and 9%/6%/19% for N-acetylaspartate.

DISCUSSION

We defined the performance of 3D-CRT-based FID-MRSI for metabolite concentration estimate mapping, showing which metabolites could be robustly quantified in which ROIs with which inter-subject CVs expected. However, the basal brain regions and lesser-signal metabolites in particular remain as a challenge due susceptibility effects from the proximity to nasal and auditory cavities. Further improvement in quantification and the mitigation of B₀ /B₁ -field inhomogeneities will be necessary to achieve reliable whole-brain coverage.

Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge

Dynamic imaging is a beneficial tool for interventions to assess physiological changes. Nonetheless during dynamic MRI, while achieving a high temporal resolution, the spatial resolution is compromised. To overcome this spatio-temporal trade-off, this research presents a super-resolution (SR) MRI reconstruction with prior knowledge based fine-tuning to maximise spatial information while reducing the required scan-time for dynamic MRIs. A U-Net based network with perceptual loss is trained on a benchmark dataset and fine-tuned using one subject-specific static high resolution MRI as prior knowledge to obtain high resolution dynamic images during the inference stage. 3D dynamic data for three subjects were acquired with different parameters to test the generalisation capabilities of the network. The method was tested for different levels of in-plane undersampling for dynamic MRI. The reconstructed dynamic SR results after fine-tuning showed higher similarity with the high resolution ground-truth, while quantitatively achieving statistically significant improvement. The average SSIM of the lowest resolution experimented during this research (6.25% of the k-space) before and after fine-tuning were 0.939 ± 0.008 and 0.957 ± 0.006 respectively. This could theoretically result in an acceleration factor of 16, which can potentially be acquired in less than half a second. The proposed approach shows that the super-resolution MRI reconstruction with prior-information can alleviate the spatio-temporal trade-off in dynamic MRI, even for high acceleration factors.

High-Resolution Magnetic Resonance Spectroscopic Imaging using a Multi-Encoder Attention U-Net with Structural and Adversarial Loss

Common to most medical imaging techniques, the spatial resolution of Magnetic Resonance Spectroscopic Imaging (MRSI) is ultimately limited by the achievable SNR. This work presents a deep learning method for 1H-MRSI spatial resolution enhancement, based on the observation that multi-parametric MRI images provide relevant spatial priors for MRSI enhancement. A Multi-encoder Attention U-Net (MAU-Net) architecture was constructed to process a MRSI metabolic map and three different MRI modalities through separate encoding paths. Spatial attention modules were incorporated to automatically learn spatial weights that highlight salient features for each MRI modality. MAU-Net was trained based on in vivo brain imaging data from patients with high-grade gliomas, using a combined loss function consisting of pixel, structural and adversarial loss. Experimental results showed that the proposed method is able to reconstruct high-quality metabolic maps with a high-resolution of 64×64 from a low-resolution of 16 × 16, with better performance compared to several baseline methods.

Hyperpolarized MRI, functional MRI, MR spectroscopy and CEST to provide metabolic information in vivo

Access to metabolic information in vivo using magnetic resonance (MR) technologies has generally been the niche of MR spectroscopy (MRS) and spectroscopic imaging (MRSI). Metabolic fluxes can be studied using the infusion of substrates labeled with magnetic isotopes, with the use of hyperpolarization especially powerful. Unfortunately, these promising methods are not yet accepted clinically, where fast, simple, and reliable measurement and diagnosis are key. Recent advances in functional MRI and chemical exchange saturation transfer (CEST) MRI allow the use of water imaging to study oxygen metabolism and tissue metabolite levels. These, together with the use of novel data analysis approaches such as machine learning for all of these metabolic MR approaches, are increasing the likelihood of their clinical translation.

Improving phase-based conductivity reconstruction by means of deep learning-based denoising of B 1 + phase data for 3T MRI

PURPOSE

To denoise B 1 + phase using a deep learning method for phase-based in vivo electrical conductivity reconstruction in a 3T MR system.

METHODS

For B 1 + phase deep-learning denoising, a convolutional neural network (U-net) was chosen. Training was performed on data sets from 10 healthy volunteers. Input data were the real and imaginary components of single averaged spin-echo data (SNR = 45), which was used to approximate the B 1 + phase. For label data, multiple signal-averaged spin-echo data (SNR = 128) were used. Testing was performed on in silico and in vivo data. Reconstructed conductivity maps were derived using phase-based conductivity reconstructions. Additionally, we investigated the usability of the network to various SNR levels, imaging contrasts, and anatomical sites (ie, T₁ , T₂ , and proton density-weighted brain images and proton density-weighted breast images. In addition, conductivity reconstructions from deep learning-based denoised data were compared with conventional image filters, which were used for data denoising in electrical properties tomography (ie, the Gaussian filtering and the Savitzky-Golay filtering).

RESULTS

The proposed deep learning-based denoising approach showed improvement for B 1 + phase for both in silico and in vivo experiments with reduced quantitative error measures compared with other methods. Subsequently, this resulted in an improvement of reconstructed conductivity maps from the denoised B 1 + phase with deep learning.

CONCLUSION

The results suggest that the proposed approach can be used as an alternative preprocessing method to denoise B 1 + maps for phase-based conductivity reconstruction without relying on image filters or signal averaging.

Accelerated MR spectroscopic imaging-a review of current and emerging techniques

Over more than 30 years in vivo MR spectroscopic imaging (MRSI) has undergone an enormous evolution from theoretical concepts in the early 1980s to the robust imaging technique that it is today. The development of both fast and efficient sampling and reconstruction techniques has played a fundamental role in this process. State-of-the-art MRSI has grown from a slow purely phase-encoded acquisition technique to a method that today combines the benefits of different acceleration techniques. These include shortening of repetition times, spatial-spectral encoding, undersampling of k-space and time domain, and use of spatial-spectral prior knowledge in the reconstruction. In this way in vivo MRSI has considerably advanced in terms of spatial coverage, spatial resolution, acquisition speed, artifact suppression, number of detectable metabolites and quantification precision. Acceleration not only has been the enabling factor in high-resolution whole-brain 1 H-MRSI, but today is also common in non-proton MRSI (31 P, 2 H and 13 C) and applied in many different organs. In this process, MRSI techniques had to constantly adapt, but have also benefitted from the significant increase of magnetic field strength boosting the signal-to-noise ratio along with high gradient fidelity and high-density receive arrays. In combination with recent trends in image reconstruction and much improved computation power, these advances led to a number of novel developments with respect to MRSI acceleration. Today MRSI allows for non-invasive and non-ionizing mapping of the spatial distribution of various metabolites' tissue concentrations in animals or humans, is applied for clinical diagnostics and has been established as an important tool for neuro-scientific and metabolism research. This review highlights the developments of the last five years and puts them into the context of earlier MRSI acceleration techniques. In addition to 1 H-MRSI it also includes other relevant nuclei and is not limited to certain body regions or specific applications.

Deep learning can accelerate and quantify simulated localized correlated spectroscopy

Nuclear magnetic resonance spectroscopy (MRS) allows for the determination of atomic structures and concentrations of different chemicals in a biochemical sample of interest. MRS is used in vivo clinically to aid in the diagnosis of several pathologies that affect metabolic pathways in the body. Typically, this experiment produces a one dimensional (1D) 1H spectrum containing several peaks that are well associated with biochemicals, or metabolites. However, since many of these peaks overlap, distinguishing chemicals with similar atomic structures becomes much more challenging. One technique capable of overcoming this issue is the localized correlated spectroscopy (L-COSY) experiment, which acquires a second spectral dimension and spreads overlapping signal across this second dimension. Unfortunately, the acquisition of a two dimensional (2D) spectroscopy experiment is extremely time consuming. Furthermore, quantitation of a 2D spectrum is more complex. Recently, artificial intelligence has emerged in the field of medicine as a powerful force capable of diagnosing disease, aiding in treatment, and even predicting treatment outcome. In this study, we utilize deep learning to: (1) accelerate the L-COSY experiment and (2) quantify L-COSY spectra. All training and testing samples were produced using simulated metabolite spectra for chemicals found in the human body. We demonstrate that our deep learning model greatly outperforms compressed sensing based reconstruction of L-COSY spectra at higher acceleration factors. Specifically, at four-fold acceleration, our method has less than 5% normalized mean squared error, whereas compressed sensing yields 20% normalized mean squared error. We also show that at low SNR (25% noise compared to maximum signal), our deep learning model has less than 8% normalized mean squared error for quantitation of L-COSY spectra. These pilot simulation results appear promising and may help improve the efficiency and accuracy of L-COSY experiments in the future.

Watching the embryo: Evolution of the microscope for the study of embryogenesis

Embryos and microscopes share a long, remarkable history and biologists have always been intrigued to watch how embryos develop under the microscope. Here we discuss the advances in microscopy which have greatly influenced our current understanding of embryogenesis. We highlight the evolution of microscopes and the optical technologies that have been instrumental in studying various developmental processes. These imaging modalities provide mechanistic insights into the dynamic cellular and molecular events which drive lineage commitment and morphogenetic changes in the developing embryo. We begin the journey with a brief history of microscopy to study embryos. First, we review the principles and optics of light, fluorescence, confocal, and electron microscopy which have been key techniques for imaging cellular and molecular events during embryonic development. Next, we discuss recent key imaging modalities such as light-sheet microscopy, which are suitable for whole embryo imaging. Further, we highlight imaging techniques like multiphoton and super resolution microscopy for beyond light diffraction limit, high resolution imaging. Lastly, we review some of the scattering-based imaging methods and techniques used for imaging human embryos.

Investigation of a Novel Deep Learning-Based Computed Tomography Perfusion Mapping Framework for Functional Lung Avoidance Radiotherapy

Functional lung avoidance radiation therapy aims to minimize dose delivery to the normal lung tissue while favoring dose deposition in the defective lung tissue based on the regional function information. However, the clinical acquisition of pulmonary functional images is resource-demanding, inconvenient, and technically challenging. This study aims to investigate the deep learning-based lung functional image synthesis from the CT domain. Forty-two pulmonary macro-aggregated albumin SPECT/CT perfusion scans were retrospectively collected from the hospital. A deep learning-based framework (including image preparation, image processing, and proposed convolutional neural network) was adopted to extract features from 3D CT images and synthesize perfusion as estimations of regional lung function. Ablation experiments were performed to assess the effects of each framework component by removing each element of the framework and analyzing the testing performances. Major results showed that the removal of the CT contrast enhancement component in the image processing resulted in the largest drop in framework performance, compared to the optimal performance (~12%). In the CNN part, all the three components (residual module, ROI attention, and skip attention) were approximately equally important to the framework performance; removing one of them resulted in a 3–5% decline in performance. The proposed CNN improved ~4% overall performance and ~350% computational efficiency, compared to the U-Net model. The deep convolutional neural network, in conjunction with image processing for feature enhancement, is capable of feature extraction from CT images for pulmonary perfusion synthesis. In the proposed framework, image processing, especially CT contrast enhancement, plays a crucial role in the perfusion synthesis. This CTPM framework provides insights for relevant research studies in the future and enables other researchers to leverage for the development of optimized CNN models for functional lung avoidance radiation therapy.

Frequency and phase correction of J-difference edited MR spectra using deep learning

PURPOSE

To investigate whether a deep learning-based (DL) approach can be used for frequency-and-phase correction (FPC) of MEGA-edited MRS data.

METHODS

Two neural networks (1 for frequency, 1 for phase) consisting of fully connected layers were trained and validated using simulated MEGA-edited MRS data. This DL-FPC was subsequently tested and compared to a conventional approach (spectral registration [SR]) and to a model-based SR implementation (mSR) using in vivo MEGA-edited MRS datasets. Additional artificial offsets were added to these datasets to further investigate performance.

RESULTS

The validation showed that DL-based FPC was capable of correcting within 0.03 Hz of frequency and 0.4°of phase offset for unseen simulated data. DL-based FPC performed similarly to SR for the unmanipulated in vivo test datasets. When additional offsets were added to these datasets, the networks still performed well. However, although SR accurately corrected for smaller offsets, it often failed for larger offsets. The mSR algorithm performed well for larger offsets, which was because the model was generated from the in vivo datasets. In addition, the computation times were much shorter using DL-based FPC or mSR compared to SR for heavily distorted spectra.

CONCLUSION

These results represent a proof of principle for the use of DL for preprocessing MRS data.

Magnetic resonance spectroscopic imaging in gliomas: clinical diagnosis and radiotherapy planning

The reprogramming of cellular metabolism is a hallmark of cancer diagnosis and prognosis. Proton magnetic resonance spectroscopic imaging (MRSI) is a non-invasive diagnostic technique for investigating brain metabolism to establish cancer diagnosis and IDH gene mutation diagnosis as well as facilitate pre-operative planning and treatment response monitoring. By allowing tissue metabolism to be quantified, MRSI provides added value to conventional MRI. MRSI can generate metabolite maps from a single volume or multiple volume elements within the whole brain. Metabolites such as NAA, Cho and Cr, as well as their ratios Cho:NAA ratio and Cho:Cr ratio, have been used to provide tumor diagnosis and aid in radiation therapy planning as well as treatment assessment. In addition to these common metabolites, 2-hydroxygluterate (2HG) has also been quantified using MRSI following the recent discovery of IDH mutations in gliomas. This has opened up targeted drug development to inhibit the mutant IDH pathway. This review provides guidance on MRSI in brain gliomas, including its acquisition, analysis methods, and evolving clinical applications.

Using deep learning to model the biological dose prediction on bulky lung cancer patients of partial stereotactic ablation radiotherapy

PURPOSE

To develop a biological dose prediction model considering tissue bio-reactions in addition to patient anatomy for achieving a more comprehensive evaluation of tumor control and promoting the automatic planning of bulky lung cancer.

METHODS

A database containing images and partial stereotactic ablation boost radiotherapy (P-SABR) plans of 94 bulky lung cancer patients was studied. Patient-specific parameters of gross tumor boost volume (GTVb), planning gross target volume (PGTV), and identified organs at risk (OARs) were extracted via Numpy and simple ITK. The original dose and structure maps for P-SABR patients were resampled to have a voxel resolution of 3.9 × 3.9 × 3 mm³ . Biological equivalent dose (BED) distributions were reprogrammed based on physical dose volumes. A developed deep learning architecture, Nestnet, was adopted as the training framework. We utilized two approaches for data organization to correlate the structures and BED: (a) BED programming before training model (B-Nestnet); (b) BED programming after the training process (D-B Nestnet). The early-stop mechanism was adopted on the validation set to avoid overfitting. The evaluation criteria of predictive accuracy contain the minimum BED of GTVb and PGTV, the maximum and the mean BED of all targets, BED-volume metrics. For comparison, we also used the original Unet for BED prediction. The absolute differences were statistically analyzed with the paired-samples t test.

RESULTS

The statistical outcomes demonstrate that D-B Nestnet model predicts biological dose distributions accurately. The average absolute biases of [max, mean] BED for GTVb, PGTV are [2.1%, 3.3%] and [2.1%, 4.7%], respectively. Averaging across most of OARs, the D-B Nestnet model is capable of predicting the errors of the max and mean BED within 6.3% and 6.1%, respectively. While the compared models performed worse with averaged max and mean BED prediction errors surpassing 10% on some specific OARs.

CONCLUSIONS

The study developed a D-B Nestnet model capable of predicting BED distribution accurately for bulky lung cancer patients in P-SABR. The predicted BED map enables a quick intuitive evaluation of tumor ablation, modification of the ablation range to improve BED of tumor targets, and quality assessment. It represents a major step forward toward automated P-SABR planning on bulky lung cancer in real clinical practice.

Operating a treatment planning system using a deep-reinforcement learning-based virtual treatment planner for prostate cancer intensity-modulated radiation therapy treatment planning

PURPOSE

In the treatment planning process of intensity-modulated radiation therapy (IMRT), a human planner operates the treatment planning system (TPS) to adjust treatment planning parameters, for example, dose volume histogram (DVH) constraints' locations and weights, to achieve a satisfactory plan for each patient. This process is usually time-consuming, and the plan quality depends on planer's experience and available planning time. In this study, we proposed to model the behaviors of human planners in treatment planning by a deep reinforcement learning (DRL)-based virtual treatment planner network (VTPN), such that it can operate the TPS in a human-like manner for treatment planning.

METHODS AND MATERIALS

Using prostate cancer IMRT as an example, we established the VTPN using a deep neural network developed. We considered an in-house optimization engine with a weighted quadratic objective function. Virtual treatment planner network was designed to observe an intermediate plan DVHs and decide the action to improve the plan by changing weights and threshold dose in the objective function. We trained the VTPN in an end-to-end DRL process in 10 patient cases. A plan score was used to measure plan quality. We demonstrated the feasibility and effectiveness of the trained VTPN in another 64 patient cases.

RESULTS

Virtual treatment planner network was trained to spontaneously learn how to adjust treatment planning parameters to generate high-quality treatment plans. In the 64 testing cases, with initialized parameters, quality score was 4.97 (±2.02), with 9.0 being the highest possible score. Using VTPN to perform treatment planning improved quality score to 8.44 (±0.48).

CONCLUSIONS

To our knowledge, this was the first time that intelligent treatment planning behaviors of human planner in external beam IMRT are autonomously encoded in an artificial intelligence system. The trained VTPN is capable of behaving in a human-like way to produce high-quality plans.

An introduction to deep learning in medical physics: advantages, potential, and challenges

As one of the most popular approaches in artificial intelligence, deep learning (DL) has attracted a lot of attention in the medical physics field over the past few years. The goals of this topical review article are twofold. First, we will provide an overview of the method to medical physics researchers interested in DL to help them start the endeavor. Second, we will give in-depth discussions on the DL technology to make researchers aware of its potential challenges and possible solutions. As such, we divide the article into two major parts. The first part introduces general concepts and principles of DL and summarizes major research resources, such as computational tools and databases. The second part discusses challenges faced by DL, present available methods to mitigate some of these challenges, as well as our recommendations.

Super-Resolution Whole-Brain 3D MR Spectroscopic Imaging for Mapping D-2-Hydroxyglutarate and Tumor Metabolism in Isocitrate Dehydrogenase 1-mutated Human Gliomas

Background Isocitrate dehydrogenase (IDH) mutations are highly frequent in glioma, producing high levels of the oncometabolite D-2-hydroxyglutarate (D-2HG). Hence, D-2HG represents a valuable imaging marker for IDH-mutated human glioma. Purpose To develop and evaluate a super-resolution three-dimensional (3D) MR spectroscopic imaging strategy to map D-2HG and tumor metabolism in IDH-mutated human glioma. Materials and Methods Between March and September 2018, participants with IDH1-mutated gliomas and healthy participants were prospectively scanned with a 3-T whole-brain 3D MR spectroscopic imaging protocol optimized for D-2HG. The acquired D-2HG maps with a voxel size of 5.2 × 5.2 × 12 mm were upsampled to a voxel size of 1.7 × 1.7 × 3 mm using a super-resolution method that combined weighted total variation, feature-based nonlocal means, and high-spatial-resolution anatomic imaging priors. Validation with simulated healthy and patient data and phantom measurements was also performed. The Mann-Whitney U test was used to check that the proposed super-resolution technique yields the highest peak signal-to-noise ratio and structural similarity index. Results Three participants with IDH1-mutated gliomas (mean age, 50 years ± 21 [standard deviation]; two men) and three healthy participants (mean age, 32 years ± 3; two men) were scanned. Twenty healthy participants (mean age, 33 years ± 5; 16 men) underwent a simulation of upsampled MR spectroscopic imaging. Super-resolution upsampling improved peak signal-to-noise ratio and structural similarity index by 62% (P < .05) and 7.3% (P < .05), respectively, for simulated data when compared with spline interpolation. Correspondingly, the proposed method significantly improved tissue contrast and structural information for the acquired 3D MR spectroscopic imaging data. Conclusion High-spatial-resolution whole-brain D-2-hydroxyglutarate imaging is possible in isocitrate dehydrogenase 1-mutated human glioma by using a super-resolution framework to upsample three-dimensional MR spectroscopic images acquired at lower resolution. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Huang and Lin in this issue.

Intelligent inverse treatment planning via deep reinforcement learning, a proof-of-principle study in high dose-rate brachytherapy for cervical cancer

Inverse treatment planning in radiation therapy is formulated as solving optimization problems. The objective function and constraints consist of multiple terms designed for different clinical and practical considerations. Weighting factors of these terms are needed to define the optimization problem. While a treatment planning optimization engine can solve the optimization problem with given weights, adjusting the weights to yield a high-quality plan is typically performed by a human planner. Yet the weight-tuning task is labor intensive, time consuming, and it critically affects the final plan quality. An automatic weight-tuning approach is strongly desired. The procedure of weight adjustment to improve the plan quality is essentially a decision-making problem. Motivated by the tremendous success in deep learning for decision making with human-level intelligence, we propose a novel framework to adjust the weights in a human-like manner. This study used inverse treatment planning in high-dose-rate brachytherapy (HDRBT) for cervical cancer as an example. We developed a weight-tuning policy network (WTPN) that observes dose volume histograms of a plan and outputs an action to adjust organ weighting factors, similar to the behaviors of a human planner. We trained the WTPN via end-to-end deep reinforcement learning. Experience replay was performed with the epsilon greedy algorithm. After training was completed, we applied the trained WTPN to guide treatment planning of five testing patient cases. It was found that the trained WTPN successfully learnt the treatment planning goals and was able to guide the weight tuning process. On average, the quality score of plans generated under the WTPN's guidance was improved by ~8.5% compared to the initial plan with arbitrarily set weights, and by 10.7% compared to the plans generated by human planners. To our knowledge, this was the first time that a tool was developed to adjust organ weights for the treatment planning optimization problem in a human-like fashion based on intelligence learnt from a training process, which was different from existing strategies based on pre-defined rules. The study demonstrated potential feasibility to develop intelligent treatment planning approaches via deep reinforcement learning.