Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning

Verification of image quality improvement by deep learning reconstruction to 1.5 T MRI in T2-weighted images of the prostate gland

This study aimed to evaluate whether the image quality of 1.5 T magnetic resonance imaging (MRI) of the prostate is equal to or higher than that of 3 T MRI by applying deep learning reconstruction (DLR). To objectively analyze the images from the 13 healthy volunteers, we measured the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of the images obtained by the 1.5 T scanner with and without DLR, as well as for images obtained by the 3 T scanner. In the subjective, T2W images of the prostate were visually evaluated by two board-certified radiologists. The SNRs and CNRs in 1.5 T images with DLR were higher than that in 3 T images. Subjective image scores were better for 1.5 T images with DLR than 3 T images. The use of the DLR technique in 1.5 T MRI substantially improved the SNR and image quality of T2W images of the prostate gland, as compared to 3 T MRI.

Dual-space high-frequency learning for transformer-based MRI super-resolution

BACKGROUND AND OBJECTIVE

Magnetic resonance imaging (MRI) can provide rich and detailed high-contrast information of soft tissues, while the scanning of MRI is time-consuming. To accelerate MR imaging, a variety of Transformer-based single image super-resolution methods are proposed in recent years, achieving promising results thanks to their superior capability of capturing long-range dependencies. Nevertheless, most existing works prioritize the design of transformer attention blocks to capture global information. The local high-frequency details, which are pivotal to faithful MRI restoration, are unfortunately neglected.

METHODS

In this work, we propose a high-frequency enhanced learning scheme to effectively improve the awareness of high frequency information in current Transformer-based MRI single image super-resolution methods. Specifically, we present two entirely plug-and-play modules designed to equip Transformer-based networks with the ability to recover high-frequency details from dual spaces: 1) in the feature space, we design a high-frequency block (Hi-Fe block) paralleled with Transformer-based attention layers to extract rich high-frequency features; while 2) in the image intensity space, we tailor a high-frequency amplification module (HFA) to further refine the high-frequency details. By fully exploiting the merits of the two modules, our framework can recover abundant and diverse high-frequency information, rendering faithful MRI super-resolved results with fine details.

RESULTS

We integrated our modules with six Transformer-based models and conducted experiments across three datasets. The results indicate that our plug-and-play modules can enhance the super-resolution performance of all foundational models to varying degrees, surpassing the capabilities of existing state-of-the-art single image super-resolution networks.

CONCLUSION

Comprehensive comparison of super-resolution images and high-frequency maps from various methods, clearly demonstrating that our module possesses the capability to restore high-frequency information, showing huge potential in clinical practice for accelerated MRI reconstruction.

Deep learning-based super-resolution of structural brain MRI at 1.5 T: application to quantitative volume measurement

OBJECTIVE

This study investigated the feasibility of using deep learning-based super-resolution (DL-SR) technique on low-resolution (LR) images to generate high-resolution (HR) MR images with the aim of scan time reduction. The efficacy of DL-SR was also assessed through the application of brain volume measurement (BVM).

MATERIALS AND METHODS

In vivo brain images acquired with 3D-T1W from various MRI scanners were utilized. For model training, LR images were generated by downsampling the original 1 mm-2 mm isotropic resolution images. Pairs of LR and HR images were used for training 3D residual dense net (RDN). For model testing, actual scanned 2 mm isotropic resolution 3D-T1W images with one-minute scan time were used. Normalized root-mean-square error (NRMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) were used for model evaluation. The evaluation also included brain volume measurement, with assessments of subcortical brain regions.

RESULTS

The results showed that DL-SR model improved the quality of LR images compared with cubic interpolation, as indicated by NRMSE (24.22% vs 30.13%), PSNR (26.19 vs 24.65), and SSIM (0.96 vs 0.95). For volumetric assessments, there were no significant differences between DL-SR and actual HR images (p > 0.05, Pearson's correlation > 0.90) at seven subcortical regions.

DISCUSSION

The combination of LR MRI and DL-SR enables addressing prolonged scan time in 3D MRI scans while providing sufficient image quality without affecting brain volume measurement.

Deep Learning Reconstruction for DWIs by EPI and FASE Sequences for Head and Neck Tumors

BACKGROUND

Diffusion-weighted images (DWI) obtained by echo-planar imaging (EPI) are frequently degraded by susceptibility artifacts. It has been suggested that DWI obtained by fast advanced spin-echo (FASE) or reconstructed with deep learning reconstruction (DLR) could be useful for image quality improvements. The purpose of this investigation using in vitro and in vivo studies was to determine the influence of sequence difference and of DLR for DWI on image quality, apparent diffusion coefficient (ADC) evaluation, and differentiation of malignant from benign head and neck tumors.

METHODS

For the in vitro study, a DWI phantom was scanned by FASE and EPI sequences and reconstructed with and without DLR. Each ADC within the phantom for each DWI was then assessed and correlated for each measured ADC and standard value by Spearman's rank correlation analysis. For the in vivo study, DWIs obtained by EPI and FASE sequences were also obtained for head and neck tumor patients. Signal-to-noise ratio (SNR) and ADC were then determined based on ROI measurements, while SNR of tumors and ADC were compared between all DWI data sets by means of Tukey's Honest Significant Difference test.

RESULTS

For the in vitro study, all correlations between measured ADC and standard reference were significant and excellent (0.92 ≤ ρ ≤ 0.99, p < 0.0001). For the in vivo study, the SNR of FASE with DLR was significantly higher than that of FASE without DLR (p = 0.02), while ADC values for benign and malignant tumors showed significant differences between each sequence with and without DLR (p < 0.05).

CONCLUSION

In comparison with EPI sequence, FASE sequence and DLR can improve image quality and distortion of DWIs without significantly influencing ADC measurements or differentiation capability of malignant from benign head and neck tumors.

Neighborhood evaluator for efficient super-resolution reconstruction of 2D medical images

BACKGROUND

Deep learning-based super-resolution (SR) algorithms aim to reconstruct low-resolution (LR) images into high-fidelity high-resolution (HR) images by learning the low- and high-frequency information. Experts' diagnostic requirements are fulfilled in medical application scenarios through the high-quality reconstruction of LR digital medical images.

PURPOSE

Medical image SR algorithms should satisfy the requirements of arbitrary resolution and high efficiency in applications. However, there is currently no relevant study available. Several SR research on natural images have accomplished the reconstruction of resolutions without limitations. However, these methodologies provide challenges in meeting medical applications due to the large scale of the model, which significantly limits efficiency. Hence, we suggest a highly effective method for reconstructing medical images at any desired resolution.

METHODS

Statistical features of medical images exhibit greater continuity in the region of neighboring pixels than natural images. Hence, the process of reconstructing medical images is comparatively less challenging. Utilizing this property, we develop a neighborhood evaluator to represent the continuity of the neighborhood while controlling the network's depth.

RESULTS

The suggested method has superior performance across seven scales of reconstruction, as evidenced by experiments conducted on panoramic radiographs and two external public datasets. Furthermore, the proposed network significantly decreases the parameter count by over 20× and the computational workload by over 10× compared to prior researches. On large-scale reconstruction, the inference speed can be enhanced by over 5×.

CONCLUSION

The novel proposed SR strategy for medical images performs efficient reconstruction at arbitrary resolution, marking a significant breakthrough in the field. The given scheme facilitates the implementation of SR in mobile medical platforms.

Enhancing Knee MR Image Clarity through Image Domain Super-Resolution Reconstruction

This study introduces a hybrid analytical super-resolution (SR) pipeline aimed at enhancing the resolution of medical magnetic resonance imaging (MRI) scans. The primary objective is to overcome the limitations of clinical MRI resolution without the need for additional expensive hardware. The proposed pipeline involves three key steps: pre-processing to re-slice and register the image stacks; SR reconstruction to combine information from three orthogonal image stacks to generate a high-resolution image stack; and post-processing using an artefact reduction convolutional neural network (ARCNN) to reduce the block artefacts introduced during SR reconstruction. The workflow was validated on a dataset of six knee MRIs obtained at high resolution using various sequences. Quantitative analysis of the method revealed promising results, showing an average mean error of 1.40 ± 2.22% in voxel intensities between the SR denoised images and the original high-resolution images. Qualitatively, the method improved out-of-plane resolution while preserving in-plane image quality. The hybrid SR pipeline also displayed robustness across different MRI sequences, demonstrating potential for clinical application in orthopaedics and beyond. Although computationally intensive, this method offers a viable alternative to costly hardware upgrades and holds promise for improving diagnostic accuracy and generating more anatomically accurate models of the human body.

Accelerated Musculoskeletal Magnetic Resonance Imaging

With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning-based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning-based accelerated MRI techniques, including undersampled MR image reconstruction, super-resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: Stage 1.

A multimodal pipeline for image correction and registration of mass spectrometry imaging with microscopy

The integration of matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) and histology plays a pivotal role in advancing our understanding of complex heterogeneous tissues, which provides a comprehensive description of biological tissue with both wide molecule coverage and high lateral resolution. Herein, we proposed a novel strategy for the correction and registration of MALDI MSI data with hematoxylin & eosin (H&E) staining images. To overcome the challenges of discrepancies in spatial resolution towards the unification of the two imaging modalities, a deep learning-based interpolation algorithm for MALDI MSI data was constructed, which enables spatial coherence and the following orientation matching between images. Coupled with the affine transformation (AT) and the subsequent moving least squares algorithm, the two types of images from one rat brain tissue section were aligned automatically with high accuracy. Moreover, we demonstrated the practicality of the developed pipeline by projecting it to a rat cerebral ischemia-reperfusion injury model, which would help decipher the link between molecular metabolism and pathological interpretation towards microregion. This new approach offers the chance for other types of bioimaging to boost the field of multimodal image fusion.

AI-assisted compressed sensing and parallel imaging sequences for MRI of patients with nasopharyngeal carcinoma: comparison of their capabilities in terms of examination time and image quality

OBJECTIVE

To compare examination time and image quality between artificial intelligence (AI)-assisted compressed sensing (ACS) technique and parallel imaging (PI) technique in MRI of patients with nasopharyngeal carcinoma (NPC).

METHODS

Sixty-six patients with pathologically confirmed NPC underwent nasopharynx and neck examination using a 3.0-T MRI system. Transverse T2-weighted fast spin-echo (FSE) sequence, transverse T1-weighted FSE sequence, post-contrast transverse T1-weighted FSE sequence, and post-contrast coronal T1-weighted FSE were obtained by both ACS and PI techniques, respectively. The signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and duration of scanning of both sets of images analyzed by ACS and PI techniques were compared. The images from the ACS and PI techniques were scored for lesion detection, margin sharpness of lesions, artifacts, and overall image quality using the 5-point Likert scale.

RESULTS

The examination time with ACS technique was significantly shorter than that with PI technique (p < 0.0001). The comparison of SNR and CNR showed that ACS technique was significantly superior with PI technique (p < 0.005). Qualitative image analysis showed that the scores of lesion detection, margin sharpness of lesions, artifacts, and overall image quality were higher in the ACS sequences than those in the PI sequences (p < 0.0001). Inter-observer agreement was evaluated for all qualitative indicators for each method, in which the results showed satisfactory-to-excellent agreement (p < 0.0001).

CONCLUSION

Compared with the PI technique, the ACS technique for MR examination of NPC can not only shorten scanning time but also improve image quality.

CLINICAL RELEVANCE STATEMENT

The artificial intelligence (AI)-assisted compressed sensing (ACS) technique shortens examination time for patients with nasopharyngeal carcinoma, while improving the image quality and examination success rate, which will benefit more patients.

KEY POINTS

• Compared with the parallel imaging (PI) technique, the artificial intelligence (AI)-assisted compressed sensing (ACS) technique not only reduced examination time, but also improved image quality. • Artificial intelligence (AI)-assisted compressed sensing (ACS) pulls the state-of-the-art deep learning technique into the reconstruction procedure and helps find an optimal balance of imaging speed and image quality.

Highly accelerated knee magnetic resonance imaging using deep neural network (DNN)-based reconstruction: prospective, multi-reader, multi-vendor study

In this prospective, multi-reader, multi-vendor study, we evaluated the performance of a commercially available deep neural network (DNN)-based MR image reconstruction in enabling accelerated 2D fast spin-echo (FSE) knee imaging. Forty-five subjects were prospectively enrolled and randomly divided into three 3T MRIs. Conventional 2D FSE and accelerated 2D FSE sequences were acquired for each subject, and the accelerated FSE images were reconstructed and enhanced with DNN-based reconstruction software (FSE-DNN). Quantitative assessments and diagnostic performances were independently evaluated by three musculoskeletal radiologists. For statistical analyses, paired t-tests, and Pearson's correlation were used for image quality comparison and inter-reader agreements. Accelerated FSE-DNN reduced scan times by 41.0% on average. FSE-DNN showed better SNR and CNR (p < 0.001). Overall image quality of FSE-DNN was comparable (p > 0.05), and diagnostic performances of FSE-DNN showed comparable lesion detection. Two of cartilage lesions were under-graded or over-graded (n = 2) while there was no significant difference in other image sets (n = 43). Overall inter-reader agreement between FSE-conventional and FSE-DNN showed good agreement (R2 = 0.76; p < 0.001). In conclusion, DNN-based reconstruction can be applied to accelerated knee imaging in multi-vendor MRI scanners, with reduced scan time and comparable image quality. This study suggests the potential for DNN-accelerated knee MRI in clinical practice.

Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges

Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.

The application of machine learning in early diagnosis of osteoarthritis: a narrative review

Osteoarthritis (OA) is the commonest musculoskeletal disease worldwide, with an increasing prevalence due to aging. It causes joint pain and disability, decreased quality of life, and a huge burden on healthcare services for society. However, the current main diagnostic methods are not suitable for early diagnosing patients of OA. The use of machine learning (ML) in OA diagnosis has increased dramatically in the past few years. Hence, in this review article, we describe the research progress in the application of ML in the early diagnosis of OA, discuss the current trends and limitations of ML approaches, and propose future research priorities to apply the tools in the field of OA. Accurate ML-based predictive models with imaging techniques that are sensitive to early changes in OA ahead of the emergence of clinical features are expected to address the current dilemma. The diagnostic ability of the fusion model that combines multidimensional information makes patient-specific early diagnosis and prognosis estimation of OA possible in the future.

Emerging Technology in Musculoskeletal MRI and CT

This article provides a focused overview of emerging technology in musculoskeletal MRI and CT. These technological advances have primarily focused on decreasing examination times, obtaining higher quality images, providing more convenient and economical imaging alternatives, and improving patient safety through lower radiation doses. New MRI acceleration methods using deep learning and novel reconstruction algorithms can reduce scanning times while maintaining high image quality. New synthetic techniques are now available that provide multiple tissue contrasts from a limited amount of MRI and CT data. Modern low-field-strength MRI scanners can provide a more convenient and economical imaging alternative in clinical practice, while clinical 7.0-T scanners have the potential to maximize image quality. Three-dimensional MRI curved planar reformation and cinematic rendering can provide improved methods for image representation. Photon-counting detector CT can provide lower radiation doses, higher spatial resolution, greater tissue contrast, and reduced noise in comparison with currently used energy-integrating detector CT scanners. Technological advances have also been made in challenging areas of musculoskeletal imaging, including MR neurography, imaging around metal, and dual-energy CT. While the preliminary results of these emerging technologies have been encouraging, whether they result in higher diagnostic performance requires further investigation.

Development of a Super-Resolution Scheme for Pediatric Magnetic Resonance Brain Imaging Through Convolutional Neural Networks

Pediatric medical imaging represents a real challenge for physicians, as children who are patients often move during the examination, and it causes the appearance of different artifacts in the images. Thus, it is not possible to obtain good quality images for this target population limiting the possibility of evaluation and diagnosis in certain pathological conditions. Specifically, magnetic resonance imaging (MRI) is a technique that requires long acquisition times and, therefore, demands the use of sedation or general anesthesia to avoid the movement of the patient, which is really damaging in this specific population. Because ALARA (as low as reasonably achievable) principles should be considered for all imaging studies, one of the most important reasons for establishing novel MRI imaging protocols is to avoid the harmful effects of anesthesia/sedation. In this context, ground-breaking concepts and novel technologies, such as artificial intelligence, can help to find a solution to these challenges while helping in the search for underlying disease mechanisms. The use of new MRI protocols and new image acquisition and/or pre-processing techniques can aid in the development of neuroimaging studies for children evaluation, and their translation to pediatric populations. In this paper, a novel super-resolution method based on a convolutional neural network (CNN) in two and three dimensions to automatically increase the resolution of pediatric brain MRI acquired in a reduced time scheme is proposed. Low resolution images have been generated from an original high resolution dataset and used as the input of the CNN, while several scaling factors have been assessed separately. Apart from a healthy dataset, we also tested our model with pathological pediatric MRI, and it successfully recovers the original image quality in both visual and quantitative ways, even for available examples of dysplasia lesions. We hope then to establish the basis for developing an innovative free-sedation protocol in pediatric anatomical MRI acquisition.

Convolutional Neural Network for Denoising Left Ventricle Magnetic Resonance Images

Medical image processing is critical in disease detection and prediction. For example, they locate lesions and measure an organ's morphological structures. Currently, cardiac magnetic resonance imaging (CMRI) plays an essential role in cardiac motion tracking and analyzing regional and global heart functions with high accuracy and reproducibility. Cardiac MRI datasets are images taken during the heart's cardiac cycles. These datasets require expert labeling to accurately recognize features and train neural networks to predict cardiac disease. Any erroneous prediction caused by image impairment will impact patients' diagnostic decisions. As a result, image preprocessing is used, including enhancement tools such as filtering and denoising. This paper introduces a denoising algorithm that uses a convolution neural network (CNN) to delineate left ventricle (LV) contours (endocardium and epicardium borders) from MRI images. With only a small amount of training data from the EMIDEC database, this network performs well for MRI image denoising.

Comparison of utility of deep learning reconstruction on 3D MRCPs obtained with three different k-space data acquisitions in patients with IPMN

OBJECTIVE

To compare the utility of deep learning reconstruction (DLR) for improving acquisition time, image quality, and intraductal papillary mucinous neoplasm (IPMN) evaluation for 3D MRCP obtained with parallel imaging (PI), multiple k-space data acquisition for each repetition time (TR) technique (Fast 3D mode multiple: Fast 3Dm) and compressed sensing (CS) with PI.

MATERIALS AND METHODS

A total of 32 IPMN patients who had undergone 3D MRCPs obtained with PI, Fast 3Dm, and CS with PI and reconstructed with and without DLR were retrospectively included in this study. Acquisition time, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) obtained with all protocols were compared using Tukey's HSD test. Results of endoscopic ultrasound, ERCP, surgery, or pathological examination were determined as standard reference, and distribution classifications were compared among all 3D MRCP protocols by McNemar's test.

RESULTS

Acquisition times of Fast 3Dm and CS with PI with and without DLR were significantly shorter than those of PI with and without DLR (p < 0.05). Each MRCP sequence with DLR showed significantly higher SNRs and CNRs than those without DLR (p < 0.05). IPMN distribution accuracy of PI with and without DLR and Fast 3Dm with DLR was significantly higher than that of Fast 3Dm without DLR and CS with PI without DLR (p < 0.05).

CONCLUSION

DLR is useful for improving image quality and IPMN evaluation capability on 3D MRCP obtained with PI, Fast 3Dm, or CS with PI. Moreover, Fast 3Dm and CS with PI may play as substitution to PI for MRCP in patients with IPMN.

KEY POINTS

• Mean examination times of multiple k-space data acquisitions for each TR and compressed sensing with parallel imaging were significantly shorter than that of parallel imaging (p < 0.0001). • When comparing image quality of 3D MRCPs with and without deep learning reconstruction, deep learning reconstruction significantly improved signal-to-noise ratio and contrast-to-noise ratio (p < 0.05). • IPMN distribution accuracies of parallel imaging with and without deep learning reconstruction (with vs. without: 88.0% vs. 88.0%) and multiple k-space data acquisitions for each TR with deep learning reconstruction (86.0%) were significantly higher than those of others (p < 0.05).

Medial compartment cartilage repair and lower extremity biomechanical changes after single-plane high tibial osteotomy of distal tibial tuberosity

OBJECTIVE

To observe the cartilage repair of the medial compartment and the biomechanical changes of the lower extremities after single-plane high tibial osteotomy of distal tibial tuberosity (DTT-HTO).

METHODS

A total of 30 patients, including 11 males and 19 females, who underwent DTT-HTO with arthroscopic operation in our hospital from January 2020 to January 2021 and underwent arthroscopic exploration again during the second operation for internal fixation were enrolled. There were 32 knees, including 13 left knees and 19 right knees. Age ranged from 50 to 78, with an average of 63.20 ± 6.61 years old. All patients completed two surgeries and were followed up, and no adverse events occurred during the period. International Cartilage Repair Society (ICRS) was used to assess the cartilage condition of the medial compartment of the knee during the two surgeries. In this paper, the weight-bearing line ratio (WBLR), the medial proximal tibia angle (MPTA), the femoral tibial angle (FTA) and the posterior tibial slope (PTS) were used to evaluate the changes of the mechanical state of the lower limbs. The visual analogue scale (VAS) score and the Western Ontario and McMaster University Osteoarthritis (WOMAC) Index score were used to evaluate the improvement in knee pain and function.

RESULTS

All patients completed two operations without adverse events and serious complications. The medial compartment cartilage of all patients had different degrees of repair during the second operation, and the difference was statistically significant compared with the first operation (P < 0.05). During the second operation, the WBLR was corrected from (17.69 ± 2.16)% to (60.90 ± 1.97)%, the MPTA was corrected from (80.72 ± 1.61)° to (89.91 ± 2.58)°, the FTA was corrected from (182.31 ± 3.03)° to (171.81 ± 2.24)°, the difference was statistically significant (P < 0.05). There was no statistical difference in PTS between the two surgeries (P > 0.05). At the second operation, the VAS score decreased from 7.50 ± 1.34 to 0.34 ± 0.85, the WOMAC score decreased from 119.50 ± 10.43 to 46.25 ± 4.13, and the difference was statistically significant (P < 0.05).

CONCLUSION

DTT-HTO can significantly correct the weight-bearing line, restore the biomechanical parameters of the lower limb to the normal range, significantly relieve pain and improve knee function, and the medial compartment cartilage repair and regeneration phenomenon will occur after the correction of the weight-bearing line.

Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging

SIGNIFICANCE

Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space.

AIM

WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics.

APPROACH

High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection.

RESULTS

Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results.

CONCLUSIONS

The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.

AI musculoskeletal clinical applications: how can AI increase my day-to-day efficiency?

Artificial intelligence (AI) is expected to bring greater efficiency in radiology by performing tasks that would otherwise require human intelligence, also at a much faster rate than human performance. In recent years, milestone deep learning models with unprecedented low error rates and high computational efficiency have shown remarkable performance for lesion detection, classification, and segmentation tasks. However, the growing field of AI has significant implications for radiology that are not limited to visual tasks. These are essential applications for optimizing imaging workflows and improving noninterpretive tasks. This article offers an overview of the recent literature on AI, focusing on the musculoskeletal imaging chain, including initial patient scheduling, optimized protocoling, magnetic resonance imaging reconstruction, image enhancement, medical image-to-image translation, and AI-aided image interpretation. The substantial developments of advanced algorithms, the emergence of massive quantities of medical data, and the interest of researchers and clinicians reveal the potential for the growing applications of AI to augment the day-to-day efficiency of musculoskeletal radiologists.

Deep Learning Reconstruction of Diffusion-weighted MRI Improves Image Quality for Prostatic Imaging

Background Deep learning reconstruction (DLR) may improve image quality. However, its impact on diffusion-weighted imaging (DWI) of the prostate has yet to be assessed. Purpose To determine whether DLR can improve image quality of diffusion-weighted MRI at b values ranging from 1000 sec/mm² to 5000 sec/mm² in patients with prostate cancer. Materials and Methods In this retrospective study, images of the prostate obtained at DWI with a b value of 0 sec/mm², DWI with a b value of 1000 sec/mm² (DWI₁₀₀₀), DWI with a b value of 3000 sec/mm² (DWI₃₀₀₀), and DWI with a b value of 5000 sec/mm² (DWI₅₀₀₀) from consecutive patients with biopsy-proven cancer from January to June 2020 were reconstructed with and without DLR. Image quality was assessed using signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) from region-of-interest analysis and qualitatively assessed using a five-point visual scoring system (1 [very poor] to 5 [excellent]) for each high-b-value DWI sequence with and without DLR. The SNR, CNR, and visual score for DWI with and without DLR were compared with the paired t test and the Wilcoxon signed rank test with Bonferroni correction, respectively. Apparent diffusion coefficients (ADCs) from DWI with and without DLR were also compared with the paired t test with Bonferroni correction. Results A total of 60 patients (mean age, 67 years; age range, 49-79 years) were analyzed. DWI with DLR showed significantly higher SNRs and CNRs than DWI without DLR (P < .001); for example, with DWI₁₀₀₀ the mean SNR was 38.7 ± 0.6 versus 17.8 ± 0.6, respectively (P < .001), and the mean CNR was 18.4 ± 5.6 versus 7.4 ± 5.6, respectively (P < .001). DWI with DLR also demonstrated higher qualitative image quality than DWI without DLR (mean score: 4.8 ± 0.4 vs 4.0 ± 0.7, respectively, with DWI₁₀₀₀ [P = .001], 3.8 ± 0.7 vs 3.0 ± 0.8 with DWI₃₀₀₀ [P = .002], and 3.1 ± 0.8 vs 2.0 ± 0.9 with DWI₅₀₀₀ [P < .001]). ADCs derived with and without DLR did not differ substantially (P > .99). Conclusion Deep learning reconstruction improves the image quality of diffusion-weighted MRI scans of prostate cancer with no impact on apparent diffusion coefficient quantitation with a 3.0-T MRI system. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Turkbey in this issue.

Deep Learning-Based Image Feature with Arthroscopy-Aided Early Diagnosis and Treatment of Meniscus Injury of Knee Joint

The aim of this study is to explore the clinical effect of deep learning-based MRI-assisted arthroscopy in the early treatment of knee meniscus sports injury. Based on convolutional neural network algorithm, Adam algorithm was introduced to optimize it, and the magnetic resonance imaging (MRI) image super-resolution reconstruction model (SRCNN) was established. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were compared between SRCNN and other algorithms. Sixty patients with meniscus injury of knee joint were studied. Arthroscopic surgery was performed according to the patients' actual type of injury, and knee scores were evaluated for all patients. Then, postoperative scores and MRI results were analyzed. The results showed that the PSNR and SSIM values of the SRCNN algorithm were (42.19 ± 4.37) dB and 0.9951, respectively, which were significantly higher than those of other algorithms (P < 0.05). Among patients with meniscus injury, 17 cases (28.33%) were treated with meniscus suture, 39 cases (65.00%) underwent secondary resection, 3 cases (5.00%) underwent partial resection, and 1 case (1.67%) underwent full resection. After meniscus suture, secondary resection, partial resection, and total resection, the knee function scores of patients after treatment were (83.17 ± 8.63), (80.06 ± 7.96), (84.34 ± 7.74), and (85.52 ± 5.97), respectively. There was no great difference in knee function scores after different methods of treatment (P > 0.05), and there were considerable differences compared with those before treatment (P < 0.01). Compared with the results of arthroscopy, there was no significant difference in the grading of meniscus injury by MRI (P > 0.05). To sum up, the SRCNN algorithm based on the deep convolutional network algorithm improved the MRI image quality and the diagnosis of knee meniscus injuries. Arthroscopic knee surgery had good results and had great clinical application and promotion value.

Residual dense network for medical magnetic resonance images super-resolution

BACKGROUND AND OBJECTIVE

High-resolution magnetic resonance images (MRI) help experts to localize lesions and diagnose diseases, but it is difficult to obtain high-resolution MRI. Furthermore, image super-resolution technology based on deep learning can effectively improve image resolution.

METHODS

In this work, we propose a medical magnetic resonance (MR) image super-resolution reconstruction method based on residual dense network (MRDN). Firstly, we input the convolutional features of the shallow layer into the residual dense block to obtain global and local features. Secondly, each layer in the residual dense block is directly connected to the previous layer to achieve reuse of features. Finally, we use sub-pixel convolution layer for upsampling and super-resolution reconstruction to get a clear high-resolution image.

RESULTS

For the 2 ×, 3 ×, and 4 × enlargement, we propose the MRDN method shows the superiority over the state-of-the-art methods on the Set5, Set14, and Urban100 benchmark datasets, extensive benchmark experiment and analysis show that the superiority of our MRDN algorithm in terms of the peak signal-to-noise ratio (PSNR) and structural similarity index indicators (SSIM).

CONCLUSION

Quantitative experiments are conducted on three public datasets: Set5, Set14 and Urban10, evaluate with commonly used evaluation metrics, and the experimental results show that the method in this paper is more effective. In addition, we reconstruct the public MR datasets, and the reconstructed high-resolution MR image has a clear structure and rich texture details.

Automatic segmentation of cardiac magnetic resonance images based on multi-input fusion network

PURPOSE

Using computer-assisted means to process a large amount of heart image data in order to speed up the diagnosis efficiency and accuracy of medical doctors has become a research worthy of investigation.

METHOD

Based on the U-Net model, this paper proposes a multi-input fusion network (MIFNet) model based on multi-scale input and feature fusion, which automatically extracts and fuses features of different input scales to realize the detection of Cardiac Magnetic Resonance Images (CMRI). The MIFNet model is trained and verified on the public data set, and then compared with the segmentation models, namely the Fully Convolutional Network (FCN) and DeepLab v1.

RESULTS

MIFNet model segmentation of CMRI significantly improved the segmentation accuracy, and the Dice value reached 97.238%. Compared with FCN and DeepLab v1, the average Hausdorff distance (HD) was reduced by 16.425%. The capacity parameter of FCN is 124.86% of MIFNet, DeepLab v1 is 103.22% of MIFNet.

CONCLUSION

Our proposed MIFNet model reduces the amount of parameters and improves the training speed while ensuring the simultaneous segmentation of overlapping targets. It can help clinicians to more quickly check the patient's CMRI focus area, and thereby improving the efficiency of diagnosis.

Progressive back-projection network for COVID-CT super-resolution

BACKGROUND AND OBJECTIVE

Recently, the COVID-19 epidemic has become more and more serious around the world, how to improve the image resolution of COVID-CT is a very important task. The network based on progressive upsampling for COVID-CT super-resolution increases the reconstruction error. This paper proposes a progressive back-projection network (PBPN) for COVID-CT super-resolution to solve this problem.

METHODS

In this paper, we propose a progressive back-projection network (PBPN) for COVID-CT super-resolution. PBPN is divided into two stages, and each stage consists of back-projection, deep feature extraction and upscaling. We design an up-projection and down-projection residual module to minimize the reconstruction error and construct a residual attention module to extract deep features. In each stage, firstly, PBPN performs back-projection to extract shallow features by two up-projection and down-projection residual modules; then, PBPN extracts deep features from the shallow features by two residual attention modules; finally, PBPN upsamples the deep features through sub-pixel convolution.

RESULTS

The proposed method achieves the improvements of about 0.14~0.47 dB/0.0012~0.0060 for × 2 scale factor, 0.02~0.08 dB/0.0024~0.0059 for × 3 scale factor, and 0.08~0.41 dB/ 0.0040~0.0147 for × 4 scale factor than state-of-the-art methods (Bicubic, SRCNN, FSRCNN, VDSR, LapSRN, DRCN and DSRN) in terms of PSNR/SSIM on benchmark datasets.

CONCLUSIONS

The proposed mehtod obtains better performance for COVID-CT super-resolution and reconstructs high-quality high-resolution COVID-CT images that contain more details and edges.

Cardiac Magnetic Resonance Images Superresolution via Multichannel Residual Attention Networks

The deep neural network has achieved good results in medical image superresolution. However, due to the medical equipment limitations and the complexity of the human body structure, it is difficult to reconstruct clear cardiac magnetic resonance (CMR) superresolution images. To reconstruct clearer CMR images, we propose a CMR image superresolution (SR) algorithm based on multichannel residual attention networks (MCRN), which uses the idea of residual learning to alleviate the difficulty of training and fully explore the feature information of the image and uses the back-projection learning mechanism to learn the interdependence between high-resolution images and low-resolution images. Furthermore, the MCRN model introduces an attention mechanism to dynamically allocate each feature map with different attention resources to discover more high-frequency information and learn the dependency between each channel of the feature map. Extensive benchmark evaluation shows that compared with state-of-the-art image SR methods, our MCRN algorithm not only improves the objective index significantly but also provides richer texture information for the reconstructed CMR images, and our MCRN algorithm is better than the Bicubic algorithm in evaluating the information entropy and average gradient of the reconstructed image quality.

Towards lower-dose PET using physics-based uncertainty-aware multimodal learning with robustness to out-of-distribution data

Radiation exposure in positron emission tomography (PET) imaging limits its usage in the studies of radiation-sensitive populations, e.g., pregnant women, children, and adults that require longitudinal imaging. Reducing the PET radiotracer dose or acquisition time reduces photon counts, which can deteriorate image quality. Recent deep-neural-network (DNN) based methods for image-to-image translation enable the mapping of low-quality PET images (acquired using substantially reduced dose), coupled with the associated magnetic resonance imaging (MRI) images, to high-quality PET images. However, such DNN methods focus on applications involving test data that match the statistical characteristics of the training data very closely and give little attention to evaluating the performance of these DNNs on new out-of-distribution (OOD) acquisitions. We propose a novel DNN formulation that models the (i) underlying sinogram-based physics of the PET imaging system and (ii) the uncertainty in the DNN output through the per-voxel heteroscedasticity of the residuals between the predicted and the high-quality reference images. Our sinogram-based uncertainty-aware DNN framework, namely, suDNN, estimates a standard-dose PET image using multimodal input in the form of (i) a low-dose/low-count PET image and (ii) the corresponding multi-contrast MRI images, leading to improved robustness of suDNN to OOD acquisitions. Results on in vivo simultaneous PET-MRI, and various forms of OOD data in PET-MRI, show the benefits of suDNN over the current state of the art, quantitatively and qualitatively.

Autoencoder-Inspired Convolutional Network-Based Super-Resolution Method in MRI

OBJECTIVE

To introduce an MRI in-plane resolution enhancement method that estimates High-Resolution (HR) MRIs from Low-Resolution (LR) MRIs.

METHOD & MATERIALS

Previous CNN-based MRI super-resolution methods cause loss of input image information due to the pooling layer. An Autoencoder-inspired Convolutional Network-based Super-resolution (ACNS) method was developed with the deconvolution layer that extrapolates the missing spatial information by the convolutional neural network-based nonlinear mapping between LR and HR features of MRI. Simulation experiments were conducted with virtual phantom images and thoracic MRIs from four volunteers. The Peak Signal-to-Noise Ratio (PSNR), Structure SIMilarity index (SSIM), Information Fidelity Criterion (IFC), and computational time were compared among: ACNS; Super-Resolution Convolutional Neural Network (SRCNN); Fast Super-Resolution Convolutional Neural Network (FSRCNN); Deeply-Recursive Convolutional Network (DRCN).

RESULTS

ACNS achieved comparable PSNR, SSIM, and IFC results to SRCNN, FSRCNN, and DRCN. However, the average computation speed of ACNS was 6, 4, and 35 times faster than SRCNN, FSRCNN, and DRCN, respectively under the computer setup used with the actual average computation time of 0.15 s per [Formula: see text].

Do Radiographic Assessments of Periodontal Bone Loss Improve with Deep Learning Methods for Enhanced Image Resolution?

Resolution plays an essential role in oral imaging for periodontal disease assessment. Nevertheless, due to limitations in acquisition tools, a considerable number of oral examinations have low resolution, making the evaluation of this kind of lesion difficult. Recently, the use of deep-learning methods for image resolution improvement has seen an increase in the literature. In this work, we performed two studies to evaluate the effects of using different resolution improvement methods (nearest, bilinear, bicubic, Lanczos, SRCNN, and SRGAN). In the first one, specialized dentists visually analyzed the quality of images treated with these techniques. In the second study, we used those methods as different pre-processing steps for inputs of convolutional neural network (CNN) classifiers (Inception and ResNet) and evaluated whether this process leads to better results. The deep-learning methods lead to a substantial improvement in the visual quality of images but do not necessarily promote better classifier performance.

Using super-resolution generative adversarial network models and transfer learning to obtain high resolution digital periapical radiographs

Periapical Radiographs are commonly used to detect several anomalies, like caries, periodontal, and periapical diseases. Even considering that digital imaging systems used nowadays tend to provide high-quality images, external factors, or even system limitations can result in a vast amount of radiographic images with low quality and resolution. Commercial solutions offer tools based on interpolation methods to increase image resolution. However, previous literature shows that these methods may create undesirable effects in the images affecting the diagnosis accuracy. One alternative is using deep learning-based super-resolution methods to achieve better high-resolution images. Nevertheless, the amount of data for training such models is limited, demanding transfer learning approaches. In this work, we propose the use of super-resolution generative adversarial network (SRGAN) models and transfer learning to achieve periapical images with higher quality and resolution. Moreover, we evaluate the influence of using the transfer learning approach and the datasets selected for it in the final generated images. For that, we performed an experiment comparing the performance of the SRGAN models (with and without transfer learning) with other super-resolution methods. Considering Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Mean Opinion Score (MOS), the results of SRGAN models using transfer learning were better on average. This superiority was also verified statistically using the Wilcoxon paired test. In the visual analysis, the high quality achieved by the SRGAN models, in general, is visible, resulting in more defined edges details and fewer blur effects.

Compressed sensing and deep learning reconstruction for women's pelvic MRI denoising: Utility for improving image quality and examination time in routine clinical practice

PURPOSE

To demonstrate the utility of compressed sensing with parallel imaging (Compressed SPEEDER) and AiCE compared with that of conventional parallel imaging (SPEEDER) for shortening examination time and improving image quality of women's pelvic MRI.

METHOD

Thirty consecutive patients with women's pelvic diseases (mean age 50 years) underwent T2-weighted imaging using Compressed SPEEDER as well as conventional SPEEDER reconstructed with and without AiCE. The examination times were recorded, and signal-to-noise ratio (SNR) was calculated for every patient. Moreover, overall image quality was assessed using a 5-point scoring system, and final scores for all patients were determined by consensus of two readers. Mean examination time, SNR and overall image quality were compared among the four data sets by Wilcoxon signed-rank test.

RESULTS

Examination times for Compressed SPEEDER with and without AiCE were significantly shorter than those for conventional SPEEDER with and without AiCE (with AiCE: p < 0.0001, without AiCE: p < 0.0001). SNR of Compressed SPEEDER and of SPEEDER with AiCE was significantly superior to that of Compressed SPEEDER without AiCE (vs. Compressed SPEEDER, p = 0.01; vs. SPEEDER, p = 0.009). Overall image quality of Compressed SPEEDER with AiCE and of SPEEDER with and without AiCE was significantly higher than that of Compressed SPEEDER without AiCE (vs. Compressed SPEEDER with AiCE, p < 0.0001; vs. SPEEDER with AiCE, p < 0.0001; SPEEDER without AiCE, p = 0.0003).

CONCLUSION

Image quality and shorten examination time for T2-weighted imaging in women's pelvic MRI can be significantly improved by using Compressed SPEEDER with AiCE in comparison with conventional SPEEDER, although other sequences were not tested.

Rapid whole-heart CMR with single volume super-resolution

BACKGROUND

Three-dimensional, whole heart, balanced steady state free precession (WH-bSSFP) sequences provide delineation of intra-cardiac and vascular anatomy. However, they have long acquisition times. Here, we propose significant speed-ups using a deep-learning single volume super-resolution reconstruction, to recover high-resolution features from rapidly acquired low-resolution WH-bSSFP images.

METHODS

A 3D residual U-Net was trained using synthetic data, created from a library of 500 high-resolution WH-bSSFP images by simulating 50% slice resolution and 50% phase resolution. The trained network was validated with 25 synthetic test data sets. Additionally, prospective low-resolution data and high-resolution data were acquired in 40 patients. In the prospective data, vessel diameters, quantitative and qualitative image quality, and diagnostic scoring was compared between the low-resolution, super-resolution and reference high-resolution WH-bSSFP data.

RESULTS

The synthetic test data showed a significant increase in image quality of the low-resolution images after super-resolution reconstruction. Prospectively acquired low-resolution data was acquired ~× 3 faster than the prospective high-resolution data (173 s vs 488 s). Super-resolution reconstruction of the low-resolution data took < 1 s per volume. Qualitative image scores showed super-resolved images had better edge sharpness, fewer residual artefacts and less image distortion than low-resolution images, with similar scores to high-resolution data. Quantitative image scores showed super-resolved images had significantly better edge sharpness than low-resolution or high-resolution images, with significantly better signal-to-noise ratio than high-resolution data. Vessel diameters measurements showed over-estimation in the low-resolution measurements, compared to the high-resolution data. No significant differences and no bias was found in the super-resolution measurements in any of the great vessels. However, a small but significant for the underestimation was found in the proximal left coronary artery diameter measurement from super-resolution data. Diagnostic scoring showed that although super-resolution did not improve accuracy of diagnosis, it did improve diagnostic confidence compared to low-resolution imaging.

CONCLUSION

This paper demonstrates the potential of using a residual U-Net for super-resolution reconstruction of rapidly acquired low-resolution whole heart bSSFP data within a clinical setting. We were able to train the network using synthetic training data from retrospective high-resolution whole heart data. The resulting network can be applied very quickly, making these techniques particularly appealing within busy clinical workflow. Thus, we believe that this technique may help speed up whole heart CMR in clinical practice.