1
|
Wang L, Zhang W, Chen W, He Z, Jia Y, Du J. Cross-Modality Reference and Feature Mutual-Projection for 3D Brain MRI Image Super-Resolution. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01139-1. [PMID: 38829472 DOI: 10.1007/s10278-024-01139-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 03/30/2024] [Accepted: 04/21/2024] [Indexed: 06/05/2024]
Abstract
High-resolution (HR) magnetic resonance imaging (MRI) can reveal rich anatomical structures for clinical diagnoses. However, due to hardware and signal-to-noise ratio limitations, MRI images are often collected with low resolution (LR) which is not conducive to diagnosing and analyzing clinical diseases. Recently, deep learning super-resolution (SR) methods have demonstrated great potential in enhancing the resolution of MRI images; however, most of them did not take the cross-modality and internal priors of MR seriously, which hinders the SR performance. In this paper, we propose a cross-modality reference and feature mutual-projection (CRFM) method to enhance the spatial resolution of brain MRI images. Specifically, we feed the gradients of HR MRI images from referenced imaging modality into the SR network to transform true clear textures to LR feature maps. Meanwhile, we design a plug-in feature mutual-projection (FMP) method to capture the cross-scale dependency and cross-modality similarity details of MRI images. Finally, we fuse all feature maps with parallel attentions to produce and refine the HR features adaptively. Extensive experiments on MRI images in the image domain and k-space show that our CRFM method outperforms existing state-of-the-art MRI SR methods.
Collapse
Affiliation(s)
- Lulu Wang
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology and Yunnan Key Laboratory of Computer Technologies Application, Kunming, 650500, China.
| | - Wanqi Zhang
- College of Computer Science, Chongqing University, Chongqing, 400044, China
| | - Wei Chen
- College of Computer Science, Chongqing University, Chongqing, 400044, China
| | - Zhongshi He
- College of Computer Science, Chongqing University, Chongqing, 400044, China
| | - Yuanyuan Jia
- Medical Data Science Academy and College of Medical Informatics, Chongqing Medical University, Chongqing, 400016, China
| | - Jinglong Du
- Medical Data Science Academy and College of Medical Informatics, Chongqing Medical University, Chongqing, 400016, China
| |
Collapse
|
2
|
Sedighin F. Tensor Ring Based Image Enhancement. JOURNAL OF MEDICAL SIGNALS & SENSORS 2024; 14:1. [PMID: 38510671 PMCID: PMC10950313 DOI: 10.4103/jmss.jmss_32_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 08/28/2023] [Accepted: 10/11/2023] [Indexed: 03/22/2024]
Abstract
Background Image enhancement, including image de-noising, super-resolution, registration, reconstruction, in-painting, and so on, is an important issue in different research areas. Different methods which have been exploited for image analysis were mostly based on matrix or low order analysis. However, recent researches show the superior power of tensor-based methods for image enhancement. Method In this article, a new method for image super-resolution using Tensor Ring decomposition has been proposed. The proposed image super-resolution technique has been derived for the super-resolution of low resolution and noisy images. The new approach is based on a modification and extension of previous tensor-based approaches used for super-resolution of datasets. In this method, a weighted combination of the original and the resulting image of the previous stage has been computed and used to provide a new input to the algorithm. Result This enables the method to do the super-resolution and de-noising simultaneously. Conclusion Simulation results show the effectiveness of the proposed approach, especially in highly noisy situations.
Collapse
Affiliation(s)
- Farnaz Sedighin
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
| |
Collapse
|
3
|
Grigas O, Maskeliūnas R, Damaševičius R. Improving Structural MRI Preprocessing with Hybrid Transformer GANs. Life (Basel) 2023; 13:1893. [PMID: 37763297 PMCID: PMC10532639 DOI: 10.3390/life13091893] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Revised: 09/01/2023] [Accepted: 09/06/2023] [Indexed: 09/29/2023] Open
Abstract
Magnetic resonance imaging (MRI) is a technique that is widely used in practice to evaluate any pathologies in the human body. One of the areas of interest is the human brain. Naturally, MR images are low-resolution and contain noise due to signal interference, the patient's body's radio-frequency emissions and smaller Tesla coil counts in the machinery. There is a need to solve this problem, as MR tomographs that have the capability of capturing high-resolution images are extremely expensive and the length of the procedure to capture such images increases by the order of magnitude. Vision transformers have lately shown state-of-the-art results in super-resolution tasks; therefore, we decided to evaluate whether we can employ them for structural MRI super-resolution tasks. A literature review showed that similar methods do not focus on perceptual image quality because upscaled images are often blurry and are subjectively of poor quality. Knowing this, we propose a methodology called HR-MRI-GAN, which is a hybrid transformer generative adversarial network capable of increasing resolution and removing noise from 2D T1w MRI slice images. Experiments show that our method quantitatively outperforms other SOTA methods in terms of perceptual image quality and is capable of subjectively generalizing to unseen data. During the experiments, we additionally identified that the visual saliency-induced index metric is not applicable to MRI perceptual quality assessment and that general-purpose denoising networks are effective when removing noise from MR images.
Collapse
Affiliation(s)
- Ovidijus Grigas
- Faculty of Informatics, Kaunas University of Technology, 50254 Kaunas, Lithuania
| | - Rytis Maskeliūnas
- Faculty of Informatics, Kaunas University of Technology, 50254 Kaunas, Lithuania
| | - Robertas Damaševičius
- Faculty of Informatics, Kaunas University of Technology, 50254 Kaunas, Lithuania
- Department of Applied Informatics, Vytautas Magnus University, 44248 Kaunas, Lithuania
| |
Collapse
|
4
|
Zhou L, Zhu M, Xiong D, Ouyang L, Ouyang Y, Chen Z, Zhang X. RNLFNet: Residual non-local Fourier network for undersampled MRI reconstruction. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
|
5
|
A novel medical text classification model with Kalman filter for clinical decision making. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
|
6
|
Ali AM, Benjdira B, Koubaa A, El-Shafai W, Khan Z, Boulila W. Vision Transformers in Image Restoration: A Survey. SENSORS (BASEL, SWITZERLAND) 2023; 23:2385. [PMID: 36904589 PMCID: PMC10006889 DOI: 10.3390/s23052385] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 02/14/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
The Vision Transformer (ViT) architecture has been remarkably successful in image restoration. For a while, Convolutional Neural Networks (CNN) predominated in most computer vision tasks. Now, both CNN and ViT are efficient approaches that demonstrate powerful capabilities to restore a better version of an image given in a low-quality format. In this study, the efficiency of ViT in image restoration is studied extensively. The ViT architectures are classified for every task of image restoration. Seven image restoration tasks are considered: Image Super-Resolution, Image Denoising, General Image Enhancement, JPEG Compression Artifact Reduction, Image Deblurring, Removing Adverse Weather Conditions, and Image Dehazing. The outcomes, the advantages, the limitations, and the possible areas for future research are detailed. Overall, it is noted that incorporating ViT in the new architectures for image restoration is becoming a rule. This is due to some advantages compared to CNN, such as better efficiency, especially when more data are fed to the network, robustness in feature extraction, and a better feature learning approach that sees better the variances and characteristics of the input. Nevertheless, some drawbacks exist, such as the need for more data to show the benefits of ViT over CNN, the increased computational cost due to the complexity of the self-attention block, a more challenging training process, and the lack of interpretability. These drawbacks represent the future research direction that should be targeted to increase the efficiency of ViT in the image restoration domain.
Collapse
Affiliation(s)
- Anas M. Ali
- Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 12435, Saudi Arabia
- Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf 32952, Egypt
| | - Bilel Benjdira
- Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 12435, Saudi Arabia
- SE & ICT Laboratory, LR18ES44, ENICarthage, University of Carthage, Tunis 1054, Tunisia
| | - Anis Koubaa
- Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 12435, Saudi Arabia
| | - Walid El-Shafai
- Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf 32952, Egypt
- Security Engineering Laboratory, Computer Science Department, Prince Sultan University, Riyadh 11586, Saudi Arabia
| | - Zahid Khan
- Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 12435, Saudi Arabia
| | - Wadii Boulila
- Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 12435, Saudi Arabia
- RIADI Laboratory, University of Manouba, Manouba 2010, Tunisia
| |
Collapse
|
7
|
Li W, Liu X, An K, Qin C, Cheng Y. Table Tennis Track Detection Based on Temporal Feature Multiplexing Network. SENSORS (BASEL, SWITZERLAND) 2023; 23:1726. [PMID: 36772762 PMCID: PMC9921165 DOI: 10.3390/s23031726] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 01/30/2023] [Accepted: 02/01/2023] [Indexed: 06/18/2023]
Abstract
Recording the trajectory of table tennis balls in real-time enables the analysis of the opponent's attacking characteristics and weaknesses. The current analysis of the ball paths mainly relied on human viewing, which lacked certain theoretical data support. In order to solve the problem of the lack of objective data analysis in the research of table tennis competition, a target detection algorithm-based table tennis trajectory extraction network was proposed to record the trajectory of the table tennis movement in video. The network improved the feature reuse rate in order to achieve a lightweight network and enhance the detection accuracy. The core of the network was the "feature store & return" module, which could store the output of the current network layer and pass the features to the input of the network layer at the next moment to achieve efficient reuse of the features. In this module, the Transformer model was used to secondarily process the features, build the global association information, and enhance the feature richness of the feature map. According to the designed experiments, the detection accuracy of the network was 96.8% for table tennis and 89.1% for target localization. Moreover, the parameter size of the model was only 7.68 MB, and the detection frame rate could reach 634.19 FPS using the hardware for the tests. In summary, the network designed in this paper has the characteristics of both lightweight and high precision in table tennis detection, and the performance of the proposed model significantly outperforms that of the existing models.
Collapse
Affiliation(s)
- Wenjie Li
- College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 201418, China
| | - Xiangpeng Liu
- College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 201418, China
| | - Kang An
- College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 201418, China
| | - Chengjin Qin
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Yuhua Cheng
- Shanghai Research Institute of Microelectronics, Peking University, Shanghai 201203, China
| |
Collapse
|
8
|
Xu Y, Dai S, Song H, Du L, Chen Y. Multi-modal brain MRI images enhancement based on framelet and local weights super-resolution. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:4258-4273. [PMID: 36899626 DOI: 10.3934/mbe.2023199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Magnetic resonance (MR) image enhancement technology can reconstruct high-resolution image from a low-resolution image, which is of great significance for clinical application and scientific research. T1 weighting and T2 weighting are the two common magnetic resonance imaging modes, each of which has its own advantages, but the imaging time of T2 is much longer than that of T1. Related studies have shown that they have very similar anatomical structures in brain images, which can be utilized to enhance the resolution of low-resolution T2 images by using the edge information of high-resolution T1 images that can be rapidly imaged, so as to shorten the imaging time needed for T2 images. In order to overcome the inflexibility of traditional methods using fixed weights for interpolation and the inaccuracy of using gradient threshold to determine edge regions, we propose a new model based on previous studies on multi-contrast MR image enhancement. Our model uses framelet decomposition to finely separate the edge structure of the T2 brain image, and uses the local regression weights calculated from T1 image to construct a global interpolation matrix, so that our model can not only guide the edge reconstruction more accurately where the weights are shared, but also carry out collaborative global optimization for the remaining pixels and their interpolated weights. Experimental results on a set of simulated MR data and two sets of real MR images show that the enhanced images obtained by the proposed method are superior to the compared methods in terms of visual sharpness or qualitative indicators.
Collapse
Affiliation(s)
- Yingying Xu
- School of Electronics and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Songsong Dai
- School of Electronics and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Haifeng Song
- School of Electronics and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Lei Du
- School of Electronics and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Ying Chen
- School of Electronics and Information Engineering, Taizhou University, Taizhou 318000, China
| |
Collapse
|