1
|
Jiang N, Zhang Y, Li Q, Fu X, Fang D. A cardiac MRI motion artifact reduction method based on edge enhancement network. Phys Med Biol 2024; 69:095004. [PMID: 38537303 DOI: 10.1088/1361-6560/ad3884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 03/26/2024] [Indexed: 04/16/2024]
Abstract
Cardiac magnetic resonance imaging (MRI) usually requires a long acquisition time. The movement of the patients during MRI acquisition will produce image artifacts. Previous studies have shown that clear MR image texture edges are of great significance for pathological diagnosis. In this paper, a motion artifact reduction method for cardiac MRI based on edge enhancement network is proposed. Firstly, the four-plane normal vector adaptive fractional differential mask is applied to extract the edge features of blurred images. The four-plane normal vector method can reduce the noise information in the edge feature maps. The adaptive fractional order is selected according to the normal mean gradient and the local Gaussian curvature entropy of the images. Secondly, the extracted edge feature maps and blurred images are input into the de-artifact network. In this network, the edge fusion feature extraction network and the edge fusion transformer network are specially designed. The former combines the edge feature maps with the fuzzy feature maps to extract the edge feature information. The latter combines the edge attention network and the fuzzy attention network, which can focus on the blurred image edges. Finally, extensive experiments show that the proposed method can obtain higher peak signal-to-noise ratio and structural similarity index measure compared to state-of-art methods. The de-artifact images have clear texture edges.
Collapse
Affiliation(s)
- Nanhe Jiang
- School of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, People's Republic of China
| | - Yucun Zhang
- School of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, People's Republic of China
| | - Qun Li
- School of Mechanical Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, People's Republic of China
| | - Xianbin Fu
- Hebei University of Environmental Engineering, Qinhuangdao, 066102, Hebei, People's Republic of China
| | - Dongqing Fang
- Capital Aerospace Machinery Co, Ltd, Fengtai, 100076, Beijing, People's Republic of China
| |
Collapse
|
2
|
Li X, Hu Y. Cooperative-Net: An end-to-end multi-task interaction network for unified reconstruction and segmentation of MR image. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 245:108045. [PMID: 38290292 DOI: 10.1016/j.cmpb.2024.108045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 01/12/2024] [Accepted: 01/21/2024] [Indexed: 02/01/2024]
Abstract
BACKGROUND AND OBJECTIVE In clinical applications, there is an increasing demand for rapid acquisition and automated analysis of magnetic resonance imaging (MRI) data. However, most existing methods focus on either MR image reconstruction from undersampled data or segmentation using fully sampled data, hardly considering MR image segmentation in fast imaging scenarios. Consequently, it is imperative to investigate a multi-task approach that can simultaneously achieve high scanning acceleration and accurate segmentation results. METHODS In this paper, we propose a novel end-to-end multi-task interaction network, termed as the Cooperative-Net, which integrates accelerated MR imaging and multi-class tissue segmentation into a unified framework. The Cooperative-Net consists of alternating reconstruction modules and segmentation modules. To facilitate effective interaction between the two tasks, we introduce the spatial-adaptive semantic guidance module, which leverages the semantic map as a structural prior to guide MR image reconstruction. Furthermore, we propose a novel unrolling network with a multi-path shrinkage structure for MR image reconstruction. This network consists of parallel learnable shrinkage paths to handle varying degrees of degradation across different frequency components in the undersampled MR image, effectively improving the quality of the recovered image. RESULTS We use two publicly available datasets, including the cardiac and knee MR datasets, to validate the efficacy of our proposed Cooperative-Net. Through qualitative and quantitative analysis, we demonstrate that our method outperforms existing state-of-the-art multi-task approaches for joint MR image reconstruction and segmentation. CONCLUSIONS The proposed Cooperative-Net is capable of achieving both high accelerated MR imaging and accurate multi-class tissue segmentation.
Collapse
Affiliation(s)
- Xiaodi Li
- School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin, 150001, China
| | - Yue Hu
- School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin, 150001, China.
| |
Collapse
|
3
|
Zhao Q, Jia Q, Chi T. U-Net deep learning model for endoscopic diagnosis of chronic atrophic gastritis and operative link for gastritis assessment staging: a prospective nested case-control study. Therap Adv Gastroenterol 2023; 16:17562848231208669. [PMID: 37928896 PMCID: PMC10624012 DOI: 10.1177/17562848231208669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2023] [Accepted: 10/02/2023] [Indexed: 11/07/2023] Open
Abstract
Background The operative link for the gastritis assessment (OLGA) system can objectively reflect the stratification of gastric cancer risk in patients with chronic atrophic gastritis (CAG). Objectives We developed a real-time video monitoring model for the endoscopic diagnosis of CAG and OLGA staging based on U-Net deep learning (DL). To further validate and improve its performance, we designed a study to evaluate the diagnostic evaluation indices. Design A prospective nested case-control study. Methods Our cohort consisted of 1306 patients from 31 July 2021 to 31 January 2022. According to the pathological results, patients in the cohort were divided into the CAG group and the chronic non-atrophic gastritis group to evaluate the diagnostic evaluation indices. Each atrophy lesion was automatically labeled and the atrophy severity was assessed by the model. Propensity score matching was used to minimize selection bias. Results The diagnostic evaluation indices and the consistency between OLGA staging and pathological diagnosis of the model were superior to those of endoscopists [sensitivity (89.31% versus 67.56%), specificity (90.46% versus 70.23%), positive predictive value (90.35% versus 69.41%), negative predictive value (89.43% versus 68.40%), accuracy rate (89.89% versus 68.89%), Youden index (79.77% versus 37.79%), odd product (79.23 versus 4.91), positive likelihood ratio (9.36 versus 2.27), negative likelihood ratio (0.12 versus 0.46)], areas under the curves (AUC) (95% CI) (0.919 (0.893-0.945) versus 0.749 (0.707-0.792), p < 0.001) and kappa (0.816 versus 0.291)]. Conclusion Our study demonstrated that the DL model can assist endoscopists in real-time diagnosis of CAG during gastroscopy and synchronous identification of high-risk OLGA stage (OLGA stages III and IV) patients. Trial registration ChiCTR2100044458.
Collapse
Affiliation(s)
- Quchuan Zhao
- Department of Gastroenterology, Xuanwu Hospital of Capital Medical University, Beijing, China
| | - Qing Jia
- Department of Anesthesiology, Guang’anmen Hospital China Academy of Chinese Medical Sciences, 5 North Court Street, Beijing 100053, China
| | - Tianyu Chi
- Department of Gastroenterology, Xuanwu Hospital of Capital Medical University, 45 Chang-Chun Street, Beijing 100053, China
| |
Collapse
|
4
|
Li X, Zhang H, Yang H, Li TQ. CS-MRI Reconstruction Using an Improved GAN with Dilated Residual Networks and Channel Attention Mechanism. SENSORS (BASEL, SWITZERLAND) 2023; 23:7685. [PMID: 37765747 PMCID: PMC10537966 DOI: 10.3390/s23187685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 08/20/2023] [Accepted: 08/30/2023] [Indexed: 09/29/2023]
Abstract
Compressed sensing (CS) MRI has shown great potential in enhancing time efficiency. Deep learning techniques, specifically generative adversarial networks (GANs), have emerged as potent tools for speedy CS-MRI reconstruction. Yet, as the complexity of deep learning reconstruction models increases, this can lead to prolonged reconstruction time and challenges in achieving convergence. In this study, we present a novel GAN-based model that delivers superior performance without the model complexity escalating. Our generator module, built on the U-net architecture, incorporates dilated residual (DR) networks, thus expanding the network's receptive field without increasing parameters or computational load. At every step of the downsampling path, this revamped generator module includes a DR network, with the dilation rates adjusted according to the depth of the network layer. Moreover, we have introduced a channel attention mechanism (CAM) to distinguish between channels and reduce background noise, thereby focusing on key information. This mechanism adeptly combines global maximum and average pooling approaches to refine channel attention. We conducted comprehensive experiments with the designed model using public domain MRI datasets of the human brain. Ablation studies affirmed the efficacy of the modified modules within the network. Incorporating DR networks and CAM elevated the peak signal-to-noise ratios (PSNR) of the reconstructed images by about 1.2 and 0.8 dB, respectively, on average, even at 10× CS acceleration. Compared to other relevant models, our proposed model exhibits exceptional performance, achieving not only excellent stability but also outperforming most of the compared networks in terms of PSNR and SSIM. When compared with U-net, DR-CAM-GAN's average gains in SSIM and PSNR were 14% and 15%, respectively. Its MSE was reduced by a factor that ranged from two to seven. The model presents a promising pathway for enhancing the efficiency and quality of CS-MRI reconstruction.
Collapse
Affiliation(s)
- Xia Li
- College of Information Engineering, China Jiliang University, Hangzhou 310018, China
| | - Hui Zhang
- College of Information Engineering, China Jiliang University, Hangzhou 310018, China
| | - Hao Yang
- College of Information Engineering, China Jiliang University, Hangzhou 310018, China
| | - Tie-Qiang Li
- Department of Clinical Science, Intervention, and Technology, Karolinska Institute, 14186 Stockholm, Sweden
- Department of Medical Radiation and Nuclear Medicine, Karolinska University Hospital, 17176 Stockholm, Sweden
| |
Collapse
|
5
|
Wang F, Kim SH, Zhao Y, Raghuram A, Veeraraghavan A, Robinson J, Hielscher AH. High-Speed Time-Domain Diffuse Optical Tomography with a Sensitivity Equation-based Neural Network. IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 2023; 9:459-474. [PMID: 37456517 PMCID: PMC10348778 DOI: 10.1109/tci.2023.3273423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/18/2023]
Abstract
Steady progress in time-domain diffuse optical tomography (TD-DOT) technology is allowing for the first time the design of low-cost, compact, and high-performance systems, thus promising more widespread clinical TD-DOT use, such as for recording brain tissue hemodynamics. TD-DOT is known to provide more accurate values of optical properties and physiological parameters compared to its frequency-domain or steady-state counterparts. However, achieving high temporal resolution is still difficult, as solving the inverse problem is computationally demanding, leading to relatively long reconstruction times. The runtime is further compromised by processes that involve 'nontrivial' empirical tuning of reconstruction parameters, which increases complexity and inefficiency. To address these challenges, we present a new reconstruction algorithm that combines a deep-learning approach with our previously introduced sensitivity-equation-based, non-iterative sparse optical reconstruction (SENSOR) code. The new algorithm (called SENSOR-NET) unfolds the iterations of SENSOR into a deep neural network. In this way, we achieve high-resolution sparse reconstruction using only learned parameters, thus eliminating the need to tune parameters prior to reconstruction empirically. Furthermore, once trained, the reconstruction time is not dependent on the number of sources or wavelengths used. We validate our method with numerical and experimental data and show that accurate reconstructions with 1 mm spatial resolution can be obtained in under 20 milliseconds regardless of the number of sources used in the setup. This opens the door for real-time brain monitoring and other high-speed DOT applications.
Collapse
Affiliation(s)
- Fay Wang
- Department of Biomedical Engineering, Columbia University, New York, NY 10027
| | - Stephen H Kim
- Department of Biomedical Engineering, New York University - Tandon School of Engineering, New York, NY 10001
| | - Yongyi Zhao
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005
| | - Ankit Raghuram
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005
| | - Ashok Veeraraghavan
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005
| | - Jacob Robinson
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005
| | - Andreas H Hielscher
- Department of Biomedical Engineering, New York University - Tandon School of Engineering, New York, NY 10001
| |
Collapse
|
6
|
Hossain MB, Kwon KC, Shinde RK, Imtiaz SM, Kim N. A Hybrid Residual Attention Convolutional Neural Network for Compressed Sensing Magnetic Resonance Image Reconstruction. Diagnostics (Basel) 2023; 13:diagnostics13071306. [PMID: 37046524 PMCID: PMC10093476 DOI: 10.3390/diagnostics13071306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 03/20/2023] [Accepted: 03/29/2023] [Indexed: 04/03/2023] Open
Abstract
We propose a dual-domain deep learning technique for accelerating compressed sensing magnetic resonance image reconstruction. An advanced convolutional neural network with residual connectivity and an attention mechanism was developed for frequency and image domains. First, the sensor domain subnetwork estimates the unmeasured frequencies of k-space to reduce aliasing artifacts. Second, the image domain subnetwork performs a pixel-wise operation to remove blur and noisy artifacts. The skip connections efficiently concatenate the feature maps to alleviate the vanishing gradient problem. An attention gate in each decoder layer enhances network generalizability and speeds up image reconstruction by eliminating irrelevant activations. The proposed technique reconstructs real-valued clinical images from sparsely sampled k-spaces that are identical to the reference images. The performance of this novel approach was compared with state-of-the-art direct mapping, single-domain, and multi-domain methods. With acceleration factors (AFs) of 4 and 5, our method improved the mean peak signal-to-noise ratio (PSNR) to 8.67 and 9.23, respectively, compared with the single-domain Unet model; similarly, our approach increased the average PSNR to 3.72 and 4.61, respectively, compared with the multi-domain W-net. Remarkably, using an AF of 6, it enhanced the PSNR by 9.87 ± 1.55 and 6.60 ± 0.38 compared with Unet and W-net, respectively.
Collapse
|
7
|
Zhao X, Yang T, Li B, Zhang X. SwinGAN: A dual-domain Swin Transformer-based generative adversarial network for MRI reconstruction. Comput Biol Med 2023; 153:106513. [PMID: 36603439 DOI: 10.1016/j.compbiomed.2022.106513] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Revised: 12/09/2022] [Accepted: 12/31/2022] [Indexed: 01/02/2023]
Abstract
Magnetic resonance imaging (MRI) is one of the most important modalities for clinical diagnosis. However, the main disadvantages of MRI are the long scanning time and the moving artifact caused by patient movement during prolonged imaging. It can also lead to patient anxiety and discomfort, so accelerated imaging is indispensable for MRI. Convolutional neural network (CNN) based methods have become the fact standard for medical image reconstruction, and generative adversarial network (GAN) have also been widely used. Nevertheless, due to the limited ability of CNN to capture long-distance information, it may lead to defects in the structure of the reconstructed images such as blurry contour. In this paper, we propose a novel Swin Transformer-based dual-domain generative adversarial network (SwinGAN) for accelerated MRI reconstruction. The SwinGAN consists of two generators: a frequency-domain generator and an image-domain generator. Both the generators utilize Swin Transformer as backbone for effectively capturing the long-distance dependencies. A contextual image relative position encoder (ciRPE) is designed to enhance the ability to capture local information. We extensively evaluate the method on the IXI brain dataset, MICCAI 2013 dataset and MRNet knee dataset. Compared with KIGAN, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are improved by 6.1% and 1.49% to 37.64 dB and 0.98 on IXI dataset respectively, which demonstrates that our model can sufficiently utilize the local and global information of image. The model shows promising performance and robustness under different undersampling masks, different acceleration rates and different datasets. But it needs high hardware requirements with the increasing of the network parameters. The code is available at: https://github.com/learnerzx/SwinGAN.
Collapse
Affiliation(s)
- Xiang Zhao
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| | - Tiejun Yang
- School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou, 450001, China; Key Laboratory of Grain Information Processing and Control (HAUT), Ministry of Education, Zhengzhou, China; Henan Key Laboratory of Grain Photoelectric Detection and Control (HAUT), Zhengzhou, Henan, China.
| | - Bingjie Li
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| | - Xin Zhang
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China
| |
Collapse
|
8
|
Hossain MB, Kwon KC, Imtiaz SM, Nam OS, Jeon SH, Kim N. De-Aliasing and Accelerated Sparse Magnetic Resonance Image Reconstruction Using Fully Dense CNN with Attention Gates. BIOENGINEERING (BASEL, SWITZERLAND) 2022; 10:bioengineering10010022. [PMID: 36671594 PMCID: PMC9854709 DOI: 10.3390/bioengineering10010022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 12/19/2022] [Accepted: 12/19/2022] [Indexed: 12/24/2022]
Abstract
When sparsely sampled data are used to accelerate magnetic resonance imaging (MRI), conventional reconstruction approaches produce significant artifacts that obscure the content of the image. To remove aliasing artifacts, we propose an advanced convolutional neural network (CNN) called fully dense attention CNN (FDA-CNN). We updated the Unet model with the fully dense connectivity and attention mechanism for MRI reconstruction. The main benefit of FDA-CNN is that an attention gate in each decoder layer increases the learning process by focusing on the relevant image features and provides a better generalization of the network by reducing irrelevant activations. Moreover, densely interconnected convolutional layers reuse the feature maps and prevent the vanishing gradient problem. Additionally, we also implement a new, proficient under-sampling pattern in the phase direction that takes low and high frequencies from the k-space both randomly and non-randomly. The performance of FDA-CNN was evaluated quantitatively and qualitatively with three different sub-sampling masks and datasets. Compared with five current deep learning-based and two compressed sensing MRI reconstruction techniques, the proposed method performed better as it reconstructed smoother and brighter images. Furthermore, FDA-CNN improved the mean PSNR by 2 dB, SSIM by 0.35, and VIFP by 0.37 compared with Unet for the acceleration factor of 5.
Collapse
Affiliation(s)
- Md. Biddut Hossain
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea
| | - Ki-Chul Kwon
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea
| | - Shariar Md Imtiaz
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea
| | - Oh-Seung Nam
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea
| | - Seok-Hee Jeon
- Department of Electronics Engineering, Incheon National University, 119 Academy-ro, Yeonsu-gu, Incheon 22012, Gyeonggi-do, Republic of Korea
| | - Nam Kim
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea
- Correspondence: ; Tel.: +82-043-261-2482
| |
Collapse
|
9
|
Kim M, Chung W. A cascade of preconditioned conjugate gradient networks for accelerated magnetic resonance imaging. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 225:107090. [PMID: 36067702 DOI: 10.1016/j.cmpb.2022.107090] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 08/10/2022] [Accepted: 08/25/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Recent unfolding based compressed sensing magnetic resonance imaging (CS-MRI) methods only reinterpret conventional CS-MRI optimization algorithms and, consequently, inherit the weaknesses of the alternating optimization strategy. In order to avoid the structural complexity of the alternating optimization strategy and achieve better reconstruction performance, we propose to directly optimize the ℓ1 regularized convex optimization problem using a deep learning approach. METHOD In order to achieve direct optimization, a system of equations solving the ℓ1 regularized optimization problem is constructed from the optimality conditions of a novel primal-dual form proposed for the effective training of the sparsifying transform. The optimal solution is obtained by a cascade of unfolding networks of the preconditioned conjugate gradient (PCG) algorithm trained to minimize the mean element-wise absolute difference (ℓ1 loss) between the terminal output and ground truth image in an end-to-end manner. The performance of the proposed method was compared with that of U-Net, PD-Net, ISTA-Net+, and the recently proposed projection-based cascaded U-Net, using single-coil knee MR images of the fastMRI dataset. RESULTS In our experiment, the proposed network outperformed existing unfolding-based networks and the complex version of U-Net in several subsampling scenarios. In particular, when using the random Cartesian subsampling mask with 25 % sampling rate, the proposed model outperformed PD-Net by 0.76 dB, ISTA-Net+ by 0.43 dB, and U-Net by 1.21 dB on the positron density without suppression (PD) dataset in term of peak signal to noise ratio. In comparison with the projection-based cascade U-Net, the proposed algorithm achieved approximately the same performance when the sampling rate was 25% with only 1.62% number of network parameters at the cost of a longer reconstruction time (approximately twice). CONCLUSION A cascade of unfolding networks of the PCG algorithm was proposed to directly optimize the ℓ1 regularized CS-MRI optimization problem. The proposed network achieved improved reconstruction performance compared with U-Net, PD-Net, and ISTA-Net+, and achieved approximately the same performance as the projection-based cascaded U-Net while using significantly fewer network parameters.
Collapse
Affiliation(s)
- Moogyeong Kim
- Department of Artificial Intelligence, Korea University, Seoul 02841 South Korea
| | - Wonzoo Chung
- Department of Artificial Intelligence, Korea University, Seoul 02841 South Korea.
| |
Collapse
|
10
|
Aghabiglou A, Eksioglu EM. Deep unfolding architecture for MRI reconstruction enhanced by adaptive noise maps. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.104016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
11
|
Liu X, Pang Y, Jin R, Liu Y, Wang Z. Dual-Domain Reconstruction Network with V-Net and K-Net for Fast MRI. Magn Reson Med 2022; 88:2694-2708. [PMID: 35942977 DOI: 10.1002/mrm.29400] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 07/05/2022] [Accepted: 07/08/2022] [Indexed: 11/10/2022]
Abstract
PURPOSE To introduce a dual-domain reconstruction network with V-Net and K-Net for accurate MR image reconstruction from undersampled k-space data. METHODS Most state-of-the-art reconstruction methods apply U-Net or cascaded U-Nets in the image domain and/or k-space domain. Nevertheless, these methods have the following problems: (1) directly applying U-Net in the k-space domain is not optimal for extracting features; (2) classical image-domain-oriented U-Net is heavyweighted and hence inefficient when cascaded many times to yield good reconstruction accuracy; (3) classical image-domain-oriented U-Net does not make full use of information of the encoder network for extracting features in the decoder network; and (4) existing methods are ineffective in simultaneously extracting and fusing features in the image domain and its dual k-space domain. To tackle these problems, we present 3 different methods: (1) V-Net, an image-domain encoder-decoder subnetwork that is more lightweight for cascading and effective in fully utilizing features in the encoder for decoding; (2) K-Net, a k-space domain subnetwork that is more suitable for extracting hierarchical features in the k-space domain, and (3) KV-Net, a dual-domain reconstruction network in which V-Nets and K-Nets are effectively combined and cascaded. RESULTS Extensive experimental results on the fastMRI dataset demonstrate that the proposed KV-Net can reconstruct high-quality images and outperform state-of-the-art approaches with fewer parameters. CONCLUSIONS To reconstruct images effectively and efficiently from incomplete k-space data, we have presented a dual-domain KV-Net to combine K-Nets and V-Nets. The KV-Net achieves better results with 9% and 5% parameters than comparable methods (XPD-Net and i-RIM).
Collapse
Affiliation(s)
- Xiaohan Liu
- Tianjin Key Lab. of Brain Inspired Intelligence Technology, School of Electrical and Information Engineering, Tianjin University, Tianjin, People's Republic of China
| | - Yanwei Pang
- Tianjin Key Lab. of Brain Inspired Intelligence Technology, School of Electrical and Information Engineering, Tianjin University, Tianjin, People's Republic of China
| | - Ruiqi Jin
- Tianjin Key Lab. of Brain Inspired Intelligence Technology, School of Electrical and Information Engineering, Tianjin University, Tianjin, People's Republic of China
| | - Yu Liu
- Tianjin Key Lab. of Brain Inspired Intelligence Technology, School of Electrical and Information Engineering, Tianjin University, Tianjin, People's Republic of China
| | - Zhenchang Wang
- Beijing Friendship Hospital, Capital Medical University, Beijing, People's Republic of China
| |
Collapse
|
12
|
Wang S, Sun G, Zheng B, Du Y. A Crop Image Segmentation and Extraction Algorithm Based on Mask RCNN. ENTROPY 2021; 23:e23091160. [PMID: 34573785 PMCID: PMC8469590 DOI: 10.3390/e23091160] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Revised: 08/25/2021] [Accepted: 08/30/2021] [Indexed: 11/16/2022]
Abstract
The wide variety of crops in the image of agricultural products and the confusion with the surrounding environment information makes it difficult for traditional methods to extract crops accurately and efficiently. In this paper, an automatic extraction algorithm is proposed for crop images based on Mask RCNN. First, the Fruits 360 Dataset label is set with Labelme. Then, the Fruits 360 Dataset is preprocessed. Next, the data are divided into a training set and a test set. Additionally, an improved Mask RCNN network model structure is established using the PyTorch 1.8.1 deep learning framework, and path aggregation and features are added to the network design enhanced functions, optimized region extraction network, and feature pyramid network. The spatial information of the feature map is saved by the bilinear interpolation method in ROIAlign. Finally, the edge accuracy of the segmentation mask is further improved by adding a micro-fully connected layer to the mask branch of the ROI output, employing the Sobel operator to predict the target edge, and adding the edge loss to the loss function. Compared with FCN and Mask RCNN and other image extraction algorithms, the experimental results demonstrate that the improved Mask RCNN algorithm proposed in this paper is better in the precision, Recall, Average precision, Mean Average Precision, and F1 scores of crop image extraction results.
Collapse
|