1
|
Vernikouskaya I, Bertsche D, Metze P, Schneider LM, Rasche V. Multi-network approach for image segmentation in non-contrast enhanced cardiac 3D MRI of arrhythmic patients. Comput Med Imaging Graph 2024; 113:102340. [PMID: 38277768 DOI: 10.1016/j.compmedimag.2024.102340] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 01/16/2024] [Accepted: 01/16/2024] [Indexed: 01/28/2024]
Abstract
Left atrial appendage (LAA) is the source of thrombi formation in more than 90% of strokes in patients with nonvalvular atrial fibrillation. Catheter-based LAA occlusion is being increasingly applied as a treatment strategy to prevent stroke. Anatomical complexity of LAA makes percutaneous occlusion commonly performed under transesophageal echocardiography (TEE) and X-ray (XR) guidance especially challenging. Image fusion techniques integrating 3D anatomical models derived from pre-procedural imaging into the live XR fluoroscopy can be applied to guide each step of the LAA closure. Cardiac magnetic resonance (CMR) imaging gains in importance for radiation-free evaluation of cardiac morphology as alternative to gold-standard TEE or computed tomography angiography (CTA). Manual delineation of cardiac structures from non-contrast enhanced CMR is, however, labor-intensive, tedious, and challenging due to the rather low contrast. Additionally, arrhythmia often impairs the image quality in ECG synchronized acquisitions causing blurring and motion artifacts. Thus, for cardiac segmentation in arrhythmic patients, there is a strong need for an automated image segmentation method. Deep learning-based methods have shown great promise in medical image analysis achieving superior performance in various imaging modalities and different clinical applications. Fully-convolutional neural networks (CNNs), especially U-Net, have become the method of choice for cardiac segmentation. In this paper, we propose an approach for automatic segmentation of cardiac structures from non-contrast enhanced CMR images of arrhythmic patients based on CNNs implemented in a multi-stage pipeline. Two-stage implementation allows subdividing the task into localization of the relevant cardiac structures and segmentation of these structures from the cropped sub-regions obtained from previous step leading to efficient and effective way of automated cardiac segmentation.
Collapse
Affiliation(s)
- Ina Vernikouskaya
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| | - Dagmar Bertsche
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| | - Patrick Metze
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| | - Leonhard M Schneider
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| | - Volker Rasche
- Department of Internal Medicine II, Ulm University Medical Center, Ulm, Germany.
| |
Collapse
|
2
|
Woo B, Engstrom C, Baresic W, Fripp J, Crozier S, Chandra SS. Automated anomaly-aware 3D segmentation of bones and cartilages in knee MR images from the Osteoarthritis Initiative. Med Image Anal 2024; 93:103089. [PMID: 38246088 DOI: 10.1016/j.media.2024.103089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 09/25/2023] [Accepted: 01/12/2024] [Indexed: 01/23/2024]
Abstract
In medical image analysis, automated segmentation of multi-component anatomical entities, with the possible presence of variable anomalies or pathologies, is a challenging task. In this work, we develop a multi-step approach using U-Net-based models to initially detect anomalies (bone marrow lesions, bone cysts) in the distal femur, proximal tibia and patella from 3D magnetic resonance (MR) images in individuals with varying grades of knee osteoarthritis. Subsequently, the extracted data are used for downstream tasks involving semantic segmentation of individual bone and cartilage volumes as well as bone anomalies. For anomaly detection, U-Net-based models were developed to reconstruct bone volume profiles of the femur and tibia in images via inpainting so anomalous bone regions could be replaced with close to normal appearances. The reconstruction error was used to detect bone anomalies. An anomaly-aware segmentation network, which was compared to anomaly-naïve segmentation networks, was used to provide a final automated segmentation of the individual femoral, tibial and patellar bone and cartilage volumes from the knee MR images which contain a spectrum of bone anomalies. The anomaly-aware segmentation approach provided up to 58% reduction in Hausdorff distances for bone segmentations compared to the results from anomaly-naïve segmentation networks. In addition, the anomaly-aware networks were able to detect bone anomalies in the MR images with greater sensitivity and specificity (area under the receiver operating characteristic curve [AUC] up to 0.896) compared to anomaly-naïve segmentation networks (AUC up to 0.874).
Collapse
Affiliation(s)
- Boyeong Woo
- School of Electrical Engineering and Computer Science, The University of Queensland, Australia.
| | - Craig Engstrom
- School of Human Movement and Nutrition Sciences, The University of Queensland, Australia
| | - William Baresic
- School of Human Movement and Nutrition Sciences, The University of Queensland, Australia
| | - Jurgen Fripp
- School of Electrical Engineering and Computer Science, The University of Queensland, Australia; Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organization, Australia
| | - Stuart Crozier
- School of Electrical Engineering and Computer Science, The University of Queensland, Australia
| | - Shekhar S Chandra
- School of Electrical Engineering and Computer Science, The University of Queensland, Australia
| |
Collapse
|
3
|
Wang X, He F, Huang X. A new method for deep learning detection of defects in X-ray images of pressure vessel welds. Sci Rep 2024; 14:6312. [PMID: 38491060 PMCID: PMC10943115 DOI: 10.1038/s41598-024-56794-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 03/11/2024] [Indexed: 03/18/2024] Open
Abstract
Given that defect detection in weld X-ray images is a critical aspect of pressure vessel manufacturing and inspection, accurate differentiation of the type, distribution, number, and area of defects in the images serves as the foundation for judging weld quality, and the segmentation method of defects in digital X-ray images is the core technology for differentiating defects. Based on the publicly available weld seam dataset GDX-ray, this paper proposes a complete technique for fault segmentation in X-ray pictures of pressure vessel welds. The key works are as follows: (1) To address the problem of a lack of defect samples and imbalanced distribution inside GDX-ray, a DA-DCGAN based on a two-channel attention mechanism is devised to increase sample data. (2) A convolutional block attention mechanism is incorporated into the coding layer to boost the accuracy of small-scale defect identification. The proposed MAU-Net defect semantic segmentation network uses multi-scale even convolution to enhance large-scale features. The proposed method can mask electrostatic interference and non-defect-class parts in the actual weld X-ray images, achieve an average segmentation accuracy of 84.75% for the GDX-ray dataset, segment and accurately rate the valid defects with a correct rating rate of 95%, and thus realize practical value in engineering.
Collapse
Affiliation(s)
- Xue Wang
- School of Electrical Engineering, Chongqing University of Science and Technology, HuXi Street, Chongqing, 401331, China
| | - Feng He
- School of Electrical Engineering, Chongqing University of Science and Technology, HuXi Street, Chongqing, 401331, China.
| | - Xu Huang
- School of Electrical Engineering, Chongqing University of Science and Technology, HuXi Street, Chongqing, 401331, China
| |
Collapse
|
4
|
Siami M, Barszcz T, Wodecki J, Zimroz R. Semantic segmentation of thermal defects in belt conveyor idlers using thermal image augmentation and U-Net-based convolutional neural networks. Sci Rep 2024; 14:5748. [PMID: 38459162 PMCID: PMC10923815 DOI: 10.1038/s41598-024-55864-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 02/28/2024] [Indexed: 03/10/2024] Open
Abstract
The belt conveyor (BC) is the main means of horizontal transportation of bulk materials at mining sites. The sudden fault in BC modules may cause unexpected stops in production lines. With the increasing number of applications of inspection mobile robots in condition monitoring (CM) of industrial infrastructure in hazardous environments, in this article we introduce an image processing pipeline for automatic segmentation of thermal defects in thermal images captured from BC idlers using a mobile robot. This study follows the fact that CM of idler temperature is an important task for preventing sudden breakdowns in BC system networks. We compared the performance of three different types of U-Net-based convolutional neural network architectures for the identification of thermal anomalies using a small number of hand-labeled thermal images. Experiments on the test data set showed that the attention residual U-Net with binary cross entropy as the loss function handled the semantic segmentation problem better than our previous research and other studied U-Net variations.
Collapse
Affiliation(s)
- Mohammad Siami
- AMC Vibro Sp. z o.o., Pilotow 2e, 31-462, Kraków, Poland.
| | - Tomasz Barszcz
- Faculty of Mechanical Engineering and Robotics, AGH University, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Jacek Wodecki
- Faculty of Geoengineering, Mining and Geology, Wroclaw University of Science and Technology, Na Grobli 15, 50-421, Wroclaw, Poland
| | - Radoslaw Zimroz
- Faculty of Geoengineering, Mining and Geology, Wroclaw University of Science and Technology, Na Grobli 15, 50-421, Wroclaw, Poland
| |
Collapse
|
5
|
Kodipalli A, Fernandes SL, Dasar S. An Empirical Evaluation of a Novel Ensemble Deep Neural Network Model and Explainable AI for Accurate Segmentation and Classification of Ovarian Tumors Using CT Images. Diagnostics (Basel) 2024; 14:543. [PMID: 38473015 DOI: 10.3390/diagnostics14050543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 02/18/2024] [Accepted: 02/29/2024] [Indexed: 03/14/2024] Open
Abstract
Ovarian cancer is one of the leading causes of death worldwide among the female population. Early diagnosis is crucial for patient treatment. In this work, our main objective is to accurately detect and classify ovarian cancer. To achieve this, two datasets are considered: CT scan images of patients with cancer and those without, and biomarker (clinical parameters) data from all patients. We propose an ensemble deep neural network model and an ensemble machine learning model for the automatic binary classification of ovarian CT scan images and biomarker data. The proposed model incorporates four convolutional neural network models: VGG16, ResNet 152, Inception V3, and DenseNet 101, with transformers applied for feature extraction. These extracted features are fed into our proposed ensemble multi-layer perceptron model for classification. Preprocessing and CNN tuning techniques such as hyperparameter optimization, data augmentation, and fine-tuning are utilized during model training. Our ensemble model outperforms single classifiers and machine learning algorithms, achieving a mean accuracy of 98.96%, a precision of 97.44%, and an F1-score of 98.7%. We compared these results with those obtained using features extracted by the UNet model, followed by classification with our ensemble model. The transformer demonstrated superior performance in feature extraction over the UNet, with a mean Dice score and mean Jaccard score of 0.98 and 0.97, respectively, and standard deviations of 0.04 and 0.06 for benign tumors and 0.99 and 0.98 with standard deviations of 0.01 for malignant tumors. For the biomarker data, the combination of five machine learning models-KNN, logistic regression, SVM, decision tree, and random forest-resulted in an improved accuracy of 92.8% compared to single classifiers.
Collapse
Affiliation(s)
- Ashwini Kodipalli
- Department of Artificial Intelligence and Data Science, Global Academy of Technology, Bangalore 560098, India
| | - Steven L Fernandes
- Department of Computer Science, Design, Journalism, Creighton University, Omaha, NE 68178, USA
| | - Santosh Dasar
- Department of Radiology, SDM College of Medical Sciences & Hospital, Shri Dharmasthala Manjunatheshwara University, Dharwad 580009, India
| |
Collapse
|
6
|
Yao W, Bai J, Liao W, Chen Y, Liu M, Xie Y. From CNN to Transformer: A Review of Medical Image Segmentation Models. J Imaging Inform Med 2024:10.1007/s10278-024-00981-7. [PMID: 38438696 DOI: 10.1007/s10278-024-00981-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 11/13/2023] [Accepted: 11/14/2023] [Indexed: 03/06/2024]
Abstract
Medical image segmentation is an important step in medical image analysis, especially as a crucial prerequisite for efficient disease diagnosis and treatment. The use of deep learning for image segmentation has become a prevalent trend. The widely adopted approach currently is U-Net and its variants. Moreover, with the remarkable success of pre-trained models in natural language processing tasks, transformer-based models like TransUNet have achieved desirable performance on multiple medical image segmentation datasets. Recently, the Segment Anything Model (SAM) and its variants have also been attempted for medical image segmentation. In this paper, we conduct a survey of the most representative seven medical image segmentation models in recent years. We theoretically analyze the characteristics of these models and quantitatively evaluate their performance on Tuberculosis Chest X-rays, Ovarian Tumors, and Liver Segmentation datasets. Finally, we discuss the main challenges and future trends in medical image segmentation. Our work can assist researchers in the related field to quickly establish medical segmentation models tailored to specific regions.
Collapse
Affiliation(s)
- Wenjian Yao
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, 610054, Chengdu, China
| | - Jiajun Bai
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, 610054, Chengdu, China
| | - Wei Liao
- Department of Obstetrics and Gynaecology, Deyang People's Hospital, 618000, Deyang, China
| | - Yuheng Chen
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, 610054, Chengdu, China
| | - Mengjuan Liu
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, 610054, Chengdu, China.
| | - Yao Xie
- Department of Obstetrics and Gynaecology, Sichuan Provincial People's Hospital, School of Medicine, University of Electronic Science and Technology of China, Chengdu, China.
- Chinese Academy of Sciences Sichuan Translational Medicine Research Hospital, 610072, Chengdu, China.
| |
Collapse
|
7
|
Zhu Y, Xu Z, Lin Y, Chen D, Ai Z, Zhang H. A Multi-Source Data Fusion Network for Wood Surface Broken Defect Segmentation. Sensors (Basel) 2024; 24:1635. [PMID: 38475171 DOI: 10.3390/s24051635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Revised: 02/27/2024] [Accepted: 02/28/2024] [Indexed: 03/14/2024]
Abstract
Wood surface broken defects seriously damage the structure of wooden products, these defects have to be detected and eliminated. However, current defect detection methods based on machine vision have difficulty distinguishing the interference, similar to the broken defects, such as stains and mineral lines, and can result in frequent false detections. To address this issue, a multi-source data fusion network based on U-Net is proposed for wood broken defect detection, combining image and depth data, to suppress the interference and achieve complete segmentation of the defects. To efficiently extract various semantic information of defects, an improved ResNet34 is designed to, respectively, generate multi-level features of the image and depth data, in which the depthwise separable convolution (DSC) and dilated convolution (DC) are introduced to decrease the computational expense and feature redundancy. To take full advantages of two types of data, an adaptive interacting fusion module (AIF) is designed to adaptively integrate them, thereby generating accurate feature representation of the broken defects. The experiments demonstrate that the multi-source data fusion network can effectively improve the detection accuracy of wood broken defects and reduce the false detections of interference, such as stains and mineral lines.
Collapse
Affiliation(s)
- Yuhang Zhu
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| | - Zhezhuang Xu
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| | - Ye Lin
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| | - Dan Chen
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| | - Zhijie Ai
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| | - Hongchuan Zhang
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China
| |
Collapse
|
8
|
Gi Y, Oh G, Jo Y, Lim H, Ko Y, Hong J, Lee E, Park S, Kwak T, Kim S, Yoon M. Study of multistep Dense U-Net-based automatic segmentation for head MRI scans. Med Phys 2024; 51:2230-2238. [PMID: 37956307 DOI: 10.1002/mp.16824] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Revised: 09/25/2023] [Accepted: 10/22/2023] [Indexed: 11/15/2023] Open
Abstract
BACKGROUND Despite extensive efforts to obtain accurate segmentation of magnetic resonance imaging (MRI) scans of a head, it remains challenging primarily due to variations in intensity distribution, which depend on the equipment and parameters used. PURPOSE The goal of this study is to evaluate the effectiveness of an automatic segmentation method for head MRI scans using a multistep Dense U-Net (MDU-Net) architecture. METHODS The MDU-Net-based method comprises two steps. The first step is to segment the scalp, skull, and whole brain from head MRI scans using a convolutional neural network (CNN). In the first step, a hybrid network is used to combine 2.5D Dense U-Net and 3D Dense U-Net structure. This hybrid network acquires logits in three orthogonal planes (axial, coronal, and sagittal) using 2.5D Dense U-Nets and fuses them by averaging. The resultant fused probability map with head MRI scans then serves as the input to a 3D Dense U-Net. In this process, different ratios of active contour loss and focal loss are applied. The second step is to segment the cerebrospinal fluid (CSF), white matter, and gray matter from extracted brain MRI scans using CNNs. In the second step, the histogram of the extracted brain MRI scans is standardized and then a 2.5D Dense U-Net is used to further segment the brain's specific tissues using the focal loss. A dataset of 100 head MRI scans from an OASIS-3 dataset was used for training, internal validation, and testing, with ratios of 80%, 10%, and 10%, respectively. Using the proposed approach, we segmented the head MRI scans into five areas (scalp, skull, CSF, white matter, and gray matter) and evaluated the segmentation results using the Dice similarity coefficient (DSC) score, Hausdorff distance (HD), and the average symmetric surface distance (ASSD) as evaluation metrics. We compared these results with those obtained using the Res-U-Net, Dense U-Net, U-Net++, Swin-Unet, and H-Dense U-Net models. RESULTS The MDU-Net model showed DSC values of 0.933, 0.830, 0.833, 0.953, and 0.917 in the scalp, skull, CSF, white matter, and gray matter, respectively. The corresponding HD values were 2.37, 2.89, 2.13, 1.52, and 1.53 mm, respectively. The ASSD values were 0.50, 1.63, 1.28, 0.26, and 0.27 mm, respectively. Comparing these results with other models revealed that the MDU-Net model demonstrated the best performance in terms of the DSC values for the scalp, CSF, white matter, and gray matter. When compared with the H-Dense U-Net model, which showed the highest performance among the other models, the MDU-Net model showed substantial improvements in the HD view, particularly in the gray matter region, with a difference of approximately 9%. In addition, in terms of the ASSD, the MDU-Net model outperformed the H-Dense U-Net model, showing an approximately 7% improvements in the white matter and approximately 9% improvements in the gray matter. CONCLUSION Compared with existing models in terms of DSC, HD, and ASSD, the proposed MDU-Net model demonstrated the best performance on average and showed its potential to enhance the accuracy of automatic segmentation for head MRI scans.
Collapse
Affiliation(s)
- Yongha Gi
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Geon Oh
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Yunhui Jo
- Institute of Global Health Technology (IGHT), Korea University, Seoul, Republic of Korea
| | - Hyeongjin Lim
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Yousun Ko
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Jinyoung Hong
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Eunjun Lee
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
| | - Sangmin Park
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
- Field Cure Ltd., Seoul, Republic of Korea
| | - Taemin Kwak
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
- Field Cure Ltd., Seoul, Republic of Korea
| | - Sangcheol Kim
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
- Field Cure Ltd., Seoul, Republic of Korea
| | - Myonggeun Yoon
- Department of Bio-medical Engineering, Korea University, Seoul, Republic of Korea
- Field Cure Ltd., Seoul, Republic of Korea
| |
Collapse
|
9
|
Domadia SG, Thakkar FN, Ardeshana MA. Segmenting brain glioblastoma using dense-attentive 3D DAF 2. Phys Med 2024; 119:103304. [PMID: 38340694 DOI: 10.1016/j.ejmp.2024.103304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Revised: 12/18/2023] [Accepted: 01/29/2024] [Indexed: 02/12/2024] Open
Abstract
Precise delineation of brain glioblastoma or tumor through segmentation is pivotal in the diagnosis, formulating treatment strategies, and evaluating therapeutic progress in patients. Precisely identifying brain glioblastoma within multimodal MRI scans poses a significant challenge in the field of medical image analysis as different intensity profiles are observed across the sub-regions, reflecting diverse tumor biological properties. For segmenting glioblastoma areas, convolutional neural networks have displayed astounding performance in recent years. This paper introduces an innovative methodology for brain glioblastoma segmentation by combining the Dense-Attention 3D U-Net network with a fusion strategy and the focal tversky loss function. By fusing information from multiple resolution segmentation maps, our model enhances its ability to discern intricate tumor boundaries. Incorporating the focal tversky loss function, we effectively emphasize critical regions and mitigate class imbalance. Recursive Convolution Block 2 is proposed after fusion to ensure efficient utilization of all accessible features while maintaining rapid convergence. The network's effectiveness is assessed using diverse datasets BraTS 2020 and BraTS 2021. Results show comparable dice similarity coefficient compared to other methods with increased efficiency and segmentation performance. Additionally, the architecture achieved an average dice similarity coefficient of 82.4% and an average hausdorff distance (HD95) of 10.426, which demonstrated consistent performance improvement compared to baseline models like U-Net, Attention U-Net, V-Net and Res U-Net and indicating the effectiveness of proposed architecture.
Collapse
|
10
|
Sohn J, Shin H, Lee J, Kim HC. Validation of Electrocardiogram Based Photoplethysmogram Generated Using U-Net Based Generative Adversarial Networks. J Healthc Inform Res 2024; 8:140-157. [PMID: 38273980 PMCID: PMC10805750 DOI: 10.1007/s41666-023-00156-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 10/24/2023] [Accepted: 11/13/2023] [Indexed: 01/27/2024]
Abstract
Photoplethysmogram (PPG) performs an important role in alarming atrial fibrillation (AF). While the importance of PPG is emphasized, there is insufficient amount of openly available atrial fibrillation PPG data. We propose a U-net-based generative adversarial network (GAN) which synthesize PPG from paired electrocardiogram (ECG). To measure the performance of the proposed GAN, we compared the generated PPG to reference PPG in terms of morphology similarity and also examined its influence on AF detection classifier performance. First, morphology was compared using two different metrics against the reference signal: percent root mean square difference (PRD) and Pearson correlation coefficient. The mean PRD and Pearson correlation coefficient were 27% and 0.94, respectively. Heart rate variability (HRV) of the reference AF ECG and the generated PPG were compared as well. The p-value of the paired t-test was 0.248, indicating that no significant difference was observed between the two HRV values. Second, to validate the generated AF PPG dataset, four different datasets were prepared combining the generated PPG and real AF PPG. Each dataset was used to optimize a classification model while maintaining the same architecture. A test dataset was prepared to test the performance of each optimized model. Subsequently, these datasets were used to test the hypothesis whether the generated data benefits the training of an AF classifier. Comparing the performance metrics of each optimized model, the training dataset consisting of generated and real AF PPG showed a test accuracy result of 0.962, which was close to that of the dataset consisting only of real AF PPG data at 0.961. Furthermore, both models yielded the same F1 score of 0.969. Lastly, using only the generated AF PPG dataset resulted in test accuracy of 0.945, indicating that the trained model was capable of generating valuable AF PPG. Therefore, it can be concluded that the generated AF PPG can be used to augment insufficient data. To summarize, this study proposes a GAN-based method to generate atrial fibrillation PPG that can be used for training atrial fibrillation PPG classification models.
Collapse
Affiliation(s)
- Jangjay Sohn
- Institute of Medical & Biological Engineering, Medical Research Center, Seoul National University College of Medicine, Seoul, Korea
- Department of Electronic Engineering, Hanyang University, Seoul, Korea
| | - Heean Shin
- Samsung SDS R&D Center, Seoul, Republic of Korea
| | - Joonnyong Lee
- Mellowing Factory Co., Ltd., 131 Sapeyong-daero 57-gil, Seocho-gu, Seoul, 06535 Republic of Korea
| | - Hee Chan Kim
- Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, Korea
- Department of Biomedical Engineering, Seoul National University College of Medicine, 103, Daehak-ro, Jongno-gu, Seoul, 03080 Republic of Korea
| |
Collapse
|
11
|
Seiche AT, Wittstruck L, Jarmer T. Weed Detection from Unmanned Aerial Vehicle Imagery Using Deep Learning-A Comparison between High-End and Low-Cost Multispectral Sensors. Sensors (Basel) 2024; 24:1544. [PMID: 38475081 DOI: 10.3390/s24051544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Revised: 02/13/2024] [Accepted: 02/14/2024] [Indexed: 03/14/2024]
Abstract
In order to meet the increasing demand for crops under challenging climate conditions, efficient and sustainable cultivation strategies are becoming essential in agriculture. Targeted herbicide use reduces environmental pollution and effectively controls weeds as a major cause of yield reduction. The key requirement is a reliable weed detection system that is accessible to a wide range of end users. This research paper introduces a self-built, low-cost, multispectral camera system and evaluates it against the high-end MicaSense Altum system. Pixel-based weed and crop classification was performed on UAV datasets collected with both sensors in maize using a U-Net. The training and testing data were generated via an index-based thresholding approach followed by annotation. As a result, the F1-score for the weed class reached 82% on the Altum system and 76% on the low-cost system, with recall values of 75% and 68%, respectively. Misclassifications occurred on the low-cost system images for small weeds and overlaps, with minor oversegmentation. However, with a precision of 90%, the results show great potential for application in automated weed control. The proposed system thereby enables sustainable precision farming for the general public. In future research, its spectral properties, as well as its use on different crops with real-time on-board processing, should be further investigated.
Collapse
Affiliation(s)
- Anna Teresa Seiche
- Institute of Computer Science, Osnabrück University, 49090 Osnabrück, Germany
| | - Lucas Wittstruck
- Institute of Computer Science, Osnabrück University, 49090 Osnabrück, Germany
| | - Thomas Jarmer
- Institute of Computer Science, Osnabrück University, 49090 Osnabrück, Germany
| |
Collapse
|
12
|
Karn PK, Abdulla WH. Advancing Ocular Imaging: A Hybrid Attention Mechanism-Based U-Net Model for Precise Segmentation of Sub-Retinal Layers in OCT Images. Bioengineering (Basel) 2024; 11:240. [PMID: 38534514 DOI: 10.3390/bioengineering11030240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 02/21/2024] [Accepted: 02/26/2024] [Indexed: 03/28/2024] Open
Abstract
This paper presents a novel U-Net model incorporating a hybrid attention mechanism for automating the segmentation of sub-retinal layers in Optical Coherence Tomography (OCT) images. OCT is an ophthalmology tool that provides detailed insights into retinal structures. Manual segmentation of these layers is time-consuming and subjective, calling for automated solutions. Our proposed model combines edge and spatial attention mechanisms with the U-Net architecture to improve segmentation accuracy. By leveraging attention mechanisms, the U-Net focuses selectively on image features. Extensive evaluations using datasets demonstrate that our model outperforms existing approaches, making it a valuable tool for medical professionals. The study also highlights the model's robustness through performance metrics such as an average Dice score of 94.99%, Adjusted Rand Index (ARI) of 97.00%, and Strength of Agreement (SOA) classifications like "Almost Perfect", "Excellent", and "Very Strong". This advanced predictive model shows promise in expediting processes and enhancing the precision of ocular imaging in real-world applications.
Collapse
Affiliation(s)
- Prakash Kumar Karn
- Department of Electrical, Computer and Software Engineering, The University of Auckland, Auckland 1010, New Zealand
| | - Waleed H Abdulla
- Department of Electrical, Computer and Software Engineering, The University of Auckland, Auckland 1010, New Zealand
| |
Collapse
|
13
|
Wang X, Zhang S, Zhang T. Crop insect pest detection based on dilated multi-scale attention U-Net. Plant Methods 2024; 20:34. [PMID: 38409023 PMCID: PMC10898010 DOI: 10.1186/s13007-024-01163-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Accepted: 02/20/2024] [Indexed: 02/28/2024]
Abstract
BACKGROUND Crop pests seriously affect the yield and quality of crops. Accurately and rapidly detecting and segmenting insect pests in crop leaves is a premise for effectively controlling insect pests. METHODS Aiming at the detection problem of irregular multi-scale insect pests in the field, a dilated multi-scale attention U-Net (DMSAU-Net) model is constructed for crop insect pest detection. In its encoder, dilated Inception is designed to replace the convolution layer in U-Net to extract the multi-scale features of insect pest images. An attention module is added to its decoder to focus on the edge of the insect pest image. RESULTS The experiments on the crop insect pest image IP102 dataset are implemented, and achieved the detection accuracy of 92.16% and IoU of 91.2%, which is 3.3% and 1.5% higher than that of MSR-RCNN, respectively. CONCLUSION The results indicate that the proposed method is effective as a new insect pest detection method. The dilated Inception can improve the accuracy of the model, and the attention module can reduce the noise generated by upsampling and accelerate model convergence. It can be concluded that the proposed method can be applied to practical crop insect pest monitoring system.
Collapse
Affiliation(s)
- Xuqi Wang
- School of Electronic Information, Xijing University, Xi'an, 710123, China
| | - Shanwen Zhang
- School of Electronic Information, Xijing University, Xi'an, 710123, China.
| | - Ting Zhang
- School of Electronic Information, Xijing University, Xi'an, 710123, China
| |
Collapse
|
14
|
Yurova A, Lychagin A, Kalinsky E, Vassilevski Y, Elizarov M, Garkavi A. Automated personalization of biomechanical knee model. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03075-5. [PMID: 38402535 DOI: 10.1007/s11548-024-03075-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 02/09/2024] [Indexed: 02/26/2024]
Abstract
PURPOSE Patient-specific biomechanical models of the knee joint can effectively aid in understanding the reasons for pathologies and improve diagnostic methods and treatment procedures. For deeper research of knee diseases, the development of biomechanical models with appropriate configurations is essential. In this study, we mainly focus on the development of a personalized biomechanical model for the investigation of knee joint pathologies related to patellar motion using automated methods. METHODS This study presents a biomechanical model created for patellar motion pathologies research and some techniques for automating the generation of the biomechanical model. To generate geometric models of bones, the U-Net neural network was adapted for 3D input datasets. The method uses the same neural network for segmentation of femur, tibia, patella and fibula. The total size of the train/validation (75/25%) dataset is 18,183 3D volumes of size [Formula: see text] voxels. The configuration of the biomechanical knee model proposed in the paper includes six degrees of freedom for the tibiofemoral and patellofemoral joints, lateral and medial contact surfaces for femur and tibia, and ligaments, representing, among other things, the medial and lateral stabilizers of the knee cap. The development of the personalized biomechanical model was carried out using the OpenSim software system. The automated model generation was implemented using OpenSim Python scripting commands. RESULTS The neural network for bones segmentation achieves mean DICE 0.9838. A biomechanical model for realistic simulation of patellar movement within the trochlear groove was proposed. Generation of personalized biomechanical models was automated. CONCLUSIONS In this paper, we have implemented a neural network for the segmentation of 3D CT scans of the knee joint to produce a biomechanical model for the study of knee cap motion pathologies. Most stages of the generation process have been automated and can be used to generate patient-specific models.
Collapse
Affiliation(s)
- Alexandra Yurova
- Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences, 8 Gubkin Str., Moscow, 119333, Russia.
| | - Alexey Lychagin
- Sechenov University, 8-2 Trubetskaya str., Moscow, 119991, Russia
| | - Eugene Kalinsky
- Sechenov University, 8-2 Trubetskaya str., Moscow, 119991, Russia
| | - Yuri Vassilevski
- Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences, 8 Gubkin Str., Moscow, 119333, Russia
- Sechenov University, 8-2 Trubetskaya str., Moscow, 119991, Russia
- Center for IT &AI, Sirius University, 1 Olympiyskii pr., Sochi, 354340, Russia
| | - Mikhail Elizarov
- Sechenov University, 8-2 Trubetskaya str., Moscow, 119991, Russia
| | - Andrey Garkavi
- Sechenov University, 8-2 Trubetskaya str., Moscow, 119991, Russia
| |
Collapse
|
15
|
Xia H, Tonooka H. Extraction of Coastal Levees Using U-Net Model with Visible and Topographic Images Observed by High-Resolution Satellite Sensors. Sensors (Basel) 2024; 24:1444. [PMID: 38474979 DOI: 10.3390/s24051444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 02/11/2024] [Accepted: 02/21/2024] [Indexed: 03/14/2024]
Abstract
Coastal levees play a role in protecting coastal areas from storm surges and high waves, and they provide important input information for inundation damage simulations. However, coastal levee data with uniformity and sufficient accuracy for inundation simulations are not always well developed. Against this background, this study proposed a method to extract coastal levees by inputting high spatial resolution optical satellite image products (RGB images, digital surface models (DSMs), and slope images that can be generated from DSM images), which have high data availability at the locations and times required for simulation, into a deep learning model. The model is based on U-Net, and post-processing for noise removal was introduced to further improve its accuracy. We also proposed a method to calculate levee height using a local maximum filter by giving DSM values to the extracted levee pixels. The validation was conducted in the coastal area of Ibaraki Prefecture in Japan as a test area. The levee mask images for training were manually created by combining these data with satellite images and Google Street View, because the levee GIS data created by the Ibaraki Prefectural Government were incomplete in some parts. First, the deep learning models were compared and evaluated, and it was shown that U-Net was more accurate than Pix2Pix and BBS-Net in identifying levees. Next, three cases of input images were evaluated: (Case 1) RGB image only, (Case 2) RGB and DSM images, and (Case 3) RGB, DSM, and slope images. Case 3 was found to be the most accurate, with an average Matthews correlation coefficient of 0.674. The effectiveness of noise removal post-processing was also demonstrated. In addition, an example of the calculation of levee heights was presented and evaluated for validity. In conclusion, this method was shown to be effective in extracting coastal levees. The evaluation of generalizability and use in actual inundation simulations are future tasks.
Collapse
Affiliation(s)
- Hao Xia
- Graduate School of Science and Engineering, Ibaraki University, Hitachi 3168511, Japan
| | - Hideyuki Tonooka
- Graduate School of Science and Engineering, Ibaraki University, Hitachi 3168511, Japan
| |
Collapse
|
16
|
Kumar Lilhore U, Simaiya S, Sharma YK, Kaswan KS, Rao KBVB, Rao VVRM, Baliyan A, Bijalwan A, Alroobaea R. A precise model for skin cancer diagnosis using hybrid U-Net and improved MobileNet-V3 with hyperparameters optimization. Sci Rep 2024; 14:4299. [PMID: 38383520 PMCID: PMC10881962 DOI: 10.1038/s41598-024-54212-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 02/09/2024] [Indexed: 02/23/2024] Open
Abstract
Skin cancer is a frequently occurring and possibly deadly disease that necessitates prompt and precise diagnosis in order to ensure efficacious treatment. This paper introduces an innovative approach for accurately identifying skin cancer by utilizing Convolution Neural Network architecture and optimizing hyperparameters. The proposed approach aims to increase the precision and efficacy of skin cancer recognition and consequently enhance patients' experiences. This investigation aims to tackle various significant challenges in skin cancer recognition, encompassing feature extraction, model architecture design, and optimizing hyperparameters. The proposed model utilizes advanced deep-learning methodologies to extract complex features and patterns from skin cancer images. We enhance the learning procedure of deep learning by integrating Standard U-Net and Improved MobileNet-V3 with optimization techniques, allowing the model to differentiate malignant and benign skin cancers. Also substituted the crossed-entropy loss function of the Mobilenet-v3 mathematical framework with a bias loss function to enhance the accuracy. The model's squeeze and excitation component was replaced with the practical channel attention component to achieve parameter reduction. Integrating cross-layer connections among Mobile modules has been proposed to leverage synthetic features effectively. The dilated convolutions were incorporated into the model to enhance the receptive field. The optimization of hyperparameters is of utmost importance in improving the efficiency of deep learning models. To fine-tune the model's hyperparameter, we employ sophisticated optimization methods such as the Bayesian optimization method using pre-trained CNN architecture MobileNet-V3. The proposed model is compared with existing models, i.e., MobileNet, VGG-16, MobileNet-V2, Resnet-152v2 and VGG-19 on the "HAM-10000 Melanoma Skin Cancer dataset". The empirical findings illustrate that the proposed optimized hybrid MobileNet-V3 model outperforms existing skin cancer detection and segmentation techniques based on high precision of 97.84%, sensitivity of 96.35%, accuracy of 98.86% and specificity of 97.32%. The enhanced performance of this research resulted in timelier and more precise diagnoses, potentially contributing to life-saving outcomes and mitigating healthcare expenditures.
Collapse
Affiliation(s)
- Umesh Kumar Lilhore
- Department of Computer Science and Engineering, Chandigarh University, Mohali, Punjab, 140413, India
| | - Sarita Simaiya
- Department of Computer Science and Engineering, Chandigarh University, Mohali, Punjab, 140413, India
| | - Yogesh Kumar Sharma
- Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Greenfield, Vaddeswaram, Guntur, AP, India
| | - Kuldeep Singh Kaswan
- School of Computing Science and Engineering, Galgotias University, Greater Noida, Uttar Pradesh, India
| | - K B V Brahma Rao
- Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Greenfield, Vaddeswaram, Guntur, AP, India
| | - V V R Maheswara Rao
- Departmentt of Computer Science and Engineering, Shri Vishnu Engineering College for Women (A), Bhimavaram, India
| | - Anupam Baliyan
- Department of Computer Science and Engineering, Chandigarh University, Mohali, Punjab, 140413, India
| | | | - Roobaea Alroobaea
- Department of Computer Science, College of Computers and Information Technology, Taif University, P. O. Box 11099, 21944, Taif, Saudi Arabia
| |
Collapse
|
17
|
Watanabe H, Fukuda H, Ezawa Y, Matsuyama E, Kondo Y, Hayashi N, Ogura T, Shimosegawa M. Automated angular measurement for puncture angle using a computer-aided method in ultrasound-guided peripheral insertion. Phys Eng Sci Med 2024:10.1007/s13246-024-01397-x. [PMID: 38358620 DOI: 10.1007/s13246-024-01397-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 01/28/2024] [Indexed: 02/16/2024]
Abstract
Ultrasound guidance has become the gold standard for obtaining vascular access. Angle information, which indicates the entry angle of the needle into the vein, is required to ensure puncture success. Although various image processing-based methods, such as deep learning, have recently been applied to improve needle visibility, these methods have limitations, in that the puncture angle to the target organ is not measured. We aim to detect the target vessel and puncture needle and to derive the puncture angle by combining deep learning and conventional image processing methods such as the Hough transform. Median cubital vein US images were obtained from 20 healthy volunteers, and images of simulated blood vessels and needles were obtained during the puncture of a simulated blood vessel in four phantoms. The U-Net architecture was used to segment images of blood vessels and needles, and various image processing methods were employed to automatically measure angles. The experimental results indicated that the mean dice coefficients of median cubital veins, simulated blood vessels, and needles were 0.826, 0.931, and 0.773, respectively. The quantitative results of angular measurement showed good agreement between the expert and automatic measurements of the puncture angle with 0.847 correlations. Our findings indicate that the proposed method achieves extremely high segmentation accuracy and automated angular measurements. The proposed method reduces the variability and time required in manual angle measurements and presents the possibility where the operator can concentrate on delicate techniques related to the direction of the needle.
Collapse
Affiliation(s)
- Haruyuki Watanabe
- School of Radiological Technology, Gunma Prefectural College of Health Sciences, Maebashi, Japan.
| | - Hironori Fukuda
- Department of Radiology, Cardiovascular Hospital of Central Japan, Shibukawa, Japan
| | - Yuina Ezawa
- School of Radiological Technology, Gunma Prefectural College of Health Sciences, Maebashi, Japan
| | - Eri Matsuyama
- Faculty of Informatics, The University of Fukuchiyama, Fukuchiyama, Japan
| | - Yohan Kondo
- Graduate School of Health Sciences, Niigata University, Niigata, Japan
| | - Norio Hayashi
- School of Radiological Technology, Gunma Prefectural College of Health Sciences, Maebashi, Japan
| | - Toshihiro Ogura
- School of Radiological Technology, Gunma Prefectural College of Health Sciences, Maebashi, Japan
| | - Masayuki Shimosegawa
- School of Radiological Technology, Gunma Prefectural College of Health Sciences, Maebashi, Japan
| |
Collapse
|
18
|
Zhang D, Li L, Zhang J, Ren J, Gu J, Li L, Jiang B, Zhang S. Quantitative Detection of Defects in Multi-Layer Lightweight Composite Structures Using THz-TDS Based on a U-Net-BiLSTM Network. Materials (Basel) 2024; 17:839. [PMID: 38399090 PMCID: PMC10890636 DOI: 10.3390/ma17040839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 01/12/2024] [Accepted: 01/16/2024] [Indexed: 02/25/2024]
Abstract
Multi-layer lightweight composite structures are widely used in the field of aviation and aerospace during the processes of manufacturing and use, and, as such, they inevitably produce defects, damage, and other quality problems, creating the need for timely non-destructive testing procedures and the convenient repair or replacement of quality problems related to the material. When using terahertz non-destructive testing technology to detect defects in multi-layer lightweight composite materials, due to the complexity of their structure and defect types, there are many signal characteristics of terahertz waves propagating in the structures, and there is no obvious rule behind them, resulting in a large gap between the recognition results and the actual ones. In this study, we introduced a U-Net-BiLSTM network that combines the strengths of the U-Net and BiLSTM networks. The U-Net network extracts the spatial features of THz signals, while the BiLSTM network captures their temporal features. By optimizing the network structure and various parameters, we obtained a model tailored to THz spectroscopy data. This model was subsequently employed for the identification and quantitative analysis of defects in multi-layer lightweight composite structures using THz non-destructive testing. The proposed U-Net-BiLSTM network achieved an accuracy of 99.45% in typical defect identification, with a comprehensive F1 score of 99.43%, outperforming the CNN, ResNet, U-Net, and BiLSTM networks. By leveraging defect classification and thickness recognition, this study successfully reconstructed three-dimensional THz defect images, thereby realizing quantitative defect detection.
Collapse
Affiliation(s)
- Dandan Zhang
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Lulu Li
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Jiyang Zhang
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Jiaojiao Ren
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Jian Gu
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Lijuan Li
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| | - Baihong Jiang
- Institute of Aerospace Special Materials and Technology, Beijing 100074, China
| | - Shida Zhang
- Key Laboratory of Optoelectronic Measurement and Control and Optical Information Transmission Technology, Ministry of Education, Changchun University of Science and Technology, Changchun 130022, China
- School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China
- Zhongshan Institute of Changchun University of Science and Technology, Zhongshan 528400, China
| |
Collapse
|
19
|
Ji Z, Mu J, Liu J, Zhang H, Dai C, Zhang X, Ganchev I. ASD-Net: a novel U-Net based asymmetric spatial-channel convolution network for precise kidney and kidney tumor image segmentation. Med Biol Eng Comput 2024:10.1007/s11517-024-03025-y. [PMID: 38326677 DOI: 10.1007/s11517-024-03025-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 01/15/2024] [Indexed: 02/09/2024]
Abstract
Early intervention in tumors can greatly improve human survival rates. With the development of deep learning technology, automatic image segmentation has taken a prominent role in the field of medical image analysis. Manually segmenting kidneys on CT images is a tedious task, and due to the diversity of these images and varying technical skills of professionals, segmentation results can be inconsistent. To address this problem, a novel ASD-Net network is proposed in this paper for kidney and kidney tumor segmentation tasks. First, the proposed network employs newly designed Adaptive Spatial-channel Convolution Optimization (ASCO) blocks to capture anisotropic information in the images. Then, other newly designed blocks, i.e., Dense Dilated Enhancement Convolution (DDEC) blocks, are utilized to enhance feature propagation and reuse it across the network, thereby improving its segmentation accuracy. To allow the network to segment complex and small kidney tumors more effectively, the Atrous Spatial Pyramid Pooling (ASPP) module is incorporated in its middle layer. With its generalized pyramid feature, this module enables the network to better capture and understand context information at various scales within the images. In addition to this, the concurrent spatial and channel squeeze & excitation (scSE) attention mechanism is adopted to better comprehend and manage context information in the images. Additional encoding layers are also added to the base (U-Net) and connected to the original encoding layer through skip connections. The resultant enhanced U-Net structure allows for better extraction and merging of high-level and low-level features, further boosting the network's ability to restore segmentation details. In addition, the combined Binary Cross Entropy (BCE)-Dice loss is utilized as the network's loss function. Experiments, conducted on the KiTS19 dataset, demonstrate that the proposed ASD-Net network outperforms the existing segmentation networks according to all evaluation metrics used, except for recall in the case of kidney tumor segmentation, where it takes the second place after Attention-UNet.
Collapse
Affiliation(s)
- Zhanlin Ji
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, People's Republic of China
| | - Juncheng Mu
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, People's Republic of China
| | - Jianuo Liu
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, People's Republic of China
| | - Haiyang Zhang
- Department of Computing, Xi'an Jiaotong-Liverpool University, Suzhou, People's Republic of China
| | - Chenxu Dai
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, People's Republic of China
| | - Xueji Zhang
- School of Biomedical Engineering, Shenzhen University Health Science Center, Shenzhen, Guangdong, 518060, People's Republic of China.
| | - Ivan Ganchev
- Telecommunications Research Centre (TRC), University of Limerick, Limerick, V94 T9PX, Ireland.
- Department of Computer Systems, University of Plovdiv "Paisii Hilendarski", Plovdiv, 4000, Bulgaria.
- Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Sofia, 1040, Bulgaria.
| |
Collapse
|
20
|
Hong JS, You WC, Sun MH, Pan HC, Lin YH, Lu YF, Chen KM, Huang TH, Lee WK, Wu YT. Deep Learning Detection and Segmentation of Brain Arteriovenous Malformation on Magnetic Resonance Angiography. J Magn Reson Imaging 2024; 59:587-598. [PMID: 37220191 DOI: 10.1002/jmri.28795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 05/06/2023] [Accepted: 05/08/2023] [Indexed: 05/25/2023] Open
Abstract
BACKGROUND The delineation of brain arteriovenous malformations (bAVMs) is crucial for subsequent treatment planning. Manual segmentation is time-consuming and labor-intensive. Applying deep learning to automatically detect and segment bAVM might help to improve clinical practice efficiency. PURPOSE To develop an approach for detecting bAVM and segmenting its nidus on Time-of-flight magnetic resonance angiography using deep learning methods. STUDY TYPE Retrospective. SUBJECTS 221 bAVM patients aged 7-79 underwent radiosurgery from 2003 to 2020. They were split into 177 training, 22 validation, and 22 test data. FIELD STRENGTH/SEQUENCE 1.5 T, Time-of-flight magnetic resonance angiography based on 3D gradient echo. ASSESSMENT The YOLOv5 and YOLOv8 algorithms were utilized to detect bAVM lesions and the U-Net and U-Net++ models to segment the nidus from the bounding boxes. The mean average precision, F1, precision, and recall were used to assess the model performance on the bAVM detection. To evaluate the model's performance on nidus segmentation, the Dice coefficient and balanced average Hausdorff distance (rbAHD) were employed. STATISTICAL TESTS The Student's t-test was used to test the cross-validation results (P < 0.05). The Wilcoxon rank test was applied to compare the median for the reference values and the model inference results (P < 0.05). RESULTS The detection results demonstrated that the model with pretraining and augmentation performed optimally. The U-Net++ with random dilation mechanism resulted in higher Dice and lower rbAHD, compared to that without that mechanism, across varying dilated bounding box conditions (P < 0.05). When combining detection and segmentation, the Dice and rbAHD were statistically different from the references calculated using the detected bounding boxes (P < 0.05). For the detected lesions in the test dataset, it showed the highest Dice of 0.82 and the lowest rbAHD of 5.3%. DATA CONCLUSION This study showed that pretraining and data augmentation improved YOLO detection performance. Properly limiting lesion ranges allows for adequate bAVM segmentation. LEVEL OF EVIDENCE 4 TECHNICAL EFFICACY STAGE: 1.
Collapse
Affiliation(s)
- Jia-Sheng Hong
- Institute of Biophotonics, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
| | - Weir-Chiang You
- Department of Radiation Oncology, Taichung Veterans General Hospital, Taichung, 407, Taiwan
| | - Ming-Hsi Sun
- Department of Neurosurgery, Taichung Veterans General Hospital, Taichung, 407, Taiwan
| | - Hung-Chuan Pan
- Department of Neurosurgery, Taichung Veterans General Hospital, Taichung, 407, Taiwan
| | - Yi-Hui Lin
- Department of Radiation Oncology, Taichung Veterans General Hospital, Taichung, 407, Taiwan
| | - Yung-Fa Lu
- Department of Radiation Oncology, Taichung Veterans General Hospital, Taichung, 407, Taiwan
| | - Kuan-Ming Chen
- Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
| | - Tzu-Hsuan Huang
- Institute of Biophotonics, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
| | - Wei-Kai Lee
- Institute of Biophotonics, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
- Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
| | - Yu-Te Wu
- Institute of Biophotonics, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei City, 112, Taiwan
| |
Collapse
|
21
|
Mathieu L, Reder M, Siah A, Ducasse A, Langlands-Perry C, Marcel TC, Morel JB, Saintenac C, Ballini E. SeptoSympto: a precise image analysis of Septoria tritici blotch disease symptoms using deep learning methods on scanned images. Plant Methods 2024; 20:18. [PMID: 38297386 PMCID: PMC10832182 DOI: 10.1186/s13007-024-01136-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 01/07/2024] [Indexed: 02/02/2024]
Abstract
BACKGROUND Investigations on plant-pathogen interactions require quantitative, accurate, and rapid phenotyping of crop diseases. However, visual assessment of disease symptoms is preferred over available numerical tools due to transferability challenges. These assessments are laborious, time-consuming, require expertise, and are rater dependent. More recently, deep learning has produced interesting results for evaluating plant diseases. Nevertheless, it has yet to be used to quantify the severity of Septoria tritici blotch (STB) caused by Zymoseptoria tritici-a frequently occurring and damaging disease on wheat crops. RESULTS We developed an image analysis script in Python, called SeptoSympto. This script uses deep learning models based on the U-Net and YOLO architectures to quantify necrosis and pycnidia on detached, flattened and scanned leaves of wheat seedlings. Datasets of different sizes (containing 50, 100, 200, and 300 leaves) were annotated to train Convolutional Neural Networks models. Five different datasets were tested to develop a robust tool for the accurate analysis of STB symptoms and facilitate its transferability. The results show that (i) the amount of annotated data does not influence the performances of models, (ii) the outputs of SeptoSympto are highly correlated with those of the experts, with a similar magnitude to the correlations between experts, and (iii) the accuracy of SeptoSympto allows precise and rapid quantification of necrosis and pycnidia on both durum and bread wheat leaves inoculated with different strains of the pathogen, scanned with different scanners and grown under different conditions. CONCLUSIONS SeptoSympto takes the same amount of time as a visual assessment to evaluate STB symptoms. However, unlike visual assessments, it allows for data to be stored and evaluated by experts and non-experts in a more accurate and unbiased manner. The methods used in SeptoSympto make it a transferable, highly accurate, computationally inexpensive, easy-to-use, and adaptable tool. This study demonstrates the potential of using deep learning to assess complex plant disease symptoms such as STB.
Collapse
Affiliation(s)
- Laura Mathieu
- PHIM Plant Health Institute, Univ Montpellier, INRAE, CIRAD, Institut Agro, IRD, Montpellier, France.
| | - Maxime Reder
- PHIM Plant Health Institute, Univ Montpellier, INRAE, CIRAD, Institut Agro, IRD, Montpellier, France
| | - Ali Siah
- BioEcoAgro, Junia, Lille University, Liège University, UPJV, Artois University, ULCO, INRAE, Lille, France
| | - Aurélie Ducasse
- PHIM Plant Health Institute, Univ Montpellier, INRAE, CIRAD, Institut Agro, IRD, Montpellier, France
| | | | | | - Jean-Benoît Morel
- PHIM Plant Health Institute, Univ Montpellier, INRAE, CIRAD, Institut Agro, IRD, Montpellier, France
| | | | - Elsa Ballini
- PHIM Plant Health Institute, Univ Montpellier, CIRAD, INRAE, IRD, Institut Agro, Montpellier, France.
| |
Collapse
|
22
|
Ritsche P, Franchi MV, Faude O, Finni T, Seynnes O, Cronin NJ. Fully Automated Analysis of Muscle Architecture from B-Mode Ultrasound Images with DL_Track_US. Ultrasound Med Biol 2024; 50:258-267. [PMID: 38007322 DOI: 10.1016/j.ultrasmedbio.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 10/25/2023] [Accepted: 10/26/2023] [Indexed: 11/27/2023]
Abstract
OBJECTIVE B-mode ultrasound can be used to image musculoskeletal tissues, but one major bottleneck is analyses of muscle architectural parameters (i.e., muscle thickness, pennation angle and fascicle length), which are most often performed manually. METHODS In this study we trained two different neural networks (classic U-Net and U-Net with VGG16 pre-trained encoder) to detect muscle fascicles and aponeuroses using a set of labeled musculoskeletal ultrasound images. We determined the best-performing model based on intersection over union and loss metrics. We then compared neural network predictions on an unseen test set with those obtained via manual analysis and two existing semi/automated analysis approaches (simple muscle architecture analysis [SMA] and UltraTrack). DL_Track_US detects the locations of the superficial and deep aponeuroses, as well as multiple fascicle fragments per image. RESULTS For single images, DL_Track_US yielded results similar to those produced by a non-trainable automated method (SMA; mean difference in fascicle length: 5.1 mm) and human manual analysis (mean difference: -2.4 mm). Between-method differences in pennation angle were within 1.5°, and mean differences in muscle thickness were less than 1 mm. Similarly, for videos, there was overlap between the results produced with UltraTrack and DL_Track_US, with intraclass correlations ranging between 0.19 and 0.88. CONCLUSION DL_Track_US is fully automated and open source and can estimate fascicle length, pennation angle and muscle thickness from single images or videos, as well as from multiple superficial muscles. We also provide a user interface and all necessary code and training data for custom model development.
Collapse
Affiliation(s)
- Paul Ritsche
- Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland.
| | - Martino V Franchi
- Department of Biomedical Sciences, University of Padova, Padova, Italy
| | - Oliver Faude
- Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
| | - Taija Finni
- Faculty of Sport and Health Sciences, University of Jyvaskyla, Jyvaskyla, Finland
| | - Olivier Seynnes
- Department for Physical Performance, Norwegian School of Sport Sciences, Oslo, Norway
| | - Neil J Cronin
- Faculty of Sport and Health Sciences, University of Jyvaskyla, Jyvaskyla, Finland; School of Sport & Exercise, University of Gloucestershire, Gloucester, UK
| |
Collapse
|
23
|
Chaoyang Z, Shibao S, Wenmao H, Pengcheng Z. FDR-TransUNet: A novel encoder-decoder architecture with vision transformer for improved medical image segmentation. Comput Biol Med 2024; 169:107858. [PMID: 38113680 DOI: 10.1016/j.compbiomed.2023.107858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 11/30/2023] [Accepted: 12/11/2023] [Indexed: 12/21/2023]
Abstract
The U-shaped and Transformer architectures have achieved exceptional performance in medical image segmentation and natural language processing, respectively. Their combination has also led to remarkable results but still suffers from enormous loss of image features during downsampling and the difficulty of recovering spatial information during upsampling. In this paper, we propose a novel encoder-decoder architecture for medical image segmentation, which has a flexibly adjustable hybrid encoder and two expanding paths decoder. The hybrid encoder incorporates the feature double reuse (FDR) block and the encoder of Vision Transformer (ViT), which can extract local and global pixel localization information, and alleviate image feature loss effectively. Meanwhile, we retain the original class-token sequence in the Vision Transformer and develop an additional corresponding expanding path. The class-token sequence and abstract image features are leveraged by two independent expanding paths with the deep-supervision strategy, which can better recover the image spatial information and accelerate model convergence. To further mitigate the feature loss and improve spatial information recovery, we introduce successive residual connections throughout the entire network. We evaluated our model on the COVID-19 lung segmentation and the infection area segmentation tasks. The mIoU index increased by 1.5 points and 3.9 points compared to other models which demonstrates a performance improvement.
Collapse
Affiliation(s)
- Zhang Chaoyang
- School of Information Engineering, HeNan University of Science and Technology, Luoyang, 471023, China
| | - Sun Shibao
- School of Information Engineering, HeNan University of Science and Technology, Luoyang, 471023, China.
| | - Hu Wenmao
- School of Information Engineering, HeNan University of Science and Technology, Luoyang, 471023, China
| | - Zhao Pengcheng
- School of Information Engineering, HeNan University of Science and Technology, Luoyang, 471023, China
| |
Collapse
|
24
|
R SSRM, T J. Multi-Scale and Spatial Information Extraction for Kidney Tumor Segmentation: A Contextual Deformable Attention and Edge-Enhanced U-Net. J Imaging Inform Med 2024; 37:151-166. [PMID: 38343255 DOI: 10.1007/s10278-023-00900-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 09/04/2023] [Accepted: 09/05/2023] [Indexed: 03/02/2024]
Abstract
Kidney tumor segmentation is a difficult task because of the complex spatial and volumetric information present in medical images. Recent advances in deep convolutional neural networks (DCNNs) have improved tumor segmentation accuracy. However, the practical usability of current CNN-based networks is constrained by their high computational complexity. Additionally, these techniques often struggle to make adaptive modifications based on the structure of the tumors, which can lead to blurred edges in segmentation results. A lightweight architecture called the contextual deformable attention and edge-enhanced U-Net (CDA2E-Net) for high-accuracy pixel-level kidney tumor segmentation is proposed to address these challenges. Rather than using complex deep encoders, the approach includes a lightweight depthwise dilated ShuffleNetV2 (LDS-Net) encoder integrated into the CDA2E-Net framework. The proposed method also contains a multiscale attention feature pyramid pooling (MAF2P) module that improves the ability of multiscale features to adapt to various tumor shapes. Finally, an edge-enhanced loss function is introduced to guide the CDA2E-Net to concentrate on tumor edge information. The CDA2E-Net is evaluated on the KiTS19 and KiTS21 datasets, and the results demonstrate its superiority over existing approaches in terms of Hausdorff distance (HD), intersection over union (IoU), and dice coefficient (DSC) metrics.
Collapse
Affiliation(s)
- Shamija Sherryl R M R
- Department of Electronics & Communication Engineering, Ponjesly College of Engineering, Nagercoil, Tamil Nadu, India.
| | - Jaya T
- Department of Electronics & Communication Engineering, Saveetha Engineering College, Thandalam, India
| |
Collapse
|
25
|
Vafaeikia P, Wagner MW, Hawkins C, Tabori U, Ertl-Wagner BB, Khalvati F. MRI-Based End-To-End Pediatric Low-Grade Glioma Segmentation and Classification. Can Assoc Radiol J 2024; 75:153-160. [PMID: 37401906 DOI: 10.1177/08465371231184780] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/05/2023] Open
Abstract
Purpose: MRI-based radiomics models can predict genetic markers in pediatric low-grade glioma (pLGG). These models usually require tumour segmentation, which is tedious and time consuming if done manually. We propose a deep learning (DL) model to automate tumour segmentation and build an end-to-end radiomics-based pipeline for pLGG classification. Methods: The proposed architecture is a 2-step U-Net based DL network. The first U-Net is trained on downsampled images to locate the tumour. The second U-Net is trained using image patches centred around the located tumour to produce more refined segmentations. The segmented tumour is then fed into a radiomics-based model to predict the genetic marker of the tumour. Results: Our segmentation model achieved a correlation value of over 80% for all volume-related radiomic features and an average Dice score of .795 in test cases. Feeding the auto-segmentation results into a radiomics model resulted in a mean area under the ROC curve (AUC) of .843, with 95% confidence interval (CI) [.78-.906] and .730, with 95% CI [.671-.789] on the test set for 2-class (BRAF V600E mutation BRAF fusion) and 3-class (BRAF V600E mutation BRAF fusion and Other) classification, respectively. This result was comparable to the AUC of .874, 95% CI [.829-.919] and .758, 95% CI [.724-.792] for the radiomics model trained and tested on the manual segmentations in 2-class and 3-class classification scenarios, respectively. Conclusion: The proposed end-to-end pipeline for pLGG segmentation and classification produced results comparable to manual segmentation when it was used for a radiomics-based genetic marker prediction model.
Collapse
Affiliation(s)
- Partoo Vafaeikia
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- The Hospital for Sick Children, Toronto, ON, Canada
| | | | - Cynthia Hawkins
- The Hospital for Sick Children, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Uri Tabori
- The Hospital for Sick Children, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Birgit B Ertl-Wagner
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- The Hospital for Sick Children, Toronto, ON, Canada
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Farzad Khalvati
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- The Hospital for Sick Children, Toronto, ON, Canada
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada
- Vector Institute, Toronto, ON, Canada
| |
Collapse
|
26
|
Dormagen N, Klein M, Schmitz AS, Thoma MH, Schwarz M. Multi-Particle Tracking in Complex Plasmas Using a Simplified and Compact U-Net. J Imaging 2024; 10:40. [PMID: 38392088 PMCID: PMC10890024 DOI: 10.3390/jimaging10020040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 01/12/2024] [Accepted: 01/15/2024] [Indexed: 02/24/2024] Open
Abstract
Detecting micron-sized particles is an essential task for the analysis of complex plasmas because a large part of the analysis is based on the initially detected positions of the particles. Accordingly, high accuracy in particle detection is desirable. Previous studies have shown that machine learning algorithms have made great progress and outperformed classical approaches. This work presents an approach for tracking micron-sized particles in a dense cloud of particles in a dusty plasma at Plasmakristall-Experiment 4 using a U-Net. The U-net is a convolutional network architecture for the fast and precise segmentation of images that was developed at the Computer Science Department of the University of Freiburg. The U-Net architecture, with its intricate design and skip connections, has been a powerhouse in achieving precise object delineation. However, as experiments are to be conducted in resource-constrained environments, such as parabolic flights, preferably with real-time applications, there is growing interest in exploring less complex U-net architectures that balance efficiency and effectiveness. We compare the full-size neural network, three optimized neural networks, the well-known StarDist and trackpy, in terms of accuracy in artificial data analysis. Finally, we determine which of the compact U-net architectures provides the best balance between efficiency and effectiveness. We also apply the full-size neural network and the the most effective compact network to the data of the PK-4 experiment. The experimental data were generated under laboratory conditions.
Collapse
Affiliation(s)
- Niklas Dormagen
- NanoP, TH Mittelhessen University of Applied Sciences, D 35392 Giessen, Germany
- I. Physikalisches Institut, Justus Liebig Universitat Giessen, D 35392 Giessen, Germany
| | - Max Klein
- NanoP, TH Mittelhessen University of Applied Sciences, D 35392 Giessen, Germany
- I. Physikalisches Institut, Justus Liebig Universitat Giessen, D 35392 Giessen, Germany
| | - Andreas S Schmitz
- I. Physikalisches Institut, Justus Liebig Universitat Giessen, D 35392 Giessen, Germany
| | - Markus H Thoma
- I. Physikalisches Institut, Justus Liebig Universitat Giessen, D 35392 Giessen, Germany
| | - Mike Schwarz
- NanoP, TH Mittelhessen University of Applied Sciences, D 35392 Giessen, Germany
| |
Collapse
|
27
|
Maruccio FC, Eppinga W, Laves MH, Navarro RF, Salvi M, Molinari F, Papaconstadopoulos P. Clinical assessment of deep learning-based uncertainty maps in lung cancer segmentation. Phys Med Biol 2024; 69:035007. [PMID: 38171012 DOI: 10.1088/1361-6560/ad1a26] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 01/03/2024] [Indexed: 01/05/2024]
Abstract
Objective. Prior to radiation therapy planning, accurate delineation of gross tumour volume (GTVs) and organs at risk (OARs) is crucial. In the current clinical practice, tumour delineation is performed manually by radiation oncologists, which is time-consuming and prone to large inter-observer variability. With the advent of deep learning (DL) models, automated contouring has become possible, speeding up procedures and assisting clinicians. However, these tools are currently used in the clinic mostly for contouring OARs, since these systems are not reliable yet for contouring GTVs. To improve the reliability of these systems, researchers have started exploring the topic of probabilistic neural networks. However, there is still limited knowledge of the practical implementation of such networks in real clinical settings.Approach. In this work, we developed a 3D probabilistic system that generates DL-based uncertainty maps for lung cancer CT segmentations. We employed the Monte Carlo (MC) dropout technique to generate probabilistic and uncertainty maps, while the model calibration was evaluated by using reliability diagrams. A clinical validation was conducted in collaboration with a radiation oncologist to qualitatively assess the value of the uncertainty estimates. We also proposed two novel metrics, namely mean uncertainty (MU) and relative uncertainty volume (RUV), as potential indicators for clinicians to assess the need for independent visual checks of the DL-based segmentation. Main results. Our study showed that uncertainty mapping effectively identified cases of under or over-contouring. Although the overconfidence of the model, a strong correlation was observed between the clinical opinion and MU metric. Moreover, both MU and RUV revealed high AUC values in discretising between low and high uncertainty cases.Significance. Our study is one of the first attempts to clinically validate uncertainty estimates in DL-based contouring. The two proposed metrics exhibited promising potential as indicators for clinicians to independently assess the quality of tumour delineation.
Collapse
Affiliation(s)
| | - Wietse Eppinga
- University Medical Centre Utrecht, Department of Radiotherapy, Heidelberglaan 100 Utrecht, 3584 CX, NL, The Netherlands
| | | | | | - Massimo Salvi
- Politecnico di Torino, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24 Torino, Piemonte, I-10129, IT, Italy
| | - Filippo Molinari
- Politecnico di Torino, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24 Torino, Piemonte, I-10129, IT, Italy
| | | |
Collapse
|
28
|
Aganj I, Mora J, Fischl B, Augustinack JC. Automatic Geometry-based Estimation of the Locus Coeruleus Region on T 1-Weighted Magnetic Resonance Images. bioRxiv 2024:2024.01.23.576958. [PMID: 38328208 PMCID: PMC10849695 DOI: 10.1101/2024.01.23.576958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
The locus coeruleus (LC) is a key brain structure implicated in cognitive function and neurodegenerative disease. Automatic segmentation of the LC is a crucial step in quantitative non-invasive analysis of the LC in large MRI cohorts. Most publicly available imaging databases for training automatic LC segmentation models take advantage of specialized contrast-enhancing (e.g., neuromelanin-sensitive) MRI. Segmentation models developed with such image contrasts, however, are not readily applicable to existing datasets with conventional MRI sequences. In this work, we evaluate the feasibility of using non-contrast neuroanatomical information to geometrically approximate the LC region from standard 3-Tesla T1-weighted images of 20 subjects from the Human Connectome Project (HCP). We employ this dataset to train and internally/externally evaluate two automatic localization methods, the Expected Label Value and the U-Net. We also test the hypothesis that using the phase image as input can improve the robustness of out-of-sample segmentation. We then apply our trained models to a larger subset of HCP, while exploratorily correlating LC imaging variables and structural connectivity with demographic and clinical data. This report contributes and provides an evaluation of two computational methods estimating neural structure.
Collapse
Affiliation(s)
- Iman Aganj
- Athinoula A. Martinos Center for Biomedical Imaging, Radiology Department, Massachusetts General Hospital, Boston, MA 02129, USA
- Radiology Department, Harvard Medical School, Boston, MA 02115, USA
| | - Jocelyn Mora
- Athinoula A. Martinos Center for Biomedical Imaging, Radiology Department, Massachusetts General Hospital, Boston, MA 02129, USA
| | - Bruce Fischl
- Athinoula A. Martinos Center for Biomedical Imaging, Radiology Department, Massachusetts General Hospital, Boston, MA 02129, USA
- Radiology Department, Harvard Medical School, Boston, MA 02115, USA
| | - Jean C. Augustinack
- Athinoula A. Martinos Center for Biomedical Imaging, Radiology Department, Massachusetts General Hospital, Boston, MA 02129, USA
- Radiology Department, Harvard Medical School, Boston, MA 02115, USA
| |
Collapse
|
29
|
Yawson AK, Walter A, Wolf N, Klüter S, Hoegen P, Adeberg S, Debus J, Frank M, Jäkel O, Giske K. Essential parameters needed for a U-Net-based segmentation of individual bones on planning CT images in the head and neck region using limited datasets for radiotherapy application. Phys Med Biol 2024; 69:035008. [PMID: 38164988 DOI: 10.1088/1361-6560/ad1996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 12/29/2023] [Indexed: 01/03/2024]
Abstract
Objective.The field of radiotherapy is highly marked by the lack of datasets even with the availability of public datasets. Our study uses a very limited dataset to provide insights on essential parameters needed to automatically and accurately segment individual bones on planning CT images of head and neck cancer patients.Approach.The study was conducted using 30 planning CT images of real patients acquired from 5 different cohorts. 15 cases from 4 cohorts were randomly selected as training and validation datasets while the remaining were used as test datasets. Four experimental sets were formulated to explore parameters such as background patch reduction, class-dependent augmentation and incorporation of a weight map on the loss function.Main results.Our best experimental scenario resulted in a mean Dice score of 0.93 ± 0.06 for other bones (skull, mandible, scapulae, clavicles, humeri and hyoid), 0.93 ± 0.02 for ribs and 0.88 ± 0.03 for vertebrae on 7 test cases from the same cohorts as the training datasets. We compared our proposed solution approach to a retrained nnU-Net and obtained comparable results for vertebral bones while outperforming in the correct identification of the left and right instances of ribs, scapulae, humeri and clavicles. Furthermore, we evaluated the generalization capability of our proposed model on a new cohort and the mean Dice score yielded 0.96 ± 0.10 for other bones, 0.95 ± 0.07 for ribs and 0.81 ± 0.19 for vertebrae on 8 test cases.Significance.With these insights, we are challenging the utilization of an automatic and accurate bone segmentation tool into the clinical routine of radiotherapy despite the limited training datasets.
Collapse
Affiliation(s)
- Ama Katseena Yawson
- German Cancer Research Center (DKFZ), Division of Medical Physics in Radiation Oncology, Heidelberg, Germany
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| | - Alexandra Walter
- German Cancer Research Center (DKFZ), Division of Medical Physics in Radiation Oncology, Heidelberg, Germany
- Karlsruhe Institute of Technology (KIT), Department of Mathematics, Karlsruhe, Germany
| | - Nora Wolf
- German Cancer Research Center (DKFZ), Division of Medical Physics in Radiation Oncology, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Sebastian Klüter
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
- University Hospital Heidelberg, Department of Radiation Oncology, Heidelberg, Germany
| | - Philip Hoegen
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
- University Hospital Heidelberg, Department of Radiation Oncology, Heidelberg, Germany
| | | | - Jürgen Debus
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
- University Hospital Heidelberg, Department of Radiation Oncology, Heidelberg, Germany
- Heidelberg Ion Therapy Center (HIT), Heidelberg, Germany
| | - Martin Frank
- Karlsruhe Institute of Technology (KIT), Department of Mathematics, Karlsruhe, Germany
| | - Oliver Jäkel
- German Cancer Research Center (DKFZ), Division of Medical Physics in Radiation Oncology, Heidelberg, Germany
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
- Heidelberg Ion Therapy Center (HIT), Heidelberg, Germany
| | - Kristina Giske
- German Cancer Research Center (DKFZ), Division of Medical Physics in Radiation Oncology, Heidelberg, Germany
- Heidelberg Institute for Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany
| |
Collapse
|
30
|
Jani VP, Ostovaneh M, Chamera E, Kato Y, Lima JAC, Ambale-Venkatesh B. Deep Learning for Automatic Volumetric Segmentation of Left Ventricular Myocardium and Ischemic Scar from Multi-Slice LGE-CMR. Eur Heart J Cardiovasc Imaging 2024:jeae022. [PMID: 38244222 DOI: 10.1093/ehjci/jeae022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 12/09/2023] [Accepted: 01/18/2024] [Indexed: 01/22/2024] Open
Abstract
PURPOSE This study details application of deep learning for automatic volumetric segmentation of left ventricular myocardium and scar and automated quantification of myocardial ischemic scar burden from late-gadolinium enhancement cardiovascular magnetic resonance (LGE-CMR). MATERIALS AND METHODS We included 501 images and manual segmentations of short-axis LGE-CMR from over 20 multinational sites, from which 377 studies were used for training and 124 studies from unique participants for internal validation. A third test set of 52 images was used for external evaluation. Three models, U-Net, Cascaded U-Net, and U-Net++, were trained with a novel adaptive weighted categorical cross entropy loss function. Model performance was evaluated using concordance correlation coefficients (CCC) for left ventricular (LV) mass and percent myocardial scar burden. RESULTS Cascaded U-Net was found to be the best model for quantification of LV mass and scar percentage. The model exhibited a mean difference of -5 ± 23 g for LV mass, -0.4 ± 11.2 g for scar mass, and -0.8 ± 7% for percent scar. CCC were 0.87, 0.77, and 0.78 for LV mass, scar mass, and percent scar burden, respectively, in the internal validation set and 0.75, 0.71, and 0.69, respectively, in the external test set. For segmental scar mass, CCC was 0.74 for apical scar, 0.91 for midventricular scar, and 0.73 for basal scar, demonstrating moderate to strong agreement. CONCLUSION We successfully trained a convolutional neural network for volumetric segmentation and analysis of left ventricular scar burden from LGE-CMR images in a large, multinational cohort of participants with ischemic scar.
Collapse
Affiliation(s)
- Vivek P Jani
- Division of Cardiology, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 524, Baltimore, MD 21297-0409, USA
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA
| | - Mohammad Ostovaneh
- Division of Cardiology, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 524, Baltimore, MD 21297-0409, USA
| | - Elzbieta Chamera
- Division of Cardiology, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 524, Baltimore, MD 21297-0409, USA
| | - Yoko Kato
- Division of Cardiology, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 524, Baltimore, MD 21297-0409, USA
| | - Joao A C Lima
- Division of Cardiology, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 524, Baltimore, MD 21297-0409, USA
| | | |
Collapse
|
31
|
Hackman A, Chen CH, Chen AWG, Chen MK. Automatic Segmentation of Membranous Glottal Gap Area with U-Net-Based Architecture. Laryngoscope 2024. [PMID: 38217455 DOI: 10.1002/lary.31266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 12/10/2023] [Accepted: 12/21/2023] [Indexed: 01/15/2024]
Abstract
BACKGROUND While videostroboscopy is recognized as the most popular approach for investigating vocal fold function, evaluating the numerical values, such as the membranous glottal gap area, remains too time consuming for clinical applications. METHODS We used a total of 2507 videostroboscopy images from 137 patients and developed five U-Net-based deep-learning image segmentation models for automatic masking of the membranous glottal gap area. To further validate the models, we used another 410 images from 41 different patients. RESULTS During development, all five models exhibited acceptable and similar metrics. While the VGG19 U-Net had a long inference time of 1654 ms, the other four models had more practical inference times, ranging from 16 to 138 ms. During further validation, Efficient U-Net demonstrated the highest intersection over union of 0.8455, the highest Dice coefficient of 0.9163, and the lowest Hausdorff distance of 1.5626. The normalized membranous glottal gap area index was also calculated and validated. Efficient U-Net and VGG19 U-Net exhibited the lowest mean squared errors (3.5476 and 3.3842) and the lowest mean absolute errors (1.8835 and 1.8396). CONCLUSIONS Automatic segmentation of the membranous glottal gap area can be achieved through U-net-based architecture. Considering the segmentation quality and speed, Efficient U-Net is a reasonable choice for this task, while the other four models remain valuable competitors. The models' masked area enables possible calculation of the normalized membranous glottal gap area and analysis of the glottal area waveform, revealing promising clinical applications for this model. LEVEL OF EVIDENCE NA Laryngoscope, 2024.
Collapse
Affiliation(s)
- Acquah Hackman
- Artificial Intelligence Development Center, Changhua Christian Hospital, Changhua, Taiwan
| | - Chih-Hua Chen
- Department of Otorhinolaryngology, Head and Neck Surgery, Changhua Christian Hospital, Changhua, Taiwan
| | - Andy Wei-Ge Chen
- Department of Otorhinolaryngology, Head and Neck Surgery, Changhua Christian Hospital, Changhua, Taiwan
- Doctoral Program in Translational Medicine, National Chung Hsing University, Taichung, Taiwan
- Rong Hsing Translational Medicine Research Center, National Chung Hsing University, Taichung, Taiwan
| | - Mu-Kuan Chen
- Department of Otorhinolaryngology, Head and Neck Surgery, Changhua Christian Hospital, Changhua, Taiwan
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan
| |
Collapse
|
32
|
Okumura N, Nishikawa T, Imafuku C, Matsuoka Y, Miyawaki Y, Kadowaki S, Nakahara M, Matsuoka Y, Koizumi N. U-Net Convolutional Neural Network for Real-Time Prediction of the Number of Cultured Corneal Endothelial Cells for Cellular Therapy. Bioengineering (Basel) 2024; 11:71. [PMID: 38247948 PMCID: PMC10813389 DOI: 10.3390/bioengineering11010071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Accepted: 01/08/2024] [Indexed: 01/23/2024] Open
Abstract
Corneal endothelial decompensation is treated by the corneal transplantation of donor corneas, but donor shortages and other problems associated with corneal transplantation have prompted investigations into tissue engineering therapies. For clinical use, cells used in tissue engineering must undergo strict quality control to ensure their safety and efficacy. In addition, efficient cell manufacturing processes are needed to make cell therapy a sustainable standard procedure with an acceptable economic burden. In this study, we obtained 3098 phase contrast images of cultured human corneal endothelial cells (HCECs). We labeled the images using semi-supervised learning and then trained a model that predicted the cell centers with a precision of 95.1%, a recall of 92.3%, and an F-value of 93.4%. The cell density calculated by the model showed a very strong correlation with the ground truth (Pearson's correlation coefficient = 0.97, p value = 8.10 × 10-52). The total cell numbers calculated by our model based on phase contrast images were close to the numbers calculated using a hemocytometer through passages 1 to 4. Our findings confirm the feasibility of using artificial intelligence-assisted quality control assessments in the field of regenerative medicine.
Collapse
Affiliation(s)
- Naoki Okumura
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Takeru Nishikawa
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Chiaki Imafuku
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Yuki Matsuoka
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Yuna Miyawaki
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Shinichi Kadowaki
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| | - Makiko Nakahara
- ActualEyes Inc., D-egg, 1 Jizodani, Koudo, Kyotanabe-City 610-0332, Kyoto, Japan; (M.N.); (Y.M.)
| | - Yasushi Matsuoka
- ActualEyes Inc., D-egg, 1 Jizodani, Koudo, Kyotanabe-City 610-0332, Kyoto, Japan; (M.N.); (Y.M.)
| | - Noriko Koizumi
- Department of Biomedical Engineering, Faculty of Life and Medical Sciences, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe-City 610-0394, Kyoto, Japan; (T.N.); (Y.M.); (Y.M.); (S.K.); (N.K.)
| |
Collapse
|
33
|
Ando S, Loh PY. Convolutional Neural Network Approaches in Median Nerve Morphological Assessment from Ultrasound Images. J Imaging 2024; 10:13. [PMID: 38248998 PMCID: PMC10817571 DOI: 10.3390/jimaging10010013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 12/27/2023] [Accepted: 12/29/2023] [Indexed: 01/23/2024] Open
Abstract
Ultrasound imaging has been used to investigate compression of the median nerve in carpal tunnel syndrome patients. Ultrasound imaging and the extraction of median nerve parameters from ultrasound images are crucial and are usually performed manually by experts. The manual annotation of ultrasound images relies on experience, and intra- and interrater reliability may vary among studies. In this study, two types of convolutional neural networks (CNNs), U-Net and SegNet, were used to extract the median nerve morphology. To the best of our knowledge, the application of these methods to ultrasound imaging of the median nerve has not yet been investigated. Spearman's correlation and Bland-Altman analyses were performed to investigate the correlation and agreement between manual annotation and CNN estimation, namely, the cross-sectional area, circumference, and diameter of the median nerve. The results showed that the intersection over union (IoU) of U-Net (0.717) was greater than that of SegNet (0.625). A few images in SegNet had an IoU below 0.6, decreasing the average IoU. In both models, the IoU decreased when the median nerve was elongated longitudinally with a blurred outline. The Bland-Altman analysis revealed that, in general, both the U-Net- and SegNet-estimated measurements showed 95% limits of agreement with manual annotation. These results show that these CNN models are promising tools for median nerve ultrasound imaging analysis.
Collapse
Affiliation(s)
- Shion Ando
- Department of Mechanical Engineering, Faculty of Engineering, Kyushu University, Fukuoka 819-0395, Japan;
| | - Ping Yeap Loh
- Department of Human Life Design and Science, Faculty of Design, Kyushu University, Fukuoka 819-0395, Japan
| |
Collapse
|
34
|
Zhang X, Su T, Zhang Y, Cui H, Tan Y, Zhu J, Xia D, Zheng H, Liang D, Ge Y. Transferring U-Net between low-dose CT denoising tasks: a validation study with varied spatial resolutions. Quant Imaging Med Surg 2024; 14:640-652. [PMID: 38223035 PMCID: PMC10784075 DOI: 10.21037/qims-23-768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Accepted: 11/09/2023] [Indexed: 01/16/2024]
Abstract
Background Recently, deep learning techniques have been widely used in low-dose computed tomography (LDCT) imaging applications for quickly generating high quality computed tomography (CT) images at lower radiation dose levels. The purpose of this study is to validate the reproducibility of the denoising performance of a given network that has been trained in advance across varied LDCT image datasets that are acquired from different imaging systems with different spatial resolutions. Methods Specifically, LDCT images with comparable noise levels but having different spatial resolutions were prepared to train the U-Net. The number of CT images used for the network training, validation and test was 2,400, 300 and 300, respectively. Afterwards, self- and cross-validations among six selected spatial resolutions (62.5, 125, 250, 375, 500, 625 µm) were studied and compared side by side. The residual variance, peak signal to noise ratio (PSNR), normalized root mean square error (NRMSE) and structural similarity (SSIM) were measured and compared. In addition, network retraining on a small number of image set was performed to fine tune the performance of transfer learning among LDCT tasks with varied spatial resolutions. Results Results demonstrated that the U-Net trained upon LDCT images having a certain spatial resolution can effectively reduce the noise of the other LDCT images having different spatial resolutions. Regardless, results showed that image artifacts would be generated during the above cross validations. For instance, noticeable residual artifacts were presented at the margin and central areas of the object as the resolution inconsistency increased. The retraining results showed that the artifacts caused by the resolution mismatch can be greatly reduced by utilizing about only 20% of the original training data size. This quantitative improvement led to a reduction in the NRMSE from 0.1898 to 0.1263 and an increase in the SSIM from 0.7558 to 0.8036. Conclusions In conclusion, artifacts would be generated when transferring the U-Net to a LDCT denoising task with different spatial resolution. To maintain the denoising performance, it is recommended to retrain the U-Net with a small amount of datasets having the same target spatial resolution.
Collapse
Affiliation(s)
- Xin Zhang
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Ting Su
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yunxin Zhang
- Department of Vascular Surgery, Beijing Jishuitan Hospital, Beijing, China
| | - Han Cui
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yuhang Tan
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Jiongtao Zhu
- College of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, China
| | - Dongmei Xia
- Key Laboratory of Low-grade Energy Utilization Technologies and Systems of Ministry of Education of China, College of Power Engineering, Chongqing University, Chongqing, China
| | - Hairong Zheng
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Dong Liang
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yongshuai Ge
- Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| |
Collapse
|
35
|
S K, Banerjee T, Priya D, Vimal Cruz M. Segmentation of Thoracic Organs through Distributed Extraction of Visual Feature Patterns Utilizing Resio-Inception U-Net and Deep Cluster Recognition Techniques. Curr Gene Ther 2024; 24:CGT-EPUB-136910. [PMID: 38310458 DOI: 10.2174/0115665232262165231201113932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 09/21/2023] [Accepted: 09/25/2023] [Indexed: 02/05/2024]
Abstract
Segmentation of medical images plays a key role in the correct identification and management of different diseases. In this study, we present a new segmentation method that meets the difficulties posed by sophisticated organ shapes in computed tomography (CT) images, particularly targeting lung, breast, and gastric cancers. Our suggested methods, Resio-Inception U-Net and Deep Cluster Recognition (RIUDCR), use a Residual Inception Architecture, which combines the power of residual connections and inception blocks to achieve cutting-edge segmentation performance while reducing the risk of overfitting. We present mathematical equations and functions that describe the design, including the encoding and decoding steps within the UC-Net system. Furthermore, we provide strong testing results that show the effectiveness of our method. Through thorough testing on varied datasets, our method regularly beats current techniques, achieving amazing precision and stability in organ task segmentation. These results show the promise of our residual inception architecture in better medical picture analysis. In summary, our research not only shows a state-of-the-art segment methodology but also reinforces its usefulness through thorough testing. The inclusion of residual inception architecture in medical picture segmentation offers good possibilities for improving the identification and management of disease planning.
Collapse
Affiliation(s)
- Karthikeyan S
- Computer Science and Engineering, KPR Institute of Engineering and Technology, Coimbatore, Tamilnadu, India
| | | | - Devi Priya
- Computer Science and Engineering, KPR Institute of Engineering and Technology, Coimbatore, Tamilnadu, India
| | | |
Collapse
|
36
|
Lee JD, Tsai CM. Advancing Barrett's Esophagus Segmentation: A Deep-Learning Ensemble Approach with Data Augmentation and Model Collaboration. Bioengineering (Basel) 2024; 11:47. [PMID: 38247924 PMCID: PMC10813459 DOI: 10.3390/bioengineering11010047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 11/15/2023] [Accepted: 11/15/2023] [Indexed: 01/23/2024] Open
Abstract
This approach provides a thorough investigation of Barrett's esophagus segmentation using deep-learning methods. This study explores various U-Net model variants with different backbone architectures, focusing on how the choice of backbone influences segmentation accuracy. By employing rigorous data augmentation techniques and ensemble strategies, the goal is to achieve precise and robust segmentation results. Key findings include the superiority of DenseNet backbones, the importance of tailored data augmentation, and the adaptability of training U-Net models from scratch. Ensemble methods are shown to enhance segmentation accuracy, and a grid search is used to fine-tune ensemble weights. A comprehensive comparison with the popular Deeplabv3+ architecture emphasizes the role of dataset characteristics. Insights into training saturation help optimize resource utilization, and efficient ensembles consistently achieve high mean intersection over union (IoU) scores, approaching 0.94. This research marks a significant advancement in Barrett's esophagus segmentation.
Collapse
Affiliation(s)
- Jiann-Der Lee
- Department of Electrical Engineering, Chang Gung University, Taoyuan 33302, Taiwan
- Department of Neurosurgery, Chang Gung Memorial Hospital at Linkou, Taoyuan 33305, Taiwan
- Department of Electrical Engineering, Ming Chi University of Technology, New Taipei City 24330, Taiwan
| | - Chih Mao Tsai
- Department of Electrical Engineering, Chang Gung University, Taoyuan 33302, Taiwan
| |
Collapse
|
37
|
Lin SY, Lin CL. Brain tumor segmentation using U-Net in conjunction with EfficientNet. PeerJ Comput Sci 2024; 10:e1754. [PMID: 38196955 PMCID: PMC10773611 DOI: 10.7717/peerj-cs.1754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 11/23/2023] [Indexed: 01/11/2024]
Abstract
According to the Ten Leading Causes of Death Statistics Report by the Ministry of Health and Welfare in 2021, cancer ranks as the leading cause of mortality. Among them, pleomorphic glioblastoma is a common type of brain cancer. Brain cancer often occurs in the brain with unclear boundaries from normal brain tissue, necessitating assistance from experienced doctors to distinguish brain tumors before surgical resection to avoid damaging critical neural structures. In recent years, with the advancement of deep learning (DL) technology, artificial intelligence (AI) plays a vital role in disease diagnosis, especially in the field of image segmentation. This technology can aid doctors in locating and measuring brain tumors, while significantly reducing manpower and time costs. Currently, U-Net is one of the primary image segmentation techniques. It utilizes skip connections to combine high-level and low-level feature information, leading to significant improvements in segmentation accuracy. To further enhance the model's performance, this study explores the feasibility of using EfficientNetV2 as an encoder in combination with U-net. Experimental results indicate that employing EfficientNetV2 as an encoder together with U-net can improve the segmentation model's Dice score (loss = 0.0866, accuracy = 0.9977, and Dice similarity coefficient (DSC) = 0.9133).
Collapse
Affiliation(s)
- Shu-You Lin
- Department of Electrical Engineering, Ming Chi University of Technology, New Taipei City, Taiwan
| | - Chun-Ling Lin
- Department of Electrical Engineering, Ming Chi University of Technology, New Taipei City, Taiwan
| |
Collapse
|
38
|
Jansen P, Arrastia JL, Baguer DO, Schmidt M, Landsberg J, Wenzel J, Emberger M, Schadendorf D, Hadaschik E, Maass P, Griewank KG. Deep learning based histological classification of adnex tumors. Eur J Cancer 2024; 196:113431. [PMID: 37980855 DOI: 10.1016/j.ejca.2023.113431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 10/26/2023] [Accepted: 10/28/2023] [Indexed: 11/21/2023]
Abstract
BACKGROUND Cutaneous adnexal tumors are a diverse group of tumors arising from structures of the hair appendages. Although often benign, malignant entities occur which can metastasize and lead to patients´ death. Correct diagnosis is critical to ensure optimal treatment and best possible patient outcome. Artificial intelligence (AI) in the form of deep neural networks has recently shown enormous potential in the field of medicine including pathology, where we and others have found common cutaneous tumors can be detected with high sensitivity and specificity. To become a widely applied tool, AI approaches will also need to reliably detect and distinguish less common tumor entities including the diverse group of cutaneous adnexal tumors. METHODS To assess the potential of AI to recognize cutaneous adnexal tumors, we selected a diverse set of these entities from five German centers. The algorithm was trained with samples from four centers and then tested on slides from the fifth center. RESULTS The neural network was able to differentiate 14 different cutaneous adnexal tumors and distinguish them from more common cutaneous tumors (i.e. basal cell carcinoma and seborrheic keratosis). The total accuracy on the test set for classifying 248 samples into these 16 diagnoses was 89.92 %. Our findings support AI can distinguish rare tumors, for morphologically distinct entities even with very limited case numbers (< 50) for training. CONCLUSION This study further underlines the enormous potential of AI in pathology which could become a standard tool to aid pathologists in routine diagnostics in the foreseeable future. The final diagnostic responsibility will remain with the pathologist.
Collapse
Affiliation(s)
- Philipp Jansen
- Department of Dermatology, University Hospital Bonn, Bonn 53127, Germany
| | | | - Daniel Otero Baguer
- Center for Industrial Mathematics, University of Bremen, Bremen 28359, Germany
| | - Maximilian Schmidt
- Center for Industrial Mathematics, University of Bremen, Bremen 28359, Germany
| | - Jennifer Landsberg
- Department of Dermatology, University Hospital Bonn, Bonn 53127, Germany
| | - Jörg Wenzel
- Department of Dermatology, University Hospital Bonn, Bonn 53127, Germany
| | - Michael Emberger
- Patholab - Labor für Pathologie Salzburg, Salzburg 5020, Austria
| | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, University Duisburg-Essen, Essen 45147, Germany
| | - Eva Hadaschik
- Department of Dermatology, University Hospital Essen, University Duisburg-Essen, Essen 45147, Germany
| | - Peter Maass
- Center for Industrial Mathematics, University of Bremen, Bremen 28359, Germany
| | - Klaus Georg Griewank
- Department of Dermatology, University Hospital Essen, University Duisburg-Essen, Essen 45147, Germany; Dermatopathologie bei Mainz, Nieder-Olm, 55268, Germany.
| |
Collapse
|
39
|
Shi XP, Lin SY, Yang ML, Huang CC, Lee JC. Image deblurring by multi-scale modified U-Net using dilated convolution. Sci Prog 2024; 107:368504241231161. [PMID: 38400510 PMCID: PMC10894552 DOI: 10.1177/00368504241231161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2024]
Abstract
In modern urban traffic systems, intersection monitoring systems are used to monitor traffic flows and track vehicles by recognizing license plates. However, intersection monitors often produce motion-blurred images because of the rapid movement of cars. If a deep learning network is used for image deblurring, the blurring of the image can be eliminated first, and then the complete vehicle information can be obtained to improve the recognition rate. To restore a dynamic blurred image to a sharp image, this paper proposes a multi-scale modified U-Net image deblurring network using dilated convolution and employs a variable scaling iterative strategy to make the scheme more adaptable to actual blurred images. Multi-scale architecture uses scale changes to learn the characteristics of different scales of images, and the use of dilated convolution can improve the advantages of the receptive field and obtain more information from features without increasing the computational cost. Experimental results are obtained using a synthetic motion-blurred image dataset and a real blurred image dataset for comparison with existing deblurring methods. The experimental results demonstrate that the image deblurring method proposed in this paper has a favorable effect on actual motion-blurred images.
Collapse
Affiliation(s)
- Xiao-Pei Shi
- School of Foreign Studies, Shaoguan University, Guangdong, China
| | - Song-Yih Lin
- Department of Aircraft Maintenance, Far East University, Tainan, Taiwan
| | - Min-Lang Yang
- Department of Civil Engineering and Geomatics, Cheng Shiu University, Kaohsiung, Taiwan
| | - Chung-Chi Huang
- Department of Electrical Engineering, Far East University, Tainan, Taiwan
| | - Jen-Chun Lee
- Department of Telecommunication Engineering, National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan
| |
Collapse
|
40
|
Zhou D, Xu L, Wang T, Wei S, Gao F, Lai X, Cao J. M-DDC: MRI based demyelinative diseases classification with U-Net segmentation and convolutional network. Neural Netw 2024; 169:108-119. [PMID: 37890361 DOI: 10.1016/j.neunet.2023.10.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Revised: 09/03/2023] [Accepted: 10/09/2023] [Indexed: 10/29/2023]
Abstract
Childhood demyelinative diseases classification (DDC) with brain magnetic resonance imaging (MRI) is crucial to clinical diagnosis. But few attentions have been paid to DDC in the past. How to accurately differentiate pediatric-onset neuromyelitis optica spectrum disorder (NMOSD) from acute disseminated encephalomyelitis (ADEM) based on MRI is challenging in DDC. In this paper, a novel architecture M-DDC based on joint U-Net segmentation network and deep convolutional network is developed. The U-Net segmentation can provide pixel-level structure information, that helps the lesion areas location and size estimation. The classification branch in DDC can detect the regions of interest inside MRIs, including the white matter regions where lesions appear. The performance of the proposed method is evaluated on MRIs of 201 subjects recorded from the Children's Hospital of Zhejiang University School of Medicine. The comparisons show that the proposed DDC achieves the highest accuracy of 99.19% and dice of 71.1% for ADEM and NMOSD classification and segmentation, respectively.
Collapse
Affiliation(s)
- Deyang Zhou
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, Hangzhou Dianzi University, 310018, China; Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang, 310018, China; HDU-ITMO Joint Institute, Hangzhou Dianzi University, Zhejiang, 310018, China.
| | - Lu Xu
- Department of Neurology, Children's Hospital, Zhejiang University School of Medicine, 310018, China.
| | - Tianlei Wang
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, Hangzhou Dianzi University, 310018, China; Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang, 310018, China.
| | - Shaonong Wei
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, Hangzhou Dianzi University, 310018, China; Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang, 310018, China; HDU-ITMO Joint Institute, Hangzhou Dianzi University, Zhejiang, 310018, China.
| | - Feng Gao
- Department of Neurology, Children's Hospital, Zhejiang University School of Medicine, 310018, China.
| | - Xiaoping Lai
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, Hangzhou Dianzi University, 310018, China; Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang, 310018, China.
| | - Jiuwen Cao
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, Hangzhou Dianzi University, 310018, China; Artificial Intelligence Institute, Hangzhou Dianzi University, Zhejiang, 310018, China.
| |
Collapse
|
41
|
Verboom SD, Caballo M, Peters J, Gommers J, van den Oever D, Broeders MJM, Teuwen J, Sechopoulos I. Deep learning-based breast region segmentation in raw and processed digital mammograms: generalization across views and vendors. J Med Imaging (Bellingham) 2024; 11:014001. [PMID: 38162417 PMCID: PMC10753125 DOI: 10.1117/1.jmi.11.1.014001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 11/25/2023] [Accepted: 12/11/2023] [Indexed: 01/03/2024] Open
Abstract
Purpose We developed a segmentation method suited for both raw (for processing) and processed (for presentation) digital mammograms (DMs) that is designed to generalize across images acquired with systems from different vendors and across the two standard screening views. Approach A U-Net was trained to segment mammograms into background, breast, and pectoral muscle. Eight different datasets, including two previously published public sets and six sets of DMs from as many different vendors, were used, totaling 322 screen film mammograms (SFMs) and 4251 DMs (2821 raw/processed pairs and 1430 only processed) from 1077 different women. Three experiments were done: first training on all SFM and processed images, second also including all raw images in training, and finally testing vendor generalization by leaving one dataset out at a time. Results The model trained on SFM and processed mammograms achieved a good overall performance regardless of projection and vendor, with a mean (±std. dev.) dice score of 0.96 ± 0.06 for all datasets combined. When raw images were included in training, the mean (±std. dev.) dice score for the raw images was 0.95 ± 0.05 and for the processed images was 0.96 ± 0.04 . Testing on a dataset with processed DMs from a vendor that was excluded from training resulted in a difference in mean dice varying between - 0.23 to + 0.02 from that of the fully trained model. Conclusions The proposed segmentation method yields accurate overall segmentation results for both raw and processed mammograms independent of view and vendor. The code and model weights are made available.
Collapse
Affiliation(s)
- Sarah D. Verboom
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
| | - Marco Caballo
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
| | - Jim Peters
- Radboud University Medical Center, Department for Health Evidence, Nijmegen, The Netherlands
| | - Jessie Gommers
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
| | - Daan van den Oever
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
| | - Mireille J. M. Broeders
- Radboud University Medical Center, Department for Health Evidence, Nijmegen, The Netherlands
- Dutch Expert Centre for Screening (LRCB), Nijmegen, The Netherlands
| | - Jonas Teuwen
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
- Netherlands Cancer Institute, Department of Radiation Oncology, Amsterdam, The Netherlands
- Memorial Sloan Kettering Cancer Center, Department of Radiology, New York, New York, United States
| | - Ioannis Sechopoulos
- Radboud University Medical Center, Department of Medical Imaging, Nijmegen, The Netherlands
- Dutch Expert Centre for Screening (LRCB), Nijmegen, The Netherlands
- University of Twente, Multi-Modality Medical Imaging, Enschede, The Netherlands
| |
Collapse
|
42
|
Bery S, Brown-Brandl TM, Jones BT, Rohrer GA, Sharma SR. Determining the Presence and Size of Shoulder Lesions in Sows Using Computer Vision. Animals (Basel) 2023; 14:131. [PMID: 38200862 PMCID: PMC10777999 DOI: 10.3390/ani14010131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 12/23/2023] [Accepted: 12/26/2023] [Indexed: 01/12/2024] Open
Abstract
Shoulder sores predominantly arise in breeding sows and often result in untimely culling. Reported prevalence rates vary significantly, spanning between 5% and 50% depending upon the type of crate flooring inside a farm, the animal's body condition, or an existing injury that causes lameness. These lesions represent not only a welfare concern but also have an economic impact due to the labor needed for treatment and medication. The objective of this study was to evaluate the use of computer vision techniques in detecting and determining the size of shoulder lesions. A Microsoft Kinect V2 camera captured the top-down depth and RGB images of sows in farrowing crates. The RGB images were collected at a resolution of 1920 × 1080. To ensure the best view of the lesions, images were selected with sows lying on their right and left sides with all legs extended. A total of 824 RGB images from 70 sows with lesions at various stages of development were identified and annotated. Three deep learning-based object detection models, YOLOv5, YOLOv8, and Faster-RCNN, pre-trained with the COCO and ImageNet datasets, were implemented to localize the lesion area. YOLOv5 was the best predictor as it was able to detect lesions with an mAP@0.5 of 0.92. To estimate the lesion area, lesion pixel segmentation was carried out on the localized region using traditional image processing techniques like Otsu's binarization and adaptive thresholding alongside DL-based segmentation models based on U-Net architecture. In conclusion, this study demonstrates the potential of computer vision techniques in effectively detecting and assessing the size of shoulder lesions in breeding sows, providing a promising avenue for improving sow welfare and reducing economic losses.
Collapse
Affiliation(s)
- Shubham Bery
- Department of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, NE 68583, USA; (S.B.); (S.R.S.)
| | - Tami M. Brown-Brandl
- Department of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, NE 68583, USA; (S.B.); (S.R.S.)
| | - Bradley T. Jones
- Genetics and Breeding Research Unit, USDA-ARS U.S. Meat Animal Research Center, Clay Center, NE 68933, USA; (B.T.J.); (G.A.R.)
| | - Gary A. Rohrer
- Genetics and Breeding Research Unit, USDA-ARS U.S. Meat Animal Research Center, Clay Center, NE 68933, USA; (B.T.J.); (G.A.R.)
| | - Sudhendu Raj Sharma
- Department of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, NE 68583, USA; (S.B.); (S.R.S.)
| |
Collapse
|
43
|
Selvaraj J, Umapathy S. CRP U-NET: a deep learning model based semantic segmentation for the detection of colorectal polyp in lower gastrointestinal tract. Biomed Phys Eng Express 2023; 10:015018. [PMID: 38100789 DOI: 10.1088/2057-1976/ad160f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Accepted: 12/15/2023] [Indexed: 12/17/2023]
Abstract
Purpose. The objectives of the proposed work are twofold. Firstly, to develop a specialized light weight CRPU-Net for the segmentation of polyps in colonoscopy images. Secondly, to conduct a comparative analysis of the performance of CRPU-Net with implemented state-of-the-art models.Methods. We have utilized two distinct colonoscopy image datasets such as CVC-ColonDB and CVC-ClinicDB. This paper introduces the CRPU-Net, a novel approach for the automated segmentation of polyps in colorectal regions. A comprehensive series of experiments was conducted using the CRPU-Net, and its performance was compared with that of state-of-the-art models such as VGG16, VGG19, U-Net and ResUnet++. Additional analysis such as ablation study, generalizability test and 5-fold cross validation were performed.Results. The CRPU-Net achieved the segmentation accuracy of 96.42% compared to state-of-the-art model like ResUnet++ (90.91%). The Jaccard coefficient of 93.96% and Dice coefficient of 95.77% was obtained by comparing the segmentation performance of the CRPU-Net with ground truth.Conclusion. The CRPU-Net exhibits outstanding performance in Segmentation of polyp and holds promise for integration into colonoscopy devices enabling efficient operation.
Collapse
Affiliation(s)
- Jothiraj Selvaraj
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - Snekhalatha Umapathy
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| |
Collapse
|
44
|
Tong MW, Tolpadi AA, Bhattacharjee R, Han M, Majumdar S, Pedoia V. Synthetic Knee MRI T 1p Maps as an Avenue for Clinical Translation of Quantitative Osteoarthritis Biomarkers. Bioengineering (Basel) 2023; 11:17. [PMID: 38247894 PMCID: PMC10812962 DOI: 10.3390/bioengineering11010017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 12/15/2023] [Accepted: 12/21/2023] [Indexed: 01/23/2024] Open
Abstract
A 2D U-Net was trained to generate synthetic T1p maps from T2 maps for knee MRI to explore the feasibility of domain adaptation for enriching existing datasets and enabling rapid, reliable image reconstruction. The network was developed using 509 healthy contralateral and injured ipsilateral knee images from patients with ACL injuries and reconstruction surgeries acquired across three institutions. Network generalizability was evaluated on 343 knees acquired in a clinical setting and 46 knees from simultaneous bilateral acquisition in a research setting. The deep neural network synthesized high-fidelity reconstructions of T1p maps, preserving textures and local T1p elevation patterns in cartilage with a normalized mean square error of 2.4% and Pearson's correlation coefficient of 0.93. Analysis of reconstructed T1p maps within cartilage compartments revealed minimal bias (-0.10 ms), tight limits of agreement, and quantification error (5.7%) below the threshold for clinically significant change (6.42%) associated with osteoarthritis. In an out-of-distribution external test set, synthetic maps preserved T1p textures, but exhibited increased bias and wider limits of agreement. This study demonstrates the capability of image synthesis to reduce acquisition time, derive meaningful information from existing datasets, and suggest a pathway for standardizing T1p as a quantitative biomarker for osteoarthritis.
Collapse
Affiliation(s)
- Michelle W. Tong
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, USA
| | - Aniket A. Tolpadi
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
- Department of Bioengineering, University of California Berkeley, Berkeley, CA 94720, USA
| | - Rupsa Bhattacharjee
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
| | - Misung Han
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
| | - Sharmila Majumdar
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
| | - Valentina Pedoia
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA 94143, USA (S.M.); (V.P.)
| |
Collapse
|
45
|
Daneshgar Rahbar M, Mousavi Mojab SZ. Enhanced U-Net with GridMask (EUGNet): A Novel Approach for Robotic Surgical Tool Segmentation. J Imaging 2023; 9:282. [PMID: 38132700 PMCID: PMC10744415 DOI: 10.3390/jimaging9120282] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2023] [Revised: 12/13/2023] [Accepted: 12/15/2023] [Indexed: 12/23/2023] Open
Abstract
This study proposed enhanced U-Net with GridMask (EUGNet) image augmentation techniques focused on pixel manipulation, emphasizing GridMask augmentation. This study introduces EUGNet, which incorporates GridMask augmentation to address U-Net's limitations. EUGNet features a deep contextual encoder, residual connections, class-balancing loss, adaptive feature fusion, GridMask augmentation module, efficient implementation, and multi-modal fusion. These innovations enhance segmentation accuracy and robustness, making it well-suited for medical image analysis. The GridMask algorithm is detailed, demonstrating its distinct approach to pixel elimination, enhancing model adaptability to occlusions and local features. A comprehensive dataset of robotic surgical scenarios and instruments is used for evaluation, showcasing the framework's robustness. Specifically, there are improvements of 1.6 percentage points in balanced accuracy for the foreground, 1.7 points in intersection over union (IoU), and 1.7 points in mean Dice similarity coefficient (DSC). These improvements are highly significant and have a substantial impact on inference speed. The inference speed, which is a critical factor in real-time applications, has seen a noteworthy reduction. It decreased from 0.163 milliseconds for the U-Net without GridMask to 0.097 milliseconds for the U-Net with GridMask.
Collapse
Affiliation(s)
- Mostafa Daneshgar Rahbar
- Department of Electrical and Computer Engineering, Lawrence Technological University, Southfield, MI 48075, USA
| | | |
Collapse
|
46
|
Yang Y, Yue S, Quan H. CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation. Network 2023:1-20. [PMID: 38050997 DOI: 10.1080/0954898x.2023.2288858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 11/23/2023] [Indexed: 12/07/2023]
Abstract
Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.
Collapse
Affiliation(s)
- Ying Yang
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Shengbin Yue
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
- Yunnan Provincial Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Haiyan Quan
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
| |
Collapse
|
47
|
Wang H, Cao P, Yang J, Zaiane O. MCA-UNet: multi-scale cross co-attentional U-Net for automatic medical image segmentation. Health Inf Sci Syst 2023; 11:10. [PMID: 36721640 PMCID: PMC9884736 DOI: 10.1007/s13755-022-00209-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Accepted: 10/01/2022] [Indexed: 01/31/2023] Open
Abstract
Medical image segmentation is a challenging task due to the high variation in shape, size and position of infections or lesions in medical images. It is necessary to construct multi-scale representations to capture image contents from different scales. However, it is still challenging for U-Net with a simple skip connection to model the global multi-scale context. To overcome it, we proposed a dense skip-connection with cross co-attention in U-Net to solve the semantic gaps for an accurate automatic medical image segmentation. We name our method MCA-UNet, which enjoys two benefits: (1) it has a strong ability to model the multi-scale features, and (2) it jointly explores the spatial and channel attentions. The experimental results on the COVID-19 and IDRiD datasets suggest that our MCA-UNet produces more precise segmentation performance for the consolidation, ground-glass opacity (GGO), microaneurysms (MA) and hard exudates (EX). The source code of this work will be released via https://github.com/McGregorWwww/MCA-UNet/.
Collapse
Affiliation(s)
- Haonan Wang
- Computer Science and Engineering, Northeastern University, Shenyang, China
- Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China
| | - Peng Cao
- Computer Science and Engineering, Northeastern University, Shenyang, China
- Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China
| | - Jinzhu Yang
- Computer Science and Engineering, Northeastern University, Shenyang, China
- Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China
| | - Osmar Zaiane
- Amii, University of Alberta, Edmonton, AB Canada
| |
Collapse
|
48
|
Lee S, Rieu Z, Kim RE, Lee M, Yen K, Yong J, Kim D. Automatic segmentation of white matter hyperintensities in T2-FLAIR with AQUA: A comparative validation study against conventional methods. Brain Res Bull 2023; 205:110825. [PMID: 38000477 DOI: 10.1016/j.brainresbull.2023.110825] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 11/05/2023] [Accepted: 11/21/2023] [Indexed: 11/26/2023]
Abstract
White matter hyperintensities (WMHs) are lesions in the white matter of the brain that are associated with cognitive decline and an increased risk of dementia. The manual segmentation of WMHs is highly time-consuming and prone to intra- and inter-variability. Therefore, automatic segmentation approaches are gaining attention as a more efficient and objective means to detect and monitor WMHs. In this study, we propose AQUA, a deep learning model designed for fully automatic segmentation of WMHs from T2-FLAIR scans, which improves upon our previous study for small lesion detection and incorporating a multicenter approach. AQUA implements a two-dimensional U-Net architecture and uses patch-based training. Additionally, the network was modified to include Bottleneck Attention Module on each convolutional block of both the encoder and decoder to enhance performance for small-sized WMH. We evaluated the performance and robustness of AQUA by comparing it with five well-known supervised and unsupervised methods for automatic segmentation of WMHs (LGA, LPA, SLS, UBO, and BIANCA). To accomplish this, we tested these six methods on the MICCAI 2017 WMH Segmentation Challenge dataset, which contains MRI images from 170 elderly participants with WMHs of presumed vascular origin, and assessed their robustness across multiple sites and scanner types. The results showed that AQUA achieved superior performance in terms of spatial (Dice = 0.72) and volumetric (logAVD = 0.10) agreement with the manual segmentation compared to the other methods. While the recall and F1-score were moderate at 0.49 and 0.59, respectively, they improved to 0.75 and 0.82 when excluding small lesions (≤ 6 voxels). Remarkably, despite being trained on a different dataset with different ethnic backgrounds, lesion loads, and scanners, AQUA's results were comparable to the top 10 ranked methods of the MICCAI challenge. The findings suggest that AQUA is effective and practical for automatic segmentation of WMHs from T2-FLAIR scans, which could help identify individuals at risk of cognitive decline and dementia and allow for early intervention and management.
Collapse
Affiliation(s)
- Soojin Lee
- Research Institute, NEUROPHET Inc., Seoul, South Korea; Pacific Parkinson's Research Centre, The University of British Columbia, Vancouver, Canada.
| | - ZunHyan Rieu
- Research Institute, NEUROPHET Inc., Seoul, South Korea
| | - Regina Ey Kim
- Research Institute, NEUROPHET Inc., Seoul, South Korea
| | - Minho Lee
- Research Institute, NEUROPHET Inc., Seoul, South Korea
| | - Kevin Yen
- Division of Neurology, Department of Medicine, University of Alberta, Edmonton, Canada
| | - Junghyun Yong
- Research Institute, NEUROPHET Inc., Seoul, South Korea
| | - Donghyeon Kim
- Research Institute, NEUROPHET Inc., Seoul, South Korea
| |
Collapse
|
49
|
Zhao Q, Feng Q, Zhang J, Xu J, Wu Z, Huang C, Yuan H. Glenoid segmentation from computed tomography scans based on a 2-stage deep learning model for glenoid bone loss evaluation. J Shoulder Elbow Surg 2023; 32:e624-e635. [PMID: 37308073 DOI: 10.1016/j.jse.2023.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 04/16/2023] [Accepted: 05/06/2023] [Indexed: 06/14/2023]
Abstract
BACKGROUND The best-fitting circle drawn by computed tomography (CT) reconstruction of the en face view of the glenoid bone to measure the bone defect is widely used in clinical application. However, there are still some limitations in practical application, which can prevent the achievement of accurate measurements. This study aimed to accurately and automatically segment the glenoid from CT scans based on a 2-stage deep learning model and to quantitatively measure the glenoid bone defect. MATERIALS AND METHODS Patients who were referred to our institution between June 2018 and February 2022 were retrospectively reviewed. The dislocation group consisted of 237 patients with a history of ≥2 unilateral shoulder dislocations within 2 years. The control group consisted of 248 individuals with no history of shoulder dislocation, shoulder developmental deformity, or other disease that may lead to abnormal morphology of the glenoid. All patients underwent CT examination with a 1-mm slice thickness and a 1-mm increment, including complete imaging of the bilateral glenoid. A residual neural network (ResNet) location model and a U-Net bone segmentation model were constructed to develop an automated segmentation model for the glenoid from CT scans. The data set was randomly divided into training (201 of 248) and test (47 of 248) data sets of control-group data and training (190 of 237) and test (47 of 237) data sets of dislocation-group data. The accuracy of the stage 1 (glenoid location) model, the mean intersection-over-union value of the stage 2 (glenoid segmentation) model, and the glenoid volume error were used to assess the performance of the model. The R2 value and Lin concordance correlation coefficient were used to assess the correlation between the prediction and the gold standard. RESULTS A total of 73,805 images were obtained after the labeling process, and each image was composed of CT images of the glenoid and its corresponding mask. The average overall accuracy of stage 1 was 99.28%; the average mean intersection-over-union value of stage 2 was 0.96. The average glenoid volume error between the predicted and true values was 9.33%. The R2 values of the predicted and true values of glenoid volume and glenoid bone loss (GBL) were 0.87 and 0.91, respectively. The Lin concordance correlation coefficient value of the predicted and true values of glenoid volume and GBL were 0.93 and 0.95, respectively. CONCLUSION The 2-stage model in this study showed a good performance in glenoid bone segmentation from CT scans and could quantitatively measure GBL, providing a data reference for subsequent clinical treatment.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Huishu Yuan
- Peking University Third Hospital, Beijing, China.
| |
Collapse
|
50
|
Xie R, Pan D, Zeng A, Xu X, Wang T, Ullah N, Ji Y. Target area distillation and section attention segmentation network for accurate 3D medical image segmentation. Health Inf Sci Syst 2023; 11:9. [PMID: 36721638 PMCID: PMC9884720 DOI: 10.1007/s13755-022-00200-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 10/21/2022] [Indexed: 01/31/2023] Open
Abstract
3D medical image segmentation has an essential role in medical image analysis, while attention mechanism has improved the performance by a large margin. However, existing methods obtained the attention coefficient in a small receptive field, resulting in possible performance limitations. Radiologists usually scan all the slices first to have an overall idea of the target, and then analyze regions of interest in multiple 2D views in clinic practice. We simulate radiologists' recognition process and propose to exploit the 3D context information in a deeper manner for accurate 3D medical images segmentation. Due to the similarity of human body structure, medical images of different populations have highly similar shape and location information, so we use target region distillation to extract the common segmented region information. Particularly, we proposed two optimizations including Target Area Distillation and Section Attention. Target Area Distillation adds positions information to the original input to let the network has an initial attention of the target, while section attention performs attention extraction in three 2D sections thus with large range of receptive field. We compare our method against several popular networks in two public datasets including ImageCHD and COVID-19. Experimental results show that our proposed method improves the segmentation Dice score by 2-4% over the state-of-the-art methods. Our code has been released to the public (Anonymous link).
Collapse
Affiliation(s)
- Ruiwei Xie
- Guangdong University of Technology, Guangzhou, Guangdong China
| | - Dan Pan
- Guangdong Polytechnic Normal University, Guangzhou, Guangdong China
| | - An Zeng
- Guangdong University of Technology, Guangzhou, Guangdong China
| | - Xiaowei Xu
- Guangdong Provincial People’s Hospital, Guangzhou, Guangdong China
| | - Tianchen Wang
- Guangdong Provincial People’s Hospital, Guangzhou, Guangdong China
| | - Najeeb Ullah
- Mardan University of Engineering and Technology, Mardan, Pakistan
| | - Yuzhu Ji
- Guangdong University of Technology, Guangzhou, Guangdong China
| |
Collapse
|