1
|
Zaman A, Hassan H, Zeng X, Khan R, Lu J, Yang H, Miao X, Cao A, Yang Y, Huang B, Guo Y, Kang Y. Adaptive Feature Medical Segmentation Network: an adaptable deep learning paradigm for high-performance 3D brain lesion segmentation in medical imaging. Front Neurosci 2024; 18:1363930. [PMID: 38680446 PMCID: PMC11047127 DOI: 10.3389/fnins.2024.1363930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2023] [Accepted: 03/04/2024] [Indexed: 05/01/2024] Open
Abstract
Introduction In neurological diagnostics, accurate detection and segmentation of brain lesions is crucial. Identifying these lesions is challenging due to its complex morphology, especially when using traditional methods. Conventional methods are either computationally demanding with a marginal impact/enhancement or sacrifice fine details for computational efficiency. Therefore, balancing performance and precision in compute-intensive medical imaging remains a hot research topic. Methods We introduce a novel encoder-decoder network architecture named the Adaptive Feature Medical Segmentation Network (AFMS-Net) with two encoder variants: the Single Adaptive Encoder Block (SAEB) and the Dual Adaptive Encoder Block (DAEB). A squeeze-and-excite mechanism is employed in SAEB to identify significant data while disregarding peripheral details. This approach is best suited for scenarios requiring quick and efficient segmentation, with an emphasis on identifying key lesion areas. In contrast, the DAEB utilizes an advanced channel spatial attention strategy for fine-grained delineation and multiple-class classifications. Additionally, both architectures incorporate a Segmentation Path (SegPath) module between the encoder and decoder, refining segmentation, enhancing feature extraction, and improving model performance and stability. Results AFMS-Net demonstrates exceptional performance across several notable datasets, including BRATs 2021, ATLAS 2021, and ISLES 2022. Its design aims to construct a lightweight architecture capable of handling complex segmentation challenges with high precision. Discussion The proposed AFMS-Net addresses the critical balance issue between performance and computational efficiency in the segmentation of brain lesions. By introducing two tailored encoder variants, the network adapts to varying requirements of speed and feature. This approach not only advances the state-of-the-art in lesion segmentation but also provides a scalable framework for future research in medical image processing.
Collapse
Affiliation(s)
- Asim Zaman
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
- Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, China
| | - Haseeb Hassan
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
| | - Xueqiang Zeng
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
| | - Rashid Khan
- School of Applied Technology, Shenzhen University, Shenzhen, China
- Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, China
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen, China
| | - Jiaxi Lu
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
| | - Huihui Yang
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
| | - Xiaoqiang Miao
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Anbo Cao
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
| | - Yingjian Yang
- Shenzhen Lanmage Medical Technology Co., Ltd, Shenzhen, China
| | - Bingding Huang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen, China
| | - Yingwei Guo
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Electrical and Information Engineering, Northeast Petroleum University, Daqing, China
| | - Yan Kang
- School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China
- College of Health Science and Environmental Engineering, Shenzhen Technology University, Shenzhen, China
- School of Applied Technology, Shenzhen University, Shenzhen, China
- Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, China
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| |
Collapse
|
2
|
Bai T, Zhou S, Pang Y, Luo J, Wang H, Du Y. An image caption model based on attention mechanism and deep reinforcement learning. Front Neurosci 2023; 17:1270850. [PMID: 37869519 PMCID: PMC10585027 DOI: 10.3389/fnins.2023.1270850] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 09/04/2023] [Indexed: 10/24/2023] Open
Abstract
Image caption technology aims to convert visual features of images, extracted by computers, into meaningful semantic information. Therefore, the computers can generate text descriptions that resemble human perception, enabling tasks such as image classification, retrieval, and analysis. In recent years, the performance of image caption has been significantly enhanced with the introduction of encoder-decoder architecture in machine translation and the utilization of deep neural networks. However, several challenges still persist in this domain. Therefore, this paper proposes a novel method to address the issue of visual information loss and non-dynamic adjustment of input images during decoding. We introduce a guided decoding network that establishes a connection between the encoding and decoding parts. Through this connection, encoding information can provide guidance to the decoding process, facilitating automatic adjustment of the decoding information. In addition, Dense Convolutional Network (DenseNet) and Multiple Instance Learning (MIL) are adopted in the image encoder, and Nested Long Short-Term Memory (NLSTM) is utilized as the decoder to enhance the extraction and parsing capability of image information during the encoding and decoding process. In order to further improve the performance of our image caption model, this study incorporates an attention mechanism to focus details and constructs a double-layer decoding structure, which facilitates the enhancement of the model in terms of providing more detailed descriptions and enriched semantic information. Furthermore, the Deep Reinforcement Learning (DRL) method is employed to train the model by directly optimizing the identical set of evaluation indexes, which solves the problem of inconsistent training and evaluation standards. Finally, the model is trained and tested on MS COCO and Flickr 30 k datasets, and the results show that the model has improved compared with commonly used models in the evaluation indicators such as BLEU, METEOR and CIDEr.
Collapse
Affiliation(s)
- Tong Bai
- School of Optoelectronic Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Sen Zhou
- Chongqing Academy of Metrology and Quality Inspection, Chongqing, China
| | - Yu Pang
- School of Optoelectronic Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Jiasai Luo
- School of Optoelectronic Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Huiqian Wang
- School of Optoelectronic Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Ya Du
- Department of Peripheral Vascular (Wound Repair), Chongqing Hospital of Traditional Chinese Medicine, Chongqing, China
| |
Collapse
|
3
|
Peng B, Ren Z, Parthasarathy S, Ning X. M 2: Mixed Models With Preferences, Popularities and Transitions for Next-Basket Recommendation. IEEE Trans Knowl Data Eng 2023; 35:4033-4046. [PMID: 37092026 PMCID: PMC10117693 DOI: 10.1109/tkde.2022.3142773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Next-basket recommendation considers the problem of recommending a set of items into the next basket that users will purchase as a whole. In this paper, we develop a novel mixed model with preferences, popularities and transitions (M2) for the next-basket recommendation. This method models three important factors in next-basket generation process: 1) users' general preferences, 2) items' global popularities and 3) transition patterns among items. Unlike existing recurrent neural network-based approaches, M2 does not use the complicated networks to model the transitions among items, or generate embeddings for users. Instead, it has a simple encoder-decoder based approach (ed-Trans) to better model the transition patterns among items. We compared M2 with different combinations of the factors with 5 state-of-the-art next-basket recommendation methods on 4 public benchmark datasets in recommending the first, second and third next basket. Our experimental results demonstrate that M2 significantly outperforms the state-of-the-art methods on all the datasets in all the tasks, with an improvement of up to 22.1%. In addition, our ablation study demonstrates that the ed-Trans is more effective than recurrent neural networks in terms of the recommendation performance. We also have a thorough discussion on various experimental protocols and evaluation metrics for next-basket recommendation evaluation.
Collapse
Affiliation(s)
- Bo Peng
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210 USA
| | - Zhiyun Ren
- Department of Biomedical Informatics, The Ohio State University, Columbus, OH 43210 USA
| | - Srinivasan Parthasarathy
- Department of Biomedical Informatics, Department of Computer Science and Engineering, and Translational Data Analytics Institute, The Ohio State University, Columbus, OH43210 USA
| | - Xia Ning
- Department of Biomedical Informatics, Department of Computer Science and Engineering, and Translational Data Analytics Institute, The Ohio State University, Columbus, OH43210 USA
| |
Collapse
|
4
|
Abstract
This study proposes a new predictive segmentation method for liver tumors detection using computed tomography (CT) liver images. In the medical imaging field, the exact localization of metastasis lesions after acquisition faces persistent problems both for diagnostic aid and treatment effectiveness. Therefore, the improvement in the diagnostic process is substantially crucial in order to increase the success chance of the management and the therapeutic follow-up. The proposed procedure highlights a computerized approach based on an encoder-decoder structure in order to provide volumetric analysis of pathologic tumors. Specifically, we developed an automatic algorithm for the liver tumors defect segmentation through the Seg-Net and U-Net architectures from metastasis CT images. In this study, we collected a dataset of 200 pathologically confirmed metastasis cancer cases. A total of 8,297 CT image slices of these cases were used developing and optimizing the proposed segmentation architecture. The model was trained and validated using 170 and 30 cases or 85% and 15% of the CT image data, respectively. Study results demonstrate the strength of the proposed approach that reveals the superlative segmentation performance as evaluated using following indices including F1-score = 0.9573, Recall = 0.9520, IOU = 0.9654, Binary cross entropy = 0.0032 and p-value <0.05, respectively. In comparison to state-of-the-art techniques, the proposed method yields a higher precision rate by specifying metastasis tumor position.
Collapse
Affiliation(s)
- Hanene Sahli
- Laboratory of Signal Image and Energy Mastery (SIME), LR13ES03, University of Tunis, ENSIT, 1008, Tunis, Tunisia
| | - Amine Ben Slama
- Laboratory of Biophysics and Medical Technologies, LR13ES07, University of Tunis EL Manar, ISTMT, 1006, Tunis, Tunisia
| | - Salam Labidi
- Laboratory of Biophysics and Medical Technologies, LR13ES07, University of Tunis EL Manar, ISTMT, 1006, Tunis, Tunisia
| |
Collapse
|
5
|
Ilyas T, Umraiz M, Khan A, Kim H. DAM: Hierarchical Adaptive Feature Selection Using Convolution Encoder Decoder Network for Strawberry Segmentation. Front Plant Sci 2021; 12:591333. [PMID: 33692814 PMCID: PMC7937933 DOI: 10.3389/fpls.2021.591333] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Accepted: 01/25/2021] [Indexed: 05/05/2023]
Abstract
Autonomous harvesters can be used for the timely cultivation of high-value crops such as strawberries, where the robots have the capability to identify ripe and unripe crops. However, the real-time segmentation of strawberries in an unbridled farming environment is a challenging task due to fruit occlusion by multiple trusses, stems, and leaves. In this work, we propose a possible solution by constructing a dynamic feature selection mechanism for convolutional neural networks (CNN). The proposed building block namely a dense attention module (DAM) controls the flow of information between the convolutional encoder and decoder. DAM enables hierarchical adaptive feature fusion by exploiting both inter-channel and intra-channel relationships and can be easily integrated into any existing CNN to obtain category-specific feature maps. We validate our attention module through extensive ablation experiments. In addition, a dataset is collected from different strawberry farms and divided into four classes corresponding to different maturity levels of fruits and one is devoted to background. Quantitative analysis of the proposed method showed a 4.1% and 2.32% increase in mean intersection over union, over existing state-of-the-art semantic segmentation models and other attention modules respectively, while simultaneously retaining a processing speed of 53 frames per second.
Collapse
Affiliation(s)
- Talha Ilyas
- Division of Electronic Engineering, Intelligent Robots Research Center, Jeonbuk National University, Jeonju, South Korea
| | - Muhammad Umraiz
- Division of Electronic Engineering, Intelligent Robots Research Center, Jeonbuk National University, Jeonju, South Korea
| | - Abbas Khan
- Division of Electronic Engineering, Intelligent Robots Research Center, Jeonbuk National University, Jeonju, South Korea
| | - Hyongsuk Kim
- Division of Electronic Engineering, Intelligent Robots Research Center, Jeonbuk National University, Jeonju, South Korea
- Division of Electronic and Information Engineering, Jeonbuk National University, Jeonju, South Korea
| |
Collapse
|
6
|
Zhang Y, Zhang S, Li Y, Zhang Y. Coarse-to-Fine Satellite Images Change Detection Framework via Boundary-Aware Attentive Network. Sensors (Basel) 2020; 20:s20236735. [PMID: 33255688 PMCID: PMC7728072 DOI: 10.3390/s20236735] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Revised: 11/12/2020] [Accepted: 11/18/2020] [Indexed: 11/30/2022]
Abstract
Timely and accurate change detection on satellite images by using computer vision techniques has been attracting lots of research efforts in recent years. Existing approaches based on deep learning frameworks have achieved good performance for the task of change detection on satellite images. However, under the scenario of disjoint changed areas in various shapes on land surface, existing methods still have shortcomings in detecting all changed areas correctly and representing the changed areas boundary. To deal with these problems, we design a coarse-to-fine detection framework via a boundary-aware attentive network with a hybrid loss to detect the change in high resolution satellite images. Specifically, we first perform an attention guided encoder-decoder subnet to obtain the coarse change map of the bi-temporal image pairs, and then apply residual learning to obtain the refined change map. We also propose a hybrid loss to provide the supervision from pixel, patch, and map levels. Comprehensive experiments are conducted on two benchmark datasets: LEBEDEV and SZTAKI to verify the effectiveness of the proposed method and the experimental results show that our model achieves state-of-the-art performance.
Collapse
Affiliation(s)
- Yi Zhang
- School of Computer Science, National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Shaanxi Provincial Key Laboratory of Speech & Image Information Processing, Northwestern Polytechnical University, Xi’an 710129, China; (Y.L.); (Y.Z.)
- Correspondence: (Y.Z.), (S.Z.)
| | - Shizhou Zhang
- School of Computer Science, National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Shaanxi Provincial Key Laboratory of Speech & Image Information Processing, Northwestern Polytechnical University, Xi’an 710129, China; (Y.L.); (Y.Z.)
- Correspondence: (Y.Z.), (S.Z.)
| | - Ying Li
- School of Computer Science, National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Shaanxi Provincial Key Laboratory of Speech & Image Information Processing, Northwestern Polytechnical University, Xi’an 710129, China; (Y.L.); (Y.Z.)
- School of Communication and Information Engineering, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
| | - Yanning Zhang
- School of Computer Science, National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Shaanxi Provincial Key Laboratory of Speech & Image Information Processing, Northwestern Polytechnical University, Xi’an 710129, China; (Y.L.); (Y.Z.)
| |
Collapse
|
7
|
Billah UH, La HM, Tavakkoli A. Deep Learning-Based Feature Silencing for Accurate Concrete Crack Detection. Sensors (Basel) 2020; 20:E4403. [PMID: 32784557 PMCID: PMC7472489 DOI: 10.3390/s20164403] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Revised: 08/02/2020] [Accepted: 08/05/2020] [Indexed: 11/30/2022]
Abstract
An autonomous concrete crack inspection system is necessary for preventing hazardous incidents arising from deteriorated concrete surfaces. In this paper, we present a concrete crack detection framework to aid the process of automated inspection. The proposed approach employs a deep convolutional neural network architecture for crack segmentation, while addressing the effect of gradient vanishing problem. A feature silencing module is incorporated in the proposed framework, capable of eliminating non-discriminative feature maps from the network to improve performance. Experimental results support the benefit of incorporating feature silencing within a convolutional neural network architecture for improving the network's robustness, sensitivity, and specificity. An added benefit of the proposed architecture is its ability to accommodate for the trade-off between specificity (positive class detection accuracy) and sensitivity (negative class detection accuracy) with respect to the target application. Furthermore, the proposed framework achieves a high precision rate and processing time than the state-of-the-art crack detection architectures.
Collapse
|
8
|
Li L, Jia T. Optical Coherence Tomography Vulnerable Plaque Segmentation Based on Deep Residual U-Net. Rev Cardiovasc Med 2020; 20:171-177. [PMID: 31601091 DOI: 10.31083/j.rcm.2019.03.5201] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2019] [Accepted: 08/14/2019] [Indexed: 11/06/2022] Open
Abstract
Automatic and accurate segmentation of intravascular optical coherence tomography imagery is of great importance in computer-aided diagnosis and in treatment of cardiovascular diseases. However, this task has not been well addressed for two reasons. First, because of the difficulty of acquisition, and the laborious labeling from personnel, optical coherence tomography image datasets are usually small. Second, optical coherence tomography images contain a variety of imaging artifacts, which hinder a clear observation of the vascular wall. In order to overcome these limitations, a new method of cardiovascular vulnerable plaque segmentation is proposed. This method constructs a novel Deep Residual U-Net to segment vulnerable plaque regions. Furthermore, in order to overcome the inaccuracy in object boundary segmentation which previous research has shown extensively, a loss function consisting of weighted cross-entropy loss and Dice coefficient is proposed to solve this problem. Thorough experiments and analysis have been carried out to verify the effectiveness and superior performance of the proposed method.
Collapse
Affiliation(s)
- Lincan Li
- College of Mechanical Engineering and Automation, Northeastern University, Shenyang, 110004, P. R. China
| | - Tong Jia
- College of Information Science and Engineering, Northeastern University, Shenyang, 110004, P. R. China
| |
Collapse
|