1
|
Zhang X, Li Q, Li W, Guo Y, Zhang J, Guo C, Chang K, Lovell NH. FD-Net: Feature Distillation Network for Oral Squamous Cell Carcinoma Lymph Node Segmentation in Hyperspectral Imagery. IEEE J Biomed Health Inform 2024; 28:1552-1563. [PMID: 38446656 DOI: 10.1109/jbhi.2024.3350245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
Oral squamous cell carcinoma (OSCC) has the characteristics of early regional lymph node metastasis. OSCC patients often have poor prognoses and low survival rates due to cervical lymph metastases. Therefore, it is necessary to rely on a reasonable screening method to quickly judge the cervical lymph metastastic condition of OSCC patients and develop appropriate treatment plans. In this study, the widely used pathological sections with hematoxylin-eosin (H&E) staining are taken as the target, and combined with the advantages of hyperspectral imaging technology, a novel diagnostic method for identifying OSCC lymph node metastases is proposed. The method consists of a learning stage and a decision-making stage, focusing on cancer and non-cancer nuclei, gradually completing the lesions' segmentation from coarse to fine, and achieving high accuracy. In the learning stage, the proposed feature distillation-Net (FD-Net) network is developed to segment the cancerous and non-cancerous nuclei. In the decision-making stage, the segmentation results are post-processed, and the lesions are effectively distinguished based on the prior. Experimental results demonstrate that the proposed FD-Net is very competitive in the OSCC hyperspectral medical image segmentation task. The proposed FD-Net method performs best on the seven segmentation evaluation indicators: MIoU, OA, AA, SE, CSI, GDR, and DICE. Among these seven evaluation indicators, the proposed FD-Net method is 1.75%, 1.27%, 0.35%, 1.9%, 0.88%, 4.45%, and 1.98% higher than the DeepLab V3 method, which ranks second in performance, respectively. In addition, the proposed diagnosis method of OSCC lymph node metastasis can effectively assist pathologists in disease screening and reduce the workload of pathologists.
Collapse
|
2
|
Jhang JY, Tsai YC, Hsu TC, Huang CR, Cheng HC, Sheu BS. Gastric Section Correlation Network for Gastric Precancerous Lesion Diagnosis. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2023; 5:434-442. [PMID: 38899022 PMCID: PMC11186652 DOI: 10.1109/ojemb.2023.3277219] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Revised: 03/24/2023] [Accepted: 05/10/2023] [Indexed: 06/21/2024] Open
Abstract
Goal: Diagnosing the corpus-predominant gastritis index (CGI) which is an early precancerous lesion in the stomach has been shown its effectiveness in identifying high gastric cancer risk patients for preventive healthcare. However, invasive biopsies and time-consuming pathological analysis are required for the CGI diagnosis. Methods: We propose a novel gastric section correlation network (GSCNet) for the CGI diagnosis from endoscopic images of three dominant gastric sections, the antrum, body and cardia. The proposed network consists of two dominant modules including the scaling feature fusion module and section correlation module. The front one aims to extract scaling fusion features which can effectively represent the mucosa under variant viewing angles and scale changes for each gastric section. The latter one aims to apply the medical prior knowledge with three section correlation losses to model the correlations of different gastric sections for the CGI diagnosis. Results: The proposed method outperforms competing deep learning methods and achieves high testing accuracy, sensitivity, and specificity of 0.957, 0.938 and 0.962, respectively. Conclusions: The proposed method is the first method to identify high gastric cancer risk patients with CGI from endoscopic images without invasive biopsies and time-consuming pathological analysis.
Collapse
Affiliation(s)
- Jyun-Yao Jhang
- Department of Computer Science and EngineeringNational Chung Hsing UniversityTaichung402Taiwan
| | - Yu-Ching Tsai
- Department of Internal MedicineTainan Hospital, Ministry of Health and WelfareTainan701Taiwan
- Department of Internal Medicine, National Cheng Kung University Hospital, College of MedicineNational Cheng Kung UniversityTainan701Taiwan
| | - Tzu-Chun Hsu
- Department of Computer Science and EngineeringNational Chung Hsing UniversityTaichung402Taiwan
| | - Chun-Rong Huang
- Cross College Elite Program, and Academy of Innovative Semiconductor and Sustainable ManufacturingNational Cheng Kung UniversityTainan701Taiwan
- Department of Computer Science and EngineeringNational Chung Hsing UniversityTaichung402Taiwan
| | - Hsiu-Chi Cheng
- Department of Internal Medicine, Institute of Clinical Medicine and Molecular MedicineNational Cheng Kung UniversityTainan701Taiwan
- Department of Internal MedicineTainan Hospital, Ministry of Health and WelfareTainan701Taiwan
| | - Bor-Shyang Sheu
- Institute of Clinical Medicine and Department of Internal Medicine, National Cheng Kung University HospitalNational Cheng Kung UniversityTainan701Taiwan
| |
Collapse
|
3
|
Semantic Segmentation of Digestive Abnormalities from WCE Images by Using AttResU-Net Architecture. Life (Basel) 2023; 13:life13030719. [PMID: 36983874 PMCID: PMC10051085 DOI: 10.3390/life13030719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 02/04/2023] [Accepted: 03/03/2023] [Indexed: 03/09/2023] Open
Abstract
Colorectal cancer is one of the most common malignancies and the leading cause of cancer death worldwide. Wireless capsule endoscopy is currently the most frequent method for detecting precancerous digestive diseases. Thus, precise and early polyps segmentation has significant clinical value in reducing the probability of cancer development. However, the manual examination is a time-consuming and tedious task for doctors. Therefore, scientists have proposed many computational techniques to automatically segment the anomalies from endoscopic images. In this paper, we present an end-to-end 2D attention residual U-Net architecture (AttResU-Net), which concurrently integrates the attention mechanism and residual units into U-Net for further polyp and bleeding segmentation performance enhancement. To reduce outside areas in an input image while emphasizing salient features, AttResU-Net inserts a sequence of attention units among related downsampling and upsampling steps. On the other hand, the residual block propagates information across layers, allowing for the construction of a deeper neural network capable of solving the vanishing gradient issue in each encoder. This improves the channel interdependencies while lowering the computational cost. Multiple publicly available datasets were employed in this work, to evaluate and verify the proposed method. Our highest-performing model was AttResU-Net, on the MICCAI 2017 WCE dataset, which achieved an accuracy of 99.16%, a Dice coefficient of 94.91%, and a Jaccard index of 90.32%. The experiment findings show that the proposed AttResU-Net overcomes its baselines and provides performance comparable to existing polyp segmentation approaches.
Collapse
|
4
|
Zhang Y, Luo L, Dou Q, Heng PA. Triplet attention and dual-pool contrastive learning for clinic-driven multi-label medical image classification. Med Image Anal 2023; 86:102772. [PMID: 36822050 DOI: 10.1016/j.media.2023.102772] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 11/21/2022] [Accepted: 02/10/2023] [Indexed: 02/18/2023]
Abstract
Multi-label classification (MLC) can attach multiple labels on single image, and has achieved promising results on medical images. But existing MLC methods still face challenging clinical realities in practical use, such as: (1) medical risks arising from misclassification, (2) sample imbalance problem among different diseases, (3) inability to classify the diseases that are not pre-defined (unseen diseases). Here, we design a hybrid label to improve the flexibility of MLC methods and alleviate the sample imbalance problem. Specifically, in the labeled training set, we remain independent labels for high-frequency diseases with enough samples and use a hybrid label to merge low-frequency diseases with fewer samples. The hybrid label can also be used to put unseen diseases in practical use. In this paper, we propose Triplet Attention and Dual-pool Contrastive Learning (TA-DCL) for multi-label medical image classification based on the aforementioned label representation. TA-DCL architecture is a triplet attention network (TAN), which combines category-attention, self-attention and cross-attention together to learn high-quality label embeddings for all disease labels by mining effective information from medical images. DCL includes dual-pool contrastive training (DCT) and dual-pool contrastive inference (DCI). DCT optimizes the clustering centers of label embeddings belonging to different disease labels to improve the discrimination of label embeddings. DCI relieves the error classification of sick cases for reducing the clinical risk and improving the ability to detect unseen diseases by contrast of differences. TA-DCL is validated on two public medical image datasets, ODIR and NIH-ChestXray14, showing superior performance than other state-of-the-art MLC methods. Code is available at https://github.com/ZhangYH0502/TA-DCL.
Collapse
Affiliation(s)
- Yuhan Zhang
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Institute of Medical Intelligence and XR, The Chinese University of Hong Kong, Hong Kong, China; Shenzhen Research Institute, The Chinese University of Hong Kong, Hong Kong, China.
| | - Luyang Luo
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| | - Qi Dou
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Institute of Medical Intelligence and XR, The Chinese University of Hong Kong, Hong Kong, China.
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Institute of Medical Intelligence and XR, The Chinese University of Hong Kong, Hong Kong, China.
| |
Collapse
|
5
|
Zhao Y, Wang X, Che T, Bao G, Li S. Multi-task deep learning for medical image computing and analysis: A review. Comput Biol Med 2023; 153:106496. [PMID: 36634599 DOI: 10.1016/j.compbiomed.2022.106496] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 12/06/2022] [Accepted: 12/27/2022] [Indexed: 12/29/2022]
Abstract
The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.
Collapse
Affiliation(s)
- Yan Zhao
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Xiuying Wang
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia.
| | - Tongtong Che
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Guoqing Bao
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia
| | - Shuyu Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
6
|
Oliveira B, Torres HR, Morais P, Veloso F, Baptista AL, Fonseca JC, Vilaça JL. A multi-task convolutional neural network for classification and segmentation of chronic venous disorders. Sci Rep 2023; 13:761. [PMID: 36641527 PMCID: PMC9840616 DOI: 10.1038/s41598-022-27089-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 12/26/2022] [Indexed: 01/16/2023] Open
Abstract
Chronic Venous Disorders (CVD) of the lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. Due to the exponential growth of the aging population and the worsening of CVD with age, it is expected that the healthcare costs and the resources needed for the treatment of CVD will increase in the coming years. The early diagnosis of CVD is fundamental in treatment planning, while the monitoring of its treatment is fundamental to assess a patient's condition and quantify the evolution of CVD. However, correct diagnosis relies on a qualitative approach through visual recognition of the various venous disorders, being time-consuming and highly dependent on the physician's expertise. In this paper, we propose a novel automatic strategy for the joint segmentation and classification of CVDs. The strategy relies on a multi-task deep learning network, denominated VENet, that simultaneously solves segmentation and classification tasks, exploiting the information of both tasks to increase learning efficiency, ultimately improving their performance. The proposed method was compared against state-of-the-art strategies in a dataset of 1376 CVD images. Experiments showed that the VENet achieved a classification performance of 96.4%, 96.4%, and 97.2% for accuracy, precision, and recall, respectively, and a segmentation performance of 75.4%, 76.7.0%, 76.7% for the Dice coefficient, precision, and recall, respectively. The joint formulation increased the robustness of both tasks when compared to the conventional classification or segmentation strategies, proving its added value, mainly for the segmentation of small lesions.
Collapse
Affiliation(s)
- Bruno Oliveira
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal. .,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal. .,Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal. .,2Ai - School of Technology, IPCA, Barcelos, Portugal. .,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal.
| | - Helena R Torres
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal.,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal.,Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal.,2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - Pedro Morais
- 2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - Fernando Veloso
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal.,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal.,2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal.,Department of Mechanical Engineering, School of Engineering, University of Minho, Guimarães, Portugal
| | | | - Jaime C Fonseca
- Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - João L Vilaça
- 2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| |
Collapse
|
7
|
Zan P, Zhong H, Zhao Y, Xu H, Hong R, Ding Q, Yue J. Research on improved intestinal image classification for LARS based on ResNet. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2022; 93:124101. [PMID: 36586901 DOI: 10.1063/5.0100192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Accepted: 11/13/2022] [Indexed: 06/17/2023]
Abstract
Low anterior rectal resection is an effective way to treat rectal cancer at present, but it is easy to cause low anterior resection syndrome after surgery; so, a comprehensive diagnosis of defecation and pelvic floor function must be carried out. There are few studies on the classification of diagnoses in the field of intestinal diseases. In response to these outstanding problems, this research will focus on the design of the intestinal function diagnosis system and the image processing and classification algorithm of the intestinal wall to verify an efficient fusion method, which can be used to diagnose the intestinal diseases in clinical medicine. The diagnostic system designed in this paper makes up for the singleness of clinical monitoring methods. At the same time, the Res-SVDNet neural network model is used to solve the problems of small intestinal image samples and network overfitting, and achieve efficient fusion diagnosis of intestinal diseases in patients. Different models were used to compare experiments on the constructed datasets to verify the applicability of the Res-SVDNet model in intestinal image classification. The accuracy of the model was 99.54%, which is several percentage points higher than other algorithm models.
Collapse
Affiliation(s)
- Peng Zan
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Hua Zhong
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Yutong Zhao
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Huiyan Xu
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Rui Hong
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Qiao Ding
- Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Jingwei Yue
- Beijing Institute of Radiation Medicine, Beijing 100850, China
| |
Collapse
|
8
|
Ramamurthy K, George TT, Shah Y, Sasidhar P. A Novel Multi-Feature Fusion Method for Classification of Gastrointestinal Diseases Using Endoscopy Images. Diagnostics (Basel) 2022; 12:2316. [PMID: 36292006 PMCID: PMC9600128 DOI: 10.3390/diagnostics12102316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Revised: 09/02/2022] [Accepted: 09/06/2022] [Indexed: 11/17/2022] Open
Abstract
The first step in the diagnosis of gastric abnormalities is the detection of various abnormalities in the human gastrointestinal tract. Manual examination of endoscopy images relies on a medical practitioner's expertise to identify inflammatory regions on the inner surface of the gastrointestinal tract. The length of the alimentary canal and the large volume of images obtained from endoscopic procedures make traditional detection methods time consuming and laborious. Recently, deep learning architectures have achieved better results in the classification of endoscopy images. However, visual similarities between different portions of the gastrointestinal tract pose a challenge for effective disease detection. This work proposes a novel system for the classification of endoscopy images by focusing on feature mining through convolutional neural networks (CNN). The model presented is built by combining a state-of-the-art architecture (i.e., EfficientNet B0) with a custom-built CNN architecture named Effimix. The proposed Effimix model employs a combination of squeeze and excitation layers and self-normalising activation layers for precise classification of gastrointestinal diseases. Experimental observations on the HyperKvasir dataset confirm the effectiveness of the proposed architecture for the classification of endoscopy images. The proposed model yields an accuracy of 97.99%, with an F1 score, precision, and recall of 97%, 97%, and 98%, respectively, which is significantly higher compared to the existing works.
Collapse
Affiliation(s)
- Karthik Ramamurthy
- Centre for Cyber Physical Systems, School of Electronics Engineering, Vellore Institute of Technology, Chennai 600127, India
| | - Timothy Thomas George
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai 600127, India
| | - Yash Shah
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai 600127, India
| | - Parasa Sasidhar
- School of Electronics Engineering, Vellore Institute of Technology, Chennai 600127, India
| |
Collapse
|