1
|
Bui DC, Song B, Kim K, Kwak JT. DAX-Net: A dual-branch dual-task adaptive cross-weight feature fusion network for robust multi-class cancer classification in pathology images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108112. [PMID: 38479146 DOI: 10.1016/j.cmpb.2024.108112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 02/15/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND AND OBJECTIVE Multi-class cancer classification has been extensively studied in digital and computational pathology due to its importance in clinical decision-making. Numerous computational tools have been proposed for various types of cancer classification. Many of them are built based on convolutional neural networks. Recently, Transformer-style networks have shown to be effective for cancer classification. Herein, we present a hybrid design that leverages both convolutional neural networks and transformer architecture to obtain superior performance in cancer classification. METHODS We propose a dual-branch dual-task adaptive cross-weight feature fusion network, called DAX-Net, which exploits heterogeneous feature representations from the convolutional neural network and Transformer network, adaptively combines them to boost their representation power, and conducts cancer classification as categorical classification and ordinal classification. For an efficient and effective optimization of the proposed model, we introduce two loss functions that are tailored to the two classification tasks. RESULTS To evaluate the proposed method, we employed colorectal and prostate cancer datasets, of which each contains both in-domain and out-of-domain test sets. For colorectal cancer, the proposed method obtained an accuracy of 88.4%, a quadratic kappa score of 0.945, and an F1 score of 0.831 for the in-domain test set, and 84.4%, 0.910, and 0.768 for the out-of-domain test set. For prostate cancer, it achieved an accuracy of 71.6%, a kappa score of 0.635, and an F1 score of 0.655 for the in-domain test set, 79.2% accuracy, 0.721 kappa score, and 0.686 F1 score for the first out-of-domain test set, and 58.1% accuracy, 0.564 kappa score, and 0.493 F1 score for the second out-of-domain test set. It is worth noting that the performance of the proposed method outperformed other competitors by significant margins, in particular, with respect to the out-of-domain test sets. CONCLUSIONS The experimental results demonstrate that the proposed method is not only accurate but also robust to varying conditions of the test sets in comparison to several, related methods. These results suggest that the proposed method can facilitate automated cancer classification in various clinical settings.
Collapse
Affiliation(s)
- Doanh C Bui
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Boram Song
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Jin Tae Kwak
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|
2
|
Li S, Li Z, Xue K, Zhou X, Ding C, Shao Y, Zhang S, Ruan T, Zheng M, Sun J. GC-CDSS: Personalized gastric cancer treatment recommendations system based on knowledge graph. Int J Med Inform 2024; 185:105402. [PMID: 38467099 DOI: 10.1016/j.ijmedinf.2024.105402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 02/25/2024] [Accepted: 03/05/2024] [Indexed: 03/13/2024]
Abstract
BACKGROUND Gastric cancer (GC) is one of the most common malignant tumors in the world, posing a serious threat to human health. Currently, gastric cancer treatment strategies emphasize a multidisciplinary team (MDT) consultation approach. However, there are numerous treatment guidelines and insights from clinical trials. The application of AI-based Clinical Decision Support System (CDSS) in tumor diagnosis and screening is increasing rapidly. OBJECTIVE The purpose of this study is to (1) summarize the treatment decision process for GC according to the treatment guidelines in China, and then create a knowledge graph (KG) for GC, (2) based on aforementioned KG, built a CDSS and conducted an initial feasibility evaluation for the current system. METHODS Firstly, we summarized the decision-making process for treatment of GC. Then, we extracted relevant decision nodes and relationships and utilized Neo4j to create the KG. After obtaining the initial node features for building the graph embedding model, graph embedding algorithm, such as Node2Vec and GraphSAGE, were used to construct the GC-CDSS. At last, a retrospective cohort study was used to compare the consistency between GC-CDSS and MDT in treatment decision making. RESULTS In current study, we introduce a GC-CDSS, which is constructed based on Chinese GC treatment guidelines knowledge graph (KG). In the KG, we define four types of nodes and four types of relationships, and it comprise a total of 207 nodes and 300 relationships. Regarding GC-CDSS, the system is capable of providing dynamic and personalized diagnostic and treatment recommendations based on the patient's condition. Furthermore, a retrospective cohort study is conducted to compare GC-CDSS recommendations with those of the MDT group, the overall consistency rate of treatment recommendations between the auxiliary decision system and MDT team is 92.96%. CONCLUSIONS We construct a GC treatment support system, GC-CDSS, based on KG. The GC-CDSS may help oncologists make treatment decisions more efficient and promote standardization in primary healthcare settings.
Collapse
Affiliation(s)
- Shuchun Li
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China
| | - Zhiang Li
- Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China
| | - Kui Xue
- Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China
| | - Xueliang Zhou
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China
| | - Chengsheng Ding
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China
| | - Yanfei Shao
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China
| | - Sen Zhang
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China
| | - Tong Ruan
- Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China.
| | - Minhua Zheng
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China.
| | - Jing Sun
- Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China; Shanghai Minimally Invasive Surgery Center, Ruijin Hospital, Shanghai Jiao Tong University, School of Medicine, Shanghai 200025, China.
| |
Collapse
|
3
|
Li Q, Xie W, Wang Y, Qin K, Huang M, Liu T, Chen Z, Chen L, Teng L, Fang Y, Ye L, Chen Z, Zhang J, Li A, Yang W, Liu S. A Deep Learning Application of Capsule Endoscopic Gastric Structure Recognition Based on a Transformer Model. J Clin Gastroenterol 2024:00004836-990000000-00271. [PMID: 38457410 DOI: 10.1097/mcg.0000000000001972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 12/26/2023] [Indexed: 03/10/2024]
Abstract
BACKGROUND Gastric structure recognition systems have become increasingly necessary for the accurate diagnosis of gastric lesions in capsule endoscopy. Deep learning, especially using transformer models, has shown great potential in the recognition of gastrointestinal (GI) images according to self-attention. This study aims to establish an identification model of capsule endoscopy gastric structures to improve the clinical applicability of deep learning to endoscopic image recognition. METHODS A total of 3343 wireless capsule endoscopy videos collected at Nanfang Hospital between 2011 and 2021 were used for unsupervised pretraining, while 2433 were for training and 118 were for validation. Fifteen upper GI structures were selected for quantifying the examination quality. We also conducted a comparison of the classification performance between the artificial intelligence model and endoscopists by the accuracy, sensitivity, specificity, and positive and negative predictive values. RESULTS The transformer-based AI model reached a relatively high level of diagnostic accuracy in gastric structure recognition. Regarding the performance of identifying 15 upper GI structures, the AI model achieved a macroaverage accuracy of 99.6% (95% CI: 99.5-99.7), a macroaverage sensitivity of 96.4% (95% CI: 95.3-97.5), and a macroaverage specificity of 99.8% (95% CI: 99.7-99.9) and achieved a high level of interobserver agreement with endoscopists. CONCLUSIONS The transformer-based AI model can accurately evaluate the gastric structure information of capsule endoscopy with the same performance as that of endoscopists, which will provide tremendous help for doctors in making a diagnosis from a large number of images and improve the efficiency of examination.
Collapse
Affiliation(s)
- Qingyuan Li
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Weijie Xie
- School of Biomedical Engineering
- Department of Information, Guangzhou First People's Hospital, School of Medicine, South China University of Technology
| | - Yusi Wang
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Kaiwen Qin
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Mei Huang
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | | | | | - Lu Chen
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Lan Teng
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Yuxin Fang
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | | | - Zhenyu Chen
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Jie Zhang
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Aimin Li
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
| | - Wei Yang
- School of Biomedical Engineering
- Pazhou Lab, Guangzhou, Guangdong
| | - Side Liu
- Guangdong Provincial Key Laboratory of Gastroenterology, Department of Gastroenterology, Nanfang Hospital
- Pazhou Lab, Guangzhou, Guangdong
- Department of Gastroenterology, Zhuhai People's Hospital, Zhuhai Hospital Affiliated with Jinan University, Zhuhai, China
| |
Collapse
|
4
|
Ayana G, Lee E, Choe SW. Vision Transformers for Breast Cancer Human Epidermal Growth Factor Receptor 2 Expression Staging without Immunohistochemical Staining. THE AMERICAN JOURNAL OF PATHOLOGY 2024; 194:402-414. [PMID: 38096984 DOI: 10.1016/j.ajpath.2023.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 10/10/2023] [Accepted: 11/20/2023] [Indexed: 12/31/2023]
Abstract
Accurate staging of human epidermal growth factor receptor 2 (HER2) expression is vital for evaluating breast cancer treatment efficacy. However, it typically involves costly and complex immunohistochemical staining, along with hematoxylin and eosin staining. This work presents customized vision transformers for staging HER2 expression in breast cancer using only hematoxylin and eosin-stained images. The proposed algorithm comprised three modules: a localization module for weakly localizing critical image features using spatial transformers, an attention module for global learning via vision transformers, and a loss module to determine proximity to a HER2 expression level based on input images by calculating ordinal loss. Results, reported with 95% CIs, reveal the proposed approach's success in HER2 expression staging: area under the receiver operating characteristic curve, 0.9202 ± 0.01; precision, 0.922 ± 0.01; sensitivity, 0.876 ± 0.01; and specificity, 0.959 ± 0.02 over fivefold cross-validation. Comparatively, this approach significantly outperformed conventional vision transformer models and state-of-the-art convolutional neural network models (P < 0.001). Furthermore, it surpassed existing methods when evaluated on an independent test data set. This work holds great importance, aiding HER2 expression staging in breast cancer treatment while circumventing the costly and time-consuming immunohistochemical staining procedure, thereby addressing diagnostic disparities in low-resource settings and low-income countries.
Collapse
Affiliation(s)
- Gelan Ayana
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; School of Biomedical Engineering, Jimma University, Jimma, Ethiopia
| | - Eonjin Lee
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea
| | - Se-Woon Choe
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea; Department of IT Convergence Engineering, Kumoh National Institute of Technology, Gumi, Republic of Korea.
| |
Collapse
|
5
|
Bai Y, Li W, An J, Xia L, Chen H, Zhao G, Gao Z. Masked autoencoders with handcrafted feature predictions: Transformer for weakly supervised esophageal cancer classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107936. [PMID: 38016392 DOI: 10.1016/j.cmpb.2023.107936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 10/28/2023] [Accepted: 11/19/2023] [Indexed: 11/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Esophageal cancer is a serious disease with a high prevalence in Eastern Asia. Histopathology tissue analysis stands as the gold standard in diagnosing esophageal cancer. In recent years, there has been a shift towards digitizing histopathological images into whole slide images (WSIs), progressively integrating them into cancer diagnostics. However, the gigapixel sizes of WSIs present significant storage and processing challenges, and they often lack localized annotations. To address this issue, multi-instance learning (MIL) has been introduced for WSI classification, utilizing weakly supervised learning for diagnosis analysis. By applying the principles of MIL to WSI analysis, it is possible to reduce the workload of pathologists by facilitating the generation of localized annotations. Nevertheless, the approach's effectiveness is hindered by the traditional simple aggregation operation and the domain shift resulting from the prevalent use of convolutional feature extractors pretrained on ImageNet. METHODS We propose a MIL-based framework for WSI analysis and cancer classification. Concurrently, we introduce employing self-supervised learning, which obviates the need for manual annotation and demonstrates versatility in various tasks, to pretrain feature extractors. This method enhances the extraction of representative features from esophageal WSI for MIL, ensuring more robust and accurate performance. RESULTS We build a comprehensive dataset of whole esophageal slide images and conduct extensive experiments utilizing this dataset. The performance on our dataset demonstrates the efficiency of our proposed MIL framework and the pretraining process, with our framework outperforming existing methods, achieving an accuracy of 93.07% and AUC (area under the curve) of 95.31%. CONCLUSION This work proposes an effective MIL method to classify WSI of esophageal cancer. The promising results indicate that our cancer classification framework holds great potential in promoting the automatic whole esophageal slide image analysis.
Collapse
Affiliation(s)
- Yunhao Bai
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Wenqi Li
- Department of Pathology, Key Laboratory of Cancer Prevention and Therapy, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China
| | - Jianpeng An
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Lili Xia
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Huazhen Chen
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Gang Zhao
- Department of Pathology, Key Laboratory of Cancer Prevention and Therapy, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China
| | - Zhongke Gao
- the School of Electrical and Information Engineering, Tianjin University, Tianjin, China.
| |
Collapse
|
6
|
Sun X, Li W, Fu B, Peng Y, He J, Wang L, Yang T, Meng X, Li J, Wang J, Huang P, Wang R. TGMIL: A hybrid multi-instance learning model based on the Transformer and the Graph Attention Network for whole-slide images classification of renal cell carcinoma. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107789. [PMID: 37722310 DOI: 10.1016/j.cmpb.2023.107789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 08/30/2023] [Accepted: 09/01/2023] [Indexed: 09/20/2023]
Abstract
BACKGROUND AND OBJECTIVES The pathological diagnosis of renal cell carcinoma is crucial for treatment. Currently, the multi-instance learning method is commonly used for whole-slide image classification of renal cell carcinoma, which is mainly based on the assumption of independent identical distribution. But this is inconsistent with the need to consider the correlation between different instances in the diagnosis process. Furthermore, the problem of high resource consumption of pathology images is still urgent to be solved. Therefore, we propose a new multi-instance learning method to solve this problem. METHODS In this study, we proposed a hybrid multi-instance learning model based on the Transformer and the Graph Attention Network, called TGMIL, to achieve whole-slide image of renal cell carcinoma classification without pixel-level annotation or region of interest extraction. Our approach is divided into three steps. First, we designed a feature pyramid with the multiple low magnifications of whole-slide image named MMFP. It makes the model incorporates richer information, and reduces memory consumption as well as training time compared to the highest magnification. Second, TGMIL amalgamates the Transformer and the Graph Attention's capabilities, adeptly addressing the loss of instance contextual and spatial. Within the Graph Attention network stream, an easy and efficient approach employing max pooling and mean pooling yields the graph adjacency matrix, devoid of extra memory consumption. Finally, the outputs of two streams of TGMIL are aggregated to achieve the classification of renal cell carcinoma. RESULTS On the TCGA-RCC validation set, a public dataset for renal cell carcinoma, the area under a receiver operating characteristic (ROC) curve (AUC) and accuracy of TGMIL were 0.98±0.0015,0.9191±0.0062, respectively. It showcased remarkable proficiency on the private validation set of renal cell carcinoma pathology images, attaining AUC of 0.9386±0.0162 and ACC of 0.9197±0.0124. Furthermore, on the public breast cancer whole-slide image test dataset, CAMELYON 16, our model showed good classification performance with an accuracy of 0.8792. CONCLUSIONS TGMIL models the diagnostic process of pathologists and shows good classification performance on multiple datasets. Concurrently, the MMFP module efficiently diminishes resource requirements, offering a novel angle for exploring computational pathology images.
Collapse
Affiliation(s)
- Xinhuan Sun
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China; Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Wuchao Li
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Bangkang Fu
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Yunsong Peng
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Junjie He
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China; Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Lihui Wang
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China
| | - Tongyin Yang
- Department of Pathology, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Xue Meng
- Department of Pathology, Affiliated Hospital of Zunyi Medical University, Zunyi, 563000, China
| | - Jin Li
- Department of Pathology, Affiliated Hospital of Zunyi Medical University, Zunyi, 563000, China
| | - Jinjing Wang
- Department of Pathology, Affiliated Hospital of Zunyi Medical University, Zunyi, 563000, China
| | - Ping Huang
- Department of Pathology, Guizhou Provincial People's Hospital, Guiyang, 550002, China
| | - Rongpin Wang
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, Guiyang, 550002, China.
| |
Collapse
|
7
|
Lee J, Han C, Kim K, Park GH, Kwak JT. CaMeL-Net: Centroid-aware metric learning for efficient multi-class cancer classification in pathology images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 241:107749. [PMID: 37579551 DOI: 10.1016/j.cmpb.2023.107749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 07/25/2023] [Accepted: 08/05/2023] [Indexed: 08/16/2023]
Abstract
BACKGROUND AND OBJECTIVE Cancer grading in pathology image analysis is a major task due to its importance in patient care, treatment, and management. The recent developments in artificial neural networks for computational pathology have demonstrated great potential to improve the accuracy and quality of cancer diagnosis. These improvements are generally ascribable to the advance in the architecture of the networks, often leading to increase in the computation and resources. In this work, we propose an efficient convolutional neural network that is designed to conduct multi-class cancer classification in an accurate and robust manner via metric learning. METHODS We propose a centroid-aware metric learning network for an improved cancer grading in pathology images. The proposed network utilizes centroids of different classes within the feature embedding space to optimize the relative distances between pathology images, which manifest the innate similarities/dissimilarities between them. For improved optimization, we introduce a new loss function and a training strategy that are tailored to the proposed network and metric learning. RESULTS We evaluated the proposed approach on multiple datasets of colorectal and gastric cancers. For the colorectal cancer, two different datasets were employed that were collected from different acquisition settings. the proposed method achieved an accuracy, F1-score, quadratic weighted kappa of 88.7%, 0.849, and 0.946 for the first dataset and 83.3%, 0.764, and 0.907 for the second dataset, respectively. For the gastric cancer, the proposed method obtained an accuracy of 85.9%, F1-score of 0.793, and quadratic weighted kappa of 0.939. We also found that the proposed method outperforms other competing models and is computationally efficient. CONCLUSIONS The experimental results demonstrate that the prediction results by the proposed network are both accurate and reliable. The proposed network not only outperformed other related methods in cancer classification but also achieved superior computational efficiency during training and inference. The future study will entail further development of the proposed method and the application of the method to other problems and domains.
Collapse
Affiliation(s)
- Jaeung Lee
- School of Electrical Engineering, Korea University, Seoul, Republic of Korea
| | - Chiwon Han
- Department of Computer Science and Engineering, Sejong University, Seoul, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Gi-Ho Park
- Department of Computer Science and Engineering, Sejong University, Seoul, Republic of Korea
| | - Jin Tae Kwak
- School of Electrical Engineering, Korea University, Seoul, Republic of Korea.
| |
Collapse
|
8
|
Alirezazadeh P, Dornaika F. Boosted Additive Angular Margin Loss for breast cancer diagnosis from histopathological images. Comput Biol Med 2023; 166:107528. [PMID: 37774559 DOI: 10.1016/j.compbiomed.2023.107528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 09/11/2023] [Accepted: 09/19/2023] [Indexed: 10/01/2023]
Abstract
Pathologists use biopsies and microscopic examination to accurately diagnose breast cancer. This process is time-consuming, labor-intensive, and costly. Convolutional neural networks (CNNs) offer an efficient and highly accurate approach to reduce analysis time and automate the diagnostic workflow in pathology. However, the softmax loss commonly used in existing CNNs leads to noticeable ambiguity in decision boundaries and lacks a clear constraint for minimizing within-class variance. In response to this problem, a solution in the form of softmax losses based on angular margin was developed. These losses were introduced in the context of face recognition, with the goal of integrating an angular margin into the softmax loss. This integration improves discrimination features during CNN training by effectively increasing the distance between different classes while reducing the variance within each class. Despite significant progress, these losses are limited to target classes only when margin penalties are applied, which may not lead to optimal effectiveness. In this paper, we introduce Boosted Additive Angular Margin Loss (BAM) to obtain highly discriminative features for breast cancer diagnosis from histopathological images. BAM not only penalizes the angle between deep features and their target class weights, but also considers angles between deep features and non-target class weights. We performed extensive experiments on the publicly available BreaKHis dataset. BAM achieved remarkable accuracies of 99.79%, 99.86%, 99.96%, and 97.65% for magnification levels of 40X, 100X, 200X, and 400X, respectively. These results show an improvement in accuracy of 0.13%, 0.34%, and 0.21% for 40X, 100X, and 200X magnifications, respectively, compared to the baseline methods. Additional experiments were performed on the BACH dataset for breast cancer classification and on the widely accepted LFW and YTF datasets for face recognition to evaluate the generalization ability of the proposed loss function. The results show that BAM outperforms state-of-the-art methods by increasing the decision space between classes and minimizing intra-class variance, resulting in improved discriminability.
Collapse
Affiliation(s)
| | - Fadi Dornaika
- Ho Chi Minh City Open University, Ho Chi Minh City, Viet Nam.
| |
Collapse
|
9
|
Atabansi CC, Nie J, Liu H, Song Q, Yan L, Zhou X. A survey of Transformer applications for histopathological image analysis: New developments and future directions. Biomed Eng Online 2023; 22:96. [PMID: 37749595 PMCID: PMC10518923 DOI: 10.1186/s12938-023-01157-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 09/15/2023] [Indexed: 09/27/2023] Open
Abstract
Transformers have been widely used in many computer vision challenges and have shown the capability of producing better results than convolutional neural networks (CNNs). Taking advantage of capturing long-range contextual information and learning more complex relations in the image data, Transformers have been used and applied to histopathological image processing tasks. In this survey, we make an effort to present a thorough analysis of the uses of Transformers in histopathological image analysis, covering several topics, from the newly built Transformer models to unresolved challenges. To be more precise, we first begin by outlining the fundamental principles of the attention mechanism included in Transformer models and other key frameworks. Second, we analyze Transformer-based applications in the histopathological imaging domain and provide a thorough evaluation of more than 100 research publications across different downstream tasks to cover the most recent innovations, including survival analysis and prediction, segmentation, classification, detection, and representation. Within this survey work, we also compare the performance of CNN-based techniques to Transformers based on recently published papers, highlight major challenges, and provide interesting future research directions. Despite the outstanding performance of the Transformer-based architectures in a number of papers reviewed in this survey, we anticipate that further improvements and exploration of Transformers in the histopathological imaging domain are still required in the future. We hope that this survey paper will give readers in this field of study a thorough understanding of Transformer-based techniques in histopathological image analysis, and an up-to-date paper list summary will be provided at https://github.com/S-domain/Survey-Paper .
Collapse
Affiliation(s)
| | - Jing Nie
- School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044 China
| | - Haijun Liu
- School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044 China
| | - Qianqian Song
- School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044 China
| | - Lingfeng Yan
- School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044 China
| | - Xichuan Zhou
- School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 400044 China
| |
Collapse
|
10
|
Li M, Hu Z, Qiu S, Zhou C, Weng J, Dong Q, Sheng X, Ren N, Zhou M. Dual-branch hybrid encoding embedded network for histopathology image classification. Phys Med Biol 2023; 68:195002. [PMID: 37647919 DOI: 10.1088/1361-6560/acf556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Accepted: 08/30/2023] [Indexed: 09/01/2023]
Abstract
Objective.Learning-based histopathology image (HI) classification methods serve as important tools for auxiliary diagnosis in the prognosis stage. However, most existing methods are focus on a single target cancer due to inter-domain differences among different cancer types, limiting their applicability to different cancer types. To overcome these limitations, this paper presents a high-performance HI classification method that aims to address inter-domain differences and provide an improved solution for reliable and practical HI classification.Approach.Firstly, we collect a high-quality hepatocellular carcinoma (HCC) dataset with enough data to verify the stability and practicability of the method. Secondly, a novel dual-branch hybrid encoding embedded network is proposed, which integrates the feature extraction capabilities of convolutional neural network and Transformer. This well-designed structure enables the network to extract diverse features while minimizing redundancy from a single complex network. Lastly, we develop a salient area constraint loss function tailored to the unique characteristics of HIs to address inter-domain differences and enhance the robustness and universality of the methods.Main results.Extensive experiments have conducted on the proposed HCC dataset and two other publicly available datasets. The proposed method demonstrates outstanding performance with an impressive accuracy of 99.09% on the HCC dataset and achieves state-of-the-art results on the other two public datasets. These remarkable outcomes underscore the superior performance and versatility of our approach in multiple HI classification.Significance.The advancements presented in this study contribute to the field of HI analysis by providing a reliable and practical solution for multiple cancer classification, potentially improving diagnostic accuracy and patient outcomes. Our code is available athttps://github.com/lms-design/DHEE-net.
Collapse
Affiliation(s)
- Mingshuai Li
- Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, 200241, People's Republic of China
| | - Zhiqiu Hu
- Department of Hepatobiliary and Pancreatic Surgery, Minhang Hospital, Fudan University, Shanghai, 201199, People's Republic of China
- Key Laboratory of Whole-Period Monitoring and Precise Intervention of Digestive Cancer of Shanghai Municipal Health Commission, Shanghai, 201199, People's Republic of China
- Institute of Fudan-Minhang Academic Health System, Minhang Hospital, Fudan University, Shanghai, 201199, People's Republic of China
| | - Song Qiu
- Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, 200241, People's Republic of China
- MOE Engineering Research Center of Software/Hardware Co-design Technology and Application, East China Normal University, Shanghai, 200241, People's Republic of China
| | - Chenhao Zhou
- Key Laboratory of Whole-Period Monitoring and Precise Intervention of Digestive Cancer of Shanghai Municipal Health Commission, Shanghai, 201199, People's Republic of China
- Department of Liver Surgery and Transplantation, Liver Cancer Institute, Zhongshan Hospital, Fudan University, Key Laboratory of Carcinogenesis and Cancer Invasion, Ministry of Education, Shanghai, 200032, People's Republic of China
| | - Jialei Weng
- Key Laboratory of Whole-Period Monitoring and Precise Intervention of Digestive Cancer of Shanghai Municipal Health Commission, Shanghai, 201199, People's Republic of China
- Department of Liver Surgery and Transplantation, Liver Cancer Institute, Zhongshan Hospital, Fudan University, Key Laboratory of Carcinogenesis and Cancer Invasion, Ministry of Education, Shanghai, 200032, People's Republic of China
| | - Qiongzhu Dong
- Key Laboratory of Whole-Period Monitoring and Precise Intervention of Digestive Cancer of Shanghai Municipal Health Commission, Shanghai, 201199, People's Republic of China
- Institute of Fudan-Minhang Academic Health System, Minhang Hospital, Fudan University, Shanghai, 201199, People's Republic of China
| | - Xia Sheng
- Department of Pathology, Minhang Hospital, Fudan University, Shanghai, 201199, People's Republic of China
| | - Ning Ren
- Key Laboratory of Whole-Period Monitoring and Precise Intervention of Digestive Cancer of Shanghai Municipal Health Commission, Shanghai, 201199, People's Republic of China
- Institute of Fudan-Minhang Academic Health System, Minhang Hospital, Fudan University, Shanghai, 201199, People's Republic of China
- Department of Liver Surgery and Transplantation, Liver Cancer Institute, Zhongshan Hospital, Fudan University, Key Laboratory of Carcinogenesis and Cancer Invasion, Ministry of Education, Shanghai, 200032, People's Republic of China
| | - Mei Zhou
- Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, 200241, People's Republic of China
| |
Collapse
|
11
|
Wang Z, Liu Y, Niu X. Application of artificial intelligence for improving early detection and prediction of therapeutic outcomes for gastric cancer in the era of precision oncology. Semin Cancer Biol 2023; 93:83-96. [PMID: 37116818 DOI: 10.1016/j.semcancer.2023.04.009] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Revised: 04/12/2023] [Accepted: 04/24/2023] [Indexed: 04/30/2023]
Abstract
Gastric cancer is a leading contributor to cancer incidence and mortality globally. Recently, artificial intelligence approaches, particularly machine learning and deep learning, are rapidly reshaping the full spectrum of clinical management for gastric cancer. Machine learning is formed from computers running repeated iterative models for progressively improving performance on a particular task. Deep learning is a subtype of machine learning on the basis of multilayered neural networks inspired by the human brain. This review summarizes the application of artificial intelligence algorithms to multi-dimensional data including clinical and follow-up information, conventional images (endoscope, histopathology, and computed tomography (CT)), molecular biomarkers, etc. to improve the risk surveillance of gastric cancer with established risk factors; the accuracy of diagnosis, and survival prediction among established gastric cancer patients; and the prediction of treatment outcomes for assisting clinical decision making. Therefore, artificial intelligence makes a profound impact on almost all aspects of gastric cancer from improving diagnosis to precision medicine. Despite this, most established artificial intelligence-based models are in a research-based format and often have limited value in real-world clinical practice. With the increasing adoption of artificial intelligence in clinical use, we anticipate the arrival of artificial intelligence-powered gastric cancer care.
Collapse
Affiliation(s)
- Zhe Wang
- Department of Digestive Diseases 1, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang 110042, Liaoning, China
| | - Yang Liu
- Department of Gastric Surgery, Cancer Hospital of China Medical University, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang 110042, Liaoning, China.
| | - Xing Niu
- China Medical University, Shenyang 110122, Liaoning, China.
| |
Collapse
|
12
|
Liu P, Ji L, Ye F, Fu B. GraphLSurv: A scalable survival prediction network with adaptive and sparse structure learning for histopathological whole-slide images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 231:107433. [PMID: 36841107 DOI: 10.1016/j.cmpb.2023.107433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Revised: 02/17/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Predicting patients' survival from gigapixel Whole-Slide Images (WSIs) has always been a challenging task. To learn effective WSI representations for survival prediction, existing deep learning methods have explored utilizing graphs to describe the complex structure inner WSIs, where graph node is respective to WSI patch. However, these graphs are often densely-connected or static, leading to some redundant or missing patch correlations. Moreover, these methods cannot be directly scaled to the very-large WSI with more than 10,000 patches. To address these, this paper proposes a scalable graph convolution network, GraphLSurv, which can efficiently learn adaptive and sparse structures to better characterize WSIs for survival prediction. METHODS GraphLSurv has three highlights in methodology: (1) it generates adaptive and sparse structures for patches so that latent patch correlations could be captured and adjusted dynamically according to prediction tasks; (2) based on the generated structure and a given graph, GraphLSurv further aggregates local microenvironmental cues into a non-local embedding using the proposed hybrid message passing network; (3) to make this network suitable for very large-scale graphs, it adopts an anchor-based technique to reduce theorical computation complexity. RESULTS The experiments on 2268 WSIs show that GraphLSurv achieves a concordance-index of 0.66132 and 0.68348, with an improvement of 3.79% and 3.41% compared to existing methods, on NLST and TCGA-BRCA, respectively. CONCLUSIONS GraphLSurv could often perform better than previous methods, which suggests that GraphLSurv could provide an important and effective means for WSI survival prediction. Moreover, this work empirically shows that adaptive and sparse structures could be more suitable than static or dense ones for modeling WSIs.
Collapse
Affiliation(s)
- Pei Liu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Xiyuan Ave, Chengdu 611731, Sichuan, China.
| | - Luping Ji
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Xiyuan Ave, Chengdu 611731, Sichuan, China.
| | - Feng Ye
- Institute of Clinical Pathology, West China Hospital, Sichuan University, Guo Xue Xiang, Chengdu 610041, Sichuan, China.
| | - Bo Fu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Xiyuan Ave, Chengdu 611731, Sichuan, China.
| |
Collapse
|
13
|
Shickel B, Lucarelli N, Rao AS, Yun D, Moon KC, Han SS, Sarder P. Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.20.23286044. [PMID: 36865174 PMCID: PMC9980230 DOI: 10.1101/2023.02.20.23286044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that sugest opportunities for future spatially aware WSI research using limited pathology datasets.
Collapse
Affiliation(s)
- Benjamin Shickel
- Dept. of Medicine—Quantitative Health, Univ. of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA; Dept. of Electrical & Computer Engineering, Univ. of Florida, Gainesville, FL, USA
| | | | - Adish S. Rao
- Dept. of Computer and Information Science and Engineering, Univ. of Florida, Gainesville, FL
| | - Donghwan Yun
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Kyung Chul Moon
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Seung Seok Han
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Pinaki Sarder
- Dept. of Medicine—Quantitative Health, Univ. of Florida, Gainesville, FL, USA
- Dept. of Biomedical Engineering, Univ. of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA; Dept. of Electrical & Computer Engineering, Univ. of Florida, Gainesville, FL, USA
| |
Collapse
|
14
|
Sun J, Wu B, Zhao T, Gao L, Xie K, Lin T, Sui J, Li X, Wu X, Ni X. Classification for thyroid nodule using ViT with contrastive learning in ultrasound images. Comput Biol Med 2023; 152:106444. [PMID: 36565481 DOI: 10.1016/j.compbiomed.2022.106444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Revised: 12/01/2022] [Accepted: 12/15/2022] [Indexed: 12/23/2022]
Abstract
The lack of representative features between benign nodules, especially level 3 of Thyroid Imaging Reporting and Data System (TI-RADS), and malignant nodules limits diagnostic accuracy, leading to inconsistent interpretation, overdiagnosis, and unnecessary biopsies. We propose a Vision-Transformer-based (ViT) thyroid nodule classification model using contrast learning, called TC-ViT, to improve accuracy of diagnosis and specificity of biopsy recommendations. ViT can explore the global features of thyroid nodules well. Nodule images are used as ROI to enhance the local features of the ViT. Contrast learning can minimize the representation distance between nodules of the same category, enhance the representation consistency of global and local features, and achieve accurate diagnosis of TI-RADS 3 or malignant nodules. The test results achieve an accuracy of 86.9%. The evaluation metrics show that the network outperforms other classical deep learning-based networks in terms of classification performance. TC-ViT can achieve automatic classification of TI-RADS 3 and malignant nodules on ultrasound images. It can also be used as a key step in computer-aided diagnosis for comprehensive analysis and accurate diagnosis. The code will be available at https://github.com/Jiawei217/TC-ViT.
Collapse
Affiliation(s)
- Jiawei Sun
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China
| | - Bobo Wu
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China
| | - Tong Zhao
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China
| | - Liugang Gao
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China
| | - Kai Xie
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China
| | - Tao Lin
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China
| | - Jianfeng Sui
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China
| | - Xiaoqin Li
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China
| | - Xiaojin Wu
- Oncology Department, Xuzhou NO.1 People's Hospital, Xuzhou 221000, China.
| | - Xinye Ni
- The Affiliated Changzhou NO.2 People's Hospital of Nanjing Medical University, Changzhou 213003, China; Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China; Center of Medical Physics, Nanjing Medical University, Changzhou 213003, China.
| |
Collapse
|
15
|
Yavuz A, Alpsoy A, Gedik EO, Celik MY, Bassorgun CI, Unal B, Elpek GO. Artificial intelligence applications in predicting the behavior of gastrointestinal cancers in pathology. Artif Intell Gastroenterol 2022; 3:142-162. [DOI: 10.35712/aig.v3.i5.142] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Revised: 11/25/2022] [Accepted: 12/14/2022] [Indexed: 12/28/2022] Open
Abstract
Recent research has provided a wealth of data supporting the application of artificial intelligence (AI)-based applications in routine pathology practice. Indeed, it is clear that these methods can significantly support an accurate and rapid diagnosis by eliminating errors, increasing reliability, and improving workflow. In addition, the effectiveness of AI in the pathological evaluation of prognostic parameters associated with behavior, course, and treatment in many types of tumors has also been noted. Regarding gastrointestinal system (GIS) cancers, the contribution of AI methods to pathological diagnosis has been investigated in many studies. On the other hand, studies focusing on AI applications in evaluating parameters to determine tumor behavior are relatively few. For this purpose, the potential of AI models has been studied over a broad spectrum, from tumor subtyping to the identification of new digital biomarkers. The capacity of AI to infer genetic alterations of cancer tissues from digital slides has been demonstrated. Although current data suggest the merit of AI-based approaches in assessing tumor behavior in GIS cancers, a wide range of challenges still need to be solved, from laboratory infrastructure to improving the robustness of algorithms, before incorporating AI applications into real-life GIS pathology practice. This review aims to present data from AI applications in evaluating pathological parameters related to the behavior of GIS cancer with an overview of the opportunities and challenges encountered in implementing AI in pathology.
Collapse
Affiliation(s)
- Aysen Yavuz
- Department of Pathology, Akdeniz University Medical School, Antalya 07070, Turkey
| | - Anil Alpsoy
- Department of Pathology, Akdeniz University Medical School, Antalya 07070, Turkey
| | - Elif Ocak Gedik
- Department of Pathology, Akdeniz University Medical School, Antalya 07070, Turkey
| | | | | | - Betul Unal
- Department of Pathology, Akdeniz University Medical School, Antalya 07070, Turkey
| | - Gulsum Ozlem Elpek
- Department of Pathology, Akdeniz University Medical School, Antalya 07070, Turkey
| |
Collapse
|
16
|
Hu W, Chen H, Liu W, Li X, Sun H, Huang X, Grzegorzek M, Li C. A comparative study of gastric histopathology sub-size image classification: From linear regression to visual transformer. Front Med (Lausanne) 2022; 9:1072109. [PMID: 36569152 PMCID: PMC9767945 DOI: 10.3389/fmed.2022.1072109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 11/18/2022] [Indexed: 12/12/2022] Open
Abstract
Introduction Gastric cancer is the fifth most common cancer in the world. At the same time, it is also the fourth most deadly cancer. Early detection of cancer exists as a guide for the treatment of gastric cancer. Nowadays, computer technology has advanced rapidly to assist physicians in the diagnosis of pathological pictures of gastric cancer. Ensemble learning is a way to improve the accuracy of algorithms, and finding multiple learning models with complementarity types is the basis of ensemble learning. Therefore, this paper compares the performance of multiple algorithms in anticipation of applying ensemble learning to a practical gastric cancer classification problem. Methods The complementarity of sub-size pathology image classifiers when machine performance is insufficient is explored in this experimental platform. We choose seven classical machine learning classifiers and four deep learning classifiers for classification experiments on the GasHisSDB database. Among them, classical machine learning algorithms extract five different image virtual features to match multiple classifier algorithms. For deep learning, we choose three convolutional neural network classifiers. In addition, we also choose a novel Transformer-based classifier. Results The experimental platform, in which a large number of classical machine learning and deep learning methods are performed, demonstrates that there are differences in the performance of different classifiers on GasHisSDB. Classical machine learning models exist for classifiers that classify Abnormal categories very well, while classifiers that excel in classifying Normal categories also exist. Deep learning models also exist with multiple models that can be complementarity. Discussion Suitable classifiers are selected for ensemble learning, when machine performance is insufficient. This experimental platform demonstrates that multiple classifiers are indeed complementarity and can improve the efficiency of ensemble learning. This can better assist doctors in diagnosis, improve the detection of gastric cancer, and increase the cure rate.
Collapse
Affiliation(s)
- Weiming Hu
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Haoyuan Chen
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Wanli Liu
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Xiaoyan Li
- Department of Pathology, Liaoning Cancer Hospital and Institute, Cancer Hospital, China Medical University, Shenyang, China
| | - Hongzan Sun
- Department of Radiology, Shengjing Hospital, China Medical University, Shenyang, China
| | - Xinyu Huang
- Institute of Medical Informatics, University of Luebeck, Luebeck, Germany
| | - Marcin Grzegorzek
- Institute of Medical Informatics, University of Luebeck, Luebeck, Germany
- Department of Knowledge Engineering, University of Economics in Katowice, Katowice, Poland
| | - Chen Li
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| |
Collapse
|
17
|
Li H, Wu P, Wang Z, Mao J, Alsaadi FE, Zeng N. A generalized framework of feature learning enhanced convolutional neural network for pathology-image-oriented cancer diagnosis. Comput Biol Med 2022; 151:106265. [PMID: 36401968 DOI: 10.1016/j.compbiomed.2022.106265] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 10/24/2022] [Accepted: 10/30/2022] [Indexed: 11/11/2022]
Abstract
In this paper, a feature learning enhanced convolutional neural network (FLE-CNN) is proposed for cancer detection from histopathology images. To build a highly generalized computer-aided diagnosis (CAD) system, an information refinement unit employing depth- and point-wise convolutions is meticulously designed, where a dual-domain attention mechanism is adopted to focus primarily on the important areas. By deploying a residual fusion unit, context information is further integrated to extract highly discriminative features with strong representation ability. Experimental results demonstrate the merits of the proposed FLE-CNN in terms of feature extraction, which has achieved average sensitivity, specificity, precision, accuracy and F1 score of 0.9992, 0.9998, 0.9992, 0.9997 and 0.9992 in a five-class cancer detection task, and in comparison to some other advanced deep learning models, above indicators have been improved by 1.23%, 0.31%, 1.24%, 0.5% and 1.26%, respectively. Moreover, the proposed FLE-CNN provides satisfactory results in three important diagnosis, which further validates that FLE-CNN is a competitive CAD model with high generalization ability.
Collapse
Affiliation(s)
- Han Li
- Department of Instrumental and Electrical Engineering, Xiamen University, Fujian 361005, China
| | - Peishu Wu
- Department of Instrumental and Electrical Engineering, Xiamen University, Fujian 361005, China
| | - Zidong Wang
- Department of Computer Science, Brunel University London, Uxbridge UB8 3PH, UK.
| | - Jingfeng Mao
- School of Electrical Engineering, Nantong University, Nantong 226019, China
| | - Fuad E Alsaadi
- Communication Systems and Networks Research Group, Department of Electrical and Computer Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Nianyin Zeng
- Department of Instrumental and Electrical Engineering, Xiamen University, Fujian 361005, China.
| |
Collapse
|