1
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
2
|
Ogundipe O, Kurt Z, Woo WL. Deep neural networks integrating genomics and histopathological images for predicting stages and survival time-to-event in colon cancer. PLoS One 2024; 19:e0305268. [PMID: 39226289 PMCID: PMC11371203 DOI: 10.1371/journal.pone.0305268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 05/26/2024] [Indexed: 09/05/2024] Open
Abstract
MOTIVATION There exists an unexplained diverse variation within the predefined colon cancer stages using only features from either genomics or histopathological whole slide images as prognostic factors. Unraveling this variation will bring about improved staging and treatment outcomes. Hence, motivated by the advancement of Deep Neural Network (DNN) libraries and complementary factors within some genomics datasets, we aggregate atypia patterns in histopathological images with diverse carcinogenic expression from mRNA, miRNA and DNA methylation as an integrative input source into a deep neural network for colon cancer stages classification, and samples stratification into low or high-risk survival groups. RESULTS The genomics-only and integrated input features return Area Under Curve-Receiver Operating Characteristic curve (AUC-ROC) of 0.97 compared with AUC-ROC of 0.78 obtained when only image features are used for the stage's classification. A further analysis of prediction accuracy using the confusion matrix shows that the integrated features have a weakly improved accuracy of 0.08% more than the accuracy obtained with genomics features. Also, the extracted features were used to split the patients into low or high-risk survival groups. Among the 2,700 fused features, 1,836 (68%) features showed statistically significant survival probability differences in aggregating samples into either low or high between the two risk survival groups. Availability and Implementation: https://github.com/Ogundipe-L/EDCNN.
Collapse
Affiliation(s)
- Olalekan Ogundipe
- Department of Computer and Information Sciences, University of Northumbria, Newcastle Upon Tyne, United Kingdom
| | - Zeyneb Kurt
- Information School, University of Sheffield, Sheffield, United Kingdom
| | - Wai Lok Woo
- Department of Computer and Information Sciences, University of Northumbria, Newcastle Upon Tyne, United Kingdom
| |
Collapse
|
3
|
Jiang W, Wang H, Dong X, Zhao Y, Long C, Chen D, Yan B, Cheng J, Lin Z, Zhuo S, Wang H, Yan J. Association of the pathomics-collagen signature with lymph node metastasis in colorectal cancer: a retrospective multicenter study. J Transl Med 2024; 22:103. [PMID: 38273371 PMCID: PMC10811897 DOI: 10.1186/s12967-024-04851-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 01/02/2024] [Indexed: 01/27/2024] Open
Abstract
BACKGROUND Lymph node metastasis (LNM) is a prognostic biomarker and affects therapeutic selection in colorectal cancer (CRC). Current evaluation methods are not adequate for estimating LNM in CRC. H&E images contain much pathological information, and collagen also affects the biological behavior of tumor cells. Hence, the objective of the study is to investigate whether a fully quantitative pathomics-collagen signature (PCS) in the tumor microenvironment can be used to predict LNM. METHODS Patients with histologically confirmed stage I-III CRC who underwent radical surgery were included in the training cohort (n = 329), the internal validation cohort (n = 329), and the external validation cohort (n = 315). Fully quantitative pathomics features and collagen features were extracted from digital H&E images and multiphoton images of specimens, respectively. LASSO regression was utilized to develop the PCS. Then, a PCS-nomogram was constructed incorporating the PCS and clinicopathological predictors for estimating LNM in the training cohort. The performance of the PCS-nomogram was evaluated via calibration, discrimination, and clinical usefulness. Furthermore, the PCS-nomogram was tested in internal and external validation cohorts. RESULTS By LASSO regression, the PCS was developed based on 11 pathomics and 9 collagen features. A significant association was found between the PCS and LNM in the three cohorts (P < 0.001). Then, the PCS-nomogram based on PCS, preoperative CEA level, lymphadenectasis on CT, venous emboli and/or lymphatic invasion and/or perineural invasion (VELIPI), and pT stage achieved AUROCs of 0.939, 0.895, and 0.893 in the three cohorts. The calibration curves identified good agreement between the nomogram-predicted and actual outcomes. Decision curve analysis indicated that the PCS-nomogram was clinically useful. Moreover, the PCS was still an independent predictor of LNM at station Nos. 1, 2, and 3. The PCS nomogram displayed AUROCs of 0.849-0.939 for the training cohort, 0.837-0.902 for the internal validation cohort, and 0.851-0.895 for the external validation cohorts in the three nodal stations. CONCLUSIONS This study proposed that PCS integrating pathomics and collagen features was significantly associated with LNM, and the PCS-nomogram has the potential to be a useful tool for predicting individual LNM in CRC patients.
Collapse
Affiliation(s)
- Wei Jiang
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
- School of Science, Jimei University, Xiamen, Fujian, 361021, People's Republic of China
| | - Huaiming Wang
- Department of General Surgery (Colorectal Surgery), The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, Guangdong, 510655, People's Republic of China
- Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, Guangdong, 510655, People's Republic of China
- Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, Guangdong, 510655, People's Republic of China
| | - Xiaoyu Dong
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Yandong Zhao
- Department of Pathology, The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, Guangdong, 510655, People's Republic of China
| | - Chenyan Long
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
- Division of Colorectal and Anal Surgery, Department of Gastrointestinal Surgery, Guangxi Medical University Cancer Hospital, Nanning, 530000, People's Republic of China
| | - Dexin Chen
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Botao Yan
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Jiaxin Cheng
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Zexi Lin
- School of Science, Jimei University, Xiamen, Fujian, 361021, People's Republic of China
| | - Shuangmu Zhuo
- School of Science, Jimei University, Xiamen, Fujian, 361021, People's Republic of China.
| | - Hui Wang
- Department of General Surgery (Colorectal Surgery), The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, Guangdong, 510655, People's Republic of China.
| | - Jun Yan
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical Medicine, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.
- Department of Gastrointestinal Surgery, Shenzhen People's Hospital, Second Clinical Medical College of Jinan University, First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, Guangdong, 518020, People's Republic of China.
| |
Collapse
|
4
|
Li JW, Wang LM, Ichimasa K, Lin KW, Ngu JCY, Ang TL. Use of artificial intelligence in the management of T1 colorectal cancer: a new tool in the arsenal or is deep learning out of its depth? Clin Endosc 2024; 57:24-35. [PMID: 37743068 PMCID: PMC10834280 DOI: 10.5946/ce.2023.036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 05/11/2023] [Indexed: 09/26/2023] Open
Abstract
The field of artificial intelligence is rapidly evolving, and there has been an interest in its use to predict the risk of lymph node metastasis in T1 colorectal cancer. Accurately predicting lymph node invasion may result in fewer patients undergoing unnecessary surgeries; conversely, inadequate assessments will result in suboptimal oncological outcomes. This narrative review aims to summarize the current literature on deep learning for predicting the probability of lymph node metastasis in T1 colorectal cancer, highlighting areas of potential application and barriers that may limit its generalizability and clinical utility.
Collapse
Affiliation(s)
- James Weiquan Li
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - Lai Mun Wang
- Department of Laboratory Medicine, Changi General Hospital, Singapore Health Services, Singapore
| | - Katsuro Ichimasa
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Kenneth Weicong Lin
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| | - James Chi-Yong Ngu
- Department of General Surgery, Changi General Hospital, Singapore Health Services, Singapore
| | - Tiing Leong Ang
- Department of Gastroenterology and Hepatology, Changi General Hospital, Singapore Health Services, Singapore
- Academic Medicine Center, Duke-NUS Medical School, Singapore
| |
Collapse
|
5
|
Griem J, Eich ML, Schallenberg S, Pryalukhin A, Bychkov A, Fukuoka J, Zayats V, Hulla W, Munkhdelger J, Seper A, Tsvetkov T, Mukhopadhyay A, Sanner A, Stieber J, Fuchs M, Babendererde N, Schömig-Markiefka B, Klein S, Buettner R, Quaas A, Tolkach Y. Artificial Intelligence-Based Tool for Tumor Detection and Quantitative Tissue Analysis in Colorectal Specimens. Mod Pathol 2023; 36:100327. [PMID: 37683932 DOI: 10.1016/j.modpat.2023.100327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 08/11/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
Digital pathology adoption allows for applying computational algorithms to routine pathology tasks. Our study aimed to develop a clinical-grade artificial intelligence (AI) tool for precise multiclass tissue segmentation in colorectal specimens (resections and biopsies) and clinically validate the tool for tumor detection in biopsy specimens. The training data set included 241 precisely manually annotated whole-slide images (WSIs) from multiple institutions. The algorithm was trained for semantic segmentation of 11 tissue classes with an additional module for biopsy WSI classification. Six case cohorts from 5 pathology departments (4 countries) were used for formal and clinical validation, digitized by 4 different scanning systems. The developed algorithm showed high precision of segmentation of different tissue classes in colorectal specimens with composite multiclass Dice score of up to 0.895 and pixel-wise tumor detection specificity and sensitivity of up to 0.958 and 0.987, respectively. In the clinical validation study on multiple external cohorts, the AI tool reached sensitivity of 1.0 and specificity of up to 0.969 for tumor detection in biopsy WSI. The AI tool analyzes most biopsy cases in less than 1 minute, allowing effective integration into clinical routine. We developed and extensively validated a highly accurate, clinical-grade tool for assistive diagnostic processing of colorectal specimens. This tool allows for quantitative deciphering of colorectal cancer tissue for development of prognostic and predictive biomarkers and personalization of oncologic care. This study is a foundation for a SemiCOL computational challenge. We open-source multiple manually annotated and weakly labeled test data sets, representing a significant contribution to the colorectal cancer computational pathology field.
Collapse
Affiliation(s)
- Johanna Griem
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | - Alexey Pryalukhin
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | - Andrey Bychkov
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Junya Fukuoka
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Vitaliy Zayats
- Laboratory for Medical Artificial Intelligence, The Resource Center for Universal Design and Rehabilitation Technologies (RCUD and RT), Moscow, Russia
| | - Wolfgang Hulla
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | | | - Alexander Seper
- Danube Private University, Medical Faculty, Krems-Stein, Austria
| | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | | | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | | | | | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Buettner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|
6
|
Jing Y, Li C, Du T, Jiang T, Sun H, Yang J, Shi L, Gao M, Grzegorzek M, Li X. A comprehensive survey of intestine histopathological image analysis using machine vision approaches. Comput Biol Med 2023; 165:107388. [PMID: 37696178 DOI: 10.1016/j.compbiomed.2023.107388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 08/06/2023] [Accepted: 08/25/2023] [Indexed: 09/13/2023]
Abstract
Colorectal Cancer (CRC) is currently one of the most common and deadly cancers. CRC is the third most common malignancy and the fourth leading cause of cancer death worldwide. It ranks as the second most frequent cause of cancer-related deaths in the United States and other developed countries. Histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of CRC. In order to improve the objectivity and diagnostic efficiency for image analysis of intestinal histopathology, Computer-aided Diagnosis (CAD) methods based on machine learning (ML) are widely applied in image analysis of intestinal histopathology. In this investigation, we conduct a comprehensive study on recent ML-based methods for image analysis of intestinal histopathology. First, we discuss commonly used datasets from basic research studies with knowledge of intestinal histopathology relevant to medicine. Second, we introduce traditional ML methods commonly used in intestinal histopathology, as well as deep learning (DL) methods. Then, we provide a comprehensive review of the recent developments in ML methods for segmentation, classification, detection, and recognition, among others, for histopathological images of the intestine. Finally, the existing methods have been studied, and the application prospects of these methods in this field are given.
Collapse
Affiliation(s)
- Yujie Jing
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China
| | - Chen Li
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China.
| | - Tianming Du
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China
| | - Tao Jiang
- School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China; International Joint Institute of Robotics and Intelligent Systems, Chengdu University of Information Technology, Chengdu, China
| | - Hongzan Sun
- Shengjing Hospital of China Medical University, Shenyang, China
| | - Jinzhu Yang
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China
| | - Liyu Shi
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China
| | - Minghe Gao
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, China
| | - Marcin Grzegorzek
- Institute for Medical Informatics, University of Luebeck, Luebeck, Germany; Department of Knowledge Engineering, University of Economics in Katowice, Katowice, Poland
| | - Xiaoyan Li
- Cancer Hospital of China Medical University, Liaoning Cancer Hospital, Shenyang, China.
| |
Collapse
|
7
|
Sahoo PK, Gupta P, Lai YC, Chiang SF, You JF, Onthoni DD, Chern YJ. Localization of Colorectal Cancer Lesions in Contrast-Computed Tomography Images via a Deep Learning Approach. Bioengineering (Basel) 2023; 10:972. [PMID: 37627857 PMCID: PMC10451186 DOI: 10.3390/bioengineering10080972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 07/26/2023] [Accepted: 07/31/2023] [Indexed: 08/27/2023] Open
Abstract
Abdominal computed tomography (CT) is a frequently used imaging modality for evaluating gastrointestinal diseases. The detection of colorectal cancer is often realized using CT before a more invasive colonoscopy. When a CT exam is performed for indications other than colorectal evaluation, the tortuous structure of the long, tubular colon makes it difficult to analyze the colon carefully and thoroughly. In addition, the sensitivity of CT in detecting colorectal cancer is greatly dependent on the size of the tumor. Missed incidental colon cancers using CT are an emerging problem for clinicians and radiologists; consequently, the automatic localization of lesions in the CT images of unprepared bowels is needed. Therefore, this study used artificial intelligence (AI) to localize colorectal cancer in CT images. We enrolled 190 colorectal cancer patients to obtain 1558 tumor slices annotated by radiologists and colorectal surgeons. The tumor sites were double-confirmed via colonoscopy or other related examinations, including physical examination or image study, and the final tumor sites were obtained from the operation records if available. The localization and training models used were RetinaNet, YOLOv3, and YOLOv8. We achieved an F1 score of 0.97 (±0.002), a mAP of 0.984 when performing slice-wise testing, 0.83 (±0.29) sensitivity, 0.97 (±0.01) specificity, and 0.96 (±0.01) accuracy when performing patient-wise testing using our derived model YOLOv8 with hyperparameter tuning.
Collapse
Affiliation(s)
- Prasan Kumar Sahoo
- Department of Computer Science and Information Engineering, Chang Gung University, Guishan, Taoyuan 33302, Taiwan; (P.K.S.); (P.G.); (D.D.O.)
- Department of Neurology, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan
| | - Pushpanjali Gupta
- Department of Computer Science and Information Engineering, Chang Gung University, Guishan, Taoyuan 33302, Taiwan; (P.K.S.); (P.G.); (D.D.O.)
| | - Ying-Chieh Lai
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan;
- Department of Metabolomics Core Lab, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan
| | - Sum-Fu Chiang
- Division of Colon and Rectal Surgery, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan; (S.-F.C.); (J.-F.Y.)
- College of Medicine, Chang Gung University, Guishan, Taoyuan 33302, Taiwan
| | - Jeng-Fu You
- Division of Colon and Rectal Surgery, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan; (S.-F.C.); (J.-F.Y.)
- College of Medicine, Chang Gung University, Guishan, Taoyuan 33302, Taiwan
| | - Djeane Debora Onthoni
- Department of Computer Science and Information Engineering, Chang Gung University, Guishan, Taoyuan 33302, Taiwan; (P.K.S.); (P.G.); (D.D.O.)
| | - Yih-Jong Chern
- Division of Colon and Rectal Surgery, Chang Gung Memorial Hospital, Linkou, New Taipei City 33305, Taiwan; (S.-F.C.); (J.-F.Y.)
- Graduate Institute of Clinical Medical Sciences, College of Medicine, Chang Gung University, Guishan, Taoyuan 33302, Taiwan
| |
Collapse
|
8
|
Chlorogiannis DD, Verras GI, Tzelepi V, Chlorogiannis A, Apostolos A, Kotis K, Anagnostopoulos CN, Antzoulas A, Davakis S, Vailas M, Schizas D, Mulita F. Tissue classification and diagnosis of colorectal cancer histopathology images using deep learning algorithms. Is the time ripe for clinical practice implementation? PRZEGLAD GASTROENTEROLOGICZNY 2023; 18:353-367. [PMID: 38572457 PMCID: PMC10985751 DOI: 10.5114/pg.2023.130337] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 05/20/2023] [Indexed: 04/05/2024]
Abstract
Colorectal cancer is one of the most prevalent types of cancer, with histopathologic examination of biopsied tissue samples remaining the gold standard for diagnosis. During the past years, artificial intelligence (AI) has steadily found its way into the field of medicine and pathology, especially with the introduction of whole slide imaging (WSI). The main outcome of interest was the composite balanced accuracy (ACC) as well as the F1 score. The average reported ACC from the collected studies was 95.8 ±3.8%. Reported F1 scores reached as high as 0.975, with an average of 89.7 ±9.8%, indicating that existing deep learning algorithms can achieve in silico distinction between malignant and benign. Overall, the available state-of-the-art algorithms are non-inferior to pathologists for image analysis and classification tasks. However, due to their inherent uniqueness in their training and lack of widely accepted external validation datasets, their generalization potential is still limited.
Collapse
Affiliation(s)
| | | | - Vasiliki Tzelepi
- Department of Pathology, School of Medicine, University of Patras, Patras, Greece
| | | | - Anastasios Apostolos
- First Department of Cardiology, Hippokration Hospital, University of Athens, Athens, Greece
| | - Konstantinos Kotis
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, Mytilene, Greece
| | | | - Andreas Antzoulas
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| | - Spyridon Davakis
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Michail Vailas
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Dimitrios Schizas
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Francesk Mulita
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| |
Collapse
|
9
|
Bousis D, Verras GI, Bouchagier K, Antzoulas A, Panagiotopoulos I, Katinioti A, Kehagias D, Kaplanis C, Kotis K, Anagnostopoulos CN, Mulita F. The role of deep learning in diagnosing colorectal cancer. PRZEGLAD GASTROENTEROLOGICZNY 2023; 18:266-273. [PMID: 37937113 PMCID: PMC10626379 DOI: 10.5114/pg.2023.129494] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 01/15/2023] [Accepted: 02/24/2023] [Indexed: 11/09/2023]
Abstract
Colon cancer is a major public health issue, affecting a growing number of individuals worldwide. Proper and early diagnosis of colon cancer is the necessary first step toward effective treatment and/or prevention of future disease relapse. Artificial intelligence and its subtypes, deep learning in particular, tend nowadays to have an expanding role in all fields of medicine, and diagnosing colon cancer is no exception. This report aims to summarize the entire application spectrum of deep learning in all diagnostic tests regarding colon cancer, from endoscopy and histologic examination to medical imaging and screening serologic tests.
Collapse
Affiliation(s)
- Dimitrios Bousis
- Department of Internal Medicine, General University Hospital of Patras, Patras, Greece
| | | | | | - Andreas Antzoulas
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| | | | | | - Dimitrios Kehagias
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| | | | - Konstantinos Kotis
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, Mytilene, Greece
| | | | - Francesk Mulita
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| |
Collapse
|
10
|
Li S, Zhu X, Zhang L, Huang C, Li D. The effect of pain-education nursing based on a mind map on postoperative pain score and quality of life in patients with colorectal cancer. Medicine (Baltimore) 2023; 102:e33562. [PMID: 37171350 PMCID: PMC10174362 DOI: 10.1097/md.0000000000033562] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/13/2023] Open
Abstract
Mind mapping is a combination of images and logical thinking in the form of pictures and texts. It presents a natural expression process that conforms to the divergent thinking of the brain, helping patients understand memory. This study aimed to investigate the impact of pain-education nursing with the aid of mind maps on postoperative pain scores and quality of life in patients with colorectal cancer. The medical records of 100 colorectal cancer patients treated from July 2020 to July 2022 were selected as retrospective research participants and divided into control and observation groups. The control group received routine pain care, and the observation group was given pain-education nursing based on a mind map. The effects on quality of life, pain, negative emotions, and comfort after nursing were compared between the 2 groups. There was no significant difference in quality of-life scores between the 2 groups before the nursing intervention (P > .05). After nursing, the physical, emotional, cognitive, and social functions of patients in the observation group were significantly higher than those in the control group. However, the overall quality of life and financial difficulties of patients in the observation group were significantly lower than those in the control group (P < .05). The observation group's social, physical, and psychological comfort scores were significantly higher than those in the control group after nursing intervention (P < .05). After nursing, the observation group's visual analog scale scores at different times after the operation were significantly lower than those of the control group. The scores on the self-rating anxiety scale and self-rating depression scale in the observation group were significantly lower than those in the control group (P < .05). Pain-education nursing based on mind maps can effectively improve the daily living ability, pain symptoms, and quality of life of patients after colorectal cancer, and there is a reference value for post-nursing care of patients after colorectal cancer.
Collapse
Affiliation(s)
- Shan Li
- Pain Department Hand Surgery, Wuhan Fourth Hospital, Wuhan, Hubei, China
| | - Xiaohong Zhu
- Department of Surgery, Central Hospital of Enshi Tujia and Miao Autonomous Prefecture, Enshi City, Hubei Province, China
| | - Lihua Zhang
- Pain Department Hand Surgery, Wuhan Fourth Hospital, Wuhan, Hubei, China
| | - Cui Huang
- Urology Gastrointestinal Surgery, Wuhan Fourth Hospital, Wuhan Fourth Hospital, Wuhan, Hubei, China
| | - Dan Li
- Urology Gastrointestinal Surgery, Wuhan Fourth Hospital, Wuhan Fourth Hospital, Wuhan, Hubei, China
| |
Collapse
|
11
|
Wong ANN, He Z, Leung KL, To CCK, Wong CY, Wong SCC, Yoo JS, Chan CKR, Chan AZ, Lacambra MD, Yeung MHY. Current Developments of Artificial Intelligence in Digital Pathology and Its Future Clinical Applications in Gastrointestinal Cancers. Cancers (Basel) 2022; 14:3780. [PMID: 35954443 PMCID: PMC9367360 DOI: 10.3390/cancers14153780] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 07/27/2022] [Accepted: 08/01/2022] [Indexed: 02/05/2023] Open
Abstract
The implementation of DP will revolutionize current practice by providing pathologists with additional tools and algorithms to improve workflow. Furthermore, DP will open up opportunities for development of AI-based tools for more precise and reproducible diagnosis through computational pathology. One of the key features of AI is its capability to generate perceptions and recognize patterns beyond the human senses. Thus, the incorporation of AI into DP can reveal additional morphological features and information. At the current rate of AI development and adoption of DP, the interest in computational pathology is expected to rise in tandem. There have already been promising developments related to AI-based solutions in prostate cancer detection; however, in the GI tract, development of more sophisticated algorithms is required to facilitate histological assessment of GI specimens for early and accurate diagnosis. In this review, we aim to provide an overview of the current histological practices in AP laboratories with respect to challenges faced in image preprocessing, present the existing AI-based algorithms, discuss their limitations and present clinical insight with respect to the application of AI in early detection and diagnosis of GI cancer.
Collapse
Affiliation(s)
- Alex Ngai Nick Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Zebang He
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Ka Long Leung
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Curtis Chun Kit To
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Chun Yin Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Sze Chuen Cesar Wong
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Jung Sun Yoo
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| | - Cheong Kin Ronald Chan
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Angela Zaneta Chan
- Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, Shatin, Hong Kong SAR, China;
| | - Maribel D. Lacambra
- Department of Anatomical and Cellular Pathology, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong SAR, China; (C.C.K.T.); (C.K.R.C.); (M.D.L.)
| | - Martin Ho Yin Yeung
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China; (A.N.N.W.); (Z.H.); (K.L.L.); (C.Y.W.); (S.C.C.W.); (J.S.Y.)
| |
Collapse
|
12
|
Special Issue: “Machine Learning for Computer-Aided Diagnosis in Biomedical Imaging”. Diagnostics (Basel) 2022; 12:diagnostics12061331. [PMID: 35741141 PMCID: PMC9222049 DOI: 10.3390/diagnostics12061331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 05/25/2022] [Indexed: 02/04/2023] Open
|
13
|
Xiao C, Zhou M, Yang X, Wang H, Tang Z, Zhou Z, Tian Z, Liu Q, Li X, Jiang W, Luo J. Accurate Prediction of Metachronous Liver Metastasis in Stage I-III Colorectal Cancer Patients Using Deep Learning With Digital Pathological Images. Front Oncol 2022; 12:844067. [PMID: 35433467 PMCID: PMC9010865 DOI: 10.3389/fonc.2022.844067] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2021] [Accepted: 03/10/2022] [Indexed: 11/13/2022] Open
Abstract
ObjectivesMetachronous liver metastasis (LM) significantly impacts the prognosis of stage I-III colorectal cancer (CRC) patients. An effective biomarker to predict LM after surgery is urgently needed. We aimed to develop deep learning-based models to assist in predicting LM in stage I-III CRC patients using digital pathological images.MethodsSix-hundred eleven patients were retrospectively included in the study and randomly divided into training (428 patients) and validation (183 patients) cohorts according to the 7:3 ratio. Digital HE images from training cohort patients were used to construct the LM risk score based on a 50-layer residual convolutional neural network (ResNet-50). An LM prediction model was established by multivariable Cox analysis and confirmed in the validation cohort. The performance of the integrated nomogram was assessed with respect to its calibration, discrimination, and clinical application value.ResultsPatients were divided into low- and high-LM risk score groups according to the cutoff value and significant differences were observed in the LM of the different risk score groups in the training and validation cohorts (P<0.001). Multivariable analysis revealed that the LM risk score, VELIPI, pT stage and pN stage were independent predictors of LM. Then, the prediction model was developed and presented as a nomogram to predict the 1-, 2-, and 3-year probability of LM. The integrated nomogram achieved satisfactory discrimination, with C-indexes of 0.807 (95% CI: 0.787, 0.827) and 0.812 (95% CI: 0.773, 0.850) and AUCs of 0.840 (95% CI: 0.795, 0.885) and 0.848 (95% CI: 0.766, 0.931) in the training and validation cohorts, respectively. Favorable calibration of the nomogram was confirmed in the training and validation cohorts. Integrated discrimination improvement and net reclassification index indicated that the integrated nomogram was superior to the traditional clinicopathological model. Decision curve analysis confirmed that the nomogram has clinical application value.ConclusionsThe LM risk score based on ResNet-50 and digital HE images was significantly associated with LM. The integrated nomogram could identify stage I-III CRC patients at high risk of LM after primary colectomy, so it may serve as a potential tool to choose the appropriate treatment to improve the prognosis of stage I-III CRC patients.
Collapse
Affiliation(s)
- Chanchan Xiao
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
- Department of Microbiology and Immunology, Institute of Geriatric Immunology, School of Medicine, Jinan University, Guangzhou, China
| | - Meihua Zhou
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
| | - Xihua Yang
- Department of Surgical Oncology, Chenzhou No. 1 People’s Hospital, Chenzhou, China
| | - Haoyun Wang
- Department of Microbiology and Immunology, Institute of Geriatric Immunology, School of Medicine, Jinan University, Guangzhou, China
| | - Zhen Tang
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
| | - Zheng Zhou
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
| | - Zeyu Tian
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
| | - Qi Liu
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
| | - Xiaojie Li
- Department of Pathology, Chenzhou No. 1 People’s Hospital, Chenzhou, China
| | - Wei Jiang
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
- Department of Surgical Oncology, Chenzhou No. 1 People’s Hospital, Chenzhou, China
- *Correspondence: Jihui Luo, ; Wei Jiang,
| | - Jihui Luo
- Department of General Surgery, Hunan Provincial People’s Hospital (The First-Affiliated Hospital of Hunan Normal University), Changsha, China
- *Correspondence: Jihui Luo, ; Wei Jiang,
| |
Collapse
|
14
|
Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review. Diagnostics (Basel) 2022; 12:diagnostics12040837. [PMID: 35453885 PMCID: PMC9028395 DOI: 10.3390/diagnostics12040837] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 02/04/2023] Open
Abstract
Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.
Collapse
|