1
|
Fu X, Ma W, Zuo Q, Qi Y, Zhang S, Zhao Y. Application of machine learning for high-throughput tumor marker screening. Life Sci 2024; 348:122634. [PMID: 38685558 DOI: 10.1016/j.lfs.2024.122634] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 03/26/2024] [Accepted: 04/10/2024] [Indexed: 05/02/2024]
Abstract
High-throughput sequencing and multiomics technologies have allowed increasing numbers of biomarkers to be mined and used for disease diagnosis, risk stratification, efficacy assessment, and prognosis prediction. However, the large number and complexity of tumor markers make screening them a substantial challenge. Machine learning (ML) offers new and effective ways to solve the screening problem. ML goes beyond mere data processing and is instrumental in recognizing intricate patterns within data. ML also has a crucial role in modeling dynamic changes associated with diseases. Used together, ML techniques have been included in automatic pipelines for tumor marker screening, thereby enhancing the efficiency and accuracy of the screening process. In this review, we discuss the general processes and common ML algorithms, and highlight recent applications of ML in tumor marker screening of genomic, transcriptomic, proteomic, and metabolomic data of patients with various types of cancers. Finally, the challenges and future prospects of the application of ML in tumor therapy are discussed.
Collapse
Affiliation(s)
- Xingxing Fu
- Key Laboratory of Biotechnology and Bioresources Utilization of Ministry of Education, Dalian Minzu University, Dalian 116600, China
| | - Wanting Ma
- Key Laboratory of Biotechnology and Bioresources Utilization of Ministry of Education, Dalian Minzu University, Dalian 116600, China
| | - Qi Zuo
- Key Laboratory of Biotechnology and Bioresources Utilization of Ministry of Education, Dalian Minzu University, Dalian 116600, China
| | - Yanfei Qi
- Centenary Institute, The University of Sydney, Sydney, NSW 2050, Australia
| | - Shubiao Zhang
- Key Laboratory of Biotechnology and Bioresources Utilization of Ministry of Education, Dalian Minzu University, Dalian 116600, China.
| | - Yinan Zhao
- Key Laboratory of Biotechnology and Bioresources Utilization of Ministry of Education, Dalian Minzu University, Dalian 116600, China
| |
Collapse
|
2
|
Wang L, Zhang Q, Zhang P, Wu B, Chen J, Gong J, Tang K, Du S, Li S. Development of an artificial intelligent model for pre-endoscopic screening of precancerous lesions in gastric cancer. Chin Med 2024; 19:90. [PMID: 38951913 PMCID: PMC11218324 DOI: 10.1186/s13020-024-00963-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 06/18/2024] [Indexed: 07/03/2024] Open
Abstract
BACKGROUND Given the high cost of endoscopy in gastric cancer (GC) screening, there is an urgent need to explore cost-effective methods for the large-scale prediction of precancerous lesions of gastric cancer (PLGC). We aim to construct a hierarchical artificial intelligence-based multimodal non-invasive method for pre-endoscopic risk screening, to provide tailored recommendations for endoscopy. METHODS From December 2022 to December 2023, a large-scale screening study was conducted in Fujian, China. Based on traditional Chinese medicine theory, we simultaneously collected tongue images and inquiry information from 1034 participants, considering the potential of these data for PLGC screening. Then, we introduced inquiry information for the first time, forming a multimodality artificial intelligence model to integrate tongue images and inquiry information for pre-endoscopic screening. Moreover, we validated this approach in another independent external validation cohort, comprising 143 participants from the China-Japan Friendship Hospital. RESULTS A multimodality artificial intelligence-assisted pre-endoscopic screening model based on tongue images and inquiry information (AITonguequiry) was constructed, adopting a hierarchical prediction strategy, achieving tailored endoscopic recommendations. Validation analysis revealed that the area under the curve (AUC) values of AITonguequiry were 0.74 for overall PLGC (95% confidence interval (CI) 0.71-0.76, p < 0.05) and 0.82 for high-risk PLGC (95% CI 0.82-0.83, p < 0.05), which were significantly and robustly better than those of the independent use of either tongue images or inquiry information alone. In addition, AITonguequiry has superior performance compared to existing PLGC screening methodologies, with the AUC value enhancing 45% in terms of PLGC screening (0.74 vs. 0.51, p < 0.05) and 52% in terms of high-risk PLGC screening (0.82 vs. 0.54, p < 0.05). In the independent external verification, the AUC values were 0.69 for PLGC and 0.76 for high-risk PLGC. CONCLUSION Our AITonguequiry artificial intelligence model, for the first time, incorporates inquiry information and tongue images, leading to a higher precision and finer-grained pre-endoscopic screening of PLGC. This enhances patient screening efficiency and alleviates patient burden.
Collapse
Affiliation(s)
- Lan Wang
- Institute for TCM-X, MOE Key Laboratory of Bioinformatics, Bioinformatics Division, BNRIST, Department of Automation, Tsinghua University, Beijing, China
| | - Qian Zhang
- Institute for TCM-X, MOE Key Laboratory of Bioinformatics, Bioinformatics Division, BNRIST, Department of Automation, Tsinghua University, Beijing, China
| | - Peng Zhang
- Institute for TCM-X, MOE Key Laboratory of Bioinformatics, Bioinformatics Division, BNRIST, Department of Automation, Tsinghua University, Beijing, China
| | - Bowen Wu
- Institute for TCM-X, MOE Key Laboratory of Bioinformatics, Bioinformatics Division, BNRIST, Department of Automation, Tsinghua University, Beijing, China
| | - Jun Chen
- Department of Epidemiology and Health Statistics, School of Public Health, Fujian Medical University, Fuzhou, China
| | - Jiamin Gong
- Department of Epidemiology and Health Statistics, School of Public Health, Fujian Medical University, Fuzhou, China
| | - Kaiqiang Tang
- Department of Control Science and Intelligence Engineering, Nanjing University, Nanjing, China
| | - Shiyu Du
- Department of Gastroenterology, China-Japan Friendship Hospital, Chaoyang District, Beijing, China.
| | - Shao Li
- Institute for TCM-X, MOE Key Laboratory of Bioinformatics, Bioinformatics Division, BNRIST, Department of Automation, Tsinghua University, Beijing, China.
| |
Collapse
|
3
|
Weng W, Yoshida N, Morinaga Y, Sugino S, Tomita Y, Kobayashi R, Inoue K, Hirose R, Dohi O, Itoh Y, Zhu X. Development of high-quality artificial intelligence for computer-aided diagnosis in determining subtypes of colorectal cancer. J Gastroenterol Hepatol 2024. [PMID: 38923607 DOI: 10.1111/jgh.16661] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 03/14/2024] [Accepted: 06/09/2024] [Indexed: 06/28/2024]
Abstract
BACKGROUND AND AIM There are no previous studies in which computer-aided diagnosis (CAD) diagnosed colorectal cancer (CRC) subtypes correctly. In this study, we developed an original CAD for the diagnosis of CRC subtypes. METHODS Pretraining for the CAD based on ResNet was performed using ImageNet and five open histopathological pretraining image datasets (HiPreD) containing 3 million images. In addition, sparse attention was introduced to improve the CAD compared to other attention networks. One thousand and seventy-two histopathological images from 29 early CRC cases at Kyoto Prefectural University of Medicine from 2019 to 2022 were collected (857 images for training and validation, 215 images for test). All images were annotated by a qualified histopathologist for segmentation of normal mucosa, adenoma, pure well-differentiated adenocarcinoma (PWDA), and moderately/poorly differentiated adenocarcinoma (MPDA). Diagnostic ability including dice sufficient coefficient (DSC) and diagnostic accuracy were evaluated. RESULTS Our original CAD, named Colon-seg, with the pretraining of both HiPreD and ImageNET showed a better DSC (88.4%) compared to CAD without both pretraining (76.8%). Regarding the attentional mechanism, Colon-seg with sparse attention showed a better DSC (88.4%) compared to other attentional mechanisms (dual: 79.7%, ECA: 80.7%, shuffle: 84.7%, SK: 86.9%). In addition, the DSC of Colon-seg (88.4%) was better than other types of CADs (TransUNet: 84.7%, MultiResUnet: 86.1%, Unet++: 86.7%). The diagnostic accuracy of Colon-seg for each histopathological type was 94.3% for adenoma, 91.8% for PWDA, and 92.8% for MPDA. CONCLUSION A deep learning-based CAD for CRC subtype differentiation was developed with pretraining and fine-tuning of abundant histopathological images.
Collapse
Affiliation(s)
- Weihao Weng
- Graduate School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Japan
| | - Naohisa Yoshida
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Yukiko Morinaga
- Department of Surgical Pathology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Satoshi Sugino
- Department of Gastroenterology, Asahi University Hospital, Gifu, Japan
| | - Yuri Tomita
- Department of Gastroenterology, Koseikai Takeda Hospital, Kyoto, Japan
| | - Reo Kobayashi
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Ken Inoue
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Ryohei Hirose
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Osamu Dohi
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Yoshito Itoh
- Department of Molecular Gastroenterology and Hepatology, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Xin Zhu
- Graduate School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Japan
| |
Collapse
|
4
|
Sayed IM, Vo DT, Alcantara J, Inouye KM, Pranadinata RF, Luo L, Boland CR, Goyal NP, Kuo DJ, Huang SC, Sahoo D, Ghosh P, Das S. Molecular Signatures for Microbe-Associated Colorectal Cancers. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.26.595902. [PMID: 38853996 PMCID: PMC11160670 DOI: 10.1101/2024.05.26.595902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2024]
Abstract
Background Genetic factors and microbial imbalances play crucial roles in colorectal cancers (CRCs), yet the impact of infections on cancer initiation remains poorly understood. While bioinformatic approaches offer valuable insights, the rising incidence of CRCs creates a pressing need to precisely identify early CRC events. We constructed a network model to identify continuum states during CRC initiation spanning normal colonic tissue to pre-cancer lesions (adenomatous polyps) and examined the influence of microbes and host genetics. Methods A Boolean network was built using a publicly available transcriptomic dataset from healthy and adenoma affected patients to identify an invariant Microbe-Associated Colorectal Cancer Signature (MACS). We focused on Fusobacterium nucleatum ( Fn ), a CRC-associated microbe, as a model bacterium. MACS-associated genes and proteins were validated by RT-qPCR, RNA seq, ELISA, IF and IHCs in tissues and colon-derived organoids from genetically predisposed mice ( CPC-APC Min+/- ) and patients (FAP, Lynch Syndrome, PJS, and JPS). Results The MACS that is upregulated in adenomas consists of four core genes/proteins: CLDN2/Claudin-2 (leakiness), LGR5/leucine-rich repeat-containing receptor (stemness), CEMIP/cell migration-inducing and hyaluronan-binding protein (epithelial-mesenchymal transition) and IL8/Interleukin-8 (inflammation). MACS was induced upon Fn infection, but not in response to infection with other enteric bacteria or probiotics. MACS induction upon Fn infection was higher in CPC-APC Min+/- organoids compared to WT controls. The degree of MACS expression in the patient-derived organoids (PDOs) generally corresponded with the known lifetime risk of CRCs. Conclusions Computational prediction followed by validation in the organoid-based disease model identified the early events in CRC initiation. MACS reveals that the CRC-associated microbes induce a greater risk in the genetically predisposed hosts, suggesting its potential use for risk prediction and targeted cancer prevention.
Collapse
|
5
|
Pan Y, Dai H, Wang S, Wang L, Li Q, Wang W, Li J, Qi D, Yang Z, Jia J, Wang Y, Fang Q, Li L, Zhou W, Song Z, Zou S. Clinically Applicable Pan-Origin Cancer Detection for Lymph Nodes via Artificial Intelligence-Based Pathology. Pathobiology 2024:1-14. [PMID: 38718783 DOI: 10.1159/000539010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Accepted: 04/09/2024] [Indexed: 06/13/2024] Open
Abstract
INTRODUCTION Lymph node metastasis is one of the most common ways of tumour metastasis. The presence or absence of lymph node involvement influences the cancer's stage, therapy, and prognosis. The integration of artificial intelligence systems in the histopathological diagnosis of lymph nodes after surgery is urgent. METHODS Here, we propose a pan-origin lymph node cancer metastasis detection system. The system is trained by over 700 whole-slide images (WSIs) and is composed of two deep learning models to locate the lymph nodes and detect cancers. RESULTS It achieved an area under the receiver operating characteristic curve (AUC) of 0.958, with a 95.2% sensitivity and 72.2% specificity, on 1,402 WSIs from 49 organs at the National Cancer Center, China. Moreover, we demonstrated that the system could perform robustly with 1,051 WSIs from 52 organs from another medical centre, with an AUC of 0.925. CONCLUSION Our research represents a step forward in a pan-origin lymph node metastasis detection system, providing accurate pathological guidance by reducing the probability of missed diagnosis in routine clinical practice.
Collapse
Affiliation(s)
- Yi Pan
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China,
| | - Hongtian Dai
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Shuhao Wang
- Thorough Lab, Thorough Future, Beijing, China
| | - Lang Wang
- Thorough Lab, Thorough Future, Beijing, China
| | - Qiting Li
- R&D Department, China Academy of Launch Vehicle Technology, Beijing, China
| | - Wenmiao Wang
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jiangtao Li
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Dan Qi
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhaoyang Yang
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jia Jia
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yaxi Wang
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Qing Fang
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Lin Li
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Weixun Zhou
- Department of Pathology, Peking Union Medical College Hospital, Beijing, China
| | - Zhigang Song
- Department of Pathology, The Chinese PLA General Hospital, Beijing, China
| | - Shuangmei Zou
- Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
6
|
Lotter W, Hassett MJ, Schultz N, Kehl KL, Van Allen EM, Cerami E. Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions. Cancer Discov 2024; 14:711-726. [PMID: 38597966 PMCID: PMC11131133 DOI: 10.1158/2159-8290.cd-23-1199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/29/2024] [Accepted: 02/28/2024] [Indexed: 04/11/2024]
Abstract
Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field. SIGNIFICANCE AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.
Collapse
Affiliation(s)
- William Lotter
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Michael J. Hassett
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Nikolaus Schultz
- Marie-Josée and Henry R. Kravis Center for Molecular Oncology, Memorial Sloan Kettering Cancer Center; New York, NY, USA
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Kenneth L. Kehl
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Eliezer M. Van Allen
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
- Cancer Program, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Ethan Cerami
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| |
Collapse
|
7
|
Yuan L, Zhou H, Xiao X, Zhang X, Chen F, Liu L, Liu J, Bao S, Tao K. Development and external validation of a transfer learning-based system for the pathological diagnosis of colorectal cancer: a large emulated prospective study. Front Oncol 2024; 14:1365364. [PMID: 38725622 PMCID: PMC11079287 DOI: 10.3389/fonc.2024.1365364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 04/11/2024] [Indexed: 05/12/2024] Open
Abstract
Background The progress in Colorectal cancer (CRC) screening and management has resulted in an unprecedented caseload for histopathological diagnosis. While artificial intelligence (AI) presents a potential solution, the predominant emphasis on slide-level aggregation performance without thorough verification of cancer in each location, impedes both explainability and transparency. Effectively addressing these challenges is crucial to ensuring the reliability and efficacy of AI in histology applications. Method In this study, we created an innovative AI algorithm using transfer learning from a polyp segmentation model in endoscopy. The algorithm precisely localized CRC targets within 0.25 mm² grids from whole slide imaging (WSI). We assessed the CRC detection capabilities at this fine granularity and examined the influence of AI on the diagnostic behavior of pathologists. The evaluation utilized an extensive dataset comprising 858 consecutive patient cases with 1418 WSIs obtained from an external center. Results Our results underscore a notable sensitivity of 90.25% and specificity of 96.60% at the grid level, accompanied by a commendable area under the curve (AUC) of 0.962. This translates to an impressive 99.39% sensitivity at the slide level, coupled with a negative likelihood ratio of <0.01, signifying the dependability of the AI system to preclude diagnostic considerations. The positive likelihood ratio of 26.54, surpassing 10 at the grid level, underscores the imperative for meticulous scrutiny of any AI-generated highlights. Consequently, all four participating pathologists demonstrated statistically significant diagnostic improvements with AI assistance. Conclusion Our transfer learning approach has successfully yielded an algorithm that can be validated for CRC histological localizations in whole slide imaging. The outcome advocates for the integration of the AI system into histopathological diagnosis, serving either as a diagnostic exclusion application or a computer-aided detection (CADe) tool. This integration has the potential to alleviate the workload of pathologists and ultimately benefit patients.
Collapse
Affiliation(s)
- Liuhong Yuan
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Henghua Zhou
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | | | - Xiuqin Zhang
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Feier Chen
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Lin Liu
- Institute of Natural Sciences, MOE-LSC, School of Mathematical Sciences, CMA-Shanghai, SJTU-Yale Joint Center for Biostatistics and Data Science, Shanghai Jiao Tong University, Shanghai, China
| | | | - Shisan Bao
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Kun Tao
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| |
Collapse
|
8
|
Jin D, Liang S, Shmatko A, Arnold A, Horst D, Grünewald TGP, Gerstung M, Bai X. Teacher-student collaborated multiple instance learning for pan-cancer PDL1 expression prediction from histopathology slides. Nat Commun 2024; 15:3063. [PMID: 38594278 PMCID: PMC11004138 DOI: 10.1038/s41467-024-46764-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 03/08/2024] [Indexed: 04/11/2024] Open
Abstract
Programmed cell death ligand 1 (PDL1), as an important biomarker, is quantified by immunohistochemistry (IHC) with few established histopathological patterns. Deep learning aids in histopathological assessment, yet heterogeneity and lacking spatially resolved annotations challenge precise analysis. Here, we present a weakly supervised learning approach using bulk RNA sequencing for PDL1 expression prediction from hematoxylin and eosin (H&E) slides. Our method extends the multiple instance learning paradigm with the teacher-student framework, which assigns dynamic pseudo-labels for intra-slide heterogeneity and retrieves unlabeled instances using temporal ensemble model distillation. The approach, evaluated on 12,299 slides across 20 solid tumor types, achieves a weighted average area under the curve of 0.83 on fresh-frozen and 0.74 on formalin-fixed specimens for 9 tumors with PDL1 as an established biomarker. Our method predicts PDL1 expression patterns, validated by IHC on 20 slides, offering insights into histologies relevant to PDL1. This demonstrates the potential of deep learning in identifying diverse histological patterns for molecular changes from H&E images.
Collapse
Affiliation(s)
- Darui Jin
- Image Processing Center, Beihang University, Beijing, 102206, China
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Shen Yuan Honors College, Beihang University, Beijing, 100191, China
| | - Shangying Liang
- Image Processing Center, Beihang University, Beijing, 102206, China
| | - Artem Shmatko
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Arnold
- Charité - Universitätsmedizin Berlin, Institute of Pathology, 10117, Berlin, Germany
| | - David Horst
- Charité - Universitätsmedizin Berlin, Institute of Pathology, 10117, Berlin, Germany
- German Cancer Consortium (DKTK), partner site Berlin, a partnership between DKFZ and Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Thomas G P Grünewald
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany.
- Division of Translational Pediatric Sarcoma Research, German Cancer Research Center (DKFZ), German Cancer Consortium (DKTK), Heidelberg, Germany.
- Hopp Children's Cancer Center (KiTZ) Heidelberg, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Heidelberg, Germany.
| | - Moritz Gerstung
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| | - Xiangzhi Bai
- Image Processing Center, Beihang University, Beijing, 102206, China.
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, 100191, China.
- Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, 100083, China.
| |
Collapse
|
9
|
Yim D, Khuntia J, Parameswaran V, Meyers A. Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review. JMIR Med Inform 2024; 12:e52073. [PMID: 38506918 PMCID: PMC10993141 DOI: 10.2196/52073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 10/12/2023] [Accepted: 01/30/2024] [Indexed: 03/21/2024] Open
Abstract
BACKGROUND Generative artificial intelligence tools and applications (GenAI) are being increasingly used in health care. Physicians, specialists, and other providers have started primarily using GenAI as an aid or tool to gather knowledge, provide information, train, or generate suggestive dialogue between physicians and patients or between physicians and patients' families or friends. However, unless the use of GenAI is oriented to be helpful in clinical service encounters that can improve the accuracy of diagnosis, treatment, and patient outcomes, the expected potential will not be achieved. As adoption continues, it is essential to validate the effectiveness of the infusion of GenAI as an intelligent technology in service encounters to understand the gap in actual clinical service use of GenAI. OBJECTIVE This study synthesizes preliminary evidence on how GenAI assists, guides, and automates clinical service rendering and encounters in health care The review scope was limited to articles published in peer-reviewed medical journals. METHODS We screened and selected 0.38% (161/42,459) of articles published between January 1, 2020, and May 31, 2023, identified from PubMed. We followed the protocols outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select highly relevant studies with at least 1 element on clinical use, evaluation, and validation to provide evidence of GenAI use in clinical services. The articles were classified based on their relevance to clinical service functions or activities using the descriptive and analytical information presented in the articles. RESULTS Of 161 articles, 141 (87.6%) reported using GenAI to assist services through knowledge access, collation, and filtering. GenAI was used for disease detection (19/161, 11.8%), diagnosis (14/161, 8.7%), and screening processes (12/161, 7.5%) in the areas of radiology (17/161, 10.6%), cardiology (12/161, 7.5%), gastrointestinal medicine (4/161, 2.5%), and diabetes (6/161, 3.7%). The literature synthesis in this study suggests that GenAI is mainly used for diagnostic processes, improvement of diagnosis accuracy, and screening and diagnostic purposes using knowledge access. Although this solves the problem of knowledge access and may improve diagnostic accuracy, it is oriented toward higher value creation in health care. CONCLUSIONS GenAI informs rather than assisting or automating clinical service functions in health care. There is potential in clinical service, but it has yet to be actualized for GenAI. More clinical service-level evidence that GenAI is used to streamline some functions or provides more automated help than only information retrieval is needed. To transform health care as purported, more studies related to GenAI applications must automate and guide human-performed services and keep up with the optimism that forward-thinking health care organizations will take advantage of GenAI.
Collapse
Affiliation(s)
- Dobin Yim
- Loyola University, Maryland, MD, United States
| | - Jiban Khuntia
- University of Colorado Denver, Denver, CO, United States
| | | | - Arlen Meyers
- University of Colorado Denver, Denver, CO, United States
| |
Collapse
|
10
|
Neto PC, Montezuma D, Oliveira SP, Oliveira D, Fraga J, Monteiro A, Monteiro J, Ribeiro L, Gonçalves S, Reinhard S, Zlobec I, Pinto IM, Cardoso JS. An interpretable machine learning system for colorectal cancer diagnosis from pathology slides. NPJ Precis Oncol 2024; 8:56. [PMID: 38443695 PMCID: PMC10914836 DOI: 10.1038/s41698-024-00539-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 02/08/2024] [Indexed: 03/07/2024] Open
Abstract
Considering the profound transformation affecting pathology practice, we aimed to develop a scalable artificial intelligence (AI) system to diagnose colorectal cancer from whole-slide images (WSI). For this, we propose a deep learning (DL) system that learns from weak labels, a sampling strategy that reduces the number of training samples by a factor of six without compromising performance, an approach to leverage a small subset of fully annotated samples, and a prototype with explainable predictions, active learning features and parallelisation. Noting some problems in the literature, this study is conducted with one of the largest WSI colorectal samples dataset with approximately 10,500 WSIs. Of these samples, 900 are testing samples. Furthermore, the robustness of the proposed method is assessed with two additional external datasets (TCGA and PAIP) and a dataset of samples collected directly from the proposed prototype. Our proposed method predicts, for the patch-based tiles, a class based on the severity of the dysplasia and uses that information to classify the whole slide. It is trained with an interpretable mixed-supervision scheme to leverage the domain knowledge introduced by pathologists through spatial annotations. The mixed-supervision scheme allowed for an intelligent sampling strategy effectively evaluated in several different scenarios without compromising the performance. On the internal dataset, the method shows an accuracy of 93.44% and a sensitivity between positive (low-grade and high-grade dysplasia) and non-neoplastic samples of 0.996. On the external test samples varied with TCGA being the most challenging dataset with an overall accuracy of 84.91% and a sensitivity of 0.996.
Collapse
Affiliation(s)
- Pedro C Neto
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
| | - Diana Montezuma
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal.
- Cancer Biology and Epigenetics Group, Research Center of IPO Porto (CI-IPOP) / RISE@CI-IPOP (Health Research Network), Portuguese Oncology Institute of Porto (IPO Porto) / Porto Comprehensive Cancer Center (Porto.CCC), R. Dr. António Bernardino de Almeida 865, Porto, 4200-072, Porto, Portugal.
- Doctoral Programme in Medical Sciences, School of Medicine and Biomedical Sciences - University of Porto (ICBAS-UP), R. Jorge de Viterbo Ferreira 228, Porto, 4050-313, Porto, Portugal.
| | - Sara P Oliveira
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal.
| | - Domingos Oliveira
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - João Fraga
- Department of Pathology, IPO-Porto, R. Dr. António Bernardino de Almeida 865, Porto, 4200-072, Porto, Portugal
| | - Ana Monteiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - João Monteiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Liliana Ribeiro
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Sofia Gonçalves
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Stefan Reinhard
- Institute of Pathology, University of Bern, Uni Bern, Murtenstrasse 31, Bern, 3008, Bern, Switzerland
| | - Inti Zlobec
- Institute of Pathology, University of Bern, Uni Bern, Murtenstrasse 31, Bern, 3008, Bern, Switzerland
| | - Isabel M Pinto
- IMP Diagnostics, Praça do Bom Sucesso, 61, sala 808, Porto, 4150-146, Porto, Portugal
| | - Jaime S Cardoso
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal
- Faculty of Engineering, University of Porto (FEUP), R. Dr. Roberto Frias, Porto, 4200-465, Porto, Portugal
| |
Collapse
|
11
|
Uchikov P, Khalid U, Kraev K, Hristov B, Kraeva M, Tenchev T, Chakarov D, Sandeva M, Dragusheva S, Taneva D, Batashki A. Artificial Intelligence in the Diagnosis of Colorectal Cancer: A Literature Review. Diagnostics (Basel) 2024; 14:528. [PMID: 38472999 DOI: 10.3390/diagnostics14050528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 02/26/2024] [Accepted: 02/27/2024] [Indexed: 03/14/2024] Open
Abstract
BACKGROUND The aim of this review is to explore the role of artificial intelligence in the diagnosis of colorectal cancer, how it impacts CRC morbidity and mortality, and why its role in clinical medicine is limited. METHODS A targeted, non-systematic review of the published literature relating to colorectal cancer diagnosis was performed with PubMed databases that were scouted to help provide a more defined understanding of the recent advances regarding artificial intelligence and their impact on colorectal-related morbidity and mortality. Articles were included if deemed relevant and including information associated with the keywords. RESULTS The advancements in artificial intelligence have been significant in facilitating an earlier diagnosis of CRC. In this review, we focused on evaluating genomic biomarkers, the integration of instruments with artificial intelligence, MR and hyperspectral imaging, and the architecture of neural networks. We found that these neural networks seem practical and yield positive results in initial testing. Furthermore, we explored the use of deep-learning-based majority voting methods, such as bag of words and PAHLI, in improving diagnostic accuracy in colorectal cancer detection. Alongside this, the autonomous and expansive learning ability of artificial intelligence, coupled with its ability to extract increasingly complex features from images or videos without human reliance, highlight its impact in the diagnostic sector. Despite this, as most of the research involves a small sample of patients, a diversification of patient data is needed to enhance cohort stratification for a more sensitive and specific neural model. We also examined the successful application of artificial intelligence in predicting microsatellite instability, showcasing its potential in stratifying patients for targeted therapies. CONCLUSIONS Since its commencement in colorectal cancer, artificial intelligence has revealed a multitude of functionalities and augmentations in the diagnostic sector of CRC. Given its early implementation, its clinical application remains a fair way away, but with steady research dedicated to improving neural architecture and expanding its applicational range, there is hope that these advanced neural software could directly impact the early diagnosis of CRC. The true promise of artificial intelligence, extending beyond the medical sector, lies in its potential to significantly influence the future landscape of CRC's morbidity and mortality.
Collapse
Affiliation(s)
- Petar Uchikov
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Usman Khalid
- Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Krasimir Kraev
- Department of Propaedeutics of Internal Diseases "Prof. Dr. Anton Mitov", Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Bozhidar Hristov
- Section "Gastroenterology", Second Department of Internal Diseases, Medical Faculty, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Maria Kraeva
- Department of Otorhinolaryngology, Medical Faculty, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Tihomir Tenchev
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Dzhevdet Chakarov
- Department of Propaedeutics of Surgical Diseases, Section of General Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| | - Milena Sandeva
- Department of Midwifery, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Snezhanka Dragusheva
- Department of Nursing Care, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Daniela Taneva
- Department of Nursing Care, Faculty of Public Health, Medical University of Plovdiv, 4000 Plovdiv, Bulgaria
| | - Atanas Batashki
- Department of Special Surgery, Faculty of Medicine, Medical University of Plovdiv, 4002 Plovdiv, Bulgaria
| |
Collapse
|
12
|
Feng X, Shu W, Li M, Li J, Xu J, He M. Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview. J Transl Med 2024; 22:131. [PMID: 38310237 PMCID: PMC10837897 DOI: 10.1186/s12967-024-04915-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/20/2024] [Indexed: 02/05/2024] Open
Abstract
The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.
Collapse
Affiliation(s)
- Xiaobing Feng
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Wen Shu
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Mingya Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyu Li
- College of Electrical and Information Engineering, Hunan University, Changsha, China
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Junyao Xu
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China
| | - Min He
- College of Electrical and Information Engineering, Hunan University, Changsha, China.
- Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, Zhejiang, China.
| |
Collapse
|
13
|
Alves VM, dos Santos Cardoso J, Gama J. Classification of Pulmonary Nodules in 2-[ 18F]FDG PET/CT Images with a 3D Convolutional Neural Network. Nucl Med Mol Imaging 2024; 58:9-24. [PMID: 38261899 PMCID: PMC10796312 DOI: 10.1007/s13139-023-00821-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 05/17/2023] [Accepted: 08/08/2023] [Indexed: 01/25/2024] Open
Abstract
Purpose 2-[18F]FDG PET/CT plays an important role in the management of pulmonary nodules. Convolutional neural networks (CNNs) automatically learn features from images and have the potential to improve the discrimination between malignant and benign pulmonary nodules. The purpose of this study was to develop and validate a CNN model for classification of pulmonary nodules from 2-[18F]FDG PET images. Methods One hundred thirteen participants were retrospectively selected. One nodule per participant. The 2-[18F]FDG PET images were preprocessed and annotated with the reference standard. The deep learning experiment entailed random data splitting in five sets. A test set was held out for evaluation of the final model. Four-fold cross-validation was performed from the remaining sets for training and evaluating a set of candidate models and for selecting the final model. Models of three types of 3D CNNs architectures were trained from random weight initialization (Stacked 3D CNN, VGG-like and Inception-v2-like models) both in original and augmented datasets. Transfer learning, from ImageNet with ResNet-50, was also used. Results The final model (Stacked 3D CNN model) obtained an area under the ROC curve of 0.8385 (95% CI: 0.6455-1.0000) in the test set. The model had a sensibility of 80.00%, a specificity of 69.23% and an accuracy of 73.91%, in the test set, for an optimised decision threshold that assigns a higher cost to false negatives. Conclusion A 3D CNN model was effective at distinguishing benign from malignant pulmonary nodules in 2-[18F]FDG PET images. Supplementary Information The online version contains supplementary material available at 10.1007/s13139-023-00821-6.
Collapse
Affiliation(s)
- Victor Manuel Alves
- Faculty of Economics, University of Porto, Rua Dr. Roberto Frias, Porto, 4200-464 Porto, Portugal
- Department of Nuclear Medicine, University Hospital Center of São João, Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal
| | - Jaime dos Santos Cardoso
- Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
| | - João Gama
- Faculty of Economics, University of Porto, Rua Dr. Roberto Frias, Porto, 4200-464 Porto, Portugal
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
| |
Collapse
|
14
|
Griem J, Eich ML, Schallenberg S, Pryalukhin A, Bychkov A, Fukuoka J, Zayats V, Hulla W, Munkhdelger J, Seper A, Tsvetkov T, Mukhopadhyay A, Sanner A, Stieber J, Fuchs M, Babendererde N, Schömig-Markiefka B, Klein S, Buettner R, Quaas A, Tolkach Y. Artificial Intelligence-Based Tool for Tumor Detection and Quantitative Tissue Analysis in Colorectal Specimens. Mod Pathol 2023; 36:100327. [PMID: 37683932 DOI: 10.1016/j.modpat.2023.100327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 08/11/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
Digital pathology adoption allows for applying computational algorithms to routine pathology tasks. Our study aimed to develop a clinical-grade artificial intelligence (AI) tool for precise multiclass tissue segmentation in colorectal specimens (resections and biopsies) and clinically validate the tool for tumor detection in biopsy specimens. The training data set included 241 precisely manually annotated whole-slide images (WSIs) from multiple institutions. The algorithm was trained for semantic segmentation of 11 tissue classes with an additional module for biopsy WSI classification. Six case cohorts from 5 pathology departments (4 countries) were used for formal and clinical validation, digitized by 4 different scanning systems. The developed algorithm showed high precision of segmentation of different tissue classes in colorectal specimens with composite multiclass Dice score of up to 0.895 and pixel-wise tumor detection specificity and sensitivity of up to 0.958 and 0.987, respectively. In the clinical validation study on multiple external cohorts, the AI tool reached sensitivity of 1.0 and specificity of up to 0.969 for tumor detection in biopsy WSI. The AI tool analyzes most biopsy cases in less than 1 minute, allowing effective integration into clinical routine. We developed and extensively validated a highly accurate, clinical-grade tool for assistive diagnostic processing of colorectal specimens. This tool allows for quantitative deciphering of colorectal cancer tissue for development of prognostic and predictive biomarkers and personalization of oncologic care. This study is a foundation for a SemiCOL computational challenge. We open-source multiple manually annotated and weakly labeled test data sets, representing a significant contribution to the colorectal cancer computational pathology field.
Collapse
Affiliation(s)
- Johanna Griem
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | - Alexey Pryalukhin
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | - Andrey Bychkov
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Junya Fukuoka
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Vitaliy Zayats
- Laboratory for Medical Artificial Intelligence, The Resource Center for Universal Design and Rehabilitation Technologies (RCUD and RT), Moscow, Russia
| | - Wolfgang Hulla
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | | | - Alexander Seper
- Danube Private University, Medical Faculty, Krems-Stein, Austria
| | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | | | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | | | | | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Buettner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|
15
|
Bilal M, Tsang YW, Ali M, Graham S, Hero E, Wahab N, Dodd K, Sahota H, Wu S, Lu W, Jahanifar M, Robinson A, Azam A, Benes K, Nimir M, Hewitt K, Bhalerao A, Eldaly H, Raza SEA, Gopalakrishnan K, Minhas F, Snead D, Rajpoot N. Development and validation of artificial intelligence-based prescreening of large-bowel biopsies taken in the UK and Portugal: a retrospective cohort study. Lancet Digit Health 2023; 5:e786-e797. [PMID: 37890902 DOI: 10.1016/s2589-7500(23)00148-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 07/10/2023] [Accepted: 07/25/2023] [Indexed: 10/29/2023]
Abstract
BACKGROUND Histopathological examination is a crucial step in the diagnosis and treatment of many major diseases. Aiming to facilitate diagnostic decision making and improve the workload of pathologists, we developed an artificial intelligence (AI)-based prescreening tool that analyses whole-slide images (WSIs) of large-bowel biopsies to identify typical, non-neoplastic, and neoplastic biopsies. METHODS This retrospective cohort study was conducted with an internal development cohort of slides acquired from a hospital in the UK and three external validation cohorts of WSIs acquired from two hospitals in the UK and one clinical laboratory in Portugal. To learn the differential histological patterns from digitised WSIs of large-bowel biopsy slides, our proposed weakly supervised deep-learning model (Colorectal AI Model for Abnormality Detection [CAIMAN]) used slide-level diagnostic labels and no detailed cell or region-level annotations. The method was developed with an internal development cohort of 5054 biopsy slides from 2080 patients that were labelled with corresponding diagnostic categories assigned by pathologists. The three external validation cohorts, with a total of 1536 slides, were used for independent validation of CAIMAN. Each WSI was classified into one of three classes (ie, typical, atypical non-neoplastic, and atypical neoplastic). Prediction scores of image tiles were aggregated into three prediction scores for the whole slide, one for its likelihood of being typical, one for its likelihood of being non-neoplastic, and one for its likelihood of being neoplastic. The assessment of the external validation cohorts was conducted by the trained and frozen CAIMAN model. To evaluate model performance, we calculated area under the convex hull of the receiver operating characteristic curve (AUROC), area under the precision-recall curve, and specificity compared with our previously published iterative draw and rank sampling (IDaRS) algorithm. We also generated heat maps and saliency maps to analyse and visualise the relationship between the WSI diagnostic labels and spatial features of the tissue microenvironment. The main outcome of this study was the ability of CAIMAN to accurately identify typical and atypical WSIs of colon biopsies, which could potentially facilitate automatic removing of typical biopsies from the diagnostic workload in clinics. FINDINGS A randomly selected subset of all large bowel biopsies was obtained between Jan 1, 2012, and Dec 31, 2017. The AI training, validation, and assessments were done between Jan 1, 2021, and Sept 30, 2022. WSIs with diagnostic labels were collected between Jan 1 and Sept 30, 2022. Our analysis showed no statistically significant differences across prediction scores from CAIMAN for typical and atypical classes based on anatomical sites of the biopsy. At 0·99 sensitivity, CAIMAN (specificity 0·5592) was more accurate than an IDaRS-based weakly supervised WSI-classification pipeline (0·4629) in identifying typical and atypical biopsies on cross-validation in the internal development cohort (p<0·0001). At 0·99 sensitivity, CAIMAN was also more accurate than IDaRS for two external validation cohorts (p<0·0001), but not for a third external validation cohort (p=0·10). CAIMAN provided higher specificity than IDaRS at some high-sensitivity thresholds (0·7763 vs 0·6222 for 0·95 sensitivity, 0·7126 vs 0·5407 for 0·97 sensitivity, and 0·5615 vs 0·3970 for 0·99 sensitivity on one of the external validation cohorts) and showed high classification performance in distinguishing between neoplastic biopsies (AUROC 0·9928, 95% CI 0·9927-0·9929), inflammatory biopsies (0·9658, 0·9655-0·9661), and atypical biopsies (0·9789, 0·9786-0·9792). On the three external validation cohorts, CAIMAN had AUROC values of 0·9431 (95% CI 0·9165-0·9697), 0·9576 (0·9568-0·9584), and 0·9636 (0·9615-0·9657) for the detection of atypical biopsies. Saliency maps supported the representation of disease heterogeneity in model predictions and its association with relevant histological features. INTERPRETATION CAIMAN, with its high sensitivity in detecting atypical large-bowel biopsies, might be a promising improvement in clinical workflow efficiency and diagnostic decision making in prescreening of typical colorectal biopsies. FUNDING The Pathology Image Data Lake for Analytics, Knowledge and Education Centre of Excellence; the UK Government's Industrial Strategy Challenge Fund; and Innovate UK on behalf of UK Research and Innovation.
Collapse
Affiliation(s)
- Mohsin Bilal
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Artificial Intelligence and Data Science, National University of Computer and Emerging Sciences, Islamabad, Pakistan
| | - Yee Wah Tsang
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Mahmoud Ali
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Histofy, Birmingham, UK
| | - Emily Hero
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Department of Pathology, University Hospitals of Leicester National Health Service Trust, Leicester, UK
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Katherine Dodd
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Harvir Sahota
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shaobin Wu
- Department of Pathology, East Suffolk and North Essex National Health Service Foundation Trust, Colchester, UK
| | - Wenqi Lu
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Andrew Robinson
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ayesha Azam
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Ksenija Benes
- Department of Pathology, The Royal Wolverhampton National Health Service Trust, Wolverhampton, UK
| | - Mohammed Nimir
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Katherine Hewitt
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Abhir Bhalerao
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Hesham Eldaly
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Kishore Gopalakrishnan
- Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - David Snead
- Warwick Medical School, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK; Department of Pathology, University Hospitals Coventry and Warwickshire National Health Service Trust, Coventry, UK; Histofy, Birmingham, UK; The Alan Turing Institute, London, UK.
| |
Collapse
|
16
|
Yang Y, Sun K, Gao Y, Wang K, Yu G. Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance. Diagnostics (Basel) 2023; 13:3115. [PMID: 37835858 PMCID: PMC10572440 DOI: 10.3390/diagnostics13193115] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/27/2023] [Accepted: 09/28/2023] [Indexed: 10/15/2023] Open
Abstract
The pathology is decisive for disease diagnosis but relies heavily on experienced pathologists. In recent years, there has been growing interest in the use of artificial intelligence in pathology (AIP) to enhance diagnostic accuracy and efficiency. However, the impressive performance of deep learning-based AIP in laboratory settings often proves challenging to replicate in clinical practice. As the data preparation is important for AIP, the paper has reviewed AIP-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. An in-depth analysis of data preparation methods is conducted, encompassing the acquisition of pathological tissue slides, data cleaning, screening, and subsequent digitization. Expert review, image annotation, dataset division for model training and validation are also discussed. Furthermore, we delve into the reasons behind the challenges in reproducing the high performance of AIP in clinical settings and present effective strategies to enhance AIP's clinical performance. The robustness of AIP depends on a randomized collection of representative disease slides, incorporating rigorous quality control and screening, correction of digital discrepancies, reasonable annotation, and sufficient data volume. Digital pathology is fundamental in clinical-grade AIP, and the techniques of data standardization and weakly supervised learning methods based on whole slide image (WSI) are effective ways to overcome obstacles of performance reproduction. The key to performance reproducibility lies in having representative data, an adequate amount of labeling, and ensuring consistency across multiple centers. Digital pathology for clinical diagnosis, data standardization and the technique of WSI-based weakly supervised learning will hopefully build clinical-grade AIP.
Collapse
Affiliation(s)
- Yuanqing Yang
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
- Department of Biomedical Engineering, School of Medical, Tsinghua University, Beijing 100084, China
| | - Kai Sun
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
- Furong Laboratory, Changsha 410013, China
| | - Yanhua Gao
- Department of Ultrasound, Shaanxi Provincial People’s Hospital, Xi’an 710068, China;
| | - Kuansong Wang
- Department of Pathology, School of Basic Medical Sciences, Central South University, Changsha 410013, China;
- Department of Pathology, Xiangya Hospital, Central South University, Changsha 410013, China
| | - Gang Yu
- Department of Biomedical Engineering, School of Basic Medical Sciences, Central South University, Changsha 410013, China; (Y.Y.); (K.S.)
| |
Collapse
|
17
|
Dong Z, Tao X, Du H, Wang J, Huang L, He C, Zhao Z, Mao X, Ai Y, Zhang B, Liu M, Xu H, Jiang Z, Sun Y, Li X, Liu Z, Chen J, Song Y, Liu G, Luo C, Li Y, Zeng X, Liu J, Zhu Y, Wu L, Yu H. Exploring the challenge of early gastric cancer diagnostic AI system face in multiple centers and its potential solutions. J Gastroenterol 2023; 58:978-989. [PMID: 37515597 DOI: 10.1007/s00535-023-02025-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 07/10/2023] [Indexed: 07/31/2023]
Abstract
BACKGROUND Artificial intelligence (AI) performed variously among test sets with different diversity due to sample selection bias, which can be stumbling block for AI applications. We previously tested AI named ENDOANGEL, diagnosing early gastric cancer (EGC) on single-center videos in man-machine competition. We aimed to re-test ENDOANGEL on multi-center videos to explore challenges applying AI in multiple centers, then upgrade ENDOANGEL and explore solutions to the challenge. METHODS ENDOANGEL was re-tested on multi-center videos retrospectively collected from 12 institutions and compared with performance in previously reported single-center videos. We then upgraded ENDOANGEL to ENDOANGEL-2022 with more training samples and novel algorithms and conducted competition between ENDOANGEL-2022 and endoscopists. ENDOANGEL-2022 was then tested on single-center videos and compared with performance in multi-center videos; the two AI systems were also compared with each other and endoscopists. RESULTS Forty-six EGCs and 54 non-cancers were included in multi-center video cohort. On diagnosing EGCs, compared with single-center videos, ENDOANGEL showed stable sensitivity (97.83% vs. 100.00%) while sharply decreased specificity (61.11% vs. 82.54%); ENDOANGEL-2022 showed similar tendency while achieving significantly higher specificity (79.63%, p < 0.01) making fewer mistakes on typical lesions than ENDOANGEL. On detecting gastric neoplasms, both AI showed stable sensitivity while sharply decreased specificity. Nevertheless, both AI outperformed endoscopists in the two competitions. CONCLUSIONS Great increase of false positives is a prominent challenge for applying EGC diagnostic AI in multiple centers due to high heterogeneity of negative cases. Optimizing AI by adding samples and using novel algorithms is promising to overcome this challenge.
Collapse
Affiliation(s)
- Zehua Dong
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Xiao Tao
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Hongliu Du
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Junxiao Wang
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Li Huang
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chiyi He
- Department of Gastroenterology, Yijishan Hospital of Wannan Medical College, Wuhu, 241001, Anhui, People's Republic of China
| | - Zhifeng Zhao
- Department of Digestive Endoscopy, The Fourth Hospital of China Medical University, Shenyang, 110032, Liaoning Province, People's Republic of China
| | - Xinli Mao
- Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, Zhejiang, China
| | - Yaowei Ai
- Department of Gastroenterology, The People's Hospital of China Three Gorges University, The First People's Hospital of Yichang, Yichang, China
| | - Beiping Zhang
- Department of Gastroenterology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Mei Liu
- Department of Gastroenterology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Hong Xu
- Department of Endoscopy, The First Hospital of Jilin University, Changchun, China
| | - Zhenyu Jiang
- Department of Gastroenterology, The Second Affiliated Hospital of Baotou Medical College, Baotou, Inner Mongolia, China
| | - Yunwei Sun
- Department of Gastroenterology, Ruijin Hospital, Shanghai Jiaotong University, Gubei Branch, Shanghai, People's Republic of China
| | - Xiuling Li
- Department of Gastroenterology, School of Clinical Medicine, Henan Provincial People's Hospital, People's Hospital of Zhengzhou University, Henan University, Zhengzhou, Henan, China
| | - Zhihong Liu
- Department of Gastroenterology, Jilin City People's Hospital, Jilin, China
| | - Jinzhong Chen
- Endoscopy Center, School of Medicine, The First Affiliated Hospital of Xiamen University, Xiamen University, Xiamen, China
| | - Ying Song
- Department of Gastroenterology, Xi'an Gaoxin Hospital, Xi'an, 710032, Shaanxi Province, China
| | - Guowei Liu
- Yi Xin Clinic, Changzhou, Jiangsu, China
| | - Chaijie Luo
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yanxia Li
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Xiaoquan Zeng
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Jun Liu
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yijie Zhu
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Lianlian Wu
- Renmin Hospital of Wuhan University, Wuhan, China.
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China.
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.
- Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Road, Wuhan, 430060, Hubei Province, China.
| | - Honggang Yu
- Renmin Hospital of Wuhan University, Wuhan, China.
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China.
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.
- Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Road, Wuhan, 430060, Hubei Province, China.
| |
Collapse
|
18
|
Chlorogiannis DD, Verras GI, Tzelepi V, Chlorogiannis A, Apostolos A, Kotis K, Anagnostopoulos CN, Antzoulas A, Davakis S, Vailas M, Schizas D, Mulita F. Tissue classification and diagnosis of colorectal cancer histopathology images using deep learning algorithms. Is the time ripe for clinical practice implementation? PRZEGLAD GASTROENTEROLOGICZNY 2023; 18:353-367. [PMID: 38572457 PMCID: PMC10985751 DOI: 10.5114/pg.2023.130337] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 05/20/2023] [Indexed: 04/05/2024]
Abstract
Colorectal cancer is one of the most prevalent types of cancer, with histopathologic examination of biopsied tissue samples remaining the gold standard for diagnosis. During the past years, artificial intelligence (AI) has steadily found its way into the field of medicine and pathology, especially with the introduction of whole slide imaging (WSI). The main outcome of interest was the composite balanced accuracy (ACC) as well as the F1 score. The average reported ACC from the collected studies was 95.8 ±3.8%. Reported F1 scores reached as high as 0.975, with an average of 89.7 ±9.8%, indicating that existing deep learning algorithms can achieve in silico distinction between malignant and benign. Overall, the available state-of-the-art algorithms are non-inferior to pathologists for image analysis and classification tasks. However, due to their inherent uniqueness in their training and lack of widely accepted external validation datasets, their generalization potential is still limited.
Collapse
Affiliation(s)
| | | | - Vasiliki Tzelepi
- Department of Pathology, School of Medicine, University of Patras, Patras, Greece
| | | | - Anastasios Apostolos
- First Department of Cardiology, Hippokration Hospital, University of Athens, Athens, Greece
| | - Konstantinos Kotis
- Intelligent Systems Lab, Department of Cultural Technology and Communication, University of the Aegean, Mytilene, Greece
| | | | - Andreas Antzoulas
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| | - Spyridon Davakis
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Michail Vailas
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Dimitrios Schizas
- Upper Gastrointestinal and General Surgery Unit, First Department of Surgery, National and Kapodistrian University of Athens, Laiko General Hospital, Athens, Greece
| | - Francesk Mulita
- Department of Surgery, General University Hospital of Patras, Patras, Greece
| |
Collapse
|
19
|
Zidane M, Makky A, Bruhns M, Rochwarger A, Babaei S, Claassen M, Schürch CM. A review on deep learning applications in highly multiplexed tissue imaging data analysis. FRONTIERS IN BIOINFORMATICS 2023; 3:1159381. [PMID: 37564726 PMCID: PMC10410935 DOI: 10.3389/fbinf.2023.1159381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2023] [Accepted: 07/12/2023] [Indexed: 08/12/2023] Open
Abstract
Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients. Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of the DL-based pipelines used in preprocessing the highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.
Collapse
Affiliation(s)
- Mohammed Zidane
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Ahmad Makky
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Matthias Bruhns
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
- Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Alexander Rochwarger
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Sepideh Babaei
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
| | - Manfred Claassen
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
- Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Christian M. Schürch
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| |
Collapse
|
20
|
Wu Y, Li Y, Xiong X, Liu X, Lin B, Xu B. Recent advances of pathomics in colorectal cancer diagnosis and prognosis. Front Oncol 2023; 13:1094869. [PMID: 37538112 PMCID: PMC10396402 DOI: 10.3389/fonc.2023.1094869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 06/13/2023] [Indexed: 08/05/2023] Open
Abstract
Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.
Collapse
Affiliation(s)
- Yihan Wu
- School of Medicine, Chongqing University, Chongqing, China
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| | - Yi Li
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
- Bioengineering College, Chongqing University, Chongqing, China
| | - Xiaomin Xiong
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
- Bioengineering College, Chongqing University, Chongqing, China
| | - Xiaohua Liu
- Bioengineering College, Chongqing University, Chongqing, China
| | - Bo Lin
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| | - Bo Xu
- School of Medicine, Chongqing University, Chongqing, China
- Chongqing Key Laboratory of Intelligent Oncology for Breast Cancer, Chongqing University Cancer Hospital, Chongqing, China
| |
Collapse
|
21
|
Xu M, Chen Z, Zheng J, Zhao Q, Yuan Z. Artificial Intelligence-Aided Optical Imaging for Cancer Theranostics. Semin Cancer Biol 2023:S1044-579X(23)00094-9. [PMID: 37302519 DOI: 10.1016/j.semcancer.2023.06.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 06/08/2023] [Accepted: 06/08/2023] [Indexed: 06/13/2023]
Abstract
The use of artificial intelligence (AI) to assist biomedical imaging have demonstrated its high accuracy and high efficiency in medical decision-making for individualized cancer medicine. In particular, optical imaging methods are able to visualize both the structural and functional information of tumors tissues with high contrast, low cost, and noninvasive property. However, no systematic work has been performed to inspect the recent advances on AI-aided optical imaging for cancer theranostics. In this review, we demonstrated how AI can guide optical imaging methods to improve the accuracy on tumor detection, automated analysis and prediction of its histopathological section, its monitoring during treatment, and its prognosis by using computer vision, deep learning and natural language processing. By contrast, the optical imaging techniques involved mainly consisted of various tomography and microscopy imaging methods such as optical endoscopy imaging, optical coherence tomography, photoacoustic imaging, diffuse optical tomography, optical microscopy imaging, Raman imaging, and fluorescent imaging. Meanwhile, existing problems, possible challenges and future prospects for AI-aided optical imaging protocol for cancer theranostics were also discussed. It is expected that the present work can open a new avenue for precision oncology by using AI and optical imaging tools.
Collapse
Affiliation(s)
- Mengze Xu
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China; Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China
| | - Zhiyi Chen
- Institute of Medical Imaging, Hengyang Medical School, University of South China, Hengyang, China
| | - Junxiao Zheng
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China
| | - Qi Zhao
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China
| | - Zhen Yuan
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China.
| |
Collapse
|
22
|
Tolkach Y, Wolgast LM, Damanakis A, Pryalukhin A, Schallenberg S, Hulla W, Eich ML, Schroeder W, Mukhopadhyay A, Fuchs M, Klein S, Bruns C, Büttner R, Gebauer F, Schömig-Markiefka B, Quaas A. Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study. Lancet Digit Health 2023; 5:e265-e275. [PMID: 37100542 DOI: 10.1016/s2589-7500(23)00027-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Revised: 01/18/2023] [Accepted: 02/02/2023] [Indexed: 04/28/2023]
Abstract
BACKGROUND Oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction are among the most common malignant epithelial tumours. Most patients receive neoadjuvant therapy before complete tumour resection. Histological assessment after resection includes identification of residual tumour tissue and areas of regressive tumour, data which are used to calculate a clinically relevant regression score. We developed an artificial intelligence (AI) algorithm for tumour tissue detection and tumour regression grading in surgical specimens from patients with oesophageal adenocarcinoma or adenocarcinoma of the oesophagogastric junction. METHODS We used one training cohort and four independent test cohorts to develop, train, and validate a deep learning tool. The material consisted of histological slides from surgically resected specimens from patients with oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction from three pathology institutes (two in Germany, one in Austria) and oesophageal cancer cohort of The Cancer Genome Atlas (TCGA). All slides were from neoadjuvantly treated patients except for those from the TCGA cohort, who were neoadjuvant-therapy naive. Data from training cohort and test cohort cases were extensively manually annotated for 11 tissue classes. A convolutional neural network was trained on the data using a supervised principle. First, the tool was formally validated using manually annotated test datasets. Next, tumour regression grading was assessed in a retrospective cohort of post-neoadjuvant therapy surgical specimens. The grading of the algorithm was compared with that of a group of 12 board-certified pathologists from one department. To further validate the tool, three pathologists processed whole resection cases with and without AI assistance. FINDINGS Of the four test cohorts, one included 22 manually annotated histological slides (n=20 patients), one included 62 sides (n=15), one included 214 slides (n=69), and the final one included 22 manually annotated histological slides (n=22). In the independent test cohorts the AI tool had high patch-level accuracy for identifying both tumour and regression tissue. When we validated the concordance of the AI tool against analyses by a group of pathologists (n=12), agreement was 63·6% (quadratic kappa 0·749; p<0·0001) at case level. The AI-based regression grading triggered true reclassification of resected tumour slides in seven cases (including six cases who had small tumour regions that were initially missed by pathologists). Use of the AI tool by three pathologists increased interobserver agreement and substantially reduced diagnostic time per case compared with working without AI assistance. INTERPRETATION Use of our AI tool in the diagnostics of oesophageal adenocarcinoma resection specimens by pathologists increased diagnostic accuracy, interobserver concordance, and significantly reduced assessment time. Prospective validation of the tool is required. FUNDING North Rhine-Westphalia state, Federal Ministry of Education and Research of Germany, and the Wilhelm Sander Foundation.
Collapse
Affiliation(s)
- Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany.
| | - Lisa Marie Wolgast
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexander Damanakis
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexey Pryalukhin
- Institute of Pathology, Landesklinikum Wiener Neustadt, Wiener Neustadt, Austria
| | - Simon Schallenberg
- Institute of Pathology, University Hospital Berlin-Charité, Berlin, Germany
| | - Wolfgang Hulla
- Institute of Pathology, Landesklinikum Wiener Neustadt, Wiener Neustadt, Austria
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Wolfgang Schroeder
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Christiane Bruns
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Florian Gebauer
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Birgid Schömig-Markiefka
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany.
| |
Collapse
|
23
|
Tsai PC, Lee TH, Kuo KC, Su FY, Lee TLM, Marostica E, Ugai T, Zhao M, Lau MC, Väyrynen JP, Giannakis M, Takashima Y, Kahaki SM, Wu K, Song M, Meyerhardt JA, Chan AT, Chiang JH, Nowak J, Ogino S, Yu KH. Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients. Nat Commun 2023; 14:2102. [PMID: 37055393 PMCID: PMC10102208 DOI: 10.1038/s41467-023-37179-4] [Citation(s) in RCA: 22] [Impact Index Per Article: 22.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 03/03/2023] [Indexed: 04/15/2023] Open
Abstract
Histopathologic assessment is indispensable for diagnosing colorectal cancer (CRC). However, manual evaluation of the diseased tissues under the microscope cannot reliably inform patient prognosis or genomic variations crucial for treatment selections. To address these challenges, we develop the Multi-omics Multi-cohort Assessment (MOMA) platform, an explainable machine learning approach, to systematically identify and interpret the relationship between patients' histologic patterns, multi-omics, and clinical profiles in three large patient cohorts (n = 1888). MOMA successfully predicts the overall survival, disease-free survival (log-rank test P-value<0.05), and copy number alterations of CRC patients. In addition, our approaches identify interpretable pathology patterns predictive of gene expression profiles, microsatellite instability status, and clinically actionable genetic alterations. We show that MOMA models are generalizable to multiple patient populations with different demographic compositions and pathology images collected from distinctive digitization methods. Our machine learning approaches provide clinically actionable predictions that could inform treatments for colorectal cancer patients.
Collapse
Affiliation(s)
- Pei-Chen Tsai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Tsung-Hua Lee
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Kun-Chi Kuo
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Fang-Yi Su
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Tsung-Lu Michael Lee
- Department of Computer Science and Information Engineering, Southern Taiwan University of Science and Technology, Tainan, Taiwan ROC
| | - Eliana Marostica
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Division of Health Sciences and Technology, Harvard-Massachusetts Institute of Technology, Boston, MA, USA
| | - Tomotaka Ugai
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Melissa Zhao
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Mai Chan Lau
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Juha P Väyrynen
- Cancer and Translational Medicine Research Unit, Medical Research Center Oulu, Oulu University Hospital and University of Oulu, Oulu, Finland
| | - Marios Giannakis
- Department of Medicine, Dana Farber Cancer Institute, Boston, MA, USA
| | | | | | - Kana Wu
- Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Mingyang Song
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | | | - Andrew T Chan
- Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
| | - Jung-Hsien Chiang
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC.
| | - Jonathan Nowak
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Shuji Ogino
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
- Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Kun-Hsing Yu
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA.
| |
Collapse
|
24
|
Lan J, Chen M, Wang J, Du M, Wu Z, Zhang H, Xue Y, Wang T, Chen L, Xu C, Han Z, Hu Z, Zhou Y, Zhou X, Tong T, Chen G. Using less annotation workload to establish a pathological auxiliary diagnosis system for gastric cancer. Cell Rep Med 2023; 4:101004. [PMID: 37044091 PMCID: PMC10140598 DOI: 10.1016/j.xcrm.2023.101004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 10/20/2022] [Accepted: 03/17/2023] [Indexed: 04/14/2023]
Abstract
Pathological diagnosis of gastric cancer requires pathologists to have extensive clinical experience. To help pathologists improve diagnostic accuracy and efficiency, we collected 1,514 cases of stomach H&E-stained specimens with complete diagnostic information to establish a pathological auxiliary diagnosis system based on deep learning. At the slide level, our system achieves a specificity of 0.8878 while maintaining a high sensitivity close to 1.0 on 269 biopsy specimens (147 malignancies) and 163 surgical specimens (80 malignancies). The classified accuracy of our system is 0.9034 at the slide level for 352 biopsy specimens (201 malignancies) from 50 medical centers. With the help of our system, the pathologists' average false-negative rate and average false-positive rate on 100 biopsy specimens (50 malignancies) are reduced to 1/5 and 1/2 of the original rates, respectively. At the same time, the average uncertainty rate and the average diagnosis time are reduced by approximately 22% and 20%, respectively.
Collapse
Affiliation(s)
- Junlin Lan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Musheng Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Jianchao Wang
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Min Du
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Zhida Wu
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Hejun Zhang
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Yuyang Xue
- School of Engineering, University of Edinburgh, Edinburgh EH8 9JU, UK
| | - Tao Wang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Lifan Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China
| | - Chaohui Xu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Zixin Han
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Ziwei Hu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Yuanbo Zhou
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Xiaogen Zhou
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350108, China; Key Lab of Medical Instrumentation & Pharmaceutical Technology of Fujian Province, Fuzhou University, Fuzhou, Fujian 350108, China; Imperial Vision Technology, Fuzhou, Fujian 350100, China.
| | - Gang Chen
- Department of Pathology, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, Fujian 350014, China; Fujian Key Laboratory of Translational Cancer Medicine, Fuzhou, Fujian 350014, China.
| |
Collapse
|
25
|
Su L, Wang Z, Shi Y, Li A, Wang M. Local augmentation based consistency learning for semi-supervised pathology image classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107446. [PMID: 36871546 DOI: 10.1016/j.cmpb.2023.107446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 02/17/2023] [Accepted: 02/23/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Labeling pathology images is often costly and time-consuming, which is quite detrimental for supervised pathology image classification that relies heavily on sufficient labeled data during training. Exploring semi-supervised methods based on image augmentation and consistency regularization may effectively alleviate this problem. Nevertheless, traditional image-based augmentation (e.g., flip) produces only a single enhancement to an image, whereas combining multiple image sources may mix unimportant image regions resulting in poor performance. In addition, the regularization losses used in these augmentation approaches typically enforce the consistency of image level predictions, and meanwhile simply require each prediction of augmented image to be consistent bilaterally, which may force pathology image features with better predictions to be wrongly aligned towards the features with worse predictions. METHODS To tackle these problems, we propose a novel semi-supervised method called Semi-LAC for pathology image classification. Specifically, we first present local augmentation technique to randomly apply different augmentations produces to each local pathology patch, which can boost the diversity of pathology image and avoid mixing unimportant regions in other images. Moreover, we further propose the directional consistency loss to enforce restrictions on the consistency of both features and prediction results, thus improving the ability of the network to obtain robust representations and achieve accurate predictions. RESULTS The proposed method is evaluated on Bioimaging2015 and BACH datasets, and the extensive experiments show the superior performance of our Semi-LAC compared with state-of-the-art methods for pathology image classification. CONCLUSIONS We conclude that using the Semi-LAC method can effectively reduce the cost for annotating pathology images, and enhance the ability of classification networks to represent pathology images by using local augmentation techniques and directional consistency loss.
Collapse
Affiliation(s)
- Lei Su
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China
| | - Zhi Wang
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China
| | - Yi Shi
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China
| | - Ao Li
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China
| | - Minghui Wang
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| |
Collapse
|
26
|
Sun K, Chen Y, Bai B, Gao Y, Xiao J, Yu G. Automatic Classification of Histopathology Images across Multiple Cancers Based on Heterogeneous Transfer Learning. Diagnostics (Basel) 2023; 13:diagnostics13071277. [PMID: 37046497 PMCID: PMC10093253 DOI: 10.3390/diagnostics13071277] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/07/2023] [Accepted: 03/23/2023] [Indexed: 03/31/2023] Open
Abstract
Background: Current artificial intelligence (AI) in histopathology typically specializes on a single task, resulting in a heavy workload of collecting and labeling a sufficient number of images for each type of cancer. Heterogeneous transfer learning (HTL) is expected to alleviate the data bottlenecks and establish models with performance comparable to supervised learning (SL). Methods: An accurate source domain model was trained using 28,634 colorectal patches. Additionally, 1000 sentinel lymph node patches and 1008 breast patches were used to train two target domain models. The feature distribution difference between sentinel lymph node metastasis or breast cancer and CRC was reduced by heterogeneous domain adaptation, and the maximum mean difference between subdomains was used for knowledge transfer to achieve accurate classification across multiple cancers. Result: HTL on 1000 sentinel lymph node patches (L-HTL-1000) outperforms SL on 1000 sentinel lymph node patches (L-SL-1-1000) (average area under the curve (AUC) and standard deviation of L-HTL-1000 vs. L-SL-1-1000: 0.949 ± 0.004 vs. 0.931 ± 0.008, p value = 0.008). There is no significant difference between L-HTL-1000 and SL on 7104 patches (L-SL-2-7104) (0.949 ± 0.004 vs. 0.948 ± 0.008, p value = 0.742). Similar results are observed for breast cancer. B-HTL-1008 vs. B-SL-1-1008: 0.962 ± 0.017 vs. 0.943 ± 0.018, p value = 0.008; B-HTL-1008 vs. B-SL-2-5232: 0.962 ± 0.017 vs. 0.951 ± 0.023, p value = 0.148. Conclusions: HTL is capable of building accurate AI models for similar cancers using a small amount of data based on a large dataset for a certain type of cancer. HTL holds great promise for accelerating the development of AI in histopathology.
Collapse
|
27
|
Chaudhury S, Sau K. A BERT encoding with Recurrent Neural Network and Long-Short Term Memory for breast cancer image classification. DECISION ANALYTICS JOURNAL 2023; 6:100177. [DOI: 10.1016/j.dajour.2023.100177] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/19/2023]
|
28
|
Yuan L, Yang L, Zhang S, Xu Z, Qin J, Shi Y, Yu P, Wang Y, Bao Z, Xia Y, Sun J, He W, Chen T, Chen X, Hu C, Zhang Y, Dong C, Zhao P, Wang Y, Jiang N, Lv B, Xue Y, Jiao B, Gao H, Chai K, Li J, Wang H, Wang X, Guan X, Liu X, Zhao G, Zheng Z, Yan J, Yu H, Chen L, Ye Z, You H, Bao Y, Cheng X, Zhao P, Wang L, Zeng W, Tian Y, Chen M, You Y, Yuan G, Ruan H, Gao X, Xu J, Xu H, Du L, Zhang S, Fu H, Cheng X. Development of a tongue image-based machine learning tool for the diagnosis of gastric cancer: a prospective multicentre clinical cohort study. EClinicalMedicine 2023; 57:101834. [PMID: 36825238 PMCID: PMC9941057 DOI: 10.1016/j.eclinm.2023.101834] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 01/04/2023] [Accepted: 01/09/2023] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND Tongue images (the colour, size and shape of the tongue and the colour, thickness and moisture content of the tongue coating), reflecting the health state of the whole body according to the theory of traditional Chinese medicine (TCM), have been widely used in China for thousands of years. Herein, we investigated the value of tongue images and the tongue coating microbiome in the diagnosis of gastric cancer (GC). METHODS From May 2020 to January 2021, we simultaneously collected tongue images and tongue coating samples from 328 patients with GC (all newly diagnosed with GC) and 304 non-gastric cancer (NGC) participants in China, and 16 S rDNA was used to characterize the microbiome of the tongue coating samples. Then, artificial intelligence (AI) deep learning models were established to evaluate the value of tongue images and the tongue coating microbiome in the diagnosis of GC. Considering that tongue imaging is more convenient and economical as a diagnostic tool, we further conducted a prospective multicentre clinical study from May 2020 to March 2022 in China and recruited 937 patients with GC and 1911 participants with NGC from 10 centres across China to further evaluate the role of tongue images in the diagnosis of GC. Moreover, we verified this approach in another independent external validation cohort that included 294 patients with GC and 521 participants with NGC from 7 centres. This study is registered at ClinicalTrials.gov, NCT01090362. FINDINGS For the first time, we found that both tongue images and the tongue coating microbiome can be used as tools for the diagnosis of GC, and the area under the curve (AUC) value of the tongue image-based diagnostic model was 0.89. The AUC values of the tongue coating microbiome-based model reached 0.94 using genus data and 0.95 using species data. The results of the prospective multicentre clinical study showed that the AUC values of the three tongue image-based models for GCs reached 0.88-0.92 in the internal verification and 0.83-0.88 in the independent external verification, which were significantly superior to the combination of eight blood biomarkers. INTERPRETATION Our results suggest that tongue images can be used as a stable method for GC diagnosis and are significantly superior to conventional blood biomarkers. The three kinds of tongue image-based AI deep learning diagnostic models that we developed can be used to adequately distinguish patients with GC from participants with NGC, even early GC and precancerous lesions, such as atrophic gastritis (AG). FUNDING The National Key R&D Program of China (2021YFA0910100), Program of Zhejiang Provincial TCM Sci-tech Plan (2018ZY006), Medical Science and Technology Project of Zhejiang Province (2022KY114, WKJ-ZJ-2104), Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer (JBZX-202006), Natural Science Foundation of Zhejiang Province (HDMY22H160008), Science and Technology Projects of Zhejiang Province (2019C03049), National Natural Science Foundation of China (82074245, 81973634, 82204828), and Chinese Postdoctoral Science Foundation (2022M713203).
Collapse
Key Words
- AFP, alpha fetoprotein
- AG, atrophic gastritis
- AI, artificial intelligence
- APINet, attentive pairwise interaction neural network
- AUC, area under the curve
- Artificial intelligence
- BC, breast cancer
- CA, carbohydrate antigen
- CEA, carcinoembryonic antigen
- CRC, colorectal cancer
- DT, decision tree learning
- EC, esophageal cancer
- GC, gastric cancer
- Gastric cancer
- HBPC, hepatobiliary pancreatic carcinoma
- HC, healthy control
- KNN, K-nearest neighbours
- LC, lung cancer
- NGC, non-gastric cancers
- PCoA, principal coordinates analysis
- SG, superficial gastritis
- SVM, support vector machine
- TCM, traditional Chinese medicine
- Tongue coating microbiome
- Tongue images
- Traditional Chinese medicine
- TransFG, transformer architecture for fine-grained recognition
Collapse
Affiliation(s)
- Li Yuan
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
- Zhejiang Key Lab of Prevention, Diagnosis and Therapy of Upper Gastrointestinal Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
| | - Lin Yang
- Artificial Intelligence and Biomedical Images Analysis Lab, School of Engineering, Westlake University, China
| | - Shichuan Zhang
- Artificial Intelligence and Biomedical Images Analysis Lab, School of Engineering, Westlake University, China
| | - Zhiyuan Xu
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
- Zhejiang Key Lab of Prevention, Diagnosis and Therapy of Upper Gastrointestinal Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
| | - Jiangjiang Qin
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
- Zhejiang Key Lab of Prevention, Diagnosis and Therapy of Upper Gastrointestinal Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
| | - Yunfu Shi
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
- Oncology Department, Tongde Hospital of Zhejiang Province, Hangzhou, 310012, China
| | - Pengcheng Yu
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Yi Wang
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Zhehan Bao
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Yuhang Xia
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Jiancheng Sun
- Department of Gastrointestinal Surgery, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, 325099, China
| | - Weiyang He
- Department of Gastrointestinal Surgery, Sichuan Cancer Hospital, Chengdu, 610042, China
| | - Tianhui Chen
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
| | - Xiaolei Chen
- Department of Gastrointestinal Surgery, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, 325099, China
| | - Can Hu
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Yunlong Zhang
- Artificial Intelligence and Biomedical Images Analysis Lab, School of Engineering, Westlake University, China
| | - Changwu Dong
- College of Traditional Chinese Medicine, Anhui University of Traditional Chinese Medicine, HeFei, 230038, China
| | - Ping Zhao
- Department of Gastrointestinal Surgery, Sichuan Cancer Hospital, Chengdu, 610042, China
| | - Yanan Wang
- College of Traditional Chinese Medicine, Anhui University of Traditional Chinese Medicine, HeFei, 230038, China
| | - Nan Jiang
- College of Traditional Chinese Medicine, Anhui University of Traditional Chinese Medicine, HeFei, 230038, China
| | - Bin Lv
- Department of Gastroenterology, First Affiliated Hospital of Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Yingwei Xue
- Gastrointestinal Surgery, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Baoping Jiao
- Department of General Surgery, Shanxi Cancer Hospital, Taiyuan, 030013, China
| | - Hongyu Gao
- Gastrointestinal Surgery, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Kequn Chai
- Oncology Department, Tongde Hospital of Zhejiang Province, Hangzhou, 310012, China
| | - Jun Li
- Department of General Surgery, Shanxi Cancer Hospital, Taiyuan, 030013, China
| | - Hao Wang
- Gastrointestinal Surgery, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Xibo Wang
- Gastrointestinal Surgery, Harbin Medical University Cancer Hospital, Harbin, 150081, China
| | - Xiaoqing Guan
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
| | - Xu Liu
- Department of Gastrointestinal Surgery, RenJi Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200025, China
| | - Gang Zhao
- Department of Gastrointestinal Surgery, RenJi Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200025, China
| | - Zhichao Zheng
- Department of Gastric Surgery, Cancer Hospital of China Medical University (Liaoning Cancer Hospital and Institute), Shenyang, 110042, China
| | - Jie Yan
- Department of Gastric Surgery, Cancer Hospital of China Medical University (Liaoning Cancer Hospital and Institute), Shenyang, 110042, China
| | - Haiyue Yu
- Department of Gastric Surgery, Cancer Hospital of China Medical University (Liaoning Cancer Hospital and Institute), Shenyang, 110042, China
| | - Luchuan Chen
- Department of Gastrointestinal Surgery, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou, 350014, China
| | - Zaisheng Ye
- Department of Gastrointestinal Surgery, Fujian Cancer Hospital, Fujian Medical University Cancer Hospital, Fuzhou, 350014, China
| | - Huaqiang You
- Department of Gastroenterology, Yuhang District People's Hospital, Hangzhou, 311199, China
| | - Yu Bao
- Department of Gastrointestinal Surgery, Sichuan Cancer Hospital, Chengdu, 610042, China
| | - Xi Cheng
- Department of Gastrointestinal Surgery, Sichuan Cancer Hospital, Chengdu, 610042, China
| | - Peizheng Zhao
- Department of Health Management Center, Yueyang Central Hospital, Yueyang, 414000, China
| | - Liang Wang
- Department of Endoscopy Center, Kecheng District People's Hospital, Quzhou, 324000, China
| | - Wenting Zeng
- Department of General Surgery, Shanxi Cancer Hospital, Taiyuan, 030013, China
| | - Yanfei Tian
- Department of Gastric Surgery, Cancer Hospital of China Medical University (Liaoning Cancer Hospital and Institute), Shenyang, 110042, China
| | - Ming Chen
- Department of Endoscopy Center, Shandong Cancer Hospital, Shandong, 250117, China
| | - You You
- Department of Health Management Center, Zigong Fourth People's Hospital, Zigong, 643099, China
| | - Guihong Yuan
- Department of Gastroenterology, Hainan Cancer Hospital, Hainan, 570312, China
| | - Hua Ruan
- Department of Chinese Surgery, Linping District Hospital of Traditional Chinese Medicine, Hangzhou, 311100, China
| | - Xiaole Gao
- The First Affiliated Hospital of Henan University of Science and Technology, Zhengzhou, 450062, China
| | - Jingli Xu
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Handong Xu
- First Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Lingbin Du
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
| | - Shengjie Zhang
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
| | - Huanying Fu
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
| | - Xiangdong Cheng
- Department of Gastric Surgery, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institutes of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Upper Gastrointestinal Tract Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
- Zhejiang Key Lab of Prevention, Diagnosis and Therapy of Upper Gastrointestinal Cancer, Zhejiang Cancer Hospital, Hangzhou, 310022, China
- Corresponding author. Department of Gastric surgery, Zhejiang Cancer Hospital, Banshan Road 1#, Hangzhou, Zhejiang, 310022, China.
| |
Collapse
|
29
|
The Role of Artificial Intelligence in Echocardiography. J Imaging 2023; 9:jimaging9020050. [PMID: 36826969 PMCID: PMC9962859 DOI: 10.3390/jimaging9020050] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 02/03/2023] [Accepted: 02/15/2023] [Indexed: 02/22/2023] Open
Abstract
Echocardiography is an integral part of the diagnosis and management of cardiovascular disease. The use and application of artificial intelligence (AI) is a rapidly expanding field in medicine to improve consistency and reduce interobserver variability. AI can be successfully applied to echocardiography in addressing variance during image acquisition and interpretation. Furthermore, AI and machine learning can aid in the diagnosis and management of cardiovascular disease. In the realm of echocardiography, accurate interpretation is largely dependent on the subjective knowledge of the operator. Echocardiography is burdened by the high dependence on the level of experience of the operator, to a greater extent than other imaging modalities like computed tomography, nuclear imaging, and magnetic resonance imaging. AI technologies offer new opportunities for echocardiography to produce accurate, automated, and more consistent interpretations. This review discusses machine learning as a subfield within AI in relation to image interpretation and how machine learning can improve the diagnostic performance of echocardiography. This review also explores the published literature outlining the value of AI and its potential to improve patient care.
Collapse
|
30
|
Computer-Assisted Diagnosis of Lymph Node Metastases in Colorectal Cancers Using Transfer Learning With an Ensemble Model. Mod Pathol 2023; 36:100118. [PMID: 36805793 DOI: 10.1016/j.modpat.2023.100118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 12/14/2022] [Accepted: 01/21/2023] [Indexed: 02/05/2023]
Abstract
Screening of lymph node metastases in colorectal cancer (CRC) can be a cumbersome task, but it is amenable to artificial intelligence (AI)-assisted diagnostic solution. Here, we propose a deep learning-based workflow for the evaluation of CRC lymph node metastases from digitized hematoxylin and eosin-stained sections. A segmentation model was trained on 100 whole-slide images (WSIs). It achieved a Matthews correlation coefficient of 0.86 (±0.154) and an acceptable Hausdorff distance of 135.59 μm (±72.14 μm), indicating a high congruence with the ground truth. For metastasis detection, 2 models (Xception and Vision Transformer) were independently trained first on a patch-based breast cancer lymph node data set and were then fine-tuned using the CRC data set. After fine-tuning, the ensemble model showed significant improvements in the F1 score (0.797-0.949; P <.00001) and the area under the receiver operating characteristic curve (0.959-0.978; P <.00001). Four independent cohorts (3 internal and 1 external) of CRC lymph nodes were used for validation in cascading segmentation and metastasis detection models. Our approach showed excellent performance, with high sensitivity (0.995, 1.0) and specificity (0.967, 1.0) in 2 validation cohorts of adenocarcinoma cases (n = 3836 slides) when comparing slide-level labels with the ground truth (pathologist reports). Similarly, an acceptable performance was achieved in a validation cohort (n = 172 slides) with mucinous and signet-ring cell histology (sensitivity, 0.872; specificity, 0.936). The patch-based classification confidence was aggregated to overlay the potential metastatic regions within each lymph node slide for visualization. We also applied our method to a consecutive case series of lymph nodes obtained over the past 6 months at our institution (n = 217 slides). The overlays of prediction within lymph node regions matched 100% when compared with a microscope evaluation by an expert pathologist. Our results provide the basis for a computer-assisted diagnostic tool for easy and efficient lymph node screening in patients with CRC.
Collapse
|
31
|
Yin Z, Yao C, Zhang L, Qi S. Application of artificial intelligence in diagnosis and treatment of colorectal cancer: A novel Prospect. Front Med (Lausanne) 2023; 10:1128084. [PMID: 36968824 PMCID: PMC10030915 DOI: 10.3389/fmed.2023.1128084] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 02/13/2023] [Indexed: 03/29/2023] Open
Abstract
In the past few decades, according to the rapid development of information technology, artificial intelligence (AI) has also made significant progress in the medical field. Colorectal cancer (CRC) is the third most diagnosed cancer worldwide, and its incidence and mortality rates are increasing yearly, especially in developing countries. This article reviews the latest progress in AI in diagnosing and treating CRC based on a systematic collection of previous literature. Most CRCs transform from polyp mutations. The computer-aided detection systems can significantly improve the polyp and adenoma detection rate by early colonoscopy screening, thereby lowering the possibility of mutating into CRC. Machine learning and bioinformatics analysis can help screen and identify more CRC biomarkers to provide the basis for non-invasive screening. The Convolutional neural networks can assist in reading histopathologic tissue images, reducing the experience difference among doctors. Various studies have shown that AI-based high-level auxiliary diagnostic systems can significantly improve the readability of medical images and help clinicians make more accurate diagnostic and therapeutic decisions. Moreover, Robotic surgery systems such as da Vinci have been more and more commonly used to treat CRC patients, according to their precise operating performance. The application of AI in neoadjuvant chemoradiotherapy has further improved the treatment and efficacy evaluation of CRC. In addition, AI represented by deep learning in gene sequencing research offers a new treatment option. All of these things have seen that AI has a promising prospect in the era of precision medicine.
Collapse
Affiliation(s)
- Zugang Yin
- Department of General Surgery, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Chenhui Yao
- Department of General Surgery, The First Affiliated Hospital of Dalian Medical University, Dalian, China
- *Correspondence: Chenhui Yao,
| | - Limin Zhang
- Department of Respiratory, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Shaohua Qi
- Institute of Laboratory Animal Science, Chinese Academy of Medical Sciences and Comparative Medicine Center, Peking Union Medical College, Beijing, China
| |
Collapse
|
32
|
Pham TD, Ravi V, Luo B, Fan C, Sun XF. Artificial intelligence fusion for predicting survival of rectal cancer patients using immunohistochemical expression of Ras homolog family member B in biopsy. EXPLORATION OF TARGETED ANTI-TUMOR THERAPY 2023; 4:1-16. [PMID: 36937315 PMCID: PMC10017185 DOI: 10.37349/etat.2023.00119] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Accepted: 10/31/2022] [Indexed: 02/10/2023] Open
Abstract
Aim The process of biomarker discovery is being accelerated with the application of artificial intelligence (AI), including machine learning. Biomarkers of diseases are useful because they are indicators of pathogenesis or measures of responses to therapeutic treatments, and therefore, play a key role in new drug development. Proteins are among the candidates for biomarkers of rectal cancer, which need to be explored using state-of-the-art AI to be utilized for prediction, prognosis, and therapeutic treatment. This paper aims to investigate the predictive power of Ras homolog family member B (RhoB) protein in rectal cancer. Methods This study introduces the integration of pretrained convolutional neural networks and support vector machines (SVMs) for classifying biopsy samples of immunohistochemical expression of protein RhoB in rectal-cancer patients to validate its biologic measure in biopsy. Features of the immunohistochemical expression images were extracted by the pretrained networks and used for binary classification by the SVMs into two groups of less and more than 5-year survival rates. Results The fusion of neural search architecture network (NASNet)-Large for deep-layer feature extraction and classifier using SVMs provided the best average classification performance with a total accuracy = 85%, prediction of survival rate of more than 5 years = 90%, and prediction of survival rate of less than 5 years = 75%. Conclusions The finding obtained from the use of AI reported in this study suggest that RhoB expression on rectal-cancer biopsy can be potentially used as a biomarker for predicting survival outcomes in rectal-cancer patients, which can be informative for clinical decision making if the patient would be recommended for preoperative therapy.
Collapse
Affiliation(s)
- Tuan D. Pham
- Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar 34754, Saudi Arabia
- Correspondence: Tuan D. Pham, Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar 34754, Saudi Arabia. ;
| | - Vinayakumar Ravi
- Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar 34754, Saudi Arabia
| | - Bin Luo
- Department of Biomedical and Clinical Sciences, Linköping University, 58183 Linköping, Sweden
- Department of Gastrointestinal Surgery, Sichuan Provincial People’s Hospital, Chengdu 610032, Sichuan, China
| | - Chuanwen Fan
- Department of Biomedical and Clinical Sciences, Linköping University, 58183 Linköping, Sweden
| | - Xiao-Feng Sun
- Department of Biomedical and Clinical Sciences, Linköping University, 58183 Linköping, Sweden
| |
Collapse
|
33
|
Wang H, Zhou Z, Li H, Xiang W, Lan Y, Dou X, Zhang X. Blood Biomarkers Panels for Screening of Colorectal Cancer and Adenoma on a Machine Learning-Assisted Detection Platform. Cancer Control 2023; 30:10732748231222109. [PMID: 38146088 PMCID: PMC10750512 DOI: 10.1177/10732748231222109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 11/01/2023] [Accepted: 11/21/2023] [Indexed: 12/27/2023] Open
Abstract
OBJECTIVE A mini-invasive and good-compliance program is critical to broaden colorectal cancer (CRC) screening and reduce CRC-related mortality. Blood testing combined with imaging examination has been proved to be feasible on screen for multicancer and guide intervention. The study aims to construct a machine learning-assisted detection platform with available multi-targets for CRC and colorectal adenoma (CRA) screening. METHODS This was a retrospective study that the blood test data from 204 CRCs, 384 CRAs, and 229 healthy controls was extracted. The classified models were constructed with 4 machine learning (ML) algorithms including support vector machine (SVM), random forest (RF), decision tree (DT), and eXtreme Gradient Boosting (XGB) based on the candidate biomarkers. The importance index was used by SHapely Adaptive exPlanations (SHAP) analysis to identify the dominant characteristics. The performance of classified models was evaluated. The most dominating features from the proposed panel were developed by logistic regression (LR) for identification CRC from control. RESULTS The candidate biomarkers consisted of 26 multi-targets panel including CEA, AFP, and so on. Among the 4 models, the SVM classifier for CRA yields the best predictive performance (the area under the receiver operating curve, AUC: .925, sensitivity: .904, and specificity: .771). As for CRC classification, the RF model with 26 candidate biomarkers provided the best predictive parameters (AUC: .941, sensitivity: .902, and specificity: .912). Compared with CEA and CA199, the predictive performance was significantly improved. The streamlined model with 6 biomarkers for CRC also obtained a good performance (AUC: .946, sensitivity: .885, and specificity: .913). CONCLUSIONS The predictive models consisting of 26 multi-targets panel would be used as a non-invasive, economical, and effective risk stratification platform, which was expected to be applied for auxiliary screening of CRA and CRC in clinical practice.
Collapse
Affiliation(s)
- Hui Wang
- School of Medicine, Anhui University of Science and Technology, Huainan, Anhui, China
- Medical Laboratory of the Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Zhiwei Zhou
- Shenzhen Luohu People's Hospital, The Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Haijun Li
- Shenzhen Luohu People's Hospital, The Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Weiguang Xiang
- Shenzhen Luohu People's Hospital, The Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Yilin Lan
- Shenzhen Luohu People's Hospital, The Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Xiaowen Dou
- Medical Laboratory of the Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| | - Xiuming Zhang
- School of Medicine, Anhui University of Science and Technology, Huainan, Anhui, China
- Medical Laboratory of the Third Affiliated Hospital of Shenzhen University, Shenzhen, Guangdong, China
| |
Collapse
|
34
|
Yang Y, Liu Z, Huang J, Sun X, Ao J, Zheng B, Chen W, Shao Z, Hu H, Yang Y, Ji M. Histological diagnosis of unprocessed breast core-needle biopsy via stimulated Raman scattering microscopy and multi-instance learning. Theranostics 2023; 13:1342-1354. [PMID: 36923541 PMCID: PMC10008736 DOI: 10.7150/thno.81784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 01/09/2023] [Indexed: 03/14/2023] Open
Abstract
Core-needle biopsy (CNB) plays a vital role in the initial diagnosis of breast cancer. However, the complex tissue processing and global shortage of pathologists have hindered traditional histopathology from timely diagnosis on fresh biopsies. In this work, we developed a full digital platform by integrating label-free stimulated Raman scattering (SRS) microscopy with weakly-supervised learning for rapid and automated cancer diagnosis on un-labelled breast CNB. Methods: We first compared the results of SRS imaging with standard hematoxylin and eosin (H&E) staining on adjacent frozen tissue sections. Then fresh unprocessed biopsy tissues were imaged by SRS to reveal diagnostic histoarchitectures. Next, weakly-supervised learning, i.e., the multi-instance learning (MIL) model was conducted to evaluate the ability to differentiate between benign and malignant cases, and compared with the performance of supervised learning model. Finally, gradient-weighted class activation mapping (Grad-CAM) and semantic segmentation were performed to spatially resolve benign/malignant areas with high efficiency. Results: We verified the ability of SRS in revealing essential histological hallmarks of breast cancer in both thin frozen sections and fresh unprocessed biopsy, generating histoarchitectures well correlated with H&E staining. Moreover, we demonstrated that weakly-supervised MIL model could achieve superior classification performance to supervised learnings, reaching diagnostic accuracy of 95% on 61 biopsy specimens. Furthermore, Grad-CAM allowed the trained MIL model to visualize the histological heterogeneity within the CNB. Conclusion: Our results indicate that MIL-assisted SRS microscopy provides rapid and accurate diagnosis on histologically heterogeneous breast CNB, and could potentially help the subsequent management of patients.
Collapse
Affiliation(s)
- Yifan Yang
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute, Fudan University, Shanghai 200433, China
| | - Zhijie Liu
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute, Fudan University, Shanghai 200433, China
| | - Jing Huang
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute, Fudan University, Shanghai 200433, China
| | - Xiangjie Sun
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai 200032, China
| | - Jianpeng Ao
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute, Fudan University, Shanghai 200433, China
| | - Bin Zheng
- Otolaryngology & Head and Neck Center, Cancer Center, Department of Otolaryngology, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| | - Wanyuan Chen
- Cancer Center, Department of Pathology, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| | - Zhiming Shao
- Department of Breast Surgery, Fudan University Shanghai Cancer Center, Department of Oncology, Shanghai Medical College, Fudan University, Shanghai 200032, China
| | - Hao Hu
- Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, 200032 China
| | - Yinlong Yang
- Department of Breast Surgery, Fudan University Shanghai Cancer Center, Department of Oncology, Shanghai Medical College, Fudan University, Shanghai 200032, China
| | - Minbiao Ji
- State Key Laboratory of Surface Physics and Department of Physics, Human Phenome Institute, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Yiwu Research Institute, Fudan University, Shanghai 200433, China
| |
Collapse
|
35
|
Requa J, Godard T, Mandal R, Balzer B, Whittemore D, George E, Barcelona F, Lambert C, Lee J, Lambert A, Larson A, Osmond G. High-fidelity detection, subtyping, and localization of five skin neoplasms using supervised and semi-supervised learning. J Pathol Inform 2022; 14:100159. [PMID: 36506813 PMCID: PMC9731861 DOI: 10.1016/j.jpi.2022.100159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 11/16/2022] [Accepted: 11/22/2022] [Indexed: 11/27/2022] Open
Abstract
Background Skin cancers are the most common malignancies diagnosed worldwide. While the early detection and treatment of pre-cancerous and cancerous skin lesions can dramatically improve outcomes, factors such as a global shortage of pathologists, increased workloads, and high rates of diagnostic discordance underscore the need for techniques that improve pathology workflows. Although AI models are now being used to classify lesions from whole slide images (WSIs), diagnostic performance rarely surpasses that of expert pathologists. Objectives The objective of the present study was to create an AI model to detect and classify skin lesions with a higher degree of sensitivity than previously demonstrated, with potential to match and eventually surpass expert pathologists to improve clinical workflows. Methods We combined supervised learning (SL) with semi-supervised learning (SSL) to produce an end-to-end multi-level skin detection system that not only detects 5 main types of skin lesions with high sensitivity and specificity, but also subtypes, localizes, and provides margin status to evaluate the proximity of the lesion to non-epidermal margins. The Supervised Training Subset consisted of 2188 random WSIs collected by the PathologyWatch (PW) laboratory between 2013 and 2018, while the Weakly Supervised Subset consisted of 5161 WSIs from daily case specimens. The Validation Set consisted of 250 curated daily case WSIs obtained from the PW tissue archives and included 50 "mimickers". The Testing Set (3821 WSIs) was composed of non-curated daily case specimens collected from July 20, 2021 to August 20, 2021 from PW laboratories. Results The performance characteristics of our AI model (i.e., Mihm) were assessed retrospectively by running the Testing Set through the Mihm Evaluation Pipeline. Our results show that the sensitivity of Mihm in classifying melanocytic lesions, basal cell carcinoma, and atypical squamous lesions, verruca vulgaris, and seborrheic keratosis was 98.91% (95% CI: 98.27%, 99.55%), 97.24% (95% CI: 96.15%, 98.33%), 95.26% (95% CI: 93.79%, 96.73%), 93.50% (95% CI: 89.14%, 97.86%), and 86.91% (95% CI: 82.13%, 91.69%), respectively. Additionally, our multi-level (i.e., patch-level, ROI-level, and WSI-level) detection algorithm includes a qualitative feature that subtypes lesions, an AI overlay in the front-end digital display that localizes diagnostic ROIs, and reports on margin status by detecting overlap between lesions and non-epidermal tissue margins. Conclusions Our AI model, developed in collaboration with dermatopathologists, detects 5 skin lesion types with higher sensitivity than previously published AI models, and provides end users with information such as subtyping, localization, and margin status in a front-end digital display. Our end-to-end system has the potential to improve pathology workflows by increasing diagnostic accuracy, expediting the course of patient care, and ultimately improving patient outcomes.
Collapse
Affiliation(s)
- James Requa
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - Tuatini Godard
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - Rajni Mandal
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - Bonnie Balzer
- Cedars-Sinai Medical Center, 8700 Beverly Blvd, Los Angeles, CA 90048, USA
| | - Darren Whittemore
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - Eva George
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | | | - Chalette Lambert
- Kirk Kerkorian School of Medicine at UNLV, University of Nevada, Las Vegas, Mail Stop: 3070, 2040 W Charleston Blvd., Las Vegas, NV 89102-2244, USA
| | - Jonathan Lee
- Bethesda Dermatopathology Laboratory, 1730 Elton Road, Silver Spring, MD 20903, USA
| | - Allison Lambert
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - April Larson
- Pathology Watch, 497 West 4800 South, Suite 201, Murray, UT 84123, USA
| | - Gregory Osmond
- Intermountain Healthcare, Saint George Regional Hospital, Department of Pathology, 1380 East Medical Center Drive, Saint George, Utah 84790, USA,Corresponding author.
| |
Collapse
|
36
|
Deep Neural Network Models for Colon Cancer Screening. Cancers (Basel) 2022; 14:cancers14153707. [PMID: 35954370 PMCID: PMC9367621 DOI: 10.3390/cancers14153707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 07/26/2022] [Accepted: 07/27/2022] [Indexed: 12/24/2022] Open
Abstract
Simple Summary Deep learning models have been shown to achieve high performance in diagnosing colon cancer compared to conventional image processing and hand-crafted machine learning methods. Hence, several studies have focused on developing hybrid learning, end-to-end, and transfer learning techniques to reduce manual interaction and for labelling the regions of interest. However, these weak learning techniques do not always provide a clear diagnosis. Therefore, it is necessary to develop a clear explainable learning method that can highlight factors and form the basis of clinical decisions. However, there has been little research carried out employing such transparent approaches. This study discussed the aforementioned models for colon cancer diagnosis. Abstract Early detection of colorectal cancer can significantly facilitate clinicians’ decision-making and reduce their workload. This can be achieved using automatic systems with endoscopic and histological images. Recently, the success of deep learning has motivated the development of image- and video-based polyp identification and segmentation. Currently, most diagnostic colonoscopy rooms utilize artificial intelligence methods that are considered to perform well in predicting invasive cancer. Convolutional neural network-based architectures, together with image patches and preprocesses are often widely used. Furthermore, learning transfer and end-to-end learning techniques have been adopted for detection and localization tasks, which improve accuracy and reduce user dependence with limited datasets. However, explainable deep networks that provide transparency, interpretability, reliability, and fairness in clinical diagnostics are preferred. In this review, we summarize the latest advances in such models, with or without transparency, for the prediction of colorectal cancer and also address the knowledge gap in the upcoming technology.
Collapse
|
37
|
Wang W, Jiang R, Cui N, Li Q, Yuan F, Xiao Z. Semi-supervised vision transformer with adaptive token sampling for breast cancer classification. Front Pharmacol 2022; 13:929755. [PMID: 35935827 PMCID: PMC9353650 DOI: 10.3389/fphar.2022.929755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2022] [Accepted: 06/29/2022] [Indexed: 12/24/2022] Open
Abstract
Various imaging techniques combined with machine learning (ML) models have been used to build computer-aided diagnosis (CAD) systems for breast cancer (BC) detection and classification. The rise of deep learning models in recent years, represented by convolutional neural network (CNN) models, has pushed the accuracy of ML-based CAD systems to a new level that is comparable to human experts. Existing studies have explored the usage of a wide spectrum of CNN models for BC detection, and supervised learning has been the mainstream. In this study, we propose a semi-supervised learning framework based on the Vision Transformer (ViT). The ViT is a model that has been validated to outperform CNN models on numerous classification benchmarks but its application in BC detection has been rare. The proposed method offers a custom semi-supervised learning procedure that unifies both supervised and consistency training to enhance the robustness of the model. In addition, the method uses an adaptive token sampling technique that can strategically sample the most significant tokens from the input image, leading to an effective performance gain. We validate our method on two datasets with ultrasound and histopathology images. Results demonstrate that our method can consistently outperform the CNN baselines for both learning tasks. The code repository of the project is available at https://github.com/FeiYee/Breast-area-TWO.
Collapse
Affiliation(s)
- Wei Wang
- Department of Breast Surgery, Hubei Provincial Clinical Research Center for Breast Cancer, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Ran Jiang
- Department of Thyroid and Breast Surgery, Maternal and Child Health Hospital of Hubei Province, Wuhan, Hubei, China
| | - Ning Cui
- Department of Ultrasound, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Qian Li
- Department of Ultrasound, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Feng Yuan
- Department of Breast Surgery, Hubei Provincial Clinical Research Center for Breast Cancer, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
- *Correspondence: Feng Yuan, ; Zhifeng Xiao,
| | - Zhifeng Xiao
- School of Engineering,Penn State Erie, The Behrend College, Erie, PA, United States
- *Correspondence: Feng Yuan, ; Zhifeng Xiao,
| |
Collapse
|
38
|
Eckardt JN, Bornhäuser M, Wendt K, Middeke JM. Semi-supervised learning in cancer diagnostics. Front Oncol 2022; 12:960984. [PMID: 35912249 PMCID: PMC9329803 DOI: 10.3389/fonc.2022.960984] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Accepted: 06/24/2022] [Indexed: 12/01/2022] Open
Abstract
In cancer diagnostics, a considerable amount of data is acquired during routine work-up. Recently, machine learning has been used to build classifiers that are tasked with cancer detection and aid in clinical decision-making. Most of these classifiers are based on supervised learning (SL) that needs time- and cost-intensive manual labeling of samples by medical experts for model training. Semi-supervised learning (SSL), however, works with only a fraction of labeled data by including unlabeled samples for information abstraction and thus can utilize the vast discrepancy between available labeled data and overall available data in cancer diagnostics. In this review, we provide a comprehensive overview of essential functionalities and assumptions of SSL and survey key studies with regard to cancer care differentiating between image-based and non-image-based applications. We highlight current state-of-the-art models in histopathology, radiology and radiotherapy, as well as genomics. Further, we discuss potential pitfalls in SSL study design such as discrepancies in data distributions and comparison to baseline SL models, and point out future directions for SSL in oncology. We believe well-designed SSL models to strongly contribute to computer-guided diagnostics in malignant disease by overcoming current hinderances in the form of sparse labeled and abundant unlabeled data.
Collapse
Affiliation(s)
- Jan-Niklas Eckardt
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- *Correspondence: Jan-Niklas Eckardt,
| | - Martin Bornhäuser
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- German Consortium for Translational Cancer Research, Heidelberg, Germany
- National Center for Tumor Disease (NCT), Dresden, Germany
| | - Karsten Wendt
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- Institute of Software and Multimedia Technology, Technical University Dresden, Dresden, Germany
| | - Jan Moritz Middeke
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| |
Collapse
|
39
|
When artificial intelligence meets PD-1/PD-L1 inhibitors: Population screening, response prediction and efficacy evaluation. Comput Biol Med 2022; 145:105499. [DOI: 10.1016/j.compbiomed.2022.105499] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 03/26/2022] [Accepted: 04/03/2022] [Indexed: 02/07/2023]
|
40
|
Neto PC, Oliveira SP, Montezuma D, Fraga J, Monteiro A, Ribeiro L, Gonçalves S, Pinto IM, Cardoso JS. iMIL4PATH: A Semi-Supervised Interpretable Approach for Colorectal Whole-Slide Images. Cancers (Basel) 2022; 14:cancers14102489. [PMID: 35626093 PMCID: PMC9139905 DOI: 10.3390/cancers14102489] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 05/13/2022] [Accepted: 05/17/2022] [Indexed: 02/04/2023] Open
Abstract
Simple Summary Nowadays, colorectal cancer is the third most incident cancer worldwide and, although it can be detected by imaging techniques, diagnosis is always based on biopsy samples. This assessment includes neoplasia grading, a subjective yet important task for pathologists. With the growing availability of digital slides, the development of robust and high-performance computer vision algorithms can help to tackle such a task. In this work, we propose an approach to automatically detect and grade lesions in colorectal biopsies with high sensitivity. The presented model attempts to support slide decision reasoning in terms of the spatial distribution of lesions, focusing the pathologist’s attention on key areas. Thus, it can be integrated into clinical practice as a second opinion or as a flag for details that may have been missed at first glance. Abstract Colorectal cancer (CRC) diagnosis is based on samples obtained from biopsies, assessed in pathology laboratories. Due to population growth and ageing, as well as better screening programs, the CRC incidence rate has been increasing, leading to a higher workload for pathologists. In this sense, the application of AI for automatic CRC diagnosis, particularly on whole-slide images (WSI), is of utmost relevance, in order to assist professionals in case triage and case review. In this work, we propose an interpretable semi-supervised approach to detect lesions in colorectal biopsies with high sensitivity, based on multiple-instance learning and feature aggregation methods. The model was developed on an extended version of the recent, publicly available CRC dataset (the CRC+ dataset with 4433 WSI), using 3424 slides for training and 1009 slides for evaluation. The proposed method attained 90.19% classification ACC, 98.8% sensitivity, 85.7% specificity, and a quadratic weighted kappa of 0.888 at slide-based evaluation. Its generalisation capabilities are also studied on two publicly available external datasets.
Collapse
Affiliation(s)
- Pedro C. Neto
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
- Correspondence:
| | - Sara P. Oliveira
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
| | - Diana Montezuma
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
- School of Medicine and Biomedical Sciences, University of Porto (ICBAS), 4050-313 Porto, Portugal
- Cancer Biology and Epigenetics Group, IPO-Porto, 4200-072 Porto, Portugal
| | - João Fraga
- Department of Pathology, IPO-Porto, 4200-072 Porto, Portugal;
| | - Ana Monteiro
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Liliana Ribeiro
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Sofia Gonçalves
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Isabel M. Pinto
- IMP Diagnostics, 4150-146 Porto, Portugal; (D.M.); (A.M.); (L.R.); (S.G.); (I.M.P.)
| | - Jaime S. Cardoso
- Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal; (S.P.O.); (J.S.C.)
- Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal
| |
Collapse
|
41
|
Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review. Diagnostics (Basel) 2022; 12:diagnostics12040837. [PMID: 35453885 PMCID: PMC9028395 DOI: 10.3390/diagnostics12040837] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 02/04/2023] Open
Abstract
Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.
Collapse
|
42
|
Ginghina O, Hudita A, Zamfir M, Spanu A, Mardare M, Bondoc I, Buburuzan L, Georgescu SE, Costache M, Negrei C, Nitipir C, Galateanu B. Liquid Biopsy and Artificial Intelligence as Tools to Detect Signatures of Colorectal Malignancies: A Modern Approach in Patient's Stratification. Front Oncol 2022; 12:856575. [PMID: 35356214 PMCID: PMC8959149 DOI: 10.3389/fonc.2022.856575] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 02/16/2022] [Indexed: 01/19/2023] Open
Abstract
Colorectal cancer (CRC) is the second most frequently diagnosed type of cancer and a major worldwide public health concern. Despite the global efforts in the development of modern therapeutic strategies, CRC prognosis is strongly correlated with the stage of the disease at diagnosis. Early detection of CRC has a huge impact in decreasing mortality while pre-lesion detection significantly reduces the incidence of the pathology. Even though the management of CRC patients is based on robust diagnostic methods such as serum tumor markers analysis, colonoscopy, histopathological analysis of tumor tissue, and imaging methods (computer tomography or magnetic resonance), these strategies still have many limitations and do not fully satisfy clinical needs due to their lack of sensitivity and/or specificity. Therefore, improvements of the current practice would substantially impact the management of CRC patients. In this view, liquid biopsy is a promising approach that could help clinicians screen for disease, stratify patients to the best treatment, and monitor treatment response and resistance mechanisms in the tumor in a regular and minimally invasive manner. Liquid biopsies allow the detection and analysis of different tumor-derived circulating markers such as cell-free nucleic acids (cfNA), circulating tumor cells (CTCs), and extracellular vesicles (EVs) in the bloodstream. The major advantage of this approach is its ability to trace and monitor the molecular profile of the patient's tumor and to predict personalized treatment in real-time. On the other hand, the prospective use of artificial intelligence (AI) in medicine holds great promise in oncology, for the diagnosis, treatment, and prognosis prediction of disease. AI has two main branches in the medical field: (i) a virtual branch that includes medical imaging, clinical assisted diagnosis, and treatment, as well as drug research, and (ii) a physical branch that includes surgical robots. This review summarizes findings relevant to liquid biopsy and AI in CRC for better management and stratification of CRC patients.
Collapse
Affiliation(s)
- Octav Ginghina
- Department II, University of Medicine and Pharmacy “Carol Davila” Bucharest, Bucharest, Romania
- Department of Surgery, “Sf. Ioan” Clinical Emergency Hospital, Bucharest, Romania
| | - Ariana Hudita
- Department of Biochemistry and Molecular Biology, University of Bucharest, Bucharest, Romania
| | - Marius Zamfir
- Department of Surgery, “Sf. Ioan” Clinical Emergency Hospital, Bucharest, Romania
| | - Andrada Spanu
- Department of Surgery, “Sf. Ioan” Clinical Emergency Hospital, Bucharest, Romania
| | - Mara Mardare
- Department of Surgery, “Sf. Ioan” Clinical Emergency Hospital, Bucharest, Romania
| | - Irina Bondoc
- Department of Surgery, “Sf. Ioan” Clinical Emergency Hospital, Bucharest, Romania
| | | | - Sergiu Emil Georgescu
- Department of Biochemistry and Molecular Biology, University of Bucharest, Bucharest, Romania
| | - Marieta Costache
- Department of Biochemistry and Molecular Biology, University of Bucharest, Bucharest, Romania
| | - Carolina Negrei
- Department of Toxicology, University of Medicine and Pharmacy “Carol Davila” Bucharest, Bucharest, Romania
| | - Cornelia Nitipir
- Department II, University of Medicine and Pharmacy “Carol Davila” Bucharest, Bucharest, Romania
- Department of Oncology, Elias University Emergency Hospital, Bucharest, Romania
| | - Bianca Galateanu
- Department of Biochemistry and Molecular Biology, University of Bucharest, Bucharest, Romania
| |
Collapse
|