1
|
Stenman S, Bétrisey S, Vainio P, Huvila J, Lundin M, Linder N, Schmitt A, Perren A, Dettmer MS, Haglund C, Arola J, Lundin J. External validation of a deep learning-based algorithm for detection of tall cells in papillary thyroid carcinoma: A multicenter study. J Pathol Inform 2024; 15:100366. [PMID: 38425542 PMCID: PMC10901856 DOI: 10.1016/j.jpi.2024.100366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Revised: 01/03/2024] [Accepted: 01/29/2024] [Indexed: 03/02/2024] Open
Abstract
The tall cell subtype (TC-PTC) is an aggressive subtype of papillary thyroid carcinoma (PTC). The TC-PTC is defined as a PTC comprising at least 30% epithelial cells that are three times as tall as they are wide. In practice, this definition is difficult to adhere to, resulting in high inter-observer variability. In this multicenter study, we validated a previously trained deep learning (DL)-based algorithm for detection of tall cells on 160 externally collected hematoxylin and eosin (HE)-stained PTC whole-slide images. In a test set of 360 manual annotations of regions of interest from 18 separate tissue sections in the external dataset, the DL-based algorithm detected TCs with a sensitivity of 90.6% and a specificity of 88.5%. The DL algorithm detected non-TC areas with a sensitivity of 81.6% and a specificity of 92.9%. In the validation datasets, 20% and 30% TC thresholds correlated with a significantly shorter relapse-free survival. In conclusion, the DL algorithm detected TCs in unseen, external scanned HE tissue slides with high sensitivity and specificity without any retraining.
Collapse
Affiliation(s)
- Sebastian Stenman
- Institute for Molecular Medicine Finland – FIMM, University of Helsinki, Tukholmankatu 8, 00290 Helsinki, Finland
- HUSLAB, Department of Pathology, HUS Diagnostic Center, Helsinki University Hospital and University of Helsinki, Haartmaninkatu 3C, 000290 HUS Helsinki, Finland
- Department of Surgery, Helsinki University Hospital, Haartmaninkatu 4, 00290 Helsinki, Finland
| | - Sylvain Bétrisey
- Institute of Tissue Medicine and Pathology, University of Bern, Murtenstrasse 31, 3008 Bern, Switzerland
| | - Paula Vainio
- Department of Pathology, University of Turku, Turku University Hospital, Kiinamyllykatu 10, 20520 Turku, Finland
| | - Jutta Huvila
- Department of Pathology, University of Turku, Turku University Hospital, Kiinamyllykatu 10, 20520 Turku, Finland
| | - Mikael Lundin
- Institute for Molecular Medicine Finland – FIMM, University of Helsinki, Tukholmankatu 8, 00290 Helsinki, Finland
| | - Nina Linder
- Institute for Molecular Medicine Finland – FIMM, University of Helsinki, Tukholmankatu 8, 00290 Helsinki, Finland
- Institute of Pathology, Klinikum Stuttgart, Kriegsbergstraße 60, 70174 Stuttgart, Germany
| | - Anja Schmitt
- Institute of Tissue Medicine and Pathology, University of Bern, Murtenstrasse 31, 3008 Bern, Switzerland
| | - Aurel Perren
- Institute of Tissue Medicine and Pathology, University of Bern, Murtenstrasse 31, 3008 Bern, Switzerland
| | - Matthias S. Dettmer
- Institute of Tissue Medicine and Pathology, University of Bern, Murtenstrasse 31, 3008 Bern, Switzerland
- The Global Health & Migration Department of Women’s and Children’s Health, Uppsala University, 75185 Uppsala, Sweden
| | - Caj Haglund
- Department of Surgery, Helsinki University Hospital, Haartmaninkatu 4, 00290 Helsinki, Finland
- Research Programs Unit, Translational Cancer Medicine, University of Helsinki, Haartmaninkatu 4, 00290 Helsinki, Finland
| | - Johanna Arola
- HUSLAB, Department of Pathology, HUS Diagnostic Center, Helsinki University Hospital and University of Helsinki, Haartmaninkatu 3C, 000290 HUS Helsinki, Finland
| | - Johan Lundin
- Institute for Molecular Medicine Finland – FIMM, University of Helsinki, Tukholmankatu 8, 00290 Helsinki, Finland
- Department of Global Public Health, Karolinska Institutet, Norrbackagatan 4, 17176 Stockholm, Sweden
- iCAN Digital Precision Cancer Medicine Flagship, Helsinki, Finland
| |
Collapse
|
2
|
Ye Z, Zhang D, Zhao Y, Chen M, Wang H, Seery S, Qu Y, Xue P, Jiang Y. Deep learning algorithms for melanoma detection using dermoscopic images: A systematic review and meta-analysis. Artif Intell Med 2024; 155:102934. [PMID: 39088883 DOI: 10.1016/j.artmed.2024.102934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 06/21/2024] [Accepted: 07/22/2024] [Indexed: 08/03/2024]
Abstract
BACKGROUND Melanoma is a serious risk to human health and early identification is vital for treatment success. Deep learning (DL) has the potential to detect cancer using imaging technologies and many studies provide evidence that DL algorithms can achieve high accuracy in melanoma diagnostics. OBJECTIVES To critically assess different DL performances in diagnosing melanoma using dermatoscopic images and discuss the relationship between dermatologists and DL. METHODS Ovid-Medline, Embase, IEEE Xplore, and the Cochrane Library were systematically searched from inception until 7th December 2021. Studies that reported diagnostic DL model performances in detecting melanoma using dermatoscopic images were included if they had specific outcomes and histopathologic confirmation. Binary diagnostic accuracy data and contingency tables were extracted to analyze outcomes of interest, which included sensitivity (SEN), specificity (SPE), and area under the curve (AUC). Subgroup analyses were performed according to human-machine comparison and cooperation. The study was registered in PROSPERO, CRD42022367824. RESULTS 2309 records were initially retrieved, of which 37 studies met our inclusion criteria, and 27 provided sufficient data for meta-analytical synthesis. The pooled SEN was 82 % (range 77-86), SPE was 87 % (range 84-90), with an AUC of 0.92 (range 0.89-0.94). Human-machine comparison had pooled AUCs of 0.87 (0.84-0.90) and 0.83 (0.79-0.86) for DL and dermatologists, respectively. Pooled AUCs were 0.90 (0.87-0.93), 0.80 (0.76-0.83), and 0.88 (0.85-0.91) for DL, and junior and senior dermatologists, respectively. Analyses of human-machine cooperation were 0.88 (0.85-0.91) for DL, 0.76 (0.72-0.79) for unassisted, and 0.87 (0.84-0.90) for DL-assisted dermatologists. CONCLUSIONS Evidence suggests that DL algorithms are as accurate as senior dermatologists in melanoma diagnostics. Therefore, DL could be used to support dermatologists in diagnostic decision-making. Although, further high-quality, large-scale multicenter studies are required to address the specific challenges associated with medical AI-based diagnostics.
Collapse
Affiliation(s)
- Zichen Ye
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Daqian Zhang
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuankai Zhao
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyang Chen
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Huike Wang
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Samuel Seery
- Population Health Sciences Institute, School of Pharmacy, Newcastle University, Newcastle NE1 7RU, United Kingdom of Great Britain and Northern Ireland
| | - Yimin Qu
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Peng Xue
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
| | - Yu Jiang
- School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
| |
Collapse
|
3
|
Chen H, Zhang B, Huang J. Recent advances and applications of artificial intelligence in 3D bioprinting. BIOPHYSICS REVIEWS 2024; 5:031301. [PMID: 39036708 PMCID: PMC11260195 DOI: 10.1063/5.0190208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 06/11/2024] [Indexed: 07/23/2024]
Abstract
3D bioprinting techniques enable the precise deposition of living cells, biomaterials, and biomolecules, emerging as a promising approach for engineering functional tissues and organs. Meanwhile, recent advances in 3D bioprinting enable researchers to build in vitro models with finely controlled and complex micro-architecture for drug screening and disease modeling. Recently, artificial intelligence (AI) has been applied to different stages of 3D bioprinting, including medical image reconstruction, bioink selection, and printing process, with both classical AI and machine learning approaches. The ability of AI to handle complex datasets, make complex computations, learn from past experiences, and optimize processes dynamically makes it an invaluable tool in advancing 3D bioprinting. The review highlights the current integration of AI in 3D bioprinting and discusses future approaches to harness the synergistic capabilities of 3D bioprinting and AI for developing personalized tissues and organs.
Collapse
Affiliation(s)
| | - Bin Zhang
- Department of Mechanical and Aerospace Engineering, Brunel University London, London, United Kingdom
| | - Jie Huang
- Department of Mechanical Engineering, University College London, London, United Kingdom
| |
Collapse
|
4
|
Hakami MA. Harnessing machine learning potential for personalised drug design and overcoming drug resistance. J Drug Target 2024; 32:918-930. [PMID: 38842417 DOI: 10.1080/1061186x.2024.2365934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 06/01/2024] [Accepted: 06/04/2024] [Indexed: 06/07/2024]
Abstract
Drug resistance in cancer treatment presents a significant challenge, necessitating innovative approaches to improve therapeutic efficacy. Integrating machine learning (ML) in cancer research is promising as ML algorithms outrival in analysing complex datasets, identifying patterns, and predicting treatment outcomes. Leveraging diverse data sources such as genomic profiles, clinical records, and drug response assays, ML uncovers molecular mechanisms of drug resistance, enabling personalised treatment, maximising efficacy and minimising adverse effects. Various ML algorithms contribute to the drug discovery process - Random Forest and Decision Trees predict drug-target interactions and aid in virtual screening, and SVM classify leads on bioactivity data. Neural Networks model QSAR to optimise lead compounds and K-means clustering group compounds with similar chemical properties aiding compound selection. Gaussian Processes predict drug responses, Bayesian Networks infer causal relationships, Autoencoders generate novel compounds, and Genetic Algorithms optimise molecular structures. These algorithms collectively enhance efficiency and success rates in drug design endeavours, from lead identification to optimisation and are cost-effective, empowering clinicians with real-time treatment monitoring and improving patient outcomes. This review highlights the immense potential of ML in revolutionising cancer care through effective drug design to reduce drug resistance, and we have also discussed various limitations and research gaps to understand better.
Collapse
Affiliation(s)
- Mohammed Ageeli Hakami
- Department of Clinical Laboratory Sciences, College of Applied Medical Sciences, Shaqra University, Al-Quwayiyah, Riyadh, Saudi Arabia
| |
Collapse
|
5
|
Huynh BN, Groendahl AR, Tomic O, Liland KH, Knudtsen IS, Hoebers F, van Elmpt W, Dale E, Malinen E, Futsaether CM. Deep learning with uncertainty estimation for automatic tumor segmentation in PET/CT of head and neck cancers: impact of model complexity, image processing and augmentation. Biomed Phys Eng Express 2024; 10:055038. [PMID: 39127060 DOI: 10.1088/2057-1976/ad6dcd] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 08/09/2024] [Indexed: 08/12/2024]
Abstract
Objective.Target volumes for radiotherapy are usually contoured manually, which can be time-consuming and prone to inter- and intra-observer variability. Automatic contouring by convolutional neural networks (CNN) can be fast and consistent but may produce unrealistic contours or miss relevant structures. We evaluate approaches for increasing the quality and assessing the uncertainty of CNN-generated contours of head and neck cancers with PET/CT as input.Approach.Two patient cohorts with head and neck squamous cell carcinoma and baseline18F-fluorodeoxyglucose positron emission tomography and computed tomography images (FDG-PET/CT) were collected retrospectively from two centers. The union of manual contours of the gross primary tumor and involved nodes was used to train CNN models for generating automatic contours. The impact of image preprocessing, image augmentation, transfer learning and CNN complexity, architecture, and dimension (2D or 3D) on model performance and generalizability across centers was evaluated. A Monte Carlo dropout technique was used to quantify and visualize the uncertainty of the automatic contours.Main results. CNN models provided contours with good overlap with the manually contoured ground truth (median Dice Similarity Coefficient: 0.75-0.77), consistent with reported inter-observer variations and previous auto-contouring studies. Image augmentation and model dimension, rather than model complexity, architecture, or advanced image preprocessing, had the largest impact on model performance and cross-center generalizability. Transfer learning on a limited number of patients from a separate center increased model generalizability without decreasing model performance on the original training cohort. High model uncertainty was associated with false positive and false negative voxels as well as low Dice coefficients.Significance.High quality automatic contours can be obtained using deep learning architectures that are not overly complex. Uncertainty estimation of the predicted contours shows potential for highlighting regions of the contour requiring manual revision or flagging segmentations requiring manual inspection and intervention.
Collapse
Affiliation(s)
- Bao Ngoc Huynh
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Aurora Rosvoll Groendahl
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
- Section of Oncology, Vestre Viken Hospital Trust, Drammen, Norway
| | - Oliver Tomic
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Kristian Hovde Liland
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Ingerid Skjei Knudtsen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway
| | - Frank Hoebers
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Reproduction, Maastricht, Netherlands
| | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Reproduction, Maastricht, Netherlands
| | - Einar Dale
- Department of Oncology, Oslo University Hospital, Oslo, Norway
| | - Eirik Malinen
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
- Department of Physics, University of Oslo, Oslo, Norway
| | | |
Collapse
|
6
|
Kludt C, Wang Y, Ahmad W, Bychkov A, Fukuoka J, Gaisa N, Kühnel M, Jonigk D, Pryalukhin A, Mairinger F, Klein F, Schultheis AM, Seper A, Hulla W, Brägelmann J, Michels S, Klein S, Quaas A, Büttner R, Tolkach Y. Next-generation lung cancer pathology: Development and validation of diagnostic and prognostic algorithms. Cell Rep Med 2024:101697. [PMID: 39178857 DOI: 10.1016/j.xcrm.2024.101697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 06/25/2024] [Accepted: 07/31/2024] [Indexed: 08/26/2024]
Abstract
Non-small cell lung cancer (NSCLC) is one of the most common malignant tumors. In this study, we develop a clinically useful computational pathology platform for NSCLC that can be a foundation for multiple downstream applications and provide immediate value for patient care optimization and individualization. We train the primary multi-class tissue segmentation algorithm on a substantial, high-quality, manually annotated dataset of whole-slide images with lung adenocarcinoma and squamous cell carcinomas. We investigate two downstream applications. NSCLC subtyping algorithm is trained and validated using a large, multi-institutional (n = 6), multi-scanner (n = 5), international cohort of NSCLC cases (slides/patients 4,097/1,527). Moreover, we develop four AI-derived, fully explainable, quantitative, prognostic parameters (based on tertiary lymphoid structure and necrosis assessment) and validate them for different clinical endpoints. The computational platform enables the high-precision, quantitative analysis of H&E-stained slides. The developed prognostic parameters facilitate robust and independent risk stratification of patients with NSCLC.
Collapse
Affiliation(s)
- Carina Kludt
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Yuan Wang
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Waleed Ahmad
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa 296-0041, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki 852-8131, Japan
| | - Junya Fukuoka
- Department of Pathology, Kameda Medical Center, Kamogawa 296-0041, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki 852-8131, Japan
| | - Nadine Gaisa
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany; Institute of Pathology, University Hospital Ulm, 89081 Ulm, Germany
| | - Mark Kühnel
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany
| | - Danny Jonigk
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany; German Center for Lung Research, DZL, BREATH, 30625 Hanover, Germany
| | - Alexey Pryalukhin
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria
| | - Fabian Mairinger
- Institute of Pathology, University Hospital Essen, 45147 Essen, Germany
| | - Franziska Klein
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Anne Maria Schultheis
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Alexander Seper
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria; Danube Private University, 3500 Krems an der Donau, Austria
| | - Wolfgang Hulla
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria
| | - Johannes Brägelmann
- University of Cologne, Faculty of Medicine and University Hospital Cologne, Department of Translational Genomics, 50937 Cologne, Germany; Mildred Scheel School of Oncology, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937 Cologne, Germany; University of Cologne, Faculty of Medicine and University Hospital Cologne, Center for Molecular Medicine Cologne, 50937 Cologne, Germany
| | - Sebastian Michels
- University of Cologne, Faculty of Medicine and University Hospital of Colone, Lung Cancer Group Cologne, Department I for Internal Medicine and Center for Integrated Oncology Aachen Bonn Cologne Dusseldorf, 50937 Cologne, Germany
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany.
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany.
| |
Collapse
|
7
|
Lee JH, Ku E, Chung YS, Kim YJ, Kim KG. Intraoperative detection of parathyroid glands using artificial intelligence: optimizing medical image training with data augmentation methods. Surg Endosc 2024:10.1007/s00464-024-11115-z. [PMID: 39138679 DOI: 10.1007/s00464-024-11115-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2024] [Accepted: 07/21/2024] [Indexed: 08/15/2024]
Abstract
BACKGROUND Postoperative hypoparathyroidism is a major complication of thyroidectomy, occurring when the parathyroid glands are inadvertently damaged during surgery. Although intraoperative images are rarely used to train artificial intelligence (AI) because of its complex nature, AI may be trained to intraoperatively detect parathyroid glands using various augmentation methods. The purpose of this study was to train an effective AI model to detect parathyroid glands during thyroidectomy. METHODS Video clips of the parathyroid gland were collected during thyroid lobectomy procedures. Confirmed parathyroid images were used to train three types of datasets according to augmentation status: baseline, geometric transformation, and generative adversarial network-based image inpainting. The primary outcome was the average precision of the performance of AI in detecting parathyroid glands. RESULTS 152 Fine-needle aspiration-confirmed parathyroid gland images were acquired from 150 patients who underwent unilateral lobectomy. The average precision of the AI model in detecting parathyroid glands based on baseline data was 77%. This performance was enhanced by applying both geometric transformation and image inpainting augmentation methods, with the geometric transformation data augmentation dataset showing a higher average precision (79%) than the image inpainting model (78.6%). When this model was subjected to external validation using a completely different thyroidectomy approach, the image inpainting method was more effective (46%) than both the geometric transformation (37%) and baseline (33%) methods. CONCLUSION This AI model was found to be an effective and generalizable tool in the intraoperative identification of parathyroid glands during thyroidectomy, especially when aided by appropriate augmentation methods. Additional studies comparing model performance and surgeon identification, however, are needed to assess the true clinical relevance of this AI model.
Collapse
Affiliation(s)
- Joon-Hyop Lee
- Division of Endocrine Surgery, Department of Surgery, Samsung Medical Center, 81 Irwon-ro, Gangnam-gu, Seoul, Korea
| | - EunKyung Ku
- Department of Digital Media, The Catholic University of Korea, 43, Jibong-ro, Wonmi-gu, Bucheon, Gyeonggi, 14662, Korea
| | - Yoo Seung Chung
- Division of Endocrine Surgery, Department of Surgery, Gachon University, College of Medicine, Gil Medical Center, Incheon, Korea
| | - Young Jae Kim
- Department of Biomedical Engineering, College of Medicine, Gachon University, Gil Medical Center, 38-13 Dokjeom-ro 3Beon-gil, Namdong-gu, Incheon, 21565, Korea
| | - Kwang Gi Kim
- Department of Biomedical Engineering, College of Medicine, Gachon University, Gil Medical Center, 38-13 Dokjeom-ro 3Beon-gil, Namdong-gu, Incheon, 21565, Korea.
| |
Collapse
|
8
|
Anyin Y, Jianping L, Mengru L, Hong Z, Xulei Z, Lianping W. Integrating bioinformatics and machine learning methods to analyze diagnostic biomarkers for HBV-induced hepatocellular carcinoma. Diagn Pathol 2024; 19:105. [PMID: 39095799 PMCID: PMC11295615 DOI: 10.1186/s13000-024-01528-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Accepted: 07/21/2024] [Indexed: 08/04/2024] Open
Abstract
Hepatocellular carcinoma (HCC) is a malignant tumor. It is estimated that approximately 50-80% of HCC cases worldwide are caused by hepatitis b virus (HBV) infection, and other pathogenic factors have been shown to promote the development of HCC when coexisting with HBV. Understanding the molecular mechanisms of HBV-induced hepatocellular carcinoma (HBV-HCC) is crucial for the prevention, diagnosis, and treatment of the disease. In this study, we analyzed the molecular mechanisms of HBV-induced HCC by combining bioinformatics and deep learning methods. Firstly, we collected a gene set related to HBV-HCC from the GEO database, performed differential analysis and WGCNA analysis to identify genes with abnormal expression in tumors and high relevance to tumors. We used three deep learning methods, Lasso, random forest, and SVM, to identify key genes RACGAP1, ECT2, and NDC80. By establishing a diagnostic model, we determined the accuracy of key genes in diagnosing HBV-HCC. In the training set, RACGAP1(AUC:0.976), ECT2(AUC:0.969), and NDC80 (AUC: 0.976) showed high accuracy. They also exhibited good accuracy in the validation set: RACGAP1(AUC:0.878), ECT2(AUC:0.731), and NDC80(AUC:0.915). The key genes were found to be highly expressed in liver cancer tissues compared to normal liver tissues, and survival analysis indicated that high expression of key genes was associated with poor prognosis in liver cancer patients. This suggests a close relationship between key genes RACGAP1, ECT2, and NDC80 and the occurrence and progression of HBV-HCC. Molecular docking results showed that the key genes could spontaneously bind to the anti-hepatocellular carcinoma drugs Lenvatinib, Regorafenib, and Sorafenib with strong binding activity. Therefore, ECT2, NDC80, and RACGAP1 may serve as potential biomarkers for the diagnosis of HBV-HCC and as targets for the development of targeted therapeutic drugs.
Collapse
Affiliation(s)
- Yang Anyin
- Department of Pharmacy, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China
| | - Liu Jianping
- Department of Pharmacy, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China
| | - Li Mengru
- Department of Hospital Infection Management Section, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China
| | - Zhang Hong
- Department of Pharmacy, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China
| | - Zhang Xulei
- Department of Liver Disease, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China.
| | - Wu Lianping
- Department of Pharmacy, Gaochun People's Hospital, Gaochun Hospital Affiliated to Jiangsu University, Nanjing, 211300, China.
| |
Collapse
|
9
|
Zheng Q, Wang X, Yang R, Fan J, Yuan J, Liu X, Wang L, Xiao Z, Chen Z. Predicting tumor mutation burden and VHL mutation from renal cancer pathology slides with self-supervised deep learning. Cancer Med 2024; 13:e70112. [PMID: 39166457 PMCID: PMC11336896 DOI: 10.1002/cam4.70112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 05/28/2024] [Accepted: 08/04/2024] [Indexed: 08/23/2024] Open
Abstract
BACKGROUND Tumor mutation burden (TMB) and VHL mutation play a crucial role in the management of patients with clear cell renal cell carcinoma (ccRCC), such as guiding adjuvant chemotherapy and improving clinical outcomes. However, the time-consuming and expensive high-throughput sequencing methods severely limit their clinical applicability. Predicting intratumoral heterogeneity poses significant challenges in biology and clinical settings. Our aimed to develop a self-supervised attention-based multiple instance learning (SSL-ABMIL) model to predict TMB and VHL mutation status from hematoxylin and eosin-stained histopathological images. METHODS We obtained whole slide images (WSIs) and somatic mutation data of 350 ccRCC patients from The Cancer Genome Atlas for developing SSL-ABMIL model. In parallel, 163 ccRCC patients from Clinical Proteomic Tumor Analysis Consortium cohort was used as independent external validation set. We systematically compared three different models (Wang-ABMIL, Ciga-ABMIL, and ImageNet-MIL) for their ability to predict TMB and VHL alterations. RESULTS We first identified two groups of populations with high- and low-TMB (cut-off point = 0.9). In two independent cohorts, the Wang-ABMIL model achieved the highest performance with decent generalization performance (AUROC = 0.83 ± 0.02 and 0.8 ± 0.04 in predicting TMB and VHL, respectively). Attention heatmaps revealed that the Wang-ABMIL model paid the highest attention to tumor regions in high-TMB patients, while in VHL mutation prediction, non-tumor regions were also assigned high attention, particularly the stromal regions infiltrated by lymphocytes. CONCLUSIONS Our results indicated that SSL-ABMIL can effectively extract histological features for predicting TMB and VHL mutation, demonstrating promising results in linking tumor morphology and molecular biology.
Collapse
Affiliation(s)
- Qingyuan Zheng
- Department of UrologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
- Institute of Urologic DiseaseRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Xinyu Wang
- Centre for Reproductive ScienceRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Rui Yang
- Department of UrologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
- Institute of Urologic DiseaseRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Junjie Fan
- University of Chinese Academy of SciencesBeijingChina
- Trusted Computing and Information Assurance LaboratoryInstitute of Software, Chinese Academy of SciencesBeijingChina
| | - Jingping Yuan
- Department of PathologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Xiuheng Liu
- Department of UrologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
- Institute of Urologic DiseaseRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Lei Wang
- Department of UrologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
- Institute of Urologic DiseaseRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Zhuoni Xiao
- Centre for Reproductive ScienceRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| | - Zhiyuan Chen
- Department of UrologyRenmin Hospital of Wuhan UniversityWuhanHubeiChina
- Institute of Urologic DiseaseRenmin Hospital of Wuhan UniversityWuhanHubeiChina
| |
Collapse
|
10
|
Jiao P, Zheng Q, Yang R, Ni X, Wu J, Chen Z, Liu X. Prediction of HER2 Status Based on Deep Learning in H&E-Stained Histopathology Images of Bladder Cancer. Biomedicines 2024; 12:1583. [PMID: 39062155 PMCID: PMC11274957 DOI: 10.3390/biomedicines12071583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2024] [Revised: 07/06/2024] [Accepted: 07/12/2024] [Indexed: 07/28/2024] Open
Abstract
Epidermal growth factor receptor 2 (HER2) has been widely recognized as one of the targets for bladder cancer immunotherapy. The key to implementing personalized treatment for bladder cancer patients lies in achieving rapid and accurate diagnosis. To tackle this challenge, we have pioneered the application of deep learning techniques to predict HER2 expression status from H&E-stained pathological images of bladder cancer, bypassing the need for intricate IHC staining or high-throughput sequencing methods. Our model, when subjected to rigorous testing within the cohort from the People's Hospital of Wuhan University, which encompasses 106 cases, has exhibited commendable performance on both the validation and test datasets. Specifically, the validation set yielded an AUC of 0.92, an accuracy of 0.86, a sensitivity of 0.87, a specificity of 0.83, and an F1 score of 86.7%. The corresponding metrics for the test set were 0.88 for AUC, 0.67 for accuracy, 0.56 for sensitivity, 0.75 for specificity, and 77.8% for F1 score. Additionally, in a direct comparison with pathologists, our model demonstrated statistically superior performance, with a p-value less than 0.05, highlighting its potential as a powerful diagnostic tool.
Collapse
Affiliation(s)
- Panpan Jiao
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Qingyuan Zheng
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Rui Yang
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Xinmiao Ni
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Jiejun Wu
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Zhiyuan Chen
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Xiuheng Liu
- Department of Urology, Renmin Hospital of Wuhan University, Wuhan 430060, China; (P.J.); (Q.Z.); (R.Y.); (X.N.); (J.W.)
- Institute of Urologic Disease, Renmin Hospital of Wuhan University, Wuhan 430060, China
| |
Collapse
|
11
|
Sokolov AV, Schiöth HB. Decoding depression: a comprehensive multi-cohort exploration of blood DNA methylation using machine learning and deep learning approaches. Transl Psychiatry 2024; 14:287. [PMID: 39009577 PMCID: PMC11250806 DOI: 10.1038/s41398-024-02992-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 06/24/2024] [Accepted: 06/27/2024] [Indexed: 07/17/2024] Open
Abstract
The causes of depression are complex, and the current diagnosis methods rely solely on psychiatric evaluations with no incorporation of laboratory biomarkers in clinical practices. We investigated the stability of blood DNA methylation depression signatures in six different populations using six public and two domestic cohorts (n = 1942) conducting mega-analysis and meta-analysis of the individual studies. We evaluated 12 machine learning and deep learning strategies for depression classification both in cross-validation (CV) and in hold-out tests using merged data from 8 separate batches, constructing models with both biased and unbiased feature selection. We found 1987 CpG sites related to depression in both mega- and meta-analysis at the nominal level, and the associated genes were nominally related to axon guidance and immune pathways based on enrichment analysis and eQTM data. Random forest classifiers achieved the highest performance (AUC 0.73 and 0.76) in CV and hold-out tests respectively on the batch-level processed data. In contrast, the methylation showed low predictive power (all AUCs < 0.57) for all classifiers in CV and no predictive power in hold-out tests when used with harmonized data. All models achieved significantly better performance (>14% gain in AUCs) with pre-selected features (selection bias), with some of the models (joint autoencoder-classifier) reaching AUCs of up to 0.91 in the final testing regardless of data preparation. Different algorithmic feature selection approaches may outperform limma, however, random forest models perform well regardless of the strategy. The results provide an overview over potential future biomarkers for depression and highlight many important methodological aspects for DNA methylation-based depression profiling including the use of machine learning strategies.
Collapse
Affiliation(s)
- Aleksandr V Sokolov
- Department of Surgical Sciences, Functional Pharmacology and Neuroscience, Uppsala University, Uppsala, Sweden
| | - Helgi B Schiöth
- Department of Surgical Sciences, Functional Pharmacology and Neuroscience, Uppsala University, Uppsala, Sweden.
| |
Collapse
|
12
|
Kildal W, Cyll K, Kalsnes J, Islam R, Julbø FM, Pradhan M, Ersvær E, Shepherd N, Vlatkovic L, Tekpli X, Garred Ø, Kristensen GB, Askautrud HA, Hveem TS, Danielsen HE. Deep learning for automated scoring of immunohistochemically stained tumour tissue sections - Validation across tumour types based on patient outcomes. Heliyon 2024; 10:e32529. [PMID: 39040241 PMCID: PMC11261074 DOI: 10.1016/j.heliyon.2024.e32529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2024] [Accepted: 06/05/2024] [Indexed: 07/24/2024] Open
Abstract
We aimed to develop deep learning (DL) models to detect protein expression in immunohistochemically (IHC) stained tissue-sections, and to compare their accuracy and performance with manually scored clinically relevant proteins in common cancer types. Five cancer patient cohorts (colon, two prostate, breast, and endometrial) were included. We developed separate DL models for scoring IHC-stained tissue-sections with nuclear, cytoplasmic, and membranous staining patterns. For training, we used images with annotations of cells with positive and negative staining from the colon cohort stained for Ki-67 and PMS2 (nuclear model), the prostate cohort 1 stained for PTEN (cytoplasmic model) and β-catenin (membranous model). The nuclear DL model was validated for MSH6 in the colon, MSH6 and PMS2 in the endometrium, Ki-67 and CyclinB1 in prostate, and oestrogen and progesterone receptors in the breast cancer cohorts. The cytoplasmic DL model was validated for PTEN and Mapre2, and the membranous DL model for CD44 and Flotillin1, all in prostate cohorts. When comparing the results of manual and DL scores in the validation sets, using manual scores as the ground truth, we observed an average correct classification rate of 91.5 % (76.9-98.5 %) for the nuclear model, 85.6 % (73.3-96.6 %) for the cytoplasmic model, and 78.4 % (75.5-84.3 %) for the membranous model. In survival analyses, manual and DL scores showed similar prognostic impact, with similar hazard ratios and p-values for all DL models. Our findings demonstrate that DL models offer a promising alternative to manual IHC scoring, providing efficiency and reproducibility across various data sources and markers.
Collapse
Affiliation(s)
- Wanja Kildal
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Karolina Cyll
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Joakim Kalsnes
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Rakibul Islam
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Frida M. Julbø
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Manohar Pradhan
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Elin Ersvær
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Neil Shepherd
- Gloucestershire Cellular Pathology Laboratory, Gloucester, GL53 7AN, UK
| | - Ljiljana Vlatkovic
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - OSBREAC
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
- Gloucestershire Cellular Pathology Laboratory, Gloucester, GL53 7AN, UK
- Department of Medical Genetics, Institute of Clinical Medicine, Faculty of Medicine, University of Oslo and Oslo University Hospital, NO-0450, Oslo, Norway
- Department of Pathology, Oslo University Hospital, NO-0424, Oslo, Norway
- Nuffield Division of Clinical Laboratory Sciences, University of Oxford, Oxford, OX3 9DU, UK
| | - Xavier Tekpli
- Department of Medical Genetics, Institute of Clinical Medicine, Faculty of Medicine, University of Oslo and Oslo University Hospital, NO-0450, Oslo, Norway
| | - Øystein Garred
- Department of Pathology, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Gunnar B. Kristensen
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Hanne A. Askautrud
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Tarjei S. Hveem
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
| | - Håvard E. Danielsen
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, NO-0424, Oslo, Norway
- Nuffield Division of Clinical Laboratory Sciences, University of Oxford, Oxford, OX3 9DU, UK
| |
Collapse
|
13
|
Gao C, Wu L, Wu W, Huang Y, Wang X, Sun Z, Xu M, Gao C. Deep learning in pulmonary nodule detection and segmentation: a systematic review. Eur Radiol 2024:10.1007/s00330-024-10907-0. [PMID: 38985185 DOI: 10.1007/s00330-024-10907-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 04/09/2024] [Accepted: 05/10/2024] [Indexed: 07/11/2024]
Abstract
OBJECTIVES The accurate detection and precise segmentation of lung nodules on computed tomography are key prerequisites for early diagnosis and appropriate treatment of lung cancer. This study was designed to compare detection and segmentation methods for pulmonary nodules using deep-learning techniques to fill methodological gaps and biases in the existing literature. METHODS This study utilized a systematic review with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, searching PubMed, Embase, Web of Science Core Collection, and the Cochrane Library databases up to May 10, 2023. The Quality Assessment of Diagnostic Accuracy Studies 2 criteria was used to assess the risk of bias and was adjusted with the Checklist for Artificial Intelligence in Medical Imaging. The study analyzed and extracted model performance, data sources, and task-focus information. RESULTS After screening, we included nine studies meeting our inclusion criteria. These studies were published between 2019 and 2023 and predominantly used public datasets, with the Lung Image Database Consortium Image Collection and Image Database Resource Initiative and Lung Nodule Analysis 2016 being the most common. The studies focused on detection, segmentation, and other tasks, primarily utilizing Convolutional Neural Networks for model development. Performance evaluation covered multiple metrics, including sensitivity and the Dice coefficient. CONCLUSIONS This study highlights the potential power of deep learning in lung nodule detection and segmentation. It underscores the importance of standardized data processing, code and data sharing, the value of external test datasets, and the need to balance model complexity and efficiency in future research. CLINICAL RELEVANCE STATEMENT Deep learning demonstrates significant promise in autonomously detecting and segmenting pulmonary nodules. Future research should address methodological shortcomings and variability to enhance its clinical utility. KEY POINTS Deep learning shows potential in the detection and segmentation of pulmonary nodules. There are methodological gaps and biases present in the existing literature. Factors such as external validation and transparency affect the clinical application.
Collapse
Affiliation(s)
- Chuan Gao
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China
| | - Linyu Wu
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China
| | - Wei Wu
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China
| | - Yichao Huang
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China
| | - Xinyue Wang
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China
| | - Zhichao Sun
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China.
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China.
| | - Maosheng Xu
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China.
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China.
| | - Chen Gao
- The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Chinese Medicine), Hangzhou, China.
- The First School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou, China.
| |
Collapse
|
14
|
Hilgers L, Ghaffari Laleh N, West NP, Westwood A, Hewitt KJ, Quirke P, Grabsch HI, Carrero ZI, Matthaei E, Loeffler CML, Brinker TJ, Yuan T, Brenner H, Brobeil A, Hoffmeister M, Kather JN. Automated curation of large-scale cancer histopathology image datasets using deep learning. Histopathology 2024; 84:1139-1153. [PMID: 38409878 DOI: 10.1111/his.15159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 12/29/2023] [Accepted: 02/09/2024] [Indexed: 02/28/2024]
Abstract
BACKGROUND Artificial intelligence (AI) has numerous applications in pathology, supporting diagnosis and prognostication in cancer. However, most AI models are trained on highly selected data, typically one tissue slide per patient. In reality, especially for large surgical resection specimens, dozens of slides can be available for each patient. Manually sorting and labelling whole-slide images (WSIs) is a very time-consuming process, hindering the direct application of AI on the collected tissue samples from large cohorts. In this study we addressed this issue by developing a deep-learning (DL)-based method for automatic curation of large pathology datasets with several slides per patient. METHODS We collected multiple large multicentric datasets of colorectal cancer histopathological slides from the United Kingdom (FOXTROT, N = 21,384 slides; CR07, N = 7985 slides) and Germany (DACHS, N = 3606 slides). These datasets contained multiple types of tissue slides, including bowel resection specimens, endoscopic biopsies, lymph node resections, immunohistochemistry-stained slides, and tissue microarrays. We developed, trained, and tested a deep convolutional neural network model to predict the type of slide from the slide overview (thumbnail) image. The primary statistical endpoint was the macro-averaged area under the receiver operating curve (AUROCs) for detection of the type of slide. RESULTS In the primary dataset (FOXTROT), with an AUROC of 0.995 [95% confidence interval [CI]: 0.994-0.996] the algorithm achieved a high classification performance and was able to accurately predict the type of slide from the thumbnail image alone. In the two external test cohorts (CR07, DACHS) AUROCs of 0.982 [95% CI: 0.979-0.985] and 0.875 [95% CI: 0.864-0.887] were observed, which indicates the generalizability of the trained model on unseen datasets. With a confidence threshold of 0.95, the model reached an accuracy of 94.6% (7331 classified cases) in CR07 and 85.1% (2752 classified cases) for the DACHS cohort. CONCLUSION Our findings show that using the low-resolution thumbnail image is sufficient to accurately classify the type of slide in digital pathology. This can support researchers to make the vast resource of existing pathology archives accessible to modern AI models with only minimal manual annotations.
Collapse
Affiliation(s)
- Lars Hilgers
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Narmin Ghaffari Laleh
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Nicholas P West
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Alice Westwood
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Katherine J Hewitt
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Philip Quirke
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Heike I Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Zunamys I Carrero
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Emylou Matthaei
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Chiara M L Loeffler
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Tanwei Yuan
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Brobeil
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany
- Tissue Bank, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|
15
|
Carini C, Seyhan AA. Tribulations and future opportunities for artificial intelligence in precision medicine. J Transl Med 2024; 22:411. [PMID: 38702711 PMCID: PMC11069149 DOI: 10.1186/s12967-024-05067-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 03/05/2024] [Indexed: 05/06/2024] Open
Abstract
Upon a diagnosis, the clinical team faces two main questions: what treatment, and at what dose? Clinical trials' results provide the basis for guidance and support for official protocols that clinicians use to base their decisions. However, individuals do not consistently demonstrate the reported response from relevant clinical trials. The decision complexity increases with combination treatments where drugs administered together can interact with each other, which is often the case. Additionally, the individual's response to the treatment varies with the changes in their condition. In practice, the drug and the dose selection depend significantly on the medical protocol and the medical team's experience. As such, the results are inherently varied and often suboptimal. Big data and Artificial Intelligence (AI) approaches have emerged as excellent decision-making tools, but multiple challenges limit their application. AI is a rapidly evolving and dynamic field with the potential to revolutionize various aspects of human life. AI has become increasingly crucial in drug discovery and development. AI enhances decision-making across different disciplines, such as medicinal chemistry, molecular and cell biology, pharmacology, pathology, and clinical practice. In addition to these, AI contributes to patient population selection and stratification. The need for AI in healthcare is evident as it aids in enhancing data accuracy and ensuring the quality care necessary for effective patient treatment. AI is pivotal in improving success rates in clinical practice. The increasing significance of AI in drug discovery, development, and clinical trials is underscored by many scientific publications. Despite the numerous advantages of AI, such as enhancing and advancing Precision Medicine (PM) and remote patient monitoring, unlocking its full potential in healthcare requires addressing fundamental concerns. These concerns include data quality, the lack of well-annotated large datasets, data privacy and safety issues, biases in AI algorithms, legal and ethical challenges, and obstacles related to cost and implementation. Nevertheless, integrating AI in clinical medicine will improve diagnostic accuracy and treatment outcomes, contribute to more efficient healthcare delivery, reduce costs, and facilitate better patient experiences, making healthcare more sustainable. This article reviews AI applications in drug development and clinical practice, making healthcare more sustainable, and highlights concerns and limitations in applying AI.
Collapse
Affiliation(s)
- Claudio Carini
- School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and Medicine, New Hunt's House, King's College London, Guy's Campus, London, UK.
- Biomarkers Consortium, Foundation of the National Institute of Health, Bethesda, MD, USA.
| | - Attila A Seyhan
- Laboratory of Translational Oncology and Experimental Cancer Therapeutics, Warren Alpert Medical School, Brown University, Providence, RI, USA.
- Department of Pathology and Laboratory Medicine, Warren Alpert Medical School, Brown University, Providence, RI, USA.
- Joint Program in Cancer Biology, Lifespan Health System and Brown University, Providence, RI, USA.
- Legorreta Cancer Center at Brown University, Providence, RI, USA.
| |
Collapse
|
16
|
Wang Z, Santa-Maria CA, Popel AS, Sulam J. Bi-level Graph Learning Unveils Prognosis-Relevant Tumor Microenvironment Patterns from Breast Multiplexed Digital Pathology. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.22.590118. [PMID: 38712207 PMCID: PMC11071347 DOI: 10.1101/2024.04.22.590118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
The tumor microenvironment is widely recognized for its central role in driving cancer progression and influencing prognostic outcomes. Despite extensive research efforts dedicated to characterizing this complex and heterogeneous environment, considerable challenges persist. In this study, we introduce a data-driven approach for identifying patterns of cell organizations in the tumor microenvironment that are associated with patient prognoses. Our methodology relies on the construction of a bi-level graph model: (i) a cellular graph, which models the intricate tumor microenvironment, and (ii) a population graph that captures inter-patient similarities, given their respective cellular graphs, by means of a soft Weisfeiler-Lehman subtree kernel. This systematic integration of information across different scales enables us to identify patient subgroups exhibiting unique prognoses while unveiling tumor microenvironment patterns that characterize them. We demonstrate our approach in a cohort of breast cancer patients, where the identified tumor microenvironment patterns result in a risk stratification system that provides complementary, new information with respect to alternative standards. Our results, which are validated in a completely independent cohort, allow for new insights into the prognostic implications of the breast tumor microenvironment, and this methodology could be applied to other cancer types more generally.
Collapse
Affiliation(s)
- Zhenzhen Wang
- Department of Biomedical Engineering, Johns Hopkins University
- Mathematical Institute for Data Science, Johns Hopkins University
| | - Cesar A Santa-Maria
- Department of Oncology, Johns Hopkins University
- Sidney Kimmel Comprehensive Cancer Center
| | | | - Jeremias Sulam
- Department of Biomedical Engineering, Johns Hopkins University
- Mathematical Institute for Data Science, Johns Hopkins University
| |
Collapse
|
17
|
Yuan L, Zhou H, Xiao X, Zhang X, Chen F, Liu L, Liu J, Bao S, Tao K. Development and external validation of a transfer learning-based system for the pathological diagnosis of colorectal cancer: a large emulated prospective study. Front Oncol 2024; 14:1365364. [PMID: 38725622 PMCID: PMC11079287 DOI: 10.3389/fonc.2024.1365364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 04/11/2024] [Indexed: 05/12/2024] Open
Abstract
Background The progress in Colorectal cancer (CRC) screening and management has resulted in an unprecedented caseload for histopathological diagnosis. While artificial intelligence (AI) presents a potential solution, the predominant emphasis on slide-level aggregation performance without thorough verification of cancer in each location, impedes both explainability and transparency. Effectively addressing these challenges is crucial to ensuring the reliability and efficacy of AI in histology applications. Method In this study, we created an innovative AI algorithm using transfer learning from a polyp segmentation model in endoscopy. The algorithm precisely localized CRC targets within 0.25 mm² grids from whole slide imaging (WSI). We assessed the CRC detection capabilities at this fine granularity and examined the influence of AI on the diagnostic behavior of pathologists. The evaluation utilized an extensive dataset comprising 858 consecutive patient cases with 1418 WSIs obtained from an external center. Results Our results underscore a notable sensitivity of 90.25% and specificity of 96.60% at the grid level, accompanied by a commendable area under the curve (AUC) of 0.962. This translates to an impressive 99.39% sensitivity at the slide level, coupled with a negative likelihood ratio of <0.01, signifying the dependability of the AI system to preclude diagnostic considerations. The positive likelihood ratio of 26.54, surpassing 10 at the grid level, underscores the imperative for meticulous scrutiny of any AI-generated highlights. Consequently, all four participating pathologists demonstrated statistically significant diagnostic improvements with AI assistance. Conclusion Our transfer learning approach has successfully yielded an algorithm that can be validated for CRC histological localizations in whole slide imaging. The outcome advocates for the integration of the AI system into histopathological diagnosis, serving either as a diagnostic exclusion application or a computer-aided detection (CADe) tool. This integration has the potential to alleviate the workload of pathologists and ultimately benefit patients.
Collapse
Affiliation(s)
- Liuhong Yuan
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Henghua Zhou
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | | | - Xiuqin Zhang
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Feier Chen
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Lin Liu
- Institute of Natural Sciences, MOE-LSC, School of Mathematical Sciences, CMA-Shanghai, SJTU-Yale Joint Center for Biostatistics and Data Science, Shanghai Jiao Tong University, Shanghai, China
| | | | - Shisan Bao
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| | - Kun Tao
- Department of Pathology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Tongren Hospital, School of Medicine Shanghai Jiaotong University, Shanghai, China
| |
Collapse
|
18
|
Wang Y, Zhang W, Chen L, Xie J, Zheng X, Jin Y, Zheng Q, Xue Q, Li B, He C, Chen H, Li Y. Development of an Interpretable Deep Learning Model for Pathological Tumor Response Assessment After Neoadjuvant Therapy. Biol Proced Online 2024; 26:10. [PMID: 38632527 PMCID: PMC11022344 DOI: 10.1186/s12575-024-00234-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/22/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Neoadjuvant therapy followed by surgery has become the standard of care for locally advanced esophageal squamous cell carcinoma (ESCC) and accurate pathological response assessment is critical to assess the therapeutic efficacy. However, it can be laborious and inconsistency between different observers may occur. Hence, we aim to develop an interpretable deep-learning model for efficient pathological response assessment following neoadjuvant therapy in ESCC. METHODS This retrospective study analyzed 337 ESCC resection specimens from 2020-2021 at the Pudong-Branch (Cohort 1) and 114 from 2021-2022 at the Puxi-Branch (External Cohort 2) of Fudan University Shanghai Cancer Center. Whole slide images (WSIs) from these two cohorts were generated using different scanning machines to test the ability of the model in handling color variations. Four pathologists independently assessed the pathological response. The senior pathologists annotated tumor beds and residual tumor percentages on WSIs to determine consensus labels. Furthermore, 1850 image patches were randomly extracted from Cohort 1 WSIs and binarily classified for tumor viability. A deep-learning model employing knowledge distillation was developed to automatically classify positive patches for each WSI and estimate the viable residual tumor percentages. Spatial heatmaps were output for model explanations and visualizations. RESULTS The approach achieved high concordance with pathologist consensus, with an R^2 of 0.8437, a RAcc_0.1 of 0.7586, a RAcc_0.3 of 0.9885, which were comparable to two senior pathologists (R^2 of 0.9202/0.9619, RAcc_0.1 of 8506/0.9425, RAcc_0.3 of 1.000/1.000) and surpassing two junior pathologists (R^2 of 0.5592/0.5474, RAcc_0.1 of 0.5287/0.5287, RAcc_0.3 of 0.9080/0.9310). Visualizations enabled the localization of residual viable tumor to augment microscopic assessment. CONCLUSION This work illustrates deep learning's potential for assisting pathological response assessment. Spatial heatmaps and patch examples provide intuitive explanations of model predictions, engendering clinical trust and adoption (Code and data will be available at https://github.com/WinnieLaugh/ESCC_Percentage once the paper has been conditionally accepted). Integrating interpretable computational pathology could help enhance the efficiency and consistency of tumor response assessment and empower precise oncology treatment decisions.
Collapse
Affiliation(s)
- Yichen Wang
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
| | - Wenhua Zhang
- Shanghai Aitrox Technology Corporation Limited, Shanghai, China
- Department of Future Technology, Shanghai University, Shanghai, China
| | - Lijun Chen
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
| | - Jun Xie
- Shanghai Aitrox Technology Corporation Limited, Shanghai, China
| | - Xuebin Zheng
- Shanghai Aitrox Technology Corporation Limited, Shanghai, China
| | - Yan Jin
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
| | - Qiang Zheng
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
| | - Qianqian Xue
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
| | - Bin Li
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
- Department of Thoracic Surgery, Fudan University Shanghai Cancer Center, Shanghai, China
| | - Chuan He
- Shanghai Aitrox Technology Corporation Limited, Shanghai, China
| | - Haiquan Chen
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032
- Department of Thoracic Surgery, Fudan University Shanghai Cancer Center, Shanghai, China
| | - Yuan Li
- Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China, 200032.
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China, 200032.
| |
Collapse
|
19
|
Kolbinger FR, Veldhuizen GP, Zhu J, Truhn D, Kather JN. Reporting guidelines in medical artificial intelligence: a systematic review and meta-analysis. COMMUNICATIONS MEDICINE 2024; 4:71. [PMID: 38605106 PMCID: PMC11009315 DOI: 10.1038/s43856-024-00492-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 03/27/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND The field of Artificial Intelligence (AI) holds transformative potential in medicine. However, the lack of universal reporting guidelines poses challenges in ensuring the validity and reproducibility of published research studies in this field. METHODS Based on a systematic review of academic publications and reporting standards demanded by both international consortia and regulatory stakeholders as well as leading journals in the fields of medicine and medical informatics, 26 reporting guidelines published between 2009 and 2023 were included in this analysis. Guidelines were stratified by breadth (general or specific to medical fields), underlying consensus quality, and target research phase (preclinical, translational, clinical) and subsequently analyzed regarding the overlap and variations in guideline items. RESULTS AI reporting guidelines for medical research vary with respect to the quality of the underlying consensus process, breadth, and target research phase. Some guideline items such as reporting of study design and model performance recur across guidelines, whereas other items are specific to particular fields and research stages. CONCLUSIONS Our analysis highlights the importance of reporting guidelines in clinical AI research and underscores the need for common standards that address the identified variations and gaps in current guidelines. Overall, this comprehensive overview could help researchers and public stakeholders reinforce quality standards for increased reliability, reproducibility, clinical validity, and public trust in AI research in healthcare. This could facilitate the safe, effective, and ethical translation of AI methods into clinical applications that will ultimately improve patient outcomes.
Collapse
Grants
- UM1 TR004402 NCATS NIH HHS
- JNK is supported by the German Federal Ministry of Health (DEEP LIVER, ZMVI1-2520DAT111) and the Max-Eder-Programme of the German Cancer Aid (grant #70113864), the German Federal Ministry of Education and Research (PEARL, 01KD2104C; CAMINO, 01EO2101; SWAG, 01KD2215A; TRANSFORM LIVER, 031L0312A), the German Academic Exchange Service (SECAI, 57616814), the German Federal Joint Committee (Transplant.KI, 01VSF21048) the European Union (ODELIA, 101057091; GENIAL, 101096312) and the National Institute for Health and Care Research (NIHR, NIHR213331) Leeds Biomedical Research Centre.
Collapse
Affiliation(s)
- Fiona R Kolbinger
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital and Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
- Regenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, IN, USA
- Department of Biostatistics and Health Data Science, Richard M. Fairbanks School of Public Health, Indiana University, Indianapolis, IN, USA
- Indiana University Simon Comprehensive Cancer Center, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Gregory P Veldhuizen
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Jiefu Zhu
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany.
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
20
|
Rigamonti A, Viatore M, Polidori R, Rahal D, Erreni M, Fumagalli MR, Zanini D, Doni A, Putignano AR, Bossi P, Voulaz E, Alloisio M, Rossi S, Zucali PA, Santoro A, Balzano V, Nisticò P, Feuerhake F, Mantovani A, Locati M, Marchesi F. Integrating AI-Powered Digital Pathology and Imaging Mass Cytometry Identifies Key Classifiers of Tumor Cells, Stroma, and Immune Cells in Non-Small Cell Lung Cancer. Cancer Res 2024; 84:1165-1177. [PMID: 38315789 PMCID: PMC10982643 DOI: 10.1158/0008-5472.can-23-1698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 11/13/2023] [Accepted: 02/01/2024] [Indexed: 02/07/2024]
Abstract
Artificial intelligence (AI)-powered approaches are becoming increasingly used as histopathologic tools to extract subvisual features and improve diagnostic workflows. On the other hand, hi-plex approaches are widely adopted to analyze the immune ecosystem in tumor specimens. Here, we aimed at combining AI-aided histopathology and imaging mass cytometry (IMC) to analyze the ecosystem of non-small cell lung cancer (NSCLC). An AI-based approach was used on hematoxylin and eosin (H&E) sections from 158 NSCLC specimens to accurately identify tumor cells, both adenocarcinoma and squamous carcinoma cells, and to generate a classifier of tumor cell spatial clustering. Consecutive tissue sections were stained with metal-labeled antibodies and processed through the IMC workflow, allowing quantitative detection of 24 markers related to tumor cells, tissue architecture, CD45+ myeloid and lymphoid cells, and immune activation. IMC identified 11 macrophage clusters that mainly localized in the stroma, except for S100A8+ cells, which infiltrated tumor nests. T cells were preferentially localized in peritumor areas or in tumor nests, the latter being associated with better prognosis, and they were more abundant in highly clustered tumors. Integrated tumor and immune classifiers were validated as prognostic on whole slides. In conclusion, integration of AI-powered H&E and multiparametric IMC allows investigation of spatial patterns and reveals tissue relevant features with clinical relevance. SIGNIFICANCE Leveraging artificial intelligence-powered H&E analysis integrated with hi-plex imaging mass cytometry provides insights into the tumor ecosystem and can translate tumor features into classifiers to predict prognosis, genotype, and therapy response.
Collapse
Affiliation(s)
- Alessandra Rigamonti
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Medical Biotechnology and Translational Medicine, University of Milan; Milan, Italy
| | - Marika Viatore
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Medical Biotechnology and Translational Medicine, University of Milan; Milan, Italy
| | - Rebecca Polidori
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Medical Biotechnology and Translational Medicine, University of Milan; Milan, Italy
| | - Daoud Rahal
- Department of Pathology, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
| | - Marco Erreni
- Unit of Advanced Optical Microscopy, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy
- Department of Biomedical Science, Humanitas University, Pieve Emanuele, Milan, Italy
| | - Maria Rita Fumagalli
- Unit of Advanced Optical Microscopy, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy
| | - Damiano Zanini
- Unit of Advanced Optical Microscopy, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy
| | - Andrea Doni
- Unit of Advanced Optical Microscopy, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy
| | - Anna Rita Putignano
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
| | - Paola Bossi
- Department of Pathology, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
| | - Emanuele Voulaz
- Department of Biomedical Science, Humanitas University, Pieve Emanuele, Milan, Italy
- Division of Thoracic Surgery, IRCCS Humanitas Research Hospital, Rozzano (Milan), Italy
| | - Marco Alloisio
- Division of Thoracic Surgery, IRCCS Humanitas Research Hospital, Rozzano (Milan), Italy
| | - Sabrina Rossi
- Medical Oncology and Hematology Unit, IRCCS Humanitas Research Hospital, Rozzano (Milan), Italy
| | - Paolo Andrea Zucali
- Department of Biomedical Science, Humanitas University, Pieve Emanuele, Milan, Italy
- Medical Oncology and Hematology Unit, IRCCS Humanitas Research Hospital, Rozzano (Milan), Italy
| | - Armando Santoro
- Department of Biomedical Science, Humanitas University, Pieve Emanuele, Milan, Italy
- Medical Oncology and Hematology Unit, IRCCS Humanitas Research Hospital, Rozzano (Milan), Italy
| | - Vittoria Balzano
- Immunology and Immunotherapy Unit, IRCCS Regina Elena National Cancer Institute, Rome, Italy
| | - Paola Nisticò
- Immunology and Immunotherapy Unit, IRCCS Regina Elena National Cancer Institute, Rome, Italy
| | | | - Alberto Mantovani
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Biomedical Science, Humanitas University, Pieve Emanuele, Milan, Italy
- The William Harvey Research Institute, Queen Mary University of London, London, United Kingdom
| | - Massimo Locati
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Medical Biotechnology and Translational Medicine, University of Milan; Milan, Italy
| | - Federica Marchesi
- Department of Immunology and Inflammation, IRCCS Humanitas Research Hospital; Rozzano (Milan), Italy
- Department of Medical Biotechnology and Translational Medicine, University of Milan; Milan, Italy
| |
Collapse
|
21
|
Zheng H, Sherazi SWA, Lee JY. A cost-sensitive deep neural network-based prediction model for the mortality in acute myocardial infarction patients with hypertension on imbalanced data. Front Cardiovasc Med 2024; 11:1276608. [PMID: 38566962 PMCID: PMC10986180 DOI: 10.3389/fcvm.2024.1276608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 03/04/2024] [Indexed: 04/04/2024] Open
Abstract
Background and objectives Hypertension is one of the most serious risk factors and the leading cause of mortality in patients with cardiovascular diseases (CVDs). It is necessary to accurately predict the mortality of patients suffering from CVDs with hypertension. Therefore, this paper proposes a novel cost-sensitive deep neural network (CSDNN)-based mortality prediction model for out-of-hospital acute myocardial infarction (AMI) patients with hypertension on imbalanced data. Methods The synopsis of our research is as follows. First, the experimental data is extracted from the Korea Acute Myocardial Infarction Registry-National Institutes of Health (KAMIR-NIH) and preprocessed with several approaches. Then the imbalanced experimental dataset is divided into training data (80%) and test data (20%). After that, we design the proposed CSDNN-based mortality prediction model, which can solve the skewed class distribution between the majority and minority classes in the training data. The threshold moving technique is also employed to enhance the performance of the proposed model. Finally, we evaluate the performance of the proposed model using the test data and compare it with other commonly used machine learning (ML) and data sampling-based ensemble models. Moreover, the hyperparameters of all models are optimized through random search strategies with a 5-fold cross-validation approach. Results and discussion In the result, the proposed CSDNN model with the threshold moving technique yielded the best results on imbalanced data. Additionally, our proposed model outperformed the best ML model and the classic data sampling-based ensemble model with an AUC of 2.58% and 2.55% improvement, respectively. It aids in decision-making and offers a precise mortality prediction for AMI patients with hypertension.
Collapse
Affiliation(s)
- Huilin Zheng
- Department of Computer Science, Chungbuk National University, Cheongju, Republic of Korea
- College of Computer Science and Engineering, Guilin University of Technology, Guilin, China
| | | | - Jong Yun Lee
- Department of Computer Science, Chungbuk National University, Cheongju, Republic of Korea
| |
Collapse
|
22
|
Arslan S, Schmidt J, Bass C, Mehrotra D, Geraldes A, Singhal S, Hense J, Li X, Raharja-Liu P, Maiques O, Kather JN, Pandya P. A systematic pan-cancer study on deep learning-based prediction of multi-omic biomarkers from routine pathology images. COMMUNICATIONS MEDICINE 2024; 4:48. [PMID: 38491101 PMCID: PMC10942985 DOI: 10.1038/s43856-024-00471-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 02/29/2024] [Indexed: 03/18/2024] Open
Abstract
BACKGROUND The objective of this comprehensive pan-cancer study is to evaluate the potential of deep learning (DL) for molecular profiling of multi-omic biomarkers directly from hematoxylin and eosin (H&E)-stained whole slide images. METHODS A total of 12,093 DL models predicting 4031 multi-omic biomarkers across 32 cancer types were trained and validated. The study included a broad range of genetic, transcriptomic, and proteomic biomarkers, as well as established prognostic markers, molecular subtypes, and clinical outcomes. RESULTS Here we show that 50% of the models achieve an area under the curve (AUC) of 0.644 or higher. The observed AUC for 25% of the models is at least 0.719 and exceeds 0.834 for the top 5%. Molecular profiling with image-based histomorphological features is generally considered feasible for most of the investigated biomarkers and across different cancer types. The performance appears to be independent of tumor purity, sample size, and class ratio (prevalence), suggesting a degree of inherent predictability in histomorphology. CONCLUSIONS The results demonstrate that DL holds promise to predict a wide range of biomarkers across the omics spectrum using only H&E-stained histological slides of solid tumors. This paves the way for accelerating diagnosis and developing more precise treatments for cancer patients.
Collapse
Affiliation(s)
| | | | | | - Debapriya Mehrotra
- Panakeia Technologies, London, UK
- Department of Pathology, Barking, Havering and Redbridge University NHS Trust, Romford, UK
| | | | - Shikha Singhal
- Panakeia Technologies, London, UK
- Department of Pathology, The Royal Wolverhampton NHS Trust, Wolverhampton, UK
| | | | - Xiusi Li
- Panakeia Technologies, London, UK
| | | | - Oscar Maiques
- Cytoskeleton and Cancer Metastasis Group, Breast Cancer Now Toby Robins Breast Cancer Research Centre, The Institute of Cancer Research, London, UK
- Cancer Biomarkers & Biotherapeutics, Barts Cancer Institute, Queen Mary University of London, John Vane Science Building, London, UK
| | - Jakob Nikolas Kather
- Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
| | | |
Collapse
|
23
|
Kovacs DG, Ladefoged CN, Andersen KF, Brittain JM, Christensen CB, Dejanovic D, Hansen NL, Loft A, Petersen JH, Reichkendler M, Andersen FL, Fischer BM. Clinical Evaluation of Deep Learning for Tumor Delineation on 18F-FDG PET/CT of Head and Neck Cancer. J Nucl Med 2024; 65:jnumed.123.266574. [PMID: 38388516 PMCID: PMC10995525 DOI: 10.2967/jnumed.123.266574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 01/29/2024] [Accepted: 01/29/2024] [Indexed: 02/24/2024] Open
Abstract
Artificial intelligence (AI) may decrease 18F-FDG PET/CT-based gross tumor volume (GTV) delineation variability and automate tumor-volume-derived image biomarker extraction. Hence, we aimed to identify and evaluate promising state-of-the-art deep learning methods for head and neck cancer (HNC) PET GTV delineation. Methods: We trained and evaluated deep learning methods using retrospectively included scans of HNC patients referred for radiotherapy between January 2014 and December 2019 (ISRCTN16907234). We used 3 test datasets: an internal set to compare methods, another internal set to compare AI-to-expert variability and expert interobserver variability (IOV), and an external set to compare internal and external AI-to-expert variability. Expert PET GTVs were used as the reference standard. Our benchmark IOV was measured using the PET GTV of 6 experts. The primary outcome was the Dice similarity coefficient (DSC). ANOVA was used to compare methods, a paired t test was used to compare AI-to-expert variability and expert IOV, an unpaired t test was used to compare internal and external AI-to-expert variability, and post hoc Bland-Altman analysis was used to evaluate biomarker agreement. Results: In total, 1,220 18F-FDG PET/CT scans of 1,190 patients (mean age ± SD, 63 ± 10 y; 858 men) were included, and 5 deep learning methods were trained using 5-fold cross-validation (n = 805). The nnU-Net method achieved the highest similarity (DSC, 0.80 [95% CI, 0.77-0.86]; n = 196). We found no evidence of a difference between expert IOV and AI-to-expert variability (DSC, 0.78 for AI vs. 0.82 for experts; mean difference of 0.04 [95% CI, -0.01 to 0.09]; P = 0.12; n = 64). We found no evidence of a difference between the internal and external AI-to-expert variability (DSC, 0.80 internally vs. 0.81 externally; mean difference of 0.004 [95% CI, -0.05 to 0.04]; P = 0.87; n = 125). PET GTV-derived biomarkers of AI were in good agreement with experts. Conclusion: Deep learning can be used to automate 18F-FDG PET/CT tumor-volume-derived imaging biomarkers, and the deep-learning-based volumes have the potential to assist clinical tumor volume delineation in radiation oncology.
Collapse
Affiliation(s)
- David G Kovacs
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark;
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Claes N Ladefoged
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
- Department of Applied Mathematics and Computer Science, Technical University of Denmark, Lyngby, Denmark
| | - Kim F Andersen
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Jane M Brittain
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Charlotte B Christensen
- Department of Clinical Physiology and Nuclear Medicine, Herlev Hospital, University of Copenhagen, Copenhagen, Denmark
| | - Danijela Dejanovic
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Naja L Hansen
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Annika Loft
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Jørgen H Petersen
- Section of Biostatistics, Institute of Public Health, Faculty of Health Sciences, University of Copenhagen, Denmark; and
| | - Michala Reichkendler
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Flemming L Andersen
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Barbara M Fischer
- Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
- PET Centre, School of Biomedical Engineering and Imaging Science, King's College London, London, United Kingdom
| |
Collapse
|
24
|
Liu X, Liu J. Aided Diagnosis Model Based on Deep Learning for Glioblastoma, Solitary Brain Metastases, and Primary Central Nervous System Lymphoma with Multi-Modal MRI. BIOLOGY 2024; 13:99. [PMID: 38392317 PMCID: PMC10887006 DOI: 10.3390/biology13020099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 01/26/2024] [Accepted: 01/27/2024] [Indexed: 02/24/2024]
Abstract
(1) Background: Diagnosis of glioblastoma (GBM), solitary brain metastases (SBM), and primary central nervous system lymphoma (PCNSL) plays a decisive role in the development of personalized treatment plans. Constructing a deep learning classification network to diagnose GBM, SBM, and PCNSL with multi-modal MRI is important and necessary. (2) Subjects: GBM, SBM, and PCNSL were confirmed by histopathology with the multi-modal MRI examination (study from 1225 subjects, average age 53 years, 671 males), 3.0 T T2 fluid-attenuated inversion recovery (T2-Flair), and Contrast-enhanced T1-weighted imaging (CE-T1WI). (3) Methods: This paper introduces MFFC-Net, a classification model based on the fusion of multi-modal MRIs, for the classification of GBM, SBM, and PCNSL. The network architecture consists of parallel encoders using DenseBlocks to extract features from different modalities of MRI images. Subsequently, an L1-norm feature fusion module is applied to enhance the interrelationships among tumor tissues. Then, a spatial-channel self-attention weighting operation is performed after the feature fusion. Finally, the classification results are obtained using the full convolutional layer (FC) and Soft-max. (4) Results: The ACC of MFFC-Net based on feature fusion was 0.920, better than the radiomics model (ACC of 0.829). There was no significant difference in the ACC compared to the expert radiologist (0.920 vs. 0.924, p = 0.774). (5) Conclusions: Our MFFC-Net model could distinguish GBM, SBM, and PCNSL preoperatively based on multi-modal MRI, with a higher performance than the radiomics model and was comparable to radiologists.
Collapse
Affiliation(s)
- Xiao Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
| | - Jie Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
| |
Collapse
|
25
|
Konugolu Venkata Sekar S, Ma H, Komolibus K, Dumlupinar G, Mickert MJ, Krawczyk K, Andersson-Engels S. High contrast breast cancer biomarker semi-quantification and immunohistochemistry imaging using upconverting nanoparticles. BIOMEDICAL OPTICS EXPRESS 2024; 15:900-909. [PMID: 38404324 PMCID: PMC10890842 DOI: 10.1364/boe.504939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 12/21/2023] [Accepted: 01/09/2024] [Indexed: 02/27/2024]
Abstract
Breast cancer is the second leading cause of cancer death in women. Current clinical treatment stratification practices open up an avenue for significant improvements, potentially through advancements in immunohistochemistry (IHC) assessments of biopsies. We report a high contrast upconverting nanoparticles (UCNP) labeling to distinguish different levels of human epidermal growth factor receptor 2 (HER2) in HER2 control pellet arrays (CPAs) and HER2-positive breast cancer tissue. A simple Fourier transform algorithm trained on CPAs was sufficient to provide a semi-quantitative HER2 assessment tool for breast cancer tissues. The UCNP labeling had a signal-to-background ratio of 40 compared to the negative control.
Collapse
Affiliation(s)
| | - Hui Ma
- Biophotonics@Tyndall, IPIC, Tyndall National Institute, Lee Maltings Complex, Dyke Parade, T12R5CP, Cork,
Ireland
- Department of Physics,
University College Cork, College Road,
Cork, T12 K8AF, Ireland
| | - Katarzyna Komolibus
- Biophotonics@Tyndall, IPIC, Tyndall National Institute, Lee Maltings Complex, Dyke Parade, T12R5CP, Cork,
Ireland
| | - Gokhan Dumlupinar
- Biophotonics@Tyndall, IPIC, Tyndall National Institute, Lee Maltings Complex, Dyke Parade, T12R5CP, Cork,
Ireland
- Department of Physics,
University College Cork, College Road,
Cork, T12 K8AF, Ireland
| | | | | | - Stefan Andersson-Engels
- Biophotonics@Tyndall, IPIC, Tyndall National Institute, Lee Maltings Complex, Dyke Parade, T12R5CP, Cork,
Ireland
- Department of Physics,
University College Cork, College Road,
Cork, T12 K8AF, Ireland
| |
Collapse
|
26
|
Truhn D, Tayebi Arasteh S, Saldanha OL, Müller-Franzes G, Khader F, Quirke P, West NP, Gray R, Hutchins GGA, James JA, Loughrey MB, Salto-Tellez M, Brenner H, Brobeil A, Yuan T, Chang-Claude J, Hoffmeister M, Foersch S, Han T, Keil S, Schulze-Hagen M, Isfort P, Bruners P, Kaissis G, Kuhl C, Nebelung S, Kather JN. Encrypted federated learning for secure decentralized collaboration in cancer image analysis. Med Image Anal 2024; 92:103059. [PMID: 38104402 PMCID: PMC10804934 DOI: 10.1016/j.media.2023.103059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 04/28/2023] [Accepted: 12/05/2023] [Indexed: 12/19/2023]
Abstract
Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers.
Collapse
Affiliation(s)
- Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany.
| | - Soroosh Tayebi Arasteh
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Oliver Lester Saldanha
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Gustav Müller-Franzes
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Firas Khader
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Philip Quirke
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Nicholas P West
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Richard Gray
- Clinical Trial Service Unit, University of Oxford, Oxford, United Kingdom
| | - Gordon G A Hutchins
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Jacqueline A James
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, United Kingdom; Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast, United Kingdom; The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom
| | - Maurice B Loughrey
- The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom; Department of Cellular Pathology, Belfast Health and Social Care Trust, Belfast, United Kingdom; Centre for Public Health, Queen's University Belfast, Belfast, United Kingdom
| | - Manuel Salto-Tellez
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, United Kingdom; Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast, United Kingdom; The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany; German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Brobeil
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany; Tissue Bank, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Tanwei Yuan
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Jenny Chang-Claude
- Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; Division of Cancer Epidemiology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sebastian Foersch
- Institute of Pathology, University Medical Center Mainz, Mainz, Germany
| | - Tianyu Han
- Physics of Molecular Imaging Systems, Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany
| | - Sebastian Keil
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Maximilian Schulze-Hagen
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Peter Isfort
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Philipp Bruners
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Georgios Kaissis
- Institute of Diagnostic and Interventional Radiology, Technical University of Munich, Munich, Germany; Artificial Intelligence in Medicine and Healthcare, Technical University of Munich, Munich, Germany; Department of Computing, Imperial College London, London, United Kingdom
| | - Christiane Kuhl
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Sven Nebelung
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany; Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom; Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|
27
|
Ma W, Wu H, Chen Y, Xu H, Jiang J, Du B, Wan M, Ma X, Chen X, Lin L, Su X, Bao X, Shen Y, Xu N, Ruan J, Jiang H, Ding Y. New techniques to identify the tissue of origin for cancer of unknown primary in the era of precision medicine: progress and challenges. Brief Bioinform 2024; 25:bbae028. [PMID: 38343328 PMCID: PMC10859692 DOI: 10.1093/bib/bbae028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Revised: 12/10/2023] [Accepted: 01/11/2024] [Indexed: 02/15/2024] Open
Abstract
Despite a standardized diagnostic examination, cancer of unknown primary (CUP) is a rare metastatic malignancy with an unidentified tissue of origin (TOO). Patients diagnosed with CUP are typically treated with empiric chemotherapy, although their prognosis is worse than those with metastatic cancer of a known origin. TOO identification of CUP has been employed in precision medicine, and subsequent site-specific therapy is clinically helpful. For example, molecular profiling, including genomic profiling, gene expression profiling, epigenetics and proteins, has facilitated TOO identification. Moreover, machine learning has improved identification accuracy, and non-invasive methods, such as liquid biopsy and image omics, are gaining momentum. However, the heterogeneity in prediction accuracy, sample requirements and technical fundamentals among the various techniques is noteworthy. Accordingly, we systematically reviewed the development and limitations of novel TOO identification methods, compared their pros and cons and assessed their potential clinical usefulness. Our study may help patients shift from empirical to customized care and improve their prognoses.
Collapse
Affiliation(s)
- Wenyuan Ma
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Hui Wu
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yiran Chen
- Department of Surgical Oncology, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, Zhejiang, China
| | - Hongxia Xu
- Zhejiang University-University of Edinburgh Institute (ZJU-UoE Institute), Zhejiang University School of Medicine, Zhejiang University, Haining, China
| | - Junjie Jiang
- Department of Gastroenterology, Affiliated Hangzhou First People's Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Bang Du
- Real Doctor AI Research Centre, School of Medicine, Zhejiang University, Hangzhou 310058, China
| | - Mingyu Wan
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xiaolu Ma
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xiaoyu Chen
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Lili Lin
- Department of Nuclear Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xinhui Su
- Department of Nuclear Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xuanwen Bao
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yifei Shen
- Department of Laboratory Medicine, the First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Nong Xu
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Jian Ruan
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Haiping Jiang
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yongfeng Ding
- Department of Medical Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| |
Collapse
|
28
|
Yang L, Yang J, Kleppe A, Danielsen HE, Kerr DJ. Personalizing adjuvant therapy for patients with colorectal cancer. Nat Rev Clin Oncol 2024; 21:67-79. [PMID: 38001356 DOI: 10.1038/s41571-023-00834-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/26/2023] [Indexed: 11/26/2023]
Abstract
The current standard-of-care adjuvant treatment for patients with colorectal cancer (CRC) comprises a fluoropyrimidine (5-fluorouracil or capecitabine) as a single agent or in combination with oxaliplatin, for either 3 or 6 months. Selection of therapy depends on conventional histopathological staging procedures, which constitute a blunt tool for patient stratification. Given the relatively marginal survival benefits that patients can derive from adjuvant treatment, improving the safety of chemotherapy regimens and identifying patients most likely to benefit from them is an area of unmet need. Patient stratification should enable distinguishing those at low risk of recurrence and a high chance of cure by surgery from those at higher risk of recurrence who would derive greater absolute benefits from chemotherapy. To this end, genetic analyses have led to the discovery of germline determinants of toxicity from fluoropyrimidines, the identification of patients at high risk of life-threatening toxicity, and enabling dose modulation to improve safety. Thus far, results from analyses of resected tissue to identify mutational or transcriptomic signatures with value as prognostic biomarkers have been rather disappointing. In the past few years, the application of artificial intelligence-driven models to digital images of resected tissue has identified potentially useful algorithms that stratify patients into distinct prognostic groups. Similarly, liquid biopsy approaches involving measurements of circulating tumour DNA after surgery are additionally useful tools to identify patients at high and low risk of tumour recurrence. In this Perspective, we provide an overview of the current landscape of adjuvant therapy for patients with CRC and discuss how new technologies will enable better personalization of therapy in this setting.
Collapse
Affiliation(s)
- Li Yang
- Department of Gastroenterology, Sichuan University, Chengdu, China
| | - Jinlin Yang
- Department of Gastroenterology, Sichuan University, Chengdu, China
| | - Andreas Kleppe
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, Oslo, Norway
- Department of Informatics, University of Oslo, Oslo, Norway
- Centre for Research-based Innovation Visual Intelligence, UiT The Arctic University of Norway, Tromsø, Norway
| | - Håvard E Danielsen
- Institute for Cancer Genetics and Informatics, Oslo University Hospital, Oslo, Norway
- Radcliffe Department of Medicine, Oxford University, Oxford, UK
| | - David J Kerr
- Radcliffe Department of Medicine, Oxford University, Oxford, UK.
| |
Collapse
|
29
|
Chen Z, Yu Y, Liu S, Du W, Hu L, Wang C, Li J, Liu J, Zhang W, Peng X. A deep learning and radiomics fusion model based on contrast-enhanced computer tomography improves preoperative identification of cervical lymph node metastasis of oral squamous cell carcinoma. Clin Oral Investig 2023; 28:39. [PMID: 38151672 DOI: 10.1007/s00784-023-05423-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Accepted: 11/21/2023] [Indexed: 12/29/2023]
Abstract
OBJECTIVES In this study, we constructed and validated models based on deep learning and radiomics to facilitate preoperative diagnosis of cervical lymph node metastasis (LNM) using contrast-enhanced computed tomography (CECT). MATERIALS AND METHODS CECT scans of 100 patients with OSCC (217 metastatic and 1973 non-metastatic cervical lymph nodes: development set, 76 patients; internally independent test set, 24 patients) who received treatment at the Peking University School and Hospital of Stomatology between 2012 and 2016 were retrospectively collected. Clinical diagnoses and pathological findings were used to establish the gold standard for metastatic cervical LNs. A reader study with two clinicians was also performed to evaluate the lymph node status in the test set. The performance of the proposed models and the clinicians was evaluated and compared by measuring using the area under the receiver operating characteristic curve (AUC), accuracy (ACC), sensitivity (SEN), and specificity (SPE). RESULTS A fusion model combining deep learning with radiomics showed the best performance (ACC, 89.2%; SEN, 92.0%; SPE, 88.9%; and AUC, 0.950 [95% confidence interval: 0.908-0.993, P < 0.001]) in the test set. In comparison with the clinicians, the fusion model showed higher sensitivity (92.0 vs. 72.0% and 60.0%) but lower specificity (88.9 vs. 97.5% and 98.8%). CONCLUSION A fusion model combining radiomics and deep learning approaches outperformed other single-technique models and showed great potential to accurately predict cervical LNM in patients with OSCC. CLINICAL RELEVANCE The fusion model can complement the preoperative identification of LNM of OSCC performed by the clinicians.
Collapse
Affiliation(s)
- Zhen Chen
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Yao Yu
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Shuo Liu
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Wen Du
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Leihao Hu
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Congwei Wang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Jiaqi Li
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Jianbo Liu
- Huafang Hanying Medical Technology Co., Ltd, No.19, West Bridge Road, Miyun District, Beijing, 101520, People's Republic of China
| | - Wenbo Zhang
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China
| | - Xin Peng
- Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Research Center of Oral Biomaterials and Digital Medical Devices & Beijing Key Laboratory of Digital Stomatology & Research Center of Engineering and Technology for Computerized Dentistry Ministry of Health & NMPA Key Laboratory for Dental Materials, No. 22, Zhongguancun South Avenue, Haidian District, Beijing, 100081, People's Republic of China.
| |
Collapse
|
30
|
Chen L, Fu G, Jiang C. Deep learning-derived 12-lead electrocardiogram-based genotype prediction for hypertrophic cardiomyopathy: a pilot study. Ann Med 2023; 55:2235564. [PMID: 37467172 PMCID: PMC10360981 DOI: 10.1080/07853890.2023.2235564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 06/25/2023] [Accepted: 07/07/2023] [Indexed: 07/21/2023] Open
Abstract
Objective: Given the psychosocial and ethical burden, patients with hypertrophic cardiomyopathy (HCMs) could benefit from the establishment of genetic probability prior to the test. This study aimed to develop a simple tool to provide genotype prediction for HCMs.Methods: A convolutional neural network (CNN) was built with the 12-lead electrocardiogram (ECG) of 124 HCMs who underwent genetic testing (GT), externally tested by predicting the genotype on another HCMs cohort (n = 54), and compared with the conventional methods (the Mayo and Toronto score). Using a third cohort of HCMs (n = 76), the role of the network in risk stratification was explored by calculating the sudden cardiac death (SCD) risk scorers (HCM risk-SCD) across the predicted genotypes. Score-CAM was employed to provide a visual explanation of the network.Results: Overall, 80 of 178 HCMs (45%) were genotype-positive. Using the 12-lead ECG as input, the network showed an area under the curve (AUC) of 0.89 (95% CI, 0.83-0.96) on the test set, outperforming the Mayo score (0.69 [95% CI, 0.65-0.78], p < 0.001) and the Toronto score (0.69 [95% CI, 0.64-0.75], p < 0.001). The network classified the third cohort into two groups (predicted genotype-negative vs. predicted genotype-positive). Compared with the former, patients predicted genotype-positive had a significantly higher HCM risk-SCD (0.04 ± 0.03 vs. 0.03 ± 0.02, p <0.01). Visualization indicated that the prediction was heavily influenced by the limb lead.Conclusions: The network demonstrated a promising ability in genotype prediction and risk assessment in HCM.
Collapse
Affiliation(s)
- LaiTe Chen
- The Affiliated Eye Hospital of Wenzhou Medical University, Wenzhou, P.R. China
| | - GuoSheng Fu
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Zhejiang, P.R. China
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Zhejiang, P.R. China
| | - ChenYang Jiang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Zhejiang, P.R. China
| |
Collapse
|
31
|
Tavolara TE, Su Z, Gurcan MN, Niazi MKK. One label is all you need: Interpretable AI-enhanced histopathology for oncology. Semin Cancer Biol 2023; 97:70-85. [PMID: 37832751 DOI: 10.1016/j.semcancer.2023.09.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 09/06/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023]
Abstract
Artificial Intelligence (AI)-enhanced histopathology presents unprecedented opportunities to benefit oncology through interpretable methods that require only one overall label per hematoxylin and eosin (H&E) slide with no tissue-level annotations. We present a structured review of these methods organized by their degree of verifiability and by commonly recurring application areas in oncological characterization. First, we discuss morphological markers (tumor presence/absence, metastases, subtypes, grades) in which AI-identified regions of interest (ROIs) within whole slide images (WSIs) verifiably overlap with pathologist-identified ROIs. Second, we discuss molecular markers (gene expression, molecular subtyping) that are not verified via H&E but rather based on overlap with positive regions on adjacent tissue. Third, we discuss genetic markers (mutations, mutational burden, microsatellite instability, chromosomal instability) that current technologies cannot verify if AI methods spatially resolve specific genetic alterations. Fourth, we discuss the direct prediction of survival to which AI-identified histopathological features quantitatively correlate but are nonetheless not mechanistically verifiable. Finally, we discuss in detail several opportunities and challenges for these one-label-per-slide methods within oncology. Opportunities include reducing the cost of research and clinical care, reducing the workload of clinicians, personalized medicine, and unlocking the full potential of histopathology through new imaging-based biomarkers. Current challenges include explainability and interpretability, validation via adjacent tissue sections, reproducibility, data availability, computational needs, data requirements, domain adaptability, external validation, dataset imbalances, and finally commercialization and clinical potential. Ultimately, the relative ease and minimum upfront cost with which relevant data can be collected in addition to the plethora of available AI methods for outcome-driven analysis will surmount these current limitations and achieve the innumerable opportunities associated with AI-driven histopathology for the benefit of oncology.
Collapse
Affiliation(s)
- Thomas E Tavolara
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Ziyu Su
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Metin N Gurcan
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - M Khalid Khan Niazi
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA.
| |
Collapse
|
32
|
Castelo-Branco L, Pellat A, Martins-Branco D, Valachis A, Derksen JWG, Suijkerbuijk KPM, Dafni U, Dellaporta T, Vogel A, Prelaj A, Groenwold RHH, Martins H, Stahel R, Bliss J, Kather J, Ribelles N, Perrone F, Hall PS, Dienstmann R, Booth CM, Pentheroudakis G, Delaloge S, Koopman M. ESMO Guidance for Reporting Oncology real-World evidence (GROW). Ann Oncol 2023; 34:1097-1112. [PMID: 37848160 DOI: 10.1016/j.annonc.2023.10.001] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 09/28/2023] [Accepted: 10/04/2023] [Indexed: 10/19/2023] Open
Affiliation(s)
- L Castelo-Branco
- Scientific and Medical Division, European Society for Medical Oncology (ESMO), Lugano, Switzerland.
| | - A Pellat
- Department of Gastroenterology and Digestive Oncology, Hôpital Cochin AP-HP, Université Paris Cité, Paris; Centre d'Épidémiologie Clinique, Hôtel Dieu, Paris, France
| | - D Martins-Branco
- Scientific and Medical Division, European Society for Medical Oncology (ESMO), Lugano, Switzerland; Université Libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (HUB), Institut Jules Bordet, Academic Trials Promoting Team (ATPT), Brussels, Belgium
| | - A Valachis
- Department of Oncology, Faculty of Medicine and Health, Örebro University Hospital, Örebro University, Örebro, Sweden
| | - J W G Derksen
- Julius Center for Health Sciences and Primary Care, Department of Epidemiology and Health Economics, University Medical Centre Utrecht, Utrecht University, Utrecht
| | - K P M Suijkerbuijk
- Department of Medical Oncology, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands
| | - U Dafni
- Laboratory of Biostatistics, Department of Nursing, National and Kapodistrian University of Athens, Athens; Frontier Science Foundation Hellas, Athens, Greece
| | - T Dellaporta
- Frontier Science Foundation Hellas, Athens, Greece
| | - A Vogel
- Department of Gastroenterology, Hepatology and Endocrinology, Medical School of Hannover, Hannover, Germany; Toronto Center of Liver Disease, Toronto General Hospital, University Health Network, Toronto; Princess Margaret Cancer Centre, University of Toronto, Toronto, Canada
| | - A Prelaj
- AI-ON-Lab, Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan; NEARLab, Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - R H H Groenwold
- Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands
| | - H Martins
- Business Research Unit, ISCTE Business School, ISCTE-IUL, Lisbon, Portugal
| | - R Stahel
- ETOP IBCSG Partners Foundation, Berne, Switzerland
| | - J Bliss
- ICR-CTSU, Division of Clinical Studies, The Institute of Cancer Research, London, UK
| | - J Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden; Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany
| | - N Ribelles
- Medical Oncology Intercenter Unit, Regional and Virgen de la Victoria University Hospitals, IBIMA, Málaga, Spain
| | - F Perrone
- Clinical Trial Unit, Istituto Nazionale Tumori IRCCS Fondazione G. Pascale, Naples, Italy
| | - P S Hall
- Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK
| | - R Dienstmann
- Oncoclinicas Precision Medicine, Oncoclinicas Group, São Paulo, Brazil; Oncology Data Science Group, Vall d'Hebron Institute of Oncology, Barcelona, Spain
| | - C M Booth
- Department of Oncology; Department of Public Health Sciences, Queen's University, Kingston, Canada
| | - G Pentheroudakis
- Scientific and Medical Division, European Society for Medical Oncology (ESMO), Lugano, Switzerland
| | - S Delaloge
- Department of Cancer Medicine, Gustave Roussy, Villejuif, France
| | - M Koopman
- Department of Medical Oncology, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
33
|
Yan F, Jiang L, Ye F, Ping J, Bowley TY, Ness SA, Li CI, Marchetti D, Tang J, Guo Y. Deep neural network based tissue deconvolution of circulating tumor cell RNA. J Transl Med 2023; 21:783. [PMID: 37925448 PMCID: PMC10625696 DOI: 10.1186/s12967-023-04663-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 10/25/2023] [Indexed: 11/06/2023] Open
Abstract
Prior research has shown that the deconvolution of cell-free RNA can uncover the tissue origin. The conventional deconvolution approaches rely on constructing a reference tissue-specific gene panel, which cannot capture the inherent variation present in actual data. To address this, we have developed a novel method that utilizes a neural network framework to leverage the entire training dataset. Our approach involved training a model that incorporated 15 distinct tissue types. Through one semi-independent and two complete independent validations, including deconvolution using a semi in silico dataset, deconvolution with a custom normal tissue mixture RNA-seq data, and deconvolution of longitudinal circulating tumor cell RNA-seq (ctcRNA) data from a cancer patient with metastatic tumors, we demonstrate the efficacy and advantages of the deep-learning approach which were exerted by effectively capturing the inherent variability present in the dataset, thus leading to enhanced accuracy. Sensitivity analyses reveal that neural network models are less susceptible to the presence of missing data, making them more suitable for real-world applications. Moreover, by leveraging the concept of organotropism, we applied our approach to trace the migration of circulating tumor cell-derived RNA (ctcRNA) in a cancer patient with metastatic tumors, thereby highlighting the potential clinical significance of early detection of cancer metastasis.
Collapse
Affiliation(s)
- Fengyao Yan
- Department of Public Health and Sciences, Sylvester Comprehensive Cancer Center, University of Miami, Miami, FL, 33136, USA
- Department of Computer Science, University of South Carolina, Columbia, SC, 29208, USA
| | - Limin Jiang
- Department of Public Health and Sciences, Sylvester Comprehensive Cancer Center, University of Miami, Miami, FL, 33136, USA
| | - Fei Ye
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, 37232, USA
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, 37232, USA
| | - Jie Ping
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, 37232, USA
| | - Tetiana Y Bowley
- Department of Internal Medicine, Comprehensive Cancer Center, University of New Mexico, Albuquerque, NM, 87131, USA
| | - Scott A Ness
- Department of Internal Medicine, Comprehensive Cancer Center, University of New Mexico, Albuquerque, NM, 87131, USA
| | - Chung-I Li
- Department of Statistics, National Cheng Kung University, Tainan, 701401, Taiwan
| | - Dario Marchetti
- Department of Internal Medicine, Comprehensive Cancer Center, University of New Mexico, Albuquerque, NM, 87131, USA
| | - Jijun Tang
- Department of Computer Science, University of South Carolina, Columbia, SC, 29208, USA
| | - Yan Guo
- Department of Public Health and Sciences, Sylvester Comprehensive Cancer Center, University of Miami, Miami, FL, 33136, USA.
| |
Collapse
|
34
|
Muti HS, Röcken C, Behrens HM, Löffler CML, Reitsam NG, Grosser B, Märkl B, Stange DE, Jiang X, Velduizen GP, Truhn D, Ebert MP, Grabsch HI, Kather JN. Deep learning trained on lymph node status predicts outcome from gastric cancer histopathology: a retrospective multicentric study. Eur J Cancer 2023; 194:113335. [PMID: 37862795 DOI: 10.1016/j.ejca.2023.113335] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 08/15/2023] [Accepted: 09/03/2023] [Indexed: 10/22/2023]
Abstract
AIM Gastric cancer (GC) is a tumour entity with highly variant outcomes. Lymph node metastasis is a prognostically adverse biomarker. We hypothesised that GC primary tissue contains information that is predictive of lymph node status and patient prognosis and that this information can be extracted using deep learning (DL). METHODS Using three patient cohorts comprising 1146 patients, we trained and validated a DL system to predict lymph node status directly from haematoxylin and eosin-stained GC tissue sections. We investigated the concordance between the DL-based prediction from the primary tumour slides (aiN score) and the histopathological lymph node status (pN). Furthermore, we assessed the prognostic value of the aiN score alone and when combined with the pN status. RESULTS The aiN score predicted the pN status reaching area under the receiver operating characteristic curves of 0.71 in the training cohort and 0.69 and 0.65 in the two test cohorts. In a multivariate Cox analysis, the aiN score was an independent predictor of patient survival with hazard ratios of 1.5 in the training cohort and of 1.3 and 2.2 in the two test cohorts. A combination of the aiN score and the pN status prognostically stratified patients by survival with p-values <0.05 in logrank tests. CONCLUSION GC primary tumour tissue contains additional prognostic information that is accessible using the aiN score. In combination with the pN status, this can be used for personalised management of GC patients after prospective validation.
Collapse
Affiliation(s)
- Hannah S Muti
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany; Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Dresden, Germany
| | - Christoph Röcken
- Department of Pathology, University Hospital Schleswig-Holstein, Kiel, Germany
| | | | - Chiara M L Löffler
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany; Department of Medicine I, University Hospital Dresden, Dresden, Germany
| | - Nic G Reitsam
- Pathology, Faculty of Medicine, University of Augsburg, Augsburg, Germany
| | - Bianca Grosser
- Pathology, Faculty of Medicine, University of Augsburg, Augsburg, Germany
| | - Bruno Märkl
- Pathology, Faculty of Medicine, University of Augsburg, Augsburg, Germany
| | - Daniel E Stange
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital Carl Gustav Carus Dresden, Dresden, Germany
| | - Xiaofeng Jiang
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| | - Gregory P Velduizen
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| | - Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Germany
| | - Matthias P Ebert
- Department of Medicine II, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany; DKFZ-Hector Cancer Institute at the University Medical Center, Mannheim, Germany; Clinical Cooperation Unit Healthy Metabolism, Center for Preventive Medicine and Digital Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany; Mannheim Institute for Innate Immunoscience (MI3), Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Heike I Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK; Department of Pathology, GROW School for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, the Netherlands
| | - Jakob N Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany; Department of Medicine I, University Hospital Dresden, Dresden, Germany; Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK; Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
35
|
Esce A, Redemann JP, Olson GT, Hanson JA, Agarwal S, Yenwongfai L, Ferreira J, Boyd NH, Bocklage T, Martin DR. Lymph Node Metastases in Papillary Thyroid Carcinoma can be Predicted by a Convolutional Neural Network: a Multi-Institution Study. Ann Otol Rhinol Laryngol 2023; 132:1373-1379. [PMID: 36896865 DOI: 10.1177/00034894231158464] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/11/2023]
Abstract
OBJECTIVES The presence of nodal metastases in patients with papillary thyroid carcinoma (PTC) has both staging and treatment implications. However, lymph nodes are often not removed during thyroidectomy. Prior work has demonstrated the capability of artificial intelligence (AI) to predict the presence of nodal metastases in PTC based on the primary tumor histopathology alone. This study aimed to replicate these results with multi-institutional data. METHODS Cases of conventional PTC were identified from the records of 2 large academic institutions. Only patients with complete pathology data, including at least 3 sampled lymph nodes, were included in the study. Tumors were designated "positive" if they had at least 5 positive lymph node metastases. First, algorithms were trained separately on each institution's data and tested independently on the other institution's data. Then, the data sets were combined and new algorithms were developed and tested. The primary tumors were randomized into 2 groups, one to train the algorithm and another to test it. A low level of supervision was used to train the algorithm. Board-certified pathologists annotated the slides. HALO-AI convolutional neural network and image software was used to perform training and testing. Receiver operator characteristic curves and the Youden J statistic were used for primary analysis. RESULTS There were 420 cases used in analyses, 45% of which were negative. The best performing single institution algorithm had an area under the curve (AUC) of 0.64 with a sensitivity and specificity of 65% and 61% respectively, when tested on the other institution's data. The best performing combined institution algorithm had an AUC of 0.84 with a sensitivity and specificity of 68% and 91% respectively. CONCLUSION A convolutional neural network can produce an accurate and robust algorithm that is capable of predicting nodal metastases from primary PTC histopathology alone even in the setting of multi-institutional data.
Collapse
Affiliation(s)
- Antoinette Esce
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Jordan P Redemann
- Department of Pathology, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Garth T Olson
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Joshua A Hanson
- Department of Pathology, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Shweta Agarwal
- Department of Pathology, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Leonard Yenwongfai
- Department of Pathology, University of Kentucky College of Medicine, Lexington, KY, USA
| | - Juanita Ferreira
- Department of Pathology, University of Kentucky College of Medicine, Lexington, KY, USA
| | - Nathan H Boyd
- Department of Surgery, Division of Otolaryngology Head and Neck Surgery, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| | - Thèrése Bocklage
- Department of Pathology, University of Kentucky College of Medicine, Lexington, KY, USA
| | - David R Martin
- Department of Pathology, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
| |
Collapse
|
36
|
Hu M, Xia X, Chen L, Jin Y, Hu Z, Xia S, Yao X. Emerging biomolecules for practical theranostics of liver hepatocellular carcinoma. Ann Hepatol 2023; 28:101137. [PMID: 37451515 DOI: 10.1016/j.aohep.2023.101137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 06/17/2023] [Accepted: 06/28/2023] [Indexed: 07/18/2023]
Abstract
Most cases of hepatocellular carcinoma (HCC) are able to be diagnosed through regular surveillance in an identifiable patient population with chronic hepatitis B or cirrhosis. Nevertheless, 50% of global cases might present incidentally owing to symptomatic advanced-stage HCC after worsening of liver dysfunction. A systematic search based on PUBMED was performed to identify relevant outcomes, covering newer surveillance modalities including secretory proteins, DNA methylation, miRNAs, and genome sequencing analysis which proposed molecular expression signatures as ideal tools in the early-stage HCC detection. In the face of low accuracy without harmonization on the analytical approaches and data interpretation for liquid biopsy, a more accurate incidence of HCC will be unveiled by using deep machine learning system and multiplex immunohistochemistry analysis. A combination of molecular-secretory biomarkers, high-definition imaging and bedside clinical indexes in a surveillance setting offers a comprehensive range of HCC potential indicators. In addition, the sequential use of numerous lines of systemic anti-HCC therapies will simultaneously benefit more patients in survival. This review provides an overview on the most recent developments in HCC theranostic platform.
Collapse
Affiliation(s)
- Miner Hu
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China
| | - Xiaojun Xia
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China
| | - Lichao Chen
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China
| | - Yunpeng Jin
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China
| | - Zhenhua Hu
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China; Division of Hepatobiliary and Pancreatic Surgery, Department of Surgery, Fourth Affiliated Hospital, School of Medicine, Zhejiang University, Yiwu, Zhejiang, China.
| | - Shudong Xia
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
| | - Xudong Yao
- International Institutes of Medicine, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
| |
Collapse
|
37
|
Xu J, Xu HL, Cao YN, Huang Y, Gao S, Wu QJ, Gong TT. The performance of deep learning on thyroid nodule imaging predicts thyroid cancer: A systematic review and meta-analysis of epidemiological studies with independent external test sets. Diabetes Metab Syndr 2023; 17:102891. [PMID: 37907027 DOI: 10.1016/j.dsx.2023.102891] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 10/06/2023] [Accepted: 10/15/2023] [Indexed: 11/02/2023]
Abstract
BACKGROUND AND AIMS It is still controversial whether deep learning (DL) systems add accuracy to thyroid nodule imaging classification based on the recent available evidence. We conducted this study to analyze the current evidence of DL in thyroid nodule imaging diagnosis in both internal and external test sets. METHODS Until the end of December 2022, PubMed, IEEE, Embase, Web of Science, and the Cochrane Library were searched. We included primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. This systematic review was registered on PROSPERO (CRD42022362892). RESULTS We evaluated evidence from 17 primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. Fourteen studies were deemed eligible for meta-analysis. The pooled sensitivity, specificity, and area under the curve (AUC) of these DL algorithms were 0.89 (95% confidence interval 0.87-0.90), 0.84 (0.82-0.86), and 0.93 (0.91-0.95), respectively. For the internal validation set, the pooled sensitivity, specificity, and AUC were 0.91 (0.89-0.93), 0.88 (0.85-0.91), and 0.96 (0.93-0.97), respectively. In the external validation set, the pooled sensitivity, specificity, and AUC were 0.87 (0.85-0.89), 0.81 (0.77-0.83), and 0.91 (0.88-0.93), respectively. Notably, in subgroup analyses, DL algorithms still demonstrated exceptional diagnostic validity. CONCLUSIONS Current evidence suggests DL-based imaging shows diagnostic performances comparable to clinicians for differentiating thyroid nodules in both the internal and external test sets.
Collapse
Affiliation(s)
- Jin Xu
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China
| | - He-Li Xu
- Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Yi-Ning Cao
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China; Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Ying Huang
- Department of Ultrasound, Shengjing Hospital of China Medical University, Shenyang, China
| | - Song Gao
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Qi-Jun Wu
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China; Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, China; Key Laboratory of Reproductive and Genetic Medicine (China Medical University), National Health Commission, Shenyang, China.
| | - Ting-Ting Gong
- Department of Obstetrics and Gynecology, Shengjing Hospital of China Medical University, Shenyang, China.
| |
Collapse
|
38
|
Dai Q, Tao Y, Liu D, Zhao C, Sui D, Xu J, Shi T, Leng X, Lu M. Ultrasound radiomics models based on multimodal imaging feature fusion of papillary thyroid carcinoma for predicting central lymph node metastasis. Front Oncol 2023; 13:1261080. [PMID: 38023240 PMCID: PMC10643192 DOI: 10.3389/fonc.2023.1261080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 10/09/2023] [Indexed: 12/01/2023] Open
Abstract
Objective This retrospective study aimed to establish ultrasound radiomics models to predict central lymph node metastasis (CLNM) based on preoperative multimodal ultrasound imaging features fusion of primary papillary thyroid carcinoma (PTC). Methods In total, 498 cases of unifocal PTC were randomly divided into two sets which comprised 348 cases (training set) and 150 cases (validition set). In addition, the testing set contained 120 cases of PTC at different times. Post-operative histopathology was the gold standard for CLNM. The following steps were used to build models: the regions of interest were segmented in PTC ultrasound images, multimodal ultrasound image features were then extracted by the deep learning residual neural network with 50-layer network, followed by feature selection and fusion; subsequently, classification was performed using three classical classifiers-adaptive boosting (AB), linear discriminant analysis (LDA), and support vector machine (SVM). The performances of the unimodal models (Unimodal-AB, Unimodal-LDA, and Unimodal-SVM) and the multimodal models (Multimodal-AB, Multimodal-LDA, and Multimodal-SVM) were evaluated and compared. Results The Multimodal-SVM model achieved the best predictive performance than the other models (P < 0.05). For the Multimodal-SVM model validation and testing sets, the areas under the receiver operating characteristic curves (AUCs) were 0.910 (95% CI, 0.894-0.926) and 0.851 (95% CI, 0.833-0.869), respectively. The AUCs of the Multimodal-SVM model were 0.920 (95% CI, 0.881-0.959) in the cN0 subgroup-1 cases and 0.828 (95% CI, 0.769-0.887) in the cN0 subgroup-2 cases. Conclusion The ultrasound radiomics model only based on the PTC multimodal ultrasound image have high clinical value in predicting CLNM and can provide a reference for treatment decisions.
Collapse
Affiliation(s)
- Quan Dai
- Department of Ultrasound, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Medicine & Laboratory of Translational Research in Ultrasound Theranostics, Chengdu, China
| | - Yi Tao
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Dongmei Liu
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Chen Zhao
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Dong Sui
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
- School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, Beijing, China
| | - Jinshun Xu
- Department of Ultrasound, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Medicine & Laboratory of Translational Research in Ultrasound Theranostics, Chengdu, China
| | - Tiefeng Shi
- Department of General Surgery, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Xiaoping Leng
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Man Lu
- Department of Ultrasound, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Medicine & Laboratory of Translational Research in Ultrasound Theranostics, Chengdu, China
| |
Collapse
|
39
|
Sehring J, Dohmen H, Selignow C, Schmid K, Grau S, Stein M, Uhl E, Mukhopadhyay A, Németh A, Amsel D, Acker T. Leveraging Attention-Based Convolutional Neural Networks for Meningioma Classification in Computational Histopathology. Cancers (Basel) 2023; 15:5190. [PMID: 37958364 PMCID: PMC10647687 DOI: 10.3390/cancers15215190] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 10/23/2023] [Accepted: 10/25/2023] [Indexed: 11/15/2023] Open
Abstract
Convolutional neural networks (CNNs) are becoming increasingly valuable tools for advanced computational histopathology, promoting precision medicine through exceptional visual decoding abilities. Meningiomas, the most prevalent primary intracranial tumors, necessitate accurate grading and classification for informed clinical decision-making. Recently, DNA methylation-based molecular classification of meningiomas has proven to be more effective in predicting tumor recurrence than traditional histopathological methods. However, DNA methylation profiling is expensive, labor-intensive, and not widely accessible. Consequently, a digital histology-based prediction of DNA methylation classes would be advantageous, complementing molecular classification. In this study, we developed and rigorously assessed an attention-based multiple-instance deep neural network for predicting meningioma methylation classes using tumor methylome data from 142 (+51) patients and corresponding hematoxylin-eosin-stained histological sections. Pairwise analysis of sample cohorts from three meningioma methylation classes demonstrated high accuracy in two combinations. The performance of our approach was validated using an independent set of 51 meningioma patient samples. Importantly, attention map visualization revealed that the algorithm primarily focuses on tumor regions deemed significant by neuropathologists, offering insights into the decision-making process of the CNN. Our findings highlight the capacity of CNNs to effectively harness phenotypic information from histological sections through computerized images for precision medicine. Notably, this study is the first demonstration of predicting clinically relevant DNA methylome information using computer vision applied to standard histopathology. The introduced AI framework holds great potential in supporting, augmenting, and expediting meningioma classification in the future.
Collapse
Affiliation(s)
- Jannik Sehring
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Hildegard Dohmen
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Carmen Selignow
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Kai Schmid
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Stefan Grau
- Department of Neurosurgery, Hospital Fulda, Pacelliallee 4, D-36043 Fulda, Germany
| | - Marco Stein
- Department of Neurosurgery, University Hospital Gießen, Klinikstr. 33, D-35392 Giessen, Germany
| | - Eberhard Uhl
- Department of Neurosurgery, University Hospital Gießen, Klinikstr. 33, D-35392 Giessen, Germany
| | - Anirban Mukhopadhyay
- Department of Computer Science, Technical University of Darmstadt, Fraunhoferstraße 5, D-64283 Darmstadt, Germany
| | - Attila Németh
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Daniel Amsel
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Till Acker
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| |
Collapse
|
40
|
Shi Y, Zhang C, Pan S, Chen Y, Miao X, He G, Wu Y, Ye H, Weng C, Zhang H, Zhou W, Yang X, Liang C, Chen D, Hong L, Su F. The diagnosis of tuberculous meningitis: advancements in new technologies and machine learning algorithms. Front Microbiol 2023; 14:1290746. [PMID: 37942080 PMCID: PMC10628659 DOI: 10.3389/fmicb.2023.1290746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 10/09/2023] [Indexed: 11/10/2023] Open
Abstract
Tuberculous meningitis (TBM) poses a diagnostic challenge, particularly impacting vulnerable populations such as infants and those with untreated HIV. Given the diagnostic intricacies of TBM, there's a pressing need for rapid and reliable diagnostic tools. This review scrutinizes the efficacy of up-and-coming technologies like machine learning in transforming TBM diagnostics and management. Advanced diagnostic technologies like targeted gene sequencing, real-time polymerase chain reaction (RT-PCR), miRNA assays, and metagenomic next-generation sequencing (mNGS) offer promising avenues for early TBM detection. The capabilities of these technologies are further augmented when paired with mass spectrometry, metabolomics, and proteomics, enriching the pool of disease-specific biomarkers. Machine learning algorithms, adept at sifting through voluminous datasets like medical imaging, genomic profiles, and patient histories, are increasingly revealing nuanced disease pathways, thereby elevating diagnostic accuracy and guiding treatment strategies. While these burgeoning technologies offer hope for more precise TBM diagnosis, hurdles remain in terms of their clinical implementation. Future endeavors should zero in on the validation of these tools through prospective studies, critically evaluating their limitations, and outlining protocols for seamless incorporation into established healthcare frameworks. Through this review, we aim to present an exhaustive snapshot of emerging diagnostic modalities in TBM, the current standing of machine learning in meningitis diagnostics, and the challenges and future prospects of converging these domains.
Collapse
Affiliation(s)
- Yi Shi
- Department of Infectious Diseases, Wenzhou Central Hospital, Wenzhou, China
- The First School of Medicine, Wenzhou Medical University, Wenzhou, China
| | - Chengxi Zhang
- School of Materials Science and Engineering, Shandong Jianzhu University, Jinan, China
| | - Shuo Pan
- The First School of Medicine, Wenzhou Medical University, Wenzhou, China
| | - Yi Chen
- The First School of Medicine, Wenzhou Medical University, Wenzhou, China
| | - Xingguo Miao
- Department of Infectious Diseases, Wenzhou Central Hospital, Wenzhou, China
- Department of Infectious Diseases, Wenzhou Sixth People’s Hospital, Wenzhou, China
- Wenzhou Key Laboratory of Diagnosis and Treatment of Emerging and Recurrent Infectious Diseases, Wenzhou, China
| | - Guoqiang He
- Postgraduate Training Base Alliance of Wenzhou Medical University, Wenzhou, China
- Wenzhou Institute, University of Chinese Academy of Sciences, Wenzhou, China
| | - Yanchan Wu
- School of Electrical and Information Engineering, Quzhou University, Quzhou, China
| | - Hui Ye
- Department of Infectious Diseases, Wenzhou Central Hospital, Wenzhou, China
- Department of Infectious Diseases, Wenzhou Sixth People’s Hospital, Wenzhou, China
- Wenzhou Key Laboratory of Diagnosis and Treatment of Emerging and Recurrent Infectious Diseases, Wenzhou, China
| | - Chujun Weng
- The Fourth Affiliated Hospital Zhejiang University School of Medicine, Yiwu, China
| | - Huanhuan Zhang
- School and Hospital of Stomatology, Wenzhou Medical University, Wenzhou, China
| | - Wenya Zhou
- School and Hospital of Stomatology, Wenzhou Medical University, Wenzhou, China
| | - Xiaojie Yang
- Wenzhou Medical University Renji College, Wenzhou, China
| | - Chenglong Liang
- The First School of Medicine, Wenzhou Medical University, Wenzhou, China
| | - Dong Chen
- Wenzhou Key Laboratory of Diagnosis and Treatment of Emerging and Recurrent Infectious Diseases, Wenzhou, China
- Wenzhou Central Blood Station, Wenzhou, China
| | - Liang Hong
- Department of Infectious Diseases, The Third Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Feifei Su
- Department of Infectious Diseases, Wenzhou Central Hospital, Wenzhou, China
- Department of Infectious Diseases, Wenzhou Sixth People’s Hospital, Wenzhou, China
- Wenzhou Key Laboratory of Diagnosis and Treatment of Emerging and Recurrent Infectious Diseases, Wenzhou, China
| |
Collapse
|
41
|
Zhong NN, Wang HQ, Huang XY, Li ZZ, Cao LM, Huo FY, Liu B, Bu LL. Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives. Semin Cancer Biol 2023; 95:52-74. [PMID: 37473825 DOI: 10.1016/j.semcancer.2023.07.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 07/11/2023] [Accepted: 07/15/2023] [Indexed: 07/22/2023]
Abstract
Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.
Collapse
Affiliation(s)
- Nian-Nian Zhong
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Han-Qi Wang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Xin-Yue Huang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Zi-Zhan Li
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Lei-Ming Cao
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Fang-Yi Huo
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Bing Liu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| | - Lin-Lin Bu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| |
Collapse
|
42
|
Sarkar S, Squire A, Diab H, Rahman MK, Perdomo A, Awosile B, Calle A, Thompson J. Effect of Tryptic Digestion on Sensitivity and Specificity in MALDI-TOF-Based Molecular Diagnostics through Machine Learning. SENSORS (BASEL, SWITZERLAND) 2023; 23:8042. [PMID: 37836873 PMCID: PMC10575185 DOI: 10.3390/s23198042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/15/2023] [Accepted: 09/21/2023] [Indexed: 10/15/2023]
Abstract
The digestion of protein into peptide fragments reduces the size and complexity of protein molecules. Peptide fragments can be analyzed with higher sensitivity (often > 102 fold) and resolution using MALDI-TOF mass spectrometers, leading to improved pattern recognition by common machine learning algorithms. In turn, enhanced sensitivity and specificity for bacterial sorting and/or disease diagnosis may be obtained. To test this hypothesis, four exemplar case studies have been pursued in which samples are sorted into dichotomous groups by machine learning (ML) software based on MALDI-TOF spectra. Samples were analyzed in 'intact' mode in which the proteins present in the sample were not digested with protease prior to MALDI-TOF analysis and separately after the standard overnight tryptic digestion of the same samples. For each case, sensitivity (sens), specificity (spc), and the Youdin index (J) were used to assess the ML model performance. The proteolytic digestion of samples prior to MALDI-TOF analysis substantially enhanced the sensitivity and specificity of dichotomous sorting. Two exceptions were when substantial differences in chemical composition between the samples were present and, in such cases, both 'intact' and 'digested' protocols performed similarly. The results suggest proteolytic digestion prior to analysis can improve sorting in MALDI/ML-based workflows and may enable improved biomarker discovery. However, when samples are easily distinguishable protein digestion is not necessary to obtain useful diagnostic results.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Jonathan Thompson
- School of Veterinary Medicine, Texas Tech University, 7671 Evans Dr., Amarillo, TX 79106, USA; (S.S.); (A.S.); (M.K.R.)
| |
Collapse
|
43
|
Wang A, Zhou J, Wang G, Zhang B, Xin H, Zhou H. Deep learning of endoscopic features for the assessment of neoadjuvant therapy response in locally advanced rectal cancer. Asian J Surg 2023; 46:3568-3574. [PMID: 37062601 DOI: 10.1016/j.asjsur.2023.03.165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 03/28/2023] [Accepted: 03/30/2023] [Indexed: 04/18/2023] Open
Abstract
BACKGROUND For locally advanced rectal cancer (LARC), accurate response evaluation is necessary to select complete responders after neoadjuvant therapy (NAT) for a watch-and-wait (W&W) strategy. Algorithms based on deep learning have shown great value in medical image analyses. Here we used deep learning algorithms of endoscopic images for the assessment of NAT response in LARC. METHOD 214 LARC patients were retrospectively included in the study. After NAT, these patients underwent total mesorectal excision (TME) surgery. Among them, 51 (23.8%) of the patients achieved a pathological complete response (pCR). 160 patients from Shanghai Changzheng Hospital were regarded as primary dataset, and the other 54 patients from Zhejiang Cancer Hospital were regarded as validation dataset. ResNet-18 and DenseNet-121 were applied to train the models based on endoscopic images after NAT. Deep learning models were valid in the validation dataset and compared to manual method. RESULTS The performances were comparable in AUC between deep learning models and manual method. For mean metrics, sensitivity (0.750 vs. 0.417) and AUC (0.716 vs. 0.601) in ResNet-18 deep learning model were higher than those in the manual method. The deep learning models were able to identify the endoscopic features associated with NAT response by the heatmaps. A diagnostic flow diagram which integrated the deep learning model to assist the clinicians in making decisions for W&W strategy was constructed. CONCLUSIONS We created deep learning models using endoscopic features for assessment of NAT in LARC. The deep learning models achieved modest accuracies and performed comparably to manual method.
Collapse
Affiliation(s)
- Anqi Wang
- Department of Colorectal Surgery, Changzheng Hospital, Navy Medical University, China
| | - Jieli Zhou
- UM-SJTU Joint Institute, Shanghai Jiao Tong University, China
| | - Gang Wang
- Department of Colorectal Surgery, Zhejiang Cancer Hospital, China
| | - Beibei Zhang
- Department of Dermatology, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, China.
| | - Hongyi Xin
- UM-SJTU Joint Institute, Shanghai Jiao Tong University, China.
| | - Haiyang Zhou
- Department of Colorectal Surgery, Changzheng Hospital, Navy Medical University, China.
| |
Collapse
|
44
|
Xiong Y, Guo W, Liang Z, Wu L, Ye G, Liang YY, Wen C, Yang F, Chen S, Zeng XW, Xu F. Deep learning-based diagnosis of osteoblastic bone metastases and bone islands in computed tomograph images: a multicenter diagnostic study. Eur Radiol 2023; 33:6359-6368. [PMID: 37060446 PMCID: PMC10415522 DOI: 10.1007/s00330-023-09573-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Revised: 03/08/2023] [Accepted: 03/20/2023] [Indexed: 04/16/2023]
Abstract
OBJECTIVE To develop and validate a deep learning (DL) model based on CT for differentiating bone islands and osteoblastic bone metastases. MATERIALS AND METHODS The patients with sclerosing bone lesions (SBLs) were retrospectively included in three hospitals. The images from site 1 were randomly assigned to the training (70%) and intrinsic verification (10%) datasets for developing the two-dimensional (2D) DL model (single-slice input) and "2.5-dimensional" (2.5D) DL model (three-slice input) and to the internal validation dataset (20%) for evaluating the performance of both models. The diagnostic performance was evaluated using the internal validation set from site 1 and additional external validation datasets from site 2 and site 3. And statistically analyze the performance of 2D and 2.5D DL models. RESULTS In total, 1918 SBLs in 728 patients in site 1, 122 SBLs in 71 patients in site 2, and 71 SBLs in 47 patients in site 3 were used to develop and test the 2D and 2.5D DL models. The best performance was obtained using the 2.5D DL model, which achieved an AUC of 0.996 (95% confidence interval [CI], 0.995-0.996), 0.958 (95% CI, 0.958-0.960), and 0.952 (95% CI, 0.951-0.953) and accuracies of 0.950, 0.902, and 0.863 for the internal validation set, the external validation set from site 2 and site 3, respectively. CONCLUSION A DL model based on a three-slice CT image input (2.5D DL model) can improve the prediction of osteoblastic bone metastases, which can facilitate clinical decision-making. KEY POINTS • This study investigated the value of deep learning models in identifying bone islands and osteoblastic bone metastases. • Three-slice CT image input (2.5D DL model) outweighed the 2D model in the classification of sclerosing bone lesions. • The 2.5D deep learning model showed excellent performance using the internal (AUC, 0.996) and two external (AUC, 0.958; AUC, 0.952) validation sets.
Collapse
Affiliation(s)
- Yuchao Xiong
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Wei Guo
- Department of Radiology, Wuhan Third Hospital, Tongren Hospital of Wuhan University, 241 Liuyang Road, Wuhan, 430063, Hubei Province, China
| | - Zhiping Liang
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Li Wu
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Guoxi Ye
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Ying-Ying Liang
- Department of Radiology, Guangzhou First People's Hospital, School of Medicine, South China University of Technology, 1Panfu Road, Guangzhou, 510180, Guangdong Province, China
| | - Chao Wen
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Feng Yang
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Song Chen
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China
| | - Xu-Wen Zeng
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China.
| | - Fan Xu
- Department of Radiology, Guangzhou Red Cross Hospital (Guangzhou Red Cross Hospital, Medical College of Jinan University), 396 Tongfu Road, Guangzhou, 510220, Guangdong Province, China.
| |
Collapse
|
45
|
Veldhuizen GP, Röcken C, Behrens HM, Cifci D, Muti HS, Yoshikawa T, Arai T, Oshima T, Tan P, Ebert MP, Pearson AT, Calderaro J, Grabsch HI, Kather JN. Deep learning-based subtyping of gastric cancer histology predicts clinical outcome: a multi-institutional retrospective study. Gastric Cancer 2023; 26:708-720. [PMID: 37269416 PMCID: PMC10361890 DOI: 10.1007/s10120-023-01398-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Accepted: 05/09/2023] [Indexed: 06/05/2023]
Abstract
INTRODUCTION The Laurén classification is widely used for Gastric Cancer (GC) histology subtyping. However, this classification is prone to interobserver variability and its prognostic value remains controversial. Deep Learning (DL)-based assessment of hematoxylin and eosin (H&E) stained slides is a potentially useful tool to provide an additional layer of clinically relevant information, but has not been systematically assessed in GC. OBJECTIVE We aimed to train, test and externally validate a deep learning-based classifier for GC histology subtyping using routine H&E stained tissue sections from gastric adenocarcinomas and to assess its potential prognostic utility. METHODS We trained a binary classifier on intestinal and diffuse type GC whole slide images for a subset of the TCGA cohort (N = 166) using attention-based multiple instance learning. The ground truth of 166 GC was obtained by two expert pathologists. We deployed the model on two external GC patient cohorts, one from Europe (N = 322) and one from Japan (N = 243). We assessed classification performance using the Area Under the Receiver Operating Characteristic Curve (AUROC) and prognostic value (overall, cancer specific and disease free survival) of the DL-based classifier with uni- and multivariate Cox proportional hazard models and Kaplan-Meier curves with log-rank test statistics. RESULTS Internal validation using the TCGA GC cohort using five-fold cross-validation achieved a mean AUROC of 0.93 ± 0.07. External validation showed that the DL-based classifier can better stratify GC patients' 5-year survival compared to pathologist-based Laurén classification for all survival endpoints, despite frequently divergent model-pathologist classifications. Univariate overall survival Hazard Ratios (HRs) of pathologist-based Laurén classification (diffuse type versus intestinal type) were 1.14 (95% Confidence Interval (CI) 0.66-1.44, p-value = 0.51) and 1.23 (95% CI 0.96-1.43, p-value = 0.09) in the Japanese and European cohorts, respectively. DL-based histology classification resulted in HR of 1.46 (95% CI 1.18-1.65, p-value < 0.005) and 1.41 (95% CI 1.20-1.57, p-value < 0.005), in the Japanese and European cohorts, respectively. In diffuse type GC (as defined by the pathologist), classifying patients using the DL diffuse and intestinal classifications provided a superior survival stratification, and demonstrated statistically significant survival stratification when combined with pathologist classification for both the Asian (overall survival log-rank test p-value < 0.005, HR 1.43 (95% CI 1.05-1.66, p-value = 0.03) and European cohorts (overall survival log-rank test p-value < 0.005, HR 1.56 (95% CI 1.16-1.76, p-value < 0.005)). CONCLUSION Our study shows that gastric adenocarcinoma subtyping using pathologist's Laurén classification as ground truth can be performed using current state of the art DL techniques. Patient survival stratification seems to be better by DL-based histology typing compared with expert pathologist histology typing. DL-based GC histology typing has potential as an aid in subtyping. Further investigations are warranted to fully understand the underlying biological mechanisms for the improved survival stratification despite apparent imperfect classification by the DL algorithm.
Collapse
Affiliation(s)
| | - Christoph Röcken
- Department of Pathology, Christian-Albrechts University, Kiel, Germany
| | | | - Didem Cifci
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Hannah Sophie Muti
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- Department of Visceral, Thoracic and Vascular Surgery, Technical University Dresden, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Takaki Yoshikawa
- Department of Gastric Surgery, National Cancer Center Hospital, Tokyo, Japan
| | - Tomio Arai
- Department of Pathology, Tokyo Metropolitan Geriatric Hospital and Institute of Gerontology, Tokyo, Japan
| | - Takashi Oshima
- Department of Gastrointestinal Surgery, Kanagawa Cancer Center, Yokohama, Japan
| | - Patrick Tan
- Duke-NUS Medical School, Singapore, Singapore
| | - Matthias P Ebert
- Department of Medicine II, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- DKFZ-Hector Cancer Institute at the University Medical Center, Mannheim, Germany
- Clinical Cooperation Unit Healthy Metabolism, Center for Preventive Medicine and Digital Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- Mannheim Institute for Innate Immunoscience (MI3), Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Alexander T Pearson
- Department of Medicine, Section of Hematology/Oncology, The University of Chicago, Chicago, IL, USA
| | - Julien Calderaro
- Université Paris Est Créteil, INSERM, IMRB, Créteil, France
- Department of Pathology, Assistance Publique-Hôpitaux de Paris, Henri Mondor-Albert Chenevier University Hospital, Créteil, France
| | - Heike I Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Pathology, GROW School for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany.
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
46
|
Escudero Sanchez L, Buddenkotte T, Al Sa’d M, McCague C, Darcy J, Rundo L, Samoshkin A, Graves MJ, Hollamby V, Browne P, Crispin-Ortuzar M, Woitek R, Sala E, Schönlieb CB, Doran SJ, Öktem O. Integrating Artificial Intelligence Tools in the Clinical Research Setting: The Ovarian Cancer Use Case. Diagnostics (Basel) 2023; 13:2813. [PMID: 37685352 PMCID: PMC10486639 DOI: 10.3390/diagnostics13172813] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 07/31/2023] [Accepted: 08/22/2023] [Indexed: 09/10/2023] Open
Abstract
Artificial intelligence (AI) methods applied to healthcare problems have shown enormous potential to alleviate the burden of health services worldwide and to improve the accuracy and reproducibility of predictions. In particular, developments in computer vision are creating a paradigm shift in the analysis of radiological images, where AI tools are already capable of automatically detecting and precisely delineating tumours. However, such tools are generally developed in technical departments that continue to be siloed from where the real benefit would be achieved with their usage. Significant effort still needs to be made to make these advancements available, first in academic clinical research and ultimately in the clinical setting. In this paper, we demonstrate a prototype pipeline based entirely on open-source software and free of cost to bridge this gap, simplifying the integration of tools and models developed within the AI community into the clinical research setting, ensuring an accessible platform with visualisation applications that allow end-users such as radiologists to view and interact with the outcome of these AI tools.
Collapse
Affiliation(s)
- Lorena Escudero Sanchez
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- National Cancer Imaging Translational Accelerator (NCITA) Consortium, UK
| | - Thomas Buddenkotte
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge CB3 0WA, UK
- Department for Diagnostic and Interventional Radiology and Nuclear Medicine, University Hospital Hamburg-Eppendorf, 20246 Hamburg, Germany
- Jung Diagnostics GmbH, 22335 Hamburg, Germany
| | - Mohammad Al Sa’d
- National Cancer Imaging Translational Accelerator (NCITA) Consortium, UK
- Cancer Imaging Centre, Department of Surgery & Cancer, Imperial College, London SW7 2AZ, UK
| | - Cathal McCague
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- Cambridge University Hospitals NHS Foundation Trust, Cambridge CB2 0QQ, UK
| | - James Darcy
- National Cancer Imaging Translational Accelerator (NCITA) Consortium, UK
- Division of Radiotherapy and Imaging, Institute of Cancer Research, London SW7 3RP, UK
| | - Leonardo Rundo
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- Department of Information and Electrical Engineering and Applied Mathematics (DIEM), University of Salerno, 84084 Fisciano, Italy
| | - Alex Samoshkin
- Office for Translational Research, School of Clinical Medicine, University of Cambridge, Cambridge CB2 0SP, UK
| | - Martin J. Graves
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cambridge University Hospitals NHS Foundation Trust, Cambridge CB2 0QQ, UK
| | - Victoria Hollamby
- Research and Information Governance, School of Clinical Medicine, University of Cambridge, Cambridge CB2 0SP, UK
| | - Paul Browne
- High Performance Computing Department, University of Cambridge, Cambridge CB3 0RB, UK
| | - Mireia Crispin-Ortuzar
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- Department of Oncology, University of Cambridge, Cambridge CB2 0XZ, UK
| | - Ramona Woitek
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- Research Centre for Medical Image Analysis and Artificial Intelligence (MIAAI), Department of Medicine, Faculty of Medicine and Dentistry, Danube Private University, 3500 Krems, Austria
| | - Evis Sala
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
- Cancer Research UK Cambridge Centre, Li Ka Shing Centre, Cambridge CB2 0RE, UK
- National Cancer Imaging Translational Accelerator (NCITA) Consortium, UK
- Cambridge University Hospitals NHS Foundation Trust, Cambridge CB2 0QQ, UK
- Dipartimento di Scienze Radiologiche ed Ematologiche, Universita Cattolica del Sacro Cuore, 00168 Rome, Italy
- Dipartimento Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Policlinico Universitario A. Gemelli IRCCS, 00168 Rome, Italy
| | - Carola-Bibiane Schönlieb
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge CB3 0WA, UK
| | - Simon J. Doran
- National Cancer Imaging Translational Accelerator (NCITA) Consortium, UK
- Division of Radiotherapy and Imaging, Institute of Cancer Research, London SW7 3RP, UK
| | - Ozan Öktem
- Department of Mathematics, KTH-Royal Institute of Technology, SE-100 44 Stockholm, Sweden
| |
Collapse
|
47
|
Huynh BN, Groendahl AR, Tomic O, Liland KH, Knudtsen IS, Hoebers F, van Elmpt W, Malinen E, Dale E, Futsaether CM. Head and neck cancer treatment outcome prediction: a comparison between machine learning with conventional radiomics features and deep learning radiomics. Front Med (Lausanne) 2023; 10:1217037. [PMID: 37711738 PMCID: PMC10498924 DOI: 10.3389/fmed.2023.1217037] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 07/07/2023] [Indexed: 09/16/2023] Open
Abstract
Background Radiomics can provide in-depth characterization of cancers for treatment outcome prediction. Conventional radiomics rely on extraction of image features within a pre-defined image region of interest (ROI) which are typically fed to a classification algorithm for prediction of a clinical endpoint. Deep learning radiomics allows for a simpler workflow where images can be used directly as input to a convolutional neural network (CNN) with or without a pre-defined ROI. Purpose The purpose of this study was to evaluate (i) conventional radiomics and (ii) deep learning radiomics for predicting overall survival (OS) and disease-free survival (DFS) for patients with head and neck squamous cell carcinoma (HNSCC) using pre-treatment 18F-fluorodeoxuglucose positron emission tomography (FDG PET) and computed tomography (CT) images. Materials and methods FDG PET/CT images and clinical data of patients with HNSCC treated with radio(chemo)therapy at Oslo University Hospital (OUS; n = 139) and Maastricht University Medical Center (MAASTRO; n = 99) were collected retrospectively. OUS data was used for model training and initial evaluation. MAASTRO data was used for external testing to assess cross-institutional generalizability. Models trained on clinical and/or conventional radiomics features, with or without feature selection, were compared to CNNs trained on PET/CT images without or with the gross tumor volume (GTV) included. Model performance was measured using accuracy, area under the receiver operating characteristic curve (AUC), Matthew's correlation coefficient (MCC), and the F1 score calculated for both classes separately. Results CNNs trained directly on images achieved the highest performance on external data for both endpoints. Adding both clinical and radiomics features to these image-based models increased performance further. Conventional radiomics including clinical data could achieve competitive performance. However, feature selection on clinical and radiomics data lead to overfitting and poor cross-institutional generalizability. CNNs without tumor and node contours achieved close to on-par performance with CNNs including contours. Conclusion High performance and cross-institutional generalizability can be achieved by combining clinical data, radiomics features and medical images together with deep learning models. However, deep learning models trained on images without contours can achieve competitive performance and could see potential use as an initial screening tool for high-risk patients.
Collapse
Affiliation(s)
- Bao Ngoc Huynh
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | | | - Oliver Tomic
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Kristian Hovde Liland
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
| | - Ingerid Skjei Knudtsen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | - Frank Hoebers
- Department of Radiation Oncology (MAASTRO), Maastricht University Medical Center, Maastricht, Netherlands
- GROW School for Oncology and Reproduction, Maastricht University Medical Center, Maastricht, Netherlands
| | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), Maastricht University Medical Center, Maastricht, Netherlands
- GROW School for Oncology and Reproduction, Maastricht University Medical Center, Maastricht, Netherlands
| | - Eirik Malinen
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
- Department of Physics, University of Oslo, Oslo, Norway
| | - Einar Dale
- Department of Oncology, Oslo University Hospital, Oslo, Norway
| | | |
Collapse
|
48
|
Verghese G, Li M, Liu F, Lohan A, Kurian NC, Meena S, Gazinska P, Shah A, Oozeer A, Chan T, Opdam M, Linn S, Gillett C, Alberts E, Hardiman T, Jones S, Thavaraj S, Jones JL, Salgado R, Pinder SE, Rane S, Sethi A, Grigoriadis A. Multiscale deep learning framework captures systemic immune features in lymph nodes predictive of triple negative breast cancer outcome in large-scale studies. J Pathol 2023; 260:376-389. [PMID: 37230111 PMCID: PMC10720675 DOI: 10.1002/path.6088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 02/27/2023] [Accepted: 04/11/2023] [Indexed: 05/27/2023]
Abstract
The suggestion that the systemic immune response in lymph nodes (LNs) conveys prognostic value for triple-negative breast cancer (TNBC) patients has not previously been investigated in large cohorts. We used a deep learning (DL) framework to quantify morphological features in haematoxylin and eosin-stained LNs on digitised whole slide images. From 345 breast cancer patients, 5,228 axillary LNs, cancer-free and involved, were assessed. Generalisable multiscale DL frameworks were developed to capture and quantify germinal centres (GCs) and sinuses. Cox regression proportional hazard models tested the association between smuLymphNet-captured GC and sinus quantifications and distant metastasis-free survival (DMFS). smuLymphNet achieved a Dice coefficient of 0.86 and 0.74 for capturing GCs and sinuses, respectively, and was comparable to an interpathologist Dice coefficient of 0.66 (GC) and 0.60 (sinus). smuLymphNet-captured sinuses were increased in LNs harbouring GCs (p < 0.001). smuLymphNet-captured GCs retained clinical relevance in LN-positive TNBC patients whose cancer-free LNs had on average ≥2 GCs, had longer DMFS (hazard ratio [HR] = 0.28, p = 0.02) and extended GCs' prognostic value to LN-negative TNBC patients (HR = 0.14, p = 0.002). Enlarged smuLymphNet-captured sinuses in involved LNs were associated with superior DMFS in LN-positive TNBC patients in a cohort from Guy's Hospital (multivariate HR = 0.39, p = 0.039) and with distant recurrence-free survival in 95 LN-positive TNBC patients of the Dutch-N4plus trial (HR = 0.44, p = 0.024). Heuristic scoring of subcapsular sinuses in LNs of LN-positive Tianjin TNBC patients (n = 85) cross-validated the association of enlarged sinuses with shorter DMFS (involved LNs: HR = 0.33, p = 0.029 and cancer-free LNs: HR = 0.21 p = 0.01). Morphological LN features reflective of cancer-associated responses are robustly quantifiable by smuLymphNet. Our findings further strengthen the value of assessment of LN properties beyond the detection of metastatic deposits for prognostication of TNBC patients. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Gregory Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Mengyuan Li
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Fangfang Liu
- Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer, Tianjin's Clinical Research Center for Cancer, Key Laboratory of Breast Cancer Prevention and Therapy, Tianjin Medical University, Ministry of EducationKey Laboratory of Cancer Prevention and TherapyTianjinPR China
| | - Amit Lohan
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Nikhil Cherian Kurian
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Swati Meena
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Patrycja Gazinska
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Biobank Research GroupLukasiewicz Research Network, PORT Polish Center for Technology DevelopmentWroclawPoland
| | - Aekta Shah
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Department of PathologyTata Memorial Centre, Tata Memorial Hospital, Homi Bhabha National InstituteMumbaiIndia
| | - Aasiyah Oozeer
- King's Health Partners Cancer Biobank, King's College LondonLondonUK
| | - Terry Chan
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
| | - Mark Opdam
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
| | - Sabine Linn
- Division of Molecular PathologyThe Netherlands Cancer InstituteAmsterdamThe Netherlands
- Department of Medical OncologyThe Netherlands Cancer Institute, Antoni van LeeuwenhoekAmsterdamThe Netherlands
- Department of PathologyUniversity Medical CentreUtrechtThe Netherlands
| | - Cheryl Gillett
- King's Health Partners Cancer Biobank, King's College LondonLondonUK
| | - Elena Alberts
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Thomas Hardiman
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Samantha Jones
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of LondonLondonUK
| | - Selvam Thavaraj
- Faculty of Dentistry, Oral & Craniofacial ScienceKing's College LondonLondonUK
- Head and Neck PathologyGuy's & St Thomas' NHS Foundation TrustLondonUK
| | - J Louise Jones
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of LondonLondonUK
| | - Roberto Salgado
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Division of ResearchPeter Mac Callum Cancer CentreMelbourneAustralia
| | - Sarah E Pinder
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Swapnil Rane
- Department of PathologyTata Memorial Centre‐ACTREC, HBNIMumbaiIndia
| | - Amit Sethi
- Department of Electrical EngineeringIndian Institute of Technology BombayMumbaiIndia
| | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Breast Cancer Now Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| |
Collapse
|
49
|
Thagaard J, Broeckx G, Page DB, Jahangir CA, Verbandt S, Kos Z, Gupta R, Khiroya R, Abduljabbar K, Acosta Haab G, Acs B, Akturk G, Almeida JS, Alvarado‐Cabrero I, Amgad M, Azmoudeh‐Ardalan F, Badve S, Baharun NB, Balslev E, Bellolio ER, Bheemaraju V, Blenman KRM, Botinelly Mendonça Fujimoto L, Bouchmaa N, Burgues O, Chardas A, Chon U Cheang M, Ciompi F, Cooper LAD, Coosemans A, Corredor G, Dahl AB, Dantas Portela FL, Deman F, Demaria S, Doré Hansen J, Dudgeon SN, Ebstrup T, Elghazawy M, Fernandez‐Martín C, Fox SB, Gallagher WM, Giltnane JM, Gnjatic S, Gonzalez‐Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hart SN, Hartman J, Hauberg S, Hewitt S, Hida AI, Horlings HM, Husain Z, Hytopoulos E, Irshad S, Janssen EAM, Kahila M, Kataoka TR, Kawaguchi K, Kharidehal D, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Kovács A, Laenkholm A, Lang‐Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Ly A, Madabhushi A, Maley SK, Manur Narasimhamurthy V, Marks DK, McDonald ES, Mehrotra R, Michiels S, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault‐Llorca F, Perera RD, Pinard CJ, Pinto‐Cardenas JC, Pruneri G, Pusztai L, Rahman A, Rajpoot NM, Rapoport BL, Rau TT, Reis‐Filho JS, Ribeiro JM, Rimm D, Roslind A, Vincent‐Salomon A, Salto‐Tellez M, Saltz J, Sayed S, Scott E, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Fineberg S, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, van Diest PJ, Verghese GE, Viale G, Vieth M, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Zin RM, Adams S, Bartlett J, Loibl S, Denkert C, Savas P, Loi S, Salgado R, Specht Stovgaard E. Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J Pathol 2023; 260:498-513. [PMID: 37608772 PMCID: PMC10518802 DOI: 10.1002/path.6155] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 06/07/2023] [Indexed: 08/24/2023]
Abstract
The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Jeppe Thagaard
- Technical University of DenmarkKongens LyngbyDenmark
- Visiopharm A/SHørsholmDenmark
| | - Glenn Broeckx
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of MedicineAntwerp UniversityAntwerpBelgium
| | - David B Page
- Earle A Chiles Research InstituteProvidence Cancer InstitutePortlandORUSA
| | - Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | - Sara Verbandt
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Zuzana Kos
- Department of Pathology and Laboratory MedicineBC Cancer Vancouver Centre, University of British ColumbiaVancouverBritish ColumbiaCanada
| | - Rajarsi Gupta
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Reena Khiroya
- Department of Cellular PathologyUniversity College Hospital LondonLondonUK
| | | | | | - Balazs Acs
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co IncRahwayNJUSA
| | - Jonas S Almeida
- Division of Cancer Epidemiology and Genetics (DCEG)National Cancer Institute (NCI)Rockville, MDUSA
| | | | - Mohamed Amgad
- Department of PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | | | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Emory University School of MedicineEmory University Winship Cancer InstituteAtlantaGAUSA
| | | | - Eva Balslev
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de MedicinaUniversidad de La FronteraTemucoChile
| | | | - Kim RM Blenman
- Department of Internal Medicine Section of Medical Oncology and Yale Cancer CenterYale School of MedicineNew HavenCTUSA
- Department of Computer ScienceYale School of Engineering and Applied ScienceNew HavenCTUSA
| | | | - Najat Bouchmaa
- Institute of Biological Sciences, Faculty of Medical SciencesMohammed VI Polytechnic University (UM6P)Ben‐GuerirMorocco
| | - Octavio Burgues
- Pathology DepartmentHospital Cliníco Universitario de Valencia/InclivaValenciaSpain
| | - Alexandros Chardas
- Department of Pathobiology & Population SciencesThe Royal Veterinary CollegeLondonUK
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR‐CTSU, Division of Clinical StudiesThe Institute of Cancer ResearchLondonUK
| | - Francesco Ciompi
- Radboud University Medical CenterDepartment of PathologyNijmegenThe Netherlands
| | - Lee AD Cooper
- Department of PathologyNorthwestern Feinberg School of MedicineChicagoILUSA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and ImmunotherapyKU LeuvenLeuvenBelgium
| | - Germán Corredor
- Biomedical Engineering DepartmentEmory UniversityAtlantaGAUSA
| | - Anders B Dahl
- Technical University of DenmarkKongens LyngbyDenmark
| | | | | | - Sandra Demaria
- Department of Radiation OncologyWeill Cornell MedicineNew YorkNYUSA
- Department of Pathology and Laboratory MedicineWeill Cornell MedicineNew YorkNYUSA
| | | | - Sarah N Dudgeon
- Conputational Biology and BioinformaticsYale UniversityNew HavenCTUSA
| | | | | | - Claudio Fernandez‐Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN‐techUniversitat Politècnica de ValènciaValenciaSpain
| | - Stephen B Fox
- Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/Onc, and Pathology, Tisch Cancer Institute – Precision Immunology InstituteIcahn School of Medicine at Mount SinaiNew YorkNYUSA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Niels Halama
- Department of Translational ImmunotherapyGerman Cancer Research CenterHeidelbergGermany
| | - Matthew G Hanna
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | | | - Steven N Hart
- Department of Laboratory Medicine and PathologyMayo ClinicRochester, MNUSA
| | - Johan Hartman
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Søren Hauberg
- Technical University of DenmarkKongens LyngbyDenmark
| | - Stephen Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer InstituteNational Institutes of HealthBethesdaMDUSA
| | - Akira I Hida
- Department of PathologyMatsuyama Shimin HospitalMatsuyamaJapan
| | - Hugo M Horlings
- Division of PathologyNetherlands Cancer Institute (NKI)AmsterdamThe Netherlands
| | | | | | - Sheeba Irshad
- King's College London & Guy's & St Thomas’ NHS TrustLondonUK
| | - Emiel AM Janssen
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | | | | | - Kosuke Kawaguchi
- Department of Breast SurgeryKyoto University Graduate School of MedicineKyotoJapan
| | | | - Andrey I Khramtsov
- Department of Pathology and Laboratory MedicineAnn & Robert H. Lurie Children's Hospital of ChicagoChicagoILUSA
| | - Umay Kiraz
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | - Pawan Kirtani
- Department of HistopathologyAakash Healthcare Super Speciality HospitalNew DelhiIndia
| | - Liudmila L Kodach
- Department of PathologyNetherlands Cancer Institute – Antoni van Leeuwenhoek HospitalAmsterdamThe Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product DevelopmentF. Hoffmann‐La Roche AGBaselSwitzerland
| | - Anikó Kovács
- Department of Clinical PathologySahlgrenska University HospitalGothenburgSweden
- Institute of Biomedicine, Sahlgrenska AcademyUniversity of GothenburgGothenburgSweden
| | - Anne‐Vibeke Laenkholm
- Department of Surgical PathologyZealand University HospitalRoskildeDenmark
- Department of Surgical PathologyUniversity of CopenhagenCopenhagenDenmark
| | - Corinna Lang‐Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Denis Larsimont
- Institut Jules BordetUniversité Libre de BruxellesBrusselsBelgium
| | - Jochen K Lennerz
- Center for Integrated DiagnosticsMassachusetts General Hospital/Harvard Medical SchoolBostonMAUSA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | - Xiaoxian Li
- Department of Pathology and Laboratory MedicineEmory UniversityAtlantaGAUSA
| | - Amy Ly
- Department of PathologyMassachusetts General HospitalBostonMAUSA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, PathologyGeorgia Institute of Technology and Emory UniversityAtlantaGAUSA
| | - Sai K Maley
- NRG Oncology/NSABP FoundationPittsburghPAUSA
| | | | | | - Elizabeth S McDonald
- Breast Cancer Translational Research GroupUniversity of PennsylvaniaPhiladelphiaPAUSA
| | - Ravi Mehrotra
- Indian Cancer Genomic AtlasPuneIndia
- Centre for Health, Innovation and Policy FoundationNoidaIndia
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, InsermUniversity Paris‐Saclay, Ligue Contre le Cancer labeled TeamVillejuifFrance
| | - Fayyaz ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and PathologyUniversity of WashingtonSeattle, WAUSA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCL and Cellular Pathology DepartmentUCLHLondonUK
| | - Shamim Mushtaq
- Department of BiochemistryZiauddin UniversityKarachiPakistan
| | - Hussain Nighat
- Pathology and Laboratory MedicineAll India Institute of Medical sciencesRaipurIndia
| | - Thomas Papathomas
- Institute of Metabolism and Systems ResearchUniversity of BirminghamBirminghamUK
- Department of Clinical PathologyDrammen Sykehus, Vestre Viken HFDrammenNorway
| | - Frederique Penault‐Llorca
- Centre Jean Perrin, Université Clermont Auvergne, INSERM, U1240 Imagerie Moléculaire et Stratégies ThéranostiquesClermont FerrandFrance
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure EngineeringUniversity of MelbourneMelbourneVictoriaAustralia
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Christopher J Pinard
- Radiogenomics LaboratorySunnybrook Health Sciences CentreTorontoOntarioCanada
- Department of Clinical Studies, Ontario Veterinary CollegeUniversity of GuelphGuelphOntarioCanada
- Department of OncologyLakeshore Animal Health PartnersMississaugaOntarioCanada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE‐AI)University of GuelphGuelphOntarioCanada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory MedicineFondazione IRCCS Istituto Nazionale dei TumoriMilanItaly
- Faculty of Medicine and SurgeryUniversity of MilanMilanItaly
| | - Lajos Pusztai
- Yale Cancer CenterYale UniversityNew HavenCTUSA
- Department of Medical Oncology, Yale School of MedicineYale UniversityNew HavenCTUSA
| | - Arman Rahman
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of RosebankJohannesburgSouth Africa
- Department of Immunology, Faculty of Health SciencesUniversity of PretoriaPretoriaSouth Africa
| | - Tilman T Rau
- Institute of PathologyUniversity Hospital Düsseldorf and Heinrich‐Heine‐University DüsseldorfDüsseldorfGermany
| | - Jorge S Reis‐Filho
- Department of Pathology and Laboratory MedicineMemorial Sloan Kettering Cancer CenterNew YorkNYUSA
| | - Joana M Ribeiro
- Département de Médecine OncologiqueGustave RoussyVillejuifFrance
| | - David Rimm
- Department of PathologyYale University School of MedicineNew HavenCTUSA
- Department of MedicineYale University School of MedicineNew HavenCTUSA
| | - Anne Roslind
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Anne Vincent‐Salomon
- Department of Diagnostic and Theranostic Medicine, Institut CurieUniversity Paris‐Sciences et LettresParisFrance
| | - Manuel Salto‐Tellez
- Integrated Pathology UnitThe Institute of Cancer ResearchLondonUK
- Precision Medicine CentreQueen's University BelfastBelfastUK
| | - Joel Saltz
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Shahin Sayed
- Department of PathologyAga Khan UniversityNairobiKenya
| | - Ely Scott
- Translational PathologyTranslational Sciences and Diagnostics/Translational Medicine/R&D, Bristol Myers SquibbPrincetonNJUSA
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.‐C. Heuson, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
- Medical Oncology Department, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
| | - Albrecht Stenzinger
- Institute of PathologyUniversity Hospital HeidelbergHeidelbergGermany
- Centers for Personalized Medicine (ZPM)HeidelbergGermany
| | | | - Daniel Sur
- Department of Medical OncologyUniversity of Medicine and Pharmacy “Iuliu Hatieganu”Cluj‐NapocaRomania
| | - Susan Fineberg
- Montefiore Medical CenterBronxNYUSA
- Albert Einstein College of MedicineBronxNYUSA
| | - Fraser Symmans
- University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Jonas Teuwen
- AI for Oncology Lab, The Netherlands Cancer InstituteAmsterdamThe Netherlands
| | | | - Trine Tramm
- Department of PathologyAarhus University HospitalAarhusDenmark
- Institute of Clinical MedicineAarhus UniversityAarhusDenmark
| | - William T Tran
- Department of Radiation OncologyUniversity of Toronto and Sunnybrook Health Sciences CentreTorontoOntarioCanada
| | - Jeroen van der Laak
- Department of PathologyRadboud University Medical CenterNijmegenThe Netherlands
| | - Paul J van Diest
- Department of PathologyUniversity Medical Center UtrechtThe Netherlands
- Johns Hopkins Oncology CenterBaltimoreMDUSA
| | - Gregory E Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Giuseppe Viale
- Department of PathologyEuropean Institute of OncologyMilanItaly
- Department of PathologyUniversity of MilanMilanItaly
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Thomas Walter
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | | | - Hannah Y Wen
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer CenterShanghaiPR China
| | - Yinyin Yuan
- Department of Translational Molecular Pathology, Division of Pathology and Laboratory MedicineThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Reena Md Zin
- Department of Pathology, Faculty of MedicineUniversiti Kebangsaan MalaysiaKuala LumpurMalaysia
| | - Sylvia Adams
- Perlmutter Cancer CenterNYU Langone HealthNew YorkNYUSA
- Department of MedicineNYU Grossman School of MedicineManhattanNYUSA
| | | | - Sibylle Loibl
- Department of Medicine and ResearchGerman Breast GroupNeu‐IsenburgGermany
| | - Carsten Denkert
- Institut für PathologiePhilipps‐Universität Marburg und Universitätsklinikum MarburgMarburgGermany
| | - Peter Savas
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Sherene Loi
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Roberto Salgado
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Elisabeth Specht Stovgaard
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
- Department of Clinical MedicineUniversity of CopenhagenCopenhagenDenmark
| |
Collapse
|
50
|
Martos O, Hoque MZ, Keskinarkaus A, Kemi N, Näpänkangas J, Eskuri M, Pohjanen VM, Kauppila JH, Seppänen T. Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal. Pathol Res Pract 2023; 248:154694. [PMID: 37494804 DOI: 10.1016/j.prp.2023.154694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Revised: 07/03/2023] [Accepted: 07/13/2023] [Indexed: 07/28/2023]
Abstract
Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987-2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient's medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.
Collapse
Affiliation(s)
- Oleg Martos
- Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
| | - Md Ziaul Hoque
- Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland.
| | - Anja Keskinarkaus
- Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
| | - Niko Kemi
- Department of Pathology, Oulu University Hospital, Finland, and University of Oulu, Finland
| | - Juha Näpänkangas
- Department of Pathology, Oulu University Hospital, Finland, and University of Oulu, Finland
| | - Maarit Eskuri
- Department of Pathology, Oulu University Hospital, Finland, and University of Oulu, Finland
| | - Vesa-Matti Pohjanen
- Department of Pathology, Oulu University Hospital, Finland, and University of Oulu, Finland
| | - Joonas H Kauppila
- Department of Surgery, Oulu University Hospital, Finland, and University of Oulu, Finland
| | - Tapio Seppänen
- Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
| |
Collapse
|