1
|
Su W, Xu C, Hu J, Chen Q, Yang Y, Ji M, Fei Y, Ma J, Jiang H, Mi L. Non-invasive screening of bladder cancer using digital microfluidics and FLIM technology combined with deep learning. JOURNAL OF BIOPHOTONICS 2024:e202400192. [PMID: 38938144 DOI: 10.1002/jbio.202400192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2024] [Revised: 05/23/2024] [Accepted: 05/27/2024] [Indexed: 06/29/2024]
Abstract
Non-invasive screening for bladder cancer is crucial for treatment and postoperative follow-up. This study combines digital microfluidics (DMF) technology with fluorescence lifetime imaging microscopy (FLIM) for urine analysis and introduces a novel non-invasive bladder cancer screening technique. Initially, the DMF was utilized to perform preliminary screening and enrichment of urine exfoliated cells from 54 participants, followed by cell staining and FLIM analysis to assess the viscosity of the intracellular microenvironment. Subsequently, a deep learning residual convolutional neural network was employed to automatically classify FLIM images, achieving a three-class prediction of high-risk (malignant), low-risk (benign), and minimal risk (normal) categories. The results demonstrated a high consistency with pathological diagnosis, with an accuracy of 91% and a precision of 93%. Notably, the method is sensitive for both high-grade and low-grade bladder cancer cases. This highly accurate non-invasive screening method presents a promising approach for bladder cancer screening with significant clinical application potential.
Collapse
Affiliation(s)
- Wenhua Su
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
| | - Chenyang Xu
- Department of Urology, Huashan Hospital, Fudan University, Shanghai, China
| | - Jinzhong Hu
- Department of Urology, Huashan Hospital, Fudan University, Shanghai, China
| | - Qiushu Chen
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
| | - Yuwei Yang
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
| | - Mingmei Ji
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
| | - Yiyan Fei
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
| | - Jiong Ma
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
- Institute of Biomedical Engineering and Technology, Academy for Engineer and Technology, Fudan University, Shanghai, China
- Shanghai Engineering Research Center of Industrial Microorganisms, The Multiscale Research Institute of Complex Systems (MRICS), School of Life Sciences, Fudan University, Shanghai, China
| | - Haowen Jiang
- Department of Urology, Huashan Hospital, Fudan University, Shanghai, China
| | - Lan Mi
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, Shanghai, China
- Institute of Biomedical Engineering and Technology, Academy for Engineer and Technology, Fudan University, Shanghai, China
| |
Collapse
|
2
|
Jahanifar M, Shephard A, Zamanitajeddin N, Graham S, Raza SEA, Minhas F, Rajpoot N. Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures. Med Image Anal 2024; 94:103132. [PMID: 38442527 DOI: 10.1016/j.media.2024.103132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 02/28/2024] [Accepted: 03/01/2024] [Indexed: 03/07/2024]
Abstract
Counting of mitotic figures is a fundamental step in grading and prognostication of several cancers. However, manual mitosis counting is tedious and time-consuming. In addition, variation in the appearance of mitotic figures causes a high degree of discordance among pathologists. With advances in deep learning models, several automatic mitosis detection algorithms have been proposed but they are sensitive to domain shift often seen in histology images. We propose a robust and efficient two-stage mitosis detection framework, which comprises mitosis candidate segmentation (Detecting Fast) and candidate refinement (Detecting Slow) stages. The proposed candidate segmentation model, termed EUNet, is fast and accurate due to its architectural design. EUNet can precisely segment candidates at a lower resolution to considerably speed up candidate detection. Candidates are then refined using a deeper classifier network, EfficientNet-B7, in the second stage. We make sure both stages are robust against domain shift by incorporating domain generalization methods. We demonstrate state-of-the-art performance and generalizability of the proposed model on the three largest publicly available mitosis datasets, winning the two mitosis domain generalization challenge contests (MIDOG21 and MIDOG22). Finally, we showcase the utility of the proposed algorithm by processing the TCGA breast cancer cohort (1,124 whole-slide images) to generate and release a repository of more than 620K potential mitotic figures (not exhaustively validated).
Collapse
Affiliation(s)
- Mostafa Jahanifar
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK.
| | - Adam Shephard
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Neda Zamanitajeddin
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Simon Graham
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK; Histofy Ltd, Birmingham, UK
| | - Shan E Ahmed Raza
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Fayyaz Minhas
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK
| | - Nasir Rajpoot
- Tissue Image Analytic (TIA) Center, Department of Computer Science, University of Warwick, UK; Histofy Ltd, Birmingham, UK.
| |
Collapse
|
3
|
Unger M, Kather JN. Deep learning in cancer genomics and histopathology. Genome Med 2024; 16:44. [PMID: 38539231 PMCID: PMC10976780 DOI: 10.1186/s13073-024-01315-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 03/13/2024] [Indexed: 07/08/2024] Open
Abstract
Histopathology and genomic profiling are cornerstones of precision oncology and are routinely obtained for patients with cancer. Traditionally, histopathology slides are manually reviewed by highly trained pathologists. Genomic data, on the other hand, is evaluated by engineered computational pipelines. In both applications, the advent of modern artificial intelligence methods, specifically machine learning (ML) and deep learning (DL), have opened up a fundamentally new way of extracting actionable insights from raw data, which could augment and potentially replace some aspects of traditional evaluation workflows. In this review, we summarize current and emerging applications of DL in histopathology and genomics, including basic diagnostic as well as advanced prognostic tasks. Based on a growing body of evidence, we suggest that DL could be the groundwork for a new kind of workflow in oncology and cancer research. However, we also point out that DL models can have biases and other flaws that users in healthcare and research need to know about, and we propose ways to address them.
Collapse
Affiliation(s)
- Michaela Unger
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany.
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
4
|
Yadav N, Pandey S, Gupta A, Dudani P, Gupta S, Rangarajan K. Data Privacy in Healthcare: In the Era of Artificial Intelligence. Indian Dermatol Online J 2023; 14:788-792. [PMID: 38099022 PMCID: PMC10718098 DOI: 10.4103/idoj.idoj_543_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Revised: 09/01/2023] [Accepted: 09/08/2023] [Indexed: 12/17/2023] Open
Abstract
Data Privacy has increasingly become a matter of concern in the era of large public digital respositories of data. This is particularly true in healthcare where data can be misused if traced back to patients, and brings with itself a myriad of possibilities. Bring custodians of data, as well as being at the helm of disigning studies and products that can potentially benefit products, healthcare professionals often find themselves unsure about ethical and legal constraints that undelie data sharing. In this review we touch upon the concerns, leal frameworks as well as some common practices in these respects.
Collapse
Affiliation(s)
- Neel Yadav
- Department of Radiodiagnosis and Interventional Radiology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| | - Saumya Pandey
- Department of Radiodiagnosis and Interventional Radiology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| | - Amit Gupta
- Department of Radiodiagnosis and Interventional Radiology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| | - Pankhuri Dudani
- Department of Dermatology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| | - Somesh Gupta
- Department of Dermatology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| | - Krithika Rangarajan
- Department of Radiodiagnosis and Interventional Radiology, All India Institute of Medical Sciences, New Delhi, Delhi, India
| |
Collapse
|
5
|
Ariotta V, Lehtonen O, Salloum S, Micoli G, Lavikka K, Rantanen V, Hynninen J, Virtanen A, Hautaniemi S. H&E image analysis pipeline for quantifying morphological features. J Pathol Inform 2023; 14:100339. [PMID: 37915837 PMCID: PMC10616375 DOI: 10.1016/j.jpi.2023.100339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 08/15/2023] [Accepted: 09/30/2023] [Indexed: 11/03/2023] Open
Abstract
Detecting cell types from histopathological images is essential for various digital pathology applications. However, large number of cells in whole-slide images (WSIs) necessitates automated analysis pipelines for efficient cell type detection. Herein, we present hematoxylin and eosin (H&E) Image Processing pipeline (HEIP) for automatied analysis of scanned H&E-stained slides. HEIP is a flexible and modular open-source software that performs preprocessing, instance segmentation, and nuclei feature extraction. To evaluate the performance of HEIP, we applied it to extract cell types from ovarian high-grade serous carcinoma (HGSC) patient WSIs. HEIP showed high precision in instance segmentation, particularly for neoplastic and epithelial cells. We also show that there is a significant correlation between genomic ploidy values and morphological features, such as major axis of the nucleus.
Collapse
Affiliation(s)
- Valeria Ariotta
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Oskari Lehtonen
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Shams Salloum
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
- Department of Pathology, University of Helsinki and HUS Diagnostic Center, Helsinki University Hospital, 00029 Helsinki, Finland
| | - Giulia Micoli
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Kari Lavikka
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Ville Rantanen
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Johanna Hynninen
- Department of Obstetrics and Gynaecology, University of Turku and Turku University Hospital, 200521 Turku, Finland
| | - Anni Virtanen
- Department of Pathology, University of Helsinki and HUS Diagnostic Center, Helsinki University Hospital, 00029 Helsinki, Finland
| | - Sampsa Hautaniemi
- Research Program in Systems Oncology, Research Programs Unit, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| |
Collapse
|
6
|
Patel P, Harmon S, Iseman R, Ludkowski O, Auman H, Hawley S, Newcomb LF, Lin DW, Nelson PS, Feng Z, Boyer HD, Tretiakova MS, True LD, Vakar-Lopez F, Carroll PR, Cooperberg MR, Chan E, Simko J, Fazli L, Gleave M, Hurtado-Coll A, Thompson IM, Troyer D, McKenney JK, Wei W, Choyke PL, Bratslavsky G, Turkbey B, Siemens DR, Squire J, Peng YP, Brooks JD, Jamaspishvili T. Artificial Intelligence-Based PTEN Loss Assessment as an Early Predictor of Prostate Cancer Metastasis After Surgery: A Multicenter Retrospective Study. Mod Pathol 2023; 36:100241. [PMID: 37343766 PMCID: PMC10592257 DOI: 10.1016/j.modpat.2023.100241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 05/23/2023] [Accepted: 06/06/2023] [Indexed: 06/23/2023]
Abstract
Phosphatase and tensin homolog (PTEN) loss is associated with adverse outcomes in prostate cancer and can be measured via immunohistochemistry. The purpose of the study was to establish the clinical application of an in-house developed artificial intelligence (AI) image analysis workflow for automated detection of PTEN loss on digital images for identifying patients at risk of early recurrence and metastasis. Postsurgical tissue microarray sections from the Canary Foundation (n = 1264) stained with anti-PTEN antibody were evaluated independently by pathologist conventional visual scoring (cPTEN) and an automated AI-based image analysis pipeline (AI-PTEN). The relationship of PTEN evaluation methods with cancer recurrence and metastasis was analyzed using multivariable Cox proportional hazard and decision curve models. Both cPTEN scoring by the pathologist and quantification of PTEN loss by AI (high-risk AI-qPTEN) were significantly associated with shorter metastasis-free survival (MFS) in univariable analysis (cPTEN hazard ratio [HR], 1.54; CI, 1.07-2.21; P = .019; AI-qPTEN HR, 2.55; CI, 1.83-3.56; P < .001). In multivariable analyses, AI-qPTEN showed a statistically significant association with shorter MFS (HR, 2.17; CI, 1.49-3.17; P < .001) and recurrence-free survival (HR, 1.36; CI, 1.06-1.75; P = .016) when adjusting for relevant postsurgical clinical nomogram (Cancer of the Prostate Risk Assessment [CAPRA] postsurgical score [CAPRA-S]), whereas cPTEN does not show a statistically significant association (HR, 1.33; CI, 0.89-2; P = .2 and HR, 1.26; CI, 0.99-1.62; P = .063, respectively) when adjusting for CAPRA-S risk stratification. More importantly, AI-qPTEN was associated with shorter MFS in patients with favorable pathological stage and negative surgical margins (HR, 2.72; CI, 1.46-5.06; P = .002). Workflow also demonstrated enhanced clinical utility in decision curve analysis, more accurately identifying men who might benefit from adjuvant therapy postsurgery. This study demonstrates the clinical value of an affordable and fully automated AI-powered PTEN assessment for evaluating the risk of developing metastasis or disease recurrence after radical prostatectomy. Adding the AI-qPTEN assessment workflow to clinical variables may affect postoperative surveillance or management options, particularly in low-risk patients.
Collapse
Affiliation(s)
- Palak Patel
- Department of Cell Biology at The Arthur and Sonia Labatt Brain Tumour Research Centre at the Hospital for Sick Children, Toronto, Ontario, Canada
| | - Stephanie Harmon
- Molecular Imaging Branch, National Cancer Institute, Bethesda, Maryland; Artificial Intelligence Resource, National Cancer Institute, Bethesda, Maryland
| | - Rachael Iseman
- Division of Cancer Biology and Genetics, Queen's University, Kingston, Ontario, Canada
| | - Olga Ludkowski
- University Health Network, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | | | | | - Lisa F Newcomb
- Department of Urology, University of Washington Medical Center, Seattle, Washington
| | - Daniel W Lin
- Department of Urology, University of Washington Medical Center, Seattle, Washington
| | - Peter S Nelson
- Division of Human Biology, Fred Hutchinson Cancer Research Center, Seattle, Washington
| | - Ziding Feng
- Program of Biostatistics and Biomathematics, Division of Public Health Sciences, Fred Hutchinson Cancer Research Center, Seattle, Washington
| | - Hilary D Boyer
- Program of Biostatistics and Biomathematics, Division of Public Health Sciences, Fred Hutchinson Cancer Research Center, Seattle, Washington
| | - Maria S Tretiakova
- Department of Pathology, University of Washington Medical Center, Seattle, Washington
| | - Larry D True
- Department of Pathology, University of Washington Medical Center, Seattle, Washington
| | - Funda Vakar-Lopez
- Department of Pathology, University of Washington Medical Center, Seattle, Washington
| | - Peter R Carroll
- Department of Urology, University of California San Francisco and Helen Diller Family, Comprehensive Cancer Center, San Francisco, California
| | - Matthew R Cooperberg
- Department of Urology, University of California San Francisco and Helen Diller Family, Comprehensive Cancer Center, San Francisco, California
| | - Emily Chan
- Department of Urology, University of California San Francisco and Helen Diller Family, Comprehensive Cancer Center, San Francisco, California
| | - Jeff Simko
- Department of Urology, University of California San Francisco and Helen Diller Family, Comprehensive Cancer Center, San Francisco, California; Department of Pathology, University of California San Francisco, San Francisco, California
| | - Ladan Fazli
- The Vancouver Prostate Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Martin Gleave
- The Vancouver Prostate Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Antonio Hurtado-Coll
- The Vancouver Prostate Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | | | - Dean Troyer
- Department of Pathology, Eastern Virginia Medical School, Norfolk, Virginia; Department of Microbiology and Molecular Cell Biology, Eastern Virginia Medical School, Norfolk, Virginia
| | | | - Wei Wei
- Department of Pathology, Cleveland Clinic, Cleveland, Ohio
| | - Peter L Choyke
- Molecular Imaging Branch, National Cancer Institute, Bethesda, Maryland
| | | | - Baris Turkbey
- Molecular Imaging Branch, National Cancer Institute, Bethesda, Maryland; Artificial Intelligence Resource, National Cancer Institute, Bethesda, Maryland
| | - D Robert Siemens
- Department of Urology, Queen's University, Kingston, Ontario, Canada
| | - Jeremy Squire
- Department of Genetics, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Yingwei P Peng
- Department of Public Health Sciences, Queen's University, Kingston, Ontario, Canada; Department of Mathematics and Statistics, Queen's University, Kingston, Ontario, Canada
| | - James D Brooks
- Department of Urology, Stanford University Medical Center, Stanford, California
| | - Tamara Jamaspishvili
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, Ontario, Canada; Department of Pathology and Molecular Medicine, SUNY Upstate Medical University, Syracuse, New York.
| |
Collapse
|
7
|
Midya A, Chakraborty J, Srouji R, Narayan RR, Boerner T, Zheng J, Pak LM, Creasy JM, Escobar LA, Harrington KA, Gonen M, D'Angelica MI, Kingham TP, Do RKG, Jarnagin WR, Simpson AL. Computerized Diagnosis of Liver Tumors From CT Scans Using a Deep Neural Network Approach. IEEE J Biomed Health Inform 2023; 27:2456-2464. [PMID: 37027632 PMCID: PMC10245221 DOI: 10.1109/jbhi.2023.3248489] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
The liver is a frequent site of benign and malignant, primary and metastatic tumors. Hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC) are the most common primary liver cancers, and colorectal liver metastasis (CRLM) is the most common secondary liver cancer. Although the imaging characteristic of these tumors is central to optimal clinical management, it relies on imaging features that are often non-specific, overlap, and are subject to inter-observer variability. Thus, in this study, we aimed to categorize liver tumors automatically from CT scans using a deep learning approach that objectively extracts discriminating features not visible to the naked eye. Specifically, we used a modified Inception v3 network-based classification model to classify HCC, ICC, CRLM, and benign tumors from pretreatment portal venous phase computed tomography (CT) scans. Using a multi-institutional dataset of 814 patients, this method achieved an overall accuracy rate of 96%, with sensitivity rates of 96%, 94%, 99%, and 86% for HCC, ICC, CRLM, and benign tumors, respectively, using an independent dataset. These results demonstrate the feasibility of the proposed computer-assisted system as a novel non-invasive diagnostic tool to classify the most common liver tumors objectively.
Collapse
|
8
|
Chapfuwa P, Tao C, Li C, Khan I, Chandross KJ, Pencina MJ, Carin L, Henao R. Calibration and Uncertainty in Neural Time-to-Event Modeling. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:1666-1680. [PMID: 33119513 PMCID: PMC8439415 DOI: 10.1109/tnnls.2020.3029631] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Models for predicting the time of a future event are crucial for risk assessment, across a diverse range of applications. Existing time-to-event (survival) models have focused primarily on preserving pairwise ordering of estimated event times (i.e., relative risk). We propose neural time-to-event models that account for calibration and uncertainty while predicting accurate absolute event times. Specifically, an adversarial nonparametric model is introduced for estimating matched time-to-event distributions for probabilistically concentrated and accurate predictions. We also consider replacing the discriminator of the adversarial nonparametric model with a survival-function matching estimator that accounts for model calibration. The proposed estimator can be used as a means of estimating and comparing conditional survival distributions while accounting for the predictive uncertainty of probabilistic models. Extensive experiments show that the distribution matching methods outperform existing approaches in terms of both calibration and concentration of time-to-event distributions.
Collapse
|
9
|
Cieslak C, Mitteldorf C, Krömer-Olbrisch T, Kempf W, Stadler R. QuPath Analysis for CD30+ Cutaneous T-Cell Lymphoma. Am J Dermatopathol 2023; 45:93-98. [PMID: 36669072 DOI: 10.1097/dad.0000000000002330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 09/10/2022] [Indexed: 01/22/2023]
Abstract
BACKGROUND Mycosis fungoides is the most common subtype of cutaneous T-cell lymphoma, in which the expression of cluster of differentiation 30 (CD30)+ subtype can now be treated with the CD30 antibody conjugate brentuximab vedotin. Diagnostic methods are based on immunohistochemical (IHC) staining followed by manual assessment by pathologists, which is always a subjective calculation. QuPath, an open-source software for digital pathology image analysis, satisfies the requirements of objective approaches. METHODS Ten samples from mycosis fungoides patients with CD30 expression at different stages were stained for CD3 and CD30 by IHC staining, scanned, and quantitative analysis was performed using QuPath (version 2.1). Each slide was independently assessed by 3 board-certified dermatopathologists. RESULTS Individual estimates for CD30+/CD3+ cells varied among the individual histopathologists (mean coefficient of variation, 0.46; range, 0-0.78). QuPath analysis showed excellent separation between the positively stained cells for CD3 and CD30 IHC and other cells and tissue structures, although the results correlated strongly with the respective mean estimates of the 3 histopathologists (Pearson-R 0.93). CONCLUSIONS The results show a high interobserver variability evaluation of IHC markers, although quantitative image analysis offer a significant advantage for comparison. This is not only relevant for clinical routine but also especially critical in therapeutic studies addressing targeted molecules.
Collapse
Affiliation(s)
- Cassandra Cieslak
- University Clinic for Dermatology, Johannes Wesling Medical Centre, Minden, Germany
- University Hospital of Ruhr-University, Bochum, Germany
| | - Christina Mitteldorf
- Department of Dermatology, Venereology and Allergology, University Medical Center Göttingen, Göttingen, Germany; and
| | - Tanja Krömer-Olbrisch
- University Clinic for Dermatology, Johannes Wesling Medical Centre, Minden, Germany
- University Hospital of Ruhr-University, Bochum, Germany
| | - Werner Kempf
- Kempf und Pfaltz Histologische Diagnostik, Zurich, Switzerland
| | - Rudolf Stadler
- University Clinic for Dermatology, Johannes Wesling Medical Centre, Minden, Germany
- University Hospital of Ruhr-University, Bochum, Germany
| |
Collapse
|
10
|
Ayorinde JOO, Citterio F, Landrò M, Peruzzo E, Islam T, Tilley S, Taylor G, Bardsley V, Liò P, Samoshkin A, Pettigrew GJ. Artificial Intelligence You Can Trust: What Matters Beyond Performance When Applying Artificial Intelligence to Renal Histopathology? J Am Soc Nephrol 2022; 33:2133-2140. [PMID: 36351761 PMCID: PMC9731632 DOI: 10.1681/asn.2022010069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
Abstract
Although still in its infancy, artificial intelligence (AI) analysis of kidney biopsy images is anticipated to become an integral aspect of renal histopathology. As these systems are developed, the focus will understandably be on developing ever more accurate models, but successful translation to the clinic will also depend upon other characteristics of the system.In the extreme, deployment of highly performant but "black box" AI is fraught with risk, and high-profile errors could damage future trust in the technology. Furthermore, a major factor determining whether new systems are adopted in clinical settings is whether they are "trusted" by clinicians. Key to unlocking trust will be designing platforms optimized for intuitive human-AI interactions and ensuring that, where judgment is required to resolve ambiguous areas of assessment, the workings of the AI image classifier are understandable to the human observer. Therefore, determining the optimal design for AI systems depends on factors beyond performance, with considerations of goals, interpretability, and safety constraining many design and engineering choices.In this article, we explore challenges that arise in the application of AI to renal histopathology, and consider areas where choices around model architecture, training strategy, and workflow design may be influenced by factors beyond the final performance metrics of the system.
Collapse
Affiliation(s)
- John O O Ayorinde
- Department of Surgery, University of Cambridge, Addenbrooke's Hospital, Cambridge, United Kingdom
| | | | | | | | | | | | | | - Victoria Bardsley
- Department of Histopathology, Addenbrooke's Hospital, Cambridge, United Kingdom
| | - Pietro Liò
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Alex Samoshkin
- Office for Translational Research, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Gavin J Pettigrew
- Department of Surgery, University of Cambridge, Addenbrooke's Hospital, Cambridge, United Kingdom
| |
Collapse
|
11
|
Mahbod A, Schaefer G, Dorffner G, Hatamikia S, Ecker R, Ellinger I. A dual decoder U-Net-based model for nuclei instance segmentation in hematoxylin and eosin-stained histological images. Front Med (Lausanne) 2022; 9:978146. [PMID: 36438040 PMCID: PMC9691672 DOI: 10.3389/fmed.2022.978146] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Accepted: 10/28/2022] [Indexed: 11/03/2023] Open
Abstract
Even in the era of precision medicine, with various molecular tests based on omics technologies available to improve the diagnosis process, microscopic analysis of images derived from stained tissue sections remains crucial for diagnostic and treatment decisions. Among other cellular features, both nuclei number and shape provide essential diagnostic information. With the advent of digital pathology and emerging computerized methods to analyze the digitized images, nuclei detection, their instance segmentation and classification can be performed automatically. These computerized methods support human experts and allow for faster and more objective image analysis. While methods ranging from conventional image processing techniques to machine learning-based algorithms have been proposed, supervised convolutional neural network (CNN)-based techniques have delivered the best results. In this paper, we propose a CNN-based dual decoder U-Net-based model to perform nuclei instance segmentation in hematoxylin and eosin (H&E)-stained histological images. While the encoder path of the model is developed to perform standard feature extraction, the two decoder heads are designed to predict the foreground and distance maps of all nuclei. The outputs of the two decoder branches are then merged through a watershed algorithm, followed by post-processing refinements to generate the final instance segmentation results. Moreover, to additionally perform nuclei classification, we develop an independent U-Net-based model to classify the nuclei predicted by the dual decoder model. When applied to three publicly available datasets, our method achieves excellent segmentation performance, leading to average panoptic quality values of 50.8%, 51.3%, and 62.1% for the CryoNuSeg, NuInsSeg, and MoNuSAC datasets, respectively. Moreover, our model is the top-ranked method in the MoNuSAC post-challenge leaderboard.
Collapse
Affiliation(s)
- Amirreza Mahbod
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, Austria
- Research Center for Medical Image Analysis and Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, Austria
| | - Gerald Schaefer
- Department of Computer Science, Loughborough University, Loughborough, United Kingdom
| | - Georg Dorffner
- Institute of Artificial Intelligence, Medical University of Vienna, Vienna, Austria
| | - Sepideh Hatamikia
- Research Center for Medical Image Analysis and Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, Austria
- Austrian Center for Medical Innovation and Technology, Wiener Neustadt, Austria
| | - Rupert Ecker
- Department of Research and Development, TissueGnostics GmbH, Vienna, Austria
| | - Isabella Ellinger
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
12
|
K-RAS Associated Gene-Mutation-Based Algorithm for Prediction of Treatment Response of Patients with Subtypes of Breast Cancer and Especially Triple-Negative Cancer. Cancers (Basel) 2022; 14:cancers14215322. [PMID: 36358741 PMCID: PMC9657686 DOI: 10.3390/cancers14215322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2022] [Revised: 10/14/2022] [Accepted: 10/25/2022] [Indexed: 11/16/2022] Open
Abstract
Purpose: There is an urgent need for developing new biomarker tools to accurately predict treatment response of breast cancer, especially the deadly triple-negative breast cancer. We aimed to develop gene-mutation-based machine learning (ML) algorithms as biomarker classifiers to predict treatment response of first-line chemotherapy with high precision. Methods: Random Forest ML was applied to screen the algorithms of various combinations of gene mutation profiles of primary tumors at diagnosis using a TCGA Cohort (n = 399) with up to 150 months follow-up as a training set and validated in a MSK Cohort (n = 807) with up to 220 months follow-up. Subtypes of breast cancer including triple-negative and luminal A (ER+, PR+ and HER2−) were also assessed. The predictive performance of the candidate algorithms as classifiers was further assessed using logistic regression, Kaplan−Meier progression-free survival (PFS) plot, and univariate/multivariate Cox proportional hazard regression analyses. Results: A novel algorithm termed the 12-Gene Algorithm based on mutation profiles of KRAS, PIK3CA, MAP3K1, MAP2K4, PTEN, TP53, CDH1, GATA3, KMT2C, ARID1A, RunX1, and ESR1, was identified. The performance of this algorithm to distinguish non-progressed (responder) vs. progressed (non-responder) to treatment in the TCGA Cohort as determined using AUC was 0.96 (95% CI 0.94−0.98). It predicted progression-free survival (PFS) with hazard ratio (HR) of 21.6 (95% CI 11.3−41.5) (p < 0.0001) in all patients. The algorithm predicted PFS in the triple-negative subgroup with HR of 19.3 (95% CI 3.7−101.3) (n = 42, p = 0.000). The 12-Gene Algorithm was validated in the MSK Cohort with a similar AUC of 0.97 (95% CI 0.96−0.98) to distinguish responder vs. non-responder patients, and had a HR of 18.6 (95% CI 4.4−79.2) to predict PFS in the triple-negative subgroup (n = 75, p < 0.0001). Conclusions: The novel 12-Gene algorithm based on multitude gene-mutation profiles identified through ML has a potential to predict breast cancer treatment response to therapies, especially in triple-negative subgroups patients, which may assist personalized therapies and reduce mortality.
Collapse
|
13
|
Rączkowska A, Paśnik I, Kukiełka M, Nicoś M, Budzinska MA, Kucharczyk T, Szumiło J, Krawczyk P, Crosetto N, Szczurek E. Deep learning-based tumor microenvironment segmentation is predictive of tumor mutations and patient survival in non-small-cell lung cancer. BMC Cancer 2022; 22:1001. [PMID: 36131239 PMCID: PMC9490924 DOI: 10.1186/s12885-022-10081-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Accepted: 09/07/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Despite the fact that tumor microenvironment (TME) and gene mutations are the main determinants of progression of the deadliest cancer in the world - lung cancer, their interrelations are not well understood. Digital pathology data provides a unique insight into the spatial composition of the TME. Various spatial metrics and machine learning approaches were proposed for prediction of either patient survival or gene mutations from this data. Still, these approaches are limited in the scope of analyzed features and in their explainability, and as such fail to transfer to clinical practice. METHODS Here, we generated 23,199 image patches from 26 hematoxylin-and-eosin (H&E)-stained lung cancer tissue sections and annotated them into 9 different tissue classes. Using this dataset, we trained a deep neural network ARA-CNN. Next, we applied the trained network to segment 467 lung cancer H&E images from The Cancer Genome Atlas (TCGA) database. We used the segmented images to compute human-interpretable features reflecting the heterogeneous composition of the TME, and successfully utilized them to predict patient survival and cancer gene mutations. RESULTS We achieved per-class AUC ranging from 0.72 to 0.99 for classifying tissue types in lung cancer with ARA-CNN. Machine learning models trained on the proposed human-interpretable features achieved a c-index of 0.723 in the task of survival prediction and AUC up to 73.5% for PDGFRB in the task of mutation classification. CONCLUSIONS We presented a framework that accurately predicted survival and gene mutations in lung adenocarcinoma patients based on human-interpretable features extracted from H&E slides. Our approach can provide important insights for designing novel cancer treatments, by linking the spatial structure of the TME in lung adenocarcinoma to gene mutations and patient survival. It can also expand our understanding of the effects that the TME has on tumor evolutionary processes. Our approach can be generalized to different cancer types to inform precision medicine strategies.
Collapse
Affiliation(s)
- Alicja Rączkowska
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Banacha 2, 02-097 Warsaw, Poland
| | - Iwona Paśnik
- Department of Clinical Pathomorphology, Medical University of Lublin, Jaczewskiego 8b, 20-090 Lublin, Poland
| | - Michał Kukiełka
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Banacha 2, 02-097 Warsaw, Poland
| | - Marcin Nicoś
- Department of Pneumology, Oncology and Allergology, Medical University of Lublin, Jaczewskiego 8, 20-090 Lublin, Poland
| | | | - Tomasz Kucharczyk
- Department of Pneumology, Oncology and Allergology, Medical University of Lublin, Jaczewskiego 8, 20-090 Lublin, Poland
| | - Justyna Szumiło
- Department of Clinical Pathomorphology, Medical University of Lublin, Jaczewskiego 8b, 20-090 Lublin, Poland
| | - Paweł Krawczyk
- Department of Pneumology, Oncology and Allergology, Medical University of Lublin, Jaczewskiego 8, 20-090 Lublin, Poland
| | - Nicola Crosetto
- Division of Genome Biology, Department of Medical Biochemistry and Biophysics, Karolinska Institutet, Tomtebodavägen 23a, 17165 Solna, Sweden
- Science for Life Laboratory, Tomtebodavägen 23a, 17165 Solna, Sweden
| | - Ewa Szczurek
- Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Banacha 2, 02-097 Warsaw, Poland
| |
Collapse
|
14
|
Lee SH, Lee Y, Jang H. Deep learning captures selective features for discrimination of microsatellite instability from pathologic tissue slides of gastric cancer. Int J Cancer 2022; 152:298-307. [DOI: 10.1002/ijc.34251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Revised: 08/12/2022] [Accepted: 08/15/2022] [Indexed: 12/24/2022]
Affiliation(s)
- Sung Hak Lee
- Department of Hospital Pathology Seoul St. Mary's Hospital
| | - Yujin Lee
- Department of Hospital Pathology St. Vincent's Hospital
| | - Hyun‐Jong Jang
- Catholic Big Data Integration Center, Department of Physiology, College of Medicine The Catholic University of Korea Seoul South Korea
| |
Collapse
|
15
|
Daniel N, Larey A, Aknin E, Osswald GA, Caldwell JM, Rochman M, Collins MH, Yang GY, Arva NC, Capocelli KE, Rothenberg ME, Savir Y. A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3211-3217. [PMID: 36085661 PMCID: PMC9552249 DOI: 10.1109/embc48229.2022.9871086] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.
Collapse
Affiliation(s)
- Nati Daniel
- Faculty of Medicine,Dept. of Physiology, Biophysics and System Biology,Technion,Israel
| | | | - Eliel Aknin
- Faculty of Industrial Engineering,Technion,Israel
| | - Garrett A. Osswald
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Julie M. Caldwell
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Mark Rochman
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Margaret H. Collins
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Dept. of Pathology,Cincinnati,OH,USA
| | - Guang-Yu Yang
- Ann & Robert H. Lurie Children's Hospital of Chicago, Northwestern University, The Feinberg School of Medicine,Dept. of Pathology,Chicago,IL,USA
| | - Nicoleta C. Arva
- Ann & Robert H. Lurie Children's Hospital of Chicago, Northwestern University, The Feinberg School of Medicine,Dept. of Pathology,Chicago,IL,USA
| | | | - Marc E. Rothenberg
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Yonatan Savir
- Faculty of Medicine,Dept. of Physiology, Biophysics and System Biology,Technion,Israel
| |
Collapse
|
16
|
Recent Applications of Artificial Intelligence from Histopathologic Image-Based Prediction of Microsatellite Instability in Solid Cancers: A Systematic Review. Cancers (Basel) 2022; 14:cancers14112590. [PMID: 35681570 PMCID: PMC9179592 DOI: 10.3390/cancers14112590] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 05/07/2022] [Accepted: 05/22/2022] [Indexed: 12/11/2022] Open
Abstract
Cancers with high microsatellite instability (MSI-H) have a better prognosis and respond well to immunotherapy. However, MSI is not tested in all cancers because of the additional costs and time of diagnosis. Therefore, artificial intelligence (AI)-based models have been recently developed to evaluate MSI from whole slide images (WSIs). Here, we aimed to assess the current state of AI application to predict MSI based on WSIs analysis in MSI-related cancers and suggest a better study design for future studies. Studies were searched in online databases and screened by reference type, and only the full texts of eligible studies were reviewed. The included 14 studies were published between 2018 and 2021, and most of the publications were from developed countries. The commonly used dataset is The Cancer Genome Atlas dataset. Colorectal cancer (CRC) was the most common type of cancer studied, followed by endometrial, gastric, and ovarian cancers. The AI models have shown the potential to predict MSI with the highest AUC of 0.93 in the case of CRC. The relatively limited scale of datasets and lack of external validation were the limitations of most studies. Future studies with larger datasets are required to implicate AI models in routine diagnostic practice for MSI prediction.
Collapse
|
17
|
Dent A, Diamandis P. Integrating computational pathology and proteomics to address tumor heterogeneity. J Pathol 2022; 257:445-453. [DOI: 10.1002/path.5905] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 03/20/2022] [Accepted: 03/30/2022] [Indexed: 11/11/2022]
Affiliation(s)
- Anglin Dent
- Department of Laboratory Medicine and Pathobiology University of Toronto Toronto Ontario M5S 1A8 Canada
- Princess Margaret Cancer Center University Health Network, Toronto, Ontario, 610 University Avenue, M5G 2C1 Canada
| | - Phedias Diamandis
- Department of Laboratory Medicine and Pathobiology University of Toronto Toronto Ontario M5S 1A8 Canada
- Princess Margaret Cancer Center University Health Network, Toronto, Ontario, 610 University Avenue, M5G 2C1 Canada
- Laboratory Medicine Program University Health Network, 200 Elizabeth Street, Toronto, ON Toronto Ontario M5G 2C4 Canada
- Department of Medical Biophysics University of Toronto Toronto Ontario M5S 1A8 Canada
| |
Collapse
|
18
|
Abstract
This overview of the molecular pathology of lung cancer includes a review of the most salient molecular alterations of the genome, transcriptome, and the epigenome. The insights provided by the growing use of next-generation sequencing (NGS) in lung cancer will be discussed, and interrelated concepts such as intertumor heterogeneity, intratumor heterogeneity, tumor mutational burden, and the advent of liquid biopsy will be explored. Moreover, this work describes how the evolving field of molecular pathology refines the understanding of different histologic phenotypes of non-small-cell lung cancer (NSCLC) and the underlying biology of small-cell lung cancer. This review will provide an appreciation for how ongoing scientific findings and technologic advances in molecular pathology are crucial for development of biomarkers, therapeutic agents, clinical trials, and ultimately improved patient care.
Collapse
Affiliation(s)
- James J Saller
- Departments of Pathology and Thoracic Oncology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida 33612, USA
| | - Theresa A Boyle
- Departments of Pathology and Thoracic Oncology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida 33612, USA
| |
Collapse
|
19
|
Artificial Intelligence for Predicting Microsatellite Instability Based on Tumor Histomorphology: A Systematic Review. Int J Mol Sci 2022; 23:ijms23052462. [PMID: 35269607 PMCID: PMC8910565 DOI: 10.3390/ijms23052462] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Accepted: 02/21/2022] [Indexed: 02/04/2023] Open
Abstract
Microsatellite instability (MSI)/defective DNA mismatch repair (dMMR) is receiving more attention as a biomarker for eligibility for immune checkpoint inhibitors in advanced diseases. However, due to high costs and resource limitations, MSI/dMMR testing is not widely performed. Some attempts are in progress to predict MSI/dMMR status through histomorphological features on H&E slides using artificial intelligence (AI) technology. In this study, the potential predictive role of this new methodology was reviewed through a systematic review. Studies up to September 2021 were searched through PubMed and Embase database searches. The design and results of each study were summarized, and the risk of bias for each study was evaluated. For colorectal cancer, AI-based systems showed excellent performance with the highest standard of 0.972; for gastric and endometrial cancers they showed a relatively low but satisfactory performance, with the highest standard of 0.81 and 0.82, respectively. However, analyzing the risk of bias, most studies were evaluated at high-risk. AI-based systems showed a high potential in predicting the MSI/dMMR status of different cancer types, and particularly of colorectal cancers. Therefore, a confirmation test should be required only for the results that are positive in the AI test.
Collapse
|
20
|
Roetzer-Pejrimovsky T, Moser AC, Atli B, Vogel CC, Mercea PA, Prihoda R, Gelpi E, Haberler C, Höftberger R, Hainfellner JA, Baumann B, Langs G, Woehrer A. The Digital Brain Tumour Atlas, an open histopathology resource. Sci Data 2022; 9:55. [PMID: 35169150 PMCID: PMC8847577 DOI: 10.1038/s41597-022-01157-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Accepted: 12/22/2021] [Indexed: 11/28/2022] Open
Abstract
Currently, approximately 150 different brain tumour types are defined by the WHO. Recent endeavours to exploit machine learning and deep learning methods for supporting more precise diagnostics based on the histological tumour appearance have been hampered by the relative paucity of accessible digital histopathological datasets. While freely available datasets are relatively common in many medical specialties such as radiology and genomic medicine, there is still an unmet need regarding histopathological data. Thus, we digitized a significant portion of a large dedicated brain tumour bank based at the Division of Neuropathology and Neurochemistry of the Medical University of Vienna, covering brain tumour cases from 1995-2019. A total of 3,115 slides of 126 brain tumour types (including 47 control tissue slides) have been scanned. Additionally, complementary clinical annotations have been collected for each case. In the present manuscript, we thoroughly discuss this unique dataset and make it publicly available for potential use cases in machine learning and digital image analysis, teaching and as a reference for external validation.
Collapse
Affiliation(s)
- Thomas Roetzer-Pejrimovsky
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria.
| | - Anna-Christina Moser
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Baran Atli
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Clemens Christian Vogel
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Petra A Mercea
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
- Department of Neurosurgery, Medical University of Vienna, Vienna, Austria
| | - Romana Prihoda
- Department of Neurosurgery, Medical University of Vienna, Vienna, Austria
- Department of Neurosurgery, University Hospital St. Poelten, St. Poelten, Austria
| | - Ellen Gelpi
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Christine Haberler
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Romana Höftberger
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Johannes A Hainfellner
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| | - Bernhard Baumann
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Georg Langs
- Department of Biomedical Imaging and Image-Guided Therapy, Computational Imaging Research Lab, Medical University of Vienna, Vienna, Austria
| | - Adelheid Woehrer
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
21
|
Teranikar T, Lim J, Ijaseun T, Lee J. Development of Planar Illumination Strategies for Solving Mysteries in the Sub-Cellular Realm. Int J Mol Sci 2022; 23:ijms23031643. [PMID: 35163562 PMCID: PMC8835835 DOI: 10.3390/ijms23031643] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 12/22/2021] [Accepted: 01/25/2022] [Indexed: 02/04/2023] Open
Abstract
Optical microscopy has vastly expanded the frontiers of structural and functional biology, due to the non-invasive probing of dynamic volumes in vivo. However, traditional widefield microscopy illuminating the entire field of view (FOV) is adversely affected by out-of-focus light scatter. Consequently, standard upright or inverted microscopes are inept in sampling diffraction-limited volumes smaller than the optical system's point spread function (PSF). Over the last few decades, several planar and structured (sinusoidal) illumination modalities have offered unprecedented access to sub-cellular organelles and 4D (3D + time) image acquisition. Furthermore, these optical sectioning systems remain unaffected by the size of biological samples, providing high signal-to-noise (SNR) ratios for objective lenses (OLs) with long working distances (WDs). This review aims to guide biologists regarding planar illumination strategies, capable of harnessing sub-micron spatial resolution with a millimeter depth of penetration.
Collapse
Affiliation(s)
| | | | | | - Juhyun Lee
- Correspondence: ; Tel.: +1-817-272-6534; Fax: +1-817-272-2251
| |
Collapse
|
22
|
Pettersen HS, Belevich I, Røyset ES, Smistad E, Simpson MR, Jokitalo E, Reinertsen I, Bakke I, Pedersen A. Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology. Front Med (Lausanne) 2022; 8:816281. [PMID: 35155486 PMCID: PMC8829033 DOI: 10.3389/fmed.2021.816281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Accepted: 12/24/2021] [Indexed: 11/13/2022] Open
Abstract
Application of deep learning on histopathological whole slide images (WSIs) holds promise of improving diagnostic efficiency and reproducibility but is largely dependent on the ability to write computer code or purchase commercial solutions. We present a code-free pipeline utilizing free-to-use, open-source software (QuPath, DeepMIB, and FastPathology) for creating and deploying deep learning-based segmentation models for computational pathology. We demonstrate the pipeline on a use case of separating epithelium from stroma in colonic mucosa. A dataset of 251 annotated WSIs, comprising 140 hematoxylin-eosin (HE)-stained and 111 CD3 immunostained colon biopsy WSIs, were developed through active learning using the pipeline. On a hold-out test set of 36 HE and 21 CD3-stained WSIs a mean intersection over union score of 95.5 and 95.3% was achieved on epithelium segmentation. We demonstrate pathologist-level segmentation accuracy and clinical acceptable runtime performance and show that pathologists without programming experience can create near state-of-the-art segmentation solutions for histopathological WSIs using only free-to-use software. The study further demonstrates the strength of open-source solutions in its ability to create generalizable, open pipelines, of which trained models and predictions can seamlessly be exported in open formats and thereby used in external solutions. All scripts, trained models, a video tutorial, and the full dataset of 251 WSIs with ~31 k epithelium annotations are made openly available at https://github.com/andreped/NoCodeSeg to accelerate research in the field.
Collapse
Affiliation(s)
- Henrik Sahlin Pettersen
- Department of Pathology, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- Department of Clinical and Molecular Medicine, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
- Clinic of Laboratory Medicine, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Ilya Belevich
- Electron Microscopy Unit, Institute of Biotechnology, Helsinki Institute of Life Science, University of Helsinki, Helsinki, Finland
| | - Elin Synnøve Røyset
- Department of Pathology, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
- Department of Clinical and Molecular Medicine, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
- Clinic of Laboratory Medicine, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Erik Smistad
- Department of Health Research, SINTEF Digital, Trondheim, Norway
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
| | - Melanie Rae Simpson
- Department of Public Health and Nursing, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
- The Clinical Research Unit for Central Norway, Trondheim, Norway
| | - Eija Jokitalo
- Electron Microscopy Unit, Institute of Biotechnology, Helsinki Institute of Life Science, University of Helsinki, Helsinki, Finland
| | - Ingerid Reinertsen
- Department of Health Research, SINTEF Digital, Trondheim, Norway
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
| | - Ingunn Bakke
- Department of Clinical and Molecular Medicine, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
- Clinic of Laboratory Medicine, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - André Pedersen
- Department of Clinical and Molecular Medicine, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway
- Department of Health Research, SINTEF Digital, Trondheim, Norway
- The Cancer Foundation, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| |
Collapse
|
23
|
Larey A, Aknin E, Daniel N, Osswald GA, Caldwell JM, Rochman M, Wasserman T, Collins MH, Arva NC, Yang GY, Rothenberg ME, Savir Y. Harnessing artificial intelligence to infer novel spatial biomarkers for the diagnosis of eosinophilic esophagitis. Front Med (Lausanne) 2022; 9:950728. [PMID: 36341260 PMCID: PMC9633847 DOI: 10.3389/fmed.2022.950728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 09/29/2022] [Indexed: 11/13/2022] Open
Abstract
Eosinophilic esophagitis (EoE) is a chronic allergic inflammatory condition of the esophagus associated with elevated esophageal eosinophils. Second only to gastroesophageal reflux disease, EoE is one of the leading causes of chronic refractory dysphagia in adults and children. EoE is a clinicopathologic disorder and the histological portion of the diagnosis requires enumerating the density of esophageal eosinophils in esophageal biopsies, and evaluating additional features such as basal zone hyperplasia is helpful. However, this task requires time-consuming, somewhat subjective manual analysis, thus reducing the ability to process the complex tissue structure and infer its relationship with the patient's clinical status. Previous artificial intelligence (AI) approaches that aimed to improve histology-based diagnosis focused on recapitulating identification and quantification of the area of maximal eosinophil density, the gold standard manual metric for determining EoE disease activity. However, this metric does not account for the distribution of eosinophils or other histological features, over the whole slide image. Here, we developed an artificial intelligence platform that infers local and spatial biomarkers based on semantic segmentation of intact eosinophils and basal zone distributions. Besides the maximal density of eosinophils [referred to as Peak Eosinophil Count (PEC)] and a maximal basal zone fraction, we identify the value of two additional metrics that reflect the distribution of eosinophils and basal zone fractions. This approach enables a decision support system that predicts EoE activity and potentially classifies the histological severity of EoE patients. We utilized a cohort that includes 1,066 biopsy slides from 400 subjects to validate the system's performance and achieved a histological severity classification accuracy of 86.70%, sensitivity of 84.50%, and specificity of 90.09%. Our approach highlights the importance of systematically analyzing the distribution of biopsy features over the entire slide and paves the way toward a personalized decision support system that will assist not only in counting cells but can also potentially improve diagnosis and provide treatment prediction.
Collapse
Affiliation(s)
- Ariel Larey
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel.,Faculty of Computer Science, Technion Israel Institute of Technology, Haifa, Israel
| | - Eliel Aknin
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel.,Faculty of Industrial Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Nati Daniel
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| | - Garrett A Osswald
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Julie M Caldwell
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Mark Rochman
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Tanya Wasserman
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| | - Margaret H Collins
- Division of Pathology, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Nicoleta C Arva
- Department of Pathology and Laboratory Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
| | - Guang-Yu Yang
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Marc E Rothenberg
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Yonatan Savir
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| |
Collapse
|
24
|
Mehrvar S, Himmel LE, Babburi P, Goldberg AL, Guffroy M, Janardhan K, Krempley AL, Bawa B. Deep Learning Approaches and Applications in Toxicologic Histopathology: Current Status and Future Perspectives. J Pathol Inform 2021; 12:42. [PMID: 34881097 PMCID: PMC8609289 DOI: 10.4103/jpi.jpi_36_21] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 07/18/2021] [Indexed: 12/13/2022] Open
Abstract
Whole slide imaging enables the use of a wide array of digital image analysis tools that are revolutionizing pathology. Recent advances in digital pathology and deep convolutional neural networks have created an enormous opportunity to improve workflow efficiency, provide more quantitative, objective, and consistent assessments of pathology datasets, and develop decision support systems. Such innovations are already making their way into clinical practice. However, the progress of machine learning - in particular, deep learning (DL) - has been rather slower in nonclinical toxicology studies. Histopathology data from toxicology studies are critical during the drug development process that is required by regulatory bodies to assess drug-related toxicity in laboratory animals and its impact on human safety in clinical trials. Due to the high volume of slides routinely evaluated, low-throughput, or narrowly performing DL methods that may work well in small-scale diagnostic studies or for the identification of a single abnormality are tedious and impractical for toxicologic pathology. Furthermore, regulatory requirements around good laboratory practice are a major hurdle for the adoption of DL in toxicologic pathology. This paper reviews the major DL concepts, emerging applications, and examples of DL in toxicologic pathology image analysis. We end with a discussion of specific challenges and directions for future research.
Collapse
Affiliation(s)
- Shima Mehrvar
- Preclinical Safety, AbbVie Inc., North Chicago, IL, USA
| | | | - Pradeep Babburi
- Business Technology Solutions, AbbVie Inc., North Chicago, IL, USA
| | | | | | | | | | | |
Collapse
|
25
|
Foundations of Machine Learning-Based Clinical Prediction Modeling: Part V-A Practical Approach to Regression Problems. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:43-50. [PMID: 34862526 DOI: 10.1007/978-3-030-85292-4_6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
This chapter goes through the steps required to train and validate a simple, machine learning-based clinical prediction model for any continuous outcome. We supply fully structured code for the readers to download and execute in parallel to this section, as well as a simulated database of 10,000 glioblastoma patients who underwent microsurgery, and predict survival from diagnosis in months. We walk the reader through each step, including import, checking, splitting of data. In terms of pre-processing, we focus on how to practically implement imputation using a k-nearest neighbor algorithm. We also illustrate how to select features based on recursive feature elimination and how to use k-fold cross validation. We demonstrate a generalized linear model, a generalized additive model, a random forest, a ridge regressor, and a Least Absolute Shrinkage and Selection Operator (LASSO) regressor. Specifically for regression, we discuss how to evaluate root mean square error (RMSE), mean average error (MAE), and the R2 statistic, as well as how a quantile-quantile plot can be used to assess the performance of the regressor along the spectrum of the outcome variable, similarly to calibration when dealing with binary outcomes. Finally, we explain how to arrive at a measure of variable importance using a universal, nonparametric method.
Collapse
|
26
|
Foundations of Machine Learning-Based Clinical Prediction Modeling: Part IV-A Practical Approach to Binary Classification Problems. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:33-41. [PMID: 34862525 DOI: 10.1007/978-3-030-85292-4_5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
We illustrate the steps required to train and validate a simple, machine learning-based clinical prediction model for any binary outcome, such as, for example, the occurrence of a complication, in the statistical programming language R. To illustrate the methods applied, we supply a simulated database of 10,000 glioblastoma patients who underwent microsurgery, and predict the occurrence of 12-month survival. We walk the reader through each step, including import, checking, and splitting of datasets. In terms of pre-processing, we focus on how to practically implement imputation using a k-nearest neighbor algorithm, and how to perform feature selection using recursive feature elimination. When it comes to training models, we apply the theory discussed in Parts I-III. We show how to implement bootstrapping and to evaluate and select models based on out-of-sample error. Specifically for classification, we discuss how to counteract class imbalance by using upsampling techniques. We discuss how the reporting of a minimum of accuracy, area under the curve (AUC), sensitivity, and specificity for discrimination, as well as slope and intercept for calibration-if possible alongside a calibration plot-is paramount. Finally, we explain how to arrive at a measure of variable importance using a universal, AUC-based method. We provide the full, structured code, as well as the complete glioblastoma survival database for the readers to download and execute in parallel to this section.
Collapse
|
27
|
The Artificial Intelligence Doctor: Considerations for the Clinical Implementation of Ethical AI. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:257-261. [PMID: 34862549 DOI: 10.1007/978-3-030-85292-4_29] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
The applications of artificial intelligence (AI) and machine learning (ML) in modern medicine are growing exponentially, and new developments are fast-paced. However, the lack of trust and appropriate legislation hinder its clinical implementation. Recently, there is a clear increase of directives and considerations on Ethical AI. However, most literature broadly deals with ethical tensions on a meta-level without offering hands-on advice in practice. In this article, we non-exhaustively cover basic practical guidelines regarding AI-specific ethical aspects, including transparency and explicability, equity and mitigation of biases, and lastly, liability.
Collapse
|
28
|
Stumpo V, Kernbach JM, van Niftrik CHB, Sebök M, Fierstra J, Regli L, Serra C, Staartjes VE. Machine Learning Algorithms in Neuroimaging: An Overview. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:125-138. [PMID: 34862537 DOI: 10.1007/978-3-030-85292-4_17] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Machine learning (ML) and artificial intelligence (AI) applications in the field of neuroimaging have been on the rise in recent years, and their clinical adoption is increasing worldwide. Deep learning (DL) is a field of ML that can be defined as a set of algorithms enabling a computer to be fed with raw data and progressively discover-through multiple layers of representation-more complex and abstract patterns in large data sets. The combination of ML and radiomics, namely the extraction of features from medical images, has proven valuable, too: Radiomic information can be used for enhanced image characterization and prognosis or outcome prediction. This chapter summarizes the basic concepts underlying ML application for neuroimaging and discusses technical aspects of the most promising algorithms, with a specific focus on Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), in order to provide the readership with the fundamental theoretical tools to better understand ML in neuroimaging. Applications are highlighted from a practical standpoint in the last section of the chapter, including: image reconstruction and restoration, image synthesis and super-resolution, registration, segmentation, classification, and outcome prediction.
Collapse
Affiliation(s)
- Vittorio Stumpo
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Julius M Kernbach
- Neurosurgical Artificial Intelligence Lab Aachen (NAILA), Department of Neurosurgery, RWTH University Hospital, Aachen, Germany
- Department of Neurosurgery, Faculty of Medicine, RWTH Aachen University, Aachen, Germany
| | - Christiaan H B van Niftrik
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Martina Sebök
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Jorn Fierstra
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Luca Regli
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Carlo Serra
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Victor E Staartjes
- Machine Intelligence in Clinical Neuroscience (MICN) Lab, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland.
| |
Collapse
|
29
|
Jang HJ, Lee A, Kang J, Song IH, Lee SH. Prediction of genetic alterations from gastric cancer histopathology images using a fully automated deep learning approach. World J Gastroenterol 2021; 27:7687-7704. [PMID: 34908807 PMCID: PMC8641056 DOI: 10.3748/wjg.v27.i44.7687] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 09/05/2021] [Accepted: 11/13/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Studies correlating specific genetic mutations and treatment response are ongoing to establish an effective treatment strategy for gastric cancer (GC). To facilitate this research, a cost- and time-effective method to analyze the mutational status is necessary. Deep learning (DL) has been successfully applied to analyze hematoxylin and eosin (H and E)-stained tissue slide images. AIM To test the feasibility of DL-based classifiers for the frequently occurring mutations from the H and E-stained GC tissue whole slide images (WSIs). METHODS From the GC dataset of The Cancer Genome Atlas (TCGA-STAD), wild-type/mutation classifiers for CDH1, ERBB2, KRAS, PIK3CA, and TP53 genes were trained on 360 × 360-pixel patches of tissue images. RESULTS The area under the curve (AUC) for the receiver operating characteristic (ROC) curves ranged from 0.727 to 0.862 for the TCGA frozen WSIs and 0.661 to 0.858 for the TCGA formalin-fixed paraffin-embedded (FFPE) WSIs. The performance of the classifier can be improved by adding new FFPE WSI training dataset from our institute. The classifiers trained for mutation prediction in colorectal cancer completely failed to predict the mutational status in GC, indicating that DL-based mutation classifiers are incompatible between different cancers. CONCLUSION This study concluded that DL could predict genetic mutations in H and E-stained tissue slides when they are trained with appropriate tissue data.
Collapse
Affiliation(s)
- Hyun-Jong Jang
- Catholic Big Data Integration Center, Department of Physiology, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - Ahwon Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - Jun Kang
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - In Hye Song
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - Sung Hak Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| |
Collapse
|
30
|
Balluet M, Sizaire F, El Habouz Y, Walter T, Pont J, Giroux B, Bouchareb O, Tramier M, Pecreaux J. Neural network fast-classifies biological images through features selecting to power automated microscopy. J Microsc 2021; 285:3-19. [PMID: 34623634 DOI: 10.1111/jmi.13062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 09/28/2021] [Indexed: 11/26/2022]
Abstract
Artificial intelligence is nowadays used for cell detection and classification in optical microscopy during post-acquisition analysis. The microscopes are now fully automated and next expected to be smart by making acquisition decisions based on the images. It calls for analysing them on the fly. Biology further imposes training on a reduced data set due to cost and time to prepare the samples and have the data sets annotated by experts. We propose a real-time image processing compliant with these specifications by balancing accurate detection and execution performance. We characterised the images using a generic, high-dimensional feature extractor. We then classified the images using machine learning to understand the contribution of each feature in decision and execution time. We found that the non-linear-classifier random forests outperformed Fisher's linear discriminant. More importantly, the most discriminant and time-consuming features could be excluded without significant accuracy loss, offering a substantial gain in execution time. It suggests a feature-group redundancy likely related to the biology of the observed cells. We offer a method to select fast and discriminant features. In our assay, a 79.6 ± 2.4% accurate classification of a cell took 68.7 ± 3.5 ms (mean ± SD, 5-fold cross-validation nested in 10 bootstrap repeats), corresponding to 14 cells per second, dispatched into eight phases of the cell cycle, using 12 feature groups and operating a consumer market ARM-based embedded system. A simple neural network offered similar performances paving the way to faster training and classification, using parallel execution on a general-purpose graphic processing unit. Finally, this strategy is also usable for deep neural networks paving the way to optimizing these algorithms for smart microscopy.
Collapse
Affiliation(s)
- Maël Balluet
- CNRS, Univ Rennes, IGDR - UMR 6290, Rennes, France.,Inscoper SAS, Cesson-Sévigné, France
| | - Florian Sizaire
- CNRS, Univ Rennes, IGDR - UMR 6290, Rennes, France.,Present address Biologics Research, Sanofi R&D, Vitry-sur-Seine, France
| | | | - Thomas Walter
- Centre for Computational Biology (CBIO), MINES ParisTech, PSL University, Paris, France.,Institut Curie, Paris, France.,INSERM, U900, Paris, France
| | | | | | | | - Marc Tramier
- CNRS, Univ Rennes, IGDR - UMR 6290, Rennes, France.,Univ Rennes, BIOSIT, UMS CNRS 3480, US INSERM 018, Rennes, France
| | | |
Collapse
|
31
|
Chauhan C, Gullapalli RR. Ethics of AI in Pathology: Current Paradigms and Emerging Issues. THE AMERICAN JOURNAL OF PATHOLOGY 2021; 191:1673-1683. [PMID: 34252382 PMCID: PMC8485059 DOI: 10.1016/j.ajpath.2021.06.011] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Revised: 06/18/2021] [Accepted: 06/24/2021] [Indexed: 02/06/2023]
Abstract
Deep learning has rapidly advanced artificial intelligence (AI) and algorithmic decision-making (ADM) paradigms, affecting many traditional fields of medicine, including pathology, which is a heavily data-centric specialty of medicine. The structured nature of pathology data repositories makes it highly attractive to AI researchers to train deep learning models to improve health care delivery. Additionally, there are enormous financial incentives driving adoption of AI and ADM due to promise of increased efficiency of the health care delivery process. AI, if used unethically, may exacerbate existing inequities of health care, especially if not implemented correctly. There is an urgent need to harness the vast power of AI in an ethically and morally justifiable manner. This review explores the key issues involving AI ethics in pathology. Issues related to ethical design of pathology AI studies and the potential risks associated with implementation of AI and ADM within the pathology workflow are discussed. Three key foundational principles of ethical AI: transparency, accountability, and governance, are described in the context of pathology. The future practice of pathology must be guided by these principles. Pathologists should be aware of the potential of AI to deliver superlative health care and the ethical pitfalls associated with it. Finally, pathologists must have a seat at the table to drive future implementation of ethical AI in the practice of pathology.
Collapse
Affiliation(s)
- Chhavi Chauhan
- American Society of Investigative Pathology, Rockville, Maryland
| | - Rama R Gullapalli
- Department of Pathology, University of New Mexico, Albuquerque, New Mexico; Department of Chemical and Biological Engineering, University of New Mexico, Albuquerque, New Mexico.
| |
Collapse
|
32
|
Corvo A, Caballero HSG, Westenberg MA, van Driel MA, van Wijk JJ. Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2021; 27:3851-3866. [PMID: 32340951 DOI: 10.1109/tvcg.2020.2990336] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Recent advances in computational and algorithmic power are evolving the field of medical imaging rapidly. In cancer research, many new directions are sought to characterize patients with additional imaging features derived from radiology and pathology images. The emerging field of Computational Pathology targets the high-throughput extraction and analysis of the spatial distribution of cells from digital histopathology images. The associated morphological and architectural features allow researchers to quantify and characterize new imaging biomarkers for cancer diagnosis, prognosis, and treatment decisions. However, while the image feature space grows, exploration and analysis become more difficult and ineffective. There is a need for dedicated interfaces for interactive data manipulation and visual analysis of computational pathology and clinical data. For this purpose, we present IIComPath, a visual analytics approach that enables clinical researchers to formulate hypotheses and create computational pathology pipelines involving cohort construction, spatial analysis of image-derived features, and cohort analysis. We demonstrate our approach through use cases that investigate the prognostic value of current diagnostic features and new computational pathology biomarkers.
Collapse
|
33
|
Czyzewski T, Daniel N, Rochman M, Caldwell JM, Osswald GA, Collins MH, Rothenberg ME, Savir Y. Machine Learning Approach for Biopsy-Based Identification of Eosinophilic Esophagitis Reveals Importance of Global features. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2021; 2:218-223. [PMID: 34505063 PMCID: PMC8425173 DOI: 10.1109/ojemb.2021.3089552] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
Goal Eosinophilic esophagitis (EoE) is an allergic inflammatory condition characterized by eosinophil accumulation in the esophageal mucosa. EoE diagnosis includes a manual assessment of eosinophil levels in mucosal biopsies-a time-consuming, laborious task that is difficult to standardize. One of the main challenges in automating this process, like many other biopsy-based diagnostics, is detecting features that are small relative to the size of the biopsy. Results In this work, we utilized hematoxylin- and eosin-stained slides from esophageal biopsies from patients with active EoE and control subjects to develop a platform based on a deep convolutional neural network (DCNN) that can classify esophageal biopsies with an accuracy of 85%, sensitivity of 82.5%, and specificity of 87%. Moreover, by combining several downscaling and cropping strategies, we show that some of the features contributing to the correct classification are global rather than specific, local features. Conclusions We report the ability of artificial intelligence to identify EoE using computer vision analysis of esophageal biopsy slides. Further, the DCNN features associated with EoE are based on not only local eosinophils but also global histologic changes. Our approach can be used for other conditions that rely on biopsy-based histologic diagnostics.
Collapse
Affiliation(s)
- Tomer Czyzewski
- Department of Physiology, Biophysics, and System Biology, Faculty of Medicine, Technion Israel Institute of Technology, Haifa 35254, Israel
| | - Nati Daniel
- Department of Physiology, Biophysics, and System Biology, Faculty of Medicine, Technion Israel Institute of Technology, Haifa 35254, Israel
| | - Mark Rochman
- Division of Allergy, and Immunology, Cincinnati Children's Hospital Medical Center, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45229 USA
| | - Julie M Caldwell
- Division of Allergy, and Immunology, Cincinnati Children's Hospital Medical Center, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45229 USA
| | - Garrett A Osswald
- Division of Allergy, and Immunology, Cincinnati Children's Hospital Medical Center, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45229 USA
| | - Margaret H Collins
- Division of Pathology, Cincinnati Children's Hospital Medical Center, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45229 USA
| | - Marc E Rothenberg
- Division of Allergy, and Immunology, Cincinnati Children's Hospital Medical Center, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH 45229 USA
| | - Yonatan Savir
- Department of Physiology, Biophysics, and System Biology, Faculty of Medicine, Technion Israel Institute of Technology, Haifa 35254, Israel
| |
Collapse
|
34
|
Jang HJ, Song IH, Lee SH. Deep Learning for Automatic Subclassification of Gastric Carcinoma Using Whole-Slide Histopathology Images. Cancers (Basel) 2021; 13:3811. [PMID: 34359712 PMCID: PMC8345042 DOI: 10.3390/cancers13153811] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Revised: 07/19/2021] [Accepted: 07/24/2021] [Indexed: 12/12/2022] Open
Abstract
Histomorphologic types of gastric cancer (GC) have significant prognostic values that should be considered during treatment planning. Because the thorough quantitative review of a tissue slide is a laborious task for pathologists, deep learning (DL) can be a useful tool to support pathologic workflow. In the present study, a fully automated approach was applied to distinguish differentiated/undifferentiated and non-mucinous/mucinous tumor types in GC tissue whole-slide images from The Cancer Genome Atlas (TCGA) stomach adenocarcinoma dataset (TCGA-STAD). By classifying small patches of tissue images into differentiated/undifferentiated and non-mucinous/mucinous tumor tissues, the relative proportion of GC tissue subtypes can be easily quantified. Furthermore, the distribution of different tissue subtypes can be clearly visualized. The patch-level areas under the curves for the receiver operating characteristic curves for the differentiated/undifferentiated and non-mucinous/mucinous classifiers were 0.932 and 0.979, respectively. We also validated the classifiers on our own GC datasets and confirmed that the generalizability of the classifiers is excellent. The results indicate that the DL-based tissue classifier could be a useful tool for the quantitative analysis of cancer tissue slides. By combining DL-based classifiers for various molecular and morphologic variations in tissue slides, the heterogeneity of tumor tissues can be unveiled more efficiently.
Collapse
Affiliation(s)
- Hyun-Jong Jang
- Catholic Big Data Integration Center, Department of Physiology, College of Medicine, The Catholic University of Korea, Seoul 06591, Korea;
| | - In-Hye Song
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, Korea;
| | - Sung-Hak Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, Korea;
| |
Collapse
|
35
|
Klein C, Zeng Q, Arbaretaz F, Devêvre E, Calderaro J, Lomenie N, Maiuri MC. Artificial Intelligence for solid tumor diagnosis in digital pathology. Br J Pharmacol 2021; 178:4291-4315. [PMID: 34302297 DOI: 10.1111/bph.15633] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Revised: 02/05/2021] [Accepted: 02/07/2021] [Indexed: 11/30/2022] Open
Abstract
Tumor diagnosis relies on the visual examination of histological slides by pathologists through a microscope eyepiece. Digital pathology, the digitalization of histological slides at high magnification with slides scanners, has raised the opportunity to extract quantitative information thanks to image analysis. In the last decade, medical image analysis has made exceptional progress due to the development of artificial intelligence (AI) algorithms. AI has been successfully used in the field of medical imaging and more recently in digital pathology. The feasibility and usefulness of AI assisted pathology tasks have been demonstrated in the very last years and we can expect those developments to be applied on routine histopathology in the future. In this review, we will describe and illustrate this technique and present the most recent applications in the field of tumor histopathology.
Collapse
Affiliation(s)
- Christophe Klein
- Centre de recherche des Cordeliers, Centre d'Imagerie, Histologie et Cytométrie (CHIC), INSERM, Sorbonne Université, Université de Paris, Paris, France
| | - Qinghe Zeng
- Centre de recherche des Cordeliers, Centre d'Imagerie, Histologie et Cytométrie (CHIC), INSERM, Sorbonne Université, Université de Paris, Paris, France.,Laboratoire d'informatique Paris Descartes (LIPADE), Université de Paris, Paris, France
| | - Floriane Arbaretaz
- Centre de recherche des Cordeliers, Centre d'Imagerie, Histologie et Cytométrie (CHIC), INSERM, Sorbonne Université, Université de Paris, Paris, France
| | - Estelle Devêvre
- Centre de recherche des Cordeliers, Centre d'Imagerie, Histologie et Cytométrie (CHIC), INSERM, Sorbonne Université, Université de Paris, Paris, France
| | - Julien Calderaro
- Département de pathologie, Hôpital Henri Mondor, Créteil, France
| | - Nicolas Lomenie
- Laboratoire d'informatique Paris Descartes (LIPADE), Université de Paris, Paris, France
| | - Maria Chiara Maiuri
- Centre de recherche des Cordeliers, Centre d'Imagerie, Histologie et Cytométrie (CHIC), INSERM, Sorbonne Université, Université de Paris, Paris, France
| |
Collapse
|
36
|
Al Mouiee D, Meijering E, Kalloniatis M, Nivison-Smith L, Williams RA, Nayagam DAX, Spencer TC, Luu CD, McGowan C, Epp SB, Shivdasani MN. Classifying Retinal Degeneration in Histological Sections Using Deep Learning. Transl Vis Sci Technol 2021; 10:9. [PMID: 34110385 PMCID: PMC8196406 DOI: 10.1167/tvst.10.7.9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
Purpose Artificial intelligence (AI) techniques are increasingly being used to classify retinal diseases. In this study we investigated the ability of a convolutional neural network (CNN) in categorizing histological images into different classes of retinal degeneration. Methods Images were obtained from a chemically induced feline model of monocular retinal dystrophy and split into training and testing sets. The training set was graded for the level of retinal degeneration and used to train various CNN architectures. The testing set was evaluated through the best architecture and graded by six observers. Comparisons between model and observer classifications, and interobserver variability were measured. Finally, the effects of using less training images or images containing half the presentable context were investigated. Results The best model gave weighted-F1 scores in the range 85% to 90%. Cohen kappa scores reached up to 0.86, indicating high agreement between the model and observers. Interobserver variability was consistent with the model-observer variability in the model's ability to match predictions with the observers. Image context restriction resulted in model performance reduction by up to 6% and at least one training set size resulted in a model performance reduction of 10% compared to the original size. Conclusions Detecting the presence and severity of up to three classes of retinal degeneration in histological data can be reliably achieved with a deep learning classifier. Translational Relevance This work lays the foundations for future AI models which could aid in the evaluation of more intricate changes occurring in retinal degeneration, particularly in other types of clinically derived image data.
Collapse
Affiliation(s)
- Daniel Al Mouiee
- Graduate School of Biomedical Engineering, University of New South Wales, Kensington, NSW, Australia.,School of Computer Science and Engineering, University of New South Wales, Kensington, NSW, Australia.,School of Biotechnology and Biomolecular Science, University of New South Wales, Kensington, NSW, Australia
| | - Erik Meijering
- Graduate School of Biomedical Engineering, University of New South Wales, Kensington, NSW, Australia.,School of Computer Science and Engineering, University of New South Wales, Kensington, NSW, Australia
| | - Michael Kalloniatis
- School of Optometry and Vision Sciences, University of New South Wales, Kensington, NSW, Australia
| | - Lisa Nivison-Smith
- School of Optometry and Vision Sciences, University of New South Wales, Kensington, NSW, Australia
| | - Richard A Williams
- Department of Pathology, University of Melbourne, Parkville, VIC, Australia
| | - David A X Nayagam
- Department of Pathology, University of Melbourne, Parkville, VIC, Australia.,The Bionics Institute of Australia, East Melbourne, VIC, Australia
| | - Thomas C Spencer
- The Bionics Institute of Australia, East Melbourne, VIC, Australia.,Department of Biomedical Engineering, University of Melbourne, Parkville, VIC, Australia
| | - Chi D Luu
- Ophthalmology, Department of Surgery, University of Melbourne, Parkville, VIC, Australia.,Centre for Eye Research Australia, Royal Victorian Eye & Ear Hospital, East Melbourne, VIC, Australia
| | - Ceara McGowan
- The Bionics Institute of Australia, East Melbourne, VIC, Australia
| | - Stephanie B Epp
- The Bionics Institute of Australia, East Melbourne, VIC, Australia
| | - Mohit N Shivdasani
- Graduate School of Biomedical Engineering, University of New South Wales, Kensington, NSW, Australia.,The Bionics Institute of Australia, East Melbourne, VIC, Australia
| |
Collapse
|
37
|
Anderson M, Pitchforth E, Asaria M, Brayne C, Casadei B, Charlesworth A, Coulter A, Franklin BD, Donaldson C, Drummond M, Dunnell K, Foster M, Hussey R, Johnson P, Johnston-Webber C, Knapp M, Lavery G, Longley M, Clark JM, Majeed A, McKee M, Newton JN, O'Neill C, Raine R, Richards M, Sheikh A, Smith P, Street A, Taylor D, Watt RG, Whyte M, Woods M, McGuire A, Mossialos E. LSE-Lancet Commission on the future of the NHS: re-laying the foundations for an equitable and efficient health and care service after COVID-19. Lancet 2021; 397:1915-1978. [PMID: 33965070 DOI: 10.1016/s0140-6736(21)00232-4] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Revised: 12/10/2020] [Accepted: 01/07/2021] [Indexed: 02/06/2023]
Affiliation(s)
- Michael Anderson
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Emma Pitchforth
- College of Medicine and Health, University of Exeter, Exeter, UK
| | - Miqdad Asaria
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Carol Brayne
- Cambridge Public Health, University of Cambridge, Cambridge, UK
| | - Barbara Casadei
- Radcliffe Department of Medicine, BHF Centre of Research Excellence, NIHR Biomedical Research Centre, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Anita Charlesworth
- The Health Foundation, London, UK; College of Social Sciences, Health Services Management Centre, University of Birmingham, Birmingham, UK
| | - Angela Coulter
- Green Templeton College, University of Oxford, Oxford, UK; Department of Regional Health Research, Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
| | - Bryony Dean Franklin
- UCL School of Pharmacy, University College London, London, UK; NIHR Imperial Patient Safety Translational Research Centre, Imperial College Healthcare NHS Trust, London, UK
| | - Cam Donaldson
- Yunus Centre for Social Business and Health, Glasgow Caledonian University, Glasgow, UK
| | | | | | - Margaret Foster
- National Health Service Wales Shared Services Partnership, Cardiff, UK
| | | | | | | | - Martin Knapp
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Gavin Lavery
- Belfast Health and Social Care Trust, Belfast, UK
| | - Marcus Longley
- Welsh Institute for Health and Social Care, University of South Wales, Pontypridd, UK
| | | | - Azeem Majeed
- Department of Primary Care and Public Health, Imperial College London, London, UK
| | - Martin McKee
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK
| | | | - Ciaran O'Neill
- School of Medicine, Dentistry and Biomedical Sciences, Queen's University Belfast, Belfast, UK
| | - Rosalind Raine
- Department of Applied Health Research, University College London, London, UK
| | - Mike Richards
- Department of Health Policy, London School of Economics and Political Science, London, UK; The Health Foundation, London, UK
| | - Aziz Sheikh
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Peter Smith
- Centre for Health Economics, University of York, York, UK; Centre for Health Economics and Policy Innovation, Imperial College London, London, UK
| | - Andrew Street
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - David Taylor
- UCL School of Pharmacy, University College London, London, UK
| | - Richard G Watt
- Department of Epidemiology and Public Health, University College London, London, UK
| | - Moira Whyte
- College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, UK
| | - Michael Woods
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Alistair McGuire
- Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Elias Mossialos
- Department of Health Policy, London School of Economics and Political Science, London, UK; Institute of Global Health Innovation, Imperial College London, London, UK.
| |
Collapse
|
38
|
Lee SH, Song IH, Jang HJ. Feasibility of deep learning-based fully automated classification of microsatellite instability in tissue slides of colorectal cancer. Int J Cancer 2021; 149:728-740. [PMID: 33851412 DOI: 10.1002/ijc.33599] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 02/19/2021] [Accepted: 04/07/2021] [Indexed: 12/16/2022]
Abstract
High levels of microsatellite instability (MSI-H) occurs in about 15% of sporadic colorectal cancer (CRC) and is an important predictive marker for response to immune checkpoint inhibitors. To test the feasibility of a deep learning (DL)-based classifier as a screening tool for MSI status, we built a fully automated DL-based MSI classifier using pathology whole-slide images (WSIs) of CRCs. On small image patches of The Cancer Genome Atlas (TCGA) CRC WSI dataset, tissue/non-tissue, normal/tumor and MSS/MSI-H classifiers were applied sequentially for the fully automated prediction of the MSI status. The classifiers were also tested on an independent cohort. Furthermore, to test how the expansion of the training data affects the performance of the DL-based classifier, additional classifier trained on both TCGA and external datasets was tested. The areas under the receiver operating characteristic curves were 0.892 and 0.972 for the TCGA and external datasets, respectively, by a classifier trained on both datasets. The performance of the DL-based classifier was much better than that of previously reported histomorphology-based methods. We speculated that about 40% of CRC slides could be screened for MSI status without molecular testing by the DL-based classifier. These results demonstrated that the DL-based method has potential as a screening tool to discriminate molecular alteration in tissue slides.
Collapse
Affiliation(s)
- Sung Hak Lee
- Department of Hospital Pathology, Seoul St. Mary's Hospital, Seoul, South Korea
| | - In Hye Song
- Department of Hospital Pathology, Seoul St. Mary's Hospital, Seoul, South Korea
| | - Hyun-Jong Jang
- Catholic Big Data Integration Center, Department of Physiology, College of Medicine, The Catholic University of Korea, Seoul, South Korea
| |
Collapse
|
39
|
Wu LL, Wang JL, Huang W, Liu X, Huang YY, Zeng J, Cui CY, Lu JB, Lin P, Long H, Zhang LJ, Wei J, Lu Y, Ma GW. Prognostic Modeling of Patients Undergoing Surgery Alone for Esophageal Squamous Cell Carcinoma: A Histopathological and Computed Tomography Based Quantitative Analysis. Front Oncol 2021; 11:565755. [PMID: 33912439 PMCID: PMC8072145 DOI: 10.3389/fonc.2021.565755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 03/17/2021] [Indexed: 12/24/2022] Open
Abstract
Objective To evaluate the effectiveness of a novel computerized quantitative analysis based on histopathological and computed tomography (CT) images for predicting the postoperative prognosis of esophageal squamous cell carcinoma (ESCC) patients. Methods We retrospectively reviewed the medical records of 153 ESCC patients who underwent esophagectomy alone and quantitatively analyzed digital histological specimens and diagnostic CT images. We cut pathological images (6000 × 6000) into 50 × 50 patches; each patient had 14,400 patches. Cluster analysis was used to process these patches. We used the pathological clusters to all patches ratio (PCPR) of each case for pathological features and we obtained 20 PCPR quantitative features. Totally, 125 computerized quantitative (20 PCPR and 105 CT) features were extracted. We used a recursive feature elimination approach to select features. A Cox hazard model with L1 penalization was used for prognostic indexing. We compared the following prognostic models: Model A: clinical features; Model B: quantitative CT and clinical features; Model C: quantitative histopathological and clinical features; and Model D: combined information of clinical, CT, and histopathology. Indices of concordance (C-index) and leave-one-out cross-validation (LOOCV) were used to assess prognostic model accuracy. Results Five PCPR and eight CT features were treated as significant indicators in ESCC prognosis. C-indices adjusted for LOOCV were comparable among four models, 0.596 (Model A) vs. 0.658 (Model B) vs. 0.651 (Model C), and improved to 0.711with Model D combining information of clinical, CT, and histopathology (all p<0.05). Using Model D, we stratified patients into low- and high-risk groups. The 3-year overall survival rates of low- and high-risk patients were 38.0% and 25.0%, respectively (p<0.001). Conclusion Quantitative prognostic modeling using a combination of clinical data, histopathological, and CT images can stratify ESCC patients with surgery alone into high-risk and low-risk groups.
Collapse
Affiliation(s)
- Lei-Lei Wu
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Jin-Long Wang
- School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China
| | - Wei Huang
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Xuan Liu
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Yang-Yu Huang
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Jing Zeng
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Chun-Yan Cui
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Jia-Bin Lu
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Peng Lin
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Hao Long
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Lan-Jun Zhang
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| | - Jun Wei
- Department of Radiology, University of Michigan, Ann Arbor, MI, United States
| | - Yao Lu
- School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China
| | - Guo-Wei Ma
- Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
40
|
Abstract
PURPOSE OF REVIEW This review aims to shed light on recent applications of artificial intelligence in urologic oncology. RECENT FINDINGS Artificial intelligence algorithms harness the wealth of patient data to assist in diagnosing, staging, treating, and monitoring genitourinary malignancies. Successful applications of artificial intelligence in urologic oncology include interpreting diagnostic imaging, pathology, and genomic annotations. Many of these algorithms, however, lack external validity and can only provide predictions based on one type of dataset. SUMMARY Future applications of artificial intelligence will need to incorporate several forms of data in order to truly make headway in urologic oncology. Researchers must actively ensure future artificial intelligence developments encompass the entire prospective patient population.
Collapse
|
41
|
Chaunzwa TL, Hosny A, Xu Y, Shafer A, Diao N, Lanuti M, Christiani DC, Mak RH, Aerts HJWL. Deep learning classification of lung cancer histology using CT images. Sci Rep 2021; 11:5471. [PMID: 33727623 PMCID: PMC7943565 DOI: 10.1038/s41598-021-84630-x] [Citation(s) in RCA: 45] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Accepted: 02/15/2021] [Indexed: 02/07/2023] Open
Abstract
Tumor histology is an important predictor of therapeutic response and outcomes in lung cancer. Tissue sampling for pathologist review is the most reliable method for histology classification, however, recent advances in deep learning for medical image analysis allude to the utility of radiologic data in further describing disease characteristics and for risk stratification. In this study, we propose a radiomics approach to predicting non-small cell lung cancer (NSCLC) tumor histology from non-invasive standard-of-care computed tomography (CT) data. We trained and validated convolutional neural networks (CNNs) on a dataset comprising 311 early-stage NSCLC patients receiving surgical treatment at Massachusetts General Hospital (MGH), with a focus on the two most common histological types: adenocarcinoma (ADC) and Squamous Cell Carcinoma (SCC). The CNNs were able to predict tumor histology with an AUC of 0.71(p = 0.018). We also found that using machine learning classifiers such as k-nearest neighbors (kNN) and support vector machine (SVM) on CNN-derived quantitative radiomics features yielded comparable discriminative performance, with AUC of up to 0.71 (p = 0.017). Our best performing CNN functioned as a robust probabilistic classifier in heterogeneous test sets, with qualitatively interpretable visual explanations to its predictions. Deep learning based radiomics can identify histological phenotypes in lung cancer. It has the potential to augment existing approaches and serve as a corrective aid for diagnosticians.
Collapse
Affiliation(s)
- Tafadzwa L. Chaunzwa
- grid.38142.3c000000041936754XArtificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA USA ,grid.62560.370000 0004 0378 8294Department of Radiation Oncology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA ,grid.413575.10000 0001 2167 1581Howard Hughes Medical Institute, Chevy Chase, MD USA
| | - Ahmed Hosny
- grid.38142.3c000000041936754XArtificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA USA ,grid.62560.370000 0004 0378 8294Department of Radiation Oncology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA
| | - Yiwen Xu
- grid.38142.3c000000041936754XArtificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA USA ,grid.62560.370000 0004 0378 8294Department of Radiation Oncology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA
| | - Andrea Shafer
- grid.38142.3c000000041936754XHarvard T.H. Chan School of Public Health, Boston, MA USA
| | - Nancy Diao
- grid.38142.3c000000041936754XHarvard T.H. Chan School of Public Health, Boston, MA USA
| | - Michael Lanuti
- grid.32224.350000 0004 0386 9924Division of Thoracic Surgery, Massachusetts General Hospital, Boston, MA USA
| | - David C. Christiani
- grid.38142.3c000000041936754XHarvard T.H. Chan School of Public Health, Boston, MA USA ,grid.32224.350000 0004 0386 9924Department of Medicine, Massachusetts General Hospital, Boston, MA USA
| | - Raymond H. Mak
- grid.38142.3c000000041936754XArtificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA USA ,grid.62560.370000 0004 0378 8294Department of Radiation Oncology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA
| | - Hugo J. W. L. Aerts
- grid.38142.3c000000041936754XArtificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA USA ,grid.62560.370000 0004 0378 8294Department of Radiation Oncology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA ,grid.65499.370000 0001 2106 9910Department of Radiology, Dana Farber Cancer Institute and Brigham and Women’s Hospital, Boston, MA USA ,grid.5012.60000 0001 0481 6099Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
42
|
Tizhoosh HR, Diamandis P, Campbell CJV, Safarpoor A, Kalra S, Maleki D, Riasatian A, Babaie M. Searching Images for Consensus: Can AI Remove Observer Variability in Pathology? THE AMERICAN JOURNAL OF PATHOLOGY 2021; 191:1702-1708. [PMID: 33636179 DOI: 10.1016/j.ajpath.2021.01.015] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Revised: 01/05/2021] [Accepted: 01/25/2021] [Indexed: 02/07/2023]
Abstract
One of the major obstacles in reaching diagnostic consensus is observer variability. With the recent success of artificial intelligence, particularly the deep networks, the question emerges as to whether the fundamental challenge of diagnostic imaging can now be resolved. This article briefly reviews the problem and how eventually both supervised and unsupervised AI technologies could help to overcome it.
Collapse
Affiliation(s)
| | - Phedias Diamandis
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada
| | - Clinton J V Campbell
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Amir Safarpoor
- Kimia Laboratory, University of Waterloo, Waterloo, Canada
| | - Shivam Kalra
- Kimia Laboratory, University of Waterloo, Waterloo, Canada
| | - Danial Maleki
- Kimia Laboratory, University of Waterloo, Waterloo, Canada
| | | | - Morteza Babaie
- Kimia Laboratory, University of Waterloo, Waterloo, Canada
| |
Collapse
|
43
|
Harmon SA, Patel PG, Sanford TH, Caven I, Iseman R, Vidotto T, Picanço C, Squire JA, Masoudi S, Mehralivand S, Choyke PL, Berman DM, Turkbey B, Jamaspishvili T. High throughput assessment of biomarkers in tissue microarrays using artificial intelligence: PTEN loss as a proof-of-principle in multi-center prostate cancer cohorts. Mod Pathol 2021; 34:478-489. [PMID: 32884130 PMCID: PMC9152638 DOI: 10.1038/s41379-020-00674-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Revised: 08/21/2020] [Accepted: 08/21/2020] [Indexed: 02/06/2023]
Abstract
Phosphatase and tensin homolog (PTEN) loss is associated with adverse outcomes in prostate cancer and has clinical potential as a prognostic biomarker. The objective of this work was to develop an artificial intelligence (AI) system for automated detection and localization of PTEN loss on immunohistochemically (IHC) stained sections. PTEN loss was assessed using IHC in two prostate tissue microarrays (TMA) (internal cohort, n = 272 and external cohort, n = 129 patients). TMA cores were visually scored for PTEN loss by pathologists and, if present, spatially annotated. Cores from each patient within the internal TMA cohort were split into 90% cross-validation (N = 2048) and 10% hold-out testing (N = 224) sets. ResNet-101 architecture was used to train core-based classification using a multi-resolution ensemble approach (×5, ×10, and ×20). For spatial annotations, single resolution pixel-based classification was trained from patches extracted at ×20 resolution, interpolated to ×40 resolution, and applied in a sliding-window fashion. A final AI-based prediction model was created from combining multi-resolution and pixel-based models. Performance was evaluated in 428 cores of external cohort. From both cohorts, a total of 2700 cores were studied, with a frequency of PTEN loss of 14.5% in internal (180/1239) and external 13.5% (43/319) cancer cores. The final AI-based prediction of PTEN status demonstrated 98.1% accuracy (95.0% sensitivity, 98.4% specificity; median dice score = 0.811) in internal cohort cross-validation set and 99.1% accuracy (100% sensitivity, 99.0% specificity; median dice score = 0.804) in internal cohort test set. Overall core-based classification in the external cohort was significantly improved in the external cohort (area under the curve = 0.964, 90.6% sensitivity, 95.7% specificity) when further trained (fine-tuned) using 15% of cohort data (19/124 patients). These results demonstrate a robust and fully automated method for detection and localization of PTEN loss in prostate cancer tissue samples. AI-based algorithms have potential to streamline sample assessment in research and clinical laboratories.
Collapse
Affiliation(s)
- Stephanie A Harmon
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- Clinical Research Directorate, Frederick National Laboratory for Cancer Research, Frederick, MD, USA
| | - Palak G Patel
- Division of Cancer Biology & Genetics, Cancer Research Institute, Queen's University, Kingston, ON, Canada
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada
- Department of Cell Biology at The Arthur and Sonia Labatt Brain Tumour Research Centre at the Hospital for Sick Children, Toronto, ON, Canada
| | - Thomas H Sanford
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- Department of Urology, Upstate Medical University, Syracuse, NY, USA
| | - Isabelle Caven
- Division of Cancer Biology & Genetics, Cancer Research Institute, Queen's University, Kingston, ON, Canada
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada
| | - Rachael Iseman
- Division of Cancer Biology & Genetics, Cancer Research Institute, Queen's University, Kingston, ON, Canada
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada
| | - Thiago Vidotto
- Department of Genetics, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Clarissa Picanço
- Department of Genetics, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Jeremy A Squire
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada
- Department of Genetics, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Samira Masoudi
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Sherif Mehralivand
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Peter L Choyke
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - David M Berman
- Division of Cancer Biology & Genetics, Cancer Research Institute, Queen's University, Kingston, ON, Canada
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada
| | - Baris Turkbey
- Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Tamara Jamaspishvili
- Division of Cancer Biology & Genetics, Cancer Research Institute, Queen's University, Kingston, ON, Canada.
- Department of Pathology and Molecular Medicine, Queen's University, Kingston, ON, Canada.
| |
Collapse
|
44
|
Generalizability of Deep Learning System for the Pathologic Diagnosis of Various Cancers. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11020808] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
The deep learning (DL)-based approaches in tumor pathology help to overcome the limitations of subjective visual examination from pathologists and improve diagnostic accuracy and objectivity. However, it is unclear how a DL system trained to discriminate normal/tumor tissues in a specific cancer could perform on other tumor types. Herein, we cross-validated the DL-based normal/tumor classifiers separately trained on the tissue slides of cancers from bladder, lung, colon and rectum, stomach, bile duct, and liver. Furthermore, we compared the differences between the classifiers trained on the frozen or formalin-fixed paraffin-embedded (FFPE) tissues. The Area under the curve (AUC) for the receiver operating characteristic (ROC) curve ranged from 0.982 to 0.999 when the tissues were analyzed by the classifiers trained on the same tissue preparation modalities and cancer types. However, the AUCs could drop to 0.476 and 0.439 when the classifiers trained for different tissue modalities and cancer types were applied. Overall, the optimal performance could be achieved only when the tissue slides were analyzed by the classifiers trained on the same preparation modalities and cancer types.
Collapse
|
45
|
Marsh JN, Liu TC, Wilson PC, Swamidass SJ, Gaut JP. Development and Validation of a Deep Learning Model to Quantify Glomerulosclerosis in Kidney Biopsy Specimens. JAMA Netw Open 2021; 4:e2030939. [PMID: 33471115 PMCID: PMC7818108 DOI: 10.1001/jamanetworkopen.2020.30939] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
IMPORTANCE A chronic shortage of donor kidneys is compounded by a high discard rate, and this rate is directly associated with biopsy specimen evaluation, which shows poor reproducibility among pathologists. A deep learning algorithm for measuring percent global glomerulosclerosis (an important predictor of outcome) on images of kidney biopsy specimens could enable pathologists to more reproducibly and accurately quantify percent global glomerulosclerosis, potentially saving organs that would have been discarded. OBJECTIVE To compare the performances of pathologists with a deep learning model on quantification of percent global glomerulosclerosis in whole-slide images of donor kidney biopsy specimens, and to determine the potential benefit of a deep learning model on organ discard rates. DESIGN, SETTING, AND PARTICIPANTS This prognostic study used whole-slide images acquired from 98 hematoxylin-eosin-stained frozen and 51 permanent donor biopsy specimen sections retrieved from 83 kidneys. Serial annotation by 3 board-certified pathologists served as ground truth for model training and for evaluation. Images of kidney biopsy specimens were obtained from the Washington University database (retrieved between June 2015 and June 2017). Cases were selected randomly from a database of more than 1000 cases to include biopsy specimens representing an equitable distribution within 0% to 5%, 6% to 10%, 11% to 15%, 16% to 20%, and more than 20% global glomerulosclerosis. MAIN OUTCOMES AND MEASURES Correlation coefficient (r) and root-mean-square error (RMSE) with respect to annotations were computed for cross-validated model predictions and on-call pathologists' estimates of percent global glomerulosclerosis when using individual and pooled slide results. Data were analyzed from March 2018 to August 2020. RESULTS The cross-validated model results of section images retrieved from 83 donor kidneys showed higher correlation with annotations (r = 0.916; 95% CI, 0.886-0.939) than on-call pathologists (r = 0.884; 95% CI, 0.825-0.923) that was enhanced when pooling glomeruli counts from multiple levels (r = 0.933; 95% CI, 0.898-0.956). Model prediction error for single levels (RMSE, 5.631; 95% CI, 4.735-6.517) was 14% lower than on-call pathologists (RMSE, 6.523; 95% CI, 5.191-7.783), improving to 22% with multiple levels (RMSE, 5.094; 95% CI, 3.972-6.301). The model decreased the likelihood of unnecessary organ discard by 37% compared with pathologists. CONCLUSIONS AND RELEVANCE The findings of this prognostic study suggest that this deep learning model provided a scalable and robust method to quantify percent global glomerulosclerosis in whole-slide images of donor kidneys. The model performance improved by analyzing multiple levels of a section, surpassing the capacity of pathologists in the time-sensitive setting of examining donor biopsy specimens. The results indicate the potential of a deep learning model to prevent erroneous donor organ discard.
Collapse
Affiliation(s)
- Jon N Marsh
- Department of Pathology and Immunology, Washington University School of Medicine in St Louis, St Louis, Missouri
- Institute for Informatics (I 2 ), Washington University School of Medicine in St Louis, St Louis, Missouri
| | - Ta-Chiang Liu
- Department of Pathology and Immunology, Washington University School of Medicine in St Louis, St Louis, Missouri
| | - Parker C Wilson
- Department of Pathology and Immunology, Washington University School of Medicine in St Louis, St Louis, Missouri
| | - S Joshua Swamidass
- Department of Pathology and Immunology, Washington University School of Medicine in St Louis, St Louis, Missouri
- Institute for Informatics (I 2 ), Washington University School of Medicine in St Louis, St Louis, Missouri
| | - Joseph P Gaut
- Department of Pathology and Immunology, Washington University School of Medicine in St Louis, St Louis, Missouri
- Department of Medicine, Washington University School of Medicine in St Louis, St Louis, Missouri
| |
Collapse
|
46
|
Chlis NK, Rausch L, Brocker T, Kranich J, Theis FJ. Predicting single-cell gene expression profiles of imaging flow cytometry data with machine learning. Nucleic Acids Res 2020; 48:11335-11346. [PMID: 33119742 PMCID: PMC7672460 DOI: 10.1093/nar/gkaa926] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2020] [Revised: 08/24/2020] [Accepted: 10/28/2020] [Indexed: 02/07/2023] Open
Abstract
High-content imaging and single-cell genomics are two of the most prominent high-throughput technologies for studying cellular properties and functions at scale. Recent studies have demonstrated that information in large imaging datasets can be used to estimate gene mutations and to predict the cell-cycle state and the cellular decision making directly from cellular morphology. Thus, high-throughput imaging methodologies, such as imaging flow cytometry can potentially aim beyond simple sorting of cell-populations. We introduce IFC-seq, a machine learning methodology for predicting the expression profile of every cell in an imaging flow cytometry experiment. Since it is to-date unfeasible to observe single-cell gene expression and morphology in flow, we integrate uncoupled imaging data with an independent transcriptomics dataset by leveraging common surface markers. We demonstrate that IFC-seq successfully models gene expression of a moderate number of key gene-markers for two independent imaging flow cytometry datasets: (i) human blood mononuclear cells and (ii) mouse myeloid progenitor cells. In the case of mouse myeloid progenitor cells IFC-seq can predict gene expression directly from brightfield images in a label-free manner, using a convolutional neural network. The proposed method promises to add gene expression information to existing and new imaging flow cytometry datasets, at no additional cost.
Collapse
Affiliation(s)
- Nikolaos-Kosmas Chlis
- Institute of Computational Biology, Helmholtz Zentrum München, Neuherberg 85764, Germany.,Roche Pharma Research and Early Development, Large Molecule Research, Roche Innovation Center Munich, Penzberg 82377, Germany
| | - Lisa Rausch
- Institute for Immunology, Medical Faculty, Ludwig Maximilian University of Munich, 82152 Planegg-Martinsried, Germany
| | - Thomas Brocker
- Institute for Immunology, Medical Faculty, Ludwig Maximilian University of Munich, 82152 Planegg-Martinsried, Germany
| | - Jan Kranich
- Institute for Immunology, Medical Faculty, Ludwig Maximilian University of Munich, 82152 Planegg-Martinsried, Germany
| | - Fabian J Theis
- Institute of Computational Biology, Helmholtz Zentrum München, Neuherberg 85764, Germany.,Department of Mathematics, Technical University of Munich, Garching 85748, Germany
| |
Collapse
|
47
|
Cho KO, Lee SH, Jang HJ. Feasibility of fully automated classification of whole slide images based on deep learning. THE KOREAN JOURNAL OF PHYSIOLOGY & PHARMACOLOGY : OFFICIAL JOURNAL OF THE KOREAN PHYSIOLOGICAL SOCIETY AND THE KOREAN SOCIETY OF PHARMACOLOGY 2020; 24:89-99. [PMID: 31908578 PMCID: PMC6940498 DOI: 10.4196/kjpp.2020.24.1.89] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 11/25/2019] [Accepted: 11/27/2019] [Indexed: 12/11/2022]
Abstract
Although microscopic analysis of tissue slides has been the basis for disease diagnosis for decades, intra- and inter-observer variabilities remain issues to be resolved. The recent introduction of digital scanners has allowed for using deep learning in the analysis of tissue images because many whole slide images (WSIs) are accessible to researchers. In the present study, we investigated the possibility of a deep learning-based, fully automated, computer-aided diagnosis system with WSIs from a stomach adenocarcinoma dataset. Three different convolutional neural network architectures were tested to determine the better architecture for tissue classifier. Each network was trained to classify small tissue patches into normal or tumor. Based on the patch-level classification, tumor probability heatmaps can be overlaid on tissue images. We observed three different tissue patterns, including clear normal, clear tumor and ambiguous cases. We suggest that longer inspection time can be assigned to ambiguous cases compared to clear normal cases, increasing the accuracy and efficiency of histopathologic diagnosis by pre-evaluating the status of the WSIs. When the classifier was tested with completely different WSI dataset, the performance was not optimal because of the different tissue preparation quality. By including a small amount of data from the new dataset for training, the performance for the new dataset was much enhanced. These results indicated that WSI dataset should include tissues prepared from many different preparation conditions to construct a generalized tissue classifier. Thus, multi-national/multi-center dataset should be built for the application of deep learning in the real world medical practice.
Collapse
Affiliation(s)
- Kyung-Ok Cho
- Department of Pharmacology, Seoul St. Mary's Hospital, Seoul 06591, Korea.,Department of Biomedicine & Health Sciences, Seoul St. Mary's Hospital, Seoul 06591, Korea.,Catholic Neuroscience Institute, Seoul St. Mary's Hospital, Seoul 06591, Korea
| | - Sung Hak Lee
- Department of Hospital Pathology, Seoul St. Mary's Hospital, Seoul 06591, Korea
| | - Hyun-Jong Jang
- Department of Biomedicine & Health Sciences, Seoul St. Mary's Hospital, Seoul 06591, Korea.,Catholic Neuroscience Institute, Seoul St. Mary's Hospital, Seoul 06591, Korea.,Department of Physiology, College of Medicine, The Catholic University of Korea, Seoul 06591, Korea
| |
Collapse
|
48
|
Jang HJ, Lee A, Kang J, Song IH, Lee SH. Prediction of clinically actionable genetic alterations from colorectal cancer histopathology images using deep learning. World J Gastroenterol 2020; 26:6207-6223. [PMID: 33177794 PMCID: PMC7596644 DOI: 10.3748/wjg.v26.i40.6207] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/28/2020] [Revised: 08/09/2020] [Accepted: 09/25/2020] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Identifying genetic mutations in cancer patients have been increasingly important because distinctive mutational patterns can be very informative to determine the optimal therapeutic strategy. Recent studies have shown that deep learning-based molecular cancer subtyping can be performed directly from the standard hematoxylin and eosin (H&E) sections in diverse tumors including colorectal cancers (CRCs). Since H&E-stained tissue slides are ubiquitously available, mutation prediction with the pathology images from cancers can be a time- and cost-effective complementary method for personalized treatment. AIM To predict the frequently occurring actionable mutations from the H&E-stained CRC whole-slide images (WSIs) with deep learning-based classifiers. METHODS A total of 629 CRC patients from The Cancer Genome Atlas (TCGA-COAD and TCGA-READ) and 142 CRC patients from Seoul St. Mary Hospital (SMH) were included. Based on the mutation frequency in TCGA and SMH datasets, we chose APC, KRAS, PIK3CA, SMAD4, and TP53 genes for the study. The classifiers were trained with 360 × 360 pixel patches of tissue images. The receiver operating characteristic (ROC) curves and area under the curves (AUCs) for all the classifiers were presented. RESULTS The AUCs for ROC curves ranged from 0.693 to 0.809 for the TCGA frozen WSIs and from 0.645 to 0.783 for the TCGA formalin-fixed paraffin-embedded WSIs. The prediction performance can be enhanced with the expansion of datasets. When the classifiers were trained with both TCGA and SMH data, the prediction performance was improved. CONCLUSION APC, KRAS, PIK3CA, SMAD4, and TP53 mutations can be predicted from H&E pathology images using deep learning-based classifiers, demonstrating the potential for deep learning-based mutation prediction in the CRC tissue slides.
Collapse
Affiliation(s)
- Hyun-Jong Jang
- Department of Physiology, Department of Biomedicine and Health Sciences, Catholic Neuroscience Institute, The Catholic University of Korea, Seoul 06591, South Korea
| | - Ahwon Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - J Kang
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - In Hye Song
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| | - Sung Hak Lee
- Department of Hospital Pathology, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul 06591, South Korea
| |
Collapse
|
49
|
Steiner DF, Chen PHC, Mermel CH. Closing the translation gap: AI applications in digital pathology. Biochim Biophys Acta Rev Cancer 2020; 1875:188452. [PMID: 33065195 DOI: 10.1016/j.bbcan.2020.188452] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 07/31/2020] [Accepted: 10/09/2020] [Indexed: 02/07/2023]
Abstract
Recent advances in artificial intelligence show tremendous promise to improve the accuracy, reproducibility, and availability of medical diagnostics across a number of medical subspecialities. This is especially true in the field of digital pathology, which has recently witnessed a surge in publications describing state-of-the-art performance for machine learning models across a wide range of diagnostic applications. Nonetheless, despite this promise, there remain significant gaps in translating applications for any of these technologies into actual clinical practice. In this review, we will first give a brief overview of the recent progress in applying AI to digitized pathology images, focusing on how these tools might be applied in clinical workflows in the near term to improve the accuracy and efficiency of pathologists. Then we define and describe in detail the various factors that need to be addressed in order to successfully close the "translation gap" for AI applications in digital pathology.
Collapse
|
50
|
Dov D, Kovalsky SZ, Assaad S, Cohen J, Range DE, Pendse AA, Henao R, Carin L. Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images. Med Image Anal 2020; 67:101814. [PMID: 33049578 DOI: 10.1016/j.media.2020.101814] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2019] [Revised: 06/16/2020] [Accepted: 07/28/2020] [Indexed: 02/07/2023]
Abstract
We consider machine-learning-based thyroid-malignancy prediction from cytopathology whole-slide images (WSI). Multiple instance learning (MIL) approaches, typically used for the analysis of WSIs, divide the image (bag) into patches (instances), which are used to predict a single bag-level label. These approaches perform poorly in cytopathology slides due to a unique bag structure: sparsely located informative instances with varying characteristics of abnormality. We address these challenges by considering multiple types of labels: bag-level malignancy and ordered diagnostic scores, as well as instance-level informativeness and abnormality labels. We study their contribution beyond the MIL setting by proposing a maximum likelihood estimation (MLE) framework, from which we derive a two-stage deep-learning-based algorithm. The algorithm identifies informative instances and assigns them local malignancy scores that are incorporated into a global malignancy prediction. We derive a lower bound of the MLE, leading to an improved training strategy based on weak supervision, that we motivate through statistical analysis. The lower bound further allows us to extend the proposed algorithm to simultaneously predict multiple bag and instance-level labels from a single output of a neural network. Experimental results demonstrate that the proposed algorithm provides competitive performance compared to several competing methods, achieves (expert) human-level performance, and allows augmentation of human decisions.
Collapse
Affiliation(s)
- David Dov
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA.
| | | | - Serge Assaad
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| | - Jonathan Cohen
- Department of Surgery, Duke University Medical Center, Durham, NC 27710, USA
| | | | - Avani A Pendse
- Department of Pathology, Duke University Medical Center, Durham, NC 27710, USA
| | - Ricardo Henao
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| | - Lawrence Carin
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| |
Collapse
|