51
|
Yin PN, Kc K, Wei S, Yu Q, Li R, Haake AR, Miyamoto H, Cui F. Histopathological distinction of non-invasive and invasive bladder cancers using machine learning approaches. BMC Med Inform Decis Mak 2020; 20:162. [PMID: 32680493 PMCID: PMC7367328 DOI: 10.1186/s12911-020-01185-z] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2019] [Accepted: 07/13/2020] [Indexed: 01/18/2023] Open
Abstract
Background One of the most challenging tasks for bladder cancer diagnosis is to histologically differentiate two early stages, non-invasive Ta and superficially invasive T1, the latter of which is associated with a significantly higher risk of disease progression. Indeed, in a considerable number of cases, Ta and T1 tumors look very similar under microscope, making the distinction very difficult even for experienced pathologists. Thus, there is an urgent need for a favoring system based on machine learning (ML) to distinguish between the two stages of bladder cancer. Methods A total of 1177 images of bladder tumor tissues stained by hematoxylin and eosin were collected by pathologists at University of Rochester Medical Center, which included 460 non-invasive (stage Ta) and 717 invasive (stage T1) tumors. Automatic pipelines were developed to extract features for three invasive patterns characteristic to the T1 stage bladder cancer (i.e., desmoplastic reaction, retraction artifact, and abundant pinker cytoplasm), using imaging processing software ImageJ and CellProfiler. Features extracted from the images were analyzed by a suite of machine learning approaches. Results We extracted nearly 700 features from the Ta and T1 tumor images. Unsupervised clustering analysis failed to distinguish hematoxylin and eosin images of Ta vs. T1 tumors. With a reduced set of features, we successfully distinguished 1177 Ta or T1 images with an accuracy of 91–96% by six supervised learning methods. By contrast, convolutional neural network (CNN) models that automatically extract features from images produced an accuracy of 84%, indicating that feature extraction driven by domain knowledge outperforms CNN-based automatic feature extraction. Further analysis revealed that desmoplastic reaction was more important than the other two patterns, and the number and size of nuclei of tumor cells were the most predictive features. Conclusions We provide a ML-empowered, feature-centered, and interpretable diagnostic system to facilitate the accurate staging of Ta and T1 diseases, which has a potential to apply to other types of cancer.
Collapse
Affiliation(s)
- Peng-Nien Yin
- Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology, 1 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Kishan Kc
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, 20 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Shishi Wei
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, 20 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Qi Yu
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, 20 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Rui Li
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, 20 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Anne R Haake
- Golisano College of Computing and Information Sciences, Rochester Institute of Technology, 20 Lomb Memorial Drive, Rochester, NY, 14623, USA
| | - Hiroshi Miyamoto
- Department of Pathology and Laboratory Medicine, University of Rochester Medical Center, 601 Elmwood Avenue, Rochester, NY, 14642, USA.
| | - Feng Cui
- Thomas H. Gosnell School of Life Sciences, Rochester Institute of Technology, 1 Lomb Memorial Drive, Rochester, NY, 14623, USA.
| |
Collapse
|
52
|
Acs B, Rantalainen M, Hartman J. Artificial intelligence as the next step towards precision pathology. J Intern Med 2020; 288:62-81. [PMID: 32128929 DOI: 10.1111/joim.13030] [Citation(s) in RCA: 173] [Impact Index Per Article: 43.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Revised: 12/16/2019] [Accepted: 12/30/2019] [Indexed: 12/13/2022]
Abstract
Pathology is the cornerstone of cancer care. The need for accuracy in histopathologic diagnosis of cancer is increasing as personalized cancer therapy requires accurate biomarker assessment. The appearance of digital image analysis holds promise to improve both the volume and precision of histomorphological evaluation. Recently, machine learning, and particularly deep learning, has enabled rapid advances in computational pathology. The integration of machine learning into routine care will be a milestone for the healthcare sector in the next decade, and histopathology is right at the centre of this revolution. Examples of potential high-value machine learning applications include both model-based assessment of routine diagnostic features in pathology, and the ability to extract and identify novel features that provide insights into a disease. Recent groundbreaking results have demonstrated that applications of machine learning methods in pathology significantly improves metastases detection in lymph nodes, Ki67 scoring in breast cancer, Gleason grading in prostate cancer and tumour-infiltrating lymphocyte (TIL) scoring in melanoma. Furthermore, deep learning models have also been demonstrated to be able to predict status of some molecular markers in lung, prostate, gastric and colorectal cancer based on standard HE slides. Moreover, prognostic (survival outcomes) deep neural network models based on digitized HE slides have been demonstrated in several diseases, including lung cancer, melanoma and glioma. In this review, we aim to present and summarize the latest developments in digital image analysis and in the application of artificial intelligence in diagnostic pathology.
Collapse
Affiliation(s)
- B Acs
- From the, Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
| | - M Rantalainen
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - J Hartman
- From the, Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
| |
Collapse
|
53
|
Liu Y. Application of artificial intelligence in clinical non-small cell lung cancer. Artif Intell Cancer 2020; 1:19-30. [DOI: 10.35713/aic.v1.i1.19] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2020] [Revised: 06/17/2020] [Accepted: 06/18/2020] [Indexed: 02/06/2023] Open
Abstract
Lung cancer is the most common cause of cancer death in the world. Early diagnosis, screening and precise individualized treatment can significantly reduce the death rate of lung cancer. Artificial intelligence (AI) has been shown to be able to help clinicians make more accurate judgments and decisions in many ways. It has been involved in the screening of lung cancer, the judgment of benign and malignant degree of pulmonary nodules, the classification of histological cancer, the differentiation of histological subtypes, the identification of genomics, the judgment of the effectiveness of treatment and even the prognosis. AI has shown that it can be an excellent assistant for clinicians. This paper reviews the application of AI in the field of non-small cell lung cancer and describes the relevant progress. Although most of the studies to evaluate the clinical application of AI in non-small cell lung cancer have not been repeatable and generalizable, the research results highlight the efforts to promote the clinical application of AI technology and influence the future treatment direction.
Collapse
Affiliation(s)
- Yong Liu
- Department of Thoracic Surgery, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430011, Hubei Province, China
| |
Collapse
|
54
|
Yang L, Zhang J, Yang G, Xu H, Lin J, Shao L, Li J, Guo C, Du Y, Guo L, Li X, Han-Zhang H, Wang C, Chuai S, Ye J, Kang Q, Liu H, Ying J, Wang Y. The prognostic value of a Methylome-based Malignancy Density Scoring System to predict recurrence risk in early-stage Lung Adenocarcinoma. Theranostics 2020; 10:7635-7644. [PMID: 32685009 PMCID: PMC7359091 DOI: 10.7150/thno.44229] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Accepted: 06/01/2020] [Indexed: 12/11/2022] Open
Abstract
Current NCCN guidelines do not recommend the use of adjuvant chemotherapy for stage IA lung adenocarcinoma patients with R0 surgery. However, 25% to 40% of patients with stage IA disease experience recurrence. Stratifying patients according to the recurrence risk may tailor adjuvant therapy and surveillance imaging for those with a higher risk. However, prognostic markers are often identified by comparing high-risk and low-risk cases which might introduce bias due to the widespread interpatient heterogeneity. Here, we developed a scoring system quantifying the degree of field cancerization in adjacent normal tissues and revealed its association with disease-free survival (DFS). Methods: We recruited a cohort of 44 patients with resected stage IA lung adenocarcinoma who did not receive adjuvant therapy. Both tumor and adjacent normal tissues were obtained from each patient and subjected to capture-based targeted genomic and epigenomic profiling. A novel methylome-based scoring system namely malignancy density ratio (MD ratio) was developed based on 39 patients by comparing tumor and corresponding adjacent normal tissues of each patient. A MD score was then obtained by Wald statistics. The correlations of MD ratio, MD score, and genomic features with clinical outcome were investigated. Results: Patients with a high-risk MD ratio showed a significantly shorter postsurgical DFS compared with those with a low-risk MD ratio (HR=4.47, P=0.01). The MD ratio was not associated with T stage (P=1), tumor cell fraction (P=0.748) nor inflammatory status (p=0.548). Patients with a high-risk MD score also demonstrated an inferior DFS (HR=4.69, P=0.039). In addition, multivariate analysis revealed EGFR 19 del (HR=5.39, P=0.012) and MD score (HR= 7.90, P=0.01) were independent prognostic markers. Conclusion: The novel methylome-based scoring system, developed by comparing the signatures between tumor and corresponding adjacent normal tissues of individual patients, largely minimizes the bias of interpatient heterogeneity and reveals a robust prognostic value in patients with resected lung adenocarcinoma.
Collapse
|
55
|
Jiang Y, Yang M, Wang S, Li X, Sun Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun (Lond) 2020; 40:154-166. [PMID: 32277744 PMCID: PMC7170661 DOI: 10.1002/cac2.12012] [Citation(s) in RCA: 154] [Impact Index Per Article: 38.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Accepted: 02/06/2020] [Indexed: 12/11/2022] Open
Abstract
The development of digital pathology and progression of state-of-the-art algorithms for computer vision have led to increasing interest in the use of artificial intelligence (AI), especially deep learning (DL)-based AI, in tumor pathology. The DL-based algorithms have been developed to conduct all kinds of work involved in tumor pathology, including tumor diagnosis, subtyping, grading, staging, and prognostic prediction, as well as the identification of pathological features, biomarkers and genetic changes. The applications of AI in pathology not only contribute to improve diagnostic accuracy and objectivity but also reduce the workload of pathologists and subsequently enable them to spend additional time on high-level decision-making tasks. In addition, AI is useful for pathologists to meet the requirements of precision oncology. However, there are still some challenges relating to the implementation of AI, including the issues of algorithm validation and interpretability, computing systems, the unbelieving attitude of pathologists, clinicians and patients, as well as regulators and reimbursements. Herein, we present an overview on how AI-based approaches could be integrated into the workflow of pathologists and discuss the challenges and perspectives of the implementation of AI in tumor pathology.
Collapse
Affiliation(s)
- Yahui Jiang
- Department of PathologyKey Laboratory of Cancer Prevention and TherapyTianjin's Clinical Research Center for CancerNational Clinical Research Center for CancerTianjin Cancer Institute and HospitalTianjin Medical UniversityTianjin300060P. R. China
| | - Meng Yang
- Department Epidemiology and BiostatisticsKey Laboratory of Cancer Prevention and TherapyTianjin's Clinical Research Center for CancerNational Clinical Research Center for CancerTianjin Cancer Institute and HospitalTianjin Medical UniversityTianjin300060P.R. China
| | - Shuhao Wang
- Institute for Interdisciplinary Information SciencesTsinghua UniversityBeijing100084P. R. China
| | - Xiangchun Li
- Department Epidemiology and BiostatisticsKey Laboratory of Cancer Prevention and TherapyTianjin's Clinical Research Center for CancerNational Clinical Research Center for CancerTianjin Cancer Institute and HospitalTianjin Medical UniversityTianjin300060P.R. China
| | - Yan Sun
- Department of PathologyKey Laboratory of Cancer Prevention and TherapyTianjin's Clinical Research Center for CancerNational Clinical Research Center for CancerTianjin Cancer Institute and HospitalTianjin Medical UniversityTianjin300060P. R. China
| |
Collapse
|
56
|
Ji MY, Yuan L, Lu SM, Gao MT, Zeng Z, Zhan N, Ding YJ, Liu ZR, Huang PX, Lu C, Dong WG. Glandular orientation and shape determined by computational pathology could identify aggressive tumor for early colon carcinoma: a triple-center study. J Transl Med 2020; 18:129. [PMID: 32178690 PMCID: PMC7077008 DOI: 10.1186/s12967-020-02297-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 03/11/2020] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND Identifying the early-stage colon adenocarcinoma (ECA) patients who have lower risk cancer vs. the higher risk cancer could improve disease prognosis. Our study aimed to explore whether the glandular morphological features determined by computational pathology could identify high risk cancer in ECA via H&E images digitally. METHODS 532 ECA patients retrospectively from 2 independent data centers, as well as 113 from The Cancer Genome Atlas (TCGA), were enrolled in this study. Four tissue microarrays (TMAs) were constructed across ECA hematoxylin and eosin (H&E) stained slides. 797 quantitative glandular morphometric features were extracted and 5 most prognostic features were identified using minimum redundancy maximum relevance to construct an image classifier. The image classifier was evaluated on D2/D3 = 223, D4 = 46, D5 = 113. The expression of Ki67 and serum CEA levels were scored on D3, aiming to explore the correlations between image classifier and immunohistochemistry data and serum CEA levels. The roles of clinicopathological data and ECAHBC were evaluated by univariate and multivariate analyses for prognostic value. RESULTS The image classifier could predict ECA recurrence (accuracy of 88.1%). ECA histomorphometric-based image classifier (ECAHBC) was an independent prognostic factor for poorer disease-specific survival [DSS, (HR = 9.65, 95% CI 2.15-43.12, P = 0.003)]. Significant correlations were observed between ECAHBC-positive patients and positivity of Ki67 labeling index (Ki67Li) and serum CEA. CONCLUSION Glandular orientation and shape could predict the high risk cancer in ECA and contribute to precision oncology. Computational pathology is emerging as a viable and objective means of identifying predictive biomarkers for cancer patients.
Collapse
Affiliation(s)
- Meng-Yao Ji
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China.,Key Laboratory of Hubei Province for Digestive System Disease, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Lei Yuan
- Department of Information Center, Wuhan University Renmin Hospital, Wuhan, Hubei, China. .,Key Laboratory of Hubei Province for Digestive System Disease, Wuhan University Renmin Hospital, Wuhan, Hubei, China.
| | - Shi-Min Lu
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China.,Key Laboratory of Hubei Province for Digestive System Disease, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Meng-Ting Gao
- Department of Information Center, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Zhi Zeng
- Department of Pathology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Na Zhan
- Department of Pathology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Yi-Juan Ding
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Zheng-Ru Liu
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Ping-Xiao Huang
- Department of Gastroenterology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Cheng Lu
- College of Computer Science, Shaanxi Normal University, Xi'an, Shaanxi, China.
| | - Wei-Guo Dong
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China.
| |
Collapse
|
57
|
Failmezger H, Muralidhar S, Rullan A, de Andrea CE, Sahai E, Yuan Y. Topological Tumor Graphs: A Graph-Based Spatial Model to Infer Stromal Recruitment for Immunosuppression in Melanoma Histology. Cancer Res 2020; 80:1199-1209. [PMID: 31874858 PMCID: PMC7985597 DOI: 10.1158/0008-5472.can-19-2268] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 11/12/2019] [Accepted: 12/10/2019] [Indexed: 01/08/2023]
Abstract
Despite the advent of immunotherapy, metastatic melanoma represents an aggressive tumor type with a poor survival outcome. The successful application of immunotherapy requires in-depth understanding of the biological basis and immunosuppressive mechanisms within the tumor microenvironment. In this study, we conducted spatially explicit analyses of the stromal-immune interface across 400 melanoma hematoxylin and eosin (H&E) specimens from The Cancer Genome Atlas. A computational pathology pipeline (CRImage) was used to classify cells in the H&E specimen into stromal, immune, or cancer cells. The estimated proportions of these cell types were validated by independent measures of tumor purity, pathologists' estimate of lymphocyte density, imputed immune cell subtypes, and pathway analyses. Spatial interactions between these cell types were computed using a graph-based algorithm (topological tumor graphs, TTG). This approach identified two stromal features, namely stromal clustering and stromal barrier, which represented the melanoma stromal microenvironment. Tumors with increased stromal clustering and barrier were associated with reduced intratumoral lymphocyte distribution and poor overall survival independent of existing prognostic factors. To explore the genomic basis of these TTG-derived stromal phenotypes, we used a deep learning approach integrating genomic (copy number) and transcriptomic data, thereby inferring a compressed representation of copy number-driven alterations in gene expression. This integrative analysis revealed that tumors with high stromal clustering and barrier had reduced expression of pathways involved in naïve CD4 signaling, MAPK, and PI3K signaling. Taken together, our findings support the immunosuppressive role of stromal cells and T-cell exclusion within the vicinity of melanoma cells. SIGNIFICANCE: Computational histology-based stromal phenotypes within the tumor microenvironment are significantly associated with prognosis and immune exclusion in melanoma.
Collapse
MESH Headings
- Adult
- Aged
- Aged, 80 and over
- Antineoplastic Agents, Immunological/pharmacology
- Antineoplastic Agents, Immunological/therapeutic use
- Biomarkers, Tumor/genetics
- Biomarkers, Tumor/metabolism
- Biopsy
- Cohort Studies
- DNA Copy Number Variations
- Deep Learning
- Drug Resistance, Neoplasm/genetics
- Drug Resistance, Neoplasm/immunology
- Follow-Up Studies
- Gene Expression Regulation, Neoplastic
- Humans
- Image Interpretation, Computer-Assisted
- Kaplan-Meier Estimate
- Lymphocytes, Tumor-Infiltrating/immunology
- Melanoma/drug therapy
- Melanoma/genetics
- Melanoma/immunology
- Melanoma/mortality
- Middle Aged
- Models, Biological
- Prognosis
- RNA-Seq
- Skin/cytology
- Skin/immunology
- Skin/pathology
- Skin Neoplasms/drug therapy
- Skin Neoplasms/genetics
- Skin Neoplasms/immunology
- Skin Neoplasms/mortality
- Spatial Analysis
- Stromal Cells/immunology
- Stromal Cells/pathology
- T-Lymphocytes/immunology
- Tumor Escape/genetics
- Tumor Escape/immunology
- Tumor Microenvironment/drug effects
- Tumor Microenvironment/genetics
- Tumor Microenvironment/immunology
Collapse
Affiliation(s)
- Henrik Failmezger
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, United Kingdom
- Division of Molecular Pathology, The Institute of Cancer Research, London, United Kingdom
| | - Sathya Muralidhar
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, United Kingdom
- Division of Molecular Pathology, The Institute of Cancer Research, London, United Kingdom
| | - Antonio Rullan
- Tumor Cell Biology Laboratory, The Francis Crick Institute, London, United Kingdom
- Targeted therapy Laboratory, The Royal Marsden NHS Foundation Trust, London, United Kingdom
| | | | - Erik Sahai
- Tumor Cell Biology Laboratory, The Francis Crick Institute, London, United Kingdom
| | - Yinyin Yuan
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, United Kingdom.
- Division of Molecular Pathology, The Institute of Cancer Research, London, United Kingdom
| |
Collapse
|
58
|
Toğaçar M, Ergen B, Cömert Z. Detection of lung cancer on chest CT images using minimum redundancy maximum relevance feature selection method with convolutional neural networks. Biocybern Biomed Eng 2020. [DOI: 10.1016/j.bbe.2019.11.004] [Citation(s) in RCA: 59] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
59
|
Tran WT, Jerzak K, Lu FI, Klein J, Tabbarah S, Lagree A, Wu T, Rosado-Mendez I, Law E, Saednia K, Sadeghi-Naini A. Personalized Breast Cancer Treatments Using Artificial Intelligence in Radiomics and Pathomics. J Med Imaging Radiat Sci 2019; 50:S32-S41. [DOI: 10.1016/j.jmir.2019.07.010] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Accepted: 07/22/2019] [Indexed: 12/19/2022]
|
60
|
Wang S, Wang T, Yang L, Yang DM, Fujimoto J, Yi F, Luo X, Yang Y, Yao B, Lin S, Moran C, Kalhor N, Weissferdt A, Minna J, Xie Y, Wistuba II, Mao Y, Xiao G. ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network. EBioMedicine 2019; 50:103-110. [PMID: 31767541 PMCID: PMC6921240 DOI: 10.1016/j.ebiom.2019.10.033] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2019] [Revised: 10/16/2019] [Accepted: 10/16/2019] [Indexed: 12/20/2022] Open
Abstract
BACKGROUND The spatial distributions of different types of cells could reveal a cancer cell's growth pattern, its relationships with the tumor microenvironment and the immune response of the body, all of which represent key "hallmarks of cancer". However, the process by which pathologists manually recognize and localize all the cells in pathology slides is extremely labor intensive and error prone. METHODS In this study, we developed an automated cell type classification pipeline, ConvPath, which includes nuclei segmentation, convolutional neural network-based tumor cell, stromal cell, and lymphocyte classification, and extraction of tumor microenvironment-related features for lung cancer pathology images. To facilitate users in leveraging this pipeline for their research, all source scripts for ConvPath software are available at https://qbrc.swmed.edu/projects/cnn/. FINDINGS The overall classification accuracy was 92.9% and 90.1% in training and independent testing datasets, respectively. By identifying cells and classifying cell types, this pipeline can convert a pathology image into a "spatial map" of tumor, stromal and lymphocyte cells. From this spatial map, we can extract features that characterize the tumor micro-environment. Based on these features, we developed an image feature-based prognostic model and validated the model in two independent cohorts. The predicted risk group serves as an independent prognostic factor, after adjusting for clinical variables that include age, gender, smoking status, and stage. INTERPRETATION The analysis pipeline developed in this study could convert the pathology image into a "spatial map" of tumor cells, stromal cells and lymphocytes. This could greatly facilitate and empower comprehensive analysis of the spatial organization of cells, as well as their roles in tumor progression and metastasis.
Collapse
Affiliation(s)
- Shidan Wang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - Tao Wang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX; Center for the Genetics of Host Defense, University of Texas Southwestern Medical Center, Dallas, TX
| | - Lin Yang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX; Department of Pathology, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences (CHCAMS), China
| | - Donghan M Yang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - Junya Fujimoto
- Department of Translational Molecular Pathology, University of Texas MD Anderson Cancer Center, Houston, TX
| | - Faliu Yi
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - Xin Luo
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - Yikun Yang
- Department of Thoracic Surgery, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences (CHCAMS), China
| | - Bo Yao
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - ShinYi Lin
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX
| | - Cesar Moran
- Department of Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Neda Kalhor
- Department of Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Annikka Weissferdt
- Department of Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - John Minna
- Hamon Center for Therapeutic Oncology Research, Department of Internal Medicine and Department of Pharmacology, University of Texas Southwestern Medical Center, Dallas, TX; Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX
| | - Yang Xie
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX; Department of Bioinformatics, University of Texas Southwestern Medical Center, Dallas, TX; Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX
| | - Ignacio I Wistuba
- Department of Translational Molecular Pathology, University of Texas MD Anderson Cancer Center, Houston, TX
| | - Yousheng Mao
- Department of Thoracic Surgery, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences (CHCAMS), China
| | - Guanghua Xiao
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX; Department of Bioinformatics, University of Texas Southwestern Medical Center, Dallas, TX; Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX.
| |
Collapse
|
61
|
Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 2019; 16:703-715. [PMID: 31399699 PMCID: PMC6880861 DOI: 10.1038/s41571-019-0252-y] [Citation(s) in RCA: 643] [Impact Index Per Article: 128.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/04/2019] [Indexed: 02/06/2023]
Abstract
In the past decade, advances in precision oncology have resulted in an increased demand for predictive assays that enable the selection and stratification of patients for treatment. The enormous divergence of signalling and transcriptional networks mediating the crosstalk between cancer, stromal and immune cells complicates the development of functionally relevant biomarkers based on a single gene or protein. However, the result of these complex processes can be uniquely captured in the morphometric features of stained tissue specimens. The possibility of digitizing whole-slide images of tissue has led to the advent of artificial intelligence (AI) and machine learning tools in digital pathology, which enable mining of subvisual morphometric phenotypes and might, ultimately, improve patient management. In this Perspective, we critically evaluate various AI-based computational approaches for digital pathology, focusing on deep neural networks and 'hand-crafted' feature-based methodologies. We aim to provide a broad framework for incorporating AI and machine learning tools into clinical oncology, with an emphasis on biomarker development. We discuss some of the challenges relating to the use of AI, including the need for well-curated validation datasets, regulatory approval and fair reimbursement strategies. Finally, we present potential future opportunities for precision oncology.
Collapse
Affiliation(s)
- Kaustav Bera
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
| | - Kurt A Schalper
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - David L Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Vamsidhar Velcheti
- Thoracic Medical Oncology, Perlmutter Cancer Center, New York University, New York, NY, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA.
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA.
| |
Collapse
|
62
|
Wang S, Yang DM, Rong R, Zhan X, Fujimoto J, Liu H, Minna J, Wistuba II, Xie Y, Xiao G. Artificial Intelligence in Lung Cancer Pathology Image Analysis. Cancers (Basel) 2019; 11:E1673. [PMID: 31661863 PMCID: PMC6895901 DOI: 10.3390/cancers11111673] [Citation(s) in RCA: 97] [Impact Index Per Article: 19.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 10/17/2019] [Accepted: 10/21/2019] [Indexed: 12/12/2022] Open
Abstract
OBJECTIVE Accurate diagnosis and prognosis are essential in lung cancer treatment selection and planning. With the rapid advance of medical imaging technology, whole slide imaging (WSI) in pathology is becoming a routine clinical procedure. An interplay of needs and challenges exists for computer-aided diagnosis based on accurate and efficient analysis of pathology images. Recently, artificial intelligence, especially deep learning, has shown great potential in pathology image analysis tasks such as tumor region identification, prognosis prediction, tumor microenvironment characterization, and metastasis detection. MATERIALS AND METHODS In this review, we aim to provide an overview of current and potential applications for AI methods in pathology image analysis, with an emphasis on lung cancer. RESULTS We outlined the current challenges and opportunities in lung cancer pathology image analysis, discussed the recent deep learning developments that could potentially impact digital pathology in lung cancer, and summarized the existing applications of deep learning algorithms in lung cancer diagnosis and prognosis. DISCUSSION AND CONCLUSION With the advance of technology, digital pathology could have great potential impacts in lung cancer patient care. We point out some promising future directions for lung cancer pathology image analysis, including multi-task learning, transfer learning, and model interpretation.
Collapse
Affiliation(s)
- Shidan Wang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Donghan M Yang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Ruichen Rong
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Xiaowei Zhan
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Junya Fujimoto
- Department of Translational Molecular Pathology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA.
| | - Hongyu Liu
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - John Minna
- Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
- Hamon Center for Therapeutic Oncology Research, UT Southwestern Medical Center, Dallas, TX 75390, USA.
- Departments of Internal Medicine and Pharmacology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Ignacio Ivan Wistuba
- Department of Translational Molecular Pathology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA.
| | - Yang Xie
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
- Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
- Department of Bioinformatics, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Guanghua Xiao
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
- Harold C. Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
- Department of Bioinformatics, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| |
Collapse
|
63
|
Turner OC, Aeffner F, Bangari DS, High W, Knight B, Forest T, Cossic B, Himmel LE, Rudmann DG, Bawa B, Muthuswamy A, Aina OH, Edmondson EF, Saravanan C, Brown DL, Sing T, Sebastian MM. Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group Article*: Opinion on the Application of Artificial Intelligence and Machine Learning to Digital Toxicologic Pathology. Toxicol Pathol 2019; 48:277-294. [DOI: 10.1177/0192623319881401] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Toxicologic pathology is transitioning from analog to digital methods. This transition seems inevitable due to a host of ongoing social and medical technological forces. Of these, artificial intelligence (AI) and in particular machine learning (ML) are globally disruptive, rapidly growing sectors of technology whose impact on the long-established field of histopathology is quickly being realized. The development of increasing numbers of algorithms, peering ever deeper into the histopathological space, has demonstrated to the scientific community that AI pathology platforms are now poised to truly impact the future of precision and personalized medicine. However, as with all great technological advances, there are implementation and adoption challenges. This review aims to define common and relevant AI and ML terminology, describe data generation and interpretation, outline current and potential future business cases, discuss validation and regulatory hurdles, and most importantly, propose how overcoming the challenges of this burgeoning technology may shape toxicologic pathology for years to come, enabling pathologists to contribute even more effectively to answering scientific questions and solving global health issues. [Box: see text]
Collapse
Affiliation(s)
- Oliver C. Turner
- Novartis, Novartis Institutes for Biomedical Research, Preclinical Safety, East Hanover, NJ, USA
| | - Famke Aeffner
- Amgen Inc, Research, Comparative Biology and Safety Sciences, San Francisco, CA, USA
| | | | - Wanda High
- High Preclinical Pathology Consulting, Rochester, NY, USA
| | - Brian Knight
- Boehringer Ingelheim Pharmaceuticals Incorporated, Nonclinical Drug Safety, Ridgefield, CT, USA
| | | | - Brieuc Cossic
- Roche, Pharmaceutical Research and Early Development (pRED), Roche Innovation Center, Basel, Switzerland
| | - Lauren E. Himmel
- Division of Animal Care, Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, TN, USA
| | | | | | | | | | - Elijah F. Edmondson
- Pathology/Histotechnology Laboratory, Frederick National Laboratory for Cancer Research, NIH, Frederick, MD, USA
| | - Chandrassegar Saravanan
- Novartis, Novartis Institutes for Biomedical Research, Preclinical Safety, Cambridge, MA, USA
| | | | - Tobias Sing
- Novartis, Novartis Institutes for Biomedical Research, NIBR Informatics, Basel, Switzerland
| | - Manu M. Sebastian
- Department of Epigenetics and Molecular Carcinogenesis, The University of Texas MD Anderson Cancer Center, Smithville, TX, USA
| |
Collapse
|
64
|
Li H, Whitney J, Bera K, Gilmore H, Thorat MA, Badve S, Madabhushi A. Quantitative nuclear histomorphometric features are predictive of Oncotype DX risk categories in ductal carcinoma in situ: preliminary findings. Breast Cancer Res 2019; 21:114. [PMID: 31623652 PMCID: PMC6798488 DOI: 10.1186/s13058-019-1200-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Accepted: 09/13/2019] [Indexed: 01/23/2023] Open
Abstract
Background Oncotype DX (ODx) is a 12-gene assay assessing the recurrence risk (high, intermediate, and low) of ductal carcinoma in situ (pre-invasive breast cancer), which guides clinicians regarding prescription of radiotherapy. However, ODx is expensive, time-consuming, and tissue-destructive. In addition, the actual prognostic meaning for the intermediate ODx risk category remains unclear. Methods In this work, we evaluated the ability of quantitative nuclear histomorphometric features extracted from hematoxylin and eosin-stained slide images of 62 ductal carcinoma in situ (DCIS) patients to distinguish between the corresponding ODx risk categories. The prognostic value of the identified image signature was further evaluated on an independent validation set of 30 DCIS patients in its ability to distinguish those DCIS patients who progressed to invasive carcinoma versus those who did not. Following nuclear segmentation and feature extraction, feature ranking strategies were employed to identify the most discriminating features between individual ODx risk categories. The selected features were then combined with machine learning classifiers to establish models to predict ODx risk categories. The model performance was evaluated using the average area under the receiver operating characteristic curve (AUC) using cross validation. In addition, an unsupervised clustering approach was also implemented to evaluate the ability of nuclear histomorphometric features to discriminate between the ODx risk categories. Results Features relating to spatial distribution, orientation disorder, and texture of nuclei were identified as most discriminating between the high ODx and the intermediate, low ODx risk categories. Additionally, the AUC of the most discriminating set of features for the different classification tasks was as follows: (1) high vs low ODx (0.68), (2) high vs. intermediate ODx (0.67), (3) intermediate vs. low ODx (0.57), (4) high and intermediate vs. low ODx (0.63), (5) high vs. low and intermediate ODx (0.66). Additionally, the unsupervised clustering resulted in intermediate ODx risk category patients being co-clustered with low ODx patients compared to high ODx. Conclusion Our results appear to suggest that nuclear histomorphometric features can distinguish high from low and intermediate ODx risk category patients. Additionally, our findings suggest that histomorphometric features for intermediate ODx were more similar to low ODx compared to high ODx risk category.
Collapse
Affiliation(s)
- Haojia Li
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA.
| | - Jon Whitney
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
| | - Kaustav Bera
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
| | - Hannah Gilmore
- University Hospitals Cleveland Medical Center, Cleveland, OH, USA
| | - Mangesh A Thorat
- Centre for Cancer Prevention, Wolfson Institute of Preventive Medicine, Queen Mary University of London, London, UK.,School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences & Medicine, King's College London, London, UK
| | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Indiana University, Indianapolis, IN, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
| |
Collapse
|
65
|
Alanni R, Hou J, Azzawi H, Xiang Y. Cancer adjuvant chemotherapy prediction model for non‐small cell lung cancer. IET Syst Biol 2019; 13:129-135. [DOI: 10.1049/iet-syb.2018.5060] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023] Open
Affiliation(s)
- Russul Alanni
- School of Information Technology, Deakin UniversityBurwoodAustralia
| | - Jingyu Hou
- School of Information Technology, Deakin UniversityBurwoodAustralia
| | - Hasseeb Azzawi
- School of Information Technology, Deakin UniversityBurwoodAustralia
| | - Yong Xiang
- School of Information Technology, Deakin UniversityBurwoodAustralia
| |
Collapse
|
66
|
Accuracy Enhanced Lung Cancer Prognosis for Improving Patient Survivability Using Proposed Gaussian Classifier System. J Med Syst 2019; 43:201. [DOI: 10.1007/s10916-019-1297-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2019] [Accepted: 04/16/2019] [Indexed: 10/26/2022]
|
67
|
Ji MY, Yuan L, Jiang XD, Zeng Z, Zhan N, Huang PX, Lu C, Dong WG. Nuclear shape, architecture and orientation features from H&E images are able to predict recurrence in node-negative gastric adenocarcinoma. J Transl Med 2019; 17:92. [PMID: 30885234 PMCID: PMC6423755 DOI: 10.1186/s12967-019-1839-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Accepted: 03/08/2019] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Identifying intestinal node-negative gastric adenocarcinoma (INGA) patients with high risk of recurrence could help perceive benefit of adjuvant therapy for INGA patients following surgical resection. This study evaluated whether the computer-extracted image features of nuclear shapes, texture, orientation, and tumor architecture on digital images of hematoxylin and eosin stained tissue, could help to predict recurrence in INGA patients. METHODS A tissue microarrays cohort of 160 retrospectively INGA cases were digitally scanned, and randomly selected as training cohort (D1 = 60), validation cohort (D2 = 100 and D3 = 100, D2 and D3 are different tumor TMA spots from the same patient), accompanied with immunohistochemistry data cohort (D3' = 100, a duplicate cohort of D3) and negative controls data cohort (D5 = 100, normal adjacent tissues). After nuclear segmentation by watershed-based method, 189 local nuclear features were captured on each TMA core and the top 5 features were selected by Wilcoxon rank sum test within D1. A morphometric-based image classifier (NGAHIC) was composed across the discriminative features and predicted the recurrence in INGA on D2. The intra-tumor heterogeneity was assessed on D3. Manual nuclear atypia grading was conducted on D1 and D2 by two pathologists. The expression of HER2 and Ki67 were detected by immunohistochemistry on D3 and D3', respectively. The association between manual grading and INGA outcome was analysis. RESULTS Independent validation results showed the NGAHIC achieved an AUC of 0.76 for recurrence prediction. NGAHIC-positive patients had poorer overall survival (P = 0.017) by univariate survival analysis. Multivariate survival analysis, controlling for T-stage, histology stage, invasion depth, demonstrated NGAHIC-positive was a reproducible prognostic factor for poorer disease-specific survival (HR = 17.24, 95% CI 3.93-75.60, P < 0.001). In contrast, human grading was only prognostic for one reader on D2. Moreover, significant correlations were observed between NGAHIC-positive patients and positivity of HER2 and Ki67 labeling index. CONCLUSIONS The NGAHIC could provide precision oncology, personalized cancer management.
Collapse
Affiliation(s)
- Meng-Yao Ji
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Lei Yuan
- Department of Information Center, Wuhan University Renmin Hospital, Wuhan, Hubei, China.
| | - Xiao-Da Jiang
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Zhi Zeng
- Department of Pathology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Na Zhan
- Department of Pathology, Wuhan University Renmin Hospital, Wuhan, Hubei, China
| | - Ping-Xiao Huang
- Department of Gastroenterology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430060, China
| | - Cheng Lu
- College of Computer Science, Shaanxi Normal University, Xi'an, Shaanxi, China.
| | - Wei-Guo Dong
- Department of Gastroenterology, Wuhan University Renmin Hospital, Wuhan, Hubei, China.
| |
Collapse
|
68
|
Corredor G, Wang X, Zhou Y, Lu C, Fu P, Syrigos K, Rimm DL, Yang M, Romero E, Schalper KA, Velcheti V, Madabhushi A. Spatial Architecture and Arrangement of Tumor-Infiltrating Lymphocytes for Predicting Likelihood of Recurrence in Early-Stage Non-Small Cell Lung Cancer. Clin Cancer Res 2019; 25:1526-1534. [PMID: 30201760 PMCID: PMC6397708 DOI: 10.1158/1078-0432.ccr-18-2013] [Citation(s) in RCA: 141] [Impact Index Per Article: 28.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2018] [Revised: 08/16/2018] [Accepted: 09/06/2018] [Indexed: 12/16/2022]
Abstract
PURPOSE The presence of a high degree of tumor-infiltrating lymphocytes (TIL) has been proven to be associated with outcome in patients with non-small cell lung cancer (NSCLC). However, recent evidence indicates that tissue architecture is also prognostic of disease-specific survival and recurrence. We show a set of descriptors (spatial TIL, SpaTIL) that capture density, and spatial colocalization of TILs and tumor cells across digital images that can predict likelihood of recurrence in early-stage NSCLC. EXPERIMENTAL DESIGN The association between recurrence in early-stage NSCLC and SpaTIL features was explored on 301 patients across four different cohorts. Cohort D1 (n = 70) was used to identify the most prognostic SpaTIL features and to train a classifier to predict the likelihood of recurrence. The classifier performance was evaluated in cohorts D2 (n = 119), D3 (n = 112), and D4 (n = 112). Two pathologists graded each sample of D1 and D2; intraobserver agreement and association between manual grading and likelihood of recurrence were analyzed. RESULTS SpaTIL was associated with likelihood of recurrence in all test sets (log-rank P < 0.02). A multivariate Cox proportional hazards analysis revealed an HR of 3.08 (95% confidence interval, 2.1-4.5, P = 7.3 × 10-5). In contrast, agreement among expert pathologists using tumor grade was moderate (Kappa = 0.5), and the manual TIL grading was only prognostic for one reader in D2 (P = 8.0 × 10-3). CONCLUSIONS A set of features related to density and spatial architecture of TILs was found to be associated with a likelihood of recurrence of early-stage NSCLC. This information could potentially be used for helping in treatment planning and management of early-stage NSCLC.See related commentary by Peled et al., p. 1449.
Collapse
Affiliation(s)
- Germán Corredor
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, Ohio
- Computer Imaging and Medical Applications Laboratory, Universidad Nacional de Colombia, Bogotá, Colombia
| | - Xiangxue Wang
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, Ohio
| | - Yu Zhou
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, Ohio
| | - Cheng Lu
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, Ohio
| | - Pingfu Fu
- Department of Population and Quantitative Health Sciences, Case Western Reserve University, Cleveland, Ohio
| | - Konstantinos Syrigos
- Department of Medicine, University of Athens, Sotiria General Hospital, Athens, Greece
| | - David L Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, Connecticut
| | - Michael Yang
- Department of Pathology-Anatomic, University Hospitals, Cleveland, Ohio
| | - Eduardo Romero
- Computer Imaging and Medical Applications Laboratory, Universidad Nacional de Colombia, Bogotá, Colombia
| | - Kurt A Schalper
- Department of Pathology, Yale University School of Medicine, New Haven, Connecticut
| | - Vamsidhar Velcheti
- Hematology and Medical Oncology Department, Cleveland Clinic, Cleveland, Ohio
| | - Anant Madabhushi
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, Ohio.
| |
Collapse
|
69
|
Teramoto A, Yamada A, Kiriyama Y, Tsukamoto T, Yan K, Zhang L, Imaizumi K, Saito K, Fujita H. Automated classification of benign and malignant cells from lung cytological images using deep convolutional neural network. INFORMATICS IN MEDICINE UNLOCKED 2019. [DOI: 10.1016/j.imu.2019.100205] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022] Open
|
70
|
Hosny A, Parmar C, Coroller TP, Grossmann P, Zeleznik R, Kumar A, Bussink J, Gillies RJ, Mak RH, Aerts HJWL. Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study. PLoS Med 2018; 15:e1002711. [PMID: 30500819 PMCID: PMC6269088 DOI: 10.1371/journal.pmed.1002711] [Citation(s) in RCA: 310] [Impact Index Per Article: 51.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/29/2018] [Accepted: 11/05/2018] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Non-small-cell lung cancer (NSCLC) patients often demonstrate varying clinical courses and outcomes, even within the same tumor stage. This study explores deep learning applications in medical imaging allowing for the automated quantification of radiographic characteristics and potentially improving patient stratification. METHODS AND FINDINGS We performed an integrative analysis on 7 independent datasets across 5 institutions totaling 1,194 NSCLC patients (age median = 68.3 years [range 32.5-93.3], survival median = 1.7 years [range 0.0-11.7]). Using external validation in computed tomography (CT) data, we identified prognostic signatures using a 3D convolutional neural network (CNN) for patients treated with radiotherapy (n = 771, age median = 68.0 years [range 32.5-93.3], survival median = 1.3 years [range 0.0-11.7]). We then employed a transfer learning approach to achieve the same for surgery patients (n = 391, age median = 69.1 years [range 37.2-88.0], survival median = 3.1 years [range 0.0-8.8]). We found that the CNN predictions were significantly associated with 2-year overall survival from the start of respective treatment for radiotherapy (area under the receiver operating characteristic curve [AUC] = 0.70 [95% CI 0.63-0.78], p < 0.001) and surgery (AUC = 0.71 [95% CI 0.60-0.82], p < 0.001) patients. The CNN was also able to significantly stratify patients into low and high mortality risk groups in both the radiotherapy (p < 0.001) and surgery (p = 0.03) datasets. Additionally, the CNN was found to significantly outperform random forest models built on clinical parameters-including age, sex, and tumor node metastasis stage-as well as demonstrate high robustness against test-retest (intraclass correlation coefficient = 0.91) and inter-reader (Spearman's rank-order correlation = 0.88) variations. To gain a better understanding of the characteristics captured by the CNN, we identified regions with the most contribution towards predictions and highlighted the importance of tumor-surrounding tissue in patient stratification. We also present preliminary findings on the biological basis of the captured phenotypes as being linked to cell cycle and transcriptional processes. Limitations include the retrospective nature of this study as well as the opaque black box nature of deep learning networks. CONCLUSIONS Our results provide evidence that deep learning networks may be used for mortality risk stratification based on standard-of-care CT images from NSCLC patients. This evidence motivates future research into better deciphering the clinical and biological basis of deep learning networks as well as validation in prospective data.
Collapse
Affiliation(s)
- Ahmed Hosny
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Chintan Parmar
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Thibaud P. Coroller
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Patrick Grossmann
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Roman Zeleznik
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Avnish Kumar
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Johan Bussink
- Department of Radiation Oncology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Robert J. Gillies
- Department of Cancer Physiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, United States of America
| | - Raymond H. Mak
- Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Hugo J. W. L. Aerts
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
- Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
| |
Collapse
|
71
|
A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis. NPJ Digit Med 2018; 1:52. [PMID: 31304331 PMCID: PMC6550189 DOI: 10.1038/s41746-018-0057-x] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Revised: 08/22/2018] [Accepted: 09/04/2018] [Indexed: 12/11/2022] Open
Abstract
Accurate prognosis is fundamental in planning an appropriate therapy for cancer patients. Consequent to the heterogeneity of the disease, intra- and inter-pathologist variability, and the inherent limitations of current pathological reporting systems, patient outcome varies considerably within similarly staged patient cohorts. This is particularly true when classifying stage II colorectal cancer patients using the current TNM guidelines. The aim of the present work is to address this problem through the use of machine learning. In particular, we introduce a data driven framework which makes use of a large number of diverse types of features, readily collected from immunofluorescence imagery. Its outstanding performance in predicting mortality in stage II patients (AUROC = 0:94), exceeds that of current clinical guidelines such as pT stage (AUROC = 0:65), and is demonstrated on a cohort of 173 colorectal cancer patients.
Collapse
|
72
|
Seshadri DR, Ramamurthi A. Nanotherapeutics to Modulate the Compromised Micro-Environment for Lung Cancers and Chronic Obstructive Pulmonary Disease. Front Pharmacol 2018; 9:759. [PMID: 30061830 PMCID: PMC6054931 DOI: 10.3389/fphar.2018.00759] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2017] [Accepted: 06/22/2018] [Indexed: 11/13/2022] Open
Abstract
The use of nanomaterials to modulate the tumor microenvironment has great potential to advance outcomes in patients with lung cancer. Nanomaterials can be used to prolong the delivery time of therapeutics enabling their specific targeting to tumors while minimizing and potentially eliminating cytotoxic effects. Using nanomaterials to deliver small-molecule inhibitors for oncogene targeted therapy and cancer immunotherapy while concurrently enabling regeneration of the extracellular matrix could enhance our therapeutic reach and improve outcomes for patients with non-small cell lung cancer (NSCLC) and chronic obstructive pulmonary disease (COPD). The objective of this review is to highlight the role nanomedicines play in improving and reversing adverse outcomes in the tumor microenvironment for advancing treatments for targeting both diseases.
Collapse
Affiliation(s)
- Dhruv R. Seshadri
- Department of Biomedical Engineering, Cleveland Clinic, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Anand Ramamurthi
- Department of Biomedical Engineering, Cleveland Clinic, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| |
Collapse
|
73
|
Feature Driven Local Cell Graph (FeDeG): Predicting Overall Survival in Early Stage Lung Cancer. MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION – MICCAI 2018 2018. [DOI: 10.1007/978-3-030-00934-2_46] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|