1
|
Botros M, de Boer OJ, Cardenas B, Bekkers EJ, Jansen M, van der Wel MJ, Sánchez CI, Meijer SL. Deep Learning for Histopathological Assessment of Esophageal Adenocarcinoma Precursor Lesions. Mod Pathol 2024; 37:100531. [PMID: 38830407 DOI: 10.1016/j.modpat.2024.100531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 05/06/2024] [Accepted: 05/28/2024] [Indexed: 06/05/2024]
Abstract
Histopathological assessment of esophageal biopsies is a key part in the management of patients with Barrett esophagus (BE) but prone to observer variability and reliable diagnostic methods are needed. Artificial intelligence (AI) is emerging as a powerful tool for aided diagnosis but often relies on abstract test and validation sets while real-world behavior is unknown. In this study, we developed a 2-stage AI system for histopathological assessment of BE-related dysplasia using deep learning to enhance the efficiency and accuracy of the pathology workflow. The AI system was developed and trained on 290 whole-slide images (WSIs) that were annotated at glandular and tissue levels. The system was designed to identify individual glands, grade dysplasia, and assign a WSI-level diagnosis. The proposed method was evaluated by comparing the performance of our AI system with that of a large international and heterogeneous group of 55 gastrointestinal pathologists assessing 55 digitized biopsies spanning the complete spectrum of BE-related dysplasia. The AI system correctly graded 76.4% of the WSIs, surpassing the performance of 53 out of the 55 participating pathologists. Furthermore, the receiver-operating characteristic analysis showed that the system's ability to predict the absence (nondysplastic BE) versus the presence of any dysplasia was with an area under the curve of 0.94 and a sensitivity of 0.92 at a specificity of 0.94. These findings demonstrate that this AI system has the potential to assist pathologists in assessment of BE-related dysplasia. The system's outputs could provide a reliable and consistent secondary diagnosis in challenging cases or be used for triaging low-risk nondysplastic biopsies, thereby reducing the workload of pathologists and increasing throughput.
Collapse
Affiliation(s)
- Michel Botros
- Department of Pathology, Amsterdam University Medical Centers, Amsterdam, The Netherlands; Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, Amsterdam, The Netherlands; Quantitative Healthcare Analysis Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands; Amsterdam Machine Learning Lab, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
| | - Onno J de Boer
- Department of Pathology, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Bryan Cardenas
- Department of Pathology, Amsterdam University Medical Centers, Amsterdam, The Netherlands; Amsterdam Machine Learning Lab, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
| | - Erik J Bekkers
- Amsterdam Machine Learning Lab, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
| | - Marnix Jansen
- Research Department of Pathology, Cancer Institute, University College London, London, United Kingdom
| | - Myrtle J van der Wel
- Department of Pathology, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Clara I Sánchez
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, Amsterdam, The Netherlands; Quantitative Healthcare Analysis Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
| | - Sybren L Meijer
- Department of Pathology, Amsterdam University Medical Centers, Amsterdam, The Netherlands.
| |
Collapse
|
2
|
Alajaji SA, Khoury ZH, Jessri M, Sciubba JJ, Sultan AS. An Update on the Use of Artificial Intelligence in Digital Pathology for Oral Epithelial Dysplasia Research. Head Neck Pathol 2024; 18:38. [PMID: 38727841 PMCID: PMC11087425 DOI: 10.1007/s12105-024-01643-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 03/30/2024] [Indexed: 05/13/2024]
Abstract
INTRODUCTION Oral epithelial dysplasia (OED) is a precancerous histopathological finding which is considered the most important prognostic indicator for determining the risk of malignant transformation into oral squamous cell carcinoma (OSCC). The gold standard for diagnosis and grading of OED is through histopathological examination, which is subject to inter- and intra-observer variability, impacting accurate diagnosis and prognosis. The aim of this review article is to examine the current advances in digital pathology for artificial intelligence (AI) applications used for OED diagnosis. MATERIALS AND METHODS We included studies that used AI for diagnosis, grading, or prognosis of OED on histopathology images or intraoral clinical images. Studies utilizing imaging modalities other than routine light microscopy (e.g., scanning electron microscopy), or immunohistochemistry-stained histology slides, or immunofluorescence were excluded from the study. Studies not focusing on oral dysplasia grading and diagnosis, e.g., to discriminate OSCC from normal epithelial tissue were also excluded. RESULTS A total of 24 studies were included in this review. Nineteen studies utilized deep learning (DL) convolutional neural networks for histopathological OED analysis, and 4 used machine learning (ML) models. Studies were summarized by AI method, main study outcomes, predictive value for malignant transformation, strengths, and limitations. CONCLUSION ML/DL studies for OED grading and prediction of malignant transformation are emerging as promising adjunctive tools in the field of digital pathology. These adjunctive objective tools can ultimately aid the pathologist in more accurate diagnosis and prognosis prediction. However, further supportive studies that focus on generalization, explainable decisions, and prognosis prediction are needed.
Collapse
Affiliation(s)
- Shahd A Alajaji
- Department of Oncology and Diagnostic Sciences, University of Maryland School of Dentistry, 650 W. Baltimore Street, 7 Floor, Baltimore, MD, 21201, USA
- Department of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia
- Division of Artificial Intelligence Research, University of Maryland School of Dentistry, Baltimore, MD, USA
| | - Zaid H Khoury
- Department of Oral Diagnostic Sciences and Research, Meharry Medical College School of Dentistry, Nashville, TN, USA
| | - Maryam Jessri
- Oral Medicine and Pathology Department, School of Dentistry, University of Queensland, Herston, QLD, Australia
- Oral Medicine Department, Metro North Hospital and Health Services, Queensland Health, Brisbane, QLD, Australia
| | - James J Sciubba
- Department of Otolaryngology, Head & Neck Surgery, The Johns Hopkins University, Baltimore, MD, USA
| | - Ahmed S Sultan
- Department of Oncology and Diagnostic Sciences, University of Maryland School of Dentistry, 650 W. Baltimore Street, 7 Floor, Baltimore, MD, 21201, USA.
- Division of Artificial Intelligence Research, University of Maryland School of Dentistry, Baltimore, MD, USA.
- University of Maryland Marlene and Stewart Greenebaum Comprehensive Cancer Center, Baltimore, MD, USA.
| |
Collapse
|
3
|
Duncan KD, Pětrošová H, Lum JJ, Goodlett DR. Mass spectrometry imaging methods for visualizing tumor heterogeneity. Curr Opin Biotechnol 2024; 86:103068. [PMID: 38310648 DOI: 10.1016/j.copbio.2024.103068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 01/09/2024] [Accepted: 01/09/2024] [Indexed: 02/06/2024]
Abstract
Profiling spatial distributions of lipids, metabolites, and proteins in tumors can reveal unique cellular microenvironments and provide molecular evidence for cancer cell dysfunction and proliferation. Mass spectrometry imaging (MSI) is a label-free technique that can be used to map biomolecules in tumors in situ. Here, we discuss current progress in applying MSI to uncover molecular heterogeneity in tumors. First, the analytical strategies to profile small molecules and proteins are outlined, and current methods for multimodal imaging to maximize biological information are highlighted. Second, we present and summarize biological insights obtained by MSI of tumor tissue. Finally, we discuss important considerations for designing MSI experiments and several current analytical challenges.
Collapse
Affiliation(s)
- Kyle D Duncan
- Department of Chemistry, Vancouver Island University, Nanaimo, British Columbia, Canada; Department of Chemistry, University of Victoria, Victoria, British Columbia, Canada.
| | - Helena Pětrošová
- University of Victoria Genome British Columbia Proteomics Center, University of Victoria, Victoria, British Columbia, Canada; Department of Biochemistry and Microbiology, University of Victoria, Victoria, British Columbia, Canada.
| | - Julian J Lum
- Department of Biochemistry and Microbiology, University of Victoria, Victoria, British Columbia, Canada; Trev and Joyce Deeley Research Centre, BC Cancer, Victoria, British Columbia, Canada
| | - David R Goodlett
- University of Victoria Genome British Columbia Proteomics Center, University of Victoria, Victoria, British Columbia, Canada; Department of Biochemistry and Microbiology, University of Victoria, Victoria, British Columbia, Canada
| |
Collapse
|
4
|
Bi S, Wang M, Pu Q, Yang J, Jiang N, Zhao X, Qiu S, Liu R, Xu R, Li X, Hu C, Yang L, Gu J, Du D. Multi-MSIProcessor: Data Visualizing and Analysis Software for Spatial Metabolomics Research. Anal Chem 2024; 96:339-346. [PMID: 38102989 DOI: 10.1021/acs.analchem.3c04192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2023]
Abstract
Mass spectrometry imaging (MSI) has emerged as a revolutionary analytical strategy in biomedical research for molecular visualization. By linking the characterization of functional metabolites with tissue architecture, it is now possible to reveal unknown biological functions of tissues. However, due to the complexity and high dimensionality of MSI data, mining bioinformatics-related peaks from batch MSI data sets and achieving complete spatially resolved metabolomics analysis remain a great challenge. Here, we propose novel MSI data processing software, Multi-MSIProcessor (MMP), which integrates the data read-in, MSI visualization, processed data preservation, and biomarker discovery functions. The MMP focuses on the AFADESI-MSI data platform but also supports mzXML and imzmL data input formats for compatibility with data generated by other MSI platforms such as MALDI/SIMS-MSI. MMP enables deep mining of batch MSI data and has flexible adaptability with the source code opened that welcomes new functions and personalized analysis strategies. Using multiple clinical biosamples with complex heterogeneity, we demonstrated that MMP can rapidly establish complete MSI analysis workflows, assess batch sample data quality, screen and annotate differential MS peaks, and obtain abnormal metabolic pathways. MMP provides a novel platform for spatial metabolomics analysis of multiple samples that could meet the diverse analysis requirements of scholars.
Collapse
Affiliation(s)
- Siwei Bi
- Department of Plastic and Burn Surgery, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Manjiangcuo Wang
- Advanced Mass Spectrometry Center, Research Core Facility, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu 610213, China
| | - Qianlun Pu
- Advanced Mass Spectrometry Center, Research Core Facility, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu 610213, China
| | - Jinxi Yang
- West China Centre of Excellence for Pancreatitis, Institute of Integrated Traditional Chinese and Western Medicine, West China Hospital/West China Medical School, Sichuan University, Chengdu 610041, China
| | - Na Jiang
- Advanced Mass Spectrometry Center, Research Core Facility, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu 610213, China
| | - Xueshan Zhao
- Department of Cardiovascular Surgery, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Siyuan Qiu
- Division of Gastrointestinal Surgery, Department of General Surgery, West China Hospital, Sichuan University, Chengdu,Sichuan 610041, China
| | - Ruiqi Liu
- Department of Plastic and Burn Surgery, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Renjie Xu
- Department of Respiratory Health West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Xia Li
- West China School of Nursing, Sichuan University, Chengdu, Sichuan 610041, China
- Department of Critical Care Medicine, West China Hospital, Sichuan University, Chengdu 610041, China
| | - Chenggong Hu
- Department of Critical Care Medicine, West China Hospital, Sichuan University, Chengdu 610041, China
| | - Lie Yang
- Division of Gastrointestinal Surgery, Department of General Surgery, West China Hospital, Sichuan University, Chengdu,Sichuan 610041, China
| | - Jun Gu
- Department of Cardiovascular Surgery, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Dan Du
- Advanced Mass Spectrometry Center, Research Core Facility, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu 610213, China
| |
Collapse
|
5
|
Krestensen KK, Heeren RMA, Balluff B. State-of-the-art mass spectrometry imaging applications in biomedical research. Analyst 2023; 148:6161-6187. [PMID: 37947390 DOI: 10.1039/d3an01495a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
Mass spectrometry imaging has advanced from a niche technique to a widely applied spatial biology tool operating at the forefront of numerous fields, most notably making a significant impact in biomedical pharmacological research. The growth of the field has gone hand in hand with an increase in publications and usage of the technique by new laboratories, and consequently this has led to a shift from general MSI reviews to topic-specific reviews. Given this development, we see the need to recapitulate the strengths of MSI by providing a more holistic overview of state-of-the-art MSI studies to provide the new generation of researchers with an up-to-date reference framework. Here we review scientific advances for the six largest biomedical fields of MSI application (oncology, pharmacology, neurology, cardiovascular diseases, endocrinology, and rheumatology). These publications thereby give examples for at least one of the following categories: they provide novel mechanistic insights, use an exceptionally large cohort size, establish a workflow that has the potential to become a high-impact methodology, or are highly cited in their field. We finally have a look into new emerging fields and trends in MSI (immunology, microbiology, infectious diseases, and aging), as applied MSI is continuously broadening as a result of technological breakthroughs.
Collapse
Affiliation(s)
- Kasper K Krestensen
- The Maastricht MultiModal Molecular Imaging (M4I) Institute, Maastricht University, 6229 ER Maastricht, The Netherlands.
| | - Ron M A Heeren
- The Maastricht MultiModal Molecular Imaging (M4I) Institute, Maastricht University, 6229 ER Maastricht, The Netherlands.
| | - Benjamin Balluff
- The Maastricht MultiModal Molecular Imaging (M4I) Institute, Maastricht University, 6229 ER Maastricht, The Netherlands.
| |
Collapse
|
6
|
Cui R, Wang L, Lin L, Li J, Lu R, Liu S, Liu B, Gu Y, Zhang H, Shang Q, Chen L, Tian D. Deep Learning in Barrett's Esophagus Diagnosis: Current Status and Future Directions. Bioengineering (Basel) 2023; 10:1239. [PMID: 38002363 PMCID: PMC10669008 DOI: 10.3390/bioengineering10111239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 10/13/2023] [Accepted: 10/16/2023] [Indexed: 11/26/2023] Open
Abstract
Barrett's esophagus (BE) represents a pre-malignant condition characterized by abnormal cellular proliferation in the distal esophagus. A timely and accurate diagnosis of BE is imperative to prevent its progression to esophageal adenocarcinoma, a malignancy associated with a significantly reduced survival rate. In this digital age, deep learning (DL) has emerged as a powerful tool for medical image analysis and diagnostic applications, showcasing vast potential across various medical disciplines. In this comprehensive review, we meticulously assess 33 primary studies employing varied DL techniques, predominantly featuring convolutional neural networks (CNNs), for the diagnosis and understanding of BE. Our primary focus revolves around evaluating the current applications of DL in BE diagnosis, encompassing tasks such as image segmentation and classification, as well as their potential impact and implications in real-world clinical settings. While the applications of DL in BE diagnosis exhibit promising results, they are not without challenges, such as dataset issues and the "black box" nature of models. We discuss these challenges in the concluding section. Essentially, while DL holds tremendous potential to revolutionize BE diagnosis, addressing these challenges is paramount to harnessing its full capacity and ensuring its widespread application in clinical practice.
Collapse
Affiliation(s)
- Ruichen Cui
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Lei Wang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Lin Lin
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Jie Li
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
- West China School of Nursing, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China
| | - Runda Lu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Shixiang Liu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Bowei Liu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Yimin Gu
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Hanlu Zhang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Qixin Shang
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Longqi Chen
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| | - Dong Tian
- Department of Thoracic Surgery, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu 610041, China; (R.C.); (L.W.); (L.L.); (J.L.); (R.L.); (S.L.); (B.L.); (Y.G.); (H.Z.); (Q.S.)
| |
Collapse
|
7
|
Walid MAA, Mollick S, Shill PC, Baowaly MK, Islam MR, Ahamad MM, Othman MA, Samad MA. Adapted Deep Ensemble Learning-Based Voting Classifier for Osteosarcoma Cancer Classification. Diagnostics (Basel) 2023; 13:3155. [PMID: 37835898 PMCID: PMC10572954 DOI: 10.3390/diagnostics13193155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 10/05/2023] [Accepted: 10/06/2023] [Indexed: 10/15/2023] Open
Abstract
The study utilizes osteosarcoma hematoxylin and the Eosin-stained image dataset, which is unevenly dispersed, and it raises concerns about the potential impact on the overall performance and reliability of any analyses or models derived from the dataset. In this study, a deep-learning-based convolution neural network (CNN) and adapted heterogeneous ensemble-learning-based voting classifier have been proposed to classify osteosarcoma. The proposed methods can also resolve the issue and develop unbiased learning models by introducing an evenly distributed training dataset. Data augmentation is employed to boost the generalization abilities. Six different pre-trained CNN models, namely MobileNetV1, Mo-bileNetV2, ResNetV250, InceptionV2, EfficientNetV2B0, and NasNetMobile, are applied and evaluated in frozen and fine-tuned-based phases. In addition, a novel CNN model and adapted heterogeneous ensemble-learning-based voting classifier developed from the proposed CNN model, fine-tuned NasNetMobile model, and fine-tuned Efficient-NetV2B0 model are also introduced to classify osteosarcoma. The proposed CNN model outperforms other pre-trained models. The Kappa score obtained from the proposed CNN model is 93.09%. Notably, the proposed voting classifier attains the highest Kappa score of 96.50% and outperforms all other models. The findings of this study have practical implications in telemedicine, mobile healthcare systems, and as a supportive tool for medical professionals.
Collapse
Affiliation(s)
- Md. Abul Ala Walid
- Department of Computer Science and Engineering, Khulna University of Engineering and Technology, Khulna 9203, Bangladesh; (M.A.A.W.)
- Department of Computer Science and Engineering, Northern University of Business and Technology, Khulna 9100, Bangladesh
| | - Swarnali Mollick
- Department of Computer Science and Engineering, Northern University of Business and Technology, Khulna 9100, Bangladesh
| | - Pintu Chandra Shill
- Department of Computer Science and Engineering, Khulna University of Engineering and Technology, Khulna 9203, Bangladesh; (M.A.A.W.)
| | - Mrinal Kanti Baowaly
- Department of Computer Science and Engineering, Bangabandhu Sheikh Mujibur Rahman Science and Technology University, Gopalganj 8100, Bangladesh; (M.K.B.)
| | - Md. Rabiul Islam
- Department of Biomedical Engineering, Islamic University, Kushtia 7003, Bangladesh
| | - Md. Martuza Ahamad
- Department of Computer Science and Engineering, Bangabandhu Sheikh Mujibur Rahman Science and Technology University, Gopalganj 8100, Bangladesh; (M.K.B.)
| | - Manal A. Othman
- Medical Education Department, College of Medicine, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia;
| | - Md Abdus Samad
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan-si 38541, Republic of Korea
| |
Collapse
|
8
|
Tolkach Y, Wolgast LM, Damanakis A, Pryalukhin A, Schallenberg S, Hulla W, Eich ML, Schroeder W, Mukhopadhyay A, Fuchs M, Klein S, Bruns C, Büttner R, Gebauer F, Schömig-Markiefka B, Quaas A. Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study. Lancet Digit Health 2023; 5:e265-e275. [PMID: 37100542 DOI: 10.1016/s2589-7500(23)00027-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Revised: 01/18/2023] [Accepted: 02/02/2023] [Indexed: 04/28/2023]
Abstract
BACKGROUND Oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction are among the most common malignant epithelial tumours. Most patients receive neoadjuvant therapy before complete tumour resection. Histological assessment after resection includes identification of residual tumour tissue and areas of regressive tumour, data which are used to calculate a clinically relevant regression score. We developed an artificial intelligence (AI) algorithm for tumour tissue detection and tumour regression grading in surgical specimens from patients with oesophageal adenocarcinoma or adenocarcinoma of the oesophagogastric junction. METHODS We used one training cohort and four independent test cohorts to develop, train, and validate a deep learning tool. The material consisted of histological slides from surgically resected specimens from patients with oesophageal adenocarcinoma and adenocarcinoma of the oesophagogastric junction from three pathology institutes (two in Germany, one in Austria) and oesophageal cancer cohort of The Cancer Genome Atlas (TCGA). All slides were from neoadjuvantly treated patients except for those from the TCGA cohort, who were neoadjuvant-therapy naive. Data from training cohort and test cohort cases were extensively manually annotated for 11 tissue classes. A convolutional neural network was trained on the data using a supervised principle. First, the tool was formally validated using manually annotated test datasets. Next, tumour regression grading was assessed in a retrospective cohort of post-neoadjuvant therapy surgical specimens. The grading of the algorithm was compared with that of a group of 12 board-certified pathologists from one department. To further validate the tool, three pathologists processed whole resection cases with and without AI assistance. FINDINGS Of the four test cohorts, one included 22 manually annotated histological slides (n=20 patients), one included 62 sides (n=15), one included 214 slides (n=69), and the final one included 22 manually annotated histological slides (n=22). In the independent test cohorts the AI tool had high patch-level accuracy for identifying both tumour and regression tissue. When we validated the concordance of the AI tool against analyses by a group of pathologists (n=12), agreement was 63·6% (quadratic kappa 0·749; p<0·0001) at case level. The AI-based regression grading triggered true reclassification of resected tumour slides in seven cases (including six cases who had small tumour regions that were initially missed by pathologists). Use of the AI tool by three pathologists increased interobserver agreement and substantially reduced diagnostic time per case compared with working without AI assistance. INTERPRETATION Use of our AI tool in the diagnostics of oesophageal adenocarcinoma resection specimens by pathologists increased diagnostic accuracy, interobserver concordance, and significantly reduced assessment time. Prospective validation of the tool is required. FUNDING North Rhine-Westphalia state, Federal Ministry of Education and Research of Germany, and the Wilhelm Sander Foundation.
Collapse
Affiliation(s)
- Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany.
| | - Lisa Marie Wolgast
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexander Damanakis
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexey Pryalukhin
- Institute of Pathology, Landesklinikum Wiener Neustadt, Wiener Neustadt, Austria
| | - Simon Schallenberg
- Institute of Pathology, University Hospital Berlin-Charité, Berlin, Germany
| | - Wolfgang Hulla
- Institute of Pathology, Landesklinikum Wiener Neustadt, Wiener Neustadt, Austria
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Wolfgang Schroeder
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Christiane Bruns
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Florian Gebauer
- Department of General, Visceral and Cancer Surgery, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Birgid Schömig-Markiefka
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany; Medical Faculty, University of Cologne, Cologne, Germany.
| |
Collapse
|
9
|
Faghani S, Codipilly DC, David Vogelsang, Moassefi M, Rouzrokh P, Khosravi B, Agarwal S, Dhaliwal L, Katzka DA, Hagen C, Lewis J, Leggett CL, Erickson BJ, Iyer PG. Development of a deep learning model for the histologic diagnosis of dysplasia in Barrett's esophagus. Gastrointest Endosc 2022; 96:918-925.e3. [PMID: 35718071 DOI: 10.1016/j.gie.2022.06.013] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 06/02/2022] [Accepted: 06/08/2022] [Indexed: 12/11/2022]
Abstract
BACKGROUND AND AIMS The risk of progression in Barrett's esophagus (BE) increases with development of dysplasia. There is a critical need to improve the diagnosis of BE dysplasia, given substantial interobserver disagreement among expert pathologists and overdiagnosis of dysplasia by community pathologists. We developed a deep learning model to predict dysplasia grade on whole-slide imaging. METHODS We digitized nondysplastic BE (NDBE), low-grade dysplasia (LGD), and high-grade dysplasia (HGD) histology slides. Two expert pathologists confirmed all histology and digitally annotated areas of dysplasia. Training, validation, and test sets were created (by a random 70/20/10 split). We used an ensemble approach combining a "you only look once" model to identify regions of interest and histology class (NDBE, LGD, or HGD) followed by a ResNet101 model pretrained on ImageNet applied to the regions of interest. Diagnostic performance was determined for the whole slide. RESULTS We included slides from 542 patients (164 NDBE, 226 LGD, and 152 HGD) yielding 8596 bounding boxes in the training set, 1946 bounding boxes in the validation set, and 840 boxes in the test set. When the ensemble model was used, sensitivity and specificity for LGD was 81.3% and 100%, respectively, and >90% for NDBE and HGD. The overall positive predictive value and sensitivity metric (calculated as F1 score) was .91 for NDBE, .90 for LGD, and 1.0 for HGD. CONCLUSIONS We successfully trained and validated a deep learning model to accurately identify dysplasia on whole-slide images. This model can potentially help improve the histologic diagnosis of BE dysplasia and the appropriate application of endoscopic therapy.
Collapse
Affiliation(s)
- Shahriar Faghani
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - D Chamil Codipilly
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| | - David Vogelsang
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - Mana Moassefi
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - Pouria Rouzrokh
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - Bardia Khosravi
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - Siddharth Agarwal
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| | - Lovekirat Dhaliwal
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| | - David A Katzka
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| | - Catherine Hagen
- Department of Pathology, Mayo Clinic, Rochester, Minnesota, USA; (5)Department of Pathology, Mayo Clinic, Jacksonville, Florida, USA
| | - Jason Lewis
- Department of Pathology, Mayo Clinic, Jacksonville, Florida, USA
| | - Cadman L Leggett
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| | - Bradley J Erickson
- Artificial Intelligence Laboratory, Mayo Clinic, Rochester, Minnesota, USA; Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA
| | - Prasad G Iyer
- Barrett's Esophagus Unit, Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|
10
|
Hu H, Laskin J. Emerging Computational Methods in Mass Spectrometry Imaging. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2203339. [PMID: 36253139 PMCID: PMC9731724 DOI: 10.1002/advs.202203339] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Revised: 09/17/2022] [Indexed: 05/10/2023]
Abstract
Mass spectrometry imaging (MSI) is a powerful analytical technique that generates maps of hundreds of molecules in biological samples with high sensitivity and molecular specificity. Advanced MSI platforms with capability of high-spatial resolution and high-throughput acquisition generate vast amount of data, which necessitates the development of computational tools for MSI data analysis. In addition, computation-driven MSI experiments have recently emerged as enabling technologies for further improving the MSI capabilities with little or no hardware modification. This review provides a critical summary of computational methods and resources developed for MSI data analysis and interpretation along with computational approaches for improving throughput and molecular coverage in MSI experiments. This review is focused on the recently developed artificial intelligence methods and provides an outlook for a future paradigm shift in MSI with transformative computational methods.
Collapse
Affiliation(s)
- Hang Hu
- Department of ChemistryPurdue University560 Oval DriveWest LafayetteIN47907USA
| | - Julia Laskin
- Department of ChemistryPurdue University560 Oval DriveWest LafayetteIN47907USA
| |
Collapse
|
11
|
Smolen JA, Wooley KL. Fluorescence lifetime image microscopy prediction with convolutional neural networks for cell detection and classification in tissues. PNAS NEXUS 2022; 1:pgac235. [PMID: 36712353 PMCID: PMC9802238 DOI: 10.1093/pnasnexus/pgac235] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 10/11/2022] [Indexed: 12/05/2022]
Abstract
Convolutional neural networks (CNNs) and other deep-learning models have proven to be transformative tools for the automated analysis of microscopy images, particularly in the domain of cellular and tissue imaging. These computer-vision models have primarily been applied with traditional microscopy imaging modalities (e.g. brightfield and fluorescence), likely due to the availability of large datasets in these regimes. However, more advanced microscopy imaging techniques could, potentially, allow for improved model performance in various computational histopathology tasks. In this work, we demonstrate that CNNs can achieve high accuracy in cell detection and classification without large amounts of data when applied to histology images acquired by fluorescence lifetime imaging microscopy (FLIM). This accuracy is higher than what would be achieved with regular single or dual-channel fluorescence images under the same settings, particularly for CNNs pretrained on publicly available fluorescent cell or general image datasets. Additionally, generated FLIM images could be predicted from just the fluorescence image data by using a dense U-Net CNN model trained on a subset of ground-truth FLIM images. These U-Net CNN generated FLIM images demonstrated high similarity to ground truth and improved accuracy in cell detection and classification over fluorescence alone when used as input to a variety of commonly used CNNs. This improved accuracy was maintained even when the FLIM images were generated by a U-Net CNN trained on only a few example FLIM images.
Collapse
Affiliation(s)
| | - Karen L Wooley
- Departments of Chemistry, Chemical Engineering, and Materials Science and Engineering, Texas A&M University, College Station, TX 77842, USA
| |
Collapse
|