1
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
2
|
Petmezas G, Papageorgiou VE, Vassilikos V, Pagourelias E, Tsaklidis G, Katsaggelos AK, Maglaveras N. Recent advancements and applications of deep learning in heart failure: Α systematic review. Comput Biol Med 2024; 176:108557. [PMID: 38728995 DOI: 10.1016/j.compbiomed.2024.108557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Revised: 04/12/2024] [Accepted: 05/05/2024] [Indexed: 05/12/2024]
Abstract
BACKGROUND Heart failure (HF), a global health challenge, requires innovative diagnostic and management approaches. The rapid evolution of deep learning (DL) in healthcare necessitates a comprehensive review to evaluate these developments and their potential to enhance HF evaluation, aligning clinical practices with technological advancements. OBJECTIVE This review aims to systematically explore the contributions of DL technologies in the assessment of HF, focusing on their potential to improve diagnostic accuracy, personalize treatment strategies, and address the impact of comorbidities. METHODS A thorough literature search was conducted across four major electronic databases: PubMed, Scopus, Web of Science and IEEE Xplore, yielding 137 articles that were subsequently categorized into five primary application areas: cardiovascular disease (CVD) classification, HF detection, image analysis, risk assessment, and other clinical analyses. The selection criteria focused on studies utilizing DL algorithms for HF assessment, not limited to HF detection but extending to any attempt in analyzing and interpreting HF-related data. RESULTS The analysis revealed a notable emphasis on CVD classification and HF detection, with DL algorithms showing significant promise in distinguishing between affected individuals and healthy subjects. Furthermore, the review highlights DL's capacity to identify underlying cardiomyopathies and other comorbidities, underscoring its utility in refining diagnostic processes and tailoring treatment plans to individual patient needs. CONCLUSIONS This review establishes DL as a key innovation in HF management, highlighting its role in advancing diagnostic accuracy and personalized care. The insights provided advocate for the integration of DL in clinical settings and suggest directions for future research to enhance patient outcomes in HF care.
Collapse
Affiliation(s)
- Georgios Petmezas
- 2nd Department of Obstetrics and Gynecology, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece; Centre for Research and Technology Hellas, Thessaloniki, Greece.
| | | | - Vasileios Vassilikos
- 3rd Department of Cardiology, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Efstathios Pagourelias
- 3rd Department of Cardiology, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - George Tsaklidis
- Department of Mathematics, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Aggelos K Katsaggelos
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA
| | - Nicos Maglaveras
- 2nd Department of Obstetrics and Gynecology, Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| |
Collapse
|
3
|
Kveton M, Hudec L, Vykopal I, Halinkovic M, Laco M, Felsoova A, Benesova W, Fabian O. Digital pathology in cardiac transplant diagnostics: from biopsies to algorithms. Cardiovasc Pathol 2024; 68:107587. [PMID: 37926351 DOI: 10.1016/j.carpath.2023.107587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 10/03/2023] [Accepted: 10/30/2023] [Indexed: 11/07/2023] Open
Abstract
In the field of heart transplantation, the ability to accurately and promptly diagnose cardiac allograft rejection is crucial. This comprehensive review explores the transformative role of digital pathology and computational pathology, especially through machine learning, in this critical domain. These methodologies harness large datasets to extract subtle patterns and valuable information that extend beyond human perceptual capabilities, potentially enhancing diagnostic outcomes. Current research indicates that these computer-based systems could offer accuracy and performance matching, or even exceeding, that of expert pathologists, thereby introducing more objectivity and reducing observer variability. Despite promising results, several challenges such as limited sample sizes, diverse data sources, and the absence of standardized protocols pose significant barriers to the widespread adoption of these techniques. The future of digital pathology in heart transplantation diagnostics depends on utilizing larger, more diverse patient cohorts, standardizing data collection, processing, and evaluation protocols, and fostering collaborative research efforts. The integration of various data types, including clinical, demographic, and imaging information, could further refine diagnostic precision. As researchers address these challenges and promote collaborative efforts, digital pathology has the potential to become an integral part of clinical practice, ultimately improving patient care in heart transplantation.
Collapse
Affiliation(s)
- Martin Kveton
- Third Faculty of Medicine, Charles University, Prague, Czech Republic; Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic.
| | - Lukas Hudec
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ivan Vykopal
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Matej Halinkovic
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Miroslav Laco
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Andrea Felsoova
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Histology and Embryology, Second Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Wanda Benesova
- Faculty of Informatics and Information Technologies, Slovak University of Technology, Bratislava, Slovakia
| | - Ondrej Fabian
- Clinical and Transplant Pathology Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic; Department of Pathology and Molecular Medicine, Third Faculty of Medicine, Charles University and Thomayer Hospital, Prague, Czech Republic
| |
Collapse
|
4
|
Hu W, Li X, Li C, Li R, Jiang T, Sun H, Huang X, Grzegorzek M, Li X. A state-of-the-art survey of artificial neural networks for Whole-slide Image analysis: From popular Convolutional Neural Networks to potential visual transformers. Comput Biol Med 2023; 161:107034. [PMID: 37230019 DOI: 10.1016/j.compbiomed.2023.107034] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 04/13/2023] [Accepted: 05/10/2023] [Indexed: 05/27/2023]
Abstract
In recent years, with the advancement of computer-aided diagnosis (CAD) technology and whole slide image (WSI), histopathological WSI has gradually played a crucial aspect in the diagnosis and analysis of diseases. To increase the objectivity and accuracy of pathologists' work, artificial neural network (ANN) methods have been generally needed in the segmentation, classification, and detection of histopathological WSI. However, the existing review papers only focus on equipment hardware, development status and trends, and do not summarize the art neural network used for full-slide image analysis in detail. In this paper, WSI analysis methods based on ANN are reviewed. Firstly, the development status of WSI and ANN methods is introduced. Secondly, we summarize the common ANN methods. Next, we discuss publicly available WSI datasets and evaluation metrics. These ANN architectures for WSI processing are divided into classical neural networks and deep neural networks (DNNs) and then analyzed. Finally, the application prospect of the analytical method in this field is discussed. The important potential method is Visual Transformers.
Collapse
Affiliation(s)
- Weiming Hu
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Xintong Li
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Chen Li
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China.
| | - Rui Li
- Microscopic Image and Medical Image Analysis Group, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Tao Jiang
- School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China; International Joint Institute of Robotics and Intelligent Systems, Chengdu University of Information Technology, Chengdu, China
| | - Hongzan Sun
- Shengjing Hospital of China Medical University, Shenyang, China
| | - Xinyu Huang
- Institute for Medical Informatics, University of Luebeck, Luebeck, Germany
| | - Marcin Grzegorzek
- Institute for Medical Informatics, University of Luebeck, Luebeck, Germany; Department of Knowledge Engineering, University of Economics in Katowice, Katowice, Poland
| | - Xiaoyan Li
- Cancer Hospital of China Medical University, Shenyang, China.
| |
Collapse
|
5
|
An automatic entropy method to efficiently mask histology whole-slide images. Sci Rep 2023; 13:4321. [PMID: 36922520 PMCID: PMC10017682 DOI: 10.1038/s41598-023-29638-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Accepted: 02/08/2023] [Indexed: 03/18/2023] Open
Abstract
Tissue segmentation of histology whole-slide images (WSI) remains a critical task in automated digital pathology workflows for both accurate disease diagnosis and deep phenotyping for research purposes. This is especially challenging when the tissue structure of biospecimens is relatively porous and heterogeneous, such as for atherosclerotic plaques. In this study, we developed a unique approach called 'EntropyMasker' based on image entropy to tackle the fore- and background segmentation (masking) task in histology WSI. We evaluated our method on 97 high-resolution WSI of human carotid atherosclerotic plaques in the Athero-Express Biobank Study, constituting hematoxylin and eosin and 8 other staining types. Using multiple benchmarking metrics, we compared our method with four widely used segmentation methods: Otsu's method, Adaptive mean, Adaptive Gaussian and slideMask and observed that our method had the highest sensitivity and Jaccard similarity index. We envision EntropyMasker to fill an important gap in WSI preprocessing, machine learning image analysis pipelines, and enable disease phenotyping beyond the field of atherosclerosis.
Collapse
|
6
|
Kiemen AL, Damanakis AI, Braxton AM, He J, Laheru D, Fishman EK, Chames P, Pérez CA, Wu PH, Wirtz D, Wood LD, Hruban RH. Tissue clearing and 3D reconstruction of digitized, serially sectioned slides provide novel insights into pancreatic cancer. MED 2023; 4:75-91. [PMID: 36773599 PMCID: PMC9922376 DOI: 10.1016/j.medj.2022.11.009] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 10/06/2022] [Accepted: 11/23/2022] [Indexed: 01/26/2023]
Abstract
Pancreatic cancer is currently the third leading cause of cancer death in the United States. The clinical hallmarks of this disease include abdominal pain that radiates to the back, the presence of a hypoenhancing intrapancreatic lesion on imaging, and widespread liver metastases. Technologies such as tissue clearing and three-dimensional (3D) reconstruction of digitized serially sectioned hematoxylin and eosin-stained slides can be used to visualize large (up to 2- to 3-centimeter cube) tissues at cellular resolution. When applied to human pancreatic cancers, these 3D visualization techniques have provided novel insights into the basis of a number of the clinical characteristics of this disease. Here, we describe the clinical features of pancreatic cancer, review techniques for clearing and the 3D reconstruction of digitized microscope slides, and provide examples that illustrate how 3D visualization of human pancreatic cancer at the microscopic level has revealed features not apparent in 2D microscopy and, in so doing, has closed the gap between bench and bedside. Compared with animal models and 2D microscopy, studies of human tissues in 3D can reveal the difference between what can happen and what does happen in human cancers.
Collapse
Affiliation(s)
- Ashley L Kiemen
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA; Department of Chemical & Biomolecular Engineering, The Johns Hopkins University, 3400 N Charles St, Baltimore, MD 21218, USA
| | - Alexander Ioannis Damanakis
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA; Department of General, Visceral, Cancer and Transplant Surgery, University Hospital of Cologne, Cologne, Germany
| | - Alicia M Braxton
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Jin He
- Department of Surgery, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Daniel Laheru
- Department of Oncology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Elliot K Fishman
- Department of Radiology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Patrick Chames
- Antibody Therapeutics and Immunotargeting Team, Aix Marseille University, CNRS, INSERM, Institut Paoli-Calmettes, CRCM, Marseille, France
| | - Cristina Almagro Pérez
- Department of Chemical & Biomolecular Engineering, The Johns Hopkins University, 3400 N Charles St, Baltimore, MD 21218, USA
| | - Pei-Hsun Wu
- Department of Chemical & Biomolecular Engineering, The Johns Hopkins University, 3400 N Charles St, Baltimore, MD 21218, USA
| | - Denis Wirtz
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA; Department of Chemical & Biomolecular Engineering, The Johns Hopkins University, 3400 N Charles St, Baltimore, MD 21218, USA
| | - Laura D Wood
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA; Department of Oncology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA.
| | - Ralph H Hruban
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA; Department of Oncology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA.
| |
Collapse
|
7
|
Rafat K, Islam S, Mahfug AA, Hossain MI, Rahman F, Momen S, Rahman S, Mohammed N. Mitigating carbon footprint for knowledge distillation based deep learning model compression. PLoS One 2023; 18:e0285668. [PMID: 37186614 PMCID: PMC10184906 DOI: 10.1371/journal.pone.0285668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 04/27/2023] [Indexed: 05/17/2023] Open
Abstract
Deep learning techniques have recently demonstrated remarkable success in numerous domains. Typically, the success of these deep learning models is measured in terms of performance metrics such as accuracy and mean average precision (mAP). Generally, a model's high performance is highly valued, but it frequently comes at the expense of substantial energy costs and carbon footprint emissions during the model building step. Massive emission of CO2 has a deleterious impact on life on earth in general and is a serious ethical concern that is largely ignored in deep learning research. In this article, we mainly focus on environmental costs and the means of mitigating carbon footprints in deep learning models, with a particular focus on models created using knowledge distillation (KD). Deep learning models typically contain a large number of parameters, resulting in a 'heavy' model. A heavy model scores high on performance metrics but is incompatible with mobile and edge computing devices. Model compression techniques such as knowledge distillation enable the creation of lightweight, deployable models for these low-resource devices. KD generates lighter models and typically performs with slightly less accuracy than the heavier teacher model (model accuracy by the teacher model on CIFAR 10, CIFAR 100, and TinyImageNet is 95.04%, 76.03%, and 63.39%; model accuracy by KD is 91.78%, 69.7%, and 60.49%). Although the distillation process makes models deployable on low-resource devices, they were found to consume an exorbitant amount of energy and have a substantial carbon footprint (15.8, 17.9, and 13.5 times more carbon compared to the corresponding teacher model). The enormous environmental cost is primarily attributable to the tuning of the hyperparameter, Temperature (τ). In this article, we propose measuring the environmental costs of deep learning work (in terms of GFLOPS in millions, energy consumption in kWh, and CO2 equivalent in grams). In order to create lightweight models with low environmental costs, we propose a straightforward yet effective method for selecting a hyperparameter (τ) using a stochastic approach for each training batch fed into the models. We applied knowledge distillation (including its data-free variant) to problems involving image classification and object detection. To evaluate the robustness of our method, we ran experiments on various datasets (CIFAR 10, CIFAR 100, Tiny ImageNet, and PASCAL VOC) and models (ResNet18, MobileNetV2, Wrn-40-2). Our novel approach reduces the environmental costs by a large margin by eliminating the requirement of expensive hyperparameter tuning without sacrificing performance. Empirical results on the CIFAR 10 dataset show that the stochastic technique achieves an accuracy of 91.67%, whereas tuning achieves an accuracy of 91.78%-however, the stochastic approach reduces the energy consumption and CO2 equivalent each by a factor of 19. Similar results have been obtained with CIFAR 100 and TinyImageNet dataset. This pattern is also observed in object detection classification on the PASCAL VOC dataset, where the tuning technique performs similarly to the stochastic technique, with a difference of 0.03% mAP favoring the stochastic technique while reducing the energy consumptions and CO2 emission each by a factor of 18.5.
Collapse
Affiliation(s)
- Kazi Rafat
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | - Sadia Islam
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | - Abdullah Al Mahfug
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | - Md Ismail Hossain
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | | | - Sifat Momen
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | - Shafin Rahman
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| | - Nabeel Mohammed
- Apurba NSU R&D Lab, Department of Electrical and Computer Engineering North South University, Dhaka, Bangladesh
| |
Collapse
|
8
|
Mostafa FA, Elrefaei LA, Fouda MM, Hossam A. A Survey on AI Techniques for Thoracic Diseases Diagnosis Using Medical Images. Diagnostics (Basel) 2022; 12:diagnostics12123034. [PMID: 36553041 PMCID: PMC9777249 DOI: 10.3390/diagnostics12123034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 11/20/2022] [Accepted: 11/22/2022] [Indexed: 12/12/2022] Open
Abstract
Thoracic diseases refer to disorders that affect the lungs, heart, and other parts of the rib cage, such as pneumonia, novel coronavirus disease (COVID-19), tuberculosis, cardiomegaly, and fracture. Millions of people die every year from thoracic diseases. Therefore, early detection of these diseases is essential and can save many lives. Earlier, only highly experienced radiologists examined thoracic diseases, but recent developments in image processing and deep learning techniques are opening the door for the automated detection of these diseases. In this paper, we present a comprehensive review including: types of thoracic diseases; examination types of thoracic images; image pre-processing; models of deep learning applied to the detection of thoracic diseases (e.g., pneumonia, COVID-19, edema, fibrosis, tuberculosis, chronic obstructive pulmonary disease (COPD), and lung cancer); transfer learning background knowledge; ensemble learning; and future initiatives for improving the efficacy of deep learning models in applications that detect thoracic diseases. Through this survey paper, researchers may be able to gain an overall and systematic knowledge of deep learning applications in medical thoracic images. The review investigates a performance comparison of various models and a comparison of various datasets.
Collapse
Affiliation(s)
- Fatma A. Mostafa
- Department of Electrical Engineering, Faculty of Engineering at Shoubra, Benha University, Cairo 11672, Egypt
| | - Lamiaa A. Elrefaei
- Department of Electrical Engineering, Faculty of Engineering at Shoubra, Benha University, Cairo 11672, Egypt
| | - Mostafa M. Fouda
- Department of Electrical and Computer Engineering, College of Science and Engineering, Idaho State University, Pocatello, ID 83209, USA
- Correspondence:
| | - Aya Hossam
- Department of Electrical Engineering, Faculty of Engineering at Shoubra, Benha University, Cairo 11672, Egypt
| |
Collapse
|
9
|
Kiemen AL, Braxton AM, Grahn MP, Han KS, Babu JM, Reichel R, Jiang AC, Kim B, Hsu J, Amoa F, Reddy S, Hong SM, Cornish TC, Thompson ED, Huang P, Wood LD, Hruban RH, Wirtz D, Wu PH. CODA: quantitative 3D reconstruction of large tissues at cellular resolution. Nat Methods 2022; 19:1490-1499. [PMID: 36280719 PMCID: PMC10500590 DOI: 10.1038/s41592-022-01650-9] [Citation(s) in RCA: 48] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 09/14/2022] [Indexed: 12/15/2022]
Abstract
A central challenge in biology is obtaining high-content, high-resolution information while analyzing tissue samples at volumes relevant to disease progression. We address this here with CODA, a method to reconstruct exceptionally large (up to multicentimeter cubed) tissues at subcellular resolution using serially sectioned hematoxylin and eosin-stained tissue sections. Here we demonstrate CODA's ability to reconstruct three-dimensional (3D) distinct microanatomical structures in pancreas, skin, lung and liver tissues. CODA allows creation of readily quantifiable tissue volumes amenable to biological research. As a testbed, we assess the microanatomy of the human pancreas during tumorigenesis within the branching pancreatic ductal system, labeling ten distinct structures to examine heterogeneity and structural transformation during neoplastic progression. We show that pancreatic precancerous lesions develop into distinct 3D morphological phenotypes and that pancreatic cancer tends to spread far from the bulk tumor along collagen fibers that are highly aligned to the 3D curves of ductal, lobular, vascular and neural structures. Thus, CODA establishes a means to transform broadly the structural study of human diseases through exploration of exhaustively labeled 3D microarchitecture.
Collapse
Affiliation(s)
- Ashley L Kiemen
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Alicia M Braxton
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Mia P Grahn
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Kyu Sang Han
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Jaanvi Mahesh Babu
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Rebecca Reichel
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ann C Jiang
- Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Bridgette Kim
- Department of Mechanical Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Jocelyn Hsu
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA
| | - Falone Amoa
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Sashank Reddy
- Department of Plastic and Reconstructive Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Seung-Mo Hong
- Department of Pathology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Toby C Cornish
- Department of Pathology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Elizabeth D Thompson
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Peng Huang
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, USA
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Laura D Wood
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ralph H Hruban
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Denis Wirtz
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Department of Oncology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
- Department of Materials Science and Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Johns Hopkins Physical Sciences-Oncology Center, The Johns Hopkins University, Baltimore, MD, USA.
- Department of Pathology, The Sol Goldman Pancreatic Cancer Research Center, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
| | - Pei-Hsun Wu
- Department of Chemical and Biomolecular Engineering, The Johns Hopkins University, Baltimore, MD, USA.
- Johns Hopkins Physical Sciences-Oncology Center, The Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
10
|
Can Sequential Images from the Same Object Be Used for Training Machine Learning Models? A Case Study for Detecting Liver Disease by Ultrasound Radiomics. AI 2022; 3:739-750. [PMID: 36168560 PMCID: PMC9511699 DOI: 10.3390/ai3030043] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Machine learning for medical imaging not only requires sufficient amounts of data for training and testing but also that the data be independent. It is common to see highly interdependent data whenever there are inherent correlations between observations. This is especially to be expected for sequential imaging data taken from time series. In this study, we evaluate the use of statistical measures to test the independence of sequential ultrasound image data taken from the same case. A total of 1180 B-mode liver ultrasound images with 5903 regions of interests were analyzed. The ultrasound images were taken from two liver disease groups, fibrosis and steatosis, as well as normal cases. Computer-extracted texture features were then used to train a machine learning (ML) model for computer-aided diagnosis. The experiment resulted in high two-category diagnosis using logistic regression, with AUC of 0.928 and high performance of multicategory classification, using random forest ML, with AUC of 0.917. To evaluate the image region independence for machine learning, Jenson–Shannon (JS) divergence was used. JS distributions showed that images of normal liver were independent from each other, while the images from the two disease pathologies were not independent. To guarantee the generalizability of machine learning models, and to prevent data leakage, multiple frames of image data acquired of the same object should be tested for independence before machine learning. Such tests can be applied to real-world medical image problems to determine if images from the same subject can be used for training.
Collapse
|
11
|
Calabrese F, Roden AC, Pavlisko E, Lunardi F, Neil D, Adam B, Hwang D, Goddard M, Berry GJ, Ivanovic M, Thüsen JVD, Gibault L, Lin CY, Wassilew K, Glass C, Westall G, Zeevi A, Levine DJ, Roux A. LUNG ALLOGRAFT STANDARDIZED HISTOLOGICAL ANALYSIS (LASHA) TEMPLATE: A RESEARCH CONSENSUS PROPOSAL. J Heart Lung Transplant 2022; 41:1487-1500. [DOI: 10.1016/j.healun.2022.06.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 06/10/2022] [Accepted: 06/24/2022] [Indexed: 11/30/2022] Open
|
12
|
|
13
|
Abstract
Perineural invasion (PNI), a sign of poor diagnosis and tumor metastasis, is common in a variety of malignant tumors. The infiltrating patterns and morphologies of tumors vary by organ and histological diversity, making PNI detection difficult in biopsy, which must be performed manually by pathologists. As the diameters of PNI nerves are measured on a millimeter scale, the PNI region is extremely small compared to the whole pathological image. In this study, an efficient deep learning-based method is proposed for detecting PNI regions in multiple types of cancers using only PNI annotations without detailed segmentation maps for each nerve and tumor cells obtained by pathologists. The key idea of the proposed method is to train the adopted deep learning model, U-Net, to capture the boundary regions where two features coexist. A boundary dilation method and a loss combination technique are proposed to improve the detection performance of PNI without requiring full segmentation maps. Experiments were conducted with various combinations of boundary dilation widths and loss functions. It is confirmed that the proposed method effectively improves PNI detection performance from 0.188 to 0.275. Additional experiments were also performed on normal nerve detection to validate the applicability of the proposed method to the general boundary detection tasks. The experimental results demonstrate that the proposed method is also effective for general tasks, and it improved nerve detection performance from 0.511 to 0.693.
Collapse
|
14
|
Peyster EG, Janowczyk A, Swamidoss A, Kethireddy S, Feldman MD, Margulies KB. Computational Analysis of Routine Biopsies Improves Diagnosis and Prediction of Cardiac Allograft Vasculopathy. Circulation 2022; 145:1563-1577. [PMID: 35405081 DOI: 10.1161/circulationaha.121.058459] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background: Cardiac allograft vasculopathy (CAV) is a leading cause of morbidity and mortality for heart transplant recipients. While clinical risk factors for CAV have been established, no personalized prognostic test exists to confidently identify patients at high vs. low risk of developing aggressive CAV. The aim of this investigation was to leverage computational methods for analyzing digital pathology images from routine endomyocardial biopsies (EMB) to develop a precision medicine tool for predicting CAV years before overt clinical presentation. Methods: Clinical data from 1-year post-transplant was collected on 302 transplant recipients from the University of Pennsylvania, including 53 'early CAV' patients and 249 'no-CAV' controls. This data was used to generate a 'clinical model' (ClinCAV-Pr) for predicting future CAV development. From this cohort, n=183 archived EMBs were collected for CD31 and modified trichrome staining and then digitally scanned. These included 1-year post-transplant EMBs from 50 'early CAV' patients and 82 no-CAV patients, as well as 51 EMBs from 'disease control' patients obtained at the time of definitive coronary angiography confirming CAV. Using biologically-inspired, hand-crafted features extracted from digitized EMBs, quantitative histologic models for differentiating no-CAV from disease controls (HistoCAV-Dx), and for predicting future CAV from 1-year post-transplant EMBs were developed (HistoCAV-Pr). The performance of histologic and clinical models for predicting future CAV (i.e. HistoCAV-Pr and ClinCAV-Pr, respectively) were compared in a held-out validation set, before being combined to assess the added predictive value of an integrated predictive model (iCAV-Pr). Results: ClinCAV-Pr achieved modest performance on the independent test set, with area under the receiver operating curve (AUROC) of 0.70. The HistoCAV-Dx model for diagnosing CAV achieved excellent discrimination, with an AUROC of 0.91, while HistoCAV-Pr model for predicting CAV achieved good performance with an AUROC of 0.80. The integrated iCAV-Pr model achieved excellent predictive performance, with an AUROC of 0.93 on the held-out test set. Conclusions: Prediction of future CAV development is greatly improved by incorporation of computationally extracted histologic features. These results suggest morphologic details contained within regularly obtained biopsy tissue have the potential to enhance precision and personalization of treatment plans for post-heart transplant patients.
Collapse
Affiliation(s)
- Eliot G Peyster
- Cardiovascular Research Institute (E.G.P., K.B.M.), University of Pennsylvania, Philadelphia
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH (A.J., A.S., S.K.)
- Department of Oncology, Lausanne University Hospital and Lausanne University, Switzerland (A.J.)
| | - Abigail Swamidoss
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH (A.J., A.S., S.K.)
| | - Samhith Kethireddy
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH (A.J., A.S., S.K.)
| | - Michael D Feldman
- Department of Pathology and Laboratory Medicine (M.D.F.), University of Pennsylvania, Philadelphia
| | - Kenneth B Margulies
- Cardiovascular Research Institute (E.G.P., K.B.M.), University of Pennsylvania, Philadelphia
| |
Collapse
|
15
|
Lee MD, Elsayed M, Chopra S, Lui YW. A No-Math Primer on the Principles of Machine Learning for Radiologists. Semin Ultrasound CT MR 2022; 43:133-141. [PMID: 35339253 DOI: 10.1053/j.sult.2022.02.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Machine learning is becoming increasingly important in both research and clinical applications in radiology due to recent technological developments, particularly in deep learning. As these technologies are translated toward clinical practice, there is a need for radiologists and radiology trainees to understand the basic principles behind them. This primer provides an accessible introduction to the vocabulary and concepts that are central to machine learning and relevant to the radiologist.
Collapse
Affiliation(s)
- Matthew D Lee
- Department of Radiology, NYU Grossman School of Medicine, New York, NY
| | - Mohammed Elsayed
- Department of Radiology, NYU Grossman School of Medicine, New York, NY
| | - Sumit Chopra
- Department of Radiology, NYU Grossman School of Medicine, New York, NY; Courant Institute of Mathematical Sciences, New York University, New York, NY
| | - Yvonne W Lui
- Department of Radiology, NYU Grossman School of Medicine, New York, NY.
| |
Collapse
|
16
|
Deep learning-enabled assessment of cardiac allograft rejection from endomyocardial biopsies. Nat Med 2022; 28:575-582. [PMID: 35314822 DOI: 10.1038/s41591-022-01709-2] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Accepted: 01/19/2022] [Indexed: 02/07/2023]
Abstract
Endomyocardial biopsy (EMB) screening represents the standard of care for detecting allograft rejections after heart transplant. Manual interpretation of EMBs is affected by substantial interobserver and intraobserver variability, which often leads to inappropriate treatment with immunosuppressive drugs, unnecessary follow-up biopsies and poor transplant outcomes. Here we present a deep learning-based artificial intelligence (AI) system for automated assessment of gigapixel whole-slide images obtained from EMBs, which simultaneously addresses detection, subtyping and grading of allograft rejection. To assess model performance, we curated a large dataset from the United States, as well as independent test cohorts from Turkey and Switzerland, which includes large-scale variability across populations, sample preparations and slide scanning instrumentation. The model detects allograft rejection with an area under the receiver operating characteristic curve (AUC) of 0.962; assesses the cellular and antibody-mediated rejection type with AUCs of 0.958 and 0.874, respectively; detects Quilty B lesions, benign mimics of rejection, with an AUC of 0.939; and differentiates between low-grade and high-grade rejections with an AUC of 0.833. In a human reader study, the AI system showed non-inferior performance to conventional assessment and reduced interobserver variability and assessment time. This robust evaluation of cardiac allograft rejection paves the way for clinical trials to establish the efficacy of AI-assisted EMB assessment and its potential for improving heart transplant outcomes.
Collapse
|
17
|
Javeed A, Khan SU, Ali L, Ali S, Imrana Y, Rahman A. Machine Learning-Based Automated Diagnostic Systems Developed for Heart Failure Prediction Using Different Types of Data Modalities: A Systematic Review and Future Directions. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:9288452. [PMID: 35154361 PMCID: PMC8831075 DOI: 10.1155/2022/9288452] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 01/15/2022] [Indexed: 12/13/2022]
Abstract
One of the leading causes of deaths around the globe is heart disease. Heart is an organ that is responsible for the supply of blood to each part of the body. Coronary artery disease (CAD) and chronic heart failure (CHF) often lead to heart attack. Traditional medical procedures (angiography) for the diagnosis of heart disease have higher cost as well as serious health concerns. Therefore, researchers have developed various automated diagnostic systems based on machine learning (ML) and data mining techniques. ML-based automated diagnostic systems provide an affordable, efficient, and reliable solutions for heart disease detection. Various ML, data mining methods, and data modalities have been utilized in the past. Many previous review papers have presented systematic reviews based on one type of data modality. This study, therefore, targets systematic review of automated diagnosis for heart disease prediction based on different types of modalities, i.e., clinical feature-based data modality, images, and ECG. Moreover, this paper critically evaluates the previous methods and presents the limitations in these methods. Finally, the article provides some future research directions in the domain of automated heart disease detection based on machine learning and multiple of data modalities.
Collapse
Affiliation(s)
- Ashir Javeed
- Aging Research Center, Karolinska Institutet, Sweden
| | - Shafqat Ullah Khan
- Department of Electrical Engineering, University of Science and Technology Bannu, Pakistan
| | - Liaqat Ali
- Department of Electronics, University of Buner, Buner, Pakistan
| | - Sardar Ali
- School of Engineering and Applied Sciences, Isra University Islamabad Campus, Pakistan
| | - Yakubu Imrana
- School of Engineering, University of Development Studies, Tamale, Ghana
- School of Computer Science and Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, China
| | - Atiqur Rahman
- Department of Computer Science, University of Science and Technology Bannu, Pakistan
| |
Collapse
|
18
|
Bao Y, Zhang J, Zhao X, Zhou H, Chen Y, Jian J, Shi T, Gao X. Deep Learning-Based Fully Automated Diagnosis of Melanocytic Lesions by Using Whole Slide Images. J DERMATOL TREAT 2022; 33:2571-2577. [PMID: 35112978 DOI: 10.1080/09546634.2022.2038772] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Background Erroneous diagnoses of melanocytic lesions (benign, atypical, and malignant types) result in inappropriate surgical treatment plans.Objective To propose a deep learning (DL)-based fully automated diagnostic method using whole slide images (WSIs) for melanocytic lesions.Methods The method consisted of patch prediction using a DL model and patient diagnosis using an aggregation module. The method was developed with 745 WSIs, and evaluated using internal and external testing sets comprising 182 WSIs and 54 WSIs, respectively. The results were compared with those of the classification by one junior and two senior pathologists. Furthermore, we compared the performance of the three pathologists in the classification of melanocytic lesions with and without the assistance of our method.Results The method achieved an accuracy of 0.963 and 0.930 on the internal and external testing sets, respectively, which was significantly higher than that of the junior pathologist (0.419 and 0.535). With assistance from the method, all three pathologists achieved higher accuracy on the internal and external testing sets; the accuracy of the junior pathologist increased by 39.0% and 30.2%, respectively (p < 0.05).Conclusion This generalizable method can accurately classify melanocytic lesions and effectively improve the diagnostic accuracy of pathologists.
Collapse
Affiliation(s)
- Yongyang Bao
- Shanghai Ninth People's Hospital, Shanghai JiaoTong University School of Medicine, Shanghai, China
| | - Jiayi Zhang
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China
| | - Xingyu Zhao
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China
| | - Henghua Zhou
- Shanghai Ninth People's Hospital, Shanghai JiaoTong University School of Medicine, Shanghai, China
| | - Ying Chen
- Shanghai Ninth People's Hospital, Shanghai JiaoTong University School of Medicine, Shanghai, China
| | - Junming Jian
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China
| | - Tianlei Shi
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China
| | - Xin Gao
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu, China.,Jinan Guoke Medical Engineering and Technology Development Co., Ltd., Jinan, Shandong, China
| |
Collapse
|
19
|
A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification and detection approaches. Artif Intell Rev 2022. [DOI: 10.1007/s10462-021-10121-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
20
|
Jiao Y, Yuan J, Sodimu OM, Qiang Y, Ding Y. Deep Neural Network-Aided Histopathological Analysis of Myocardial Injury. Front Cardiovasc Med 2022; 8:724183. [PMID: 35083295 PMCID: PMC8784602 DOI: 10.3389/fcvm.2021.724183] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Accepted: 12/17/2021] [Indexed: 11/13/2022] Open
Abstract
Deep neural networks have become the mainstream approach for analyzing and interpreting histology images. In this study, we established and validated an interpretable DNN model to assess endomyocardial biopsy (EMB) data of patients with myocardial injury. Deep learning models were used to extract features and classify EMB histopathological images of heart failure cases diagnosed with either ischemic cardiomyopathy or idiopathic dilated cardiomyopathy and non-failing cases (organ donors without a history of heart failure). We utilized the gradient-weighted class activation mapping (Grad-CAM) technique to emphasize injured regions, providing an entry point to assess the dominant morphology in the process of a comprehensive evaluation. To visualize clustered regions of interest (ROI), we utilized uniform manifold approximation and projection (UMAP) embedding for dimension reduction. We further implemented a multi-model ensemble mechanism to improve the quantitative metric (area under the receiver operating characteristic curve, AUC) to 0.985 and 0.992 on ROI-level and case-level, respectively, outperforming the achievement of 0.971 ± 0.017 and 0.981 ± 0.020 based on the sub-models. Collectively, this new methodology provides a robust and interpretive framework to explore local histopathological patterns, facilitating the automatic and high-throughput quantification of cardiac EMB analysis.
Collapse
|
21
|
From imaging a single cell to implementing precision medicine: an exciting new era. Emerg Top Life Sci 2021; 5:837-847. [PMID: 34889448 PMCID: PMC8786301 DOI: 10.1042/etls20210219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 11/24/2021] [Accepted: 11/24/2021] [Indexed: 11/17/2022]
Abstract
In the age of high-throughput, single-cell biology, single-cell imaging has evolved not only in terms of technological advancements but also in its translational applications. The synchronous advancements of imaging and computational biology have produced opportunities of merging the two, providing the scientific community with tools towards observing, understanding, and predicting cellular and tissue phenotypes and behaviors. Furthermore, multiplexed single-cell imaging and machine learning algorithms now enable patient stratification and predictive diagnostics of clinical specimens. Here, we provide an overall summary of the advances in single-cell imaging, with a focus on high-throughput microscopy phenomics and multiplexed proteomic spatial imaging platforms. We also review various computational tools that have been developed in recent years for image processing and downstream applications used in biomedical sciences. Finally, we discuss how harnessing systems biology approaches and data integration across disciplines can further strengthen the exciting applications and future implementation of single-cell imaging on precision medicine.
Collapse
|
22
|
Glass C, Lafata KJ, Jeck W, Horstmeyer R, Cooke C, Everitt J, Glass M, Dov D, Seidman MA. The Role of Machine Learning in Cardiovascular Pathology. Can J Cardiol 2021; 38:234-245. [PMID: 34813876 DOI: 10.1016/j.cjca.2021.11.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Revised: 11/15/2021] [Accepted: 11/17/2021] [Indexed: 02/07/2023] Open
Abstract
Machine learning has seen slow but steady uptake in diagnostic pathology over the past decade to assess digital whole-slide images. Machine learning tools have incredible potential to standardise, and likely even improve, histopathologic diagnoses, but they are not yet widely used in clinical practice. We describe the principles of these tools and technologies and some successful preclinical and pretranslational efforts in cardiovascular pathology, as well as a roadmap for moving forward. In nonhuman animal models, one proof-of-principle application is in rodent progressive cardiomyopathy, which is of particular significance to drug toxicity studies. Basic science successes include screening the quality of differentiated stem cells and characterising cardiomyocyte developmental stages, with potential applications for research and toxicology/drug safety screening using derived or native human pluripotent stem cells differentiated into cardiomyocytes. Translational studies of particular note include those with success in diagnosing the various forms of heart allograft rejection. For fully realising the value of these tools in clinical cardiovascular pathology, we identify 3 essential challenges. First is image quality standardisation to ensure that algorithms can be developed and implemented on robust, consistent data. The second is consensus diagnosis; experts don't always agree, and thus "truth" may be difficult to establish, but the algorithms themselves may provide a solution. The third is the need for large-enough data sets to facilitate robust algorithm development, necessitating large cross-institutional shared image databases. The power of histopathology-based machine learning technologies is tremendous, and we outline the next steps needed to capitalise on this power.
Collapse
Affiliation(s)
- Carolyn Glass
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA.
| | - Kyle J Lafata
- Department of Radiology, Duke University School of Medicine, Durham, North Carolina, USA; Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina, USA; Department of Electrical and Computer Engineering, Duke Pratt School of Engineering, Duke University, Durham, North Carolina, USA
| | - William Jeck
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA
| | - Roarke Horstmeyer
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Department of Biomedical Engineering, Duke Pratt School of Engineering, Durham, North Carolina, USA
| | - Colin Cooke
- Department of Electrical and Computer Engineering, Duke Pratt School of Engineering, Duke University, Durham, North Carolina, USA
| | - Jeffrey Everitt
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Department of Pathology, Duke University Medical Center, Durham, North Carolina, USA
| | - Matthew Glass
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Department of Anesthesiology, Duke University Medical Center, Durham, North Carolina, USA
| | - David Dov
- Division of Artificial Intelligence and Computational Pathology, Duke AI Health, Duke University Medical Center, Durham, North Carolina, USA; Tel-Aviv Sourasky Medical Center, Tel Aviv, Israel
| | - Michael A Seidman
- Laboratory Medicine Program, University Health Network, Toronto, Ontario, Canada; Department of Laboratory Medicine and Pathobiology, University of Toronto, Ontario, Canada
| |
Collapse
|
23
|
Cheng X, Manandhar I, Aryal S, Joe B. Application of Artificial Intelligence in Cardiovascular Medicine. Compr Physiol 2021; 11:2455-2466. [PMID: 34558666 DOI: 10.1002/cphy.c200034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The advent of advances in machine learning (ML)-based techniques has popularized wide applications of artificial intelligence (AI) in various fields ranging from robotics to medicine. In recent years, there has been a surge in the application of AI to research in cardiovascular medicine, which is largely driven by the availability of large-scale clinical and multi-omics datasets. Such applications are providing a new perspective for a better understanding of cardiovascular disease (CVD), which could be used to develop novel diagnostic and therapeutic strategies. For example, studies have shown that ML has a substantial potential for early diagnosis of different types of CVD, prediction of adverse disease outcomes such as heart failure, and development of newer and personalized treatments. In this article, we provide an overview and discuss the current status of a wide range of AI applications, including machine learning, reinforcement learning, and deep learning, in cardiovascular medicine. © 2021 American Physiological Society. Compr Physiol 11:1-12, 2021.
Collapse
Affiliation(s)
- Xi Cheng
- Bioinformatics & Artificial Intelligence Laboratory, Center for Hypertension and Precision Medicine, Program in Physiological Genomics, Department of Physiology and Pharmacology, University of Toledo College of Medicine and Life Sciences, Toledo, Ohio, USA
| | - Ishan Manandhar
- Bioinformatics & Artificial Intelligence Laboratory, Center for Hypertension and Precision Medicine, Program in Physiological Genomics, Department of Physiology and Pharmacology, University of Toledo College of Medicine and Life Sciences, Toledo, Ohio, USA
| | - Sachin Aryal
- Bioinformatics & Artificial Intelligence Laboratory, Center for Hypertension and Precision Medicine, Program in Physiological Genomics, Department of Physiology and Pharmacology, University of Toledo College of Medicine and Life Sciences, Toledo, Ohio, USA
| | - Bina Joe
- Bioinformatics & Artificial Intelligence Laboratory, Center for Hypertension and Precision Medicine, Program in Physiological Genomics, Department of Physiology and Pharmacology, University of Toledo College of Medicine and Life Sciences, Toledo, Ohio, USA
| |
Collapse
|
24
|
Abstract
This review provides the feasible literature on drug discovery through ML tools and techniques that are enforced in every phase of drug development to accelerate the research process and deduce the risk and expenditure in clinical trials. Machine learning techniques improve the decision-making in pharmaceutical data across various applications like QSAR analysis, hit discoveries, de novo drug architectures to retrieve accurate outcomes. Target validation, prognostic biomarkers, digital pathology are considered under problem statements in this review. ML challenges must be applicable for the main cause of inadequacy in interpretability outcomes that may restrict the applications in drug discovery. In clinical trials, absolute and methodological data must be generated to tackle many puzzles in validating ML techniques, improving decision-making, promoting awareness in ML approaches, and deducing risk failures in drug discovery.
Collapse
Affiliation(s)
- Suresh Dara
- Department of Computer Science and Engineering, B V Raju Institute of Technology, Narsapur, Medak, 502313 Telangana India
| | - Swetha Dhamercherla
- Department of Computer Science and Engineering, B V Raju Institute of Technology, Narsapur, Medak, 502313 Telangana India
| | - Surender Singh Jadav
- Centre for Molecular Cancer Research (CMCR) and Vishnu Institute of Pharmaceutical Education and Research (VIPER), Narsapur, Medak, 502313 Telangana India
| | - CH Madhu Babu
- Department of Computer Science and Engineering, B V Raju Institute of Technology, Narsapur, Medak, 502313 Telangana India
| | - Mohamed Jawed Ahsan
- Department of Pharmaceutical Chemistry, Maharishi Arvind College of Pharmacy, Jaipur, 302023 Rajasthan India
| |
Collapse
|
25
|
Pham TD. Time-frequency time-space long short-term memory networks for image classification of histopathological tissue. Sci Rep 2021; 11:13703. [PMID: 34211077 PMCID: PMC8249635 DOI: 10.1038/s41598-021-93160-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 06/22/2021] [Indexed: 02/07/2023] Open
Abstract
Image analysis in histopathology provides insights into the microscopic examination of tissue for disease diagnosis, prognosis, and biomarker discovery. Particularly for cancer research, precise classification of histopathological images is the ultimate objective of the image analysis. Here, the time-frequency time-space long short-term memory network (TF-TS LSTM) developed for classification of time series is applied for classifying histopathological images. The deep learning is empowered by the use of sequential time-frequency and time-space features extracted from the images. Furthermore, unlike conventional classification practice, a strategy for class modeling is designed to leverage the learning power of the TF-TS LSTM. Tests on several datasets of histopathological images of haematoxylin-and-eosin and immunohistochemistry stains demonstrate the strong capability of the artificial intelligence (AI)-based approach for producing very accurate classification results. The proposed approach has the potential to be an AI tool for robust classification of histopathological images.
Collapse
Affiliation(s)
- Tuan D Pham
- Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar, 31952, Saudi Arabia.
| |
Collapse
|
26
|
Peyster EG, Arabyarmohammadi S, Janowczyk A, Azarianpour-Esfahani S, Sekulic M, Cassol C, Blower L, Parwani A, Lal P, Feldman MD, Margulies KB, Madabhushi A. An automated computational image analysis pipeline for histological grading of cardiac allograft rejection. Eur Heart J 2021; 42:2356-2369. [PMID: 33982079 PMCID: PMC8216729 DOI: 10.1093/eurheartj/ehab241] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Revised: 01/26/2021] [Accepted: 04/14/2021] [Indexed: 12/11/2022] Open
Abstract
AIM Allograft rejection is a serious concern in heart transplant medicine. Though endomyocardial biopsy with histological grading is the diagnostic standard for rejection, poor inter-pathologist agreement creates significant clinical uncertainty. The aim of this investigation is to demonstrate that cellular rejection grades generated via computational histological analysis are on-par with those provided by expert pathologists. METHODS AND RESULTS The study cohort consisted of 2472 endomyocardial biopsy slides originating from three major US transplant centres. The 'Computer-Assisted Cardiac Histologic Evaluation (CACHE)-Grader' pipeline was trained using an interpretable, biologically inspired, 'hand-crafted' feature extraction approach. From a menu of 154 quantitative histological features relating the density and orientation of lymphocytes, myocytes, and stroma, a model was developed to reproduce the 4-grade clinical standard for cellular rejection diagnosis. CACHE-grader interpretations were compared with independent pathologists and the 'grade of record', testing for non-inferiority (δ = 6%). Study pathologists achieved a 60.7% agreement [95% confidence interval (CI): 55.2-66.0%] with the grade of record, and pair-wise agreement among all human graders was 61.5% (95% CI: 57.0-65.8%). The CACHE-Grader met the threshold for non-inferiority, achieving a 65.9% agreement (95% CI: 63.4-68.3%) with the grade of record and a 62.6% agreement (95% CI: 60.3-64.8%) with all human graders. The CACHE-Grader demonstrated nearly identical performance in internal and external validation sets (66.1% vs. 65.8%), resilience to inter-centre variations in tissue processing/digitization, and superior sensitivity for high-grade rejection (74.4% vs. 39.5%, P < 0.001). CONCLUSION These results show that the CACHE-grader pipeline, derived using intuitive morphological features, can provide expert-quality rejection grading, performing within the range of inter-grader variability seen among human pathologists.
Collapse
Affiliation(s)
- Eliot G Peyster
- Cardiovascular Institute, University of Pennsylvania, 3400 Civic Center Blvd, Smilow TRC 11th floor, Philadelphia, PA 19104, USA
| | - Sara Arabyarmohammadi
- Department of Computer and Data Sciences, Case Western Reserve University, 10900 Euclid Avenue, Nord Hall Suite 500, Cleveland, OH 44106, USA
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Case Western Reserve University, 10900 Euclid Avenue, Nord Hall Suite 500, Cleveland, OH 44106, USA
| | - Sepideh Azarianpour-Esfahani
- Department of Biomedical Engineering, Case Western Reserve University, 10900 Euclid Avenue, Nord Hall Suite 500, Cleveland, OH 44106, USA
| | - Miroslav Sekulic
- Department of Pathology, University Hospitals Cleveland Medical Center, 11100 Euclid Ave, Cleveland, OH 44106, USA
| | - Clarissa Cassol
- Department of Pathology, Ohio State University Wexner Medical Center, 450 W 10th Ave, Columbus, OH 43210, USA
| | - Luke Blower
- Department of Pathology, Ohio State University Wexner Medical Center, 450 W 10th Ave, Columbus, OH 43210, USA
| | - Anil Parwani
- Department of Pathology, Ohio State University Wexner Medical Center, 450 W 10th Ave, Columbus, OH 43210, USA
| | - Priti Lal
- Department of Pathology and Laboratory Medicine, University of Pennsylvania, 3400 Spruce Street 6 Founders, Philadelphia, PA 19104, USA
| | - Michael D Feldman
- Department of Pathology and Laboratory Medicine, University of Pennsylvania, 3400 Spruce Street 6 Founders, Philadelphia, PA 19104, USA
| | - Kenneth B Margulies
- Cardiovascular Institute, University of Pennsylvania, 3400 Civic Center Blvd, Smilow TRC 11th floor, Philadelphia, PA 19104, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, 10900 Euclid Avenue, Nord Hall Suite 500, Cleveland, OH 44106, USA
| |
Collapse
|
27
|
Kuklyte J, Fitzgerald J, Nelissen S, Wei H, Whelan A, Power A, Ahmad A, Miarka M, Gregson M, Maxwell M, Raji R, Lenihan J, Finn-Moloney E, Rafferty M, Cary M, Barale-Thomas E, O’Shea D. Evaluation of the Use of Single- and Multi-Magnification Convolutional Neural Networks for the Determination and Quantitation of Lesions in Nonclinical Pathology Studies. Toxicol Pathol 2021; 49:815-842. [PMID: 33618634 PMCID: PMC8091423 DOI: 10.1177/0192623320986423] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Digital pathology platforms with integrated artificial intelligence have the potential to increase the efficiency of the nonclinical pathologist's workflow through screening and prioritizing slides with lesions and highlighting areas with specific lesions for review. Herein, we describe the comparison of various single- and multi-magnification convolutional neural network (CNN) architectures to accelerate the detection of lesions in tissues. Different models were evaluated for defining performance characteristics and efficiency in accurately identifying lesions in 5 key rat organs (liver, kidney, heart, lung, and brain). Cohorts for liver and kidney were collected from TG-GATEs open-source repository, and heart, lung, and brain from internally selected R&D studies. Annotations were performed, and models were trained on each of the available lesion classes in the available organs. Various class-consolidation approaches were evaluated from generalized lesion detection to individual lesion detections. The relationship between the amount of annotated lesions and the precision/accuracy of model performance is elucidated. The utility of multi-magnification CNN implementations in specific tissue subtypes is also demonstrated. The use of these CNN-based models offers users the ability to apply generalized lesion detection to whole-slide images, with the potential to generate novel quantitative data that would not be possible with conventional image analysis techniques.
Collapse
Affiliation(s)
| | | | | | - Haolin Wei
- Deciphex, Dublin City University, Dublin, Ireland
| | - Aoife Whelan
- Deciphex, Dublin City University, Dublin, Ireland
| | - Adam Power
- Deciphex, Dublin City University, Dublin, Ireland
| | - Ajaz Ahmad
- Deciphex, Dublin City University, Dublin, Ireland
| | | | - Mark Gregson
- Deciphex, Dublin City University, Dublin, Ireland
| | | | - Ruka Raji
- Deciphex, Dublin City University, Dublin, Ireland
| | | | | | | | - Maurice Cary
- Pathology Experts GmbH, Technologie Zentrum Witterswil, Witters, Switzerland
| | | | - Donal O’Shea
- Deciphex, Dublin City University, Dublin, Ireland
| |
Collapse
|
28
|
Siebelt M, Das D, Van Den Moosdijk A, Warren T, Van Der Putten P, Van Der Weegen W. Machine learning algorithms trained with pre-hospital acquired history-taking data can accurately differentiate diagnoses in patients with hip complaints. Acta Orthop 2021; 92:254-257. [PMID: 33573429 PMCID: PMC8231380 DOI: 10.1080/17453674.2021.1884408] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Background and purpose - Machine learning (ML) techniques are a form of artificial intelligence able to analyze big data. Analyzing the outcome of (digital) questionnaires, ML might recognize different patterns in answers that might relate to different types of pathology. With this study, we investigated the proof-of-principle of ML-based diagnosis in patients with hip complaints using a digital questionnaire and the Kellgren and Lawrence (KL) osteoarthritis score.Patients and methods - 548 patients (> 55 years old) scheduled for consultation of hip complaints were asked to participate in this study and fill in an online questionnaire. Our questionnaire consists of 27 questions related to general history-taking and validated patient-related outcome measures (Oxford Hip Score and a Numeric Rating Scale for pain). 336 fully completed questionnaires were related to their classified diagnosis (either hip osteoarthritis, bursitis or tendinitis, or other pathology). Different AI techniques were used to relate questionnaire outcome and hip diagnoses. Resulting area under the curve (AUC) and classification accuracy (CA) are reported to identify the best scoring AI model. The accuracy of different ML models was compared using questionnaire outcome with and without radiologic KL scores for degree of osteoarthritis.Results - The most accurate ML model for diagnosis of patients with hip complaints was the Random Forest model (AUC 82%, 95% CI 0.78-0.86; CA 69%, CI 0.64-0.74) and most accurate analysis with addition of KL scores was with a Support Vector Machine model (AUC 89%, CI 0.86-0.92; CA 83%, CI 0.79-0.87).Interpretation - Analysis of self-reported online questionnaires related to hip complaints can differentiate between basic hip pathologies. The addition of radiological scores for osteoarthritis further improves these outcomes.
Collapse
Affiliation(s)
- Michiel Siebelt
- Department of Orthopedic Surgery, St Anna Hospital, Geldrop;
| | - Dirk Das
- Department of Orthopedic Surgery, St Anna Hospital, Geldrop;
| | | | - Tristan Warren
- Department of Orthopedic Surgery, St Anna Hospital, Geldrop;
| | - Peter Van Der Putten
- Leiden Institute of Advanced Computer Science, Leiden University Leiden, The Netherlands
| | | |
Collapse
|
29
|
Gavrielides MA, Miller M, Hagemann IS, Abdelal H, Alipour Z, Chen JF, Salari B, Sun L, Zhou H, Seidman JD. Clinical Decision Support for Ovarian Carcinoma Subtype Classification: A Pilot Observer Study With Pathology Trainees. Arch Pathol Lab Med 2021; 144:869-877. [PMID: 31816269 DOI: 10.5858/arpa.2019-0390-oa] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/16/2019] [Indexed: 11/06/2022]
Abstract
CONTEXT.— Clinical decision support (CDS) systems could assist less experienced pathologists with certain diagnostic tasks for which subspecialty training or extensive experience is typically needed. The effect of decision support on pathologist performance for such diagnostic tasks has not been examined. OBJECTIVE.— To examine the impact of a CDS tool for the classification of ovarian carcinoma subtypes by pathology trainees in a pilot observer study using digital pathology. DESIGN.— Histologic review on 90 whole slide images from 75 ovarian cancer patients was conducted by 6 pathology residents using: (1) unaided review of whole slide images, and (2) aided review, where in addition to whole slide images observers used a CDS tool that provided information about the presence of 8 histologic features important for subtype classification that were identified previously by an expert in gynecologic pathology. The reference standard of ovarian subtype consisted of majority consensus from a panel of 3 gynecologic pathology experts. RESULTS.— Aided review improved pairwise concordance with the reference standard for 5 of 6 observers by 3.3% to 17.8% (for 2 observers, increase was statistically significant) and mean interobserver agreement by 9.2% (not statistically significant). Observers benefited the most when the CDS tool prompted them to look for missed histologic features that were definitive for a certain subtype. Observer performance varied widely across cases with unanimous and nonunanimous reference classification, supporting the need for balancing data sets in terms of case difficulty. CONCLUSIONS.— Findings showed the potential of CDS systems to close the knowledge gap between pathologists for complex diagnostic tasks.
Collapse
Affiliation(s)
- Marios A Gavrielides
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Meghan Miller
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Ian S Hagemann
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Heba Abdelal
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Zahra Alipour
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Jie-Fu Chen
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Behzad Salari
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Lulu Sun
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Huifang Zhou
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| | - Jeffrey D Seidman
- From the Division of Imaging, Diagnostics, and Software Reliability, Office of Engineering and Science Laboratories (Dr Gavrielides and Ms Miller), and the Office of In Vitro Diagnostics and Radiological Health, Division of Molecular Genetics and Pathology (Dr Seidman), Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland; the Department of Bioengineering, University of Maryland, College Park (Ms Miller); and the Departments of Pathology and Immunology (Drs Hagemann, Abdelal, Alipour, Chen, Salari, Sun, and Zhou) and Obstetrics and Gynecology (Dr Hagemann), Washington University School of Medicine, St Louis, Missouri. Ms Miller is currently with PCTEST Engineering Laboratory, Columbia, Maryland
| |
Collapse
|
30
|
Banerjee A, Chen S, Fatemifar G, Zeina M, Lumbers RT, Mielke J, Gill S, Kotecha D, Freitag DF, Denaxas S, Hemingway H. Machine learning for subtype definition and risk prediction in heart failure, acute coronary syndromes and atrial fibrillation: systematic review of validity and clinical utility. BMC Med 2021; 19:85. [PMID: 33820530 PMCID: PMC8022365 DOI: 10.1186/s12916-021-01940-7] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 02/12/2021] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND Machine learning (ML) is increasingly used in research for subtype definition and risk prediction, particularly in cardiovascular diseases. No existing ML models are routinely used for cardiovascular disease management, and their phase of clinical utility is unknown, partly due to a lack of clear criteria. We evaluated ML for subtype definition and risk prediction in heart failure (HF), acute coronary syndromes (ACS) and atrial fibrillation (AF). METHODS For ML studies of subtype definition and risk prediction, we conducted a systematic review in HF, ACS and AF, using PubMed, MEDLINE and Web of Science from January 2000 until December 2019. By adapting published criteria for diagnostic and prognostic studies, we developed a seven-domain, ML-specific checklist. RESULTS Of 5918 studies identified, 97 were included. Across studies for subtype definition (n = 40) and risk prediction (n = 57), there was variation in data source, population size (median 606 and median 6769), clinical setting (outpatient, inpatient, different departments), number of covariates (median 19 and median 48) and ML methods. All studies were single disease, most were North American (n = 61/97) and only 14 studies combined definition and risk prediction. Subtype definition and risk prediction studies respectively had limitations in development (e.g. 15.0% and 78.9% of studies related to patient benefit; 15.0% and 15.8% had low patient selection bias), validation (12.5% and 5.3% externally validated) and impact (32.5% and 91.2% improved outcome prediction; no effectiveness or cost-effectiveness evaluations). CONCLUSIONS Studies of ML in HF, ACS and AF are limited by number and type of included covariates, ML methods, population size, country, clinical setting and focus on single diseases, not overlap or multimorbidity. Clinical utility and implementation rely on improvements in development, validation and impact, facilitated by simple checklists. We provide clear steps prior to safe implementation of machine learning in clinical practice for cardiovascular diseases and other disease areas.
Collapse
Affiliation(s)
- Amitava Banerjee
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK.
- Health Data Research UK, University College London, London, UK.
- University College London Hospitals NHS Trust, 235 Euston Road, London, UK.
- Barts Health NHS Trust, The Royal London Hospital, Whitechapel Rd, London, UK.
| | - Suliang Chen
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK
- Health Data Research UK, University College London, London, UK
| | - Ghazaleh Fatemifar
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK
- Health Data Research UK, University College London, London, UK
| | | | - R Thomas Lumbers
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK
- Health Data Research UK, University College London, London, UK
- University College London Hospitals NHS Trust, 235 Euston Road, London, UK
| | - Johanna Mielke
- Bayer AG, Division Pharmaceuticals, Open Innovation & Digital Technologies, Wuppertal, Germany
| | - Simrat Gill
- University of Birmingham Institute of Cardiovascular Sciences and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Dipak Kotecha
- University of Birmingham Institute of Cardiovascular Sciences and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Department of Cardiology, University Medical Centre Utrecht, Utrecht, the Netherlands
| | - Daniel F Freitag
- Bayer AG, Division Pharmaceuticals, Open Innovation & Digital Technologies, Wuppertal, Germany
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK
- Health Data Research UK, University College London, London, UK
- The Alan Turing Institute, London, UK
| | - Harry Hemingway
- Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, UK
- Health Data Research UK, University College London, London, UK
- University College London Hospitals Biomedical Research Centre (UCLH BRC), London, UK
| |
Collapse
|
31
|
Tran L, Chi L, Bonti A, Abdelrazek M, Chen YPP. Mortality Prediction of Patients With Cardiovascular Disease Using Medical Claims Data Under Artificial Intelligence Architectures: Validation Study. JMIR Med Inform 2021; 9:e25000. [PMID: 33792549 PMCID: PMC8050753 DOI: 10.2196/25000] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 11/17/2020] [Accepted: 12/05/2020] [Indexed: 11/23/2022] Open
Abstract
Background Cardiovascular disease (CVD) is the greatest health problem in Australia, which kills more people than any other disease and incurs enormous costs for the health care system. In this study, we present a benchmark comparison of various artificial intelligence (AI) architectures for predicting the mortality rate of patients with CVD using structured medical claims data. Compared with other research in the clinical literature, our models are more efficient because we use a smaller number of features, and this study could help health professionals accurately choose AI models to predict mortality among patients with CVD using only claims data before a clinic visit. Objective This study aims to support health clinicians in accurately predicting mortality among patients with CVD using only claims data before a clinic visit. Methods The data set was obtained from the Medicare Benefits Scheme and Pharmaceutical Benefits Scheme service information in the period between 2004 and 2014, released by the Department of Health Australia in 2016. It included 346,201 records, corresponding to 346,201 patients. A total of five AI algorithms, including four classical machine learning algorithms (logistic regression [LR], random forest [RF], extra trees [ET], and gradient boosting trees [GBT]) and a deep learning algorithm, which is a densely connected neural network (DNN), were developed and compared in this study. In addition, because of the minority of deceased patients in the data set, a separate experiment using the Synthetic Minority Oversampling Technique (SMOTE) was conducted to enrich the data. Results Regarding model performance, in terms of discrimination, GBT and RF were the models with the highest area under the receiver operating characteristic curve (97.8% and 97.7%, respectively), followed by ET (96.8%) and LR (96.4%), whereas DNN was the least discriminative (95.3%). In terms of reliability, LR predictions were the least calibrated compared with the other four algorithms. In this study, despite increasing the training time, SMOTE was proven to further improve the model performance of LR, whereas other algorithms, especially GBT and DNN, worked well with class imbalanced data. Conclusions Compared with other research in the clinical literature involving AI models using claims data to predict patient health outcomes, our models are more efficient because we use a smaller number of features but still achieve high performance. This study could help health professionals accurately choose AI models to predict mortality among patients with CVD using only claims data before a clinic visit.
Collapse
Affiliation(s)
- Linh Tran
- School of Info Technology, Deakin University, Burwood, Australia
| | - Lianhua Chi
- Department of Computer Science and Information Technology, La Trobe University, Bundoora, Australia
| | - Alessio Bonti
- School of Info Technology, Deakin University, Burwood, Australia
| | | | - Yi-Ping Phoebe Chen
- Department of Computer Science and Information Technology, La Trobe University, Bundoora, Australia
| |
Collapse
|
32
|
Mahajan A, Mahajan SM. Deep Learning Methods and Their Application to Nursing Workflows: Technology and Perspectives. Comput Inform Nurs 2021; 39:1-6. [PMID: 33417313 DOI: 10.1097/cin.0000000000000702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Affiliation(s)
- Amey Mahajan
- Author Affiliations: C2OPS Inc, Cupertino, CA (Mr Mahajan); and VA Palo Alto Health Care System, Palo Alto, CA (Dr Mahajan)
| | | |
Collapse
|
33
|
Jin L, Pan R, Huang L, Zhang H, Jiang M, Zhao H. Family nursing with the assistance of network improves clinical outcome and life quality in patients underwent coronary artery bypass grafting: A consolidated standards of reporting trials-compliant randomized controlled trial. Medicine (Baltimore) 2020; 99:e23488. [PMID: 33327282 PMCID: PMC7738076 DOI: 10.1097/md.0000000000023488] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Family nursing with the assistance of network (FNAN) improves nurses' practice and provides family/community-oriented nursing care. This study aimed to explore the effects of FNAN on the clinical outcome and life quality of coronary atherosclerotic heart disease (CHD) patients underwent coronary artery bypass grafting (CABG). TRIAL DESIGN This study is a randomized, placebo-controlled and double-blind trial. METHODS One-hundred and twelve patients underwent CABG were randomly divided into control group (CG, routine family nursing care) and experimental group (EG, FNAN) and the allocation ratio was 1:1. The situation of anxiety and depression were analyzed using the Hamilton Anxiety Scale (HAMA) scale and Hamilton Depression Scale (HAMD). Sleep quality was measured by using Pittsburgh Sleep Quality Index (PSQI). Lung function parameters were measured, including minute ventilation (MVV), partial pressure of oxygen (PaO2), partial pressure of arterial carbon dioxide (PaCO2), oxygen saturation measurement by pulse oximetry (SpO2), forced expiratory volume in 1 second (FEV1) and forced vital capacity (FVC). Life quality was measured by using Chronic Obstructive Pulmonary Disease Assessment Test (CAT). RESULTS After a 3-month intervention, 10 and 6 patients were lost in the CG and EG groups, respectively. The scores of HAMA, HAMD, PSQI and CAT were reduced in the EG group when compared with the CG group (P < .05). The values of MVV, PaO2, SpO2, FEV1 and FVC in the EG group was higher than those in the CG group whereas the levels of PaCO2 in the EG group was lower than those in the CG group (P < .05). PSQI score had a strong relationship with the values of MVV, PaO2, PaCO2, SpO2, FEV1, and FVC. CONCLUSION FNAN improves the clinical outcome and life quality in the patients underwent CABG.
Collapse
|
34
|
Duong Van Huyen JP, Fedrigo M, Fishbein GA, Leone O, Neil D, Marboe C, Peyster E, von der Thüsen J, Loupy A, Mengel M, Revelo MP, Adam B, Bruneval P, Angelini A, Miller DV, Berry GJ. The XVth Banff Conference on Allograft Pathology the Banff Workshop Heart Report: Improving the diagnostic yield from endomyocardial biopsies and Quilty effect revisited. Am J Transplant 2020; 20:3308-3318. [PMID: 32476272 DOI: 10.1111/ajt.16083] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2020] [Revised: 05/14/2020] [Accepted: 05/15/2020] [Indexed: 01/25/2023]
Abstract
The XVth Banff Conference on Allograft Pathology meeting was held on September 23-27, 2019, in Pittsburgh, Pennsylvania, USA. During this meeting, two main topics in cardiac transplant pathology were addressed: (a) Improvement of endomyocardial biopsy (EMB) accuracy for the diagnosis of rejection and other significant injury patterns, and (b) the orphaned lesion known as Quilty effect or nodular endocardial infiltrates. Molecular technologies have evolved in recent years, deciphering pathophysiology of cardiac rejection. Diagnostically, it is time to integrate the histopathology of EMBs and molecular data. The goal is to incorporate molecular pathology, performed on the same paraffin block as a companion test for histopathology, to yield more accurate and objective EMB interpretation. Application of digital image analysis from hematoxylin and eosin (H&E) stain to multiplex labeling is another means of extracting additional information from EMBs. New concepts have emerged exploring the multifaceted significance of myocardial injury, minimal rejection patterns supported by molecular profiles, and lesions of arteriolitis/vasculitis in the setting of T cell-mediated rejection (TCMR) and antibody-mediated rejection (AMR). The orphaned lesion known as Quilty effect or nodular endocardial infiltrates. A state-of-the-art session with historical aspects and current dilemmas was reviewed, and possible pathogenesis proposed, based on advances in immunology to explain conflicting data. The Quilty effect will be the subject of a multicenter project to explore whether it functions as a tertiary lymphoid organ.
Collapse
Affiliation(s)
- Jean-Paul Duong Van Huyen
- Paris Translational Research Center for Organ Transplantation, INSERM U970 and Université de Paris, Paris, France.,Department of Pathology, Necker Hospital, Paris, France
| | - Marny Fedrigo
- Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padua, Padua, Italy
| | - Gregory A Fishbein
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, USA
| | - Ornella Leone
- Sant'Orsola-Malpighi University Hospital, Bologna, Italy
| | - Desley Neil
- Department of Cellular Pathology, Queen Elizabeth Hospital Birmingham and Institute of Immunology and Immunotherapy, University of Birmingham, Birmingham, UK
| | - Charles Marboe
- Department of Pathology and Cell Biology, Columbia University, New York, New York, USA
| | - Eliot Peyster
- Cardiovascular Research Institute, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | | | - Alexandre Loupy
- Paris Translational Research Center for Organ Transplantation, INSERM U970 and Université de Paris, Paris, France.,Department of Nephrology and Transplantation, Necker-Enfants Hospital, Paris, France
| | - Michael Mengel
- Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Alberta, Canada
| | - Monica P Revelo
- Department of Pathology, University of Utah, Salt Lake City, Utah, USA
| | - Benjamin Adam
- Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Alberta, Canada
| | - Patrick Bruneval
- Paris Translational Research Center for Organ Transplantation, INSERM U970 and Université de Paris, Paris, France
| | - Annalisa Angelini
- Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padua, Padua, Italy
| | | | - Gerald J Berry
- Department of Pathology, Stanford University, Stanford, California, USA
| |
Collapse
|
35
|
Olsen CR, Mentz RJ, Anstrom KJ, Page D, Patel PA. Clinical applications of machine learning in the diagnosis, classification, and prediction of heart failure. Am Heart J 2020; 229:1-17. [PMID: 32905873 DOI: 10.1016/j.ahj.2020.07.009] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/14/2020] [Accepted: 07/08/2020] [Indexed: 12/25/2022]
Abstract
Machine learning and artificial intelligence are generating significant attention in the scientific community and media. Such algorithms have great potential in medicine for personalizing and improving patient care, including in the diagnosis and management of heart failure. Many physicians are familiar with these terms and the excitement surrounding them, but many are unfamiliar with the basics of these algorithms and how they are applied to medicine. Within heart failure research, current applications of machine learning include creating new approaches to diagnosis, classifying patients into novel phenotypic groups, and improving prediction capabilities. In this paper, we provide an overview of machine learning targeted for the practicing clinician and evaluate current applications of machine learning in the diagnosis, classification, and prediction of heart failure.
Collapse
Affiliation(s)
- Cameron R Olsen
- Division of Cardiology, Duke University Medical Center, Durham, NC.
| | - Robert J Mentz
- Division of Cardiology, Duke University Medical Center, Durham, NC
| | - Kevin J Anstrom
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC
| | - David Page
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC
| | - Priyesh A Patel
- Sanger Heart and Vascular Institute, Atrium Health, Charlotte, NC
| |
Collapse
|
36
|
Masters K. Artificial Intelligence developments in medical education: a conceptual and practical framework. MEDEDPUBLISH 2020; 9:239. [PMID: 38058891 PMCID: PMC10697470 DOI: 10.15694/mep.2020.000239.1] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/08/2023] Open
Abstract
This article was migrated. The article was marked as recommended. As Artificial Intelligence (AI) develops in medicine, there is a greater awareness that medical education may also benefit from AI. Many AI projects are already underway, and many more are still to come. Most medical education administrators and educators are aware of AI, but are not necessarily familiar enough with it to understand the areas of possible application in both using AI in medical education, and also in the content areas that need to be raised with their students. As such, they are at a disadvantage of not understanding the current lie of the land, and are at an even greater disadvantage of not being able to influence and guide future AI projects. This paper attempts to provide an AI conceptual and practical framework for medical education administrators and educators, so that they may have a clearer understanding of the current situation, and may be better placed to guide future AI developments to meet their needs in medical education.
Collapse
|
37
|
Abstract
PURPOSE OF REVIEW The goal of this review is to summarize the state of big data analyses in the study of heart failure (HF). We discuss the use of big data in the HF space, focusing on "omics" and clinical data. We address some limitations of this data, as well as their future potential. RECENT FINDINGS Omics are providing insight into plasmal and myocardial molecular profiles in HF patients. The introduction of single cell and spatial technologies is a major advance that will reshape our understanding of cell heterogeneity and function as well as tissue architecture. Clinical data analysis focuses on HF phenotyping and prognostic modeling. Big data approaches are increasingly common in HF research. The use of methods designed for big data, such as machine learning, may help elucidate the biology underlying HF. However, important challenges remain in the translation of this knowledge into improvements in clinical care.
Collapse
Affiliation(s)
- Jan D Lanzer
- Institute for Computational Biomedicine, Bioquant, Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Heidelberg, Germany
- Faculty of Biosciences, Heidelberg University, Heidelberg, Germany
- Internal Medicine II, Heidelberg University Hospital, Heidelberg, Germany
| | - Florian Leuschner
- Department of Cardiology, Medical University Hospital, Heidelberg, Germany
- DZHK (German Centre for Cardiovascular Research), Heidelberg, Germany
| | - Rafael Kramann
- Department of Nephrology and Clinical Immunology, RWTH Aachen University, Aachen, Germany
- Department of Internal Medicine, Nephrology and Transplantation, Erasmus Medical Center, Rotterdam, The Netherlands
| | - Rebecca T Levinson
- Institute for Computational Biomedicine, Bioquant, Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Heidelberg, Germany
- Internal Medicine II, Heidelberg University Hospital, Heidelberg, Germany
| | - Julio Saez-Rodriguez
- Institute for Computational Biomedicine, Bioquant, Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Heidelberg, Germany.
- Joint Research Centre for Computational Biomedicine (JRC-COMBINE), Faculty of Medicine, RWTH Aachen University, Aachen, Germany.
| |
Collapse
|
38
|
Mathur P, Srivastava S, Xu X, Mehta JL. Artificial Intelligence, Machine Learning, and Cardiovascular Disease. CLINICAL MEDICINE INSIGHTS-CARDIOLOGY 2020; 14:1179546820927404. [PMID: 32952403 PMCID: PMC7485162 DOI: 10.1177/1179546820927404] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2019] [Accepted: 04/23/2020] [Indexed: 12/11/2022]
Abstract
Artificial intelligence (AI)-based applications have found widespread
applications in many fields of science, technology, and medicine. The use of
enhanced computing power of machines in clinical medicine and diagnostics has
been under exploration since the 1960s. More recently, with the advent of
advances in computing, algorithms enabling machine learning, especially deep
learning networks that mimic the human brain in function, there has been renewed
interest to use them in clinical medicine. In cardiovascular medicine, AI-based
systems have found new applications in cardiovascular imaging, cardiovascular
risk prediction, and newer drug targets. This article aims to describe different
AI applications including machine learning and deep learning and their
applications in cardiovascular medicine. AI-based applications have enhanced our
understanding of different phenotypes of heart failure and congenital heart
disease. These applications have led to newer treatment strategies for different
types of cardiovascular diseases, newer approach to cardiovascular drug therapy
and postmarketing survey of prescription drugs. However, there are several
challenges in the clinical use of AI-based applications and interpretation of
the results including data privacy, poorly selected/outdated data, selection
bias, and unintentional continuance of historical biases/stereotypes in the data
which can lead to erroneous conclusions. Still, AI is a transformative
technology and has immense potential in health care.
Collapse
Affiliation(s)
- Pankaj Mathur
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Shweta Srivastava
- Department of Radiology, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Xiaowei Xu
- Department of Information Science, University of Arkansas at Little Rock, Little Rock, AR USA
| | - Jawahar L Mehta
- Division of Cardiology, Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| |
Collapse
|
39
|
Acs B, Rantalainen M, Hartman J. Artificial intelligence as the next step towards precision pathology. J Intern Med 2020; 288:62-81. [PMID: 32128929 DOI: 10.1111/joim.13030] [Citation(s) in RCA: 177] [Impact Index Per Article: 44.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Revised: 12/16/2019] [Accepted: 12/30/2019] [Indexed: 12/13/2022]
Abstract
Pathology is the cornerstone of cancer care. The need for accuracy in histopathologic diagnosis of cancer is increasing as personalized cancer therapy requires accurate biomarker assessment. The appearance of digital image analysis holds promise to improve both the volume and precision of histomorphological evaluation. Recently, machine learning, and particularly deep learning, has enabled rapid advances in computational pathology. The integration of machine learning into routine care will be a milestone for the healthcare sector in the next decade, and histopathology is right at the centre of this revolution. Examples of potential high-value machine learning applications include both model-based assessment of routine diagnostic features in pathology, and the ability to extract and identify novel features that provide insights into a disease. Recent groundbreaking results have demonstrated that applications of machine learning methods in pathology significantly improves metastases detection in lymph nodes, Ki67 scoring in breast cancer, Gleason grading in prostate cancer and tumour-infiltrating lymphocyte (TIL) scoring in melanoma. Furthermore, deep learning models have also been demonstrated to be able to predict status of some molecular markers in lung, prostate, gastric and colorectal cancer based on standard HE slides. Moreover, prognostic (survival outcomes) deep neural network models based on digitized HE slides have been demonstrated in several diseases, including lung cancer, melanoma and glioma. In this review, we aim to present and summarize the latest developments in digital image analysis and in the application of artificial intelligence in diagnostic pathology.
Collapse
Affiliation(s)
- B Acs
- From the, Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
| | - M Rantalainen
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - J Hartman
- From the, Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
| |
Collapse
|
40
|
Abstract
This article presents a real-time approach for classification of burn depth based on B-mode ultrasound imaging. A grey-level co-occurrence matrix (GLCM) computed from the ultrasound images of the tissue is employed to construct the textural feature set and the classification is performed using nonlinear support vector machine and kernel Fisher discriminant analysis. A leave-one-out cross-validation is used for the independent assessment of the classifiers. The model is tested for pair-wise binary classification of four burn conditions in ex vivo porcine skin tissue: (i) 200 °F for 10 s, (ii) 200 °F for 30 s, (iii) 450 °F for 10 s, and (iv) 450 °F for 30 s. The average classification accuracy for pairwise separation is 99% with just over 30 samples in each burn group and the average multiclass classification accuracy is 93%. The results highlight that the ultrasound imaging-based burn classification approach in conjunction with the GLCM texture features provide an accurate assessment of altered tissue characteristics with relatively moderate sample sizes, which is often the case with experimental and clinical datasets. The proposed method is shown to have the potential to assist with the real-time clinical assessment of burn degrees, particularly for discriminating between superficial and deep second degree burns, which is challenging in clinical practice.
Collapse
|
41
|
Peyster EG, Wang C, Ishola F, Remeniuk B, Hoyt C, Feldman MD, Margulies KB. In Situ Immune Profiling of Heart Transplant Biopsies Improves Diagnostic Accuracy and Rejection Risk Stratification. JACC Basic Transl Sci 2020; 5:328-340. [PMID: 32368693 PMCID: PMC7188920 DOI: 10.1016/j.jacbts.2020.01.015] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2019] [Revised: 01/24/2020] [Accepted: 01/24/2020] [Indexed: 12/15/2022]
Abstract
Recognizing that guideline-directed histologic grading of endomyocardial biopsy tissue samples for rejection surveillance has limited diagnostic accuracy, quantitative, in situ characterization was performed of several important immune cell types in a retrospective cohort of clinical endomyocardial tissue samples. Differences between cases were identified and were grouped by histologic grade versus clinical rejection trajectory, with significantly increased programmed death ligand 1+, forkhead box P3+, and cluster of differentiation 68+ cells suppressed in clinically evident rejections, especially cases with marked clinical-histologic discordance. Programmed death ligand 1+, forkhead box P3+, and cluster of differentiation 68+ cell proportions are also significantly higher in "never-rejection" when compared with "future-rejection." These findings suggest that in situ immune modulators regulate the severity of cardiac allograft rejection.
Collapse
Affiliation(s)
- Eliot G Peyster
- Cardiovascular Research Institute, University of Pennsylvania, Philadelphia, Pennsylvania
| | | | | | | | | | - Michael D Feldman
- Department of Pathology and Laboratory Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Kenneth B Margulies
- Cardiovascular Research Institute, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
42
|
Kubach J, Muhlebner-Fahrngruber A, Soylemezoglu F, Miyata H, Niehusmann P, Honavar M, Rogerio F, Kim SH, Aronica E, Garbelli R, Vilz S, Popp A, Walcher S, Neuner C, Scholz M, Kuerten S, Schropp V, Roeder S, Eichhorn P, Eckstein M, Brehmer A, Kobow K, Coras R, Blumcke I, Jabari S. Same same but different: A Web-based deep learning application revealed classifying features for the histopathologic distinction of cortical malformations. Epilepsia 2020; 61:421-432. [PMID: 32080846 DOI: 10.1111/epi.16447] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2019] [Revised: 01/23/2020] [Accepted: 01/23/2020] [Indexed: 12/14/2022]
Abstract
OBJECTIVE The microscopic review of hematoxylin-eosin-stained images of focal cortical dysplasia type IIb and cortical tuber of tuberous sclerosis complex remains challenging. Both entities are distinct subtypes of human malformations of cortical development that share histopathological features consisting of neuronal dyslamination with dysmorphic neurons and balloon cells. We trained a convolutional neural network (CNN) to classify both entities and visualize the results. Additionally, we propose a new Web-based deep learning application as proof of concept of how deep learning could enter the pathologic routine. METHODS A digital processing pipeline was developed for a series of 56 cases of focal cortical dysplasia type IIb and cortical tuber of tuberous sclerosis complex to obtain 4000 regions of interest and 200 000 subsamples with different zoom and rotation angles to train a neural network. Guided gradient-weighted class activation maps (Guided Grad-CAMs) were generated to visualize morphological features used by the CNN to distinguish both entities. RESULTS Our best-performing network achieved 91% accuracy and 0.88 area under the receiver operating characteristic curve at the tile level for an unseen test set. Novel histopathologic patterns were found through the visualized Guided Grad-CAMs. These patterns were assembled into a classification score to augment decision-making in routine histopathology workup. This score was successfully validated by 11 expert neuropathologists and 12 nonexperts, boosting nonexperts to expert level performance. SIGNIFICANCE Our newly developed Web application combines the visualization of whole slide images with the possibility of deep learning-aided classification between focal cortical dysplasia IIb and tuberous sclerosis complex. This approach will help to introduce deep learning applications and visualization for the histopathologic diagnosis of rare and difficult-to-classify brain lesions.
Collapse
Affiliation(s)
- Joshua Kubach
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Angelika Muhlebner-Fahrngruber
- Department of (Neuro)Pathology, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, the Netherlands
| | - Figen Soylemezoglu
- Department of Pathology, Faculty of Medicine, Hacettepe University, Ankara, Turkey
| | - Hajime Miyata
- Department of Neuropathology, Research Institute for Brain and Blood Vessels, Akita Cerebrospinal and Cardiovascular Center, Akita, Japan
| | - Pitt Niehusmann
- Department of Neurology/Pathology, Oslo University Hospital, Oslo, Norway
| | - Mrinalini Honavar
- Department of Anatomic Pathology, Pedro Hispano Hospital, Matosinhos, Portugal
| | - Fabio Rogerio
- Department of Pathology, State University of Campinas, Campinas, Brazil
| | - Se-Hoon Kim
- Department of Pathology, Yonsei University College of Medicine, Seoul, Korea
| | - Eleonora Aronica
- Department of (Neuro)Pathology, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, the Netherlands.,Stichting Epilepsie Instellingen Nederland, Zwolle, the Netherlands
| | | | - Samuel Vilz
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Alexander Popp
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Stefan Walcher
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Christoph Neuner
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Michael Scholz
- Institute of Anatomy, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
| | - Stefanie Kuerten
- Institute of Anatomy, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
| | - Verena Schropp
- Institute of Anatomy, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
| | - Sebastian Roeder
- Department of Neurology, University Hospitals Erlangen, Erlangen, Germany
| | - Philip Eichhorn
- Institute of Pathology, University Hospitals Erlangen, Erlangen, Germany
| | - Markus Eckstein
- Institute of Pathology, University Hospitals Erlangen, Erlangen, Germany
| | - Axel Brehmer
- Institute of Anatomy, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
| | - Katja Kobow
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Roland Coras
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Ingmar Blumcke
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| | - Samir Jabari
- Institute of Neuropathology, University Hospitals, Erlangen, Germany
| |
Collapse
|
43
|
Orbit Image Analysis: An open-source whole slide image analysis tool. PLoS Comput Biol 2020; 16:e1007313. [PMID: 32023239 PMCID: PMC7028292 DOI: 10.1371/journal.pcbi.1007313] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2019] [Revised: 02/18/2020] [Accepted: 12/11/2019] [Indexed: 11/20/2022] Open
Abstract
We describe Orbit Image Analysis, an open-source whole slide image analysis tool. The tool consists of a generic tile-processing engine which allows the execution of various image analysis algorithms provided by either Orbit itself or from other open-source platforms using a tile-based map-reduce execution framework. Orbit Image Analysis is capable of sophisticated whole slide imaging analyses due to several key features. First, Orbit has machine-learning capabilities. This deep learning segmentation can be integrated with complex object detection for analysis of intricate tissues. In addition, Orbit can run locally as standalone or connect to the open-source image server OMERO. Another important characteristic is its scale-out functionality, using the Apache Spark framework for distributed computing. In this paper, we describe the use of Orbit in three different real-world applications: quantification of idiopathic lung fibrosis, nerve fibre density quantification, and glomeruli detection in the kidney.
Collapse
|
44
|
Schau GF, Burlingame EA, Thibault G, Anekpuritanang T, Wang Y, Gray JW, Corless C, Chang YH. Predicting primary site of secondary liver cancer with a neural estimator of metastatic origin. J Med Imaging (Bellingham) 2020; 7:012706. [PMID: 34541020 PMCID: PMC8441834 DOI: 10.1117/1.jmi.7.1.012706] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Accepted: 02/03/2020] [Indexed: 12/20/2022] Open
Abstract
Purpose: Pathologists rely on relevant clinical information, visual inspection of stained tissue slide morphology, and sophisticated molecular diagnostics to accurately infer the biological origin of secondary metastatic cancer. While highly effective, this process is expensive in terms of time and clinical resources. We seek to develop and evaluate a computer vision system designed to reasonably infer metastatic origin of secondary liver cancer directly from digitized histopathological whole slide images of liver biopsy. Approach: We illustrate a two-stage deep learning approach to accomplish this task. We first train a model to identify spatially localized regions of cancerous tumor within digitized hematoxylin and eosin (H&E)-stained tissue sections of secondary liver cancer based on a pathologist's annotation of several whole slide images. Then, a second model is trained to generate predictions of the cancers' metastatic origin belonging to one of three distinct clinically relevant classes as confirmed by immunohistochemistry. Results: Our approach achieves a classification accuracy of 90.2% in determining metastatic origin of whole slide images from a held-out test set, which compares favorably to an established clinical benchmark by three board-certified pathologists whose accuracies ranged from 90.2% to 94.1% on the same prediction task. Conclusions: We illustrate the potential impact of deep learning systems to leverage morphological and structural features of H&E-stained tissue sections to guide pathological and clinical determination of the metastatic origin of secondary liver cancers.
Collapse
Affiliation(s)
- Geoffrey F. Schau
- Oregon Health and Science University, Computational Biology Program, Biomedical Engineering Department, Portland, Oregon, United States
- Oregon Health and Science University, OHSU Center for Spatial Systems Biomedicine, Biomedical Engineering Department, Portland, Oregon, United States
| | - Erik A. Burlingame
- Oregon Health and Science University, Computational Biology Program, Biomedical Engineering Department, Portland, Oregon, United States
- Oregon Health and Science University, OHSU Center for Spatial Systems Biomedicine, Biomedical Engineering Department, Portland, Oregon, United States
| | - Guillaume Thibault
- Oregon Health and Science University, OHSU Center for Spatial Systems Biomedicine, Biomedical Engineering Department, Portland, Oregon, United States
| | - Tauangtham Anekpuritanang
- Oregon Health and Science University, Knight Diagnostic Laboratories, Portland, Oregon, United States
- Mahidol University, Department of Pathology, Faculty of Medicine Siriraj Hospital, Bangkok, Thailand
| | - Ying Wang
- Oregon Health and Science University, Knight Diagnostic Laboratories, Portland, Oregon, United States
| | - Joe W. Gray
- Oregon Health and Science University, OHSU Center for Spatial Systems Biomedicine, Biomedical Engineering Department, Portland, Oregon, United States
- Oregon Health and Science University, Knight Cancer Institute, Portland, Oregon, United States
| | - Christopher Corless
- Oregon Health and Science University, Knight Diagnostic Laboratories, Portland, Oregon, United States
- Oregon Health and Science University, Knight Cancer Institute, Portland, Oregon, United States
- Oregon Health and Science University, Department of Pathology, Portland, Oregon, United States
| | - Young H. Chang
- Oregon Health and Science University, Computational Biology Program, Biomedical Engineering Department, Portland, Oregon, United States
- Oregon Health and Science University, OHSU Center for Spatial Systems Biomedicine, Biomedical Engineering Department, Portland, Oregon, United States
- Oregon Health and Science University, Knight Cancer Institute, Portland, Oregon, United States
| |
Collapse
|
45
|
Muscogiuri G, Chiesa M, Trotta M, Gatti M, Palmisano V, Dell'Aversana S, Baessato F, Cavaliere A, Cicala G, Loffreno A, Rizzon G, Guglielmo M, Baggiano A, Fusini L, Saba L, Andreini D, Pepi M, Rabbat MG, Guaricci AI, De Cecco CN, Colombo G, Pontone G. Performance of a deep learning algorithm for the evaluation of CAD-RADS classification with CCTA. Atherosclerosis 2019; 294:25-32. [PMID: 31945615 DOI: 10.1016/j.atherosclerosis.2019.12.001] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Revised: 12/01/2019] [Accepted: 12/06/2019] [Indexed: 12/22/2022]
Abstract
BACKGROUND AND AIMS Artificial intelligence (AI) is increasing its role in diagnosis of patients with suspicious coronary artery disease. The aim of this manuscript is to develop a deep convolutional neural network (CNN) to classify coronary computed tomography angiography (CCTA) in the correct Coronary Artery Disease Reporting and Data System (CAD-RADS) category. METHODS Two hundred eighty eight patients who underwent clinically indicated CCTA were included in this single-center retrospective study. The CCTAs were stratified by CAD-RADS scores by expert readers and considered as reference standard. A deep CNN was designed and tested on the CCTA dataset and compared to on-site reading. The deep CNN analyzed the diagnostic accuracy of the following three Models based on CAD-RADS classification: Model A (CAD-RADS 0 vs CAD-RADS 1-2 vs CAD-RADS 3,4,5), Model 1 (CAD-RADS 0 vs CAD-RADS>0), Model 2 (CAD-RADS 0-2 vs CAD-RADS 3-5). Time of analysis for both physicians and CNN were recorded. RESULTS Model A showed a sensitivity, specificity, negative predictive value, positive predictive value and accuracy of 47%, 74%, 77%, 46% and 60%, respectively. Model 1 showed a sensitivity, specificity, negative predictive value, positive predictive value and accuracy of 66%, 91%, 92%, 63%, 86%, respectively. Conversely, Model 2 demonstrated the following sensitivity, specificity, negative predictive value, positive predictive value and accuracy: 82%, 58%, 74%, 69%, 71%, respectively. Time of analysis was significantly lower using CNN as compared to on-site reading (530.5 ± 179.1 vs 104.3 ± 1.4 sec, p=0.01) CONCLUSIONS: Deep CNN yielded accurate automated classification of patients with CAD-RADS.
Collapse
Affiliation(s)
| | | | - Michela Trotta
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Marco Gatti
- Department of Surgical Sciences, Radiology Institute, University of Turin, Turin, Italy
| | - Vitanio Palmisano
- Department of Medical Imaging, University of Cagliari, Monserrato, Italy
| | - Serena Dell'Aversana
- Department of Advanced Biomedical Sciences, University of Naples "Federico II,", Naples, Italy
| | - Francesca Baessato
- Section of Cardiology, Department of Medicine, University of Verona, Verona, Italy
| | - Annachiara Cavaliere
- Department of Medicine, Institute of Radiology, University of Padova, Padua, Italy
| | - Gloria Cicala
- Section of Radiology, Department of Medicine and Surgery, University of Parma, Parma, Italy
| | | | - Giulia Rizzon
- Department of Medicine, Institute of Radiology, University of Padova, Padua, Italy
| | | | | | - Laura Fusini
- Centro Cardiologico Monzino, IRCCS, Milan, Italy
| | - Luca Saba
- Department of Medical Imaging, University of Cagliari, Monserrato, Italy
| | - Daniele Andreini
- Centro Cardiologico Monzino, IRCCS, Milan, Italy; Department of Cardiovascular Sciences and Community Health, University of Milan, Italy
| | - Mauro Pepi
- Centro Cardiologico Monzino, IRCCS, Milan, Italy
| | - Mark G Rabbat
- Loyola University of Chicago, Chicago, IL, USA; Edward Hines Jr. VA Hospital, Hines, IL, USA
| | - Andrea I Guaricci
- Institute of Cardiovascular Disease, Department of Emergency and Organ Transplantation, University Hospital "Policlinico Consorziale" of Bari, Bari, Italy
| | - Carlo N De Cecco
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA
| | | | | |
Collapse
|
46
|
Bándi P, Balkenhol M, van Ginneken B, van der Laak J, Litjens G. Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks. PeerJ 2019; 7:e8242. [PMID: 31871843 PMCID: PMC6924324 DOI: 10.7717/peerj.8242] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2019] [Accepted: 11/19/2019] [Indexed: 12/20/2022] Open
Abstract
Modern pathology diagnostics is being driven toward large scale digitization of microscopic tissue sections. A prerequisite for its safe implementation is the guarantee that all tissue present on a glass slide can also be found back in the digital image. Whole-slide scanners perform a tissue segmentation in a low resolution overview image to prevent inefficient high-resolution scanning of empty background areas. However, currently applied algorithms can fail in detecting all tissue regions. In this study, we developed convolutional neural networks to distinguish tissue from background. We collected 100 whole-slide images of 10 tissue samples—staining categories from five medical centers for development and testing. Additionally, eight more images of eight unfamiliar categories were collected for testing only. We compared our fully-convolutional neural networks to three traditional methods on a range of resolution levels using Dice score and sensitivity. We also tested whether a single neural network can perform equivalently to multiple networks, each specialized in a single resolution. Overall, our solutions outperformed the traditional methods on all the tested resolutions. The resolution-agnostic network achieved average Dice scores between 0.97 and 0.98 across the tested resolution levels, only 0.0069 less than the resolution-specific networks. Finally, its excellent generalization performance was demonstrated by achieving averages of 0.98 Dice score and 0.97 sensitivity on the eight unfamiliar images. A future study should test this network prospectively.
Collapse
Affiliation(s)
- Péter Bándi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Maschenka Balkenhol
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
47
|
Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G, Li B, Madabhushi A, Shah P, Spitzer M, Zhao S. Applications of machine learning in drug discovery and development. Nat Rev Drug Discov 2019; 18:463-477. [PMID: 30976107 DOI: 10.1038/s41573-019-0024-5] [Citation(s) in RCA: 931] [Impact Index Per Article: 186.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Drug discovery and development pipelines are long, complex and depend on numerous factors. Machine learning (ML) approaches provide a set of tools that can improve discovery and decision making for well-specified questions with abundant, high-quality data. Opportunities to apply ML occur in all stages of drug discovery. Examples include target validation, identification of prognostic biomarkers and analysis of digital pathology data in clinical trials. Applications have ranged in context and methodology, with some approaches yielding accurate predictions and insights. The challenges of applying ML lie primarily with the lack of interpretability and repeatability of ML-generated results, which may limit their application. In all areas, systematic and comprehensive high-dimensional data still need to be generated. With ongoing efforts to tackle these issues, as well as increasing awareness of the factors needed to validate ML approaches, the application of ML can promote data-driven decision making and has the potential to speed up the process and reduce failure rates in drug discovery and development.
Collapse
Affiliation(s)
- Jessica Vamathevan
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
| | - Dominic Clark
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | | | - Ian Dunham
- Open Targets and European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | - Edgardo Ferran
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | - George Lee
- Bristol-Myers Squibb, Princeton, NJ, USA
| | - Bin Li
- Takeda Pharmaceuticals International Co., Cambridge, MA, USA
| | - Anant Madabhushi
- Case Western Reserve University, Cleveland, OH, USA.,Louis Stokes Cleveland Veterans Affair Medical Center, Cleveland, OH, USA
| | | | - Michaela Spitzer
- Open Targets and European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | - Shanrong Zhao
- Pfizer Worldwide Research and Development, Cambridge, MA, USA
| |
Collapse
|
48
|
Turner OC, Aeffner F, Bangari DS, High W, Knight B, Forest T, Cossic B, Himmel LE, Rudmann DG, Bawa B, Muthuswamy A, Aina OH, Edmondson EF, Saravanan C, Brown DL, Sing T, Sebastian MM. Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group Article*: Opinion on the Application of Artificial Intelligence and Machine Learning to Digital Toxicologic Pathology. Toxicol Pathol 2019; 48:277-294. [DOI: 10.1177/0192623319881401] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Toxicologic pathology is transitioning from analog to digital methods. This transition seems inevitable due to a host of ongoing social and medical technological forces. Of these, artificial intelligence (AI) and in particular machine learning (ML) are globally disruptive, rapidly growing sectors of technology whose impact on the long-established field of histopathology is quickly being realized. The development of increasing numbers of algorithms, peering ever deeper into the histopathological space, has demonstrated to the scientific community that AI pathology platforms are now poised to truly impact the future of precision and personalized medicine. However, as with all great technological advances, there are implementation and adoption challenges. This review aims to define common and relevant AI and ML terminology, describe data generation and interpretation, outline current and potential future business cases, discuss validation and regulatory hurdles, and most importantly, propose how overcoming the challenges of this burgeoning technology may shape toxicologic pathology for years to come, enabling pathologists to contribute even more effectively to answering scientific questions and solving global health issues. [Box: see text]
Collapse
Affiliation(s)
- Oliver C. Turner
- Novartis, Novartis Institutes for Biomedical Research, Preclinical Safety, East Hanover, NJ, USA
| | - Famke Aeffner
- Amgen Inc, Research, Comparative Biology and Safety Sciences, San Francisco, CA, USA
| | | | - Wanda High
- High Preclinical Pathology Consulting, Rochester, NY, USA
| | - Brian Knight
- Boehringer Ingelheim Pharmaceuticals Incorporated, Nonclinical Drug Safety, Ridgefield, CT, USA
| | | | - Brieuc Cossic
- Roche, Pharmaceutical Research and Early Development (pRED), Roche Innovation Center, Basel, Switzerland
| | - Lauren E. Himmel
- Division of Animal Care, Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, TN, USA
| | | | | | | | | | - Elijah F. Edmondson
- Pathology/Histotechnology Laboratory, Frederick National Laboratory for Cancer Research, NIH, Frederick, MD, USA
| | - Chandrassegar Saravanan
- Novartis, Novartis Institutes for Biomedical Research, Preclinical Safety, Cambridge, MA, USA
| | | | - Tobias Sing
- Novartis, Novartis Institutes for Biomedical Research, NIBR Informatics, Basel, Switzerland
| | - Manu M. Sebastian
- Department of Epigenetics and Molecular Carcinogenesis, The University of Texas MD Anderson Cancer Center, Smithville, TX, USA
| |
Collapse
|
49
|
Sun H, Zeng X, Xu T, Peng G, Ma Y. Computer-Aided Diagnosis in Histopathological Images of the Endometrium Using a Convolutional Neural Network and Attention Mechanisms. IEEE J Biomed Health Inform 2019; 24:1664-1676. [PMID: 31581102 DOI: 10.1109/jbhi.2019.2944977] [Citation(s) in RCA: 52] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Uterine cancer (also known as endometrial cancer) can seriously affect the female reproductive system, and histopathological image analysis is the gold standard for diagnosing endometrial cancer. Due to the limited ability to model the complicated relationships between histopathological images and their interpretations, existing computer-aided diagnosis (CAD) approaches using traditional machine learning algorithms often failed to achieve satisfying results. In this study, we develop a CAD approach based on a convolutional neural network (CNN) and attention mechanisms, called HIENet. In the ten-fold cross-validation on ∼3,300 hematoxylin and eosin (H&E) image patches from ∼500 endometrial specimens, HIENet achieved a 76.91 ± 1.17% (mean ± s. d.) accuracy for four classes of endometrial tissue, i.e., normal endometrium, endometrial polyp, endometrial hyperplasia, and endometrial adenocarcinoma. Also, HIENet obtained an area-under-the-curve (AUC) of 0.9579 ± 0.0103 with an 81.04 ± 3.87% sensitivity and 94.78 ± 0.87% specificity in a binary classification task that detected endometrioid adenocarcinoma. Besides, in the external validation on 200 H&E image patches from 50 randomly-selected female patients, HIENet achieved an 84.50% accuracy in the four-class classification task, as well as an AUC of 0.9829 with a 77.97% (95% confidence interval, CI, 65.27%∼87.71%) sensitivity and 100% (95% CI, 97.42%∼100.00%) specificity. The proposed CAD method outperformed three human experts and five CNN-based classifiers regarding overall classification performance. It was also able to provide pathologists better interpretability of diagnoses by highlighting the histopathological correlations of local pixel-level image features to morphological characteristics of endometrial tissue.
Collapse
|
50
|
Abstract
Due to the massive data sets available for drug candidates, modern drug discovery has advanced to the big data era. Central to this shift is the development of artificial intelligence approaches to implementing innovative modeling based on the dynamic, heterogeneous, and large nature of drug data sets. As a result, recently developed artificial intelligence approaches such as deep learning and relevant modeling studies provide new solutions to efficacy and safety evaluations of drug candidates based on big data modeling and analysis. The resulting models provided deep insights into the continuum from chemical structure to in vitro, in vivo, and clinical outcomes. The relevant novel data mining, curation, and management techniques provided critical support to recent modeling studies. In summary, the new advancement of artificial intelligence in the big data era has paved the road to future rational drug development and optimization, which will have a significant impact on drug discovery procedures and, eventually, public health.
Collapse
Affiliation(s)
- Hao Zhu
- Department of Chemistry and Center for Computational and Integrative Biology, Rutgers University, Camden, New Jersey 08102, USA;
| |
Collapse
|