1
|
Jaume G, de Brot S, Song AH, Williamson DFK, Oldenburg L, Zhang A, Chen RJ, Asin J, Blatter S, Dettwiler M, Goepfert C, Grau-Roma L, Soto S, Keller SM, Rottenberg S, del-Pozo J, Pettit R, Le LP, Mahmood F. Deep Learning-based Modeling for Preclinical Drug Safety Assessment. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.20.604430. [PMID: 39091793 PMCID: PMC11291027 DOI: 10.1101/2024.07.20.604430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/04/2024]
Abstract
In drug development, assessing the toxicity of candidate compounds is crucial for successfully transitioning from preclinical research to early-stage clinical trials. Drug safety is typically assessed using animal models with a manual histopathological examination of tissue sections to characterize the dose-response relationship of the compound - a time-intensive process prone to inter-observer variability and predominantly involving tedious review of cases without abnormalities. Artificial intelligence (AI) methods in pathology hold promise to accelerate this assessment and enhance reproducibility and objectivity. Here, we introduce TRACE, a model designed for toxicologic liver histopathology assessment capable of tackling a range of diagnostic tasks across multiple scales, including situations where labeled data is limited. TRACE was trained on 15 million histopathology images extracted from 46,734 digitized tissue sections from 157 preclinical studies conducted on Rattus norvegicus. We show that TRACE can perform various downstream toxicology tasks spanning histopathological response assessment, lesion severity scoring, morphological retrieval, and automatic dose-response characterization. In an independent reader study, TRACE was evaluated alongside ten board-certified veterinary pathologists and achieved higher concordance with the consensus opinion than the average of the pathologists. Our study represents a substantial leap over existing computational models in toxicology by offering the first framework for accelerating and automating toxicological pathology assessment, promoting significant progress with faster, more consistent, and reliable diagnostic processes.
Collapse
Affiliation(s)
- Guillaume Jaume
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA
| | - Simone de Brot
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
- COMPATH, Institute of Animal Pathology, University of Bern, Switzerland
- Bern Center for Precision Medicine, University of Bern, Switzerland
| | - Andrew H. Song
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA
| | - Drew F. K. Williamson
- Department of Pathology & Laboratory Medicine, Emory University School of Medicine, Atlanta, GA
| | - Lukas Oldenburg
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA
| | - Richard J. Chen
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA
| | - Javier Asin
- California Animal Health and Food Safety Laboratory, University of California-Davis, San Bernardino, CA
- School of Veterinary Medicine, Department of Pathology, University of California-Davis, Davis, CA
| | - Sohvi Blatter
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
| | | | - Christine Goepfert
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
- COMPATH, Institute of Animal Pathology, University of Bern, Switzerland
| | - Llorenç Grau-Roma
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
- COMPATH, Institute of Animal Pathology, University of Bern, Switzerland
| | - Sara Soto
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
| | | | - Sven Rottenberg
- Institute of Animal Pathology, Vetsuisse, University of Bern, Switzerland
- COMPATH, Institute of Animal Pathology, University of Bern, Switzerland
- Bern Center for Precision Medicine, University of Bern, Switzerland
- Department for BioMedical Research, University of Bern, Switzerland
| | - Jorge del-Pozo
- Royal (Dick) School of Veterinary Studies, Roslin, United-Kingdom
| | - Rowland Pettit
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
| | - Long Phi Le
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Harvard Data Science Initiative, Harvard University, Cambridge, MA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA
- Harvard Data Science Initiative, Harvard University, Cambridge, MA
| |
Collapse
|
2
|
Zingman I, Stierstorfer B, Lempp C, Heinemann F. Learning image representations for anomaly detection: Application to discovery of histological alterations in drug development. Med Image Anal 2024; 92:103067. [PMID: 38141454 DOI: 10.1016/j.media.2023.103067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Revised: 12/01/2023] [Accepted: 12/19/2023] [Indexed: 12/25/2023]
Abstract
We present a system for anomaly detection in histopathological images. In histology, normal samples are usually abundant, whereas anomalous (pathological) cases are scarce or not available. Under such settings, one-class classifiers trained on healthy data can detect out-of-distribution anomalous samples. Such approaches combined with pre-trained Convolutional Neural Network (CNN) representations of images were previously employed for anomaly detection (AD). However, pre-trained off-the-shelf CNN representations may not be sensitive to abnormal conditions in tissues, while natural variations of healthy tissue may result in distant representations. To adapt representations to relevant details in healthy tissue we propose training a CNN on an auxiliary task that discriminates healthy tissue of different species, organs, and staining reagents. Almost no additional labeling workload is required, since healthy samples come automatically with aforementioned labels. During training we enforce compact image representations with a center-loss term, which further improves representations for AD. The proposed system outperforms established AD methods on a published dataset of liver anomalies. Moreover, it provided comparable results to conventional methods specifically tailored for quantification of liver anomalies. We show that our approach can be used for toxicity assessment of candidate drugs at early development stages and thereby may reduce expensive late-stage drug attrition.
Collapse
Affiliation(s)
- Igor Zingman
- Drug Discovery Sciences, Boehringer Ingelheim Pharma GmbH and Co., Biberach an der Riß, Germany.
| | - Birgit Stierstorfer
- Non-Clinical Drug Safety, Boehringer Ingelheim Pharma GmbH and Co., Biberach an der Riß, Germany
| | - Charlotte Lempp
- Drug Discovery Sciences, Boehringer Ingelheim Pharma GmbH and Co., Biberach an der Riß, Germany
| | - Fabian Heinemann
- Drug Discovery Sciences, Boehringer Ingelheim Pharma GmbH and Co., Biberach an der Riß, Germany.
| |
Collapse
|
3
|
Maedera S, Mizuno T, Kusuhara H. Investigation of latent representation of toxicopathological images extracted by CNN model for understanding compound properties in vivo. Comput Biol Med 2024; 168:107748. [PMID: 38016375 DOI: 10.1016/j.compbiomed.2023.107748] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Revised: 10/25/2023] [Accepted: 11/20/2023] [Indexed: 11/30/2023]
Abstract
Toxicopathological images acquired during safety assessment elucidate an individual's biological responses to a given compound, and their numerization can yield valuable insights contributing to the assessment of compound properties. Currently, toxicopathological images are mainly encoded as pathological findings, evaluated by pathologists, which introduces challenges when used as input for modeling, specifically in terms of representation capability and comparability. In this study, we assessed the usefulness of latent representations extracted from toxicopathological images using Convolutional Neural Network (CNN) in estimating compound properties in vivo. Special emphasis was placed on examining the impact of learning pathological findings, the depth of frozen layers during learning, and the selection of the layer for latent representation. Our findings demonstrate that a machine learning model fed with the latent representation as input surpassed the performance of a model directly employing pathological findings as input, particularly in the classification of a compound's Mechanism of Action and in predicting late-phase findings from early-phase images in repeated-dose tests. While learning pathological findings did improve accuracy, the magnitude of improvement was relatively modest. Similarly, the effect of freezing layers during learning was also limited. Notably, the selection of the layer for latent representation had a substantial impact on the accurate estimation of compound properties in vivo.
Collapse
Affiliation(s)
- Shotaro Maedera
- Laboratory of Molecular Pharmacokinetics, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, Japan
| | - Tadahaya Mizuno
- Laboratory of Molecular Pharmacokinetics, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, Japan.
| | - Hiroyuki Kusuhara
- Laboratory of Molecular Pharmacokinetics, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo, Japan
| |
Collapse
|
4
|
Bonnevie ED, Dobrzynski E, Steiner D, Hildebrand D, Monslow J, Singh M, Decman V, Krull DL. A machine learning approach toward automating spatial identification of LAG3+/CD3+ cells in ulcerative colitis. Sci Rep 2023; 13:21759. [PMID: 38066073 PMCID: PMC10709428 DOI: 10.1038/s41598-023-49163-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 12/05/2023] [Indexed: 12/18/2023] Open
Abstract
Over the past decade, automation of digital image analysis has become commonplace in both research and clinical settings. Spurred by recent advances in artificial intelligence and machine learning (AI/ML), tissue sub-compartments and cellular phenotypes within those compartments can be identified with higher throughput and accuracy than ever before. Recently, immune checkpoints have emerged as potential targets for auto-immune diseases. As such, spatial identification of these proteins along with immune cell markers (e.g., CD3+/LAG3+ T-cells) is a crucial step in understanding the potential and/or efficacy of such treatments. Here, we describe a semi-automated imaging and analysis pipeline that identifies CD3+/LAG3+ cells in colorectal tissue sub-compartments. While chromogenic staining has been a clinical mainstay and the resulting brightfield images have been utilized in AI/ML approaches in the past, there are associated drawbacks in phenotyping algorithms that can be overcome by fluorescence imaging. To address these tradeoffs, we developed an analysis pipeline combining the strengths of brightfield and fluorescence images. In this assay, immunofluorescence imaging was conducted to identify phenotypes followed by coverslip removal and hematoxylin and eosin staining of the same section to inform an AI/ML tissue segmentation algorithm. This assay proved to be robust in both tissue segmentation and phenotyping, was compatible with automated workflows, and revealed presence of LAG3+ T-cells in ulcerative colitis biopsies with spatial context preserved.
Collapse
Affiliation(s)
| | | | | | | | | | - Mohan Singh
- Cellular Biomarkers, GSK, Upper Providence, USA
| | | | | |
Collapse
|
5
|
Shiffman S, Rios Piedra EA, Adedeji AO, Ruff CF, Andrews RN, Katavolos P, Liu E, Forster A, Brumm J, Fuji RN, Sullivan R. Analysis of cellularity in H&E-stained rat bone marrow tissue via deep learning. J Pathol Inform 2023; 14:100333. [PMID: 37743975 PMCID: PMC10514468 DOI: 10.1016/j.jpi.2023.100333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/18/2023] [Accepted: 08/19/2023] [Indexed: 09/26/2023] Open
Abstract
Our objective was to develop an automated deep-learning-based method to evaluate cellularity in rat bone marrow hematoxylin and eosin whole slide images for preclinical safety assessment. We trained a shallow CNN for segmenting marrow, 2 Mask R-CNN models for segmenting megakaryocytes (MKCs), and small hematopoietic cells (SHCs), and a SegNet model for segmenting red blood cells. We incorporated the models into a pipeline that identifies and counts MKCs and SHCs in rat bone marrow. We compared cell segmentation and counts that our method generated to those that pathologists generated on 10 slides with a range of cell depletion levels from 10 studies. For SHCs, we compared cell counts that our method generated to counts generated by Cellpose and Stardist. The median Dice and object Dice scores for MKCs using our method vs pathologist consensus and the inter- and intra-pathologist variation were comparable, with overlapping first-third quartile ranges. For SHCs, the median scores were close, with first-third quartile ranges partially overlapping intra-pathologist variation. For SHCs, in comparison to Cellpose and Stardist, counts from our method were closer to pathologist counts, with a smaller 95% limits of agreement range. The performance of the bone marrow analysis pipeline supports its incorporation into routine use as an aid for hematotoxicity assessment by pathologists. The pipeline could help expedite hematotoxicity assessment in preclinical studies and consequently could expedite drug development. The method may enable meta-analysis of rat bone marrow characteristics from future and historical whole slide images and may generate new biological insights from cross-study comparisons.
Collapse
Affiliation(s)
- Smadar Shiffman
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Edgar A. Rios Piedra
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Adeyemi O. Adedeji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Catherine F. Ruff
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Rachel N. Andrews
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Paula Katavolos
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- Bristol Myers Squibb, New Brunswick, NJ 08901, USA
| | - Evan Liu
- Genentech Research and Early Development (gRED), Department of Development Sciences Informatics, Genentech Inc, South San Francisco, USA
| | - Ashley Forster
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- University of Pennsylvania School of Veterinary Medicine, Philadelphia, PA 19104, USA
| | - Jochen Brumm
- Genentech Research and Early Development (gRED), Department of Nonclinical Biostatistics, Genentech Inc, South San Francisco, USA
| | - Reina N. Fuji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Ruth Sullivan
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| |
Collapse
|
6
|
Obrecht M, Zurbruegg S, Accart N, Lambert C, Doelemeyer A, Ledermann B, Beckmann N. Magnetic resonance imaging and ultrasound elastography in the context of preclinical pharmacological research: significance for the 3R principles. Front Pharmacol 2023; 14:1177421. [PMID: 37448960 PMCID: PMC10337591 DOI: 10.3389/fphar.2023.1177421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 06/16/2023] [Indexed: 07/18/2023] Open
Abstract
The 3Rs principles-reduction, refinement, replacement-are at the core of preclinical research within drug discovery, which still relies to a great extent on the availability of models of disease in animals. Minimizing their distress, reducing their number as well as searching for means to replace them in experimental studies are constant objectives in this area. Due to its non-invasive character in vivo imaging supports these efforts by enabling repeated longitudinal assessments in each animal which serves as its own control, thereby enabling to reduce considerably the animal utilization in the experiments. The repetitive monitoring of pathology progression and the effects of therapy becomes feasible by assessment of quantitative biomarkers. Moreover, imaging has translational prospects by facilitating the comparison of studies performed in small rodents and humans. Also, learnings from the clinic may be potentially back-translated to preclinical settings and therefore contribute to refining animal investigations. By concentrating on activities around the application of magnetic resonance imaging (MRI) and ultrasound elastography to small rodent models of disease, we aim to illustrate how in vivo imaging contributes primarily to reduction and refinement in the context of pharmacological research.
Collapse
Affiliation(s)
- Michael Obrecht
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Stefan Zurbruegg
- Neurosciences Department, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Nathalie Accart
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Christian Lambert
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Arno Doelemeyer
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Birgit Ledermann
- 3Rs Leader, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Nicolau Beckmann
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| |
Collapse
|
7
|
Abu Haeyeh Y, Ghazal M, El-Baz A, Talaat IM. Development and Evaluation of a Novel Deep-Learning-Based Framework for the Classification of Renal Histopathology Images. Bioengineering (Basel) 2022; 9:423. [PMID: 36134972 PMCID: PMC9495730 DOI: 10.3390/bioengineering9090423] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 08/11/2022] [Accepted: 08/23/2022] [Indexed: 12/24/2022] Open
Abstract
Kidney cancer has several types, with renal cell carcinoma (RCC) being the most prevalent and severe type, accounting for more than 85% of adult patients. The manual analysis of whole slide images (WSI) of renal tissues is the primary tool for RCC diagnosis and prognosis. However, the manual identification of RCC is time-consuming and prone to inter-subject variability. In this paper, we aim to distinguish between benign tissue and malignant RCC tumors and identify the tumor subtypes to support medical therapy management. We propose a novel multiscale weakly-supervised deep learning approach for RCC subtyping. Our system starts by applying the RGB-histogram specification stain normalization on the whole slide images to eliminate the effect of the color variations on the system performance. Then, we follow the multiple instance learning approach by dividing the input data into multiple overlapping patches to maintain the tissue connectivity. Finally, we train three multiscale convolutional neural networks (CNNs) and apply decision fusion to their predicted results to obtain the final classification decision. Our dataset comprises four classes of renal tissues: non-RCC renal parenchyma, non-RCC fat tissues, clear cell RCC (ccRCC), and clear cell papillary RCC (ccpRCC). The developed system demonstrates a high classification accuracy and sensitivity on the RCC biopsy samples at the slide level. Following a leave-one-subject-out cross-validation approach, the developed RCC subtype classification system achieves an overall classification accuracy of 93.0% ± 4.9%, a sensitivity of 91.3% ± 10.7%, and a high classification specificity of 95.6% ± 5.2%, in distinguishing ccRCC from ccpRCC or non-RCC tissues. Furthermore, our method outperformed the state-of-the-art Resnet-50 model.
Collapse
Affiliation(s)
- Yasmine Abu Haeyeh
- College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates
| | - Mohammed Ghazal
- College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates
| | - Ayman El-Baz
- BioImaging Lab, Bioengineering Department, University of Louisville, Louisville, KY 40292, USA
| | - Iman M. Talaat
- Clinical Sciences Department, College of Medicine, University of Sharjah, Sharjah 27272, United Arab Emirates
| |
Collapse
|
8
|
Tang Q, Nie F, Zhao Q, Chen W. A merged molecular representation deep learning method for blood-brain barrier permeability prediction. Brief Bioinform 2022; 23:6674486. [PMID: 36002937 DOI: 10.1093/bib/bbac357] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2022] [Revised: 07/27/2022] [Accepted: 07/30/2022] [Indexed: 12/30/2022] Open
Abstract
The ability of a compound to permeate across the blood-brain barrier (BBB) is a significant factor for central nervous system drug development. Thus, for speeding up the drug discovery process, it is crucial to perform high-throughput screenings to predict the BBB permeability of the candidate compounds. Although experimental methods are capable of determining BBB permeability, they are still cost-ineffective and time-consuming. To complement the shortcomings of existing methods, we present a deep learning-based multi-model framework model, called Deep-B3, to predict the BBB permeability of candidate compounds. In Deep-B3, the samples are encoded in three kinds of features, namely molecular descriptors and fingerprints, molecular graph and simplified molecular input line entry system (SMILES) text notation. The pre-trained models were built to extract latent features from the molecular graph and SMILES. These features depicted the compounds in terms of tabular data, image and text, respectively. The validation results yielded from the independent dataset demonstrated that the performance of Deep-B3 is superior to that of the state-of-the-art models. Hence, Deep-B3 holds the potential to become a useful tool for drug development. A freely available online web-server for Deep-B3 was established at http://cbcb.cdutcm.edu.cn/deepb3/, and the source code and dataset of Deep-B3 are available at https://github.com/GreatChenLab/Deep-B3.
Collapse
Affiliation(s)
- Qiang Tang
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Basic Medical Science, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China
| | - Fulei Nie
- School of Public Health, North China University of Science and Technology, Tangshan 063210, China
| | - Qi Zhao
- School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan, 114051, China
| | - Wei Chen
- State Key Laboratory of Southwestern Chinese Medicine Resources, School of Basic Medical Science, Chengdu University of Traditional Chinese Medicine, Chengdu 611137, China.,School of Public Health, North China University of Science and Technology, Tangshan 063210, China
| |
Collapse
|
9
|
Gámez Serna C, Romero-Palomo F, Arcadu F, Funk J, Schumacher V, Janowczyk A. MMO-Net (Multi-Magnification Organ Network): A use case for Organ Identification using Multiple Magnifications in Preclinical Pathology Studies. J Pathol Inform 2022; 13:100126. [PMID: 36268069 PMCID: PMC9577048 DOI: 10.1016/j.jpi.2022.100126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 07/04/2022] [Accepted: 07/12/2022] [Indexed: 11/29/2022] Open
Abstract
Identifying organs within histology images is a fundamental and non-trivial step in toxicological digital pathology workflows as multiple organs often appear on the same whole slide image (WSI). Previous works in automated tissue classification have investigated the use of single magnifications, and demonstrated limitations when attempting to identify small and contiguous organs at low magnifications. In order to overcome these shortcomings, we present a multi-magnification convolutional neural network (CNN), called MMO-Net, which employs context and cellular detail from different magnifications to facilitate the recognition of complex organs. Across N=320 WSI from 3 contract research organization (CRO) laboratories, we demonstrate state-of-the-art organ detection and segmentation performance of 7 rat organs with and without lesions: liver, kidney, thyroid gland, parathyroid gland, urinary bladder, salivary gland, and mandibular lymph node (AUROC=0.99-1.0 for all organs, Dice≥0.9 except parathyroid (0.73)). Evaluation takes place at both inter- and intra CRO levels, suggesting strong generalizability performance. Results are qualitatively reviewed using visualization masks to ensure separation of organs in close proximity (e.g., thyroid vs parathyroid glands). MMO-Net thus offers organ localization that serves as a potential quality control tool to validate WSI metadata and as a preprocessing step for subsequent organ-specific artificial intelligence (AI) use cases. To facilitate research in this area, all associated WSI and metadata used for this study are being made freely available, forming a first of its kind dataset for public use.
Collapse
Affiliation(s)
- Citlalli Gámez Serna
- Roche Pharma Research and Early Development (pRED), Oncology, Roche Innovation Center Basel, Basel, Switzerland
| | - Fernando Romero-Palomo
- Roche Pharma Research and Early Development (pRED), Pharmaceutical Sciences, Roche Innovation Center Basel, Basel, Switzerland
| | - Filippo Arcadu
- Roche Pharma Research and Early Development Informatics (pREDi), Safety Development Informatics, Roche Innovation Center Basel, Basel, Switzerland
| | - Jürgen Funk
- Roche Pharma Research and Early Development (pRED), Pharmaceutical Sciences, Roche Innovation Center Basel, Basel, Switzerland
| | - Vanessa Schumacher
- Roche Pharma Research and Early Development (pRED), Pharmaceutical Sciences, Roche Innovation Center Basel, Basel, Switzerland
| | - Andrew Janowczyk
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
- Precision Oncology Center, University of Lausanne, Lausanne, Switzerland
| |
Collapse
|
10
|
Mehrvar S, Himmel LE, Babburi P, Goldberg AL, Guffroy M, Janardhan K, Krempley AL, Bawa B. Deep Learning Approaches and Applications in Toxicologic Histopathology: Current Status and Future Perspectives. J Pathol Inform 2021; 12:42. [PMID: 34881097 PMCID: PMC8609289 DOI: 10.4103/jpi.jpi_36_21] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 07/18/2021] [Indexed: 12/13/2022] Open
Abstract
Whole slide imaging enables the use of a wide array of digital image analysis tools that are revolutionizing pathology. Recent advances in digital pathology and deep convolutional neural networks have created an enormous opportunity to improve workflow efficiency, provide more quantitative, objective, and consistent assessments of pathology datasets, and develop decision support systems. Such innovations are already making their way into clinical practice. However, the progress of machine learning - in particular, deep learning (DL) - has been rather slower in nonclinical toxicology studies. Histopathology data from toxicology studies are critical during the drug development process that is required by regulatory bodies to assess drug-related toxicity in laboratory animals and its impact on human safety in clinical trials. Due to the high volume of slides routinely evaluated, low-throughput, or narrowly performing DL methods that may work well in small-scale diagnostic studies or for the identification of a single abnormality are tedious and impractical for toxicologic pathology. Furthermore, regulatory requirements around good laboratory practice are a major hurdle for the adoption of DL in toxicologic pathology. This paper reviews the major DL concepts, emerging applications, and examples of DL in toxicologic pathology image analysis. We end with a discussion of specific challenges and directions for future research.
Collapse
Affiliation(s)
- Shima Mehrvar
- Preclinical Safety, AbbVie Inc., North Chicago, IL, USA
| | | | - Pradeep Babburi
- Business Technology Solutions, AbbVie Inc., North Chicago, IL, USA
| | | | | | | | | | | |
Collapse
|
11
|
Lee HN, Seo HD, Kim EM, Han BS, Kang JS. Classification of Mouse Lung Metastatic Tumor with Deep Learning. Biomol Ther (Seoul) 2021; 30:179-183. [PMID: 34725310 PMCID: PMC8902456 DOI: 10.4062/biomolther.2021.130] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 08/31/2021] [Accepted: 09/13/2021] [Indexed: 11/26/2022] Open
Abstract
Traditionally, pathologists microscopically examine tissue sections to detect pathological lesions; the many slides that must be evaluated impose severe work burdens. Also, diagnostic accuracy varies by pathologist training and experience; better diagnostic tools are required. Given the rapid development of computer vision, automated deep learning is now used to classify microscopic images, including medical images. Here, we used a Inception-v3 deep learning model to detect mouse lung metastatic tumors via whole slide imaging (WSI); we cropped the images to 151 by 151 pixels. The images were divided into training (53.8%) and test (46.2%) sets (21,017 and 18,016 images, respectively). When images from lung tissue containing tumor tissues were evaluated, the model accuracy was 98.76%. When images from normal lung tissue were evaluated, the model accuracy (“no tumor”) was 99.87%. Thus, the deep learning model distinguished metastatic lesions from normal lung tissue. Our approach will allow the rapid and accurate analysis of various tissues.
Collapse
Affiliation(s)
- Ha Neul Lee
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| | - Hong-Deok Seo
- Department of Industrial Promotion, Spatial Information Industry Promotion Agency, Seongnam 13487, Republic of Korea
| | - Eui-Myoung Kim
- Department of Spatial Information Engineering, Namseoul University, Cheonan 31020, Republic of Korea
| | - Beom Seok Han
- Department of Pharmaceutical Engineering, Hoseo University, Asan 31499, Republic of Korea
| | - Jin Seok Kang
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| |
Collapse
|
12
|
Freyre CAC, Spiegel S, Gubser Keller C, Vandemeulebroecke M, Hoefling H, Dubost V, Cörek E, Moulin P, Hossain I. Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology-A Machine Learning Approach to Histopathology. Toxicol Pathol 2021; 49:798-814. [PMID: 33625320 DOI: 10.1177/0192623320987202] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Several deep learning approaches have been proposed to address the challenges in computational pathology by learning structural details in an unbiased way. Transfer learning allows starting from a learned representation of a pretrained model to be directly used or fine-tuned for a new domain. However, in histopathology, the problem domain is tissue-specific and putting together a labelled data set is challenging. On the other hand, whole slide-level annotations, such as biomarker levels, are much easier to obtain. We compare two pretrained models, one histology-specific and one from ImageNet on various computational pathology tasks. We show that a domain-specific model (HistoNet) contains richer information for biomarker classification, localization of biomarker-relevant morphology within a slide, and the prediction of expert-graded features. We use a weakly supervised approach to discriminate slides based on biomarker level and simultaneously predict which regions contribute to that prediction. We employ multitask learning to show that learned representations correlate with morphological features graded by expert pathologists. All of these results are demonstrated in the context of renal toxicity in a mechanistic study of compound toxicity in rat models. Our results emphasize the importance of histology-specific models and their knowledge representations for solving a wide range of computational pathology tasks.
Collapse
Affiliation(s)
| | | | | | | | - Holger Hoefling
- 33413Novartis Institutes for Biomedical Research (NIBR), Basel, Switzerland
| | - Valerie Dubost
- 33413Novartis Institutes for Biomedical Research (NIBR), Basel, Switzerland
| | - Emre Cörek
- 130383University of Basel, Basel, Switzerland
| | - Pierre Moulin
- 33413Novartis Institutes for Biomedical Research (NIBR), Basel, Switzerland
| | - Imtiaz Hossain
- 33413Novartis Institutes for Biomedical Research (NIBR), Basel, Switzerland
| |
Collapse
|