1
|
Shiffman S, Rios Piedra EA, Adedeji AO, Ruff CF, Andrews RN, Katavolos P, Liu E, Forster A, Brumm J, Fuji RN, Sullivan R. Analysis of cellularity in H&E-stained rat bone marrow tissue via deep learning. J Pathol Inform 2023; 14:100333. [PMID: 37743975 PMCID: PMC10514468 DOI: 10.1016/j.jpi.2023.100333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/18/2023] [Accepted: 08/19/2023] [Indexed: 09/26/2023] Open
Abstract
Our objective was to develop an automated deep-learning-based method to evaluate cellularity in rat bone marrow hematoxylin and eosin whole slide images for preclinical safety assessment. We trained a shallow CNN for segmenting marrow, 2 Mask R-CNN models for segmenting megakaryocytes (MKCs), and small hematopoietic cells (SHCs), and a SegNet model for segmenting red blood cells. We incorporated the models into a pipeline that identifies and counts MKCs and SHCs in rat bone marrow. We compared cell segmentation and counts that our method generated to those that pathologists generated on 10 slides with a range of cell depletion levels from 10 studies. For SHCs, we compared cell counts that our method generated to counts generated by Cellpose and Stardist. The median Dice and object Dice scores for MKCs using our method vs pathologist consensus and the inter- and intra-pathologist variation were comparable, with overlapping first-third quartile ranges. For SHCs, the median scores were close, with first-third quartile ranges partially overlapping intra-pathologist variation. For SHCs, in comparison to Cellpose and Stardist, counts from our method were closer to pathologist counts, with a smaller 95% limits of agreement range. The performance of the bone marrow analysis pipeline supports its incorporation into routine use as an aid for hematotoxicity assessment by pathologists. The pipeline could help expedite hematotoxicity assessment in preclinical studies and consequently could expedite drug development. The method may enable meta-analysis of rat bone marrow characteristics from future and historical whole slide images and may generate new biological insights from cross-study comparisons.
Collapse
Affiliation(s)
- Smadar Shiffman
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Edgar A. Rios Piedra
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Adeyemi O. Adedeji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Catherine F. Ruff
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Rachel N. Andrews
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Paula Katavolos
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- Bristol Myers Squibb, New Brunswick, NJ 08901, USA
| | - Evan Liu
- Genentech Research and Early Development (gRED), Department of Development Sciences Informatics, Genentech Inc, South San Francisco, USA
| | - Ashley Forster
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- University of Pennsylvania School of Veterinary Medicine, Philadelphia, PA 19104, USA
| | - Jochen Brumm
- Genentech Research and Early Development (gRED), Department of Nonclinical Biostatistics, Genentech Inc, South San Francisco, USA
| | - Reina N. Fuji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Ruth Sullivan
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| |
Collapse
|
2
|
Kurnia KA, Lin YT, Farhan A, Malhotra N, Luong CT, Hung CH, Roldan MJM, Tsao CC, Cheng TS, Hsiao CD. Deep Learning-Based Automatic Duckweed Counting Using StarDist and Its Application on Measuring Growth Inhibition Potential of Rare Earth Elements as Contaminants of Emerging Concerns. TOXICS 2023; 11:680. [PMID: 37624185 PMCID: PMC10457735 DOI: 10.3390/toxics11080680] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Revised: 08/03/2023] [Accepted: 08/06/2023] [Indexed: 08/26/2023]
Abstract
In recent years, there have been efforts to utilize surface water as a power source, material, and food. However, these efforts are impeded due to the vast amounts of contaminants and emerging contaminants introduced by anthropogenic activities. Herbicides such as Glyphosate and Glufosinate are commonly known to contaminate surface water through agricultural industries. In contrast, some emerging contaminants, such as rare earth elements, have started to enter the surface water from the production and waste of electronic products. Duckweeds are angiosperms from the Lemnaceae family and have been used for toxicity tests in aquatic environments, mainly those from the genus Lemna, and have been approved by OECD. In this study, we used duckweed from the genus Wolffia, which is smaller and considered a good indicator of metal pollutants in the aquatic environment. The growth rate of duckweed is the most common endpoint in observing pollutant toxicity. In order to observe and mark the fronds automatically, we used StarDist, a machine learning-based tool. StarDist is available as a plugin in ImageJ, simplifying and assisting the counting process. Python also helps arrange, manage, and calculate the inhibition percentage after duckweeds are exposed to contaminants. The toxicity test results showed Dysprosium to be the most toxic, with an IC50 value of 14.6 ppm, and Samarium as the least toxic, with an IC50 value of 279.4 ppm. In summary, we can provide a workflow for automatic frond counting using StarDist integrated with ImageJ and Python to simplify the detection, counting, data management, and calculation process.
Collapse
Affiliation(s)
- Kevin Adi Kurnia
- Department of Chemistry, Chung Yuan Christian University, Chung-Li 32023, Taiwan; (K.A.K.); (A.F.)
- Department of Bioscience Technology, Chung Yuan Christian University, Chung-Li 32023, Taiwan;
| | - Ying-Ting Lin
- Department of Biotechnology, College of Life Science, Kaohsiung Medical University, Kaohsiung City 80708, Taiwan;
- Drug Development & Value Creation Research Center, Kaohsiung Medical University, Kaohsiung City 80708, Taiwan
| | - Ali Farhan
- Department of Chemistry, Chung Yuan Christian University, Chung-Li 32023, Taiwan; (K.A.K.); (A.F.)
- Department of Bioscience Technology, Chung Yuan Christian University, Chung-Li 32023, Taiwan;
| | - Nemi Malhotra
- Department of Bioscience Technology, Chung Yuan Christian University, Chung-Li 32023, Taiwan;
| | - Cao Thang Luong
- Department of Chemical Engineering & Institute of Biotechnology and Chemical Engineering, I-Shou University, Da-Shu, Kaohsiung City 84001, Taiwan; (C.T.L.); (C.-H.H.)
| | - Chih-Hsin Hung
- Department of Chemical Engineering & Institute of Biotechnology and Chemical Engineering, I-Shou University, Da-Shu, Kaohsiung City 84001, Taiwan; (C.T.L.); (C.-H.H.)
| | - Marri Jmelou M. Roldan
- Faculty of Pharmacy, The Graduate School, University of Santo Tomas, Manila 1008, Philippines;
| | - Che-Chia Tsao
- Department of Biological Sciences and Technology, National University of Tainan, Tainan 70005, Taiwan;
| | - Tai-Sheng Cheng
- Department of Biological Sciences and Technology, National University of Tainan, Tainan 70005, Taiwan;
| | - Chung-Der Hsiao
- Department of Chemistry, Chung Yuan Christian University, Chung-Li 32023, Taiwan; (K.A.K.); (A.F.)
- Department of Bioscience Technology, Chung Yuan Christian University, Chung-Li 32023, Taiwan;
- Center for Nanotechnology, Chung Yuan Christian University, Chung-Li 32023, Taiwan
- Research Center for Aquatic Toxicology and Pharmacology, Chung Yuan Christian University, Chung-Li 32023, Taiwan
| |
Collapse
|
3
|
Multimodal sentiment system and method based on CRNN-SVM. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08366-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2023]
Abstract
AbstractTraditional sentiment analysis focuses on text-level sentiment mining, transforming sentiment mining into classification or regression problems, resulting in a sentiment analysis low accuracy rate. Sentiment analysis refers to the use of natural language processing, text analysis, and computational linguistics to systematically identify, extract, quantify, and study sentimental states. Therefore, more scholars have begun to focus on speech recognition and facial expression recognition research, and extracting and analysing people’s sentiment tendencies can improve sentiment recognition accuracy. Traditional single-modal sentiment analysis can no longer meet people’s needs. Therefore, this paper proposes a multimodal sentiment analysis method based on the multimodal sentiment analysis method that can obtain more sentimental information sources and help people make better decisions. The experimental results in this paper show that the highest recognition rates of CNN-SVM, RNN-SVM, and CRNN-SVM were 76.8%, 71.2%, and 93.5%, respectively. It can be seen that CRNN-SVM has the highest sentiment tendency recognition rate in deep learning, so it is suitable to apply CRNN-SVM to sentiment tendency analysis system design in this paper. The average accuracy rate of the system designed in this paper was 91%, and the stability was also very strong, which shows that the system designed in this paper is meaningful. The main contribution of this paper is based on the limitations of single-mode emotion analysis. It proposes a multimode emotion analysis method and introduces a convolutional neural network to help people obtain more emotional information sources to meet their needs.
Collapse
|
4
|
Poitout-Belissent F, Vitsky A, Smith MA, Sirivelu MP. Methodologies and Emerging Technologies for the Evaluation of the Hematopoietic System. Toxicol Pathol 2022; 50:867-870. [DOI: 10.1177/01926233221128755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Hematology and bone marrow analysis is central to our understanding of the hematopoietic system and how it responds to insults, and this session presented during the 2022 STP symposium provided a review of current and novel approaches for the evaluation of the hematopoietic system in the context of nonclinical investigations. This publication summarizes the information presented on novel approaches for evaluation of the hematopoietic system using automated hematology analyzers, including details around the quantitative assessment of bone marrow cell suspensions as well as introducing several newly available hematology parameters. It was followed by a discussion on intravital microscopy and live cell imaging and how these methods can assist with de-risking hematopoiesis-associated safety concerns, and a review of recent assays using artificial intelligence for the evaluation of bone marrow.
Collapse
|
5
|
Mehrvar S, Himmel LE, Babburi P, Goldberg AL, Guffroy M, Janardhan K, Krempley AL, Bawa B. Deep Learning Approaches and Applications in Toxicologic Histopathology: Current Status and Future Perspectives. J Pathol Inform 2021; 12:42. [PMID: 34881097 PMCID: PMC8609289 DOI: 10.4103/jpi.jpi_36_21] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 07/18/2021] [Indexed: 12/13/2022] Open
Abstract
Whole slide imaging enables the use of a wide array of digital image analysis tools that are revolutionizing pathology. Recent advances in digital pathology and deep convolutional neural networks have created an enormous opportunity to improve workflow efficiency, provide more quantitative, objective, and consistent assessments of pathology datasets, and develop decision support systems. Such innovations are already making their way into clinical practice. However, the progress of machine learning - in particular, deep learning (DL) - has been rather slower in nonclinical toxicology studies. Histopathology data from toxicology studies are critical during the drug development process that is required by regulatory bodies to assess drug-related toxicity in laboratory animals and its impact on human safety in clinical trials. Due to the high volume of slides routinely evaluated, low-throughput, or narrowly performing DL methods that may work well in small-scale diagnostic studies or for the identification of a single abnormality are tedious and impractical for toxicologic pathology. Furthermore, regulatory requirements around good laboratory practice are a major hurdle for the adoption of DL in toxicologic pathology. This paper reviews the major DL concepts, emerging applications, and examples of DL in toxicologic pathology image analysis. We end with a discussion of specific challenges and directions for future research.
Collapse
Affiliation(s)
- Shima Mehrvar
- Preclinical Safety, AbbVie Inc., North Chicago, IL, USA
| | | | - Pradeep Babburi
- Business Technology Solutions, AbbVie Inc., North Chicago, IL, USA
| | | | | | | | | | | |
Collapse
|
6
|
Hondelink LM, Hüyük M, Postmus PE, Smit VTHBM, Blom S, von der Thüsen JH, Cohen D. Development and validation of a supervised deep learning algorithm for automated whole-slide programmed death-ligand 1 tumour proportion score assessment in non-small cell lung cancer. Histopathology 2021; 80:635-647. [PMID: 34786761 PMCID: PMC9299490 DOI: 10.1111/his.14571] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 09/08/2021] [Accepted: 09/21/2021] [Indexed: 12/24/2022]
Abstract
AIMS Immunohistochemical programmed death-ligand 1 (PD-L1) staining to predict responsiveness to immunotherapy in patients with advanced non-small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, with up to 20% discordance around cutoff points. The aim of this study was to develop a new deep learning-based PD-L1 tumour proportion score (TPS) algorithm, trained and validated on a routine diagnostic dataset of digitised PD-L1 (22C3, laboratory-developed test)-stained samples. METHODS AND RESULTS We designed a fully supervised deep learning algorithm for whole-slide PD-L1 assessment, consisting of four sequential convolutional neural networks (CNNs), using aiforia create software. We included 199 whole slide images (WSIs) of 'routine diagnostic' histology samples from stage IV NSCLC patients, and trained the algorithm by using a training set of 60 representative cases. We validated the algorithm by comparing the algorithm TPS with the reference score in a held-out validation set. The algorithm had similar concordance with the reference score (79%) as the pathologists had with one another (75%). The intraclass coefficient was 0.96 and Cohen's κ coefficient was 0.69 for the algorithm. Around the 1% and 50% cutoff points, concordance was also similar between pathologists and the algorithm. CONCLUSIONS We designed a new, deep learning-based PD-L1 TPS algorithm that is similarly able to assess PD-L1 expression in daily routine diagnostic cases as pathologists. Successful validation on routine diagnostic WSIs and detailed visual feedback show that this algorithm meets the requirements for functioning as a 'scoring assistant'.
Collapse
Affiliation(s)
- Liesbeth M Hondelink
- Department of Pathology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Melek Hüyük
- Department of Pulmonology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Pieter E Postmus
- Department of Pulmonology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Vincent T H B M Smit
- Department of Pathology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Sami Blom
- Aiforia Technologies Oy, Helsinki, Finland
| | | | - Danielle Cohen
- Department of Pathology, Leiden University Medical Centre, Leiden, The Netherlands
| |
Collapse
|
7
|
Lee HN, Seo HD, Kim EM, Han BS, Kang JS. Classification of Mouse Lung Metastatic Tumor with Deep Learning. Biomol Ther (Seoul) 2021; 30:179-183. [PMID: 34725310 PMCID: PMC8902456 DOI: 10.4062/biomolther.2021.130] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 08/31/2021] [Accepted: 09/13/2021] [Indexed: 11/26/2022] Open
Abstract
Traditionally, pathologists microscopically examine tissue sections to detect pathological lesions; the many slides that must be evaluated impose severe work burdens. Also, diagnostic accuracy varies by pathologist training and experience; better diagnostic tools are required. Given the rapid development of computer vision, automated deep learning is now used to classify microscopic images, including medical images. Here, we used a Inception-v3 deep learning model to detect mouse lung metastatic tumors via whole slide imaging (WSI); we cropped the images to 151 by 151 pixels. The images were divided into training (53.8%) and test (46.2%) sets (21,017 and 18,016 images, respectively). When images from lung tissue containing tumor tissues were evaluated, the model accuracy was 98.76%. When images from normal lung tissue were evaluated, the model accuracy (“no tumor”) was 99.87%. Thus, the deep learning model distinguished metastatic lesions from normal lung tissue. Our approach will allow the rapid and accurate analysis of various tissues.
Collapse
Affiliation(s)
- Ha Neul Lee
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| | - Hong-Deok Seo
- Department of Industrial Promotion, Spatial Information Industry Promotion Agency, Seongnam 13487, Republic of Korea
| | - Eui-Myoung Kim
- Department of Spatial Information Engineering, Namseoul University, Cheonan 31020, Republic of Korea
| | - Beom Seok Han
- Department of Pharmaceutical Engineering, Hoseo University, Asan 31499, Republic of Korea
| | - Jin Seok Kang
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| |
Collapse
|
8
|
Zuraw A, Aeffner F. Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review. Vet Pathol 2021; 59:6-25. [PMID: 34521285 DOI: 10.1177/03009858211040484] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Since whole-slide imaging has been commercially available for over 2 decades, digital pathology has become a constantly expanding aspect of the pathology profession that will continue to significantly impact how pathologists conduct their craft. While some aspects, such as whole-slide imaging for archiving, consulting, and teaching, have gained broader acceptance, other facets such as quantitative tissue image analysis and artificial intelligence-based assessments are still met with some reservations. While most vendors in this space have focused on diagnostic applications, that is, viewing one or few slides at a time, some are developing solutions tailored more specifically to the various aspects of veterinary pathology including updated diagnostic, discovery, and research applications. This has especially advanced the use of digital pathology in toxicologic pathology and drug development, for primary reads as well as peer reviews. It is crucial that pathologists gain a deeper understanding of digital pathology and tissue image analysis technology and their applications in order to fully use these tools in a way that enhances and improves the pathologist's assessment as well as work environment. This review focuses on an updated introduction to the basics of digital pathology and image analysis and introduces emerging topics around artificial intelligence and machine learning.
Collapse
Affiliation(s)
| | - Famke Aeffner
- Amgen Inc, Amgen Research, South San Francisco, CA, USA
| |
Collapse
|
9
|
Turner OC, Knight B, Zuraw A, Litjens G, Rudmann DG. Mini Review: The Last Mile-Opportunities and Challenges for Machine Learning in Digital Toxicologic Pathology. Toxicol Pathol 2021; 49:714-719. [PMID: 33590805 DOI: 10.1177/0192623321990375] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
The 2019 manuscript by the Special Interest Group on Digital Pathology and Image Analysis of the Society of Toxicologic pathology suggested that a synergism between artificial intelligence (AI) and machine learning (ML) technologies and digital toxicologic pathology would improve the daily workflow and future impact of toxicologic pathologists globally. Now 2 years later, the authors of this review consider whether, in their opinion, there is any evidence that supports that thesis. Specifically, we consider the opportunities and challenges for applying ML (the study of computer algorithms that are able to learn from example data and extrapolate the learned information to unseen data) algorithms in toxicologic pathology and how regulatory bodies are navigating this rapidly evolving field. Although we see similarities with the "Last Mile" metaphor, the weight of evidence suggests that toxicologic pathologists should approach ML with an equal dose of skepticism and enthusiasm. There are increasing opportunities for impact in our field that leave the authors cautiously excited and optimistic. Toxicologic pathologists have the opportunity to critically evaluate ML applications with a "call-to-arms" mentality. Why should we be late adopters? There is ample evidence to encourage engagement, growth, and leadership in this field.
Collapse
Affiliation(s)
- Oliver C Turner
- Novartis, 98557Novartis Institutes for BioMedical Research, Preclinical Safety, East Hanover, NJ, USA
| | - Brian Knight
- 435339Boehringer Ingelheim Pharmaceuticals Incorporated, Nonclinical Drug Safety, Ridgefield, CT, USA
| | | | - Geert Litjens
- Diagnostic Image Analysis Group Radboud University Medical Center Nijmegen, the Netherlands
| | | |
Collapse
|