1
|
Saxena P, Aggarwal SK, Sinha A, Saxena S, Singh AK. Review of computer-assisted diagnosis model to classify follicular lymphoma histology. Cell Biochem Funct 2024; 42:e4088. [PMID: 38973163 DOI: 10.1002/cbf.4088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Revised: 04/15/2024] [Accepted: 06/28/2024] [Indexed: 07/09/2024]
Abstract
The field of image processing is experiencing significant advancements to support professionals in analyzing histological images obtained from biopsies. The primary objective is to enhance the process of diagnosis and prognostic evaluations. Various forms of cancer can be diagnosed by employing different segmentation techniques followed by postprocessing approaches that can identify distinct neoplastic areas. Using computer approaches facilitates a more objective and efficient study of experts. The progressive advancement of histological image analysis holds significant importance in modern medicine. This paper provides an overview of the current advances in segmentation and classification approaches for images of follicular lymphoma. This research analyzes the primary image processing techniques utilized in the various stages of preprocessing, segmentation of the region of interest, classification, and postprocessing as described in the existing literature. The study also examines the strengths and weaknesses associated with these approaches. Additionally, this study encompasses an examination of validation procedures and an exploration of prospective future research roads in the segmentation of neoplasias.
Collapse
Affiliation(s)
- Pranshu Saxena
- School of Computer Science Engineering & Technology, Bennett University, Greater Noida, Uttar Pradesh, India
| | - Sahil Kumar Aggarwal
- Department of Information Technology, ABES Engineering College, Ghaziabad, India
| | - Amit Sinha
- Department of Information Technology, ABES Engineering College, Ghaziabad, India
| | - Sandeep Saxena
- Greater Noida Institute of Technology, Greater Noida, India
| | | |
Collapse
|
2
|
Naji H, Sancere L, Simon A, Büttner R, Eich ML, Lohneis P, Bożek K. HoLy-Net: Segmentation of histological images of diffuse large B-cell lymphoma. Comput Biol Med 2024; 170:107978. [PMID: 38237235 DOI: 10.1016/j.compbiomed.2024.107978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 10/30/2023] [Accepted: 01/08/2024] [Indexed: 02/28/2024]
Abstract
Over the last years, there has been large progress in automated segmentation and classification methods in histological whole slide images (WSIs) stained with hematoxylin and eosin (H&E). Current state-of-the-art (SOTA) techniques are based on diverse datasets of H&E-stained WSIs of different types of predominantly solid cancer. However, there is a scarcity of methods and datasets enabling segmentation of tumors of the lymphatic system (lymphomas). Here, we propose a solution for segmentation of diffuse large B-cell lymphoma (DLBCL), the most common non-Hodgkin's lymphoma. Our method applies to both H&E-stained slides and to a broad range of markers stained with immunohistochemistry (IHC). While IHC staining is an important tool in cancer diagnosis and treatment decisions, there are few automated segmentation and classification methods for IHC-stained WSIs. To address the challenges of nuclei segmentation in H&E- and IHC-stained DLBCL images, we propose HoLy-Net - a HoVer-Net-based deep learning model for lymphoma segmentation. We train two different models, one for segmenting H&E- and one for IHC-stained images and compare the test results with the SOTA methods as well as with the original version of HoVer-Net. Subsequently, we segment patient WSIs and perform single cell-level analysis of different cell types to identify patient-specific tumor characteristics such as high level of immune infiltration. Our method outperforms general-purpose segmentation methods for H&E staining in lymphoma WSIs (with an F1 score of 0.899) and is also a unique automated method for IHC slide segmentation (with an F1 score of 0.913). With our solution, we provide a new dataset we denote LyNSeC (lymphoma nuclear segmentation and classification) containing 73,931 annotated cell nuclei from H&E and 87,316 from IHC slides. Our method and dataset open up new avenues for quantitative, large-scale studies of morphology and microenvironment of lymphomas overlooked by the current automated segmentation methods.
Collapse
Affiliation(s)
- Hussein Naji
- Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany; Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany.
| | - Lucas Sancere
- Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany; Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany
| | - Adrian Simon
- Institute of Pathology, University Hospital Cologne, Germany
| | | | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Germany
| | - Philipp Lohneis
- Institute of Pathology, University Hospital Cologne, Germany; Hämatopathologie Lübeck, Germany
| | - Katarzyna Bożek
- Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany; Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Germany
| |
Collapse
|
3
|
Lindemann MC, Glänzer L, Roeth AA, Schmitz-Rode T, Slabu I. Towards Realistic 3D Models of Tumor Vascular Networks. Cancers (Basel) 2023; 15:5352. [PMID: 38001612 PMCID: PMC10670125 DOI: 10.3390/cancers15225352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 11/03/2023] [Accepted: 11/03/2023] [Indexed: 11/26/2023] Open
Abstract
For reliable in silico or in vitro investigations in, for example, biosensing and drug delivery applications, accurate models of tumor vascular networks down to the capillary size are essential. Compared to images acquired with conventional medical imaging techniques, digitalized histological tumor slices have a higher resolution, enabling the delineation of capillaries. Volume rendering procedures can then be used to generate a 3D model. However, the preparation of such slices leads to misalignments in relative slice orientation between consecutive slices. Thus, image registration algorithms are necessary to re-align the slices. Here, we present an algorithm for the registration and reconstruction of a vascular network from histologic slices applied to 169 tumor slices. The registration includes two steps. First, consecutive images are incrementally pre-aligned using feature- and area-based transformations. Second, using the previous transformations, parallel registration for all images is enabled. Combining intensity- and color-based thresholds along with heuristic analysis, vascular structures are segmented. A 3D interpolation technique is used for volume rendering. This results in a 3D vascular network with approximately 400-450 vessels with diameters down to 25-30 µm. A delineation of vessel structures with close distance was limited in areas of high structural density. Improvement can be achieved by using images with higher resolution and or machine learning techniques.
Collapse
Affiliation(s)
- Max C. Lindemann
- Institute of Applied Medical Engineering, Helmholtz Institute, Medical Faculty, RWTH Aachen University, Pauwelsstraße 20, 52074 Aachen, Germany (L.G.); (T.S.-R.)
| | - Lukas Glänzer
- Institute of Applied Medical Engineering, Helmholtz Institute, Medical Faculty, RWTH Aachen University, Pauwelsstraße 20, 52074 Aachen, Germany (L.G.); (T.S.-R.)
| | - Anjali A. Roeth
- Department of General, Visceral and Transplant Surgery, RWTH Aachen University Hospital, Pauwelsstrasse 30, 52074 Aachen, Germany
- Department of Surgery, Maastricht University, P. Debyelaan 25, 6229 HX Maastricht, The Netherlands
| | - Thomas Schmitz-Rode
- Institute of Applied Medical Engineering, Helmholtz Institute, Medical Faculty, RWTH Aachen University, Pauwelsstraße 20, 52074 Aachen, Germany (L.G.); (T.S.-R.)
| | - Ioana Slabu
- Institute of Applied Medical Engineering, Helmholtz Institute, Medical Faculty, RWTH Aachen University, Pauwelsstraße 20, 52074 Aachen, Germany (L.G.); (T.S.-R.)
| |
Collapse
|
4
|
Osteosarcoma Detection from Whole Slide Images Using Multi-Feature Non-Seed-Based Region Growing Segmentation and Feature Extraction. Neural Process Lett 2022. [DOI: 10.1007/s11063-022-10914-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
5
|
Daniel N, Larey A, Aknin E, Osswald GA, Caldwell JM, Rochman M, Collins MH, Yang GY, Arva NC, Capocelli KE, Rothenberg ME, Savir Y. A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3211-3217. [PMID: 36085661 PMCID: PMC9552249 DOI: 10.1109/embc48229.2022.9871086] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.
Collapse
Affiliation(s)
- Nati Daniel
- Faculty of Medicine,Dept. of Physiology, Biophysics and System Biology,Technion,Israel
| | | | - Eliel Aknin
- Faculty of Industrial Engineering,Technion,Israel
| | - Garrett A. Osswald
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Julie M. Caldwell
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Mark Rochman
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Margaret H. Collins
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Dept. of Pathology,Cincinnati,OH,USA
| | - Guang-Yu Yang
- Ann & Robert H. Lurie Children's Hospital of Chicago, Northwestern University, The Feinberg School of Medicine,Dept. of Pathology,Chicago,IL,USA
| | - Nicoleta C. Arva
- Ann & Robert H. Lurie Children's Hospital of Chicago, Northwestern University, The Feinberg School of Medicine,Dept. of Pathology,Chicago,IL,USA
| | | | - Marc E. Rothenberg
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine,Division of Allergy and Immunology,Dept. of Pediatrics,OH,USA
| | - Yonatan Savir
- Faculty of Medicine,Dept. of Physiology, Biophysics and System Biology,Technion,Israel
| |
Collapse
|
6
|
Teranikar T, Lim J, Ijaseun T, Lee J. Development of Planar Illumination Strategies for Solving Mysteries in the Sub-Cellular Realm. Int J Mol Sci 2022; 23:1643. [PMID: 35163562 PMCID: PMC8835835 DOI: 10.3390/ijms23031643] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Revised: 12/22/2021] [Accepted: 01/25/2022] [Indexed: 02/04/2023] Open
Abstract
Optical microscopy has vastly expanded the frontiers of structural and functional biology, due to the non-invasive probing of dynamic volumes in vivo. However, traditional widefield microscopy illuminating the entire field of view (FOV) is adversely affected by out-of-focus light scatter. Consequently, standard upright or inverted microscopes are inept in sampling diffraction-limited volumes smaller than the optical system's point spread function (PSF). Over the last few decades, several planar and structured (sinusoidal) illumination modalities have offered unprecedented access to sub-cellular organelles and 4D (3D + time) image acquisition. Furthermore, these optical sectioning systems remain unaffected by the size of biological samples, providing high signal-to-noise (SNR) ratios for objective lenses (OLs) with long working distances (WDs). This review aims to guide biologists regarding planar illumination strategies, capable of harnessing sub-micron spatial resolution with a millimeter depth of penetration.
Collapse
Affiliation(s)
| | | | | | - Juhyun Lee
- Department of Bioengineering, University of Texas at Arlington, Arlington, TX 75022, USA; (T.T.); (J.L.); (T.I.)
| |
Collapse
|
7
|
Ray S, Das A, Dhal KG, Gálvez J, Naskar PK. Whale Optimizer-Based Clustering for Breast Histopathology Image Segmentation. INTERNATIONAL JOURNAL OF SWARM INTELLIGENCE RESEARCH 2022. [DOI: 10.4018/ijsir.302611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Breast histopathology image segmentation is a complex task due to indiscernibly correlated and noisy regions of interest. Breast histopathological images are composed of different types of cells. Some of these cells can be harmful for humans due to the presence of cancer. Under such circumstances, many segmentation techniques for automatic detection of cancer cells have been proposed considering clustering schemes. However, such clustering methodologies are sensitive to initial cluster centers, which promote false-positive solutions. This paper presents the use of the Whale Optimization Algorithm (WOA) for proper clustering segmentation of breast histopathological images to overcome clustering issues. Also, a rigorous comparative study is conducted among the proposed approach and several state-of-art Nature-Inspired Optimization Algorithms (NIOAs) and traditional clustering techniques. The numerical results indicate that the proposed approach outperforms the other utilized clustering methods in terms of precision, robustness, and quality of the segmented outputs.
Collapse
Affiliation(s)
- Swarnajit Ray
- Maulana Abul Kalam Azad University of Technology, India
| | | | | | | | | |
Collapse
|
8
|
Larey A, Aknin E, Daniel N, Osswald GA, Caldwell JM, Rochman M, Wasserman T, Collins MH, Arva NC, Yang GY, Rothenberg ME, Savir Y. Harnessing artificial intelligence to infer novel spatial biomarkers for the diagnosis of eosinophilic esophagitis. Front Med (Lausanne) 2022; 9:950728. [PMID: 36341260 PMCID: PMC9633847 DOI: 10.3389/fmed.2022.950728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Accepted: 09/29/2022] [Indexed: 11/13/2022] Open
Abstract
Eosinophilic esophagitis (EoE) is a chronic allergic inflammatory condition of the esophagus associated with elevated esophageal eosinophils. Second only to gastroesophageal reflux disease, EoE is one of the leading causes of chronic refractory dysphagia in adults and children. EoE is a clinicopathologic disorder and the histological portion of the diagnosis requires enumerating the density of esophageal eosinophils in esophageal biopsies, and evaluating additional features such as basal zone hyperplasia is helpful. However, this task requires time-consuming, somewhat subjective manual analysis, thus reducing the ability to process the complex tissue structure and infer its relationship with the patient's clinical status. Previous artificial intelligence (AI) approaches that aimed to improve histology-based diagnosis focused on recapitulating identification and quantification of the area of maximal eosinophil density, the gold standard manual metric for determining EoE disease activity. However, this metric does not account for the distribution of eosinophils or other histological features, over the whole slide image. Here, we developed an artificial intelligence platform that infers local and spatial biomarkers based on semantic segmentation of intact eosinophils and basal zone distributions. Besides the maximal density of eosinophils [referred to as Peak Eosinophil Count (PEC)] and a maximal basal zone fraction, we identify the value of two additional metrics that reflect the distribution of eosinophils and basal zone fractions. This approach enables a decision support system that predicts EoE activity and potentially classifies the histological severity of EoE patients. We utilized a cohort that includes 1,066 biopsy slides from 400 subjects to validate the system's performance and achieved a histological severity classification accuracy of 86.70%, sensitivity of 84.50%, and specificity of 90.09%. Our approach highlights the importance of systematically analyzing the distribution of biopsy features over the entire slide and paves the way toward a personalized decision support system that will assist not only in counting cells but can also potentially improve diagnosis and provide treatment prediction.
Collapse
Affiliation(s)
- Ariel Larey
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel.,Faculty of Computer Science, Technion Israel Institute of Technology, Haifa, Israel
| | - Eliel Aknin
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel.,Faculty of Industrial Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Nati Daniel
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| | - Garrett A Osswald
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Julie M Caldwell
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Mark Rochman
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Tanya Wasserman
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| | - Margaret H Collins
- Division of Pathology, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Nicoleta C Arva
- Department of Pathology and Laboratory Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States
| | - Guang-Yu Yang
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Marc E Rothenberg
- Division of Allergy and Immunology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Yonatan Savir
- Department of Physiology, Biophysics and Systems Biology, Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| |
Collapse
|
9
|
Bhattacharjee S, Ikromjanov K, Carole KS, Madusanka N, Cho NH, Hwang YB, Sumon RI, Kim HC, Choi HK. Cluster Analysis of Cell Nuclei in H&E-Stained Histological Sections of Prostate Cancer and Classification Based on Traditional and Modern Artificial Intelligence Techniques. Diagnostics (Basel) 2021; 12:diagnostics12010015. [PMID: 35054182 PMCID: PMC8774423 DOI: 10.3390/diagnostics12010015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 12/14/2021] [Accepted: 12/20/2021] [Indexed: 11/16/2022] Open
Abstract
Biomarker identification is very important to differentiate the grade groups in the histopathological sections of prostate cancer (PCa). Assessing the cluster of cell nuclei is essential for pathological investigation. In this study, we present a computer-based method for cluster analyses of cell nuclei and performed traditional (i.e., unsupervised method) and modern (i.e., supervised method) artificial intelligence (AI) techniques for distinguishing the grade groups of PCa. Two datasets on PCa were collected to carry out this research. Histopathology samples were obtained from whole slides stained with hematoxylin and eosin (H&E). In this research, state-of-the-art approaches were proposed for color normalization, cell nuclei segmentation, feature selection, and classification. A traditional minimum spanning tree (MST) algorithm was employed to identify the clusters and better capture the proliferation and community structure of cell nuclei. K-medoids clustering and stacked ensemble machine learning (ML) approaches were used to perform traditional and modern AI-based classification. The binary and multiclass classification was derived to compare the model quality and results between the grades of PCa. Furthermore, a comparative analysis was carried out between traditional and modern AI techniques using different performance metrics (i.e., statistical parameters). Cluster features of the cell nuclei can be useful information for cancer grading. However, further validation of cluster analysis is required to accomplish astounding classification results.
Collapse
Affiliation(s)
| | - Kobiljon Ikromjanov
- Department of Digital Anti-Aging Healthcare, u-AHRC, Inje University, Gimhae 50834, Korea; (K.I.); (K.S.C.); (Y.-B.H.); (R.I.S.); (H.-C.K.)
| | - Kouayep Sonia Carole
- Department of Digital Anti-Aging Healthcare, u-AHRC, Inje University, Gimhae 50834, Korea; (K.I.); (K.S.C.); (Y.-B.H.); (R.I.S.); (H.-C.K.)
| | - Nuwan Madusanka
- School of Computing & IT, Sri Lanka Technological Campus, Paduka 10500, Sri Lanka;
| | - Nam-Hoon Cho
- Department of Pathology, Yonsei University Hospital, Seoul 03722, Korea;
| | - Yeong-Byn Hwang
- Department of Digital Anti-Aging Healthcare, u-AHRC, Inje University, Gimhae 50834, Korea; (K.I.); (K.S.C.); (Y.-B.H.); (R.I.S.); (H.-C.K.)
| | - Rashadul Islam Sumon
- Department of Digital Anti-Aging Healthcare, u-AHRC, Inje University, Gimhae 50834, Korea; (K.I.); (K.S.C.); (Y.-B.H.); (R.I.S.); (H.-C.K.)
| | - Hee-Cheol Kim
- Department of Digital Anti-Aging Healthcare, u-AHRC, Inje University, Gimhae 50834, Korea; (K.I.); (K.S.C.); (Y.-B.H.); (R.I.S.); (H.-C.K.)
| | - Heung-Kook Choi
- Department of Computer Engineering, u-AHRC, Inje University, Gimhae 50834, Korea;
- Correspondence: ; Tel.: +82-10-6733-3437
| |
Collapse
|
10
|
Abstract
Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.
Collapse
|
11
|
Dif N, Elberrichi Z. Efficient Regularization Framework for Histopathological Image Classification Using Convolutional Neural Networks. INTERNATIONAL JOURNAL OF COGNITIVE INFORMATICS AND NATURAL INTELLIGENCE 2020. [DOI: 10.4018/ijcini.2020100104] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Deep learning methods are characterized by their capacity to learn data representation compared to the traditional machine learning algorithms. However, these methods are prone to overfitting on small volumes of data. The objective of this research is to overcome this limitation by improving the generalization in the proposed deep learning framework based on various techniques: data augmentation, small models, optimizer selection, and ensemble learning. For ensembling, the authors used selected models from different checkpoints and both voting and unweighted average methods for combination. The experimental study on the lymphomas histopathological dataset highlights the efficiency of the MobileNet2 network combined with the stochastic gradient descent (SGD) optimizer in terms of generalization. The best results have been achieved by the combination of the best three checkpoint models (98.67% of accuracy). These findings provide important insights into the efficiency of the checkpoint ensemble learning method for histopathological image classification.
Collapse
Affiliation(s)
- Nassima Dif
- EEDIS Laboratory ,Djillali Liabes University, Sidi Bel Abbes, Algeria
| | | |
Collapse
|
12
|
Dresp-Langley B, Wandeto JM. Pixel precise unsupervised detection of viral particle proliferation in cellular imaging data. INFORMATICS IN MEDICINE UNLOCKED 2020; 20:100433. [PMID: 32984498 PMCID: PMC7502244 DOI: 10.1016/j.imu.2020.100433] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2020] [Revised: 09/16/2020] [Accepted: 09/16/2020] [Indexed: 11/20/2022] Open
Abstract
Cellular and molecular imaging techniques and models have been developed to characterize single stages of viral proliferation after focal infection of cells in vitro. The fast and automatic classification of cell imaging data may prove helpful prior to any further comparison of representative experimental data to mathematical models of viral propagation in host cells. Here, we use computer generated images drawn from a reproduction of an imaging model from a previously published study of experimentally obtained cell imaging data representing progressive viral particle proliferation in host cell monolayers. Inspired by experimental time-based imaging data, here in this study viral particle increase in time is simulated by a one-by-one increase, across images, in black or gray single pixels representing dead or partially infected cells, and hypothetical remission by a one-by-one increase in white pixels coding for living cells in the original image model. The image simulations are submitted to unsupervised learning by a Self-Organizing Map (SOM) and the Quantization Error in the SOM output (SOM-QE) is used for automatic classification of the image simulations as a function of the represented extent of viral particle proliferation or cell recovery. Unsupervised classification by SOM-QE of 160 model images, each with more than three million pixels, is shown to provide a statistically reliable, pixel precise, and fast classification model that outperforms human computer-assisted image classification by RGB image mean computation. The automatic classification procedure proposed here provides a powerful approach to understand finely tuned mechanisms in the infection and proliferation of virus in cell lines in vitro or other cells. Automatic classification of cell imaging data for in vitro viral propagation in host cells. Economical in terms of computation times for rapid change/no change detection in large image data prior to human decision making. Simulating viral proliferation/cell recovery by progressive and selective single pixel changes in contrast polarity and/or intensity.
Collapse
Affiliation(s)
| | - John Mwangi Wandeto
- Department of Information Technology, Dedan Kimathi University of Technology, Kenya
| |
Collapse
|
13
|
Enhancing Multi-tissue and Multi-scale Cell Nuclei Segmentation with Deep Metric Learning. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10020615] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
(1) Background: The segmentation of cell nuclei is an essential task in a wide range of biomedical studies and clinical practices. The full automation of this process remains a challenge due to intra- and internuclear variations across a wide range of tissue morphologies, differences in staining protocols and imaging procedures. (2) Methods: A deep learning model with metric embeddings such as contrastive loss and triplet loss with semi-hard negative mining is proposed in order to accurately segment cell nuclei in a diverse set of microscopy images. The effectiveness of the proposed model was tested on a large-scale multi-tissue collection of microscopy image sets. (3) Results: The use of deep metric learning increased the overall segmentation prediction by 3.12% in the average value of Dice similarity coefficients as compared to no metric learning. In particular, the largest gain was observed for segmenting cell nuclei in H&E -stained images when deep learning network and triplet loss with semi-hard negative mining were considered for the task. (4) Conclusion: We conclude that deep metric learning gives an additional boost to the overall learning process and consequently improves the segmentation performance. Notably, the improvement ranges approximately between 0.13% and 22.31% for different types of images in the terms of Dice coefficients when compared to no metric deep learning.
Collapse
|
14
|
Barricelli BR, Casiraghi E, Gliozzo J, Huber V, Leone BE, Rizzi A, Vergani B. ki67 nuclei detection and ki67-index estimation: a novel automatic approach based on human vision modeling. BMC Bioinformatics 2019; 20:733. [PMID: 31881821 PMCID: PMC6935242 DOI: 10.1186/s12859-019-3285-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Accepted: 11/19/2019] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND The protein ki67 (pki67) is a marker of tumor aggressiveness, and its expression has been proven to be useful in the prognostic and predictive evaluation of several types of tumors. To numerically quantify the pki67 presence in cancerous tissue areas, pathologists generally analyze histochemical images to count the number of tumor nuclei marked for pki67. This allows estimating the ki67-index, that is the percentage of tumor nuclei positive for pki67 over all the tumor nuclei. Given the high image resolution and dimensions, its estimation by expert clinicians is particularly laborious and time consuming. Though automatic cell counting techniques have been presented so far, the problem is still open. RESULTS In this paper we present a novel automatic approach for the estimations of the ki67-index. The method starts by exploiting the STRESS algorithm to produce a color enhanced image where all pixels belonging to nuclei are easily identified by thresholding, and then separated into positive (i.e. pixels belonging to nuclei marked for pki67) and negative by a binary classification tree. Next, positive and negative nuclei pixels are processed separately by two multiscale procedures identifying isolated nuclei and separating adjoining nuclei. The multiscale procedures exploit two Bayesian classification trees to recognize positive and negative nuclei-shaped regions. CONCLUSIONS The evaluation of the computed results, both through experts' visual assessments and through the comparison of the computed indexes with those of experts, proved that the prototype is promising, so that experts believe in its potential as a tool to be exploited in the clinical practice as a valid aid for clinicians estimating the ki67-index. The MATLAB source code is open source for research purposes.
Collapse
Affiliation(s)
- Barbara Rita Barricelli
- Department of Information Engineering, Università degli Studi di Brescia, Via Branze 38, 25123 Brescia, Italy
| | - Elena Casiraghi
- Department of Computer Science, Università degli Studi di Milano, Via Celoria 18, 20133 Milan, Italy
| | - Jessica Gliozzo
- Fondazione IRCCS Ca’ Granda - Ospedale Maggiore Policlinico, Department of Dermatology, Viale Regina Marghertita, 20122 Milan, Italy
| | - Veronica Huber
- Unit of Immunotherapy of Human Tumors, Department of Research, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Biagio Eugenio Leone
- School of Medicine and Surgery, Università degli Studi di Milano-Bicocca, Via Cadore 48, 20900 Monza, Italy
| | - Alessandro Rizzi
- Department of Computer Science, Università degli Studi di Milano, Via Celoria 18, 20133 Milan, Italy
| | - Barbara Vergani
- School of Medicine and Surgery, Università degli Studi di Milano-Bicocca, Via Cadore 48, 20900 Monza, Italy
| |
Collapse
|
15
|
Bándi P, Balkenhol M, van Ginneken B, van der Laak J, Litjens G. Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks. PeerJ 2019; 7:e8242. [PMID: 31871843 PMCID: PMC6924324 DOI: 10.7717/peerj.8242] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2019] [Accepted: 11/19/2019] [Indexed: 12/20/2022] Open
Abstract
Modern pathology diagnostics is being driven toward large scale digitization of microscopic tissue sections. A prerequisite for its safe implementation is the guarantee that all tissue present on a glass slide can also be found back in the digital image. Whole-slide scanners perform a tissue segmentation in a low resolution overview image to prevent inefficient high-resolution scanning of empty background areas. However, currently applied algorithms can fail in detecting all tissue regions. In this study, we developed convolutional neural networks to distinguish tissue from background. We collected 100 whole-slide images of 10 tissue samples—staining categories from five medical centers for development and testing. Additionally, eight more images of eight unfamiliar categories were collected for testing only. We compared our fully-convolutional neural networks to three traditional methods on a range of resolution levels using Dice score and sensitivity. We also tested whether a single neural network can perform equivalently to multiple networks, each specialized in a single resolution. Overall, our solutions outperformed the traditional methods on all the tested resolutions. The resolution-agnostic network achieved average Dice scores between 0.97 and 0.98 across the tested resolution levels, only 0.0069 less than the resolution-specific networks. Finally, its excellent generalization performance was demonstrated by achieving averages of 0.98 Dice score and 0.97 sensitivity on the eight unfamiliar images. A future study should test this network prospectively.
Collapse
Affiliation(s)
- Péter Bándi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Maschenka Balkenhol
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
16
|
Bukenya F, Nerissa C, Serres S, Pardon MC, Bai L. An automated method for segmentation and quantification of blood vessels in histology images. Microvasc Res 2019; 128:103928. [PMID: 31676310 DOI: 10.1016/j.mvr.2019.103928] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2018] [Revised: 09/11/2019] [Accepted: 09/16/2019] [Indexed: 02/01/2023]
Abstract
Alzheimer's disease (AD) is a chronic neuro-degenerative disease that adversely affect many people on a global scale. Despite different diagnostic and therapeutic treatment, there is no cure for AD. The brain is one of the most complex organ and researchers are still trying to understand so as to find a cure. OBJECTIVE To complement the efforts of clinical researchers engaged in research in alzheimer's disease, accurate segmentation and quantification of blood vessels in brain images is required. METHOD For robust segmentation of blood vessels even in the presence of colour variation, we introduce a fully automated morphological tool that can extract and quantify vessels from haematoxylin and diaminobenzidine stained histology brain image. The method, exploits saturation channel of stained image slides, ISODATA threshold method is applied to obtain a binary image. This helps in eliminating background and remaining with only blood vessels. A one-stage procedure that includes eliminating small artefacts is performed on the binary mask. The intensity of the image is transformed. Joining is performed to deal with fragmentation of intact blood vessels on the images, and artefactual appearance of the blood vessel structures. The artefactual fragments based on measured incoherence with neighbouring tissue are removed. The vessels are then labelled to facilitate quantification. Morphometric measurements are used during the vessel quantification assess both vessels with lumen and vessels without lumen. We have quantified the diameter of blood vessels. RESULTS The image processing technique is developed in close collaboration with neuroscientist experts to help clinician. We have evaluated our proposed approach qualitatively. The method was validated against their manual quantification results. Qualitative results show that the method can indeed segment the blood vessels in the presence of colour variations and artefacts. The quantitative method produces fairly better results.
Collapse
Affiliation(s)
- Faiza Bukenya
- School of Computer Science, University of Nottingham, Nottingham NG8 1BB, United Kingdom.
| | - Culi Nerissa
- School of Life Sciences, University of Nottingham, Nottingham NG7 2UH, United Kingdom.
| | - Sébastien Serres
- School of Life Sciences, University of Nottingham, Nottingham NG7 2UH, United Kingdom.
| | | | - Li Bai
- School of Computer Science, University of Nottingham, Nottingham NG8 1BB, United Kingdom.
| |
Collapse
|
17
|
Konkankit CC, King AP, Knopf KM, Southard TL, Wilson JJ. In Vivo Anticancer Activity of a Rhenium(I) Tricarbonyl Complex. ACS Med Chem Lett 2019; 10:822-827. [PMID: 31098006 DOI: 10.1021/acsmedchemlett.9b00128] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2019] [Accepted: 04/16/2019] [Indexed: 12/22/2022] Open
Abstract
The rhenium(I) complex fac-[Re(CO)3(2,9-dimethyl-1,10-phenanthroline)(OH2)]+ (1) was previously shown to exhibit potent in vitro anticancer activity in a manner distinct from conventional platinum-based drugs (J. Am. Chem. Soc. 2017, 139, 14302-14314). In this study, we report further efforts to explore its aqueous speciation and antitumor activity. The cellular uptake of 1 was measured in A2780 and cisplatin-resistant A2780CP70 ovarian cancer cells by inductively coupled plasma mass spectrometry, revealing similar uptake efficiency in both cell lines. High accumulation in the mitochondria was observed, contradicting prior fluorescence microscopy studies. The luminescence of 1 is highly dependent on pH and coordination environment, making fluorescence microscopy somewhat unreliable for determining compound localization. The in vivo anticancer activity of 1 was evaluated in mice bearing patient-derived ovarian cancer tumor xenografts. These studies conclusively show that 1 is capable of inhibiting tumor growth, providing further credibility for the use of these compounds as anticancer agents.
Collapse
Affiliation(s)
- Chilaluck C. Konkankit
- Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, United States
| | - A. Paden King
- Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, United States
| | - Kevin M. Knopf
- Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, United States
| | - Teresa L. Southard
- Department of Biomedical Sciences, Cornell University, Ithaca, New York 14853, United States
| | - Justin J. Wilson
- Department of Chemistry and Chemical Biology, Cornell University, Ithaca, New York 14853, United States
| |
Collapse
|
18
|
Neagu AN. Proteome Imaging: From Classic to Modern Mass Spectrometry-Based Molecular Histology. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2019; 1140:55-98. [PMID: 31347042 DOI: 10.1007/978-3-030-15950-4_4] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
In order to overcome the limitations of classic imaging in Histology during the actually era of multiomics, the multi-color "molecular microscope" by its emerging "molecular pictures" offers quantitative and spatial information about thousands of molecular profiles without labeling of potential targets. Healthy and diseased human tissues, as well as those of diverse invertebrate and vertebrate animal models, including genetically engineered species and cultured cells, can be easily analyzed by histology-directed MALDI imaging mass spectrometry. The aims of this review are to discuss a range of proteomic information emerging from MALDI mass spectrometry imaging comparative to classic histology, histochemistry and immunohistochemistry, with applications in biology and medicine, concerning the detection and distribution of structural proteins and biological active molecules, such as antimicrobial peptides and proteins, allergens, neurotransmitters and hormones, enzymes, growth factors, toxins and others. The molecular imaging is very well suited for discovery and validation of candidate protein biomarkers in neuroproteomics, oncoproteomics, aging and age-related diseases, parasitoproteomics, forensic, and ecotoxicology. Additionally, in situ proteome imaging may help to elucidate the physiological and pathological mechanisms involved in developmental biology, reproductive research, amyloidogenesis, tumorigenesis, wound healing, neural network regeneration, matrix mineralization, apoptosis and oxidative stress, pain tolerance, cell cycle and transformation under oncogenic stress, tumor heterogeneity, behavior and aggressiveness, drugs bioaccumulation and biotransformation, organism's reaction against environmental penetrating xenobiotics, immune signaling, assessment of integrity and functionality of tissue barriers, behavioral biology, and molecular origins of diseases. MALDI MSI is certainly a valuable tool for personalized medicine and "Eco-Evo-Devo" integrative biology in the current context of global environmental challenges.
Collapse
Affiliation(s)
- Anca-Narcisa Neagu
- Laboratory of Animal Histology, Faculty of Biology, "Alexandru Ioan Cuza" University of Iasi, Iasi, Romania.
| |
Collapse
|
19
|
Roy K, Banik D, Bhattacharjee D, Nasipuri M. Patch-based system for Classification of Breast Histology images using deep learning. Comput Med Imaging Graph 2019; 71:90-103. [DOI: 10.1016/j.compmedimag.2018.11.003] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2018] [Revised: 11/26/2018] [Accepted: 11/28/2018] [Indexed: 12/29/2022]
|
20
|
Casiraghi E, Huber V, Frasca M, Cossa M, Tozzi M, Rivoltini L, Leone BE, Villa A, Vergani B. A novel computational method for automatic segmentation, quantification and comparative analysis of immunohistochemically labeled tissue sections. BMC Bioinformatics 2018; 19:357. [PMID: 30367588 PMCID: PMC6191943 DOI: 10.1186/s12859-018-2302-3] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Background In the clinical practice, the objective quantification of histological results is essential not only to define objective and well-established protocols for diagnosis, treatment, and assessment, but also to ameliorate disease comprehension. Software The software MIAQuant_Learn presented in this work segments, quantifies and analyzes markers in histochemical and immunohistochemical images obtained by different biological procedures and imaging tools. MIAQuant_Learn employs supervised learning techniques to customize the marker segmentation process with respect to any marker color appearance. Our software expresses the location of the segmented markers with respect to regions of interest by mean-distance histograms, which are numerically compared by measuring their intersection. When contiguous tissue sections stained by different markers are available, MIAQuant_Learn aligns them and overlaps the segmented markers in a unique image enabling a visual comparative analysis of the spatial distribution of each marker (markers’ relative location). Additionally, it computes novel measures of markers’ co-existence in tissue volumes depending on their density. Conclusions Applications of MIAQuant_Learn in clinical research studies have proven its effectiveness as a fast and efficient tool for the automatic extraction, quantification and analysis of histological sections. It is robust with respect to several deficits caused by image acquisition systems and produces objective and reproducible results. Thanks to its flexibility, MIAQuant_Learn represents an important tool to be exploited in basic research where needs are constantly changing.
Collapse
Affiliation(s)
- Elena Casiraghi
- Department of Computer Science "Giovanni Degli Antoni", Università degli Studi di Milano, Via Celoria 18, 20135, Milan, Italy.
| | - Veronica Huber
- Unit of Immunotherapy of Human Tumors, Department of Experimental Oncology and Molecular Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Marco Frasca
- Department of Computer Science "Giovanni Degli Antoni", Università degli Studi di Milano, Via Celoria 18, 20135, Milan, Italy
| | - Mara Cossa
- Unit of Immunotherapy of Human Tumors, Department of Experimental Oncology and Molecular Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Matteo Tozzi
- Department of medicine and surgery, Vascular Surgery, University of Insubria Hospital, Varese, Italy
| | - Licia Rivoltini
- Unit of Immunotherapy of Human Tumors, Department of Experimental Oncology and Molecular Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | | | - Antonello Villa
- School of Medicine and Surgery, University of Milano Bicocca, Monza, Italy.,Consorzio MIA - Microscopy and Image Analysis, University of Milano Bicocca, Monza, Italy
| | - Barbara Vergani
- School of Medicine and Surgery, University of Milano Bicocca, Monza, Italy.,Consorzio MIA - Microscopy and Image Analysis, University of Milano Bicocca, Monza, Italy
| |
Collapse
|