1
|
Kundu S, Sair H, Sherr EH, Mukherjee P, Rohde GK. Discovering the gene-brain-behavior link in autism via generative machine learning. SCIENCE ADVANCES 2024; 10:eadl5307. [PMID: 38865470 PMCID: PMC11168471 DOI: 10.1126/sciadv.adl5307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 05/08/2024] [Indexed: 06/14/2024]
Abstract
Autism is traditionally diagnosed behaviorally but has a strong genetic basis. A genetics-first approach could transform understanding and treatment of autism. However, isolating the gene-brain-behavior relationship from confounding sources of variability is a challenge. We demonstrate a novel technique, 3D transport-based morphometry (TBM), to extract the structural brain changes linked to genetic copy number variation (CNV) at the 16p11.2 region. We identified two distinct endophenotypes. In data from the Simons Variation in Individuals Project, detection of these endophenotypes enabled 89 to 95% test accuracy in predicting 16p11.2 CNV from brain images alone. Then, TBM enabled direct visualization of the endophenotypes driving accurate prediction, revealing dose-dependent brain changes among deletion and duplication carriers. These endophenotypes are sensitive to articulation disorders and explain a portion of the intelligence quotient variability. Genetic stratification combined with TBM could reveal new brain endophenotypes in many neurodevelopmental disorders, accelerating precision medicine, and understanding of human neurodiversity.
Collapse
Affiliation(s)
- Shinjini Kundu
- Department of Radiology, Johns Hopkins University School of Medicine, Baltimore, USA
| | - Haris Sair
- Department of Radiology, Johns Hopkins University School of Medicine, Baltimore, USA
| | - Elliott H. Sherr
- Department of Neurology, University of California San Francisco, San Francisco, USA
| | - Pratik Mukherjee
- Department of Radiology, University of California San Francisco, San Francisco, USA
| | - Gustavo K. Rohde
- Department of Biomedical Engineering, University of Virginia, Charlottesville, USA
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, USA
| |
Collapse
|
2
|
Slabaugh G, Beltran L, Rizvi H, Deloukas P, Marouli E. Applications of machine and deep learning to thyroid cytology and histopathology: a review. Front Oncol 2023; 13:958310. [PMID: 38023130 PMCID: PMC10661921 DOI: 10.3389/fonc.2023.958310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 10/12/2023] [Indexed: 12/01/2023] Open
Abstract
This review synthesises past research into how machine and deep learning can improve the cyto- and histopathology processing pipelines for thyroid cancer diagnosis. The current gold-standard preoperative technique of fine-needle aspiration cytology has high interobserver variability, often returns indeterminate samples and cannot reliably identify some pathologies; histopathology analysis addresses these issues to an extent, but it requires surgical resection of the suspicious lesions so cannot influence preoperative decisions. Motivated by these issues, as well as by the chronic shortage of trained pathologists, much research has been conducted into how artificial intelligence could improve current pipelines and reduce the pressure on clinicians. Many past studies have indicated the significant potential of automated image analysis in classifying thyroid lesions, particularly for those of papillary thyroid carcinoma, but these have generally been retrospective, so questions remain about both the practical efficacy of these automated tools and the realities of integrating them into clinical workflows. Furthermore, the nature of thyroid lesion classification is significantly more nuanced in practice than many current studies have addressed, and this, along with the heterogeneous nature of processing pipelines in different laboratories, means that no solution has proven itself robust enough for clinical adoption. There are, therefore, multiple avenues for future research: examine the practical implementation of these algorithms as pathologist decision-support systems; improve interpretability, which is necessary for developing trust with clinicians and regulators; and investigate multiclassification on diverse multicentre datasets, aiming for methods that demonstrate high performance in a process- and equipment-agnostic manner.
Collapse
Affiliation(s)
- Greg Slabaugh
- Digital Environment Research Institute, Queen Mary University of London, London, United Kingdom
| | - Luis Beltran
- Barts Health NHS Trust, The Royal London Hospital, London, United Kingdom
| | - Hasan Rizvi
- Barts Health NHS Trust, The Royal London Hospital, London, United Kingdom
| | - Panos Deloukas
- William Harvey Research Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| | - Eirini Marouli
- Digital Environment Research Institute, Queen Mary University of London, London, United Kingdom
- Barts Health NHS Trust, The Royal London Hospital, London, United Kingdom
- William Harvey Research Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| |
Collapse
|
3
|
Kussaibi H, Alsafwani N. Trends in AI-powered Classification of Thyroid Neoplasms Based on Histopathology Images - a Systematic Review. Acta Inform Med 2023; 31:280-286. [PMID: 38379694 PMCID: PMC10875959 DOI: 10.5455/aim.2023.31.280-286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 12/20/2023] [Indexed: 02/22/2024] Open
Abstract
Background Assessment of thyroid nodules histopathology using AI is crucial for an accurate diagnosis. This systematic review analyzes recent works employing deep learning approaches for classifying thyroid nodules based on histopathology images, evaluating their performance, and identifying limitations. Methods Eligibility criteria focused on peer-reviewed English papers published in the last 5 years, applying deep learning to categorize thyroid histopathology images. The PubMed database was searched using relevant keyword combinations. Results Out of 103 articles, 11 studies met inclusion criteria. They used convolutional neural networks to classify thyroid neoplasm. Most studies aimed for basic tumor subtyping; however, 3 studies targeted the prediction of tumor-associated mutation. Accuracy ranged from 77% to 100%, with most over 90%. Discussion The findings from our analysis reveal the effectiveness of deep learning in identifying discriminative morphological patterns from histopathology images, thus enhancing the accuracy of thyroid nodule histopathological classification. Key limitations were small sample sizes, subjective annotation, and limited dataset diversity. Further research with larger diverse datasets, model optimization, and real-world validation is essential to translate these tools into clinical practice.
Collapse
Affiliation(s)
- Haitham Kussaibi
- Department of Pathology, College of Medicine, Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia
| | - Noor Alsafwani
- Department of Pathology, College of Medicine, Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia
| |
Collapse
|
4
|
Beier F, Beinert R, Steidl G. On a Linear Gromov-Wasserstein Distance. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:7292-7305. [PMID: 36378791 DOI: 10.1109/tip.2022.3221286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Gromov-Wasserstein distances are generalization of Wasserstein distances, which are invariant under distance preserving transformations. Although a simplified version of optimal transport in Wasserstein spaces, called linear optimal transport (LOT), was successfully used in practice, there does not exist a notion of linear Gromov-Wasserstein distances so far. In this paper, we propose a definition of linear Gromov-Wasserstein distances. We motivate our approach by a generalized LOT model, which is based on barycentric projection maps of transport plans. Numerical examples illustrate that the linear Gromov-Wasserstein distances, similarly as LOT, can replace the expensive computation of pairwise Gromov-Wasserstein distances in applications like shape classification.
Collapse
|
5
|
Xue Y, Zhou Y, Wang T, Chen H, Wu L, Ling H, Wang H, Qiu L, Ye D, Wang B. Accuracy of Ultrasound Diagnosis of Thyroid Nodules Based on Artificial Intelligence-Assisted Diagnostic Technology: A Systematic Review and Meta-Analysis. Int J Endocrinol 2022; 2022:9492056. [PMID: 36193283 PMCID: PMC9525757 DOI: 10.1155/2022/9492056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Revised: 08/04/2022] [Accepted: 08/24/2022] [Indexed: 12/07/2022] Open
Abstract
BACKGROUND Ultrasonography (US) is the most common method of identifying thyroid nodules, but US images require an experienced surgeon for identification. Many artificial intelligence (AI) techniques such as computer-aided diagnostic systems (CAD), deep learning (DL), and machine learning (ML) have been used to assist in the diagnosis of thyroid nodules, but whether AI techniques can improve the diagnostic accuracy of thyroid nodules still needs to be explored. OBJECTIVE To clarify the accuracy of AI-based thyroid nodule US images for differentiating benign and malignant thyroid nodules. METHODS A search strategy of "subject terms + key words" was used to search PubMed, Cochrane Library, Embase, Web of Science, China Biology Medicine (CBM), and China National Knowledge Infrastructure (CNKI) for studies on AI-assisted diagnosis of thyroid nodules based on US images. The summarized receiver operating characteristic (SROC) curve and the pooled sensitivity and specificity were used to assess the performance of the diagnostic tests. The quality assessment of diagnostics accuracy studies-2 (QUADAS-2) tool was used to assess the methodological quality of the included studies. The Review Manager 5.3 and Stata 15 were used to process the data. Subgroup analysis was based on the integrity of data collection. RESULTS A total of 25 studies with 17,429 US images of thyroid nodules were included. AI-assisted diagnostic techniques had better diagnostic efficacy in the diagnosis of benign and malignant thyroid nodules: sensitivity 0.88 (95% CI: (0.85-0.90)), specificity 0.81 (95% CI: 0.74-0.86), diagnostic odds ratio (DOR) 30 (95% CI: 19-46). The SROC curve indicated that the area under the curve (AUC) was 0.92 (95% CI: 0.89-0.94). Threshold effect analysis showed a Spearman correlation coefficient: 0.17 < 0.5, suggesting no threshold effect for the included studies. After a meta-regression analysis of 4 different subgroups, the results showed a statistically significant effect of mean age ≥50 years on heterogeneity. Compared with studies with an average age of ≥50 years, AI-assisted diagnostic techniques had higher diagnostic performance in studies with an average age of <50 years (0.89 (95% CI: 0.87-0.92) vs. 0.80 (95% CI: 0.73-0.88)), (0.83 (95% CI: 0.77-0.88) vs. 0.73 (95% CI: 0.60-0.87)). CONCLUSIONS AI-assisted diagnostic techniques had good diagnostic efficacy for thyroid nodules. For the diagnosis of <50 year olds, AI-assisted diagnostic technology was more effective in diagnosis.
Collapse
Affiliation(s)
- Yu Xue
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Ying Zhou
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Tingrui Wang
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Huijuan Chen
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Lingling Wu
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Huayun Ling
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Hong Wang
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Lijuan Qiu
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Dongqing Ye
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| | - Bin Wang
- Department of Epidemiology and Biostatistics, School of Public Health, Anhui Medical University, Hefei, Anhui, China
- Inflammation and Immune Mediated Diseases Laboratory of Anhui Province, Hefei, Anhui, China
| |
Collapse
|
6
|
Han B, Zhang M, Gao X, Wang Z, You F, Li H. Automatic classification method of thyroid pathological images using multiple magnification factors. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.07.024] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
7
|
Machine learning methods for automated classification of tumors with papillary thyroid carcinoma-like nuclei: A quantitative analysis. PLoS One 2021; 16:e0257635. [PMID: 34550999 PMCID: PMC8457451 DOI: 10.1371/journal.pone.0257635] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 09/04/2021] [Indexed: 11/19/2022] Open
Abstract
When approaching thyroid gland tumor classification, the differentiation between samples with and without "papillary thyroid carcinoma-like" nuclei is a daunting task with high inter-observer variability among pathologists. Thus, there is increasing interest in the use of machine learning approaches to provide pathologists real-time decision support. In this paper, we optimize and quantitatively compare two automated machine learning methods for thyroid gland tumor classification on two datasets to assist pathologists in decision-making regarding these methods and their parameters. The first method is a feature-based classification originating from common image processing and consists of cell nucleus segmentation, feature extraction, and subsequent thyroid gland tumor classification utilizing different classifiers. The second method is a deep learning-based classification which directly classifies the input images with a convolutional neural network without the need for cell nucleus segmentation. On the Tharun and Thompson dataset, the feature-based classification achieves an accuracy of 89.7% (Cohen's Kappa 0.79), compared to the deep learning-based classification of 89.1% (Cohen's Kappa 0.78). On the Nikiforov dataset, the feature-based classification achieves an accuracy of 83.5% (Cohen's Kappa 0.46) compared to the deep learning-based classification 77.4% (Cohen's Kappa 0.35). Thus, both automated thyroid tumor classification methods can reach the classification level of an expert pathologist. To our knowledge, this is the first study comparing feature-based and deep learning-based classification regarding their ability to classify samples with and without papillary thyroid carcinoma-like nuclei on two large-scale datasets.
Collapse
|
8
|
Aldroubi A, Li S, Rohde GK. PARTITIONING SIGNAL CLASSES USING TRANSPORT TRANSFORMS FOR DATA ANALYSIS AND MACHINE LEARNING. SAMPLING THEORY, SIGNAL PROCESSING, AND DATA ANALYSIS 2021; 19:6. [PMID: 35547330 PMCID: PMC9090194 DOI: 10.1007/s43670-021-00009-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 04/21/2021] [Indexed: 06/15/2023]
Abstract
A relatively new set of transport-based transforms (CDT, R-CDT, LOT) have shown their strength and great potential in various image and data processing tasks such as parametric signal estimation, classification, cancer detection among many others. It is hence worthwhile to elucidate some of the mathematical properties that explain the successes of these transforms when they are used as tools in data analysis, signal processing or data classification. In particular, we give conditions under which classes of signals that are created by algebraic generative models are transformed into convex sets by the transport transforms. Such convexification of the classes simplify the classification and other data analysis and processing problems when viewed in the transform domain. More specifically, we study the extent and limitation of the convexification ability of these transforms under an algebraic generative modeling framework. We hope that this paper will serve as an introduction to these transforms and will encourage mathematicians and other researchers to further explore the theoretical underpinnings and algorithmic tools that will help understand the successes of these transforms and lay the groundwork for further successful applications.
Collapse
Affiliation(s)
| | - Shiying Li
- Imaging and Data Science Laboratory Department of Biomedical Engineering University of Virginia
| | - Gustavo K Rohde
- Imaging and Data Science Laboratory Department of Biomedical Engineering Department of Electrical and Computer Engineering University of Virginia
| |
Collapse
|
9
|
Abstract
Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.
Collapse
|
10
|
Li LR, Du B, Liu HQ, Chen C. Artificial Intelligence for Personalized Medicine in Thyroid Cancer: Current Status and Future Perspectives. Front Oncol 2021; 10:604051. [PMID: 33634025 PMCID: PMC7899964 DOI: 10.3389/fonc.2020.604051] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 12/21/2020] [Indexed: 12/12/2022] Open
Abstract
Thyroid cancers (TC) have increasingly been detected following advances in diagnostic methods. Risk stratification guided by refined information becomes a crucial step toward the goal of personalized medicine. The diagnosis of TC mainly relies on imaging analysis, but visual examination may not reveal much information and not enable comprehensive analysis. Artificial intelligence (AI) is a technology used to extract and quantify key image information by simulating complex human functions. This latent, precise information contributes to stratify TC on the distinct risk and drives tailored management to transit from the surface (population-based) to a point (individual-based). In this review, we started with several challenges regarding personalized care in TC, for example, inconsistent rating ability of ultrasound physicians, uncertainty in cytopathological diagnosis, difficulty in discriminating follicular neoplasms, and inaccurate prognostication. We then analyzed and summarized the advances of AI to extract and analyze morphological, textural, and molecular features to reveal the ground truth of TC. Consequently, their combination with AI technology will make individual medical strategies possible.
Collapse
Affiliation(s)
- Ling-Rui Li
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| | - Bo Du
- School of Computer Science, Wuhan University, Wuhan, China.,Institute of Artificial Intelligence, Wuhan University, Wuhan, China
| | - Han-Qing Liu
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chuang Chen
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
11
|
An experimental study on classification of thyroid histopathology images using transfer learning. Pattern Recognit Lett 2020. [DOI: 10.1016/j.patrec.2020.09.020] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
12
|
Radial Basis Function Artificial Neural Network for the Investigation of Thyroid Cytological Lesions. J Thyroid Res 2020; 2020:5464787. [PMID: 33299540 PMCID: PMC7707952 DOI: 10.1155/2020/5464787] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Revised: 07/17/2020] [Accepted: 10/24/2020] [Indexed: 01/21/2023] Open
Abstract
Objective This study investigates the potential of an artificial intelligence (AI) methodology, the radial basis function (RBF) artificial neural network (ANN), in the evaluation of thyroid lesions. Study Design. The study was performed on 447 patients who had both cytological and histological evaluation in agreement. Cytological specimens were prepared using liquid-based cytology, and the histological result was based on subsequent surgical samples. Each specimen was digitized; on these images, nuclear morphology features were measured by the use of an image analysis system. The extracted measurements (41,324 nuclei) were separated into two sets: the training set that was used to create the RBF ANN and the test set that was used to evaluate the RBF performance. The system aimed to predict the histological status as benign or malignant. Results The RBF ANN obtained in the training set has sensitivity 82.5%, specificity 94.6%, and overall accuracy 90.3%, while in the test set, these indices were 81.4%, 90.0%, and 86.9%, respectively. Algorithm was used to classify patients on the basis of the RBF ANN, the overall sensitivity was 95.0%, the specificity was 95.5%, and no statistically significant difference was observed. Conclusion AI techniques and especially ANNs, only in the recent years, have been studied extensively. The proposed approach is promising to avoid misdiagnoses and assists the everyday practice of the cytopathology. The major drawback in this approach is the automation of a procedure to accurately detect and measure cell nuclei from the digitized images.
Collapse
|
13
|
Dov D, Kovalsky SZ, Assaad S, Cohen J, Range DE, Pendse AA, Henao R, Carin L. Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images. Med Image Anal 2020; 67:101814. [PMID: 33049578 DOI: 10.1016/j.media.2020.101814] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2019] [Revised: 06/16/2020] [Accepted: 07/28/2020] [Indexed: 02/07/2023]
Abstract
We consider machine-learning-based thyroid-malignancy prediction from cytopathology whole-slide images (WSI). Multiple instance learning (MIL) approaches, typically used for the analysis of WSIs, divide the image (bag) into patches (instances), which are used to predict a single bag-level label. These approaches perform poorly in cytopathology slides due to a unique bag structure: sparsely located informative instances with varying characteristics of abnormality. We address these challenges by considering multiple types of labels: bag-level malignancy and ordered diagnostic scores, as well as instance-level informativeness and abnormality labels. We study their contribution beyond the MIL setting by proposing a maximum likelihood estimation (MLE) framework, from which we derive a two-stage deep-learning-based algorithm. The algorithm identifies informative instances and assigns them local malignancy scores that are incorporated into a global malignancy prediction. We derive a lower bound of the MLE, leading to an improved training strategy based on weak supervision, that we motivate through statistical analysis. The lower bound further allows us to extend the proposed algorithm to simultaneously predict multiple bag and instance-level labels from a single output of a neural network. Experimental results demonstrate that the proposed algorithm provides competitive performance compared to several competing methods, achieves (expert) human-level performance, and allows augmentation of human decisions.
Collapse
Affiliation(s)
- David Dov
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA.
| | | | - Serge Assaad
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| | - Jonathan Cohen
- Department of Surgery, Duke University Medical Center, Durham, NC 27710, USA
| | | | - Avani A Pendse
- Department of Pathology, Duke University Medical Center, Durham, NC 27710, USA
| | - Ricardo Henao
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| | - Lawrence Carin
- Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
| |
Collapse
|
14
|
Abstract
Pathologists are adopting whole slide images (WSIs) for diagnosis, thanks to recent FDA approval of WSI systems as class II medical devices. In response to new market forces and recent technology advances outside of pathology, a new field of computational pathology has emerged that applies artificial intelligence (AI) and machine learning algorithms to WSIs. Computational pathology has great potential for augmenting pathologists' accuracy and efficiency, but there are important concerns regarding trust of AI due to the opaque, black-box nature of most AI algorithms. In addition, there is a lack of consensus on how pathologists should incorporate computational pathology systems into their workflow. To address these concerns, building computational pathology systems with explainable AI (xAI) mechanisms is a powerful and transparent alternative to black-box AI models. xAI can reveal underlying causes for its decisions; this is intended to promote safety and reliability of AI for critical tasks such as pathology diagnosis. This article outlines xAI enabled applications in anatomic pathology workflow that improves efficiency and accuracy of the practice. In addition, we describe HistoMapr-Breast, an initial xAI enabled software application for breast core biopsies. HistoMapr-Breast automatically previews breast core WSIs and recognizes the regions of interest to rapidly present the key diagnostic areas in an interactive and explainable manner. We anticipate xAI will ultimately serve pathologists as an interactive computational guide for computer-assisted primary diagnosis.
Collapse
|
15
|
Elliott Range DD, Dov D, Kovalsky SZ, Henao R, Carin L, Cohen J. Application of a machine learning algorithm to predict malignancy in thyroid cytopathology. Cancer Cytopathol 2020; 128:287-295. [PMID: 32012493 DOI: 10.1002/cncy.22238] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2019] [Revised: 12/11/2019] [Accepted: 12/12/2019] [Indexed: 02/06/2023]
Abstract
BACKGROUND The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC) comprises 6 categories used for the diagnosis of thyroid fine-needle aspiration biopsy (FNAB). Each category has an associated risk of malignancy, which is important in the management of a thyroid nodule. More accurate predictions of malignancy may help to reduce unnecessary surgery. A machine learning algorithm (MLA) was developed to evaluate thyroid FNAB via whole slide images (WSIs) to predict malignancy. METHODS Files were searched for all thyroidectomy specimens with preceding FNAB over 8 years. All cytologic and surgical pathology diagnoses were recorded and correlated for each nodule. One representative slide from each case was scanned to create a WSI. An MLA was designed to identify follicular cells and predict the malignancy of the final pathology. The test set comprised cases blindly reviewed by a cytopathologist who assigned a TBSRTC category. The area under the receiver operating characteristic curve was used to assess the MLA performance. RESULTS Nine hundred eight FNABs met the criteria. The MLA predicted malignancy with a sensitivity and specificity of 92.0% and 90.5%, respectively. The areas under the curve for the prediction of malignancy by the cytopathologist and the MLA were 0.931 and 0.932, respectively. CONCLUSIONS The performance of the MLA in predicting thyroid malignancy from FNAB WSIs is comparable to the performance of an expert cytopathologist. When the MLA and electronic medical record diagnoses are combined, the performance is superior to the performance of either alone. An MLA may be used as an adjunct to FNAB to assist in refining the indeterminate categories.
Collapse
Affiliation(s)
| | - David Dov
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, North Carolina
| | - Shahar Z Kovalsky
- Department of Mathematics, Trinity College of Arts and Sciences, Duke University, Durham, North Carolina
| | - Ricardo Henao
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, North Carolina.,Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina
| | - Lawrence Carin
- Department of Electrical and Computer Engineering, Pratt School of Engineering, Duke University, Durham, North Carolina
| | - Jonathan Cohen
- Department of Head and Neck Surgery and Communication Sciences, Duke University School of Medicine, Durham, North Carolina
| |
Collapse
|
16
|
Halicek M, Shahedi M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Head and Neck Cancer Detection in Digitized Whole-Slide Histology Using Convolutional Neural Networks. Sci Rep 2019; 9:14043. [PMID: 31575946 PMCID: PMC6773771 DOI: 10.1038/s41598-019-50313-x] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2019] [Accepted: 09/10/2019] [Indexed: 01/01/2023] Open
Abstract
Primary management for head and neck cancers, including squamous cell carcinoma (SCC), involves surgical resection with negative cancer margins. Pathologists guide surgeons during these operations by detecting cancer in histology slides made from the excised tissue. In this study, 381 digitized, histological whole-slide images (WSI) from 156 patients with head and neck cancer were used to train, validate, and test an inception-v4 convolutional neural network. The proposed method is able to detect and localize primary head and neck SCC on WSI with an AUC of 0.916 for patients in the SCC testing group and 0.954 for patients in the thyroid carcinoma testing group. Moreover, the proposed method is able to diagnose WSI with cancer versus normal slides with an AUC of 0.944 and 0.995 for the SCC and thyroid carcinoma testing groups, respectively. For comparison, we tested the proposed, diagnostic method on an open-source dataset of WSI from sentinel lymph nodes with breast cancer metastases, CAMELYON 2016, to obtain patch-based cancer localization and slide-level cancer diagnoses. The experimental design yields a robust method with potential to help create a tool to increase efficiency and accuracy of pathologists detecting head and neck cancers in histological images.
Collapse
Affiliation(s)
- Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA.,Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA
| | - James V Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA, USA
| | - Amy Y Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA, USA
| | - Larry L Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Baran D Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX, USA. .,Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, TX, USA. .,Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
17
|
Yan J, Deng C, Luo L, Wang X, Yao X, Shen L, Huang H. Identifying Imaging Markers for Predicting Cognitive Assessments Using Wasserstein Distances Based Matrix Regression. Front Neurosci 2019; 13:668. [PMID: 31354405 PMCID: PMC6636330 DOI: 10.3389/fnins.2019.00668] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2019] [Accepted: 06/11/2019] [Indexed: 11/18/2022] Open
Abstract
Alzheimer's disease (AD) is a severe type of neurodegeneration which worsens human memory, thinking and cognition along a temporal continuum. How to identify the informative phenotypic neuroimaging markers and accurately predict cognitive assessment are crucial for early detection and diagnosis Alzheimer's disease. Regression models are widely used to predict the relationship between imaging biomarkers and cognitive assessment, and identify discriminative neuroimaging markers. Most existing methods use different matrix norms as the similarity measures of the empirical loss or regularization to improve the prediction performance, but ignore the inherent geometry of the cognitive data. To tackle this issue, in this paper we propose a novel robust matrix regression model with imposing Wasserstein distances on both loss function and regularization. It successfully integrate Wasserstein distance into the regression model, which can excavate the latent geometry of cognitive data. We introduce an efficient algorithm to solve the proposed new model with convergence analysis. Empirical results on cognitive data of the ADNI cohort demonstrate the great effectiveness of the proposed method for clinical cognitive predication.
Collapse
Affiliation(s)
- Jiexi Yan
- School of Electronic Engineering, Xidian University, Xi'an, China
| | - Cheng Deng
- School of Electronic Engineering, Xidian University, Xi'an, China
| | - Lei Luo
- Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, United States
| | - Xiaoqian Wang
- Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, United States
| | - Xiaohui Yao
- Department of Biostatistics, Epidemiology and Informatics Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Li Shen
- Department of Biostatistics, Epidemiology and Informatics Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Heng Huang
- Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, United States
| |
Collapse
|
18
|
Crowson MG, Ranisau J, Eskander A, Babier A, Xu B, Kahmke RR, Chen JM, Chan TCY. A contemporary review of machine learning in otolaryngology-head and neck surgery. Laryngoscope 2019; 130:45-51. [PMID: 30706465 DOI: 10.1002/lary.27850] [Citation(s) in RCA: 69] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2018] [Accepted: 01/11/2019] [Indexed: 11/07/2022]
Abstract
One of the key challenges with big data is leveraging the complex network of information to yield useful clinical insights. The confluence of massive amounts of health data and a desire to make inferences and insights on these data has produced a substantial amount of interest in machine-learning analytic methods. There has been a drastic increase in the otolaryngology literature volume describing novel applications of machine learning within the past 5 years. In this timely contemporary review, we provide an overview of popular machine-learning techniques, and review recent machine-learning applications in otolaryngology-head and neck surgery including neurotology, head and neck oncology, laryngology, and rhinology. Investigators have realized significant success in validated models with model sensitivities and specificities approaching 100%. Challenges remain in the implementation of machine-learning algorithms. This may be in part the unfamiliarity of these techniques to clinician leaders on the front lines of patient care. Spreading awareness and confidence in machine learning will follow with further validation and proof-of-value analyses that demonstrate model performance superiority over established methods. We are poised to see a greater influx of machine-learning applications to clinical problems in otolaryngology-head and neck surgery, and it is prudent for providers to understand the potential benefits and limitations of these technologies. Laryngoscope, 130:45-51, 2020.
Collapse
Affiliation(s)
- Matthew G Crowson
- Department of Otolaryngology-Head and Neck Surgery, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
| | - Jonathan Ranisau
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada
| | - Antoine Eskander
- Department of Otolaryngology-Head and Neck Surgery, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
- Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
| | - Aaron Babier
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada
| | - Bin Xu
- Department of Pathology, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
| | - Russel R Kahmke
- Division of Otolaryngology-Head and Neck Surgery, Duke University Medical Center, Durham, North Carolina, U.S.A
| | - Joseph M Chen
- Department of Otolaryngology-Head and Neck Surgery, Sunnybrook Health Sciences Center, Toronto, Ontario, Canada
| | - Timothy C Y Chan
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
19
|
Savala R, Dey P, Gupta N. Artificial neural network model to distinguish follicular adenoma from follicular carcinoma on fine needle aspiration of thyroid. Diagn Cytopathol 2017; 46:244-249. [DOI: 10.1002/dc.23880] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2017] [Revised: 12/01/2017] [Accepted: 12/11/2017] [Indexed: 11/06/2022]
Affiliation(s)
- Rajiv Savala
- Department of Pathology; Postgraduate Institute of Medical Education and Research; Chandigarh India
| | - Pranab Dey
- Department of Cytology; Post Graduate Institute of Medical Education and Research; Chandigarh India
| | - Nalini Gupta
- Department of Cytology; Post Graduate Institute of Medical Education and Research; Chandigarh India
| |
Collapse
|
20
|
Thorpe M, Park S, Kolouri S, Rohde GK, Slepčev D. A Transportation Lp Distance for Signal Analysis. JOURNAL OF MATHEMATICAL IMAGING AND VISION 2017; 59:187-210. [PMID: 30233108 PMCID: PMC6141213 DOI: 10.1007/s10851-017-0726-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2016] [Accepted: 03/13/2017] [Indexed: 06/08/2023]
Abstract
Transport based distances, such as the Wasserstein distance and earth mover'sdistance, have been shown to be an effective tool in signal and image analysis. The success of transport based distances is in part due to their Lagrangian nature which allows it to capture the important variations in many signal classes. However these distances require the signal to be nonnegative and normalized. Furthermore, the signals are considered as measures and compared by redistributing (transporting) them, which does not directly take into account the signal intensity. Here we study a transport-based distance, called the TLp distance, that combines Lagrangian and intensity modelling and is directly applicable to general, non-positive and multi-channelled signals. The distance can be computed by existing numerical methods. We give an overview of the basic properties of this distance and applications to classification, with multi-channelled non-positive one-dimensional signals and two-dimensional images, and color transfer.
Collapse
Affiliation(s)
| | - Serim Park
- Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | | | | | - Dejan Slepčev
- Carnegie Mellon University, Pittsburgh, PA 15213, USA
| |
Collapse
|
21
|
Yoon J, Jo Y, Kim MH, Kim K, Lee S, Kang SJ, Park Y. Identification of non-activated lymphocytes using three-dimensional refractive index tomography and machine learning. Sci Rep 2017; 7:6654. [PMID: 28751719 PMCID: PMC5532204 DOI: 10.1038/s41598-017-06311-y] [Citation(s) in RCA: 64] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2017] [Accepted: 06/05/2017] [Indexed: 01/31/2023] Open
Abstract
Identification of lymphocyte cell types are crucial for understanding their pathophysiological roles in human diseases. Current methods for discriminating lymphocyte cell types primarily rely on labelling techniques with magnetic beads or fluorescence agents, which take time and have costs for sample preparation and may also have a potential risk of altering cellular functions. Here, we present the identification of non-activated lymphocyte cell types at the single-cell level using refractive index (RI) tomography and machine learning. From the measurements of three-dimensional RI maps of individual lymphocytes, the morphological and biochemical properties of the cells are quantitatively retrieved. To construct cell type classification models, various statistical classification algorithms are compared, and the k-NN (k = 4) algorithm was selected. The algorithm combines multiple quantitative characteristics of the lymphocyte to construct the cell type classifiers. After optimizing the feature sets via cross-validation, the trained classifiers enable identification of three lymphocyte cell types (B, CD4+ T, and CD8+ T cells) with high sensitivity and specificity. The present method, which combines RI tomography and machine learning for the first time to our knowledge, could be a versatile tool for investigating the pathophysiological roles of lymphocytes in various diseases including cancers, autoimmune diseases, and virus infections.
Collapse
Affiliation(s)
- Jonghee Yoon
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute Health Science and Technology, Daejeon, 34141, Republic of Korea
- Department of Physics, University of Cambridge, Cambridge, CB3 0HE, UK
| | - YoungJu Jo
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute Health Science and Technology, Daejeon, 34141, Republic of Korea
| | - Min-Hyeok Kim
- Department of Biological Sciences, KAIST, Daejeon, 34141, Republic of Korea
| | - Kyoohyun Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute Health Science and Technology, Daejeon, 34141, Republic of Korea
| | - SangYun Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute Health Science and Technology, Daejeon, 34141, Republic of Korea
| | - Suk-Jo Kang
- Department of Biological Sciences, KAIST, Daejeon, 34141, Republic of Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea.
- KAIST Institute Health Science and Technology, Daejeon, 34141, Republic of Korea.
- Tomocube, Inc., Daejeon, 34051, Republic of Korea.
| |
Collapse
|
22
|
Kolouri S, Park S, Thorpe M, Slepčev D, Rohde GK. Optimal Mass Transport: Signal processing and machine-learning applications. IEEE SIGNAL PROCESSING MAGAZINE 2017; 34:43-59. [PMID: 29962824 PMCID: PMC6024256 DOI: 10.1109/msp.2017.2695801] [Citation(s) in RCA: 57] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Transport-based techniques for signal and data analysis have received increased attention recently. Given their ability to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications including content-based retrieval, cancer detection, image super-resolution, and statistical machine learning, to name a few, and shown to produce state of the art results in several applications. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here we provide a practical overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications. Software accompanying this tutorial is available at [43].
Collapse
|
23
|
Zimmerman-Moreno G, Marin I, Lindner M, Barshack I, Garini Y, Konen E, Mayer A. Automatic classification of cancer cells in multispectral microscopic images of lymph node samples. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2016:3973-3976. [PMID: 28269155 DOI: 10.1109/embc.2016.7591597] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Histopathological analysis is crucial for the diagnosis of a large number of cancer types. A lot of progress has been made in the development of molecular based assays, but many of the cases still require the careful analysis of the stained tissue under a bright-field microscope and its analysis. This procedure is costly and time-consuming. We present a novel method for classification of cancer cells in lymph node images. It is based on the measurement of the spectral image of hematoxylin and eosin stained sample under the microscope and the analysis of the acquired data using state of the art machine learning techniques. The method is based on the analysis of the spectral information of the cells as well as their morphological properties. A large number of descriptors is extracted for each cell location, which are used to train a supervised classifier which discriminates between normal and cancer cells. We show that a reliable analysis can be made with detection rate (recall) of 81%-100% for the cancer class.
Collapse
|
24
|
|
25
|
Angel Arul Jothi J, Mary Anita Rajam V. Effective segmentation of orphan annie-eye nuclei from papillary thyroid carcinoma histopathology images using a probabilistic model and region-based active contour. Biomed Signal Process Control 2016. [DOI: 10.1016/j.bspc.2016.06.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
26
|
Macedo AA, Pessotti HC, Almansa LF, Felipe JC, Kimura ET. Morphometric information to reduce the semantic gap in the characterization of microscopic images of thyroid nodules. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2016; 130:162-174. [PMID: 27208531 DOI: 10.1016/j.cmpb.2016.03.017] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2015] [Revised: 02/05/2016] [Accepted: 03/15/2016] [Indexed: 06/05/2023]
Abstract
BACKGROUND The analyses of several systems for medical-imaging processing typically support the extraction of image attributes, but do not comprise some information that characterizes images. For example, morphometry can be applied to find new information about the visual content of an image. The extension of information may result in knowledge. Subsequently, results of mappings can be applied to recognize exam patterns, thus improving the accuracy of image retrieval and allowing a better interpretation of exam results. Although successfully applied in breast lesion images, the morphometric approach is still poorly explored in thyroid lesions due to the high subjectivity thyroid examinations. OBJECTIVE This paper presents a theoretical-practical study, considering Computer Aided Diagnosis (CAD) and Morphometry, to reduce the semantic discontinuity between medical image features and human interpretation of image content. METHOD The proposed method aggregates the content of microscopic images characterized by morphometric information and other image attributes extracted by traditional object extraction algorithms. This method carries out segmentation, feature extraction, image labeling and classification. Morphometric analysis was included as an object extraction method in order to verify the improvement of its accuracy for automatic classification of microscopic images. RESULTS To validate this proposal and verify the utility of morphometric information to characterize thyroid images, a CAD system was created to classify real thyroid image-exams into Papillary Cancer, Goiter and Non-Cancer. Results showed that morphometric information can improve the accuracy and precision of image retrieval and the interpretation of results in computer-aided diagnosis. For example, in the scenario where all the extractors are combined with the morphometric information, the CAD system had its best performance (70% of precision in Papillary cases). CONCLUSION Results signalized a positive use of morphometric information from images to reduce semantic discontinuity between human interpretation and image characterization.
Collapse
Affiliation(s)
- Alessandra A Macedo
- University of São Paulo, Faculty of Philosophy, Sciences and Letters of Ribeirão Preto, Computer Science and Mathematics Department, Biomedical Informatics Group, Ribeirão Preto - SP, Brazil.
| | - Hugo C Pessotti
- University of São Paulo, Faculty of Philosophy, Sciences and Letters of Ribeirão Preto, Computer Science and Mathematics Department, Bioinformatics Graduate Program, Ribeirão Preto - SP, Brazil.
| | - Luciana F Almansa
- University of São Paulo, Faculty of Philosophy, Sciences and Letters of Ribeirão Preto, Computer Science and Mathematics Department, Bioinformatics Graduate Program, Ribeirão Preto - SP, Brazil.
| | - Joaquim C Felipe
- University of São Paulo, Faculty of Philosophy, Sciences and Letters of Ribeirão Preto, Computer Science and Mathematics Department, Biomedical Informatics Group, Ribeirão Preto - SP, Brazil.
| | - Edna T Kimura
- University of São Paulo, Biomedical Sciences Institute, Department of Cell and Developmental Biology, São Paulo - SP, Brazil.
| |
Collapse
|
27
|
Kolouri S, Tosun AB, Ozolek JA, Rohde GK. A continuous linear optimal transport approach for pattern analysis in image datasets. PATTERN RECOGNITION 2016; 51:453-462. [PMID: 26858466 PMCID: PMC4742369 DOI: 10.1016/j.patcog.2015.09.019] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
We present a new approach to facilitate the application of the optimal transport metric to pattern recognition on image databases. The method is based on a linearized version of the optimal transport metric, which provides a linear embedding for the images. Hence, it enables shape and appearance modeling using linear geometric analysis techniques in the embedded space. In contrast to previous work, we use Monge's formulation of the optimal transport problem, which allows for reasonably fast computation of the linearized optimal transport embedding for large images. We demonstrate the application of the method to recover and visualize meaningful variations in a supervised-learning setting on several image datasets, including chromatin distribution in the nuclei of cells, galaxy morphologies, facial expressions, and bird species identification. We show that the new approach allows for high-resolution construction of modes of variations and discrimination and can enhance classification accuracy in a variety of image discrimination problems.
Collapse
Affiliation(s)
- Soheil Kolouri
- Biomedical Engineering Department, Carnegie Mellon University, C120 Hamerschlag Hall, Pittsburgh, PA 15213, USA
- Corresponding author. Tel.: +1 412 801 1063. (S. Kolouri), http://andrew.cmu.edu/user/skolouri (S. Kolouri)
| | - Akif B. Tosun
- Biomedical Engineering Department, Carnegie Mellon University, C120 Hamerschlag Hall, Pittsburgh, PA 15213, USA
| | - John A. Ozolek
- Biomedical Engineering Department, Carnegie Mellon University, C120 Hamerschlag Hall, Pittsburgh, PA 15213, USA
- Department of Pathology, Children's Hospital of Pittsburgh, Pittsburgh, PA 15224, USA
| | - Gustavo K. Rohde
- Biomedical Engineering Department, Carnegie Mellon University, C120 Hamerschlag Hall, Pittsburgh, PA 15213, USA
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| |
Collapse
|
28
|
Pouliakis A, Karakitsou E, Margari N, Bountris P, Haritou M, Panayiotides J, Koutsouris D, Karakitsos P. Artificial Neural Networks as Decision Support Tools in Cytopathology: Past, Present, and Future. Biomed Eng Comput Biol 2016; 7:1-18. [PMID: 26917984 PMCID: PMC4760671 DOI: 10.4137/becb.s31601] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2015] [Revised: 01/17/2016] [Accepted: 01/19/2016] [Indexed: 02/05/2023] Open
Abstract
OBJECTIVE This study aims to analyze the role of artificial neural networks (ANNs) in cytopathology. More specifically, it aims to highlight the importance of employing ANNs in existing and future applications and in identifying unexplored or poorly explored research topics. STUDY DESIGN A systematic search was conducted in scientific databases for articles related to cytopathology and ANNs with respect to anatomical places of the human body where cytopathology is performed. For each anatomic system/organ, the major outcomes described in the scientific literature are presented and the most important aspects are highlighted. RESULTS The vast majority of ANN applications are related to cervical cytopathology, specifically for the ANN-based, semiautomated commercial diagnostic system PAPNET. For cervical cytopathology, there is a plethora of studies relevant to the diagnostic accuracy; in addition, there are also efforts evaluating cost-effectiveness and applications on primary, secondary, or hybrid screening. For the rest of the anatomical sites, such as the gastrointestinal system, thyroid gland, urinary tract, and breast, there are significantly less efforts relevant to the application of ANNs. Additionally, there are still anatomical systems for which ANNs have never been applied on their cytological material. CONCLUSIONS Cytopathology is an ideal discipline to apply ANNs. In general, diagnosis is performed by experts via the light microscope. However, this approach introduces subjectivity, because this is not a universal and objective measurement process. This has resulted in the existence of a gray zone between normal and pathological cases. From the analysis of related articles, it is obvious that there is a need to perform more thorough analyses, using extensive number of cases and particularly for the nonexplored organs. Efforts to apply such systems within the laboratory test environment are required for their future uptake.
Collapse
Affiliation(s)
- Abraham Pouliakis
- Department of Cytopathology, National and Kapodistrian University of Athens, Medical School, Attikon University Hospital, Athens, Greece
| | - Efrossyni Karakitsou
- 2nd Department of Pathology, National and Kapodistrian University of Athens, Medical School, Attikon University Hospital, Athens, Greece
| | - Niki Margari
- Department of Cytopathology, National and Kapodistrian University of Athens, Medical School, Attikon University Hospital, Athens, Greece
| | - Panagiotis Bountris
- Biomedical Engineering Laboratory, National Technical University of Athens, Athens, Greece
| | - Maria Haritou
- Institute of Communication and Computer Systems, Athens, Greece
| | - John Panayiotides
- 2nd Department of Pathology, National and Kapodistrian University of Athens, Medical School, Attikon University Hospital, Athens, Greece
| | - Dimitrios Koutsouris
- Biomedical Engineering Laboratory, National Technical University of Athens, Athens, Greece
| | - Petros Karakitsos
- Department of Cytopathology, National and Kapodistrian University of Athens, Medical School, Attikon University Hospital, Athens, Greece
| |
Collapse
|
29
|
Kolouri S, Park SR, Rohde GK. The Radon Cumulative Distribution Transform and Its Application to Image Classification. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2016; 25:920-934. [PMID: 26685245 PMCID: PMC4871726 DOI: 10.1109/tip.2015.2509419] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
Invertible image representation methods (transforms) are routinely employed as low-level image processing operations based on which feature extraction and recognition algorithms are developed. Most transforms in current use (e.g., Fourier, wavelet, and so on) are linear transforms and, by themselves, are unable to substantially simplify the representation of image classes for classification. Here, we describe a nonlinear, invertible, low-level image processing transform based on combining the well-known Radon transform for image data, and the 1D cumulative distribution transform proposed earlier. We describe a few of the properties of this new transform, and with both theoretical and experimental results show that it can often render certain problems linearly separable in a transform space.
Collapse
Affiliation(s)
- Soheil Kolouri
- Department of Biomedical Engineering, Carnegie Mellon
University, Pittsburgh, PA, 15213
| | - Se Rim Park
- Department of Electrical and Computer Engineering, Carnegie
Mellon University, Pittsburgh, PA, 15213
| | - Gustavo K. Rohde
- Department of Biomedical Engineering, Carnegie Mellon
University, Pittsburgh, PA, 15213
- Department of Electrical and Computer Engineering, Carnegie
Mellon University, Pittsburgh, PA, 15213
- Lane Center for Computational Biology, Carnegie Mellon
University, Pittsburgh, PA, 15213
| |
Collapse
|
30
|
Xing F, Yang L. Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review. IEEE Rev Biomed Eng 2016; 9:234-63. [PMID: 26742143 PMCID: PMC5233461 DOI: 10.1109/rbme.2016.2515127] [Citation(s) in RCA: 213] [Impact Index Per Article: 26.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.
Collapse
|
31
|
Barker J, Hoogi A, Depeursinge A, Rubin DL. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal 2015; 30:60-71. [PMID: 26854941 DOI: 10.1016/j.media.2015.12.002] [Citation(s) in RCA: 107] [Impact Index Per Article: 11.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2015] [Revised: 12/02/2015] [Accepted: 12/07/2015] [Indexed: 02/07/2023]
Abstract
Computerized analysis of digital pathology images offers the potential of improving clinical care (e.g. automated diagnosis) and catalyzing research (e.g. discovering disease subtypes). There are two key challenges thwarting computerized analysis of digital pathology images: first, whole slide pathology images are massive, making computerized analysis inefficient, and second, diverse tissue regions in whole slide images that are not directly relevant to the disease may mislead computerized diagnosis algorithms. We propose a method to overcome both of these challenges that utilizes a coarse-to-fine analysis of the localized characteristics in pathology images. An initial surveying stage analyzes the diversity of coarse regions in the whole slide image. This includes extraction of spatially localized features of shape, color and texture from tiled regions covering the slide. Dimensionality reduction of the features assesses the image diversity in the tiled regions and clustering creates representative groups. A second stage provides a detailed analysis of a single representative tile from each group. An Elastic Net classifier produces a diagnostic decision value for each representative tile. A weighted voting scheme aggregates the decision values from these tiles to obtain a diagnosis at the whole slide level. We evaluated our method by automatically classifying 302 brain cancer cases into two possible diagnoses (glioblastoma multiforme (N = 182) versus lower grade glioma (N = 120)) with an accuracy of 93.1% (p << 0.001). We also evaluated our method in the dataset provided for the 2014 MICCAI Pathology Classification Challenge, in which our method, trained and tested using 5-fold cross validation, produced a classification accuracy of 100% (p << 0.001). Our method showed high stability and robustness to parameter variation, with accuracy varying between 95.5% and 100% when evaluated for a wide range of parameters. Our approach may be useful to automatically differentiate between the two cancer subtypes.
Collapse
Affiliation(s)
- Jocelyn Barker
- Department of Medicine (Stanford Biomedical Informatics Research), Stanford University School of Medicine, CA, USA.
| | - Assaf Hoogi
- Department of Radiology, Stanford University School of Medicine, CA, USA.
| | - Adrien Depeursinge
- Department of Radiology, Stanford University School of Medicine, CA, USA; Institute of Information Systems, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland.
| | - Daniel L Rubin
- Department of Radiology, Stanford University School of Medicine, CA, USA; Department of Medicine (Stanford Biomedical Informatics Research), Stanford University School of Medicine, CA, USA.
| |
Collapse
|
32
|
Zhang X, Xing F, Su H, Yang L, Zhang S. High-throughput histopathological image analysis via robust cell segmentation and hashing. Med Image Anal 2015; 26:306-15. [PMID: 26599156 PMCID: PMC4679540 DOI: 10.1016/j.media.2015.10.005] [Citation(s) in RCA: 66] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2014] [Revised: 05/13/2015] [Accepted: 10/16/2015] [Indexed: 11/27/2022]
Abstract
Computer-aided diagnosis of histopathological images usually requires to examine all cells for accurate diagnosis. Traditional computational methods may have efficiency issues when performing cell-level analysis. In this paper, we propose a robust and scalable solution to enable such analysis in a real-time fashion. Specifically, a robust segmentation method is developed to delineate cells accurately using Gaussian-based hierarchical voting and repulsive balloon model. A large-scale image retrieval approach is also designed to examine and classify each cell of a testing image by comparing it with a massive database, e.g., half-million cells extracted from the training dataset. We evaluate this proposed framework on a challenging and important clinical use case, i.e., differentiation of two types of lung cancers (the adenocarcinoma and squamous carcinoma), using thousands of lung microscopic tissue images extracted from hundreds of patients. Our method has achieved promising accuracy and running time by searching among half-million cells .
Collapse
Affiliation(s)
- Xiaofan Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
| | - Fuyong Xing
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Hai Su
- Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Lin Yang
- Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA; Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA
| | - Shaoting Zhang
- Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC 28223, USA.
| |
Collapse
|
33
|
Rohde GK, Ozolek JA, Parwani AV, Pantanowitz L. Carnegie Mellon University bioimaging day 2014: Challenges and opportunities in digital pathology. J Pathol Inform 2014; 5:32. [PMID: 25250190 PMCID: PMC4168545 DOI: 10.4103/2153-3539.139712] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2014] [Accepted: 06/24/2014] [Indexed: 01/16/2023] Open
Abstract
Recent advances in digital imaging is impacting the practice of pathology. One of the key enabling technologies that is leading the way towards this transformation is the use of whole slide imaging (WSI) which allows glass slides to be converted into large image files that can be shared, stored, and analyzed rapidly. Many applications around this novel technology have evolved in the last decade including education, research and clinical applications. This publication highlights a collection of abstracts, each corresponding to a talk given at Carnegie Mellon University's (CMU) Bioimaging Day 2014 co-sponsored by the Biomedical Engineering and Lane Center for Computational Biology Departments at CMU. Topics related specifically to digital pathology are presented in this collection of abstracts. These include topics related to digital workflow implementation, imaging and artifacts, storage demands, and automated image analysis algorithms.
Collapse
Affiliation(s)
- Gustavo K Rohde
- Department of Biomedical Engineering, Lane Center for Computational Biology, Carnegie Mellon University, Pittsburgh, PA, USA
| | - John A Ozolek
- Department of Pathology, Children's Hospital of Pittsburgh University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | - Anil V Parwani
- Department of Pathology, Division of Pathology Informatics, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Liron Pantanowitz
- Department of Pathology, Division of Pathology Informatics, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| |
Collapse
|