1
|
Shifat-E-Rabbi M, Pathan NS, Li S, Zhuang Y, Rubaiyat AHM, Rohde GK. Linear optimal transport subspaces for point set classification. Res Sq 2024:rs.3.rs-4106387. [PMID: 38562684 PMCID: PMC10984092 DOI: 10.21203/rs.3.rs-4106387/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Learning from point sets is an essential component in many computer vision and machine learning applications. Native, unordered, and permutation invariant set structure space is challenging to model, particularly for point set classification under spatial deformations. Here we propose a framework for classifying point sets experiencing certain types of spatial deformations, with a particular emphasis on datasets featuring affine deformations. Our approach employs the Linear Optimal Transport (LOT) transform to obtain a linear embedding of set-structured data. Utilizing the mathematical properties of the LOT transform, we demonstrate its capacity to accommodate variations in point sets by constructing a convex data space, effectively simplifying point set classification problems. Our method, which employs a nearest-subspace algorithm in the LOT space, demonstrates label efficiency, non-iterative behavior, and requires no hyper-parameter tuning. It achieves competitive accuracies compared to state-of-the-art methods across various point set classification tasks. Furthermore, our approach exhibits robustness in out-of-distribution scenarios where training and test distributions vary in terms of deformation magnitudes.
Collapse
Affiliation(s)
- Mohammad Shifat-E-Rabbi
- Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh
| | - Naqib Sad Pathan
- Imaging and Data Science Laboratory and the Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA, USA
| | - Shiying Li
- Department of Mathematics, University of North Carolina - Chapel Hill, NC, USA
| | - Yan Zhuang
- Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center, MD, USA
| | - Abu Hasnat Mohammad Rubaiyat
- Imaging and Data Science Laboratory and the Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA, USA
| | - Gustavo K. Rohde
- Imaging and Data Science Laboratory, the Department of Biomedical Engineering, and the Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA, USA
| |
Collapse
|
2
|
Rubaiyat AHM, Li S, Yin X, Shifat-E-Rabbi M, Zhuang Y, Rohde GK. End-to-End Signal Classification in Signed Cumulative Distribution Transform Space. IEEE Trans Pattern Anal Mach Intell 2024; PP:1-14. [PMID: 38427542 DOI: 10.1109/tpami.2024.3372455] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/03/2024]
Abstract
This paper presents a new end-to-end signal classification method using the signed cumulative distribution transform (SCDT). We adopt a transport generative model to define the classification problem. We then make use of mathematical properties of the SCDT to render the problem easier in transform domain, and solve for the class of an unknown sample using a nearest local subspace (NLS) search algorithm in SCDT domain. Experiments show that the proposed method provides high accuracy classification results while being computationally cheap, data efficient, and robust to out-of-distribution samples with respect to the existing end-to-end classification methods. The implementation of the proposed method in Python language is integrated as a part of the software package PyTransKit.
Collapse
|
3
|
Shifat-E-Rabbi M, Zhuang Y, Li S, Rubaiyat AHM, Yin X, Rohde GK. Invariance encoding in sliced-Wasserstein space for image classification with limited training data. Pattern Recognit 2023; 137:109268. [PMID: 36713887 PMCID: PMC9879373 DOI: 10.1016/j.patcog.2022.109268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Deep convolutional neural networks (CNNs) are broadly considered to be state-of-the-art generic end-to-end image classification systems. However, they are known to underperform when training data are limited and thus require data augmentation strategies that render the method computationally expensive and not always effective. Rather than using a data augmentation strategy to encode invariances as typically done in machine learning, here we propose to mathematically augment a nearest subspace classification model in sliced-Wasserstein space by exploiting certain mathematical properties of the Radon Cumulative Distribution Transform (R-CDT), a recently introduced image transform. We demonstrate that for a particular type of learning problem, our mathematical solution has advantages over data augmentation with deep CNNs in terms of classification accuracy and computational complexity, and is particularly effective under a limited training data setting. The method is simple, effective, computationally efficient, non-iterative, and requires no parameters to be tuned. Python code implementing our method is available at https://github.com/rohdelab/mathematical augmentation. Our method is integrated as a part of the software package PyTransKit, which is available at https://github.com/rohdelab/PyTransKit.
Collapse
Affiliation(s)
- Mohammad Shifat-E-Rabbi
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Yan Zhuang
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Shiying Li
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Abu Hasnat Mohammad Rubaiyat
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Xuwang Yin
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Gustavo K. Rohde
- Imaging and Data Science Laboratory, University of Virginia, Charlottesville, VA 22908, USA
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22908, USA
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22908, USA
| |
Collapse
|
4
|
Zhang C, Herbig M, Zhou Y, Nishikawa M, Shifat-E-Rabbi M, Kanno H, Yang R, Ibayashi Y, Xiao TH, Rohde GK, Sato M, Kodera S, Daimon M, Yatomi Y, Goda K. Real-time intelligent classification of COVID-19 and thrombosis via massive image-based analysis of platelet aggregates. Cytometry A 2023. [PMID: 36772915 DOI: 10.1002/cyto.a.24721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Revised: 11/03/2022] [Accepted: 02/03/2023] [Indexed: 02/12/2023]
Abstract
Microvascular thrombosis is a typical symptom of COVID-19 and shows similarities to thrombosis. Using a microfluidic imaging flow cytometer, we measured the blood of 181 COVID-19 samples and 101 non-COVID-19 thrombosis samples, resulting in a total of 6.3 million bright-field images. We trained a convolutional neural network to distinguish single platelets, platelet aggregates, and white blood cells and performed classical image analysis for each subpopulation individually. Based on derived single-cell features for each population, we trained machine learning models for classification between COVID-19 and non-COVID-19 thrombosis, resulting in a patient testing accuracy of 75%. This result indicates that platelet formation differs between COVID-19 and non-COVID-19 thrombosis. All analysis steps were optimized for efficiency and implemented in an easy-to-use plugin for the image viewer napari, allowing the entire analysis to be performed within seconds on mid-range computers, which could be used for real-time diagnosis.
Collapse
Affiliation(s)
- Chenqi Zhang
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Maik Herbig
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Yuqi Zhou
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Masako Nishikawa
- Department of Clinical Laboratory, University of Tokyo Hospital, Tokyo, Japan
| | - Mohammad Shifat-E-Rabbi
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Hiroshi Kanno
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Ruoxi Yang
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Yuma Ibayashi
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Ting-Hui Xiao
- Department of Chemistry, The University of Tokyo, Tokyo, Japan
| | - Gustavo K Rohde
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA.,Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Masataka Sato
- Department of Cardiovascular Medicine, The University of Tokyo Hospital, Tokyo, Japan
| | - Satoshi Kodera
- Department of Cardiovascular Medicine, The University of Tokyo Hospital, Tokyo, Japan
| | - Masao Daimon
- Department of Clinical Laboratory, University of Tokyo Hospital, Tokyo, Japan
| | - Yutaka Yatomi
- Department of Clinical Laboratory, University of Tokyo Hospital, Tokyo, Japan
| | - Keisuke Goda
- Department of Chemistry, The University of Tokyo, Tokyo, Japan.,Department of Bioengineering, University of California, Los Angeles, California, USA.,CYBO, Tokyo, Japan.,Institute of Technological Sciences, Wuhan University, Hubei, China
| |
Collapse
|
5
|
Nishikawa M, Kanno H, Zhou Y, Xiao TH, Suzuki T, Ibayashi Y, Harmon J, Takizawa S, Hiramatsu K, Nitta N, Kameyama R, Peterson W, Takiguchi J, Shifat-E-Rabbi M, Zhuang Y, Yin X, Rubaiyat AHM, Deng Y, Zhang H, Miyata S, Rohde GK, Iwasaki W, Yatomi Y, Goda K. Massive image-based single-cell profiling reveals high levels of circulating platelet aggregates in patients with COVID-19. Nat Commun 2021; 12:7135. [PMID: 34887400 PMCID: PMC8660840 DOI: 10.1038/s41467-021-27378-2] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Accepted: 11/16/2021] [Indexed: 12/19/2022] Open
Abstract
A characteristic clinical feature of COVID-19 is the frequent incidence of microvascular thrombosis. In fact, COVID-19 autopsy reports have shown widespread thrombotic microangiopathy characterized by extensive diffuse microthrombi within peripheral capillaries and arterioles in lungs, hearts, and other organs, resulting in multiorgan failure. However, the underlying process of COVID-19-associated microvascular thrombosis remains elusive due to the lack of tools to statistically examine platelet aggregation (i.e., the initiation of microthrombus formation) in detail. Here we report the landscape of circulating platelet aggregates in COVID-19 obtained by massive single-cell image-based profiling and temporal monitoring of the blood of COVID-19 patients (n = 110). Surprisingly, our analysis of the big image data shows the anomalous presence of excessive platelet aggregates in nearly 90% of all COVID-19 patients. Furthermore, results indicate strong links between the concentration of platelet aggregates and the severity, mortality, respiratory condition, and vascular endothelial dysfunction level of COVID-19 patients.
Collapse
Affiliation(s)
- Masako Nishikawa
- Department of Clinical Laboratory Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Hiroshi Kanno
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Yuqi Zhou
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Ting-Hui Xiao
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan.
| | - Takuma Suzuki
- Department of Computational Biology and Medical Sciences, The University of Tokyo, Chiba, 277-8562, Japan
| | - Yuma Ibayashi
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Jeffrey Harmon
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Shigekazu Takizawa
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Kotaro Hiramatsu
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
- Research Center for Spectrochemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | | | - Risako Kameyama
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Walker Peterson
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Jun Takiguchi
- Department of Clinical Laboratory Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan
| | | | - Yan Zhuang
- Department of Electrical and Computer Engineering, University of Virginia, Virginia, 22908, USA
| | - Xuwang Yin
- Department of Electrical and Computer Engineering, University of Virginia, Virginia, 22908, USA
| | | | - Yunjie Deng
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Hongqian Zhang
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Shigeki Miyata
- Research and Development Department, Central Blood Institute, Japanese Red Cross Society, Tokyo, 135-8521, Japan
| | - Gustavo K Rohde
- Department of Biomedical Engineering, University of Virginia, Virginia, 22908, USA
- Department of Electrical and Computer Engineering, University of Virginia, Virginia, 22908, USA
| | - Wataru Iwasaki
- Department of Computational Biology and Medical Sciences, The University of Tokyo, Chiba, 277-8562, Japan
- Department of Biological Sciences, The University of Tokyo, Tokyo, 113-0033, Japan
- Department of Integrated Biosciences, The University of Tokyo, Chiba, 277-8562, Japan
| | - Yutaka Yatomi
- Department of Clinical Laboratory Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
| | - Keisuke Goda
- Department of Chemistry, The University of Tokyo, Tokyo, 113-0033, Japan.
- Institute of Technological Sciences, Wuhan University, 430072, Hubei, China.
- Department of Bioengineering, University of California, Los Angeles, California, 90095, USA.
| |
Collapse
|
6
|
Shifat-E-Rabbi M, Yin X, Rubaiyat AHM, Li S, Kolouri S, Aldroubi A, Nichols JM, Rohde GK. Radon Cumulative Distribution Transform Subspace Modeling for Image Classification. J Math Imaging Vis 2021; 63:1185-1203. [PMID: 35464640 PMCID: PMC9032314 DOI: 10.1007/s10851-021-01052-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Accepted: 07/16/2021] [Indexed: 06/14/2023]
Abstract
We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method - utilizing a nearest-subspace algorithm in the R-CDT space - is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at [1].
Collapse
Affiliation(s)
| | | | | | - Shiying Li
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22908, USA
| | - Soheil Kolouri
- Department of Computer Science, Vanderbilt University, Nashville, TN 37212, USA
| | - Akram Aldroubi
- Department of Mathematics, Vanderbilt University, Nashville, TN 37212, USA
| | | | - Gustavo K. Rohde
- Department of Biomedical Engineering and the Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22908, USA
| |
Collapse
|
7
|
Kundu S, Ashinsky BG, Bouhrara M, Dam EB, Demehri S, Shifat-E-Rabbi M, Spencer RG, Urish KL, Rohde GK. Reply to Roemer and Guermazi: Early biochemical changes on MRI can predict risk of symptomatic progression. Proc Natl Acad Sci U S A 2021; 118:e2024679118. [PMID: 33836609 PMCID: PMC7980416 DOI: 10.1073/pnas.2024679118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Affiliation(s)
- Shinjini Kundu
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213;
- Medical Scientist Training Program, University of Pittsburgh, Pittsburgh, PA 15213
| | - Beth G Ashinsky
- Laboratory of Clinical Investigation, Magnetic Resonance Imaging and Spectroscopy Section, National Institute on Aging, National Institutes of Health, Baltimore, MD 21224
| | - Mustapha Bouhrara
- Laboratory of Clinical Investigation, Magnetic Resonance Imaging and Spectroscopy Section, National Institute on Aging, National Institutes of Health, Baltimore, MD 21224
| | - Erik B Dam
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Shadpour Demehri
- Department of Radiology, The Johns Hopkins Hospital, Baltimore, MD 21287
| | - M Shifat-E-Rabbi
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22904
| | - Richard G Spencer
- Laboratory of Clinical Investigation, Magnetic Resonance Imaging and Spectroscopy Section, National Institute on Aging, National Institutes of Health, Baltimore, MD 21224
| | - Kenneth L Urish
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213
- Arthritis and Arthroplasty Design Group, The Bone and Joint Center, Magee Womens Hospital of the University of Pittsburgh Medical Center, Pittsburgh, PA 15213
- Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA 15213
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15261
- Clinical and Translational Science Institute, University of Pittsburgh, Pittsburgh, PA 15261
| | - Gustavo K Rohde
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA 22904
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904
| |
Collapse
|
8
|
Shifat-E-Rabbi M, Yin X, Fitzgerald CE, Rohde GK. Cell Image Classification: A Comparative Overview. Cytometry A 2020; 97:347-362. [PMID: 32040260 DOI: 10.1002/cyto.a.23984] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Revised: 10/18/2019] [Accepted: 01/18/2020] [Indexed: 12/13/2022]
Abstract
Cell image classification methods are currently being used in numerous applications in cell biology and medicine. Applications include understanding the effects of genes and drugs in screening experiments, understanding the role and subcellular localization of different proteins, as well as diagnosis and prognosis of cancer from images acquired using cytological and histological techniques. The article also reviews three main approaches for cell image classification most often used: numerical feature extraction, end-to-end classification with neural networks (NNs), and transport-based morphometry (TBM). In addition, we provide comparisons on four different cell imaging datasets to highlight the relative strength of each method. The results computed using four publicly available datasets show that numerical features tend to carry the best discriminative information for most of the classification tasks. Results also show that NN-based methods produce state-of-the-art results in the dataset that contains a relatively large number of training samples. Data augmentation or the choice of a more recently reported architecture does not necessarily improve the classification performance of NNs in the datasets with limited number of training samples. If understanding and visualization are desired aspects, TBM methods can offer the ability to invert classification functions, and thus can aid in the interpretation of results. These and other comparison outcomes are discussed with the aim of clarifying the advantages and disadvantages of each method. © 2020 International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Mohammad Shifat-E-Rabbi
- Imaging and Data Science Lab, Charlottesville, Virginia, 22903
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, 22903
| | - Xuwang Yin
- Imaging and Data Science Lab, Charlottesville, Virginia, 22903
- Department of Electrical & Computer Engineering, University of Virginia, Charlottesville, Virginia, 22903
| | - Cailey E Fitzgerald
- Imaging and Data Science Lab, Charlottesville, Virginia, 22903
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, 22903
| | - Gustavo K Rohde
- Imaging and Data Science Lab, Charlottesville, Virginia, 22903
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, 22903
- Department of Electrical & Computer Engineering, University of Virginia, Charlottesville, Virginia, 22903
| |
Collapse
|