1
|
Long J, Zhang Q, Lu X, Wen J, Zhao L, Xie W. Multi-scale locality preserving projection for partial multi-view incomplete multi-label learning. Neural Netw 2024; 180:106748. [PMID: 39332211 DOI: 10.1016/j.neunet.2024.106748] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2024] [Revised: 08/25/2024] [Accepted: 09/17/2024] [Indexed: 09/29/2024]
Abstract
Amidst advancements in feature extraction techniques, research on multi-view multi-label classifications has attracted widespread interest in recent years. However, real-world scenarios often pose a challenge where the completeness of multiple views and labels cannot be ensured. At present, only a handful of techniques have attempted to address the complex issue of partial multi-view incomplete multi-label classification, and the majority of these approaches overlook the significance of manifold structures between instances. To tackle these challenges, we propose a novel partial multi-view incomplete multi-label learning model, termed MSLPP. Differing from existing studies, MSLPP emphasizes retaining the effective inherent structure of data during the feature extraction process, thereby facilitating a richer semantic information extraction. Specifically, MSLPP captures and integrates four types of information: the distance and similarity information in the original feature space, and the distance and similarity information in the extracted feature space. Further, by adopting the graph embedding technique, it simultaneously preserves the intrinsic structure with multi-scale information through a constraint term. Moreover, taking into account the negative impact of the missing views on the model and the possible impact of missing views on the data inherent structure, we further propose a shielding strategy for missing views, which not only eliminates the negative effects of missing views on the model but also more accurately captures the inherent data structure. The experimental results on five widely recognized datasets indicate that the model performs better than many excellent methods.
Collapse
Affiliation(s)
- Jiang Long
- College of Big Data and Information Engineering, Guizhou University, Guiyang, China
| | - Qi Zhang
- Faculty of Data Science, City University of Macau, Macao Special Administrative Region of China
| | - Xiaohuan Lu
- College of Big Data and Information Engineering, Guizhou University, Guiyang, China.
| | - Jie Wen
- Shenzhen Key Laboratory of Visual Object Detection and Recognition , Harbin Institute of Technology, Shenzhen, 518055, China
| | - Lian Zhao
- College of Big Data and Information Engineering, Guizhou University, Guiyang, China
| | - Wulin Xie
- College of Big Data and Information Engineering, Guizhou University, Guiyang, China
| |
Collapse
|
2
|
Cui J, Xie Y, Liu C, Huang Q, Li M, Wen J. Deep dual incomplete multi-view multi-label classification via label semantic-guided contrastive learning. Neural Netw 2024; 180:106674. [PMID: 39236408 DOI: 10.1016/j.neunet.2024.106674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2024] [Revised: 08/14/2024] [Accepted: 08/28/2024] [Indexed: 09/07/2024]
Abstract
Multi-view multi-label learning (MVML) aims to train a model that can explore the multi-view information of the input sample to obtain its accurate predictions of multiple labels. Unfortunately, a majority of existing MVML methods are based on the assumption of data completeness, making them useless in practical applications with partially missing views or some uncertain labels. Recently, many approaches have been proposed for incomplete data, but few of them can handle the case of both missing views and labels. Moreover, these few existing works commonly ignore potentially valuable information about unknown labels or do not sufficiently explore latent label information. Therefore, in this paper, we propose a label semantic-guided contrastive learning method named LSGC for the dual incomplete multi-view multi-label classification problem. Concretely, LSGC employs deep neural networks to extract high-level features of samples. Inspired by the observation of exploiting label correlations to improve the feature discriminability, we introduce a graph convolutional network to effectively capture label semantics. Furthermore, we introduce a new sample-label contrastive loss to explore the label semantic information and enhance the feature representation learning. For missing labels, we adopt a pseudo-label filling strategy and develop a weighting mechanism to explore the confidently recovered label information. We validate the framework on five standard datasets and the experimental results show that our method achieves superior performance in comparison with the state-of-the-art methods.
Collapse
Affiliation(s)
- Jinrong Cui
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, China
| | - Yazi Xie
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, China
| | - Chengliang Liu
- Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China
| | - Qiong Huang
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, China; Guangzhou Key Laboratory of Intelligent Agriculture, Guangzhou, 510642, China.
| | - Mu Li
- Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China.
| | - Jie Wen
- Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China.
| |
Collapse
|
3
|
Li J, Liao L, Jia M, Chen Z, Liu X. Latent relation shared learning for endometrial cancer diagnosis with incomplete multi-modality medical images. iScience 2024; 27:110509. [PMID: 39161958 PMCID: PMC11332793 DOI: 10.1016/j.isci.2024.110509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2024] [Revised: 05/22/2024] [Accepted: 07/11/2024] [Indexed: 08/21/2024] Open
Abstract
Magnetic resonance imaging (MRI), ultrasound (US), and contrast-enhanced ultrasound (CEUS) can provide different image data about uterus, which have been used in the preoperative assessment of endometrial cancer. In practice, not all the patients have complete multi-modality medical images due to the high cost or long examination period. Most of the existing methods need to perform data cleansing or discard samples with missing modalities, which will influence the performance of the model. In this work, we propose an incomplete multi-modality images data fusion method based on latent relation shared to overcome this limitation. The shared space contains the common latent feature representation and modality-specific latent feature representation from the complete and incomplete multi-modality data, which jointly exploits both consistent and complementary information among multiple images. The experimental results show that our method outperforms the current representative approaches in terms of classification accuracy, sensitivity, specificity, and area under curve (AUC). Furthermore, our method performs well under varying imaging missing rates.
Collapse
Affiliation(s)
- Jiaqi Li
- School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
- Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing 100081, China
| | - Lejian Liao
- School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
- Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing 100081, China
| | - Meihuizi Jia
- School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
- Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing 100081, China
| | - Zhendong Chen
- School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
- Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing 100081, China
| | - Xin Liu
- Department of Ultrasound, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| |
Collapse
|
4
|
Wen J, Liu C, Deng S, Liu Y, Fei L, Yan K, Xu Y. Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:11396-11408. [PMID: 37030862 DOI: 10.1109/tnnls.2023.3260349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
View missing and label missing are two challenging problems in the applications of multi-view multi-label classification scenery. In the past years, many efforts have been made to address the incomplete multi-view learning or incomplete multi-label learning problem. However, few works can simultaneously handle the challenging case with both the incomplete issues. In this article, we propose a new incomplete multi-view multi-label learning network to address this challenging issue. The proposed method is composed of four major parts: view-specific deep feature extraction network, weighted representation fusion module, classification module, and view-specific deep decoder network. By, respectively, integrating the view missing information and label missing information into the weighted fusion module and classification module, the proposed method can effectively reduce the negative influence caused by two such incomplete issues and sufficiently explore the available data and label information to obtain the most discriminative feature extractor and classifier. Furthermore, our method can be trained in both supervised and semi-supervised manners, which has important implications for flexible deployment. Experimental results on five benchmarks in supervised and semi-supervised cases demonstrate that the proposed method can greatly enhance the classification performance on the difficult incomplete multi-view multi-label classification tasks with missing labels and missing views.
Collapse
|
5
|
Cheng C, Liu W, Fan Z, Feng L, Jia Z. A novel transformer autoencoder for multi-modal emotion recognition with incomplete data. Neural Netw 2024; 172:106111. [PMID: 38237444 DOI: 10.1016/j.neunet.2024.106111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2023] [Revised: 12/18/2023] [Accepted: 01/05/2024] [Indexed: 02/28/2024]
Abstract
Multi-modal signals have become essential data for emotion recognition since they can represent emotions more comprehensively. However, in real-world environments, it is often impossible to acquire complete data on multi-modal signals, and the problem of missing modalities causes severe performance degradation in emotion recognition. Therefore, this paper represents the first attempt to use a transformer-based architecture, aiming to fill the modality-incomplete data from partially observed data for multi-modal emotion recognition (MER). Concretely, this paper proposes a novel unified model called transformer autoencoder (TAE), comprising a modality-specific hybrid transformer encoder, an inter-modality transformer encoder, and a convolutional decoder. The modality-specific hybrid transformer encoder bridges a convolutional encoder and a transformer encoder, allowing the encoder to learn local and global context information within each particular modality. The inter-modality transformer encoder builds and aligns global cross-modal correlations and models long-range contextual information with different modalities. The convolutional decoder decodes the encoding features to produce more precise recognition. Besides, a regularization term is introduced into the convolutional decoder to force the decoder to fully leverage the complete and incomplete data for emotional recognition of missing data. 96.33%, 95.64%, and 92.69% accuracies are attained on the available data of the DEAP and SEED-IV datasets, and 93.25%, 92.23%, and 81.76% accuracies are obtained on the missing data. Particularly, the model acquires a 5.61% advantage with 70% missing data, demonstrating that the model outperforms some state-of-the-art approaches in incomplete multi-modal learning.
Collapse
Affiliation(s)
- Cheng Cheng
- Department of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Wenzhe Liu
- School of Information Engineering, Huzhou University, Huzhou, China
| | - Zhaoxin Fan
- Renmin University of China, Psyche AI Inc, Beijing, China
| | - Lin Feng
- Department of Computer Science and Technology, Dalian University of Technology, Dalian, China.
| | - Ziyu Jia
- Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
6
|
Liu X. Incomplete Multiple Kernel Alignment Maximization for Clustering. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:1412-1424. [PMID: 34596533 DOI: 10.1109/tpami.2021.3116948] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Multiple kernel alignment (MKA) maximization criterion has been widely applied into multiple kernel clustering (MKC) and many variants have been recently developed. Though demonstrating superior clustering performance in various applications, it is observed that none of them can effectively handle incomplete MKC, where parts or all of the pre-specified base kernel matrices are incomplete. To address this issue, we propose to integrate the imputation of incomplete kernel matrices and MKA maximization for clustering into a unified learning framework. The clustering of MKA maximization guides the imputation of incomplete kernel elements, and the completed kernel matrices are in turn combined to conduct the subsequent MKC. These two procedures are alternately performed until convergence. By this way, the imputation and MKC processes are seamlessly connected, with the aim to achieve better clustering performance. Besides theoretically analyzing the clustering generalization error bound, we empirically evaluate the clustering performance on several multiple kernel learning (MKL) benchmark datasets, and the results indicate the superiority of our algorithm over existing state-of-the-art counterparts. Our codes and data are publicly available at https://xinwangliu.github.io/.
Collapse
|
7
|
Du X, Jiang X, Lin J. Multinomial Logistic Factor Regression for Multi-source Functional Block-wise Missing Data. PSYCHOMETRIKA 2023; 88:975-1001. [PMID: 37268759 DOI: 10.1007/s11336-023-09918-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 03/23/2023] [Indexed: 06/04/2023]
Abstract
Multi-source functional block-wise missing data arise more commonly in medical care recently with the rapid development of big data and medical technology, hence there is an urgent need to develop efficient dimension reduction to extract important information for classification under such data. However, most existing methods for classification problems consider high-dimensional data as covariates. In the paper, we propose a novel multinomial imputed-factor Logistic regression model with multi-source functional block-wise missing data as covariates. Our main contribution is to establishing two multinomial factor regression models by using the imputed multi-source functional principal component scores and imputed canonical scores as covariates, respectively, where the missing factors are imputed by both the conditional mean imputation and the multiple block-wise imputation approaches. Specifically, the univariate FPCA is carried out for the observable data of each data source firstly to obtain the univariate principal component scores and the eigenfunctions. Then, the block-wise missing univariate principal component scores instead of the block-wise missing functional data are imputed by the conditional mean imputation method and the multiple block-wise imputation method, respectively. After that, based on the imputed univariate factors, the multi-source principal component scores are constructed by using the relationship between the multi-source principal component scores and the univariate principal component scores; and at the same time, the canonical scores are obtained by the multiple-set canonial correlation analysis. Finally, the multinomial imputed-factor Logistic regression model is established with the multi-source principal component scores or the canonical scores as factors. Numerical simulations and real data analysis on ADNI data show the proposed method works well.
Collapse
Affiliation(s)
- Xiuli Du
- College of Mathematical Sciences, Nanjing Normal University, Nanjing, 210023, China.
| | - Xiaohu Jiang
- College of Mathematical Sciences, Nanjing Normal University, Nanjing, 210023, China
| | - Jinguan Lin
- Institute of Statistics and Data Science, Nanjing Audit University, Nanjing, 211815, China
| |
Collapse
|
8
|
Gentry AE, Kirkpatrick RM, Peterson RE, Webb BT. Missingness adapted group informed clustered (MAGIC)-LASSO: a novel paradigm for phenotype prediction to improve power for genetic loci discovery. Front Genet 2023; 14:1162690. [PMID: 37547462 PMCID: PMC10399453 DOI: 10.3389/fgene.2023.1162690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 07/03/2023] [Indexed: 08/08/2023] Open
Abstract
Introduction: The availability of large-scale biobanks linking genetic data, rich phenotypes, and biological measures is a powerful opportunity for scientific discovery. However, real-world collections frequently have extensive missingness. While missing data prediction is possible, performance is significantly impaired by block-wise missingness inherent to many biobanks. Methods: To address this, we developed Missingness Adapted Group-wise Informed Clustered (MAGIC)-LASSO which performs hierarchical clustering of variables based on missingness followed by sequential Group LASSO within clusters. Variables are pre-filtered for missingness and balance between training and target sets with final models built using stepwise inclusion of features ranked by completeness. This research has been conducted using the UK Biobank (n > 500 k) to predict unmeasured Alcohol Use Disorders Identification Test (AUDIT) scores. Results: The phenotypic correlation between measured and predicted total score was 0.67 while genetic correlations between independent subjects was high >0.86. Discussion: Phenotypic and genetic correlations in real data application, as well as simulations, demonstrate the method has significant accuracy and utility for increasing power for genetic loci discovery.
Collapse
Affiliation(s)
- Amanda Elswick Gentry
- Department of Psychiatry, Virginia Institute for Psychiatric and Behavioral Genetics, Virginia Commonwealth University, Richmond, VA, United States
| | - Robert M. Kirkpatrick
- Department of Psychiatry, Virginia Institute for Psychiatric and Behavioral Genetics, Virginia Commonwealth University, Richmond, VA, United States
| | - Roseann E. Peterson
- Department of Psychiatry, Virginia Institute for Psychiatric and Behavioral Genetics, Virginia Commonwealth University, Richmond, VA, United States
- Department of Psychiatry and Behavioral Sciences, Institute for Genomics in Health, SUNY Downstate Health Sciences University, Brooklyn, NY, United States
| | - Bradley T. Webb
- Department of Psychiatry, Virginia Institute for Psychiatric and Behavioral Genetics, Virginia Commonwealth University, Richmond, VA, United States
- GenOmics and Translational Research Center, RTI International, Research Triangle Park, NC, United States
| |
Collapse
|
9
|
Kline A, Wang H, Li Y, Dennis S, Hutch M, Xu Z, Wang F, Cheng F, Luo Y. Multimodal machine learning in precision health: A scoping review. NPJ Digit Med 2022; 5:171. [PMID: 36344814 PMCID: PMC9640667 DOI: 10.1038/s41746-022-00712-8] [Citation(s) in RCA: 74] [Impact Index Per Article: 37.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 10/14/2022] [Indexed: 11/09/2022] Open
Abstract
Machine learning is frequently being leveraged to tackle problems in the health sector including utilization for clinical decision-support. Its use has historically been focused on single modal data. Attempts to improve prediction and mimic the multimodal nature of clinical expert decision-making has been met in the biomedical field of machine learning by fusing disparate data. This review was conducted to summarize the current studies in this field and identify topics ripe for future research. We conducted this review in accordance with the PRISMA extension for Scoping Reviews to characterize multi-modal data fusion in health. Search strings were established and used in databases: PubMed, Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 128 articles were included in the analysis. The most common health areas utilizing multi-modal methods were neurology and oncology. Early fusion was the most common data merging strategy. Notably, there was an improvement in predictive performance when using data fusion. Lacking from the papers were clear clinical deployment strategies, FDA-approval, and analysis of how using multimodal approaches from diverse sub-populations may improve biases and healthcare disparities. These findings provide a summary on multimodal data fusion as applied to health diagnosis/prognosis problems. Few papers compared the outputs of a multimodal approach with a unimodal prediction. However, those that did achieved an average increase of 6.4% in predictive accuracy. Multi-modal machine learning, while more robust in its estimations over unimodal methods, has drawbacks in its scalability and the time-consuming nature of information concatenation.
Collapse
Affiliation(s)
- Adrienne Kline
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Hanyin Wang
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Yikuan Li
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Saya Dennis
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Meghan Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Zhenxing Xu
- Department of Population Health Sciences, Cornell University, New York, 10065, NY, USA
| | - Fei Wang
- Department of Population Health Sciences, Cornell University, New York, 10065, NY, USA
| | - Feixiong Cheng
- Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, 44195, OH, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA.
| |
Collapse
|
10
|
Jeong SW, Cho HH, Lee S, Park H. Robust multimodal fusion network using adversarial learning for brain tumor grading. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 226:107165. [PMID: 36215857 DOI: 10.1016/j.cmpb.2022.107165] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 09/26/2022] [Accepted: 09/27/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVE Gliomas are graded using multimodal magnetic resonance imaging, which provides important information for treatment and prognosis. When modalities are missing, the grading is degraded. We propose a robust brain tumor grading model that can handle missing modalities. METHODS Our method was developed and tested on Brain Tumor Segmentation Challenge 2017 dataset (n = 285) via nested five-fold cross-validation. Our method adopts adversarial learning to generate the features of missing modalities relative to the features obtained from a full set of modalities in the latent space. An attention-based fusion block across modalities fuses the features of each available modality into a shared representation. Our method's results are compared to those of two other models where 15 missing-modality scenarios are explicitly considered and a joint training approach with random dropouts is used. RESULTS Our method outperforms the two competing methods in classifying high-grade gliomas (HGGs) and low-grade gliomas (LGGs), achieving an area under the curve of 87.76% on average for all missing-modality scenarios. The activation maps derived with our method confirm that it focuses on the enhancing portion of the tumor in HGGs and on the edema and non-enhancing portions of the tumor in LGGs, which is consistent with prior expertise. An ablation study shows the added benefits of a fusion block and adversarial learning for handling missing modalities. CONCLUSION Our method shows robust grading of gliomas in all cases of missing modalities. Our proposed network might have positive implications in glioma care by learning features robust to missing modalities.
Collapse
Affiliation(s)
- Seung-Wan Jeong
- Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea; Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea
| | - Hwan-Ho Cho
- Department of Medical Aritifical Intelligence, Konyang University, Daejon, Republic of Korea
| | - Seunghak Lee
- Core Research & Development Center, Korea University Ansan Hospital, Ansan, Republic of Korea
| | - Hyunjin Park
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea; School of Electronic and Electrical Engineering, Sungkyunkwan University, Suwon, Republic of Korea.
| |
Collapse
|
11
|
Pan Y, Liu M, Xia Y, Shen D. Disease-Image-Specific Learning for Diagnosis-Oriented Neuroimage Synthesis With Incomplete Multi-Modality Data. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022; 44:6839-6853. [PMID: 34156939 PMCID: PMC9297233 DOI: 10.1109/tpami.2021.3091214] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
Incomplete data problem is commonly existing in classification tasks with multi-source data, particularly the disease diagnosis with multi-modality neuroimages, to track which, some methods have been proposed to utilize all available subjects by imputing missing neuroimages. However, these methods usually treat image synthesis and disease diagnosis as two standalone tasks, thus ignoring the specificity conveyed in different modalities, i.e., different modalities may highlight different disease-relevant regions in the brain. To this end, we propose a disease-image-specific deep learning (DSDL) framework for joint neuroimage synthesis and disease diagnosis using incomplete multi-modality neuroimages. Specifically, with each whole-brain scan as input, we first design a Disease-image-Specific Network (DSNet) with a spatial cosine module to implicitly model the disease-image specificity. We then develop a Feature-consistency Generative Adversarial Network (FGAN) to impute missing neuroimages, where feature maps (generated by DSNet) of a synthetic image and its respective real image are encouraged to be consistent while preserving the disease-image-specific information. Since our FGAN is correlated with DSNet, missing neuroimages can be synthesized in a diagnosis-oriented manner. Experimental results on three datasets suggest that our method can not only generate reasonable neuroimages, but also achieve state-of-the-art performance in both tasks of Alzheimer's disease identification and mild cognitive impairment conversion prediction.
Collapse
|
12
|
Li J, Zhang Q, Chen S, Fang K. Weighted multiple blockwise imputation method for high-dimensional regression with blockwise missing data. J STAT COMPUT SIM 2022. [DOI: 10.1080/00949655.2022.2109636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2022]
Affiliation(s)
- Jingmao Li
- Department of Statistics and Data Science, School of Economics, Xiamen University, Xiamen, People's Republic of China
| | - Qingzhao Zhang
- Department of Statistics and Data Science, School of Economics, Xiamen University, Xiamen, People's Republic of China
- The Wang Yanan Institute for Studies in Economics, Xiamen University, Xiamen, People's Republic of China
| | - Song Chen
- College of Micro-Finance, Taizhou University, Taizhou, People's Republic of China
| | - Kuangnan Fang
- Department of Statistics and Data Science, School of Economics, Xiamen University, Xiamen, People's Republic of China
| |
Collapse
|
13
|
Li J, Yu G, Li Q, Liu Y. Sample-wise Combined Missing Effect Model with Penalization. J Comput Graph Stat 2022; 32:263-274. [PMID: 37274355 PMCID: PMC10237115 DOI: 10.1080/10618600.2022.2070172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2020] [Accepted: 04/11/2022] [Indexed: 10/18/2022]
Abstract
Modern high-dimensional statistical inference often faces the problem of missing data. In recent decades, many studies have focused on this topic and provided strategies including complete-sample analysis and imputation procedures. However, complete-sample analysis discards information of incomplete samples, while imputation procedures have accumulative errors from each single imputation. In this paper, we propose a new method, Sample-wise COmbined missing effect Model with penalization (SCOM), to deal with missing data occurring in predictors. Instead of imputing the predictors, SCOM estimates the combined effect caused by all missing data for each incomplete sample. SCOM makes full use of all available data. It is robust with respect to various missing mechanisms. Theoretical studies show the oracle inequality for the proposed estimator, and the consistency of variable selection and combined missing effect selection. Simulation studies and an application to the Residential Building Data also illustrate the effectiveness of the proposed SCOM.
Collapse
Affiliation(s)
- Jialu Li
- School of Mathematics and Statistics, Beijing Institute of Technology
| | - Guan Yu
- Department of Biostatistics, State University of New York at Buffalo
| | - Qizhai Li
- LSC, NCMIS, Academy of Mathematics and Systems Science, University of Chinese Academy of Sciences
| | - Yufeng Liu
- Department of Statistics and Operations Research, Carolina Center for Genome Science, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill
- Department of Genetics, Carolina Center for Genome Science, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill
- Department of Biostatistics, Carolina Center for Genome Science, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill
| |
Collapse
|
14
|
Yu G, Hou S. Integrative nearest neighbor classifier for block-missing multi-modality data. Stat Methods Med Res 2022; 31:1242-1262. [PMID: 35301917 DOI: 10.1177/09622802221084596] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In modern biomedical classification applications, data are often collected from multiple modalities, ranging from various omics technologies to brain scans. As different modalities provide complementary information, classifiers using multi-modality data usually have good classification performance. However, in many studies, due to the high cost of measures, in a lot of samples, some modalities are missing and therefore all data from those modalities are missing completely. In this case, the training data set is a block-missing multi-modality data set. In this paper, considering such classification problems, we develop a new weighted nearest neighbors classifier, called the integrative nearest neighbor (INN) classifier. INN harnesses all available information in the training data set and the feature vector of the test data point effectively to predict the class label of the test data point without deleting or imputing any missing data. Given a test data point, INN determines the weights on the training samples adaptively by minimizing the worst-case upper bound on the estimation error of the regression function over a convex class of functions. Our simulation study shows that INN outperforms common weighted nearest neighbors classifiers that only use complete training samples or modalities that are available in each sample. It performs better than methods that impute the missing data as well, even for the case where some modalities are missing not at random. The effectiveness of INN has been also demonstrated by our theoretical studies and a real application from the Alzheimer's disease neuroimaging initiative.
Collapse
Affiliation(s)
- Guan Yu
- Department of Biostatistics, 12292The State University of New York at Buffalo, NY, USA
| | - Surui Hou
- Department of Biostatistics, 12292The State University of New York at Buffalo, NY, USA
| |
Collapse
|
15
|
Dimension reduction for block-missing data based on sparse sliced inverse regression. Comput Stat Data Anal 2022. [DOI: 10.1016/j.csda.2021.107348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
16
|
Saad M, He S, Thorstad W, Gay H, Barnett D, Zhao Y, Ruan S, Wang X, Li H. Learning-based Cancer Treatment Outcome Prognosis using Multimodal Biomarkers. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2022; 6:231-244. [PMID: 35520102 PMCID: PMC9066560 DOI: 10.1109/trpms.2021.3104297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
Predicting early in treatment whether a tumor is likely to be responsive is a difficult yet important task to support clinical decision-making. Studies have shown that multimodal biomarkers could provide complementary information and lead to more accurate treatment outcome prognosis than unimodal biomarkers. However, the prognosis accuracy could be affected by multimodal data heterogeneity and incompleteness. The small-sized and imbalance datasets also bring additional challenges for training a designed prognosis model. In this study, a modular framework employing multimodal biomarkers for cancer treatment outcome prediction was proposed. It includes four modules of synthetic data generation, deep feature extraction, multimodal feature fusion, and classification to address the challenges described above. The feasibility and advantages of the designed framework were demonstrated through an example study, in which the goal was to stratify oropharyngeal squamous cell carcinoma (OPSCC) patients with low- and high-risks of treatment failures by use of positron emission tomography (PET) image data and microRNA (miRNA) biomarkers. The superior prognosis performance and the comparison with other methods demonstrated the efficiency of the proposed framework and its ability of enabling seamless integration, validation and comparison of various algorithms in each module of the framework. The limitation and future work was discussed as well.
Collapse
Affiliation(s)
- Maliazurina Saad
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA. She is now with the MD Anderson Cancer Center, Houston, TX, USA
| | - Shenghua He
- Department of Computer Science and Engineering, Washington University, Saint louis, MO, USA
| | - Wade Thorstad
- Department of Radiation Oncology, Washington University School of Medicine, Saint louis, MO, USA
| | - Hiram Gay
- Department of Radiation Oncology, Washington University School of Medicine, Saint louis, MO, USA
| | - Daniel Barnett
- Carle Cancer Center, Carle Foundation Hospital, Urbana, IL, USA
| | - Yujie Zhao
- Mao Clinic at Florida, Jacksonville, FL, USA
| | - Su Ruan
- Laboratoire LITIS (EA 4108), Equipe Quantif, University of Rouen, France
| | - Xiaowei Wang
- Department of Pharmacology and Bioengineering, University of Illinois at Chicago, Chicago, IL, USA
| | - Hua Li
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Cancer Center at Illinois, and Carle Foundation Hospital, Urbana, IL, USA
| |
Collapse
|
17
|
|
18
|
|
19
|
Zhang J, Wu J, Li Q, Caselli RJ, Thompson PM, Ye J, Wang Y. Multi-Resemblance Multi-Target Low-Rank Coding for Prediction of Cognitive Decline With Longitudinal Brain Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:2030-2041. [PMID: 33798076 PMCID: PMC8363167 DOI: 10.1109/tmi.2021.3070780] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
An effective presymptomatic diagnosis and treatment of Alzheimer's disease (AD) would have enormous public health benefits. Sparse coding (SC) has shown strong potential for longitudinal brain image analysis in preclinical AD research. However, the traditional SC computation is time-consuming and does not explore the feature correlations that are consistent over the time. In addition, longitudinal brain image cohorts usually contain incomplete image data and clinical labels. To address these challenges, we propose a novel two-stage Multi-Resemblance Multi-Target Low-Rank Coding (MMLC) method, which encourages that sparse codes of neighboring longitudinal time points are resemblant to each other, favors sparse code low-rankness to reduce the computational cost and is resilient to both source and target data incompleteness. In stage one, we propose an online multi-resemblant low-rank SC method to utilize the common and task-specific dictionaries in different time points to immune to incomplete source data and capture the longitudinal correlation. In stage two, supported by a rigorous theoretical analysis, we develop a multi-target learning method to address the missing clinical label issue. To solve such a multi-task low-rank sparse optimization problem, we propose multi-task stochastic coordinate coding with a sequence of closed-form update steps which reduces the computational costs guaranteed by a theoretical convergence proof. We apply MMLC on a publicly available neuroimaging cohort to predict two clinical measures and compare it with six other methods. Our experimental results show our proposed method achieves superior results on both computational efficiency and predictive accuracy and has great potential to assist the AD prevention.
Collapse
|
20
|
Deng Y, Tang X, Qu A. Correlation Tensor Decomposition and Its Application in Spatial Imaging Data. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2021.1938083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Yujia Deng
- Department of Statistics, University of Illinois, Urbana-Champaign, IL
| | - Xiwei Tang
- Department of Statistics, University of Virginia, Charlottesville, VA
| | - Annie Qu
- Department of Statistics, University of California, Irvine, CA
| |
Collapse
|
21
|
Gao X, Shi F, Shen D, Liu M. Task-induced Pyramid and Attention GAN for Multimodal Brain Image Imputation and Classification in Alzheimers disease. IEEE J Biomed Health Inform 2021; 26:36-43. [PMID: 34280112 DOI: 10.1109/jbhi.2021.3097721] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
With the advance of medical imaging technologies, multimodal images such as magnetic resonance images (MRI) and positron emission tomography (PET) can capture subtle structural and functional changes of brain, facilating the diagnosis of brain diseases such as Alzheimers disease (AD). In practice, multimodal images may be incomplete since PET is often missing due to high financial cost or availability. Most of existing methods simply excluded subjects with missing data, which unfortunately reduced sample size. In addition, how to extract and combine multimodal features is still challenging. To address these problems, we propose a deep learning framework to integrate a task-induced pyramid and attention generative adversarial network (TPA-GAN) with a pathwise transfer dense convolution network (PT-DCN) for imputation and also classification of multimodal brain images. First, we propose a TPA-GAN to integrate pyramid convolution and attention module as well as disease classification task into GAN for generating the missing PET data with their MRI. Then, with the imputed multimodal brain images, we build a dense convolution network with pathwise transfer blocks to gradually learn and combine multimodal features for final disease classification. Experiments are performed on ADNI-1 and ADNI-2 datasets to evaluate our proposed method, achiving superior performance in image imputation and brain disease diagnosis compared to state-of-the-art methods.
Collapse
|
22
|
Dong A, Li Z, Wang M, Shen D, Liu M. High-Order Laplacian Regularized Low-Rank Representation for Multimodal Dementia Diagnosis. Front Neurosci 2021; 15:634124. [PMID: 33776639 PMCID: PMC7994898 DOI: 10.3389/fnins.2021.634124] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Accepted: 01/25/2021] [Indexed: 11/15/2022] Open
Abstract
Multimodal heterogeneous data, such as structural magnetic resonance imaging (MRI), positron emission tomography (PET), and cerebrospinal fluid (CSF), are effective in improving the performance of automated dementia diagnosis by providing complementary information on degenerated brain disorders, such as Alzheimer's prodromal stage, i.e., mild cognitive impairment. Effectively integrating multimodal data has remained a challenging problem, especially when these heterogeneous data are incomplete due to poor data quality and patient dropout. Besides, multimodal data usually contain noise information caused by different scanners or imaging protocols. The existing methods usually fail to well handle these heterogeneous and noisy multimodal data for automated brain dementia diagnosis. To this end, we propose a high-order Laplacian regularized low-rank representation method for dementia diagnosis using block-wise missing multimodal data. The proposed method was evaluated on 805 subjects (with incomplete MRI, PET, and CSF data) from the real Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results suggest the effectiveness of our method in three tasks of brain disease classification, compared with the state-of-the-art methods.
Collapse
Affiliation(s)
- Aimei Dong
- School of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Science), Jinan, China
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Zhigang Li
- School of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Science), Jinan, China
| | - Mingliang Wang
- College of Computer Science and Technology, Nanjing University of Aeronautics & Astronautics, Nanjing, China
| | - Dinggang Shen
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| |
Collapse
|
23
|
Zhou J, Hu L, Jiang Y, Liu L. A Correlation Analysis between SNPs and ROIs of Alzheimer's Disease Based on Deep Learning. BIOMED RESEARCH INTERNATIONAL 2021; 2021:8890513. [PMID: 33628827 PMCID: PMC7886593 DOI: 10.1155/2021/8890513] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/20/2020] [Revised: 12/23/2020] [Accepted: 01/27/2021] [Indexed: 12/31/2022]
Abstract
Motivation. At present, the research methods for image genetics of Alzheimer's disease based on machine learning are mainly divided into three steps: the first step is to preprocess the original image and gene information into digital signals that are easy to calculate; the second step is feature selection aiming at eliminating redundant signals and obtain representative features; and the third step is to build a learning model and predict the unknown data with regression or bivariate correlation analysis. This type of method requires manual extraction of feature single-nucleotide polymorphisms (SNPs), and the extraction process relies on empirical knowledge to a certain extent, such as linkage imbalance and gene function information in a group sparse model, which puts forward certain requirements for applicable scenarios and application personnel. To solve the problems of insufficient biological significance and large errors in the previous methods of association analysis and disease diagnosis, this paper presents a method of correlation analysis and disease diagnosis between SNP and region of interest (ROI) based on a deep learning model. It is a data-driven method, which has no obvious feature selection process. Results. The deep learning method adopted in this paper has no obvious feature extraction process relying on prior knowledge and model assumptions. From the results of correlation analysis between SNP and ROI, this method is complementary to other regression model methods in application scenarios. In order to improve the disease diagnosis performance of deep learning, we use the deep learning model to integrate SNP characteristics and ROI characteristics. The SNP feature, ROI feature, and SNP-ROI joint feature were input into the deep learning model and trained by cross-validation technique. The experimental results show that the SNP-ROI joint feature describes the information of the samples from different angles, which makes the diagnosis accuracy higher.
Collapse
Affiliation(s)
- Juan Zhou
- School of Software, East China Jiaotong University, Nanchang 330013, China
| | - Linfeng Hu
- School of Software, East China Jiaotong University, Nanchang 330013, China
| | - Yu Jiang
- School of Software, East China Jiaotong University, Nanchang 330013, China
| | - Liyue Liu
- School of Software, East China Jiaotong University, Nanchang 330013, China
| |
Collapse
|
24
|
Zhu H, Li G, Lock EF. Generalized integrative principal component analysis for multi-type data with block-wise missing structure. Biostatistics 2020; 21:302-318. [PMID: 30247540 DOI: 10.1093/biostatistics/kxy052] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2018] [Accepted: 08/15/2018] [Indexed: 12/19/2022] Open
Abstract
High-dimensional multi-source data are encountered in many fields. Despite recent developments on the integrative dimension reduction of such data, most existing methods cannot easily accommodate data of multiple types (e.g. binary or count-valued). Moreover, multi-source data often have block-wise missing structure, i.e. data in one or more sources may be completely unobserved for a sample. The heterogeneous data types and presence of block-wise missing data pose significant challenges to the integration of multi-source data and further statistical analyses. In this article, we develop a low-rank method, called generalized integrative principal component analysis (GIPCA), for the simultaneous dimension reduction and imputation of multi-source block-wise missing data, where different sources may have different data types. We also devise an adapted Bayesian information criterion (BIC) criterion for rank estimation. Comprehensive simulation studies demonstrate the efficacy of the proposed method in terms of rank estimation, signal recovery, and missing data imputation. We apply GIPCA to a mortality study. We achieve accurate block-wise missing data imputation and identify intriguing latent mortality rate patterns with sociological relevance.
Collapse
Affiliation(s)
- Huichen Zhu
- The Department of Biostatistics, Columbia University, 722 West 168th St., New York, NY, USA
| | - Gen Li
- The Department of Biostatistics, Columbia University, 722 West 168th St., New York, NY, USA
| | - Eric F Lock
- The Division of Biostatistics, School of Public Health, University of Minneapolis, 420 Delaware Street S.E., Minneapolis, MN, USA
| |
Collapse
|
25
|
Sserwadda A, Rekik I. Topology-guided cyclic brain connectivity generation using geometric deep learning. J Neurosci Methods 2020; 353:108988. [PMID: 33160020 DOI: 10.1016/j.jneumeth.2020.108988] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Revised: 09/23/2020] [Accepted: 10/20/2020] [Indexed: 11/19/2022]
Abstract
BACKGROUND There is a growing need for analyzing medical data such as brain connectomes. However, the unavailability of large-scale training samples increases risks of model over-fitting. Recently, deep learning (DL) architectures quickly gained momentum in synthesizing medical data. However, such frameworks are primarily designed for Euclidean data (e.g., images), overlooking geometric data (e.g., brain connectomes). A few existing geometric DL works that aimed to predict a target brain connectome from a source one primarily focused on domain alignment and were agnostic to preserving the connectome topology. NEW METHOD To address the above limitations, firstly, we adapt the graph translation generative adversarial network (GT GAN) architecture to brain connectomic data. Secondly, we extend the baseline GT GAN to a cyclic graph translation (CGT) GAN, allowing bidirectional brain network translation between the source and target views. Finally, to preserve the topological strength of brain regions of interest (ROIs), we impose a topological strength constraint on the CGT GAN learning, thereby introducing CGTS GAN architecture. COMPARISON WITH EXISTING METHODS We compared CGTS with graph translation methods and its ablated versions. RESULTS Our deep graph network outperformed the baseline comparison method and its ablated versions in mean squared error (MSE) using multiview autism spectrum disorder connectomic dataset. CONCLUSION We designed a topology-aware bidirectional brain connectome synthesis framework rooted in geometric deep learning, which can be used for data augmentation in clinical diagnosis.
Collapse
Affiliation(s)
- Abubakhari Sserwadda
- BASIRA Lab, Faculty of Computer and Informatics, Istanbul Technical University, Istanbul, Turkey
| | - Islem Rekik
- BASIRA Lab, Faculty of Computer and Informatics, Istanbul Technical University, Istanbul, Turkey; School of Science and Engineering, Computing, University of Dundee, UK
| |
Collapse
|
26
|
Liu X, Chen K, Weidman D, Wu T, Lure F, Li J. A Novel Transfer Learning Model for Predictive Analytics using Incomplete Multimodality Data. IISE TRANSACTIONS 2020; 53:1010-1022. [PMID: 37397785 PMCID: PMC10312385 DOI: 10.1080/24725854.2020.1798569] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/17/2018] [Revised: 05/10/2020] [Accepted: 06/14/2020] [Indexed: 07/04/2023]
Abstract
Multimodality datasets are becoming increasingly common in various domains to provide complementary information for predictive analytics. One significant challenge in fusing multimodality data is that the multiple modalities are not universally available for all samples due to cost and accessibility constraints. This results in a unique data structure called Incomplete Multimodality Dataset (IMD). We propose a novel Incomplete-Multimodality Transfer Learning (IMTL) model that builds a predictive model for each sub-cohort of samples with the same missing modality pattern, and meanwhile couples the model estimation processes for different sub-cohorts to allow for transfer learning. We develop an Expectation-Maximization (EM) algorithm to estimate the parameters of IMTL and further extend it to a collaborative learning paradigm that is specifically valuable for patient privacy preservation in health care applications. We prove two advantageous properties of IMTL: the ability for out-of-sample prediction and a theoretical guarantee for a larger Fisher information compared with models without transfer learning. IMTL is applied to diagnosis and prognosis of the Alzheimer's Disease (AD) at an early stage called Mild Cognitive Impairment (MCI) using incomplete multimodality imaging data. IMTL achieves higher accuracy than competing methods without transfer learning. Supplementary materials are available for this article on the publisher's website.
Collapse
Affiliation(s)
- Xiaonan Liu
- Industrial Engineering, Arizona State University, Tempe, AZ, USA
| | - Kewei Chen
- Banner Alzheimer’s Institute, Phoenix, AZ, USA
| | | | - Teresa Wu
- Industrial Engineering, Arizona State University, Tempe, AZ, USA
| | | | - Jing Li
- Industrial Engineering, Arizona State University, Tempe, AZ, USA
| | | |
Collapse
|
27
|
Liu M, Zhang J, Lian C, Shen D. Weakly Supervised Deep Learning for Brain Disease Prognosis Using MRI and Incomplete Clinical Scores. IEEE TRANSACTIONS ON CYBERNETICS 2020; 50:3381-3392. [PMID: 30932861 PMCID: PMC8034591 DOI: 10.1109/tcyb.2019.2904186] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
As a hot topic in brain disease prognosis, predicting clinical measures of subjects based on brain magnetic resonance imaging (MRI) data helps to assess the stage of pathology and predict future development of the disease. Due to incomplete clinical labels/scores, previous learning-based studies often simply discard subjects without ground-truth scores. This would result in limited training data for learning reliable and robust models. Also, existing methods focus only on using hand-crafted features (e.g., image intensity or tissue volume) of MRI data, and these features may not be well coordinated with prediction models. In this paper, we propose a weakly supervised densely connected neural network (wiseDNN) for brain disease prognosis using baseline MRI data and incomplete clinical scores. Specifically, we first extract multiscale image patches (located by anatomical landmarks) from MRI to capture local-to-global structural information of images, and then develop a weakly supervised densely connected network for task-oriented extraction of imaging features and joint prediction of multiple clinical measures. A weighted loss function is further employed to make full use of all available subjects (even those without ground-truth scores at certain time-points) for network training. The experimental results on 1469 subjects from both ADNI-1 and ADNI-2 datasets demonstrate that our proposed method can efficiently predict future clinical measures of subjects.
Collapse
|
28
|
Khatri U, Kwon GR. An Efficient Combination among sMRI, CSF, Cognitive Score, and APOE ε4 Biomarkers for Classification of AD and MCI Using Extreme Learning Machine. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2020; 2020:8015156. [PMID: 32565773 PMCID: PMC7292973 DOI: 10.1155/2020/8015156] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2019] [Revised: 01/13/2020] [Accepted: 02/17/2020] [Indexed: 11/17/2022]
Abstract
Alzheimer's disease (AD) is the most common cause of dementia and a progressive neurodegenerative condition, characterized by a decline in cognitive function. Symptoms usually appear gradually and worsen over time, becoming severe enough to interfere with individual daily tasks. Thus, the accurate diagnosis of both AD and the prodromal stage (i.e., mild cognitive impairment (MCI)) is crucial for timely treatment. As AD is inherently dynamic, the relationship between AD indicators is unclear and varies over time. To address this issue, we first aimed at investigating differences in atrophic patterns between individuals with AD and MCI and healthy controls (HCs). Then we utilized multiple biomarkers, along with filter- and wrapper-based feature selection and an extreme learning machine- (ELM-) based approach, with 10-fold cross-validation for classification. Increasing efforts are focusing on the use of multiple biomarkers, which can be useful for the diagnosis of AD and MCI. However, optimum combinations have yet to be identified and most multimodal analyses use only volumetric measures obtained from magnetic resonance imaging (MRI). Anatomical structural MRI (sMRI) measures have also so far mostly been used separately. The full possibilities of using anatomical MRI for AD detection have thus yet to be explored. In this study, three measures (cortical thickness, surface area, and gray matter volume), obtained from sMRI through preprocessing for brain atrophy measurements; cerebrospinal fluid (CSF), for quantification of specific proteins; cognitive score, as a measure of cognitive performance; and APOE ε4 allele status were utilized. Our results show that a combination of specific biomarkers performs well, with accuracies of 97.31% for classifying AD vs. HC, 91.72% for MCI vs. HC, 87.91% for MCI vs. AD, and 83.38% for MCIs vs. MCIc, respectively, when evaluated using the proposed algorithm. Meanwhile, the areas under the curve (AUC) from the receiver operating characteristic (ROC) curves combining multiple biomarkers provided better classification performance. The proposed features combination and selection algorithm effectively classified AD and MCI, and MCIs vs. MCIc, the most challenging classification task, and therefore could increase the accuracy of AD classification in clinical practice. Furthermore, we compared the performance of the proposed method with SVM classifiers, using a cross-validation method with Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets.
Collapse
Affiliation(s)
- Uttam Khatri
- Department. of Information and Communication Engineering, Chosun University, 309 Pilmun-Daero, Dong-Gu, Gwangju 61452, Republic of Korea
| | - Goo-Rak Kwon
- Department. of Information and Communication Engineering, Chosun University, 309 Pilmun-Daero, Dong-Gu, Gwangju 61452, Republic of Korea
| |
Collapse
|
29
|
Liu X, Wang L, Zhu X, Li M, Zhu E, Liu T, Liu L, Dou Y, Yin J. Absent Multiple Kernel Learning Algorithms. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2020; 42:1303-1316. [PMID: 30703009 DOI: 10.1109/tpami.2019.2895608] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Multiple kernel learning (MKL) has been intensively studied during the past decade. It optimally combines the multiple channels of each sample to improve classification performance. However, existing MKL algorithms cannot effectively handle the situation where some channels of the samples are missing, which is not uncommon in practical applications. This paper proposes three absent MKL (AMKL) algorithms to address this issue. Different from existing approaches where missing channels are first imputed and then a standard MKL algorithm is deployed on the imputed data, our algorithms directly classify each sample based on its observed channels, without performing imputation. Specifically, we define a margin for each sample in its own relevant space, a space corresponding to the observed channels of that sample. The proposed AMKL algorithms then maximize the minimum of all sample-based margins, and this leads to a difficult optimization problem. We first provide two two-step iterative algorithms to approximately solve this problem. After that, we show that this problem can be reformulated as a convex one by applying the representer theorem. This makes it readily be solved via existing convex optimization packages. In addition, we provide a generalization error bound to justify the proposed AMKL algorithms from a theoretical perspective. Extensive experiments are conducted on nine UCI and six MKL benchmark datasets to compare the proposed algorithms with existing imputation-based methods. As demonstrated, our algorithms achieve superior performance and the improvement is more significant with the increase of missing ratio.
Collapse
|
30
|
Healthcare predictive analytics for disease progression: a longitudinal data fusion approach. J Intell Inf Syst 2020. [DOI: 10.1007/s10844-020-00606-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
31
|
Affiliation(s)
- Fei Xue
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA
| | - Annie Qu
- Department of Statistics, University of California Irvine, Irvine, CA
| |
Collapse
|
32
|
Mining incomplete clinical data for the early assessment of Kawasaki disease based on feature clustering and convolutional neural networks. Artif Intell Med 2020; 105:101859. [DOI: 10.1016/j.artmed.2020.101859] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2019] [Revised: 02/26/2020] [Accepted: 04/03/2020] [Indexed: 12/20/2022]
|
33
|
Enzymes to unravel bioproducts architecture. Biotechnol Adv 2020; 41:107546. [PMID: 32275940 DOI: 10.1016/j.biotechadv.2020.107546] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2019] [Revised: 02/20/2020] [Accepted: 04/03/2020] [Indexed: 11/20/2022]
Abstract
Enzymes are essential and ubiquitous biocatalysts involved in various metabolic pathways and used in many industrial processes. Here, we reframe enzymes not just as biocatalysts transforming bioproducts but also as sensitive probes for exploring the structure and composition of complex bioproducts, like meat tissue, dairy products and plant materials, in both food and non-food bioprocesses. This review details the global strategy and presents the most recent investigations to prepare and use enzymes as relevant probes, with a focus on glycoside-hydrolases involved in plant deconstruction and proteases and lipases involved in food digestion. First, to expand the enzyme repertoire to fit bioproduct complexity, novel enzymes are mined from biodiversity and can be artificially engineered. Enzymes are further characterized by exploring sequence/structure/dynamics/function relationships together with the environmental factors influencing enzyme interactions with their substrates. Then, the most advanced experimental and theoretical approaches developed for exploring bioproducts at various scales (from nanometer to millimeter) using active and inactive enzymes as probes are illustrated. Overall, combining multimodal and multiscale approaches brings a better understanding of native-form or transformed bioproduct architecture and composition, and paves the way to mainstream the use of enzymes as probes.
Collapse
|
34
|
Forouzannezhad P, Abbaspour A, Li C, Fang C, Williams U, Cabrerizo M, Barreto A, Andrian J, Rishe N, Curiel RE, Loewenstein D, Duara R, Adjouadi M. A Gaussian-based model for early detection of mild cognitive impairment using multimodal neuroimaging. J Neurosci Methods 2020; 333:108544. [PMID: 31838182 PMCID: PMC11163390 DOI: 10.1016/j.jneumeth.2019.108544] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2018] [Revised: 11/26/2019] [Accepted: 12/05/2019] [Indexed: 12/14/2022]
Abstract
BACKGROUND Diagnosis of early mild cognitive impairment (EMCI) as a prodromal stage of Alzheimer's disease (AD) with its delineation from the cognitively normal (CN) group remains a challenging but essential step for the planning of early treatment. Although several studies have focused on the MCI diagnosis, this study introduces the early stage of MCI to assess more thoroughly the earliest signs of disease manifestation and progression. NEW METHOD We used random forest feature selection model with a Gaussian-based algorithm to perform method evaluation. This integrated method serves to define multivariate normal distributions in order to classify different stages of AD, with the focus placed on detecting EMCI subjects in the most challenging classification of CN vs. EMCI. RESULTS Using 896 participants classified into the four categories of CN, EMCI, late mild cognitive impairment (LMCI) and AD, the results show that the EMCI group can be delineated from the CN group with a relatively high accuracy of 78.8% and sensitivity of 81.3%. COMPARISON WITH EXISTING METHOD(S) The feature selection model and classifier are compared with some other prominent algorithms. Although higher accuracy has been achieved using the Gaussian process (GP) model (78.8%) over the SVM classifier (75.6%) for CN vs. EMCI classification, with 0.05 being the cutoff for significance, and based on student's t-test, it was determined that the differences for accuracy, sensitivity, specificity between the GP method and support vector machine (SVM) are not statistically significant. CONCLUSION Addressing the challenging classification of CN vs. EMCI provides useful information to help clinicians and researchers determine essential measures that can help in the early detection of AD.
Collapse
Affiliation(s)
- Parisa Forouzannezhad
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Alireza Abbaspour
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Chunfei Li
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Chen Fang
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Ulyana Williams
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Mercedes Cabrerizo
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Armando Barreto
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Jean Andrian
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA
| | - Naphtali Rishe
- School of Computing and Information Sciences, Florida International University, Miami, FL, USA
| | - Rosie E Curiel
- Center for Cognitive Neuroscience and Aging, Department of Psychiatry and Behavioral Sciences, University of Miami School of Medicine, Miami, FL, USA; 1Florida Alzheimer's Disease Research Center (ADRC), University of Florida, Gainesville, FL, USA
| | - David Loewenstein
- Center for Cognitive Neuroscience and Aging, Department of Psychiatry and Behavioral Sciences, University of Miami School of Medicine, Miami, FL, USA; 1Florida Alzheimer's Disease Research Center (ADRC), University of Florida, Gainesville, FL, USA; Wien Center for Alzheimer's Disease and Memory Disorders, Mount Sinai Medical Center, Miami Beach, FL, USA
| | - Ranjan Duara
- 1Florida Alzheimer's Disease Research Center (ADRC), University of Florida, Gainesville, FL, USA; Wien Center for Alzheimer's Disease and Memory Disorders, Mount Sinai Medical Center, Miami Beach, FL, USA
| | - Malek Adjouadi
- Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA; 1Florida Alzheimer's Disease Research Center (ADRC), University of Florida, Gainesville, FL, USA
| |
Collapse
|
35
|
Zhou T, Thung KH, Liu M, Shi F, Zhang C, Shen D. Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data. Med Image Anal 2020; 60:101630. [PMID: 31927474 PMCID: PMC8260095 DOI: 10.1016/j.media.2019.101630] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2018] [Revised: 12/15/2019] [Accepted: 12/19/2019] [Indexed: 12/21/2022]
Abstract
Fusing multi-modality data is crucial for accurate identification of brain disorder as different modalities can provide complementary perspectives of complex neurodegenerative disease. However, there are at least four common issues associated with the existing fusion methods. First, many existing fusion methods simply concatenate features from each modality without considering the correlations among different modalities. Second, most existing methods often make prediction based on a single classifier, which might not be able to address the heterogeneity of the Alzheimer's disease (AD) progression. Third, many existing methods often employ feature selection (or reduction) and classifier training in two independent steps, without considering the fact that the two pipelined steps are highly related to each other. Forth, there are missing neuroimaging data for some of the participants (e.g., missing PET data), due to the participants' "no-show" or dropout. In this paper, to address the above issues, we propose an early AD diagnosis framework via novel multi-modality latent space inducing ensemble SVM classifier. Specifically, we first project the neuroimaging data from different modalities into a latent space, and then map the learned latent representations into the label space to learn multiple diversified classifiers. Finally, we obtain the more reliable classification results by using an ensemble strategy. More importantly, we present a Complete Multi-modality Latent Space (CMLS) learning model for complete multi-modality data and also an Incomplete Multi-modality Latent Space (IMLS) learning model for incomplete multi-modality data. Extensive experiments using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset have demonstrated that our proposed models outperform other state-of-the-art methods.
Collapse
Affiliation(s)
- Tao Zhou
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, NC 27599, USA; Inception Institute of Artificial Intelligence, Abu Dhabi 51133, United Arab Emirates.
| | - Kim-Han Thung
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, NC 27599, USA.
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, NC 27599, USA.
| | - Feng Shi
- United Imaging Intelligence, Shanghai, China.
| | - Changqing Zhang
- School of Computer Science and Technology, Tianjin University, Tianjin 300072, China.
| | - Dinggang Shen
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, NC 27599, USA; Department of Brain and Cognitive Engineering, Korea University, Seoul 02841, Republic of Korea.
| |
Collapse
|
36
|
Estimation of connectional brain templates using selective multi-view network normalization. Med Image Anal 2020; 59:101567. [DOI: 10.1016/j.media.2019.101567] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2019] [Revised: 09/23/2019] [Accepted: 09/27/2019] [Indexed: 11/19/2022]
|
37
|
Demir U, Gharsallaoui MA, Rekik I. Clustering-Based Deep Brain MultiGraph Integrator Network for Learning Connectional Brain Templates. UNCERTAINTY FOR SAFE UTILIZATION OF MACHINE LEARNING IN MEDICAL IMAGING, AND GRAPHS IN BIOMEDICAL IMAGE ANALYSIS 2020. [DOI: 10.1007/978-3-030-60365-6_11] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
38
|
Yu G, Li Q, Shen D, Liu Y. Optimal Sparse Linear Prediction for Block-missing Multi-modality Data without Imputation. J Am Stat Assoc 2020; 115:1406-1419. [PMID: 34824484 DOI: 10.1080/01621459.2019.1632079] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
In modern scientific research, data are often collected from multiple modalities. Since different modalities could provide complementary information, statistical prediction methods using multi-modality data could deliver better prediction performance than using single modality data. However, one special challenge for using multi-modality data is related to block-missing data. In practice, due to dropouts or the high cost of measures, the observations of a certain modality can be missing completely for some subjects. In this paper, we propose a new DIrect Sparse regression procedure using COvariance from Multi-modality data (DISCOM). Our proposed DISCOM method includes two steps to find the optimal linear prediction of a continuous response variable using block-missing multi-modality predictors. In the first step, rather than deleting or imputing missing data, we make use of all available information to estimate the covariance matrix of the predictors and the cross-covariance vector between the predictors and the response variable. The proposed new estimate of the covariance matrix is a linear combination of the identity matrix, the estimates of the intra-modality covariance matrix and the cross-modality covariance matrix. Flexible estimates for both the sub-Gaussian and heavy-tailed cases are considered. In the second step, based on the estimated covariance matrix and the estimated cross-covariance vector, an extended Lasso-type estimator is used to deliver a sparse estimate of the coefficients in the optimal linear prediction. The number of samples that are effectively used by DISCOM is the minimum number of samples with available observations from two modalities, which can be much larger than the number of samples with complete observations from all modalities. The effectiveness of the proposed method is demonstrated by theoretical studies, simulated examples, and a real application from the Alzheimer's Disease Neuroimaging Initiative. The comparison between DISCOM and some existing methods also indicates the advantages of our proposed method.
Collapse
Affiliation(s)
- Guan Yu
- Department of Biostatistics, State University of New York at Buffalo
| | - Quefeng Li
- Department of Biostatistics, University of North Carolina at Chapel Hill
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill. He is also affiliated with the Department of Brain and Cognitive Engineering, Korea University, Seoul 136-701, Korea
| | - Yufeng Liu
- Department of Statistics and Operations Research, Department of Genetics, Department of Biostatistics, Carolina Center for Genome Science, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, NC 27599
| |
Collapse
|
39
|
Li Y, Wang L, Zhou J, Ye J. Multi-task learning based survival analysis for multi-source block-wise missing data. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.07.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
|
40
|
Zhou T, Liu M, Thung KH, Shen D. Latent Representation Learning for Alzheimer's Disease Diagnosis With Incomplete Multi-Modality Neuroimaging and Genetic Data. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:2411-2422. [PMID: 31021792 PMCID: PMC8034601 DOI: 10.1109/tmi.2019.2913158] [Citation(s) in RCA: 75] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
The fusion of complementary information contained in multi-modality data [e.g., magnetic resonance imaging (MRI), positron emission tomography (PET), and genetic data] has advanced the progress of automated Alzheimer's disease (AD) diagnosis. However, multi-modality based AD diagnostic models are often hindered by the missing data, i.e., not all the subjects have complete multi-modality data. One simple solution used by many previous studies is to discard samples with missing modalities. However, this significantly reduces the number of training samples, thus leading to a sub-optimal classification model. Furthermore, when building the classification model, most existing methods simply concatenate features from different modalities into a single feature vector without considering their underlying associations. As features from different modalities are often closely related (e.g., MRI and PET features are extracted from the same brain region), utilizing their inter-modality associations may improve the robustness of the diagnostic model. To this end, we propose a novel latent representation learning method for multi-modality based AD diagnosis. Specifically, we use all the available samples (including samples with incomplete modality data) to learn a latent representation space. Within this space, we not only use samples with complete multi-modality data to learn a common latent representation, but also use samples with incomplete multi-modality data to learn independent modality-specific latent representations. We then project the latent representations to the label space for AD diagnosis. We perform experiments using 737 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, and the experimental results verify the effectiveness of our proposed method.
Collapse
Affiliation(s)
- Tao Zhou
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
- Inception Institute of Artificial Intelligence, Abu Dhabi 51133, United Arab Emirates
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Kim-Han Thung
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Dinggang Shen
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
- Department of Brain and Cognitive Engineering, Korea University, Seoul 02841, Republic of Korea
| |
Collapse
|
41
|
Tang X, Bi X, Qu A. Individualized Multilayer Tensor Learning With an Application in Imaging Analysis. J Am Stat Assoc 2019. [DOI: 10.1080/01621459.2019.1585254] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Affiliation(s)
- Xiwei Tang
- Department of Statistics, University of Virginia, Charlottesville, VA
| | - Xuan Bi
- Department of Information and Decision Sciences, Carlson School of Management, University of Minnesota, Minneapolis, MN
| | - Annie Qu
- Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, IL
| |
Collapse
|
42
|
Zhang C, Adeli E, Wu Z, Li G, Lin W, Shen D. Infant Brain Development Prediction With Latent Partial Multi-View Representation Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:909-918. [PMID: 30307859 PMCID: PMC6450718 DOI: 10.1109/tmi.2018.2874964] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The early postnatal period witnesses rapid and dynamic brain development. However, the relationship between brain anatomical structure and cognitive ability is still unknown. Currently, there is no explicit model to characterize this relationship in the literature. In this paper, we explore this relationship by investigating the mapping between morphological features of the cerebral cortex and cognitive scores. To this end, we introduce a multi-view multi-task learning approach to intuitively explore complementary information from different time-points and handle the missing data issue in longitudinal studies simultaneously. Accordingly, we establish a novel model, latent partial multi-view representation learning. Our approach regards data from different time-points as different views and constructs a latent representation to capture the complementary information from incomplete time-points. The latent representation explores the complementarity across different time-points and improves the accuracy of prediction. The minimization problem is solved by the alternating direction method of multipliers. Experimental results on both synthetic and real data validate the effectiveness of our proposed algorithm.
Collapse
Affiliation(s)
- Changqing Zhang
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina, USA and College of Intelligence and Computing, Tianjin University, Tianjin, China, ()
| | - Ehsan Adeli
- Department of Psychiatry and Behavioral Sciences, Stanford University, California, USA, ()
| | - Zhengwang Wu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina, USA, ()
| | - Gang Li
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina, USA, ()
| | - Weili Lin
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina, USA, ()
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina, USA, and also with Department of Brain and Cognitive Engineering, Korea University, Seoul 02841, Republic of Korea, ()
| |
Collapse
|
43
|
Collazos-Huertas D, Cárdenas-Peña D, Castellanos-Dominguez G. Instance-Based Representation Using Multiple Kernel Learning for Predicting Conversion to Alzheimer Disease. Int J Neural Syst 2019; 29:1850042. [DOI: 10.1142/s0129065718500429] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The early detection of Alzheimer’s disease and quantification of its progression poses multiple difficulties for machine learning algorithms. Two of the most relevant issues are related to missing data and results interpretability. To deal with both issues, we introduce a methodology to predict conversion of mild cognitive impairment patients to Alzheimer’s from structural brain MRI volumes. First, we use morphological measures of each brain structure to build an instance-based feature mapping that copes with missed follow-up visits. Then, the extracted multiple feature mappings are combined into a single representation through the convex combination of reproducing kernels. The weighting parameters per structure are tuned based on the maximization of the centered-kernel alignment criterion. We evaluate the proposed methodology on a couple of well-known classification machines employing the ADNI database devoted to assessing the combined prognostic value of several AD biomarkers. The obtained experimental results show that our proposed method of Instance-based representation using multiple kernel learning enables detecting mild cognitive impairment as well as predicting conversion to Alzheimers disease within three years from the initial screening. Besides, the brain structures with larger combination weights are directly related to memory and cognitive functions.
Collapse
Affiliation(s)
- D. Collazos-Huertas
- Signal Processing and Recognition Group, Universidad Nacional de Colombia, Km 9 Vía al Aeropuerto la Nubia, Manizales, Colombia
| | - D. Cárdenas-Peña
- Signal Processing and Recognition Group, Universidad Nacional de Colombia, Km 9 Vía al Aeropuerto la Nubia, Manizales, Colombia
| | - G. Castellanos-Dominguez
- Signal Processing and Recognition Group, Universidad Nacional de Colombia, Km 9 Vía al Aeropuerto la Nubia, Manizales, Colombia
| |
Collapse
|
44
|
Dhifallah S, Rekik I. Clustering-based multi-view network fusion for estimating brain network atlases of healthy and disordered populations. J Neurosci Methods 2019; 311:426-435. [DOI: 10.1016/j.jneumeth.2018.09.028] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2018] [Revised: 09/11/2018] [Accepted: 09/25/2018] [Indexed: 01/11/2023]
|
45
|
Zhou T, Thung KH, Zhu X, Shen D. Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Hum Brain Mapp 2018; 40:1001-1016. [PMID: 30381863 DOI: 10.1002/hbm.24428] [Citation(s) in RCA: 110] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2017] [Revised: 09/04/2018] [Accepted: 10/03/2018] [Indexed: 12/13/2022] Open
Abstract
In this article, the authors aim to maximally utilize multimodality neuroimaging and genetic data for identifying Alzheimer's disease (AD) and its prodromal status, Mild Cognitive Impairment (MCI), from normal aging subjects. Multimodality neuroimaging data such as MRI and PET provide valuable insights into brain abnormalities, while genetic data such as single nucleotide polymorphism (SNP) provide information about a patient's AD risk factors. When these data are used together, the accuracy of AD diagnosis may be improved. However, these data are heterogeneous (e.g., with different data distributions), and have different number of samples (e.g., with far less number of PET samples than the number of MRI or SNPs). Thus, learning an effective model using these data is challenging. To this end, we present a novel three-stage deep feature learning and fusion framework, where deep neural network is trained stage-wise. Each stage of the network learns feature representations for different combinations of modalities, via effective training using the maximum number of available samples. Specifically, in the first stage, we learn latent representations (i.e., high-level features) for each modality independently, so that the heterogeneity among modalities can be partially addressed, and high-level features from different modalities can be combined in the next stage. In the second stage, we learn joint latent features for each pair of modality combination by using the high-level features learned from the first stage. In the third stage, we learn the diagnostic labels by fusing the learned joint latent features from the second stage. To further increase the number of samples during training, we also use data at multiple scanning time points for each training subject in the dataset. We evaluate the proposed framework using Alzheimer's disease neuroimaging initiative (ADNI) dataset for AD diagnosis, and the experimental results show that the proposed framework outperforms other state-of-the-art methods.
Collapse
Affiliation(s)
- Tao Zhou
- Department of Radiology and the Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, North Carolina
| | - Kim-Han Thung
- Department of Radiology and the Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, North Carolina
| | - Xiaofeng Zhu
- Department of Radiology and the Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, North Carolina
| | - Dinggang Shen
- Department of Radiology and the Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, North Carolina.,Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
| |
Collapse
|
46
|
Lei B, Yang P, Zhuo Y, Zhou F, Ni D, Chen S, Xiao X, Wang T. Neuroimaging Retrieval via Adaptive Ensemble Manifold Learning for Brain Disease Diagnosis. IEEE J Biomed Health Inform 2018; 23:1661-1673. [PMID: 30281500 DOI: 10.1109/jbhi.2018.2872581] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Alzheimer's disease (AD) is a neurodegenerative and non-curable disease, with serious cognitive impairment, such as dementia. Clinically, it is critical to study the disease with multi-source data in order to capture a global picture of it. In this respect, an adaptive ensemble manifold learning (AEML) algorithm is proposed to retrieve multi-source neuroimaging data. Specifically, an objective function based on manifold learning is formulated to impose geometrical constraints by similarity learning. The complementary characteristics of various sources of brain disease data for disorder discovery are investigated by tuning weights from ensemble learning. In addition, a generalized norm is explicitly explored for adaptive sparseness degree control. The proposed AEML algorithm is evaluated by the public AD neuroimaging initiative database. Results obtained from the extensive experiments demonstrate that our algorithm outperforms the traditional methods.
Collapse
|
47
|
Thung KH, Yap PT, Shen D. Joint Robust Imputation and Classification for Early Dementia Detection Using Incomplete Multi-modality Data. PREDICTIVE INTELLIGENCE IN MEDICINE. PRIME (WORKSHOP) 2018; 11121:51-59. [PMID: 34447978 PMCID: PMC8386184 DOI: 10.1007/978-3-030-00320-3_7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
It is vital to identify Mild Cognitive Impairment (MCI) subjects who will progress to Alzheimer's Disease (AD), so that early treatment can be administered. Recent studies show that using complementary information from multi-modality data may improve the model performance of the above prediction problem. However, multi-modality data is often incomplete, causing the prediction models that rely on complete data unusable. One way to deal with this issue is by first imputing the missing values, and then building a classifier based on the completed data. This two-step approach, however, may generate non-optimal classifier output, as the errors of the imputation may propagate to the classifier during training. To address this issue, we propose a unified framework that jointly performs feature selection, data denoising, missing values imputation, and classifier learning. To this end, we use a low-rank constraint to impute the missing values and denoise the data simultaneously, while using a regression model for feature selection and classification. The feature weights learned by the regression model are integrated into the low rank formulation to focus on discriminative features when denoising and imputing data, while the resulting low-rank matrix is used for classifier learning. These two components interact and correct each other iteratively using Alternating Direction Method of Multiplier (ADMM). The experimental results using incomplete multi-modality ADNI dataset shows that our proposed method outperforms other comparison methods.
Collapse
Affiliation(s)
- Kim-Han Thung
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - Pew-Thian Yap
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, USA
| |
Collapse
|
48
|
Abstract
Multi-modality data convey complementary information that can be used to improve the accuracy of prediction models in disease diagnosis. However, effectively integrating multi-modality data remains a challenging problem, especially when the data are incomplete. For instance, more than half of the subjects in the Alzheimer's disease neuroimaging initiative (ADNI) database have no fluorodeoxyglucose positron emission tomography and cerebrospinal fluid data. Currently, there are two commonly used strategies to handle the problem of incomplete data: 1) discard samples having missing features; and 2) impute those missing values via specific techniques. In the first case, a significant amount of useful information is lost and, in the second case, additional noise and artifacts might be introduced into the data. Also, previous studies generally focus on the pairwise relationships among subjects, without considering their underlying complex (e.g., high-order) relationships. To address these issues, in this paper, we propose a multi-hypergraph learning method for dealing with incomplete multimodality data. Specifically, we first construct multiple hypergraphs to represent the high-order relationships among subjects by dividing them into several groups according to the availability of their data modalities. A hypergraph regularized transductive learning method is then applied to these groups for automatic diagnosis of brain diseases. Extensive evaluation of the proposed method using all subjects in the baseline ADNI database indicates that our method achieves promising results in AD/MCI classification, compared with the state-of-the-art methods.
Collapse
Affiliation(s)
- Mingxia Liu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina 27599, USA
| | - Yue Gao
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina 27599, USA
| | - Pew-Thian Yap
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina 27599, USA
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, North Carolina 27599, USA
- Department of Brain and Cognitive Engineering, Korea University, Seoul 02841, Republic of Korea
| |
Collapse
|
49
|
Islam J, Zhang Y. Brain MRI analysis for Alzheimer's disease diagnosis using an ensemble system of deep convolutional neural networks. Brain Inform 2018; 5:2. [PMID: 29881892 PMCID: PMC6170939 DOI: 10.1186/s40708-018-0080-3] [Citation(s) in RCA: 104] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2018] [Accepted: 04/18/2018] [Indexed: 01/11/2023] Open
Abstract
Alzheimer's disease is an incurable, progressive neurological brain disorder. Earlier detection of Alzheimer's disease can help with proper treatment and prevent brain tissue damage. Several statistical and machine learning models have been exploited by researchers for Alzheimer's disease diagnosis. Analyzing magnetic resonance imaging (MRI) is a common practice for Alzheimer's disease diagnosis in clinical research. Detection of Alzheimer's disease is exacting due to the similarity in Alzheimer's disease MRI data and standard healthy MRI data of older people. Recently, advanced deep learning techniques have successfully demonstrated human-level performance in numerous fields including medical image analysis. We propose a deep convolutional neural network for Alzheimer's disease diagnosis using brain MRI data analysis. While most of the existing approaches perform binary classification, our model can identify different stages of Alzheimer's disease and obtains superior performance for early-stage diagnosis. We conducted ample experiments to demonstrate that our proposed model outperformed comparative baselines on the Open Access Series of Imaging Studies dataset.
Collapse
Affiliation(s)
- Jyoti Islam
- Department of Computer Science, Georgia State University, Atlanta, GA 30302-5060 USA
| | - Yanqing Zhang
- Department of Computer Science, Georgia State University, Atlanta, GA 30302-5060 USA
| |
Collapse
|
50
|
Adeli E, Meng Y, Li G, Lin W, Shen D. Multi-task prediction of infant cognitive scores from longitudinal incomplete neuroimaging data. Neuroimage 2018; 185:783-792. [PMID: 29709627 DOI: 10.1016/j.neuroimage.2018.04.052] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2017] [Revised: 03/26/2018] [Accepted: 04/23/2018] [Indexed: 01/13/2023] Open
Abstract
Early postnatal brain undergoes a stunning period of development. Over the past few years, research on dynamic infant brain development has received increased attention, exhibiting how important the early stages of a child's life are in terms of brain development. To precisely chart the early brain developmental trajectories, longitudinal studies with data acquired over a long-enough period of infants' early life is essential. However, in practice, missing data from different time point(s) during the data gathering procedure is often inevitable. This leads to incomplete set of longitudinal data, which poses a major challenge for such studies. In this paper, prediction of multiple future cognitive scores with incomplete longitudinal imaging data is modeled into a multi-task machine learning framework. To efficiently learn this model, we account for selection of informative features (i.e., neuroimaging morphometric measurements for different time points), while preserving the structural information and the interrelation between these multiple cognitive scores. Several experiments are conducted on a carefully acquired in-house dataset, and the results affirm that we can predict the cognitive scores measured at the age of four years old, using the imaging data of earlier time points, as early as 24 months of age, with a reasonable performance (i.e., root mean square error of 0.18).
Collapse
Affiliation(s)
- Ehsan Adeli
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC 27599, United States; Department of Psychiatry & Behavioral Sciences, Stanford University, Stanford, CA 94305, United States.
| | - Yu Meng
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC 27599, United States; Department of Computer Science, University of North Carolina at Chapel Hill, NC 27599, United States
| | - Gang Li
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC 27599, United States
| | - Weili Lin
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC 27599, United States
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC 27599, United States; Department of Brain & Cognitive Eng, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|