Ensemble Manifold Regularized Multi-Modal Graph Convolutional Network for Cognitive Ability Prediction

Deep learning-based flocculation sensor for automatic control of flocculant dose in sludge dewatering processes during wastewater treatment

In sludge dewatering of most wastewater treatment plants (WWTPs), the dose of polymer flocculant is manually adjusted through direct visual inspection of the flocs without the aid of any instruments. Although there is a demand for the development of automatic control of flocculant dosing, this has been challenging owing to the lack of a reliable flocculation sensor. To address this issue, this study developed a novel image sensor for measuring the degree of flocculation (DF) based on deep learning. Two types of sludge samples were used in the laboratory-scale flocculation tests: excess sludge and mixtures of excess sludge and raw wastewater. To search for a deep learning regression model suitable for DF inference, ten models, including convolutional neural networks, vision transformers, and a multilayer perceptron MLP mixer, were comparatively analysed. The ConvNeXt and MLP mixer models showed the highest accuracies with coefficients of determination (R2) greater than 0.9. The region contributing to the DF inference in the flocculation images was visualised using a modulus-weighted, gradient-weighted regression activation map. A prototype of the flocculation sensor was constructed using an inexpensive EdgeAI device. This device comprises a single-board computer and an integrated graphical processing unit (GPU) and is equipped with a quantised ConvNeXt model. The maximum inference speed of the sensor was 12.8 frames per second (FPS). The flocculation control tests showed that the sensor could control the DF to the target value by adjusting the polymer flocculant dose. Therefore, the flocculation sensor can provide a data-driven approach to the management of the flocculation process, thereby facilitating the automation of WWTPs.

A Demographic-Conditioned Variational Autoencoder for fMRI Distribution Sampling and Removal of Confounds

Objective

fMRI and derived measures such as functional connectivity (FC) have been used to predict brain age, general fluid intelligence, psychiatric disease status, and preclinical neurodegenerative disease. However, it is not always clear that all demographic confounds, such as age, sex, and race, have been removed from fMRI data. Additionally, many fMRI datasets are restricted to authorized researchers, making dissemination of these valuable data sources challenging.

Methods

We create a variational autoencoder (VAE)-based model, DemoVAE, to decorrelate fMRI features from demographics and generate high-quality synthetic fMRI data based on user-supplied demographics. We train and validate our model using two large, widely used datasets, the Philadelphia Neurodevel-opmental Cohort (PNC) and Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP).

Results

We find that DemoVAE recapitulates group differences in fMRI data while capturing the full breadth of individual variations. Significantly, we also find that most clinical and computerized battery fields that are correlated with fMRI data are not correlated with DemoVAE latents. An exception are several fields related to schizophrenia medication and symptom severity.

Conclusion

Our model generates fMRI data that captures the full distribution of FC better than traditional VAE or GAN models. We also find that most prediction using fMRI data is dependent on correlation with, and prediction of, demographics.

Significance

Our DemoVAE model allows for generation of high quality synthetic data conditioned on subject demographics as well as the removal of the confounding effects of demographics. We identify that FC-based prediction tasks are highly influenced by demographic confounds.

Interpretable Cognitive Ability Prediction: A Comprehensive Gated Graph Transformer Framework for Analyzing Functional Brain Networks

Graph convolutional deep learning has emerged as a promising method to explore the functional organization of the human brain in neuroscience research. This paper presents a novel framework that utilizes the gated graph transformer (GGT) model to predict individuals' cognitive ability based on functional connectivity (FC) derived from fMRI. Our framework incorporates prior spatial knowledge and uses a random-walk diffusion strategy that captures the intricate structural and functional relationships between different brain regions. Specifically, our approach employs learnable structural and positional encodings (LSPE) in conjunction with a gating mechanism to efficiently disentangle the learning of positional encoding (PE) and graph embeddings. Additionally, we utilize the attention mechanism to derive multi-view node feature embeddings and dynamically distribute propagation weights between each node and its neighbors, which facilitates the identification of significant biomarkers from functional brain networks and thus enhances the interpretability of the findings. To evaluate our proposed model in cognitive ability prediction, we conduct experiments on two large-scale brain imaging datasets: the Philadelphia Neurodevelopmental Cohort (PNC) and the Human Connectome Project (HCP). The results show that our approach not only outperforms existing methods in prediction accuracy but also provides superior explainability, which can be used to identify important FCs underlying cognitive behaviors.

A Learnable Counter-Condition Analysis Framework for Functional Connectivity-Based Neurological Disorder Diagnosis

To understand the biological characteristics of neurological disorders with functional connectivity (FC), recent studies have widely utilized deep learning-based models to identify the disease and conducted post-hoc analyses via explainable models to discover disease-related biomarkers. Most existing frameworks consist of three stages, namely, feature selection, feature extraction for classification, and analysis, where each stage is implemented separately. However, if the results at each stage lack reliability, it can cause misdiagnosis and incorrect analysis in afterward stages. In this study, we propose a novel unified framework that systemically integrates diagnoses (i.e., feature selection and feature extraction) and explanations. Notably, we devised an adaptive attention network as a feature selection approach to identify individual-specific disease-related connections. We also propose a functional network relational encoder that summarizes the global topological properties of FC by learning the inter-network relations without pre-defined edges between functional networks. Last but not least, our framework provides a novel explanatory power for neuroscientific interpretation, also termed counter-condition analysis. We simulated the FC that reverses the diagnostic information (i.e., counter-condition FC): converting a normal brain to be abnormal and vice versa. We validated the effectiveness of our framework by using two large resting-state functional magnetic resonance imaging (fMRI) datasets, Autism Brain Imaging Data Exchange (ABIDE) and REST-meta-MDD, and demonstrated that our framework outperforms other competing methods for disease identification. Furthermore, we analyzed the disease-related neurological patterns based on counter-condition analysis.

Somatomotor-visual resting state functional connectivity increases after 2 years in the UK Biobank longitudinal cohort

Purpose

Functional magnetic resonance imaging (fMRI) and functional connectivity (FC) have been used to follow aging in both children and older adults. Robust changes have been observed in children, in which high connectivity among all brain regions changes to a more modular structure with maturation. We examine FC changes in older adults after 2 years of aging in the UK Biobank (UKB) longitudinal cohort.

Approach

We process fMRI connectivity data using the Power264 atlas and then test whether the average internetwork FC changes in the 2722-subject longitudinal cohort are statistically significant using a Bonferroni-corrected t -test. We also compare the ability of Power264 and UKB-provided, independent component analysis (ICA)-based FC to determine which of a longitudinal scan pair is older. Finally, we investigate cross-sectional FC changes as well as differences due to differing scanner tasks in the UKB, Philadelphia Neurodevelopmental Cohort, and Alzheimer's Disease Neuroimaging Initiative datasets.

Results

We find a 6.8% average increase in somatomotor network (SMT)-visual network (VIS) connectivity from younger to older scans (corrected p < 10 - 15 ) that occurs in male, female, older subject (> 65 years old), and younger subject (< 55 years old) groups. Among all internetwork connections, the average SMT-VIS connectivity is the best predictor of relative scan age. Using the full FC and a training set of 2000 subjects, one is able to predict which scan is older 82.5% of the time using either the full Power264 FC or the UKB-provided ICA-based FC.

Conclusions

We conclude that SMT-VIS connectivity increases with age in the UKB longitudinal cohort and that resting state FC increases with age in the UKB cross-sectional cohort.

Carbon Dioxide Concentration Estimation in Nonuniform Temperature Fields Based on Single-Pass Tunable Diode Laser Absorption Spectroscopy

We propose a novel technique to accurately predict carbon dioxide (CO2) concentrations even in flow fields with temperature gradients based on a single laser path absorption spectrum measurement and machine learning. Concentration measurements in typical tunable diode laser absorption spectroscopy are based on a ratio of two integrated absorbances, each from a spectral line with different temperature dependence. However, the inferred concentrations can deviate significantly from the actual concentrations in the presence of temperature gradients. Furthermore, it is also difficult to find an analytical expression to compensate for the effect of nonuniform temperature profiles on concentration measurements. In this study, the entire absorption feature was considered since its shape and peak intensities vary with temperature and concentration. Specifically, a predictive model is obtained in a data-driven manner that can identify and compensate for the effect of a nonuniform temperature field on the spectrum. Despite a very detailed understanding of the CO2 absorption spectrum, it is nearly impossible to collect sufficient spectra for model acquisition by varying all temperature gradient conditions. Therefore, the model was obtained using only simulated data, much like the concept of a "digital twin". Finally, the predictive performance of the acquired model was verified using experimental data. In all test cases, the predictive performance of the model was superior to that of the two-line method. Additionally, a gradient-weighted regression activation mapping analysis confirmed that the model utilizes both the peak intensities as well as the change in the shape of absorption lines for prediction.

Somatomotor-Visual Resting State Functional Connectivity Increases After Two Years in the UK Biobank Longitudinal Cohort

Purpose

Functional magnetic resonance imaging (fMRI) and functional connectivity (FC) have been used to follow aging in both children and older adults. Robust changes have been observed in children, where high connectivity among all brain regions changes to a more modular structure with maturation. In this work, we examine changes in FC in older adults after two years of aging in the UK Biobank longitudinal cohort.

Approach

We process data using the Power264 atlas, then test whether FC changes in the 2,722-subject longitudinal cohort are statistically significant using a Bonferroni-corrected t-test. We also compare the ability of Power264 and UKB-provided, ICA-based FC to determine which of a longitudinal scan pair is older.

Results

We find a 6.8% average increase in SMT-VIS connectivity from younger to older scan (from ρ = 0.39 to ρ = 0.42 ) that occurs in male, female, older subject (> 65 years old), and younger subject (< 55 years old) groups. Among all inter-network connections, this average SMT-VIS connectivity is the best predictor of relative scan age, accurately predicting which scan is older 57% of the time. Using the full FC and a training set of 2,000 subjects, one is able to predict which scan is older 82.5% of the time using either the full Power264 FC or the UKB-provided ICA-based FC.

Conclusions

We conclude that SMT-VIS connectivity increases in the longitudinal cohort, while resting state FC increases generally with age in the cross-sectional cohort. However, we consider the possibility of a change in resting state scanner task between UKB longitudinal data acquisitions.

Dynamic weighted hypergraph convolutional network for brain functional connectome analysis

The hypergraph structure has been utilized to characterize the brain functional connectome (FC) by capturing the high order relationships among multiple brain regions of interest (ROIs) compared with a simple graph. Accordingly, hypergraph neural network (HGNN) models have emerged and provided efficient tools for hypergraph embedding learning. However, most existing HGNN models can only be applied to pre-constructed hypergraphs with a static structure during model training, which might not be a sufficient representation of the complex brain networks. In this study, we propose a dynamic weighted hypergraph convolutional network (dwHGCN) framework to consider a dynamic hypergraph with learnable hyperedge weights. Specifically, we generate hyperedges based on sparse representation and calculate the hyper similarity as node features. The hypergraph and node features are fed into a neural network model, where the hyperedge weights are updated adaptively during training. The dwHGCN facilitates the learning of brain FC features by assigning larger weights to hyperedges with higher discriminative power. The weighting strategy also improves the interpretability of the model by identifying the highly active interactions among ROIs shared by a common hyperedge. We validate the performance of the proposed model on two classification tasks with three paradigms functional magnetic resonance imaging (fMRI) data from Philadelphia Neurodevelopmental Cohort. Experimental results demonstrate the superiority of our proposed method over existing hypergraph neural networks. We believe our model can be applied to other applications in neuroimaging for its strength in representation learning and interpretation.

Latent Similarity Identifies Important Functional Connections for Phenotype Prediction

OBJECTIVE

Endophenotypes such as brain age and fluid intelligence are important biomarkers of disease status. However, brain imaging studies to identify these biomarkers often encounter limited numbers of subjects but high dimensional imaging features, hindering reproducibility. Therefore, we develop an interpretable, multivariate classification/regression algorithm, called Latent Similarity (LatSim), suitable for small sample size but high feature dimension datasets.

METHODS

LatSim combines metric learning with a kernel similarity function and softmax aggregation to identify task-related similarities between subjects. Inter-subject similarity is utilized to improve performance on three prediction tasks using multi-paradigm fMRI data. A greedy selection algorithm, made possible by LatSim's computational efficiency, is developed as an interpretability method.

RESULTS

LatSim achieved significantly higher predictive accuracy at small sample sizes on the Philadelphia Neurodevelopmental Cohort (PNC) dataset. Connections identified by LatSim gave superior discriminative power compared to those identified by other methods. We identified 4 functional brain networks enriched in connections for predicting brain age, sex, and intelligence.

CONCLUSION

We find that most information for a predictive task comes from only a few (1-5) connections. Additionally, we find that the default mode network is over-represented in the top connections of all predictive tasks.

SIGNIFICANCE

We propose a novel prediction algorithm for small sample, high feature dimension datasets and use it to identify connections in task fMRI data. Our work can lead to new insights in both algorithm design and neuroscience research.

Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

Advances in artificial intelligence (AI), especially deep learning (DL), have facilitated magnetic resonance imaging (MRI) data analysis, enabling AI-assisted medical image diagnoses and prognoses. However, most of the DL models are considered as "black boxes". There is an unmet need to demystify DL models so domain experts can trust these high-performance DL models. This has resulted in a sub-domain of AI research called explainable artificial intelligence (XAI). In the last decade, many experts have dedicated their efforts to developing novel XAI methods that are competent at visualizing and explaining the logic behind data-driven DL models. However, XAI techniques are still in their infancy for medical MRI image analysis. This study aims to outline the XAI applications that are able to interpret DL models for MRI data analysis. We first introduce several common MRI data modalities. Then, a brief history of DL models is discussed. Next, we highlight XAI frameworks and elaborate on the principles of multiple popular XAI methods. Moreover, studies on XAI applications in MRI image analysis are reviewed across the tissues/organs of the human body. A quantitative analysis is conducted to reveal the insights of MRI researchers on these XAI techniques. Finally, evaluations of XAI methods are discussed. This survey presents recent advances in the XAI domain for explaining the DL models that have been utilized in MRI applications.

Simultaneous measurement of carbon emission and gas temperature via laser-induced breakdown spectroscopy coupled with machine learning

A method, which can accurately measure carbon emission and gas temperature simultaneously in real-time from a laser-induced breakdown spectrum (LIBS) via machine learning, is proposed in this study. In typical, peak intensity ratios had been used to map species concentrations prior to plasma formation, after removing the broadband continuum of the spectrum; however, the dependence of these peak intensity ratios on the concentration changes with the change in gas density. Therefore, considering the fact that the strength and shape of this broadband continuum is a function of the gas density for a given optical setup, we attempted to collect a spectrum by shortening the time delay after the laser fire, such that the spectrum can contain some of the broadband continuum. Since the analytical quantification of this broadband continuum is not trivial, we employed a machine learning approach to acquire a model that simultaneously predicts the gas temperature and CO₂ concentration. The predictive performance of the model trained with spectra that contain the broadband continuum was much better than that without it; the gradient-weighted regression activation mapping (Grad-RAM) analysis revealed that the model utilizes the broadband spectrum for temperature prediction and correction of changes in peak intensity due to temperature changes in the concentration prediction process.

An explainable autoencoder with multi-paradigm fMRI fusion for identifying differences in dynamic functional connectivity during brain development

Multi-paradigm deep learning models show great potential for dynamic functional connectivity (dFC) analysis by integrating complementary information. However, many of them cannot use information from different paradigms effectively and have poor explainability, that is, the ability to identify significant features that contribute to decision making. In this paper, we propose a multi-paradigm fusion-based explainable deep sparse autoencoder (MF-EDSAE) to address these issues. Considering explainability, the MF-EDSAE is constructed based on a deep sparse autoencoder (DSAE). For integrating information effectively, the MF-EDASE contains the nonlinear fusion layer and multi-paradigm hypergraph regularization. We apply the model to the Philadelphia Neurodevelopmental Cohort and demonstrate it achieves better performance in detecting dynamic FC (dFC) that differ significantly during brain development than the single-paradigm DSAE. The experimental results show that children have more dispersive dFC patterns than adults. The function of the brain transits from undifferentiated systems to specialized networks during brain development. Meanwhile, adults have stronger connectivities between task-related functional networks for a given task than children. As the brain develops, the patterns of the global dFC change more quickly when stimulated by a task.

Tackling the Accuracy–Interpretability Trade-off: Interpretable Deep Learning Models for Satellite Image-based Real Estate Appraisal

Deep learning models fuel many modern decision support systems, because they typically provide high predictive performance. Among other domains, deep learning is used in real-estate appraisal, where it allows to extend the analysis from hard facts only (e.g., size, age) to also consider more implicit information about the location or appearance of houses in the form of image data. However, one downside of deep learning models is their intransparent mechanic of decision making, which leads to a trade-off between accuracy and interpretability. This limits their applicability for tasks where a justification of the decision is necessary. Therefore, in this paper, we first combine different perspectives on interpretability into a multi-dimensional framework for a socio-technical perspective on explainable artificial intelligence. Second, we measure the performance gains of using multi-view deep learning which leverages additional image data (satellite images) for real estate appraisal. Third, we propose and test a novel post-hoc explainability method called Grad-Ram. This modified version of Grad-Cam mitigates the intransparency of convolutional neural networks (CNNs) for predicting continuous outcome variables. With this, we try to reduce the accuracy-interpretability trade-off of multi-view deep learning models. Our proposed network architecture outperforms traditional hedonic regression models by 34% in terms of MAE. Furthermore, we find that the used satellite images are the second most important predictor after square feet in our model and that the network learns interpretable patterns about the neighborhood structure and density.

Multi-Modal Imaging Genetics Data Fusion via a Hypergraph-Based Manifold Regularization: Application to Schizophrenia Study

Recent studies show that multi-modal data fusion techniques combine information from diverse sources for comprehensive diagnosis and prognosis of complex brain disorder, often resulting in improved accuracy compared to single-modality approaches. However, many existing data fusion methods extract features from homogeneous networs, ignoring heterogeneous structural information among multiple modalities. To this end, we propose a Hypergraph-based Multi-modal data Fusion algorithm, namely HMF. Specifically, we first generate a hypergraph similarity matrix to represent the high-order relationships among subjects, and then enforce the regularization term based upon both the inter- and intra-modality relationships of the subjects. Finally, we apply HMF to integrate imaging and genetics datasets. Validation of the proposed method is performed on both synthetic data and real samples from schizophrenia study. Results show that our algorithm outperforms several competing methods, and reveals significant interactions among risk genes, environmental factors and abnormal brain regions.

An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders

It has been proven that neuropsychiatric disorders (NDs) can be associated with both structures and functions of brain regions. Thus, data about structures and functions could be usefully combined in a comprehensive analysis. While brain structural MRI (sMRI) images contain anatomic and morphological information about NDs, functional MRI (fMRI) images carry complementary information. However, efficient extraction and fusion of sMRI and fMRI data remains challenging. In this study, we develop an enhanced multi-modal graph convolutional network (MME-GCN) in a binary classification between patients with NDs and healthy controls, based on the fusion of the structural and functional graphs of the brain region. First, based on the same brain atlas, we construct structural and functional graphs from sMRI and fMRI data, respectively. Second, we use machine learning to extract important features from the structural graph network. Third, we use these extracted features to adjust the corresponding edge weights in the functional graph network. Finally, we train a multi-layer GCN and use it in binary classification task. MME-GCN achieved 93.71% classification accuracy on the open data set provided by the Consortium for Neuropsychiatric Phenomics. In addition, we analyzed the important features selected from the structural graph and verified them in the functional graph. Using MME-GCN, we found several specific brain connections important to NDs.

DeepPN: a deep parallel neural network based on convolutional neural network and graph convolutional network for predicting RNA-protein binding sites

Background

Addressing the laborious nature of traditional biological experiments by using an efficient computational approach to analyze RNA-binding proteins (RBPs) binding sites has always been a challenging task. RBPs play a vital role in post-transcriptional control. Identification of RBPs binding sites is a key step for the anatomy of the essential mechanism of gene regulation by controlling splicing, stability, localization and translation. Traditional methods for detecting RBPs binding sites are time-consuming and computationally-intensive. Recently, the computational method has been incorporated in researches of RBPs. Nevertheless, lots of them not only rely on the sequence data of RNA but also need additional data, for example the secondary structural data of RNA, to improve the performance of prediction, which needs the pre-work to prepare the learnable representation of structural data.

Results

To reduce the dependency of those pre-work, in this paper, we introduce DeepPN, a deep parallel neural network that is constructed with a convolutional neural network (CNN) and graph convolutional network (GCN) for detecting RBPs binding sites. It includes a two-layer CNN and GCN in parallel to extract the hidden features, followed by a fully connected layer to make the prediction. DeepPN discriminates the RBP binding sites on learnable representation of RNA sequences, which only uses the sequence data without using other data, for example the secondary or tertiary structure data of RNA. DeepPN is evaluated on 24 datasets of RBPs binding sites with other state-of-the-art methods. The results show that the performance of DeepPN is comparable to the published methods.

Conclusion

The experimental results show that DeepPN can effectively capture potential hidden features in RBPs and use these features for effective prediction of binding sites.

Multimodal Brain Connectomics-Based Prediction of Parkinson’s Disease Using Graph Attention Networks

Background

A multimodal connectomic analysis using diffusion and functional MRI can provide complementary information on the structure–function network dynamics involved in complex neurodegenerative network disorders such as Parkinson’s disease (PD). Deep learning-based graph neural network models generate higher-level embeddings that could capture intricate structural and functional regional interactions related to PD.

Objective

This study aimed at investigating the role of structure–function connections in predicting PD, by employing an end-to-end graph attention network (GAT) on multimodal brain connectomes along with an interpretability framework.

Methods

The proposed GAT model was implemented to generate node embeddings from the structural connectivity matrix and multimodal feature set containing morphological features and structural and functional network features of PD patients and healthy controls. Graph classification was performed by extracting topmost node embeddings, and the interpretability framework was implemented using saliency analysis and attention maps. Moreover, we also compared our model with unimodal models as well as other state-of-the-art models.

Results

Our proposed GAT model with a multimodal feature set demonstrated superior classification performance over a unimodal feature set. Our model demonstrated superior classification performance over other comparative models, with 10-fold CV accuracy and an F1 score of 86% and a moderate test accuracy of 73%. The interpretability framework highlighted the structural and functional topological influence of motor network and cortico-subcortical brain regions, among which structural features were correlated with onset of PD. The attention maps showed dependency between large-scale brain regions based on their structural and functional characteristics.

Conclusion

Multimodal brain connectomic markers and GAT architecture can facilitate robust prediction of PD pathology and provide an attention mechanism-based interpretability framework that can highlight the pathology-specific relation between brain regions.

Brain Functional Connectivity Analysis via Graphical Deep Learning

OBJECTIVE

Graphical deep learning models provide a desirable way for brain functional connectivity analysis. However, the application of current graph deep learning models to brain network analysis is challenging due to the limited sample size and complex relationships between different brain regions.

METHOD

In this work, a graph convolutional network (GCN) based framework is proposed by exploiting the information from both region-to-region connectivities of the brain and subject-subject relationships. We first construct an affinity subject-subject graph followed by GCN analysis. A Laplacian regularization term is introduced in our model to tackle the overfitting problem. We apply and validate the proposed model to the Philadelphia Neurodevelopmental Cohort for the brain cognition study.

RESULTS

Experimental analysis shows that our proposed framework outperforms other competing models in classifying groups with low and high Wide Range Achievement Test (WRAT) scores. Moreover, to examine each brain region's contribution to cognitive function, we use the occlusion sensitivity analysis method to identify cognition-related brain functional networks. The results are consistent with previous research yet yield new findings.

CONCLUSION AND SIGNIFICANCE

Our study demonstrates that GCN incorporating prior knowledge about brain networks offers a powerful way to detect important brain networks and regions associated with cognitive functions.