An explainable multiscale LSTM model with wavelet transform and layer-wise relevance propagation for daily streamflow forecasting

Developing an accurate and reliable daily streamflow forecasting model is important for facilitating the efficient resource planning and management of hydrological systems. In this study, an explainable multiscale long short-term memory (XM-LSTM) model is proposed for effective daily streamflow forecasting by integrating the à trous wavelet transform (ATWT) for decomposing data, the Boruta algorithm for identifying model inputs, and the layer-wise relevance propagation (LRP) for explaining the prediction results. The proposed XM-LSTM is tested by performing multi-step-ahead forecasting of daily streamflow at four stations in the middle and lower reaches of the Yangtze River basin and compared with the X-LSTM. The X-LSTM is formed by coupling the long short-term memory (LSTM) with the LRP. For comparison, the inputs of these two models are identified by the Boruta selection algorithm. The results show that all models exhibit good ability to forecast daily streamflow, however, the prediction performance decreases as the lead time increases. The XM-LSTM provides a better forecasting performance than the X-LSTM, suggesting the ability of the ATWT to improve the LSTM for daily streamflow forecasting. Moreover, the correlation scores analysis by the LRP shows that the ATWT can extract useful information that influences the daily streamflow from the raw predictors, and the water level has the most significant contribution to streamflow prediction. Accordingly, the XM-LSTM model can be viewed as a potentially useful approach for increasing the accuracy and explainability of streamflow forecasting.

Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex

Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach - based on a convolutional neural network and an explanation technique - to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.

A new method applied for explaining the landing patterns: Interpretability analysis of machine learning

As one of many fundamental sports techniques, the landing maneuver is also frequently used in clinical injury screening and diagnosis. However, the landing patterns are different under different constraints, which will cause great difficulties for clinical experts in clinical diagnosis. Machine learning (ML) have been very successful in solving a variety of clinical diagnosis tasks, but they all have the disadvantage of being black boxes and rarely provide and explain useful information about the reasons for making a particular decision. The current work validates the feasibility of applying an explainable ML (XML) model constructed by Layer-wise Relevance Propagation (LRP) for landing pattern recognition in clinical biomechanics. This study collected 560 groups landing data. By incorporating these landing data into the XML model as input signals, the prediction results were interpreted based on the relevance score (RS) derived from LRP. The interpretation obtained from XML was evaluated comprehensively from the statistical perspective based on Statistical Parametric Mapping (SPM) and Effect Size. The RS has excellent statistical characteristics in the interpretation of landing patterns between classes, and also conforms to the clinical characteristics of landing pattern recognition. The current work highlights the applicability of XML methods that can not only satisfy the traditional decision problem between classes, but also largely solve the lack of transparency in landing pattern recognition. We provide a feasible framework for realizing interpretability of ML decision results in landing analysis, providing a methodological reference and solid foundation for future clinical diagnosis and biomechanical analysis.

Convolutional neural network based data interpretable framework for Alzheimer's treatment planning

Alzheimer's disease (AD) is a neurological disorder that predominantly affects the brain. In the coming years, it is expected to spread rapidly, with limited progress in diagnostic techniques. Various machine learning (ML) and artificial intelligence (AI) algorithms have been employed to detect AD using single-modality data. However, recent developments in ML have enabled the application of these methods to multiple data sources and input modalities for AD prediction. In this study, we developed a framework that utilizes multimodal data (tabular data, magnetic resonance imaging (MRI) images, and genetic information) to classify AD. As part of the pre-processing phase, we generated a knowledge graph from the tabular data and MRI images. We employed graph neural networks for knowledge graph creation, and region-based convolutional neural network approach for image-to-knowledge graph generation. Additionally, we integrated various explainable AI (XAI) techniques to interpret and elucidate the prediction outcomes derived from multimodal data. Layer-wise relevance propagation was used to explain the layer-wise outcomes in the MRI images. We also incorporated submodular pick local interpretable model-agnostic explanations to interpret the decision-making process based on the tabular data provided. Genetic expression values play a crucial role in AD analysis. We used a graphical gene tree to identify genes associated with the disease. Moreover, a dashboard was designed to display XAI outcomes, enabling experts and medical professionals to easily comprehend the prediction results.

Modeling biological individuality using machine learning: A study on human gait

Human gait is a complex and unique biological process that can offer valuable insights into an individual's health and well-being. In this work, we leverage a machine learning-based approach to model individual gait signatures and identify factors contributing to inter-individual variability in gait patterns. We provide a comprehensive analysis of gait individuality by (1) demonstrating the uniqueness of gait signatures in a large-scale dataset and (2) highlighting the gait characteristics that are most distinctive to each individual. We utilized the data from three publicly available datasets comprising 5368 bilateral ground reaction force recordings during level overground walking from 671 distinct healthy individuals. Our results show that individuals can be identified with a prediction accuracy of 99.3% by using the bilateral signals of all three ground reaction force components, with only 10 out of 1342 recordings in our test data being misclassified. This indicates that the combination of bilateral ground reaction force signals with all three components provides a more comprehensive and accurate representation of an individual's gait signature. The highest accuracy was achieved by (linear) Support Vector Machines (99.3%), followed by Random Forests (98.7%), Convolutional Neural Networks (95.8%), and Decision Trees (82.8%). The proposed approach provides a powerful tool to better understand biological individuality and has potential applications in personalized healthcare, clinical diagnosis, and therapeutic interventions.

Improving 3D convolutional neural network comprehensibility via interactive visualization of relevance maps: evaluation in Alzheimer's disease

Background

Although convolutional neural networks (CNNs) achieve high diagnostic accuracy for detecting Alzheimer’s disease (AD) dementia based on magnetic resonance imaging (MRI) scans, they are not yet applied in clinical routine. One important reason for this is a lack of model comprehensibility. Recently developed visualization methods for deriving CNN relevance maps may help to fill this gap as they allow the visualization of key input image features that drive the decision of the model. We investigated whether models with higher accuracy also rely more on discriminative brain regions predefined by prior knowledge.

Methods

We trained a CNN for the detection of AD in N = 663 T1-weighted MRI scans of patients with dementia and amnestic mild cognitive impairment (MCI) and verified the accuracy of the models via cross-validation and in three independent samples including in total N = 1655 cases. We evaluated the association of relevance scores and hippocampus volume to validate the clinical utility of this approach. To improve model comprehensibility, we implemented an interactive visualization of 3D CNN relevance maps, thereby allowing intuitive model inspection.

Results

Across the three independent datasets, group separation showed high accuracy for AD dementia versus controls (AUC ≥ 0.91) and moderate accuracy for amnestic MCI versus controls (AUC ≈ 0.74). Relevance maps indicated that hippocampal atrophy was considered the most informative factor for AD detection, with additional contributions from atrophy in other cortical and subcortical regions. Relevance scores within the hippocampus were highly correlated with hippocampal volumes (Pearson’s r ≈ −0.86, p < 0.001).

Conclusion

The relevance maps highlighted atrophy in regions that we had hypothesized a priori. This strengthens the comprehensibility of the CNN models, which were trained in a purely data-driven manner based on the scans and diagnosis labels. The high hippocampus relevance scores as well as the high performance achieved in independent samples support the validity of the CNN models in the detection of AD-related MRI abnormalities. The presented data-driven and hypothesis-free CNN modeling approach might provide a useful tool to automatically derive discriminative features for complex diagnostic tasks where clear clinical criteria are still missing, for instance for the differential diagnosis between various types of dementia.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13195-021-00924-2.

Identification of voxel-based texture abnormalities as new biomarkers for schizophrenia and major depressive patients using layer-wise relevance propagation on deep learning decisions

Non-segmented MRI brain images are used for the identification of new Magnetic Resonance Imaging (MRI) biomarkers able to differentiate between schizophrenic patients (SCZ), major depressive patients (MD) and healthy controls (HC). Brain texture measures such as entropy and contrast, capturing the neighboring variation of MRI voxel intensities, were computed and fed into deep learning technique for group classification. Layer-wise relevance was applied for the localization of the classification results. Texture feature map of non-segmented brain MRI scans were extracted from 141 SCZ, 103 MD and 238 HC. The gray level co-occurrence matrix (GLCM) was calculated on a voxel-by-voxel basis in a cube of voxels. Deep learning tested if texture feature map could predict diagnostic group membership of three classes under a binary classification (SCZ vs. HC, MD vs. HC, SCZ vs. MD). The method was applied in a repeated nested cross-validation scheme and cross-validated feature selection. The regions with the highest relevance (positive/negative) are presented. The method was applied on non-segmented images reducing the computation complexity and the error associated with segmentation process.

Novel relative relevance score for estimating brain connectivity from fMRI data using an explainable neural network approach

BACKGROUND

Functional integration or connectivity in brain is directional, non-linear as well as variable in time-lagged dependence. Deep neural networks (DNN) have become an indispensable tool everywhere, by learning higher levels of abstract and complex patterns from raw data. However, in neuroscientific community they generally work as black-boxes, leading to the explanation of results difficult and less intuitive. We aim to propose a brain-connectivity measure based on an explainable NN (xNN) approach.

NEW METHOD

We build a NN-based predictor for regression problem. Since we aim to determine the contribution/relevance of past data-point from one region i in the prediction of current data-point from another region j, i.e. the higher-order connectivity between two brain-regions, we employ layer-wise relevance propagation (Bach et al., 2015) (LRP, a method for explaining DNN predictions), which has not been done before to the best of our knowledge. Specifically, we propose a novel score depending on weights as a quantitative measure of connectivity, called as relative relevance score (xNN-RRS). The RRS is an intuitive and transparent score. We provide an interpretation of the trained NN weights with-respect-to the brain-connectivity.

RESULTS

Face validity of our approach is demonstrated with experiments on simulated data, over existing methods. We also demonstrate construct validity of xNN-RRS in a resting-state fMRI experiment.

COMPARISON

Our approach shows superior performance, in terms of accuracy and computational complexity, over existing state-of-the-art methods for brain-connectivity estimation.

CONCLUSION

The proposed method is promising to serve as a first post-hoc explainable NN-approach for brain-connectivity analysis in clinical applications.

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge.