The NMT Scalp EEG Dataset: An Open-Source Annotated Dataset of Healthy and Pathological EEG Recordings for Predictive Modeling

Concept-based AI interpretability in physiological time-series data: Example of abnormality detection in electroencephalography

BACKGROUND AND OBJECTIVE

Despite recent performance advancements, deep learning models are not yet adopted in clinical practice on a wide scale. The intrinsic intransparency of such systems is commonly cited as one major reason for this reluctance. This has motivated methods that aim to provide explanations of model functioning. Known limitations of feature-based explanations have led to an increased interest in concept-based interpretability. Testing with Concept Activation Vectors (TCAV) employs human-understandable, abstract concepts to explain model behavior. The method has previously been applied to the medical domain in the context of electronic health records, retinal fundus images and magnetic resonance imaging.

METHODS

We explore the usage of TCAV for building interpretable models on physiological time series, using an example of abnormality detection in electroencephalography (EEG). For this purpose, we adopt the XceptionTime model, which is suitable for multi-channel physiological data of variable sizes. The model provides state-of-the-art performance on raw EEG data and is publically available. We propose and test several ideas regarding concept definition through metadata mining, using additional labeled EEG data and extracting interpretable signal characteristics in the form of frequencies. By including our own hospital data with analog labeling, we further evaluate the robustness of our approach.

RESULTS

The tested concepts show a TCAV score distribution that is in line with the clinical expectations, i.e. concepts known to have strong links with EEG pathologies (such as epileptiform discharges) received higher scores than the neutral concepts (e.g. sex). The scores were consistent across the applied concept generation strategies.

CONCLUSIONS

TCAV has the potential to improve interpretability of deep learning applied to multi-channel signals as well as to detect possible biases in the data. Still, further work on developing the strategies for concept definition and validation on clinical physiological time series is needed to better understand how to extract clinically relevant information from the concept sensitivity scores.

Reaching the ceiling? Empirical scaling behaviour for deep EEG pathology classification

Machine learning techniques, particularly deep convolutional neural networks (ConvNets), are increasingly being used to automate clinical EEG analysis, with the potential to reduce the clinical burden and improve patient care. However, further research is required before they can be used in clinical settings, particularly regarding the impact of the number of training samples and model parameters on their testing error. To address this, we present a comprehensive study of the empirical scaling behaviour of ConvNets for EEG pathology classification. We analysed the testing error with increasing the training samples and model size for four different ConvNet architectures. The focus of our experiments is width scaling, and we have increased the number of parameters to up to 1.8 million. Our evaluation was based on two publicly available datasets: the Temple University Hospital (TUH) Abnormal EEG Corpus and the TUH Abnormal Expansion Balanced EEG Corpus, which together contain 10,707 training samples. The results show that the testing error follows a saturating power-law with both model and dataset size. This pattern is consistent across different datasets and ConvNet architectures. Furthermore, empirically observed accuracies saturate at 85%-87%, which may be due to an imperfect inter-rater agreement on the clinical labels. The empirical scaling behaviour of the test performance with dataset and model size has significant implications for deep EEG pathology classification research and practice. Our findings highlight the potential of deep ConvNets for high-performance EEG pathology classification, and the identified scaling relationships provide valuable recommendations for the advancement of automated EEG diagnostics.

Amplifying pathological detection in EEG signaling pathways through cross-dataset transfer learning

Pathology diagnosis based on EEG signals and decoding brain activity holds immense importance in understanding neurological disorders. With the advancement of artificial intelligence methods and machine learning techniques, the potential for accurate data-driven diagnoses and effective treatments has grown significantly. However, applying machine learning algorithms to real-world datasets presents diverse challenges at multiple levels. The scarcity of labeled data, especially in low regime scenarios with limited availability of real patient cohorts due to high costs of recruitment, underscores the vital deployment of scaling and transfer learning techniques. In this study, we explore a real-world pathology classification task to highlight the effectiveness of data and model scaling and cross-dataset knowledge transfer. As such, we observe varying performance improvements through data scaling, indicating the need for careful evaluation and labeling. Additionally, we identify the challenges of possible negative transfer and emphasize the significance of some key components to overcome distribution shifts and potential spurious correlations and achieve positive transfer. We see improvement in the performance of the target model on the target (NMT) datasets by using the knowledge from the source dataset (TUAB) when a low amount of labeled data was available. Our findings demonstrated that a small and generic model (e.g. ShallowNet) performs well on a single dataset, however, a larger model (e.g. TCN) performs better in transfer learning when leveraging a larger and more diverse dataset.

The NWRD Dataset: An Open-Source Annotated Segmentation Dataset of Diseased Wheat Crop

Wheat stripe rust disease (WRD) is extremely detrimental to wheat crop health, and it severely affects the crop yield, increasing the risk of food insecurity. Manual inspection by trained personnel is carried out to inspect the disease spread and extent of damage to wheat fields. However, this is quite inefficient, time-consuming, and laborious, owing to the large area of wheat plantations. Artificial intelligence (AI) and deep learning (DL) offer efficient and accurate solutions to such real-world problems. By analyzing large amounts of data, AI algorithms can identify patterns that are difficult for humans to detect, enabling early disease detection and prevention. However, deep learning models are data-driven, and scarcity of data related to specific crop diseases is one major hindrance in developing models. To overcome this limitation, in this work, we introduce an annotated real-world semantic segmentation dataset named the NUST Wheat Rust Disease (NWRD) dataset. Multileaf images from wheat fields under various illumination conditions with complex backgrounds were collected, preprocessed, and manually annotated to construct a segmentation dataset specific to wheat stripe rust disease. Classification of WRD into different types and categories is a task that has been solved in the literature; however, semantic segmentation of wheat crops to identify the specific areas of plants and leaves affected by the disease remains a challenge. For this reason, in this work, we target semantic segmentation of WRD to estimate the extent of disease spread in wheat fields. Sections of fields where the disease is prevalent need to be segmented to ensure that the sick plants are quarantined and remedial actions are taken. This will consequently limit the use of harmful fungicides only on the targeted disease area instead of the majority of wheat fields, promoting environmentally friendly and sustainable farming solutions. Owing to the complexity of the proposed NWRD segmentation dataset, in our experiments, promising results were obtained using the UNet semantic segmentation model and the proposed adaptive patching with feedback (APF) technique, which produced a precision of 0.506, recall of 0.624, and F1 score of 0.557 for the rust class.

An extended clinical EEG dataset with 15,300 automatically labelled recordings for pathology decoding

Automated clinical EEG analysis using machine learning (ML) methods is a growing EEG research area. Previous studies on binary EEG pathology decoding have mainly used the Temple University Hospital (TUH) Abnormal EEG Corpus (TUAB) which contains approximately 3,000 manually labelled EEG recordings. To evaluate and eventually even improve the generalisation performance of machine learning methods for EEG pathology, decoding larger, publicly available datasets is required. A number of studies addressed the automatic labelling of large open-source datasets as an approach to create new datasets for EEG pathology decoding, but little is known about the extent to which training on larger, automatically labelled dataset affects decoding performances of established deep neural networks. In this study, we automatically created additional pathology labels for the Temple University Hospital (TUH) EEG Corpus (TUEG) based on the medical reports using a rule-based text classifier. We generated a dataset of 15,300 newly labelled recordings, which we call the TUH Abnormal Expansion EEG Corpus (TUABEX), and which is five times larger than the TUAB. Since the TUABEX contains more pathological (75%) than non-pathological (25%) recordings, we then selected a balanced subset of 8,879 recordings, the TUH Abnormal Expansion Balanced EEG Corpus (TUABEXB). To investigate how training on a larger, automatically labelled dataset affects the decoding performance of deep neural networks, we applied four established deep convolutional neural networks (ConvNets) to the task of pathological versus non-pathological classification and compared the performance of each architecture after training on different datasets. The results show that training on the automatically labelled TUABEXB dataset rather than training on the manually labelled TUAB dataset increases accuracies on TUABEXB and even for TUAB itself for some architectures. We argue that automatically labelling of large open-source datasets can be used to efficiently utilise the massive amount of EEG data stored in clinical archives. We make the proposed TUABEXB available open source and thus offer a new dataset for EEG machine learning research.

Automatic Detection of Abnormal EEG Signals Using WaveNet and LSTM

Neurological disorders have an extreme impact on global health, affecting an estimated one billion individuals worldwide. According to the World Health Organization (WHO), these neurological disorders contribute to approximately six million deaths annually, representing a significant burden. Early and accurate identification of brain pathological features in electroencephalogram (EEG) recordings is crucial for the diagnosis and management of these disorders. However, manual evaluation of EEG recordings is not only time-consuming but also requires specialized skills. This problem is exacerbated by the scarcity of trained neurologists in the healthcare sector, especially in low- and middle-income countries. These factors emphasize the necessity for automated diagnostic processes. With the advancement of machine learning algorithms, there is a great interest in automating the process of early diagnoses using EEGs. Therefore, this paper presents a novel deep learning model consisting of two distinct paths, WaveNet-Long Short-Term Memory (LSTM) and LSTM, for the automatic detection of abnormal raw EEG data. Through multiple ablation experiments, we demonstrated the effectiveness and importance of all parts of our proposed model. The performance of our proposed model was evaluated using TUH abnormal EEG Corpus V.2.0.0. (TUAB) and achieved a high classification accuracy of 88.76%, which is higher than in the existing state-of-the-art research studies. Moreover, we demonstrated the generalization of our proposed model by evaluating it on another independent dataset, TUEP, without any hyperparameter tuning or adjustment. The obtained accuracy was 97.45% for the classification between normal and abnormal EEG recordings, confirming the robustness of our proposed model.