Using causal discovery for feature selection in multivariate numerical time series

Gait classification for early detection and severity rating of Parkinson's disease based on hybrid signal processing and machine learning methods

Parkinson's disease (PD) is one of the cognitive degenerative disorders of the central nervous system that affects the motor system. Gait dysfunction represents the pathology of motor symptom while gait analysis provides clinicians with subclinical information reflecting subtle differences between PD patients and healthy controls (HCs). Currently neurologists usually assess several clinical manifestations of the PD patients and rate the severity level according to some established criteria. This is highly dependent on clinician's expertise which is subjective and ineffective. In the present study we address these issues by proposing a hybrid signal processing and machine learning based gait classification system for gait anomaly detection and severity rating of PD patients. Time series of vertical ground reaction force (VGRF) data are utilized to represent discriminant gait information. First, phase space of the VGRF is reconstructed, in which the properties associated with the nonlinear gait system dynamics are preserved. Then Shannon energy is used to extract the characteristic envelope of the phase space signal. Third, Shannon energy envelope is decomposed into high and low resonance components using dual Q-factor signal decomposition derived from tunable Q-factor wavelet transform. Note that the high Q-factor component consists largely of sustained oscillatory behavior, while the low Q-factor component consists largely of transients and oscillations that are not sustained. Fourth, variational mode decomposition is employed to decompose high and low resonance components into different intrinsic modes and provide representative features. Finally features are fed to five different types of machine learning based classifiers for the anomaly detection and severity rating of PD patients based on Hohen and Yahr (HY) scale. The effectiveness of this strategy is verified using a Physionet gait database consisting of 93 idiopathic PD patients and 73 age-matched asymptomatic HCs. When evaluated with 10-fold cross-validation method for early PD detection and severity rating, the highest classification accuracy is reported to be 98.20 % and 96.69 % , respectively, by using the support vector machine classifier. Compared with other state-of-the-art methods, the results demonstrate superior performance and support the validity of the proposed method.

Myocardial infarction detection using ITD, DWT and deterministic learning based on ECG signals

Nowadays, cardiovascular diseases (CVD) is one of the prime causes of human mortality, which has received tremendous and elaborative research interests regarding the prevention issue. Myocardial ischemia is a kind of CVD which will lead to myocardial infarction (MI). The diagnostic criterion of MI is supplemented with clinical judgement and several electrocardiographic (ECG) or vectorcardiographic (VCG) programs. However the visual inspection of ECG or VCG signals by cardiologists is tedious, laborious and subjective. To overcome such disadvantages, numerous MI detection techniques including signal processing and artificial intelligence tools have been developed. In this study, we propose a novel technique for automatic detection of MI based on disparity of cardiac system dynamics and synthesis of the standard 12-lead and Frank XYZ leads. First, 12-lead ECG signals are synthesized with Frank XYZ leads to build a hybrid 4-dimensional cardiac vector, which is decomposed into a series of proper rotation components (PRCs) by using the intrinsic time-scale decomposition (ITD) method. The novel cardiac vector may fully reflect the pathological alterations provoked by MI and may be correlated to the disparity of cardiac system dynamics between healthy and MI subjects. ITD is employed to measure the variability of cardiac vector and the first PRCs are extracted as predominant PRCs which contain most of the cardiac vector's energy. Second, four levels discrete wavelet transform with third-order Daubechies (db3) wavelet function is employed to decompose the predominant PRCs into different frequency bands, which combines with three-dimensional phase space reconstruction to derive features. The properties associated with the cardiac system dynamics are preserved. Since the frequency components above 40 Hz are lack of use in ECG analysis, in order to reduce the feature dimension, the advisable sub-band (D4) is selected for feature acquisition. Third, neural networks are then used to model, identify and classify cardiac system dynamics between normal (healthy) and MI cardiac vector signals. The difference of cardiac system dynamics between healthy control and MI cardiac vector is computed and used for the detection of MI based on a bank of estimators. Finally, experiments are carried out on the PhysioNet PTB database to assess the effectiveness of the proposed method, in which conventional 12-lead and Frank XYZ leads ECG signal fragments from 148 patients with MI and 52 healthy controls were extracted. By using the tenfold cross-validation style, the achieved average classification accuracy is reported to be 98.20%. Results verify the effectiveness of the proposed method which can serve as a potential candidate for the automatic detection of MI in the clinical application.

A novel plantar pressure analysis method to signify gait dynamics in Parkinson's disease

Plantar pressure can signify the gait performance of patients with Parkinson's disease (PD). This study proposed a plantar pressure analysis method with the dynamics feature of the sub-regions plantar pressure signals. Specifically, each side's plantar pressure signals were divided into five sub-regions. Moreover, a dynamics feature extractor (DFE) was designed to extract features of the sub-regions signals. The radial basis function neural network (RBFNN) was used to learn and store gait dynamics. And a classification mechanism based on the output error in RBFNN was proposed. The classification accuracy of the proposed method achieved 100.00% in PD diagnosis and 95.89% in severity assessment on the online dataset, and 96.00% in severity assessment on our dataset. The experimental results suggested that the proposed method had the capability to signify the gait dynamics of PD patients.

Online Causal Feature Selection for Streaming Features

Recently, causal feature selection (CFS) has attracted considerable attention due to its outstanding interpretability and predictability performance. Such a method primarily includes the Markov blanket (MB) discovery and feature selection based on Granger causality. Representatively, the max-min MB (MMMB) can mine an optimal feature subset, i.e., MB; however, it is unsuitable for streaming features. Online streaming feature selection (OSFS) via online process streaming features can determine parents and children (PC), a subset of MB; however, it cannot mine the MB of the target attribute ( T ), i.e., a given feature, thus resulting in insufficient prediction accuracy. The Granger selection method (GSM) establishes a causal matrix of all features by performing excessively time; however, it cannot achieve a high prediction accuracy and only forecasts fixed multivariate time series data. To address these issues, we proposed an online CFS for streaming features (OCFSSFs) that mine MB containing PC and spouse and adopt the interleaving PC and spouse learning method. Furthermore, it distinguishes between PC and spouse in real time and can identify children with parents online when identifying spouses. We experimentally evaluated the proposed algorithm on synthetic datasets using precision, recall, and distance. In addition, the algorithm was tested on real-world and time series datasets using classification precision, the number of selected features, and running time. The results validated the effectiveness of the proposed algorithm.

Granger causality using Jacobian in neural networks

Granger causality is a commonly used method for uncovering information flow and dependencies in a time series. Here, we introduce JGC (Jacobian Granger causality), a neural network-based approach to Granger causality using the Jacobian as a measure of variable importance, and propose a variable selection procedure for inferring Granger causal variables with this measure, using criteria of significance and consistency. The resulting approach performs consistently well compared to other approaches in identifying Granger causal variables, the associated time lags, as well as interaction signs. In addition, we also discuss the need for contemporaneous variables in Granger causal modeling as well as how these neural network-based approaches reduce the impact of nonseparability in dynamical systems, a problem where predictive information on a target variable is not unique to its causes, but also contained in the history of the target variable itself.

Dimensionality reduction and ensemble of LSTMs for antimicrobial resistance prediction

Bacterial resistance to antibiotics has been rapidly increasing, resulting in low antibiotic effectiveness even treating common infections. The presence of resistant pathogens in environments such as a hospital Intensive Care Unit (ICU) exacerbates the critical admission-acquired infections. This work focuses on the prediction of antibiotic resistance in Pseudomonas aeruginosa nosocomial infections at the ICU, using Long Short-Term Memory (LSTM) artificial neural networks as the predictive method. The analyzed data were extracted from the Electronic Health Records (EHR) of patients admitted to the University Hospital of Fuenlabrada from 2004 to 2019 and were modeled as Multivariate Time Series. A data-driven dimensionality reduction method is built by adapting three feature importance techniques from the literature to the considered data and proposing an algorithm for selecting the most appropriate number of features. This is done using LSTM sequential capabilities so that the temporal aspect of features is taken into account. Furthermore, an ensemble of LSTMs is used to reduce the variance in performance. Our results indicate that the patient's admission information, the antibiotics administered during the ICU stay, and the previous antimicrobial resistance are the most important risk factors. Compared to other conventional dimensionality reduction schemes, our approach is able to improve performance while reducing the number of features for most of the experiments. In essence, the proposed framework achieve, in a computationally cost-efficient manner, promising results for supporting decisions in this clinical task, characterized by high dimensionality, data scarcity, and concept drift.

Assessment of Quarterly, Semiannual and Annual Models to Forecast Monthly Rainfall Anomalies: The Case of a Tropical Andean Basin

Rainfall forecasting is essential to manage water resources and make timely decisions to mitigate adverse effects related to unexpected events. Considering that rainfall drivers can change throughout the year, one approach to implementing forecasting models is to generate a model for each period in which the mechanisms are nearly constant, e.g., each season. The chosen predictors can be more robust, and the resulting models perform better. However, it has not been assessed whether the approach mentioned above offers better performance in forecasting models from a practical perspective in the tropical Andean region. This study evaluated quarterly, semiannual and annual models for forecasting monthly rainfall anomalies in an Andean basin to show if models implemented for fewer months outperform accuracy; all the models forecast rainfall on a monthly scale. Lagged rainfall and climate indices were used as predictors. Support vector regression (SVR) was used to select the most relevant predictors and train the models. The results showed a better performance of the annual models mainly due to the greater amount of data that SVR can take advantage of in training. If the training of the annual models had less data, the quarterly models would be the best. In conclusion, the annual models show greater accuracy in the rainfall forecast.

Discovering Ecological Relationships in Flowing Freshwater Ecosystems

Knowledge of ecological responses to changes in the environment is vital to design appropriate measures for conserving biodiversity. Experimental studies are the standard to identify ecological cause-effect relationships, but their results do not necessarily translate to field situations. Deriving ecological cause-effect relationships from observational field data is, however, challenging due to potential confounding influences of unmeasured variables. Here, we present a causal discovery algorithm designed to reveal ecological relationships in rivers and streams from observational data. Our algorithm (a) takes into account the spatial structure of the river network, (b) reveals the complete network of ecological relationships, and (c) shows the directions of these relationships. We apply our algorithm to data collected in the US state of Ohio to better understand causes of reductions in fish and invertebrate community integrity. We found that nitrogen is a key variable underlying fish and invertebrate community integrity in Ohio, likely negatively impacting both. We also found that fish and community integrity are each linked to one physical habitat quality variable. Our algorithm further revealed a split between physical habitat quality and water quality variables, indicating that causal relations between these groups of variables are likely absent. Our approach is able to reveal networks of ecological relationships in rivers and streams based on observational data, without the need to formulate a priori hypotheses. This is an asset particularly for diagnostic assessments of the ecological state and potential causes of biodiversity impairment in rivers and streams.

Evaluation of Machine Learning Predictions of a Highly Resolved Time Series of Chlorophyll-a Concentration

Pelagic chlorophyll-a concentrations are key for evaluation of the environmental status and productivity of marine systems, and data can be provided by in situ measurements, remote sensing and modelling. However, modelling chlorophyll-a is not trivial due to its nonlinear dynamics and complexity. In this study, chlorophyll-a concentrations for the Helgoland Roads time series were modeled using a number of measured water and environmental parameters. We chose three common machine learning algorithms from the literature: the support vector machine regressor, neural networks multi-layer perceptron regressor and random forest regressor. Results showed that the support vector machine regressor slightly outperformed other models. The evaluation with a test dataset and verification with an independent validation dataset for chlorophyll-a concentrations showed a good generalization capacity, evaluated by the root mean squared errors of less than 1 µg L−1. Feature selection and engineering are important and improved the models significantly, as measured in performance, improving the adjusted R2 by a minimum of 48%. We tested SARIMA in comparison and found that the univariate nature of SARIMA does not allow for better results than the machine learning models. Additionally, the computer processing time needed was much higher (prohibitive) for SARIMA.

A Multiattention-Based Supervised Feature Selection Method for Multivariate Time Series

Feature selection is a known technique to preprocess the data before performing any data mining task. In multivariate time series (MTS) prediction, feature selection needs to find both the most related variables and their corresponding delays. Both aspects, to a certain extent, represent essential characteristics of system dynamics. However, the variable and delay selection for MTS is a challenging task when the system is nonlinear and noisy. In this paper, a multiattention-based supervised feature selection method is proposed. It translates the feature weight generation problem into a bidirectional attention generation problem with two parallel placed attention modules. The input 2D data are sliced into 1D data from two orthogonal directions, and each attention module generates attention weights from their respective dimensions. To facilitate the feature selection from the global perspective, we proposed a global weight generation method that calculates a dot product operation on the weight values of the two dimensions. To avoid the disturbance of attention weights due to noise and duplicated features, the final feature weight matrix is calculated based on the statistics of the entire training set. Experimental results show that this proposed method achieves the best performance on compared synthesized, small, medium, and practical industrial datasets, compared to several state-of-the-art baseline feature selection methods.

Investigating Water Quality Data Using Principal Component Analysis and Granger Causality

This work investigates the inter-relationships among stream water quality indicators, hydroclimatic variables (e.g., precipitation, river discharge), and land characteristics (e.g., soil type, land use), which is crucial to developing effective methods for water quality protection. The potential of using statistical tools, such as Principal Component (PC) and Granger causality analyses, for this purpose is assessed across 10 watersheds in the Eastern United States. The PC analysis shows consistency across the ten locations, with most of the variation explained by the first two PCs, except for the least developed watershed that presents three PCs. Results show that stronger Granger causality relationships and correlation coefficients are identified when considering a lag of one day, compared to longer lags. This is mainly due to the watersheds’ limited size and, thus, their fast hydrological response. The strongest Granger causalities are observed when water temperature and dissolved oxygen concentration are considered as the effect of the other variables, which corroborates the importance of these two water properties. This work also demonstrates how watershed size and land use can impact causalities between hydrometeorological variables and water quality, thus, highlighting how complex these relationships are even in a region characterized by overall similar climatology.

Measuring the Coupling Direction between Neural Oscillations with Weighted Symbolic Transfer Entropy

Neural oscillations reflect rhythmic fluctuations in the synchronization of neuronal populations and play a significant role in neural processing. To further understand the dynamic interactions between different regions in the brain, it is necessary to estimate the coupling direction between neural oscillations. Here, we developed a novel method, termed weighted symbolic transfer entropy (WSTE), that combines symbolic transfer entropy (STE) and weighted probability distribution to measure the directionality between two neuronal populations. The traditional STE ignores the degree of difference between the amplitude values of a time series. In our proposed WSTE method, this information is picked up by utilizing a weighted probability distribution. The simulation analysis shows that the WSTE method can effectively estimate the coupling direction between two neural oscillations. In comparison with STE, the new method is more sensitive to the coupling strength and is more robust against noise. When applied to epileptic electrocorticography data, a significant coupling direction from the anterior nucleus of thalamus (ANT) to the seizure onset zone (SOZ) was detected during seizures. Considering the superiorities of the WSTE method, it is greatly advantageous to measure the coupling direction between neural oscillations and consequently characterize the information flow between different brain regions.

Classification of myocardial infarction based on hybrid feature extraction and artificial intelligence tools by adopting tunable-Q wavelet transform (TQWT), variational mode decomposition (VMD) and neural networks

Cardiovascular diseases (CVD) is the leading cause of human mortality and morbidity around the world, in which myocardial infarction (MI) is a silent condition that irreversibly damages the heart muscles. Currently, electrocardiogram (ECG) is widely used by the clinicians to diagnose MI patients due to its inexpensiveness and non-invasive nature. Pathological alterations provoked by MI cause slow conduction by increasing axial resistance on coupling between cells. This issue may cause abnormal patterns in the dynamics of the tip of the cardiac vector in the ECG signals. However, manual interpretation of the pathological alternations induced by MI is a time-consuming, tedious and subjective task. To overcome such disadvantages, computer-aided diagnosis techniques including signal processing and artificial intelligence tools have been developed. In this study we propose a novel technique for automatic detection of MI based on hybrid feature extraction and artificial intelligence tools. Tunable quality factor (Q-factor) wavelet transform (TQWT), variational mode decomposition (VMD) and phase space reconstruction (PSR) are utilized to extract representative features to form cardiac vectors with synthesis of the standard 12-lead and Frank XYZ leads. They are combined with neural networks to model, identify and detect abnormal patterns in the dynamics of cardiac system caused by MI. First, 12-lead ECG signals are reduced to 3-dimensional VCG signals, which are synthesized with Frank XYZ leads to build a hybrid 4-dimensional cardiac vector. Second, this vector is decomposed into a set of frequency subbands with a number of decomposition levels by using the TQWT method. Third, VMD is employed to decompose the subband of the 4-dimensional cardiac vector into different intrinsic modes, in which the first intrinsic mode contains the majority of the cardiac vector's energy and is considered to be the predominant intrinsic mode. It is selected to construct the reference variable for analysis. Fourth, phase space of the reference variable is reconstructed, in which the properties associated with the nonlinear cardiac system dynamics are preserved. Three-dimensional (3D) PSR together with Euclidean distance (ED) has been utilized to derive features, which demonstrate significant difference in cardiac system dynamics between normal (healthy) and MI cardiac vector signals. Fifth, cardiac system dynamics can be modeled and identified using neural networks, which employ the ED of 3D PSR of the reference variable as the input features. The difference of cardiac system dynamics between healthy control and MI cardiac vector is computed and used for the detection of MI based on a bank of estimators. Finally, data sets, which include conventional 12-lead and Frank XYZ leads ECG signal fragments from 148 patients with MI and 52 healthy controls from PTB diagnostic ECG database, are used for evaluation. By using the 10-fold cross-validation style, the achieved average classification accuracy is reported to be 97.98%. Currently, ST segment evaluation is one of the major and traditional ways for the MI detection. However, there exist weak or even undetectable ST segments in many ECG signals. Since the proposed method does not rely on the information of ST waves, it can serve as a complementary MI detection algorithm in the intensive care unit (ICU) of hospitals to assist the clinicians in confirming their diagnosis. Overall, our results verify that the proposed features may satisfactorily reflect cardiac system dynamics, and are complementary to the existing ECG features for automatic cardiac function analysis.

Feature selection based multivariate time series forecasting: An application to antibiotic resistance outbreaks prediction

Antimicrobial resistance has become one of the most important health problems and global action plans have been proposed globally. Prevention plays a key role in these actions plan and, in this context, we propose the use of Artificial Intelligence, specifically Time Series Forecasting techniques, for predicting future outbreaks of Methicillin-resistant Staphylococcus aureus (MRSA). Infection incidence forecasting is approached as a Feature Selection based Time Series Forecasting problem using multivariate time series composed of incidence of Staphylococcus aureus Methicillin-sensible and MRSA infections, influenza incidence and total days of therapy of both of Levofloxacin and Oseltamivir antimicrobials. Data were collected from the University Hospital of Getafe (Spain) from January 2009 to January 2018, using months as time granularity. The main contributions of the work are the following: the applications of wrapper feature selection methods where the search strategy is based on multi-objective evolutionary algorithms (MOEA) along with evaluators based on the most powerful state-of-the-art regression algorithms. The performance of the feature selection methods has been measured using the root mean square error (RMSE) and mean absolute error (MAE) performance metrics. A novel multi-criteria decision-making process is proposed in order to select the most satisfactory forecasting model, using the metrics previously mentioned, as well as the slopes of model prediction lines in the 1, 2 and 3 steps-ahead predictions. The multi-criteria decision-making process is applied to the best models resulting from a ranking of databases and regression algorithms obtained through multiple statistical tests. Finally, to the best of our knowledge, this is the first time that a feature selection based multivariate time series methodology is proposed for antibiotic resistance forecasting. Final results show that the best model according to the proposed multi-criteria decision making process provides a RMSE = (0.1349, 0.1304, 0.1325) and a MAE = (0.1003, 0.096, 0.0987) for 1, 2, and 3 steps-ahead predictions.

A feature selection strategy for gene expression time series experiments with hidden Markov models

Studies conducted in time series could be far more informative than those that only capture a specific moment in time. However, when it comes to transcriptomic data, time points are sparse creating the need for a constant search for methods capable of extracting information out of experiments of this kind. We propose a feature selection algorithm embedded in a hidden Markov model applied to gene expression time course data on either single or even multiple biological conditions. For the latter, in a simple case-control study features or genes are selected under the assumption of no change over time for the control samples, while the case group must have at least one change. The proposed model reduces the feature space according to a two-state hidden Markov model. The two states define change/no-change in gene expression. Features are ranked in consonance with three scores: number of changes across time, magnitude of such changes and quality of replicates as a measure of how much they deviate from the mean. An important highlight is that this strategy overcomes the few samples limitation, common in transcriptome experiments through a process of data transformation and rearrangement. To prove this method, our strategy was applied to three publicly available data sets. Results show that feature domain is reduced by up to 90% leaving only few but relevant features yet with findings consistent to those previously reported. Moreover, our strategy proved to be robust, stable and working on studies where sample size is an issue otherwise. Hence, even with two biological replicates and/or three time points our method proves to work well.

Metaheuristic approaches in biopharmaceutical process development data analysis

There is a growing interest in mining and handling of big data, which has been rapidly accumulating in the repositories of bioprocess industries. Biopharmaceutical industries are no exception; the implementation of advanced process control strategies based on multivariate monitoring techniques in biopharmaceutical production gave rise to the generation of large amounts of data. Real-time measurements of critical quality and performance attributes collected during production can be highly useful to understand and model biopharmaceutical processes. Data mining can facilitate the extraction of meaningful relationships pertaining to these bioprocesses, and predict the performance of future cultures. This review evaluates the suitability of various metaheuristic methods available for data pre-processing, which would involve the handling of missing data, the visualisation of the data, and dimension reduction; and for data processing, which would focus on modelling of the data and the optimisation of these models in the context of biopharmaceutical process development. The advantages and the associated challenges of employing different methodologies in pre-processing and processing of the data are discussed. In light of these evaluations, a summary guideline is proposed for handling and analysis of the data generated in biopharmaceutical process development.

Classification of gait patterns between patients with Parkinson's disease and healthy controls using phase space reconstruction (PSR), empirical mode decomposition (EMD) and neural networks

Parkinson's disease (PD) is a common neurodegenerative disorder that affects human's quality of life, especially leading to locomotor deficits such as postural instability and gait disturbances. Gait signal is one of the best features to characterize and detect movement disorders caused by a malfunction in parts of the brain and nervous system of the patients with PD. Various classification approaches using spatiotemporal gait variables have been presented earlier to classify Parkinson's gait. In this study we propose a novel method for gait pattern classification between patients with PD and healthy controls, based upon phase space reconstruction (PSR), empirical mode decomposition (EMD) and neural networks. First, vertical ground reaction forces (GRFs) at specific positions of human feet are captured and then phase space is reconstructed. The properties associated with the gait system dynamics are preserved in the reconstructed phase space. Three-dimensional (3D) PSR together with Euclidean distance (ED) has been used. These measured parameters demonstrate significant difference in gait dynamics between the two groups and have been utilized to form a reference variable set. Second, reference variables are decomposed into Intrinsic Mode Functions (IMFs) using EMD, and the third IMFs are extracted and served as gait features. Third, neural networks are then used as the classifier to distinguish between patients with PD and healthy controls based on the difference of gait dynamics preserved in the gait features between the two groups. Finally, experiments are carried out on 93 PD patients and 73 healthy subjects to assess the effectiveness of the proposed method. By using 2-fold, 10-fold and leave-one-out cross-validation styles, the correct classification rates are reported to be 91.46%, 96.99% and 98.80%, respectively. Compared with other state-of-the-art methods, the results demonstrate superior performance and the proposed method can serve as a potential candidate for the automatic and non-invasive classification between patients with PD and healthy subjects.

Integrated Theory- and Data-driven Feature Selection in Gene Expression Data Analysis

The exponential growth of high dimensional biological data has led to a rapid increase in demand for automated approaches for knowledge production. Existing methods rely on two general approaches to address this challenge: 1) the Theory-driven approach, which utilizes prior accumulated knowledge, and 2) the Data-driven approach, which solely utilizes the data to deduce scientific knowledge. Both of these approaches alone suffer from bias toward past/present knowledge, as they fail to incorporate all of the current knowledge that is available to make new discoveries. In this paper, we show how an integrated method can effectively address the high dimensionality of big biological data, which is a major problem for pure data-driven analysis approaches. We realize our approach in a novel two-step analytical workflow that incorporates a new feature selection paradigm as the first step to handling high-throughput gene expression data analysis and that utilizes graphical causal modeling as the second step to handle the automatic extraction of causal relationships. Our results, on real-world clinical datasets from The Cancer Genome Atlas (TCGA), demonstrate that our method is capable of intelligently selecting genes for learning effective causal networks.