Estimating and improving the signal-to-noise ratio of time series by symbolic dynamics

Spectral invariants of ergodic symbolic systems for pattern recognition and anomaly detection

Despite tangible advances in machine learning (ML) over the last few decades, many of the ML techniques still suffer from fundamental issues like overfitting and lack of explainability. These issues mandate requirements for mathematical rigor to ensure robust learning from observed data. In this context, topological invariants in data manifolds provide a rich representation of the underlying dynamical system, which can be utilized for developing a mathematically rigorous ML tool to characterize the dynamical behaviour and operational phases of the underlying process. This paper aims to investigate spectral invariants of symbolic systems for detecting changes in topological characteristics of data manifolds. A novel ML approach is proposed, where commutator norms are used on sequences of endomorphisms to symbolically describe dynamical systems on probability spaces with ergodic measures. The objective here is to detect topological invariants of data manifolds that can be used for signal processing, pattern recognition, and anomaly detection. The proposed ML approach is validated on models of selected chaotic dynamical systems for prompt detection of phase transitions. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.

Mutual Information of Multiple Rhythms for EEG Signals

Electroencephalograms (EEG) are one of the most commonly used measures to study brain functioning at a macroscopic level. The structure of the EEG time series is composed of many neural rhythms interacting at different spatiotemporal scales. This interaction is often named as cross frequency coupling, and consists of transient couplings between various parameters of different rhythms. This coupling has been hypothesized to be a basic mechanism involved in cognitive functions. There are several methods to measure cross frequency coupling between two rhythms but no single method has been selected as the gold standard. Current methods only serve to explore two rhythms at a time, are computationally demanding, and impose assumptions about the nature of the signal. Here we present a new approach based on Information Theory in which we can characterize the interaction of more than two rhythms in a given EEG time series. It estimates the mutual information of multiple rhythms (MIMR) extracted from the original signal. We tested this measure using simulated and real empirical data. We simulated signals composed of three frequencies and background noise. When the coupling between each frequency component was manipulated, we found a significant variation in the MIMR. In addition, we found that MIMR was sensitive to real EEG time series collected with open vs. closed eyes, and intra-cortical recordings from epileptic and non-epileptic signals registered at different regions of the brain. MIMR is presented as a tool to explore multiple rhythms, easy to compute and without a priori assumptions.

Symbolic Entropy Analysis and Its Applications

This editorial explains the scope of the special issue and provides a thematic introduction to the contributed papers.

Bayesian Nonparametric Modeling of Categorical Data for Information Fusion and Causal Inference

This paper presents a nonparametric regression model of categorical time series in the setting of conditional tensor factorization and Bayes network. The underlying algorithms are developed to provide a flexible and parsimonious representation for fusion of correlated information from heterogeneous sources, which can be used to improve the performance of prediction tasks and infer the causal relationship between key variables. The proposed method is first illustrated by numerical simulation and then validated with two real-world datasets: (1) experimental data, collected from a swirl-stabilized lean-premixed laboratory-scale combustor, for detection of thermoacoustic instabilities and (2) publicly available economics data for causal inference-making.

Mind the Noise When Identifying Computational Models of Cognition from Brain Activity

The aim of this study was to analyze how measurement error affects the validity of modeling studies in computational neuroscience. A synthetic validity test was created using simulated P300 event-related potentials as an example. The model space comprised four computational models of single-trial P300 amplitude fluctuations which differed in terms of complexity and dependency. The single-trial fluctuation of simulated P300 amplitudes was computed on the basis of one of the models, at various levels of measurement error and at various numbers of data points. Bayesian model selection was performed based on exceedance probabilities. At very low numbers of data points, the least complex model generally outperformed the data-generating model. Invalid model identification also occurred at low levels of data quality and under low numbers of data points if the winning model's predictors were closely correlated with the predictors from the data-generating model. Given sufficient data quality and numbers of data points, the data-generating model could be correctly identified, even against models which were very similar to the data-generating model. Thus, a number of variables affects the validity of computational modeling studies, and data quality and numbers of data points are among the main factors relevant to the issue. Further, the nature of the model space (i.e., model complexity, model dependency) should not be neglected. This study provided quantitative results which show the importance of ensuring the validity of computational modeling via adequately prepared studies. The accomplishment of synthetic validity tests is recommended for future applications. Beyond that, we propose to render the demonstration of sufficient validity via adequate simulations mandatory to computational modeling studies.

Rényi entropy measure of noise-aided information transmission in a binary channel

This paper analyzes a binary channel by means of information measures based on the Rényi entropy. The analysis extends, and contains as a special case, the classic reference model of binary information transmission based on the Shannon entropy measure. The extended model is used to investigate further possibilities and properties of stochastic resonance or noise-aided information transmission. The results demonstrate that stochastic resonance occurs in the information channel and is registered by the Rényi entropy measures at any finite order, including the Shannon order. Furthermore, in definite conditions, when seeking the Rényi information measures that best exploit stochastic resonance, then nontrivial orders differing from the Shannon case usually emerge. In this way, through binary information transmission, stochastic resonance identifies optimal Rényi measures of information differing from the classic Shannon measure. A confrontation of the quantitative information measures with visual perception is also proposed in an experiment of noise-aided binary image transmission.

Towards dynamical system models of language-related brain potentials

Event-related brain potentials (ERP) are important neural correlates of cognitive processes. In the domain of language processing, the N400 and P600 reflect lexical-semantic integration and syntactic processing problems, respectively. We suggest an interpretation of these markers in terms of dynamical system theory and present two nonlinear dynamical models for syntactic computations where different processing strategies correspond to functionally different regions in the system's phase space.

Enhancing dominant modes in nonstationary time series by means of the symbolic resonance analysis

We present the symbolic resonance analysis (SRA) as a viable method for addressing the problem of enhancing a weakly dominant mode in a mixture of impulse responses obtained from a nonlinear dynamical system. We demonstrate this using results from a numerical simulation with Duffing oscillators in different domains of their parameter space, and by analyzing event-related brain potentials (ERPs) from a language processing experiment in German as a representative application. In this paradigm, the averaged ERPs exhibit an N400 followed by a sentence final negativity. Contemporary sentence processing models predict a late positivity (P600) as well. We show that the SRA is able to unveil the P600 evoked by the critical stimuli as a weakly dominant mode from the covering sentence final negativity.

Auto-adaptive averaging: detecting artifacts in event-related potential data using a fully automated procedure

The auto-adaptive averaging procedure proposed here classifies artifacts in event-related potential data by optimizing the signal-to-noise ratio. This method rank orders single trials according to the impact of each trial on the ERP average. Then, the minimum residual background noise level in the ERP data is determined at each step in the averaging process. Trials having a negative impact on the residual background noise are discarded from the averaging procedure. Simulations showed that ERP estimates obtained by the auto-adaptive averaging procedure were either better or comparable to those obtained by single trial artifact detection methods at their most optimum configuration, in particular during long duration artifacts. Experimental data from a working memory task further illustrate the effectiveness of the method.

Topographic voltage and coherence mapping of brain potentials by means of the symbolic resonance analysis

We apply the recently developed symbolic resonance analysis to electroencephalographic measurements of event-related brain potentials (ERPs) in a language processing experiment by using a three-symbol static encoding with varying thresholds for analyzing the ERP epochs, followed by a spin-flip transformation as a nonlinear filter. We compute an estimator of the signal-to-noise ratio (SNR) for the symbolic dynamics measuring the coherence of threshold-crossing events. Hence, we utilize the inherent noise of the EEG for sweeping the underlying ERP components beyond the encoding thresholds. Plotting the SNR computed within the time window of a particular ERP component (the N400) against the encoding thresholds, we find different resonance curves for the experimental conditions. The maximal differences of the SNR lead to the estimation of optimal encoding thresholds. We show that topographic brain maps of the optimal threshold voltages and of their associated coherence differences are able to dissociate the underlying physiological processes, while corresponding maps gained from the customary voltage averaging technique are unable to do so.

Finding needles in haystacks: Symbolic resonance analysis of event-related potentials unveils different processing demands

Previous ERP studies have found an N400-P600 pattern in sentences in which the number of arguments does not match the number of arguments that the verb can take. In the present study, we elaborate on this question by investigating whether the case of the mismatching object argument in German (accusative/direct object versus dative/indirect object) affects processing differently. In general, both types of mismatches elicited a biphasic N400-P600 response in the ERP. However, traditional voltage average analysis was unable to reveal differences between the two mismatching conditions, that is, between a mismatching accusative versus dative. Therefore, we employed a recently developed method on ERP data analysis, the symbolic resonance analysis (SRA), where EEG epochs are symbolically encoded in sequences of three symbols depending on a given parameter, the encoding threshold. We found a larger proportion of threshold crossing events with negative polarity in the N400 time window for a mismatching dative argument compared to a mismatching accusative argument. By contrast, the proportion of threshold crossing events with positive polarity was smaller for dative in the P600 time window. We argue that this difference is due to the phenomenon of "free dative" in German. This result also shows that the SRA provides a useful tool for revealing ERP differences that cannot be discovered using the traditional voltage average analysis.

Is it Positive or Negative? On Determining ERP Components

In most experiments using event-related brain potentials (ERPs), there is a straightforward way to define--on theoretical grounds-which of the conditions tested is the experimental condition and which is the control condition. If, however, theoretical assumptions do not give sufficient and unambiguous information to decide this question, then the interpretation of an ERP effect becomes difficult, especially if one takes into account that certain effects can be both a positivity or a negativity on the basis of the morphology of the pattern as well as with respect to peak latency (regard for example, N400 and P345). Exemplified with an ERP experiment on language processing, we present such a critical case and offer a possible solution on the basis of nonlinear data analysis. We show that a generalized polarity histogram, the word statistics of symbolic dynamics, is in principle able to distinguish negative going ERP components from positive ones when an appropriate encoding strategy, the half wave encoding is employed. We propose statistical criteria which allow to determine ERP components on purely methodological grounds.

Synaptic transmission of chaotic spike trains between primary afferent fiber and spinal dorsal horn neuron in the rat

Primary sensory neurons can generate irregular burst firings in which the existence of significant deterministic behaviors of chaotic dynamics has been proved with nonlinear time series analysis. But how well the deterministic characteristics and neural information of presynaptic chaotic spike trains were transmitted into postsynaptic spike trains is still an open question. Here we investigated the synaptic transmission of chaotic spike trains between primary Adelta afferent fiber and spinal dorsal horn neuron. Two kinds of basic stimulus unit, brief burst and single pulse, were employed by us to comprise chaotic stimulus trains. For time series analysis, we defined "events" as the longest sequences of spikes with all interspike intervals less than or equal to a certain threshold and extracted the interevent intervals (IEIs) from spike trains. Return map analysis of the IEI series showed that the main temporal structure of chaotic input trains could be detected in postsynaptic output trains, especially under brief-burst stimulation. Using correlation dimension and nonlinear prediction methods, we found that synaptic transmission could influence the nonlinear characteristics of chaotic trains, such as fractal dimension and short-term predictability, with greater influence made under single-pulse stimulation. By calculating the mutual information between input and output trains, we found the information carried by presynaptic spike trains could not be completely transmitted at primary afferent synapses, and that brief bursts could more reliably transmit the information carried by chaotic input trains across synapses. These results indicate that although unreliability exists during synaptic transmission, the main deterministic characteristics of chaotic burst trains can be transmitted across primary afferent synapses. Moreover, brief bursts that come from the periphery can more reliably transmit neural information between primary afferent fibers and spinal dorsal horn neurons.

Detecting subthreshold events in noisy data by symbolic dynamics

We show that a symmetric threshold crossing detector can be described by a symbolic dynamics of a static three-symbol encoding which is highly efficient to detect subthreshold events in noisy nonstationary data. After computing instantaneous word statistics and running cylinder entropies, we introduce a mean-field transformation of the three-symbol dynamics considered as a Potts-spin lattice onto a distribution of two symbols. This transformed word statistics enables one to derive an estimator of the signal-to-noise ratio (SNR). Subthreshold events are then proven by a prominent peak of the SNR estimator as a function of the noise intensity.