Entropy-based metrics for predicting choice behavior based on local response to reward

Nonlinear analysis and recognition of epileptic EEG signals in different stages

It is a hot problem in epilepsy research to detect and predict seizures by EEG signals. Clinically, it is generally observed that there are only sudden abnormal signals during the ictal stage, but there is no significant difference in the EEG signal between the interictal and preictal stages. To solve the problem that preictal signals are difficult to recognize clinically, and then effectively improve the recognition efficiency of epileptic seizures, so, in this paper, some nonlinear methods are comprehensively used to extract the hidden information in the EEG signals in different stages, namely, phase space reconstruction (PSR), Poincaré section (PS), synchroextracting transform (SET), and machine learning for EEG signal analysis. First, PSR based on C-C method is used, and the results show that there are different diffuse attractor trajectories of the signals in different stages. Second, the confidence ellipse (CE) is constructed by using the scatter diagram of the corresponding trajectory on PS, and the aspect ratio and area of the ellipse are calculated. The results show that there is an interesting transitional phenomenon in preictal stage. To recognize ictal and preictal signals, time-frequency (TF) spectrums, which are processed by SET, are fed into the convolutional neural network (CNN) classifier. The accuracy of recognizing ictal and preictal signals reaches 99.7% and 93.7%, respectively. To summarize, our results based on nonlinear method provide new research ideas for seizure detection and prediction.NEW & NOTEWORTHY Our results based on nonlinear method have better practical significance and clinical application value and improved the prediction efficiency of epileptic EEG signals effectively. This work provides direct insight into the application of these biomarkers for seizure detection and prediction.

Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum

Lack of cognitive flexibility is a hallmark of substance use disorders and has been associated with drug-induced synaptic plasticity in the dorsomedial striatum (DMS). Yet the possible impact of altered plasticity on real-time striatal neural dynamics during decision-making is unclear. Here, we identified persistent impairments induced by chronic ethanol (EtOH) exposure on cognitive flexibility and striatal decision signals. After a substantial withdrawal period from prior EtOH vapor exposure, male, but not female, rats exhibited reduced adaptability and exploratory behavior during a dynamic decision-making task. Reinforcement learning models showed that prior EtOH exposure enhanced learning from rewards over omissions. Notably, neural signals in the DMS related to the decision outcome were enhanced, while those related to choice and choice-outcome conjunction were reduced, in EtOH-treated rats compared to the controls. These findings highlight the profound impact of chronic EtOH exposure on adaptive decision-making, pinpointing specific changes in striatal representations of actions and outcomes as underlying mechanisms for cognitive deficits.

The role of rat prelimbic cortex in decision making

The frontal cortex plays a critical role in decision-making. One specific frontal area, the anterior cingulate cortex, has been identified as crucial for setting a threshold for how much evidence is needed before a choice is made (Domenech & Dreher, 2010). Threshold is a key concept in drift diffusion models, a popular framework used to understand decision-making processes. Here, we investigated the role of the prelimbic cortex, part of the rodent cingulate cortex, in decision making. Male and female rats learned to choose between stimuli associated with high and low value rewards. Females learned faster, were more selective in their responses, and integrated information about the stimuli more quickly. By contrast, males learned more slowly and showed a decrease in their decision thresholds during choice learning. Inactivating the prelimbic cortex in female and male rats sped up decision making without affecting choice accuracy. Drift diffusion modeling found selective effects of prelimbic cortex inactivation on the decision threshold, which was reduced with increasing doses of the GABA-A agonist muscimol. Stimulating the prelimbic cortex through mu opioid receptors slowed the animals' choice latencies and increased the decision threshold. These findings provide the first causal evidence that the prelimbic cortex directly influences decision processes. Additionally, they suggest possible sex-based differences in early choice learning.

Tracking subjects' strategies in behavioural choice experiments at trial resolution

Investigating how, when, and what subjects learn during decision-making tasks requires tracking their choice strategies on a trial-by-trial basis. Here, we present a simple but effective probabilistic approach to tracking choice strategies at trial resolution using Bayesian evidence accumulation. We show this approach identifies both successful learning and the exploratory strategies used in decision tasks performed by humans, non-human primates, rats, and synthetic agents. Both when subjects learn and when rules change the exploratory strategies of win-stay and lose-shift, often considered complementary, are consistently used independently. Indeed, we find the use of lose-shift is strong evidence that subjects have latently learnt the salient features of a new rewarded rule. Our approach can be extended to any discrete choice strategy, and its low computational cost is ideally suited for real-time analysis and closed-loop control.

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.