A tradeoff relationship between internal monitoring and external feedback during the dynamic process of reinforcement learning

Effective behavior monitoring, including internal monitoring/error detection and external monitoring/feedback, is very pivotal for reinforcement learning. However, less attention has been paid to internal monitoring and the dynamic learning performance in reinforcement learning, and there is still a heated debate on which kind of external feedback is relied on in the reinforcement learning. In order to address these questions, an adaption probabilistic selection task was used to examine the effect of the internal monitoring, external feedback and the relationship between them for approach learners and avoidance learners during dynamic learning process of reinforcement learning and behavior adaption. Error-related negativity (ERN), feedback-related negativity (FRN) and feedback-related P300 are three ERPs components, which can be used as the indexes of internal monitoring, external feedback and behavior adaption. For our results, the ERN effect of avoidance learners become large in block 3, which is earlier than approach learners (block 4). This phenomenon suggests that avoidance learners learned faster than approach learners. In addition, the FRN amplitude of avoidance learners in block 4 was significantly smaller than the other three blocks. The aforementioned results demonstrated a tradeoff relationship between the ERN and FRN effects.

Errors and Action Monitoring: Errare Humanum Est Sed Corrigere Possibile

It was recognized long ago by Seneca through his famous "errare humanum est." that the human information processing system is intrinsically fallible. What is newer is the fact that, at least in sensorimotor information processing realized under time pressure, errors are largely dealt with by several (psycho)physiological-specific mechanisms: prevention, detection, inhibition, correction, and, if these mechanisms finally fail, strategic behavioral adjustments following errors. In this article, we review several datasets from laboratory experiments, showing that the human information processing system is well equipped not only to detect and correct errors when they occur but also to detect, inhibit, and correct them even before they fully develop. We argue that these (psycho)physiological mechanisms are important to consider when the brain works in everyday settings in order to render work systems more resilient to human errors and, thus, safer.

Detrended Fluctuation, Coherence, and Spectral Power Analysis of Activation Rearrangement in EEG Dynamics During Cognitive Workload

In the study of human cognitive activity using electroencephalogram (EEG), the brain dynamics parameters and characteristics play a crucial role. They allow to investigate the changes in functionality depending on the environment and task performance process, and also to access the intensity of the brain activity in various locations of the cortex and its dependencies. Usually, the dynamics of activation of different brain areas during the cognitive tasks are being studied by spectral analysis based on power spectral density (PSD) estimation, and coherence analysis, which are de facto standard tools in quantitative characterization of brain activity. PSD and coherence reflect the strength of oscillations and similarity of the emergence of these oscillations in the brain, respectively, while the concept of stability of brain activity over time is not well defined and less formalized. We propose to employ the detrended fluctuation analysis (DFA) as a measure of the EEG persistence over time, and use the DFA scaling exponent as its quantitative characteristics. We applied DFA to the study of the changes in activation in brain dynamics during mental calculations and united it with PSD and coherence estimation. In the experiment, EEGs during resting state and mental serial subtraction from 36 subjects were recorded and analyzed in four frequency ranges: θ1 (4.1-5.8 Hz), θ2 (5.9-7.4 Hz), β1 (13-19.9 Hz), and β2 (20-25 Hz). PSD maps to access the intensity of cortex activation and coherence to quantify the connections between different brain areas were calculated, the distribution of DFA scaling exponent over the head surface was exploited to measure the time characteristics of the dynamics of brain activity. Obtained arrangements of DFA scaling exponent suggest that normal functioning of the brain is characterized by long-term temporal correlations in the cortex. Topographical distribution of the DFA scaling exponent was comparable for θ and β frequency bands, demonstrating the largest values of DFA scaling exponent during cognitive activation. The study shows that the long-term temporal correlations evaluated by DFA can be of great interest for diagnosis of the variety of brain dysfunctions of different etiology in the future.

The Sustained Influence of an Error on Future Decision-Making

Post-error slowing (PES) is consistently observed in decision-making tasks after negative feedback. Yet, findings are inconclusive as to whether PES supports performance accuracy. We addressed the role of PES by employing drift diffusion modeling which enabled us to investigate latent processes of reaction times and accuracy on a large-scale dataset (>5,800 participants) of a visual search experiment with emotional face stimuli. In our experiment, post-error trials were characterized by both adaptive and non-adaptive decision processes. An adaptive increase in participants' response threshold was sustained over several trials post-error. Contrarily, an initial decrease in evidence accumulation rate, followed by an increase on the subsequent trials, indicates a momentary distraction of task-relevant attention and resulted in an initial accuracy drop. Higher values of decision threshold and evidence accumulation on the post-error trial were associated with higher accuracy on subsequent trials which further gives credence to these parameters' role in post-error adaptation. Finally, the evidence accumulation rate post-error decreased when the error trial presented angry faces, a finding suggesting that the post-error decision can be influenced by the error context. In conclusion, we demonstrate that error-related response adaptations are multi-component processes that change dynamically over several trials post-error.