Predictions through evidence accumulation over time

Behavioral and neural measures of confidence using a novel auditory pitch identification task

Observers can discriminate between correct versus incorrect perceptual decisions with feelings of confidence. The centro-parietal positivity build-up rate (CPP slope) has been suggested as a likely neural signature of accumulated evidence, which may guide both perceptual performance and confidence. However, CPP slope also covaries with reaction time, which also covaries with confidence in previous studies, and performance and confidence typically covary; thus, CPP slope may index signatures of perceptual performance rather than confidence per se. Moreover, perceptual metacognition-including neural correlates-has largely been studied in vision, with few exceptions. Thus, we lack understanding of domain-general neural signatures of perceptual metacognition outside vision. Here we designed a novel auditory pitch identification task and collected behavior with simultaneous 32-channel EEG in healthy adults. Participants saw two tone labels which varied in tonal distance on each trial (e.g., C vs D, C vs F), then heard a single auditory tone; they identified which label was correct and rated confidence. We found that pitch identification confidence varied with tonal distance, but performance, metacognitive sensitivity (trial-by-trial covariation of confidence with accuracy), and reaction time did not. Interestingly, however, while CPP slope covaried with performance and reaction time, it did not significantly covary with confidence. We interpret these results to mean that CPP slope is likely a signature of first-order perceptual processing and not confidence-specific signals or computations in auditory tasks. Our novel pitch identification task offers a valuable method to examine the neural correlates of auditory and domain-general perceptual confidence.

Bayesian interpretation of the prefrontal P2 ERP component based on stimulus/response mapping uncertainty

The brain can be seen as a predictive system continuously computing prior information to guess posterior probabilities minimizing sources of uncertainty. To test this Bayesian view of the brain, event-related potentials (ERP) methods have been used focusing on the well-known P3 component, traditionally associated with decision-making processes and sources of uncertainty regarding target probability. Another ERP component linked with decision-making is the prefrontal P2 (pP2) component, which has never been considered within the Bayesian framework. To test which source of uncertainty could be associated with the pP2, uncertainty induced by target probability and stimulus-response (S/R) mapping were modulated in three visuomotor tasks. Results showed that the pP2 had the largest amplitude in the task with the largest uncertainty regarding the S/R mapping and degraded as the S/R mapping became more predictable. The P3 was maximal in the tasks with larger uncertainty regarding the target probability. While we confirmed the P3 association with target probability, we extended our knowledge on the pP2 associating it with S/R mapping uncertainty. This component, which has been previously localized within the anterior insular cortex, may minimize S/R mapping uncertainty allowing response-related evidence accumulation and comparing current events with internal representations to extract action-related probabilities.

Action-outcome delays modulate the temporal expansion of intended outcomes

The phenomenon of intentional binding pertains to the perceived connection between a voluntary action and its anticipated result. When an individual intends an outcome, it appears to subjectively extend in time due to a pre-activation of the intended result, particularly evident at shorter action-outcome delays. However, there is a concern that the operationalisation of intention might have led to a mixed interpretation of the outcome expansion attributed to the pre-activation of intention, given the sensitivity of time perception and intentional binding to external cues that could accelerate the realisation of expectations. To investigate the expansion dynamics of an intended outcome, we employed a modified version of the temporal bisection task in two experiments. Experiment 1 considered the action-outcome delay as a within-subject factor, while experiment 2 treated it as a between-subject factor. The results revealed that the temporal expansion of an intended outcome was only evident under the longer action-outcome delay condition. We attribute this observation to working memory demands and attentional allocation due to temporal relevancy and not due to pre-activation. The discrepancy in effects across studies is explained by operationalising different components of the intentional binding effect, guided by the cue integration theory. Moreover, we discussed speculative ideas regarding the involvement of specific intentions based on the proximal intent distal intent (PIDI) theory and whether causality plays a role in temporal binding. Our study contributes to the understanding of how intention influences time perception and sheds light on how various methodological factors, cues, and delays can impact the dynamics of temporal expansion associated with an intended outcome.

Early Detection of Human Decision-Making in Concealed Object Visual Searching Tasks: An EEG-BiLSTM Study

Detecting concealed objects presents a significant challenge for human and artificial intelligent systems. Detecting concealed objects task necessitates a high level of human attention and cognitive effort to complete the task successfully. Thus, in this study, we use concealed objects as stimuli for our decision-making experimental paradigms to quantify participants' decision-making performance. We applied a deep learning model, Bi-directional Long Short Term Memory (BiLSTM), to predict the participant's decision accuracy by using their electroencephalogram (EEG) signals as input. The classifier model demonstrated high accuracy, reaching 96.1% with an epoching time range of 500 ms following the stimulus event onset. The results revealed that the parietal-occipital brain region provides highly informative information for the classifier in the concealed visual searching tasks. Furthermore, the neural mechanism underlying the concealed visual-searching and decision-making process was explained by analyzing serial EEG components. The findings of this study could contribute to the development of a fault alert system, which has the potential to improve human decision-making performance.

Intention affects fairness processing: Evidence from behavior and representational similarity analysis of event-related potential signals

In an ultimatum game, the responder must decide between pursuing self-interest and insisting on fairness, and these choices are affected by the intentions of the proposer. However, the time course of this social decision-making process is unclear. Representational similarity analysis (RSA) is a useful technique for linking brain activity with rich behavioral data sets. In this study, electroencephalography (EEG) was used to measure the time course of neural responses to proposed allocation schemes with different intentions. Twenty-eight participants played an ultimatum game as responders. They had to choose between accepting and rejecting the fair or unfair money allocation schemes of proposers. The schemes were offered based on the proposer's selfish intention (monetary gain), altruistic intention (donation to charity), or ambiguous intention (unknown to the responder). We used a spatiotemporal RSA and inter-subject RSA (IS-RSA) to explore the connections between event-related potentials (ERPs) after offer presentation and intention presentation with four types of behavioral data (acceptance, response time, fairness ratings, and pleasantness ratings). The spatiotemporal RSA results revealed that only response time variation was linked with the difference in ERPs at 432-592 ms after offer presentation on the posterior parietal and prefrontal regions. Meanwhile, the IS-RSA results found a significant association between inter-individual differences in response time and differences in ERP activity at 596-812 ms after the presentation of ambiguous intention, particularly in the prefrontal region. This study expands the intention-based reciprocal model to the third-party context and demonstrates that brain activity can represent response time differences in social decision-making.

Monkey Dorsolateral Prefrontal Cortex Represents Abstract Visual Sequences during a No-Report Task

Monitoring sequential information is an essential component of our daily lives. Many of these sequences are abstract, in that they do not depend on the individual stimuli, but do depend on an ordered set of rules (e.g., chop then stir when cooking). Despite the ubiquity and utility of abstract sequential monitoring, little is known about its neural mechanisms. Human rostrolateral prefrontal cortex (RLPFC) exhibits specific increases in neural activity (i.e., "ramping") during abstract sequences. Monkey dorsolateral prefrontal cortex (DLPFC) has been shown to represent sequential information in motor (not abstract) sequence tasks, and contains a subregion, area 46, with homologous functional connectivity to human RLPFC. To test the prediction that area 46 may represent abstract sequence information, and do so with parallel dynamics to those found in humans, we conducted functional magnetic resonance imaging (fMRI) in three male monkeys. When monkeys performed no-report abstract sequence viewing, we found that left and right area 46 responded to abstract sequential changes. Interestingly, responses to rule and number changes overlapped in right area 46 and left area 46 exhibited responses to abstract sequence rules with changes in ramping activation, similar to that observed in humans. Together, these results indicate that monkey DLPFC monitors abstract visual sequential information, potentially with a preference for different dynamics in the two hemispheres. More generally, these results show that abstract sequences are represented in functionally homologous regions across monkeys and humans.SIGNIFICANCE STATEMENT Daily, we complete sequences that are "abstract" because they depend on an ordered set of rules (e.g., chop then stir when cooking) rather than the identity of individual items. Little is known about how the brain tracks, or monitors, this abstract sequential information. Based on previous human work showing abstract sequence related dynamics in an analogous area, we tested whether monkey dorsolateral prefrontal cortex (DLPFC), specifically area 46, represents abstract sequential information using awake monkey functional magnetic resonance imaging (fMRI). We found that area 46 responded to abstract sequence changes, with a preference for more general responses on the right and dynamics similar to humans on the left. These results suggest that abstract sequences are represented in functionally homologous regions across monkeys and humans.

Brain dynamics of visual anticipation during spatial occlusion tasks in expert tennis players

Stimulus identification and action outcome understanding for a rapid and accurate response selection, play a fundamental role in racquet sports. Here, we investigated the neurodynamics of visual anticipation in tennis manipulating the postural and kinematic information associated with the body of opponents by means of a spatial occlusion protocol. Event Related Potentials (ERPs) were evaluated in two groups of professional tennis players (N = 37) with different levels of expertise, while they observed pictures of opponents and predicted the landing position as fast and accurately as possible. The observed action was manipulated by deleting different body districts of the opponent (legs, ball, racket and arm, trunk). Full body image (no occlusion) was used as control condition. The worst accuracy and the slowest response time were observed in the occlusion of trunk and ball. The former was associated with a reduced amplitude of the ERP components likely linked to body processing (the N1 in the right hemisphere) and visual-motor integration awareness (the pP1), as well as with an increase of the late frontal negativity (the pN2), possibly reflecting an effort by the insula to recover and/or complete the most correct sensory-motor representation. In both occlusions, a decrease in the pP2 may reflect an impairment of decisional processes upon action execution following sensory evidence accumulation. Enhanced amplitude of the P3 and the pN2 components were found in more experienced players, suggesting a greater allocation of resources in the process connecting sensory encoding and response execution, and sensory-motor representation.

The impacts of training on change deafness and build-up in a flicker task

Performance on auditory change detection tasks can be improved by training. We examined the stimulus specificity of these training effects in behavior and ERPs. A flicker change detection task was employed in which spatialized auditory scenes were alternated until a "change" or "same" response was made. For half of the trials, scenes were identical. The other half contained changes in the spatial locations of objects from scene to scene. On Day 1, participants were either trained on this auditory change detection task (trained group), or trained on a non-auditory change detection task (control group). On Day 2, all participants were tested on the flicker task while EEG was recorded. The trained group showed greater change detection accuracy than the control group. They were less biased to respond "same" and showed full generalization of learning from trained to novel auditory objects. ERPs for "change" compared to "same" trials showed more negative going P1, N1, and P2 amplitudes, as well as a larger P3b amplitude. The P3b amplitude also differed between the trained and control group, with larger amplitudes for the trained group. Analysis of ERPs to scenes viewed prior to a decision revealed build-up of a difference between "change" and "same" trials in N1 and P2. Results demonstrate that training has an impact early in the "same" versus "change" decision-making process, and that the flicker paradigm combined with the ERP method can be used to study the build-up of change detection in auditory scenes.

Intention-based and sensory-based predictions

We inhabit a continuously changing world, where the ability to anticipate future states of the environment is critical for adaptation. Anticipation can be achieved by learning about the causal or temporal relationship between sensory events, as well as by learning to act on the environment to produce an intended effect. Together, sensory-based and intention-based predictions provide the flexibility needed to successfully adapt. Yet it is currently unknown whether the two sources of information are processed independently to form separate predictions, or are combined into a common prediction. To investigate this, we ran an experiment in which the final tone of two possible four-tone sequences could be predicted from the preceding tones in the sequence and/or from the participants' intention to trigger that final tone. This tone could be congruent with both sensory-based and intention-based predictions, incongruent with both, or congruent with one while incongruent with the other. Trials where predictions were incongruent with each other yielded similar prediction error responses irrespectively of the violated prediction, indicating that both predictions were formulated and coexisted simultaneously. The violation of intention-based predictions yielded late additional error responses, suggesting that those violations underwent further differential processing which the violations of sensory-based predictions did not receive.

Neural Encoding of Auditory Statistics

The human brain extracts statistical regularities embedded in real-world scenes to sift through the complexity stemming from changing dynamics and entwined uncertainty along multiple perceptual dimensions (e.g., pitch, timbre, location). While there is evidence that sensory dynamics along different auditory dimensions are tracked independently by separate cortical networks, how these statistics are integrated to give rise to unified objects remains unknown, particularly in dynamic scenes that lack conspicuous coupling between features. Using tone sequences with stochastic regularities along spectral and spatial dimensions, this study examines behavioral and electrophysiological responses from human listeners (male and female) to changing statistics in auditory sequences and uses a computational model of predictive Bayesian inference to formulate multiple hypotheses for statistical integration across features. Neural responses reveal multiplexed brain responses reflecting both local statistics along individual features in frontocentral networks, together with global (object-level) processing in centroparietal networks. Independent tracking of local surprisal along each acoustic feature reveals linear modulation of neural responses, while global melody-level statistics follow a nonlinear integration of statistical beliefs across features to guide perception. Near identical results are obtained in separate experiments along spectral and spatial acoustic dimensions, suggesting a common mechanism for statistical inference in the brain. Potential variations in statistical integration strategies and memory deployment shed light on individual variability between listeners in terms of behavioral efficacy and fidelity of neural encoding of stochastic change in acoustic sequences.SIGNIFICANCE STATEMENT The world around us is complex and ever changing: in everyday listening, sound sources evolve along multiple dimensions, such as pitch, timbre, and spatial location, and they exhibit emergent statistical properties that change over time. In the face of this complexity, the brain builds an internal representation of the external world by collecting statistics from the sensory input along multiple dimensions. Using a Bayesian predictive inference model, this work considers alternative hypotheses for how statistics are combined across sensory dimensions. Behavioral and neural responses from human listeners show the brain multiplexes two representations, where local statistics along each feature linearly affect neural responses, and global statistics nonlinearly combine statistical beliefs across dimensions to shape perception of stochastic auditory sequences.

Predictive Modulation of Corticospinal Excitability and Implicit Encoding of Movement Probability in Schizophrenia

The ability to infer from uncertain information is impaired in schizophrenia and is associated with hallucinations and false beliefs. The accumulation of information is a key process for generating a predictive internal model, which statistically estimates an outcome from a specific situation. This study examines if updating the predictive model by the accumulation of information in absence of feedback is impaired in schizophrenia. We explored the implicit adaptation to the probability of being instructed to perform a movement (33%-Go, 50%-Go, or 66%-Go) in a Go/NoGo task in terms of reaction times (RTs), electromyographic activity, and corticospinal excitability (CSE) of primary motor cortex (M1). CSE was assessed at two time points to evaluate prediction of the upcoming instruction based on previously accumulated information: at rest (preceding the warning signal) and at the Go/NoGo signal onset. Three groups were compared: patients with schizophrenia (n = 20), unaffected siblings (n = 16), and healthy controls (n = 20). Controls and siblings showed earlier movement onset and increased CSE with higher Go probability. CSE adaptation seemed long-lasting, because the two CSE measures, at least 1500 ms apart, strongly correlated. Patients with schizophrenia failed to show movement onset (RT) adaptation and modulation of CSE. In contrast, all groups decreased movement duration with increasing Go probability. Modulation of CSE in the anticipatory phase of the potential movement reflected the estimation of upcoming response probability in unaffected controls and siblings. Impaired modulation of CSE supports the hypothesis that implicit adaptation to probabilistic context is altered in schizophrenia.

"Hit the missing stimulus". A simultaneous EEG-fMRI study to localize the generators of endogenous ERPs in an omitted target paradigm

Event-Related Potentials (ERPs) occurring independently from any stimulus are purely endogenous (emitted potentials) and their neural generators can be unequivocally linked with cognitive processes. In the present study, the subjects performed two similar visual counting tasks: a standard two-stimulus oddball, and an omitted-target oddball task, characterized by the physical absence of the target stimulus. Our investigation aimed at localizing the neural sources of the scalp-recorded endogenous/emitted ERPs. To optimize the source localization, the high temporal resolution of electrophysiology was combined with the fine spatial information provided by the simultaneous recording of functional magnetic resonance (fMRI). Both tasks identified two endogenous ERP components in the 300 to 520 ms interval. An earlier component, pP2, showed a bilateral generator in the anterior Insula. A later P3 component (P3b) was generated bilaterally in the temporal-parietal junction, the premotor and motor area and the anterior intraparietal sulcus (this latter one only in the standard oddball). Anticipatory slow waves (beginning 900 to 500 ms pre-stimulus), also of endogenous nature, were produced by the inferior and middle frontal gyrus and the supplementary and cingulate motor areas. Our protocol disentangled pre- from post-stimulus fMRI activations and provided original clues to the psychophysiological interpretation of emitted/endogenous ERPs.