When temporal prediction errs: ERP responses to delayed action-feedback onset

Hallucination Proneness Alters Sensory Feedback Processing in Self-voice Production

BACKGROUND

Sensory suppression occurs when hearing one's self-generated voice, as opposed to passively listening to one's own voice. Quality changes in sensory feedback to the self-generated voice can increase attentional control. These changes affect the self-other voice distinction and might lead to hearing voices in the absence of an external source (ie, auditory verbal hallucinations). However, it is unclear how changes in sensory feedback processing and attention allocation interact and how this interaction might relate to hallucination proneness (HP).

STUDY DESIGN

Participants varying in HP self-generated (via a button-press) and passively listened to their voice that varied in emotional quality and certainty of recognition-100% neutral, 60%-40% neutral-angry, 50%-50% neutral-angry, 40%-60% neutral-angry, 100% angry, during electroencephalography (EEG) recordings.

STUDY RESULTS

The N1 auditory evoked potential was more suppressed for self-generated than externally generated voices. Increased HP was associated with (1) an increased N1 response to the self- compared with externally generated voices, (2) a reduced N1 response for angry compared with neutral voices, and (3) a reduced N2 response to unexpected voice quality in sensory feedback (60%-40% neutral-angry) compared with neutral voices.

CONCLUSIONS

The current study highlights an association between increased HP and systematic changes in the emotional quality and certainty in sensory feedback processing (N1) and attentional control (N2) in self-voice production in a nonclinical population. Considering that voice hearers also display these changes, these findings support the continuum hypothesis.

Violation of identity-specific action-effect prediction increases pupil size and attenuates auditory event-related potentials at P2 latencies when action-effects are behaviorally relevant

Self-initiated sensory action effects are widely assumed to lead to less intense perception and reduced neural responses compared to externally triggered stimuli (sensory attenuation). However, it is unclear if sensory attenuation occurs in all cases of action-effect prediction. Specifically, when predicted action-effects are relevant to determine follow-up actions attenuation could be detrimental. We quantified auditory event-related potentials (ERP) in electroencephalography (EEG) when human participants created two-sound sequences by pressing two keys on a keyboard associated with different pitch, giving rise to identity-specific action-effect prediction after the first keypress. The first sound corresponded to (congruent) or violated (incongruent) the predicted pitch and was either relevant for the selection of the second keypress to correctly complete the sequence (Relevance) or irrelevant (Control Movement), or there was only one keypress and sound (Baseline). We found a diminished P2-timed ERP component in incongruent compared to congruent trials when the sound was relevant for the subsequent action. This effect of action-effect prediction was due to an ERP reduction for incongruent relevant sounds compared to incongruent irrelevant sounds at P2 latencies and correlated negatively with modulations of pupil dilation. Contrary to our expectation, we did not observe an N1 modulation by congruency in any condition. Attenuation of the N1 component seems absent for predicted identity-specific auditory action effects, while P2-timed ERPs as well as pupil size are sensitive to predictability, at least when action effects are relevant for the selection of the next action. Incongruent relevant stimuli thereby take a special place and seem to be subject to attentional modulations and error processing.

Suppression and omission effects in auditory predictive processing-Two of the same?

Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two different modulations of the auditory N1 event-related brain potential component are often discussed as an expression of auditory predictive processing. The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). Both phenomena were explained by action-related forward modelling, which takes place when the sensory input is predictable: prediction error signals are reduced when predicted sensory input is presented (N1 suppression) and elicited when predicted sensory input is omitted (omission N1). This common theoretical account is appealing but has not yet been directly tested. We manipulated the predictability of a sound in a self-generation paradigm in which, in two conditions, either 80% or 50% of the button presses did generate a sound, inducing a strong or a weak expectation for the occurrence of the sound. Consistent with the forward modelling account, an omission N1 was observed in the 80% but not in the 50% condition. However, N1 suppression was highly similar in both conditions. Thus, our results demonstrate a clear effect of predictability for the omission N1 but not for the N1 suppression. These results imply that the two phenomena rely (at least in part) on different mechanisms and challenge prediction related accounts of N1 suppression.

The neural network of sensory attenuation: A neuroimaging meta-analysis

Sensory attenuation refers to the reduction in sensory intensity resulting from self-initiated actions compared to stimuli initiated externally. A classic example is scratching oneself without feeling itchy. This phenomenon extends across various sensory modalities, including visual, auditory, somatosensory, and nociceptive stimuli. The internal forward model proposes that during voluntary actions, an efferent copy of the action command is sent out to predict sensory feedback. This predicted sensory feedback is then compared with the actual sensory feedback, leading to the suppression or reduction of sensory stimuli originating from self-initiated actions. To further elucidate the neural mechanisms underlying sensory attenuation effect, we conducted an extensive meta-analysis of functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies. Utilizing activation likelihood estimation (ALE) analysis, our results revealed significant activations in a prominent cluster encompassing the right superior temporal gyrus (rSTG), right middle temporal gyrus (rMTG), and right insula when comparing external-generated with self-generated conditions. Additionally, significant activation was observed in the right anterior cerebellum when comparing self-generated to external-generated conditions. Further analysis using meta-analytic connectivity modeling (MACM) unveiled distinct brain networks co-activated with the rMTG and right cerebellum, respectively. Based on these findings, we propose that sensory attenuation arises from the suppression of reflexive inputs elicited by self-initiated actions through the internal forward modeling of a cerebellum-centered action prediction network, enabling the "sensory conflict detection" regions to effectively discriminate between inputs resulting from self-induced actions and those originating externally.

Neural signatures of memory gain through active exploration in an oculomotor-auditory learning task

Active engagement improves learning and memory, and self- versus externally generated stimuli are processed differently: perceptual intensity and neural responses are attenuated. Whether the attenuation is linked to memory formation remains unclear. This study investigates whether active oculomotor control over auditory stimuli-controlling for movement and stimulus predictability-benefits associative learning, and studies the underlying neural mechanisms. Using EEG and eye tracking we explored the impact of control during learning on the processing and memory recall of arbitrary oculomotor-auditory associations. Participants (N = 23) learned associations through active exploration or passive observation, using a gaze-controlled interface to generate sounds. Our results show faster learning progress in the active condition. ERPs time-locked to the onset of sound stimuli showed that learning progress was linked to an attenuation of the P3a component. The detection of matching movement-sound pairs triggered a target-matching P3b. There was no general modulation of ERPs through active learning. However, we found continuous variation in the strength of the memory benefit across participants: some benefited more strongly from active control during learning than others. This was paralleled in the strength of the N1 attenuation effect for self-generated stimuli, which was correlated with memory gain in active learning. Our results show that control helps learning and memory and modulates sensory responses. Individual differences during sensory processing predict the strength of the memory benefit. Taken together, these results help to disentangle the effects of agency, unspecific motor-based neuromodulation, and predictability on ERP components and establish a link between self-generation effects and active learning memory gain.

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Actions in the real world have immediate sensory consequences. Mimicking these in digital environments is within reach, but technical constraints usually impose a certain latency (delay) between user actions and system responses. It is important to assess the impact of this latency on the users, ideally with measurement techniques that do not interfere with their digital experience. One such unobtrusive technique is electroencephalography (EEG), which can capture the users' brain activity associated with motor responses and sensory events by extracting event-related potentials (ERPs) from the continuous EEG recording. Here we exploit the fact that the amplitude of sensory ERP components (specifically, N1 and P2) reflects the degree to which the sensory event was perceived as an expected consequence of an own action (self-generation effect). Participants (N = 24) elicit auditory events in a virtual-reality (VR) setting by entering codes on virtual keypads to open doors. In a within-participant design, the delay between user input and sound presentation is manipulated across blocks. Occasionally, the virtual keypad is operated by a simulated robot instead, yielding a control condition with externally generated sounds. Results show that N1 (but not P2) amplitude is reduced for self-generated relative to externally generated sounds, and P2 (but not N1) amplitude is modulated by delay of sound presentation in a graded manner. This dissociation between N1 and P2 effects maps back to basic research on self-generation of sounds. We suggest P2 amplitude as a candidate read-out to assess the quality and immersiveness of digital environments with respect to system latency.

Time discrimination and change detection could share a common brain network: findings of a task-based fMRI study

Introduction

Over the past few years, several studies have described the brain activation pattern related to both time discrimination (TD) and change detection processes. We hypothesize that both processes share a common brain network which may play a significant role in more complex cognitive processes. The main goal of this proof-of-concept study is to describe the pattern of brain activity involved in TD and oddball detection (OD) paradigms, and in processes requiring higher cognitive effort.

Methods

We designed an experimental task, including an auditory test tool to assess TD and OD paradigms, which was conducted under functional magnetic resonance imaging (fMRI) in 14 healthy participants. We added a cognitive control component into both paradigms in our test tool. We used the general linear model (GLM) to analyze the individual fMRI data images and the random effects model for group inference.

Results

We defined the areas of brain activation related to TD and OD paradigms. We performed a conjunction analysis of contrast TD (task > control) and OD (task > control) patterns, finding both similarities and significant differences between them.

Discussion

We conclude that change detection and other cognitive processes requiring an increase in cognitive effort require participation of overlapping functional and neuroanatomical components, suggesting the presence of a common time and change detection network. This is of particular relevance for future research on normal cognitive functioning in the healthy population, as well as for the study of cognitive impairment and clinical manifestations associated with various neuropsychiatric conditions such as schizophrenia.

Auditory N1 and P2 Attenuation in Action Observation: An Event-Related Potential Study Considering Effects of Temporal Predictability and Individualism

Tones that are generated by self-performed actions elicit attenuated N1 and P2 amplitudes, as measured by electroencephalography (EEG), compared to identical external tones, which is referred to as neurophysiological sensory attenuation (SA). At the same time, self-generated tones are perceived as less loud compared to external tones (perceptual SA). Action observation led in part to a similar neurophysiological and perceptual SA. The perceptual SA in observers was found in comparison to tones that were temporally predictable, and one study suggested that perceptual SA in observers might depend on the cultural dimension of individualism. In this study, we examined neurophysiological SA for tones elicited by self-performed and observed actions during simultaneous EEG acquisitions in two participants, extending the paradigm with a visual cue condition controlling for effects of temporal predictability. Moreover, we investigated the effect of individualism on neurophysiological SA in action observation. Relative to un-cued external tones, the N1 was only descriptively reduced for tones that were elicited by self-performed or observed actions and significantly attenuated for cued external tones. A P2 attenuation effect relative to un-cued external tones was found in all three conditions, with stronger effects for self- and other-generated tones than for cued external tones. We found no evidence for an effect of individualism. These findings add to previous evidence for neurophysiological SA in action performance and observation with a paradigm well-controlled for the effect of predictability and individualism, showing differential effects of the former on the N1 and P2 components, and no effect of the latter.

My view on your actions: Dynamic changes in viewpoint-dependent auditory ERP attenuation during action observation

It has been suggested that during action observation, a sensory representation of the observed action is mapped onto one's own motor system. However, it is largely unexplored what this may imply for the early processing of the action's sensory consequences, whether the observational viewpoint exerts influence on this and how such a modulatory effect might change over time. We tested whether the event-related potential of auditory effects of actions observed from a first- versus third-person perspective show amplitude reductions compared with externally generated sounds, as revealed for self-generated sounds. Multilevel modeling on trial-level data showed distinct dynamic patterns for the two viewpoints on reductions of the N1, P2, and N2 components. For both viewpoints, an N1 reduction for sounds generated by observed actions versus externally generated sounds was observed. However, only during first-person observation, we found a temporal dynamic within experimental runs (i.e., the N1 reduction only emerged with increasing trial number), indicating time-variant, viewpoint-dependent processes involved in sensorimotor prediction during action observation. For the P2, only a viewpoint-independent reduction was found for sounds elicited by observed actions, which disappeared in the second half of the experiment. The opposite pattern was found in an exploratory analysis concerning the N2, revealing a reduction that increased in the second half of the experiment, and, moreover, a temporal dynamic within experimental runs for the first-person perspective, possibly reflecting an agency-related process. Overall, these results suggested that the processing of auditory outcomes of observed actions is dynamically modulated by the viewpoint over time.

Action-based predictions affect visual perception, neural processing, and pupil size, regardless of temporal predictability

Sensory consequences of one's own action are often perceived as less intense, and lead to reduced neural responses, compared to externally generated stimuli. Presumably, such sensory attenuation is due to predictive mechanisms based on the motor command (efference copy). However, sensory attenuation has also been observed outside the context of voluntary action, namely when stimuli are temporally predictable. Here, we aimed at disentangling the effects of motor and temporal predictability-based mechanisms on the attenuation of sensory action consequences. During fMRI data acquisition, participants (N = 25) judged which of two visual stimuli was brighter. In predictable blocks, the stimuli appeared temporally aligned with their button press (active) or aligned with an automatically generated cue (passive). In unpredictable blocks, stimuli were presented with a variable delay after button press/cue, respectively. Eye tracking was performed to investigate pupil-size changes and to ensure proper fixation. Self-generated stimuli were perceived as darker and led to less neural activation in visual areas than their passive counterparts, indicating sensory attenuation for self-generated stimuli independent of temporal predictability. Pupil size was larger during self-generated stimuli, which correlated negatively with the blood oxygenation level dependent (BOLD) response: the larger the pupil, the smaller the BOLD amplitude in visual areas. Our results suggest that sensory attenuation in visual cortex is driven by action-based predictive mechanisms rather than by temporal predictability. This effect may be related to changes in pupil diameter. Altogether, these results emphasize the role of the efference copy in the processing of sensory action consequences.

The Role of Action-Effect Contingency on Sensory Attenuation in the Absence of Movement

Stimuli that have been generated by a person's own willed motor actions generally elicit a suppressed electrophysiological, as well as phenomenological, response than identical stimuli that have been externally generated. This well-studied phenomenon, known as sensory attenuation, has mostly been studied by comparing ERPs evoked by self-initiated and externally generated sounds. However, most studies have assumed a uniform action-effect contingency, in which a motor action leads to a resulting sensation 100% of the time. In this study, we investigated the effect of manipulating the probability of action-effect contingencies on the sensory attenuation effect. In Experiment 1, participants watched a moving, marked tickertape while EEG was recorded. In the full-contingency (FC) condition, participants chose whether to press a button by a certain mark on the tickertape. If a button press had not occurred by the mark, a sound would be played a second later 100% of the time. If the button was pressed before the mark, the sound was not played. In the no-contingency (NC) condition, participants observed the same tickertape; in contrast, however, if participants did not press the button by the mark, a sound would occur only 50% of the time (NC-inaction). Furthermore, in the NC condition, if a participant pressed the button before the mark, a sound would also play 50% of the time (NC-action). In Experiment 2, the design was identical, except that a willed action (as opposed to a willed inaction) triggered the sound in the FC condition. The results were consistent across the two experiments: Although there were no differences in N1 amplitude between conditions, the amplitude of the Tb and P2 components were smaller in the FC condition compared with the NC-inaction condition, and the amplitude of the P2 component was also smaller in the FC condition compared with the NC-action condition. The results suggest that the effect of contingency on electrophysiological indices of sensory attenuation may be indexed primarily by the Tb and P2 components, rather than the N1 component which is most commonly studied.

Attenuation of auditory N2 for self-modulated tones during continuous actions

Event-related potentials elicited by tones generated by one's own discrete actions (e.g., button presses) are attenuated compared to those elicited by tones generated externally. The present study investigated whether ERP attenuation would occur when the timing or pitch of tones is modulated by continuous actions, as for such actions, a weak association between actions and their auditory consequences is assumed. In a modulation condition, participants modulated the time interval between tones (Experiment 1) or the pitch of tones (Experiment 2) by turning a steering wheel. In a listening condition, participants listened to the same tones as in the modulation condition without any action. The results revealed that the amplitude of N2 elicited by tones decreased in the modulation compared to listening conditions, consistently in the two experiments, suggesting relatively higher-order auditory processing can be mainly influenced by the prediction of action consequences when continuous actions modulate features of auditory stimuli.

The auditory brain in action: Intention determines predictive processing in the auditory system-A review of current paradigms and findings

According to the ideomotor theory, action may serve to produce desired sensory outcomes. Perception has been widely described in terms of sensory predictions arising due to top-down input from higher order cortical areas. Here, we demonstrate that the action intention results in reliable top-down predictions that modulate the auditory brain responses. We bring together several lines of research, including sensory attenuation, active oddball, and action-related omission studies: Together, the results suggest that the intention-based predictions modulate several steps in the sound processing hierarchy, from preattentive to evaluation-related processes, also when controlling for additional prediction sources (i.e., sound regularity). We propose an integrative theoretical framework—the extended auditory event representation system (AERS), a model compatible with the ideomotor theory, theory of event coding, and predictive coding. Initially introduced to describe regularity-based auditory predictions, we argue that the extended AERS explains the effects of action intention on auditory processing while additionally allowing studying the differences and commonalities between intention- and regularity-based predictions—we thus believe that this framework could guide future research on action and perception.

Midfrontal theta oscillation encodes haptic delay

Haptic technologies aim to simulate tactile or kinesthetic interactions with a physical or virtual environment in order to enhance user experience and/or performance. However, due to stringent communication and computational needs, the user experience is influenced by delayed haptic feedback. While delayed feedback is well understood in the visual and auditory modalities, little research has systematically examined the neural correlates associated with delayed haptic feedback. In this paper, we used electroencephalography (EEG) to study sensory and cognitive neural correlates caused by haptic delay during passive and active tasks performed using a haptic device and a computer screen. Results revealed that theta power oscillation was significantly higher at the midfrontal cortex under the presence of haptic delay. Sensory correlates represented by beta rebound were found to be similar in the passive task and different in the active task under the delayed and synchronous conditions. Additionally, the event related potential (ERP) P200 component is modulated under the haptic delay condition during the passive task. The P200 amplitude significantly reduced in the last 20% of trials during the passive task and in the absence of haptic delay. Results suggest that haptic delay could be associated with increased cognitive control processes including multi-sensory divided attention followed by conflict detection and resolution with an earlier detection during the active task. Additionally, haptic delay tends to generate greater perceptual attention that does not significantly decay across trials during the passive task.

Sensory attenuation in the absence of movement: Differentiating motor action from sense of agency

Sensory attenuation is the phenomenon that stimuli generated by willed motor actions elicit a smaller neurophysiological response than those generated by external sources. It has mostly been investigated in the auditory domain, by comparing ERPs evoked by self-initiated (active condition) and externally-generated (passive condition) sounds. The mechanistic basis of sensory attenuation has been argued to involve a duplicate of the motor command being used to predict sensory consequences of self-generated movements. An alternative possibility is that the effect is driven by between-condition differences in participants' sense of agency over the sound. In this paper, we disambiguated the effects of motor-action and sense of agency on sensory attenuation with a novel experimental paradigm. In Experiment 1, participants watched a moving, marked tickertape while EEG was recorded. In the active condition, participants chose whether to press a button by a certain mark on the tickertape. If a button-press had not occurred by the mark, then a tone would be played 1 s later. If the button was pressed prior to the mark, the tone was not played. In the passive condition, participants passively watched the animation, and were informed about whether a tone would be played on each trial. The design for Experiment 2 was identical, except that the contingencies were reversed (i.e., a button-press by the mark led to a tone). The results were consistent across the two experiments: while there were no differences in N1 amplitude between the active and passive conditions, the amplitude of the Tb component was suppressed in the active condition. The amplitude of the P2 component was enhanced in the active condition in both Experiments 1 and 2. These results suggest that motor-actions and sense of agency have differential effects on sensory attenuation to sounds and are indexed with different ERP components.

Sensory attenuation is modulated by the contrasting effects of predictability and control

Self-generated stimuli have been found to elicit a reduced sensory response compared with externally-generated stimuli. However, much of the literature has not adequately controlled for differences in the temporal predictability and temporal control of stimuli. In two experiments, we compared the N1 (and P2) components of the auditory-evoked potential to self- and externally-generated tones that differed with respect to these two factors. In Experiment 1 (n = 42), we found that increasing temporal predictability reduced N1 amplitude in a manner that may often account for the observed reduction in sensory response to self-generated sounds. We also observed that reducing temporal control over the tones resulted in a reduction in N1 amplitude. The contrasting effects of temporal predictability and temporal control on N1 amplitude meant that sensory attenuation prevailed when controlling for each. Experiment 2 (n = 38) explored the potential effect of selective attention on the results of Experiment 1 by modifying task requirements such that similar levels of attention were allocated to the visual stimuli across conditions. The results of Experiment 2 replicated those of Experiment 1, and suggested that the observed effects of temporal control and sensory attenuation were not driven by differences in attention. Given that self- and externally-generated sensations commonly differ with respect to both temporal predictability and temporal control, findings of the present study may necessitate a re-evaluation of the experimental paradigms used to study sensory attenuation.