Recalibration of subjective simultaneity between self-generated movement and delayed auditory feedback

Monitoring prediction errors facilitates cognition in action

Cognition in action requires strategic allocation of attention between internal processes and the sensory environment. We hypothesized that this resource allocation could be facilitated by mechanisms that predict sensory results of self-generated actions. Sensory signals conforming to predictions would be safely ignored to facilitate focus on internally generated content, whereas those violating predictions would draw attention for additional scrutiny. During a visual-verbal serial digit-recall task, we varied the temporal relationship between task-irrelevant keypresses and auditory distractors so that the distractors were either temporally coupled or decoupled with keypresses. Consistent with our hypothesis, distractors were more likely to interfere with target maintenance and intrude into working memory when they were decoupled from keypresses, thereby violating action-based sensory predictions. Interference was maximal when sounds preceded keypresses, suggesting that stimuli were most distracting when their timing was inconsistent with expected action-sensation contingencies. In a follow-up experiment, neither auditory nor visual cues to distractor timing produced similar effects, suggesting a unique action-based mechanism. These results suggest that action-based sensory predictions are used to dynamically optimize attentional allocation during cognition in action. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

'Robot' Hand Illusion under Delayed Visual Feedback: Relationship between the Senses of Ownership and Agency

The rubber hand illusion (RHI) is an illusion of the self-ownership of a rubber hand that is touched synchronously with one’s own hand. While the RHI relates to visual and tactile integration, we can also consider a similar illusion with visual and motor integration on a fake hand. We call this a “robot hand illusion” (RoHI), which relates to both the senses of ownership and agency. Here we investigate the effect of delayed visual feedback on the RoHI. Participants viewed a virtual computer graphic hand controlled by their hand movement recorded using a data glove device. We inserted delays of various lengths between the participant’s hand and the virtual hand movements (90–590 ms), and the RoHI effects for each delay condition were systematically tested using a questionnaire. The results showed that the participants felt significantly greater RoHI effects with temporal discrepancies of less than 190 ms compared with longer temporal discrepancies, both in the senses of ownership and agency. Additionally, participants felt significant, but weaker, RoHI effects with temporal discrepancies of 290–490 ms in the sense of agency, but not in the sense of ownership. The participants did not feel a RoHI with temporal discrepancies of 590 ms in either the senses of agency or ownership. Our results suggest that a time window of less than 200 ms is critical for multi-sensory integration processes constituting self-body image.

Neural Basis of the Time Window for Subjective Motor-Auditory Integration

Temporal contiguity between an action and corresponding auditory feedback is crucial to the perception of self-generated sound. However, the neural mechanisms underlying motor–auditory temporal integration are unclear. Here, we conducted four experiments with an oddball paradigm to examine the specific event-related potentials (ERPs) elicited by delayed auditory feedback for a self-generated action. The first experiment confirmed that a pitch-deviant auditory stimulus elicits mismatch negativity (MMN) and P300, both when it is generated passively and by the participant’s action. In our second and third experiments, we investigated the ERP components elicited by delayed auditory feedback for a self-generated action. We found that delayed auditory feedback elicited an enhancement of P2 (enhanced-P2) and a N300 component, which were apparently different from the MMN and P300 components observed in the first experiment. We further investigated the sensitivity of the enhanced-P2 and N300 to delay length in our fourth experiment. Strikingly, the amplitude of the N300 increased as a function of the delay length. Additionally, the N300 amplitude was significantly correlated with the conscious detection of the delay (the 50% detection point was around 200 ms), and hence reduction in the feeling of authorship of the sound (the sense of agency). In contrast, the enhanced-P2 was most prominent in short-delay (≤200 ms) conditions and diminished in long-delay conditions. Our results suggest that different neural mechanisms are employed for the processing of temporally deviant and pitch-deviant auditory feedback. Additionally, the temporal window for subjective motor–auditory integration is likely about 200 ms, as indicated by these auditory ERP components.

Auditory dominance in motor-sensory temporal recalibration

Perception of synchrony between one’s own action (e.g. a finger tap) and the sensory feedback thereof (e.g. a flash or click) can be shifted after exposure to an induced delay (temporal recalibration effect, TRE). It remains elusive, however, whether the same mechanism underlies motor-visual (MV) and motor-auditory (MA) TRE. We examined this by measuring crosstalk between MV- and MA-delayed feedbacks. During an exposure phase, participants pressed a mouse at a constant pace while receiving visual or auditory feedback that was either delayed (+150 ms) or subjectively synchronous (+50 ms). During a post-test, participants then tried to tap in sync with visual or auditory pacers. TRE manifested itself as a compensatory shift in the tap–pacer asynchrony (a larger anticipation error after exposure to delayed feedback). In experiment 1, MA and MV feedback were either both synchronous (MV-sync and MA-sync) or both delayed (MV-delay and MA-delay), whereas in experiment 2, different delays were mixed across alternating trials (MV-sync and MA-delay or MV-delay and MA-sync). Exposure to consistent delays induced equally large TREs for auditory and visual pacers with similar build-up courses. However, with mixed delays, we found that synchronized sounds erased MV-TRE, but synchronized flashes did not erase MA-TRE. These results suggest that similar mechanisms underlie MA- and MV-TRE, but that auditory feedback is more potent than visual feedback to induce a rearrangement of motor-sensory timing.

Delayed Visual Feedback of One's Own Action Promotes Sense of Control for Auditory Events

Sense of control refers to one’s feelings to control environmental events through one’s own action. A prevailing view is that the sense of control is strong (or is not diminished) when predicted sensory signals, which are generated in motor control mechanisms, are consistent with afferent sensory signals. Such intact sense of control often leads to the misjudgment of temporal relation between timings of one’s action and its effect (so-called, intentional binding). The present study showed that the intentional binding could be enhanced by the delayed visual feedback of an agent’s action. We asked participants to press a button to produce a tone as action outcome. In some conditions, they were given the delayed visual feedback of their button press. Participants judged whether the onset of the auditory outcome was delayed from the timing of their button press. Consequently, delay detection thresholds were significantly higher when the feedback was given 0.2 and 0.4 s delays than when no feedback was displayed to the participants. The results indicate that action agents misjudge the timing of their action (button press) in the presence of the delayed visual feedback of their action. Interestingly, delay detection thresholds were strongly correlated with the subjective magnitude of the sense of control. Thus, the sense of control is possibly determined by cross-modal processing for action-related and outcome-related sensory signals.