On the importance of being first: serial order effects in the interaction between action plans and ongoing actions

What is behind partial repetition costs? Event-files do not fully occupy bound feature codes

Feature binding accounts state that features of perceived and produced events are bound into event-files. Performance while responding to an event is impaired when some, as opposed to all or none, of this event's features already belong to a previous event-file. While these partial repetition costs are generally considered to be indicators of feature binding, their cause is still unclear. Possibly, features are fully occupied when bound in an event-file and must be unbound in a time-consuming process before they can enter a novel event-file. In this study, we tested this code occupation account. Participants responded to the font color (target) of a word (distractor) by pressing one of three keys (response) while ignoring the word meaning. We measured partial repetition costs from prime to probe while introducing an intermediate trial. We compared sequences in which this intermediate trial did not repeat any prime features and sequences in which it repeated either the prime response or distractor. Partial repetition costs occurred in the probe, even when one (vs. none) of the prime features repeated in the intermediate trial, although significantly reduced. Thus, single bindings do not fully occupy feature codes. By ruling out a possible mechanism behind partial repetition costs, the present study contributes to the further specification of feature binding accounts.

Perception and action as viewed from the Theory of Event Coding: a multi-lab replication and effect size estimation of common experimental designs

The Theory of Event Coding (TEC) has influenced research on action and perception across the past two decades. It integrates several seminal empirical phenomena and it has continued to stimulate novel experimental approaches on the representational foundations of action control and perceptual experience. Yet, many of the most notable results surrounding TEC originate from an era of psychological research that relied on rather small sample sizes as judged by today's standards. This state hampers future research aiming to build on previous phenomena. We, therefore, provide a multi-lab re-assessment of the following six classical observations: response-effect compatibility, action-induced blindness, response-effect learning, stimulus-response binding, code occupation, and short-term response-effect binding. Our major goal is to provide precise estimates of corresponding effect sizes to facilitate future scientific endeavors. These effect sizes turned out to be considerably smaller than in the original reports, thus allowing for informed decisions on how to address each phenomenon in future work. Of note, the most relevant results of the original observations were consistently obtained in the present experiments as well.

Manual action re-planning interferes with the maintenance process of working memory: an ERP investigation

The current study investigated the re-planning of the grasping movements, its functional interactions with working memory (WM), and underlying neurophysiological activity. Mainly, the current study investigated the movement re-planning interference with WM domains (verbal, visuospatial) and processes (maintenance, retrieval). We combined a cognitive-motor dual-task paradigm with an EEG setting. Thirty-six participants completed the verbal and visuospatial versions of a WM task concurrently with a manual task which required performing a grasp-and-place movement by keeping the initial movement plan (prepared movement condition) or changing it for reversing the movement direction (re-planned movement condition). ERPs were extracted for the prepared and re-planned conditions in the verbal and visuospatial tasks separately during the maintenance and retrieval processes. ERP analyses showed that during the maintenance process of both the verbal and visuospatial tasks, the re-planned movements compared to the prepared movements generated a larger positive slow wave with a centroparietal maximum between 200 and 700. We interpreted this ERP effect as a P300 component for the re-planned movements. There was no ERP difference between the planned and re-planned movements during the retrieval process. Accordingly, we suggest that re-planning the grasp-and-place movement interfered at least with the maintenance of the verbal and visuospatial domains, resulting in the re-planning costs. More generally, the current study provides the initial neurophysiological investigations of the movement re-planning–WM interactions during grasping movements, and contributes to a better understanding of the neurocognitive mechanisms underlying manual action flexibility.

Ideomotor learning: Time to generalize a longstanding principle

The ideomotor principle holds that anticipating the sensory consequences of a movement triggers an associated motor response. Even though this framework dates back to the 19th century, it continues to lie at the heart of many contemporary approaches to human action control. Here we specifically focus on the ideomotor learning mechanism that has to precede action initiation via effect anticipation. Traditional approaches to this learning mechanism focused on establishing novel action-effect (or response-effect) associations. Here we apply the theoretical concept of common coding for action and perception to argue that the same learning principle should result in response-response and stimulus-stimulus associations just as well. Generalizing ideomotor learning in such a way results in a powerful and general framework of ideomotor action control, and it allows for integrating the two seemingly separate fields of ideomotor approaches and hierarchical learning.

Partial Repetition Costs are Reduced but not Eliminated with Practice

Often, we depart from an intended course of events to react to sudden situational demands (an intervening event) before resuming the originally planned action. Executing an action to an intervening event can be delayed if the features of this action plan partly overlap with an action plan retained in working memory (WM) compared to when they completely overlap or do not overlap. This delay is referred to as a partial repetition cost (PRC). PRCs are typically attributed to code confusion between action plans in WM. We tested this by training the component action plans extensively to reduce their reliance on WM. If PRCs are caused by code confusion within WM, then PRCs should be reduced and possibly eliminated with extensive practice. To test this, participants performed a partial repetition (PR) task after 0, 4 and 8.5 sessions of stimulus-response (S-R) training. In the PR task, participants saw two visual events. They retained an action to the first event while executing a speeded action to a second (intervening) event; afterwards, they executed the retained action. The two action plans either partly overlapped or did not overlap. Results showed that extensive (S-R and PR task) practice reduced but did not eliminate PRCs. A reduction in PRCs (code confusion) with practice is compatible with memory models that assume action events become more specific and less reliant on WM with practice. These findings merit expansions of PR tasks to other domains and broader conceptions of action plans that incorporate the formal structure of memory models.

Multiple-object tracking and visually guided touch

Multiple-object tracking (MOT) involves keeping track of the positions of multiple independent target items as they move among distractors. According to Pylyshyn (Cognition, 80, 127-158, 2001), the item individuation mechanism used in MOT is also used in visually guided touch. To test this, we compared single-task MOT (MOT alone) with dual-task MOT (MOT while touching items that changed colour), looking for interference: cases where single-task performance was worse than dual-task. Touching items that changed colour interfered with MOT, but effects varied depending on whether the item touched was a target or distractor in MOT. Touching distractors always reduced MOT performance more than touching targets. Touching targets during MOT did not interfere when there was only a single target to track but interfered more once there were two or more targets. We also measured interference based on latencies to touch items that changed colour, comparing single and dual-task conditions (touch alone, touch + track). MOT interfered with touch, increasing RT to touch items that changed colour, with latencies significantly higher when those items were distractors rather than targets. Overall, there was general interference (differences between single and dual-task performance), as might be expected if coordinating the two tasks required a common limited resource such as general attention or working memory. However, there was also differential interference that varied based on whether the touched item was a target or distractor in MOT. This differential interference suggests the specific mechanisms used in MOT may also play a role in visually guided touch.

Partial repetition between action plans delays responses to ideomotor compatible stimuli

Often one must depart from an intended course of events to react to sudden situational demands before resuming his or her original action retained in working memory. Retaining an action plan in working memory (WM) can delay or facilitate the execution of an intervening action when the action features of the two action plans partly overlap (partial repetition) compared to when they do not overlap. We investigated whether partial repetition costs (PRCs) or benefits (PRBs) occur when the intervening event is an ideomotor-compatible stimulus that is a biological representation of the response required by the participant. Participants viewed two visual events and retained an action plan to the first event (A) while executing a speeded response to the second, intervening event (B). In Experiment 1A, the two visual events were ideomotor compatible, non-ideomotor compatible (abstract), or one was ideomotor compatible, and the other abstract. Results showed PRCs for all event A-B stimulus combinations with reduced PRCs for intervening, ideomotor compatible events. In contrast to previous research, there was no evidence that ideomotor-compatible actions were automatic and bypassed the selection bottleneck. Experiment 1B confirmed PRCs for ideomotor compatible stimuli that more accurately mimicked the required response. Findings suggest that mechanisms for activating, selecting, and retaining action plans are similar between ideomotor compatible and abstract visual events. We conclude that PRCs occur in response to intervening events when action plans are generated offline and rely on WM, including those for ideomotor-compatible stimuli; but PRBs may be restricted to actions generated online. This conclusion is consistent with the perceptual-motor framework by Goodale and Milner (Trends in Neuroscience 15:22-25, 1992).

Task relevance determines binding of effect features in action planning

Action planning can be construed as the temporary binding of features of perceptual action effects. While previous research demonstrated binding for task-relevant, body-related effect features, the role of task-irrelevant or environment-related effect features in action planning is less clear. Here, we studied whether task-relevance or body-relatedness determines feature binding in action planning. Participants planned an action A, but before executing it initiated an intermediate action B. Each action relied on a body-related effect feature (index vs. middle finger movement) and an environment-related effect feature (cursor movement towards vs. away from a reference object). In Experiments 1 and 2, both effects were task-relevant. Performance in action B suffered from partial feature overlap with action A compared to full feature repetition or alternation, which is in line with binding of both features while planning action A. Importantly, this cost disappeared when all features were available but only body-related features were task-relevant (Experiment 3). When only the environment-related effect of action A was known in advance, action B benefitted when it aimed at the same (vs. a different) environment-related effect (Experiment 4). Consequently, the present results support the idea that task relevance determines whether binding of body-related and environment-related effect features takes place while the pre-activation of environment-related features without binding them primes feature-overlapping actions.

Free-choice and forced-choice actions: Shared representations and conservation of cognitive effort

We examined two questions regarding the interplay of planned and ongoing actions. First: Do endogenous (free-choice) and exogenous (forced-choice) triggers of action plans activate similar cognitive representations? And, second: Are free-choice decisions biased by future action goals retained in working memory? Participants planned and retained a forced-choice action to one visual event (A) while executing an immediate forced-choice or free-choice action (action B) to a second visual event (B); then the retained action (A) was executed. We found performance costs for action B if the two action plans partly overlapped versus did not overlap (partial repetition costs). This held true even when action B required a free-choice response indicating that forced-choice and free-choice actions are represented similarly. Partial repetition costs for free-choice actions were evident regardless of whether participants did or did not show free-choice response biases. Also, a subset of participants showed a bias to freely choose actions that did not overlap (vs. did overlap) with the action plan retained in memory, which led to improved performance in executing action B and recalling action A. Because cognitive effort is likely required to resolve feature code competition and confusion assumed to underlie partial repetition costs, this free-choice decision bias may serve to conserve cognitive effort and preserve the future action goal retained in working memory.

Forward and backward recall of serial actions: Exploring the temporal dynamics of working memory for instruction

The ability to flexibly retrieve and implement sequences of actions is essential to motor learning and planning. Recent research has indicated that serial memory for instructions is influenced by presentation modality (spoken vs. visual demonstration) and recall modality (verbal vs. enacted recall). The present study extended this work by investigating the impact of recall direction (forward vs. backward), in addition to that of presentation and recall modality, on working memory for instruction sequences in healthy young adults. Experiment 1 (N = 24) showed that adults were more accurate in backward than forward verbal recall following spoken instructions. In contrast, enacted recall was not influenced by recall direction. Experiment 2 (N = 24) used visual demonstration of instruction sequences and found similar performance levels in forward and backward recall. Experiment 3 (N = 24) replicated the findings from Experiment 1 and 2, along with the previous observation of an advantage for demonstrated over spoken presentation. In addition, the beneficial effects of enacted recall and visual demonstration also emerged in an analysis of response times, specifically in reduced preparation and recall duration. Demonstrated instructions improved maintenance of all items while backward recall enhanced memory of later items in the sequence. These findings provide new insights into the cognitive processes and temporal dynamics of working memory for serial actions and instructions.

Action plan interrupted: resolution of proactive interference while coordinating execution of multiple action plans during sleep deprivation

The ability to retain an action plan to execute another is necessary for most complex, goal-directed behavior. Research shows that executing an action plan to an interrupting event can be delayed when it partly overlaps (vs. does not overlap) with the retained action plan. This phenomenon is known as partial repetition costs (PRCs). PRCs reflect proactive interference, which may be resolved by inhibitory, executive control processes. We investigated whether these inhibitory processes are compromised due to one night of sleep deprivation. Participants were randomized to a sleep-deprived group or a well-rested control group. All participants performed an action planning task at baseline after a full night of sleep, and again either after a night of sleep deprivation (sleep-deprived group) or a full night of sleep (control group). In this task, two visual events occurred in a sequence. Participants retained an action plan to the first event in working memory while executing a speeded action to the second (interrupting) event; afterwards, they executed the action to the first event. The two action plans either partly overlapped (required the same hand) or did not (required different hands). Results showed slower responses to the interrupting event during sleep deprivation compared to baseline and the control group. However, the magnitude of the PRCs was no different during sleep deprivation compared to baseline and the control group. Thus, one night of sleep deprivation slowed global responses to the interruption, but inhibitory processes involved in reducing proactive interference while responding to an interrupting event were not compromised. These findings are consistent with other studies that show sleep deprivation degrades global task performance, but does not necessarily degrade performance on isolated, executive control components of cognition. The possibility that our findings involve local as opposed to central inhibition is also discussed.

Dissociating action-effect activation and effect-based response selection

Anticipated action effects have been shown to govern action selection and initiation, as described in ideomotor theory, and they have also been demonstrated to determine crosstalk between different tasks in multitasking studies. Such effect-based crosstalk was observed not only in a forward manner (with a first task influencing performance in a following second task) but also in a backward manner (the second task influencing the preceding first task), suggesting that action effect codes can become activated prior to a capacity-limited processing stage often denoted as response selection. The process of effect-based response production, by contrast, has been proposed to be capacity-limited. These observations jointly suggest that effect code activation can occur independently of effect-based response production, though this theoretical implication has not been tested directly at present. We tested this hypothesis by employing a dual-task set-up in which we manipulated the ease of effect-based response production (via response-effect compatibility) in an experimental design that allows for observing forward and backward crosstalk. We observed robust crosstalk effects and response-effect compatibility effects alike, but no interaction between both effects. These results indicate that effect activation can occur in parallel for several tasks, independently of effect-based response production, which is confined to one task at a time.

Following Instructions in Patients With Schizophrenia: The Benefits of Actions at Encoding and Recall

The ability to follow spoken instructions is important to everyday functioning but has seldom been studied in patients with schizophrenia (SZ). Recent evidence suggests that action-based processing may facilitate the ability to follow instructions, which relies largely on working memory. In this study, we tested the hypothesis that SZ patients may also benefit from action-based advantages in following instructions. Forty-eight clinically stable SZ patients and 48 demographic- and IQ-matched controls completed a following spoken instruction span task involving varied encoding and recall conditions. While SZ patients were impaired in the overall performance of following spoken instructions, this deficit could be attributed to working memory impairment. More importantly, SZ patients showed action-based advantages both at the encoding and retrieval stage to the same extent as healthy controls. Specifically, both healthy controls and SZ patients showed improved memory performance when they additionally performed the actions, or watched the experimenter carrying out the actions compared with simply listening to spoken instructions during the encoding stage. During the retrieval stage, memory was improved when they recalled the instructions by physical enactment compared with oral repetition. The present study provides the first empirical evidence for the impairment in the ability to follow instructions in SZ. We have shown that involving action-based processing in the encoding and retrieval stage facilitated memory of instructions, indicating that the enactment advantage in working memory also applies to SZ patients. These findings provide useful insights for clinical interventions and cognitive remediation for SZ patients.

Components of action representations evoked when identifying manipulable objects

We examined the influence of holding planned hand actions in working memory on the time taken to visually identify objects with handles. Features of the hand actions and position of the object's handle were congruent or incongruent on two dimensions: alignment (left vs. right) and orientation (horizontal vs. vertical). When an object was depicted in an upright view, subjects were slower to name it when its handle was congruent with the planned hand actions on one dimension but incongruent on the other, relative to when the object handle and actions were congruent on both or neither dimension. This pattern is consistent with many other experiments demonstrating that a cost occurs when there is partial feature overlap between a planned action and a perceived target. An opposite pattern of results was obtained when the depicted object appeared in a 90° rotated view (e.g., a beer mug on its side), suggesting that the functional goal associated with the object (e.g., drinking from an upright beer mug) was taken into account during object perception and that this knowledge superseded the influence of the action afforded by the depicted view of the object. These results have implications for the relationship between object perception and action representations, and for the mechanisms that support the identification of rotated objects.

The exception does not rule: attention constrains form preparation in word production

Form preparation in word production, the benefit of exploiting a useful common sound (such as the first phoneme) of iteratively spoken small groups of words, is notoriously fastidious, exhibiting a seemingly categorical, all-or-none character and a corresponding susceptibility to "killers" of preparation. In particular, the presence of a single exception item in a group of otherwise phonologically consistent words has been found to eliminate the benefit of knowing a majority characteristic. This has been interpreted to mean that form preparation amounts to partial production and thus provides a window on fundamental processes of phonological word encoding (e.g., Levelt, Roelofs, & Meyer, 1999). However, preparation of only fully distributed properties appears to be nonoptimal and is difficult to reconcile with the sensitivity of cognitive responses to probabilities in other domains. We show here that the all-or-none characteristic of form preparation is specific to task format. Preparation for sets that included an exception item occurred in ecologically valid production tasks, picture naming (Experiment 1), and word naming (Experiment 2). Preparation failed only in the commonly used, but indirect and resource-intensive, associative cuing task (Experiment 3). We outline an account of form preparation in which anticipation of word-initial phonological fragments uses a limited-capacity, sustained attentional capability that points to rather than enacts possibilities for imminent speech.