Stroking me softly: Body-related effects in effect-based action control

Action control costs in task selection: Agents avoid actions with incompatible movement and effect features

When a movement triggers effects with incompatible features, conflict between action and effect features creates costs for action planning and initiation. We investigated whether such action control costs also factor into action choices in terms of the principle of least effort. Participants completed a reaction-time experiment, where they were instructed to perform left and right mouse swipes in response to directional cues presented on the screen. Participants could select between two action options on each trial: Depending on which part of the screen (upper or lower) the action was performed in, the swipe resulted in a visual stimulus moving in the same (compatible) or in the opposite (incompatible) direction as the mouse. Incompatible action-effect mappings did indeed incur action control costs. In accordance with effort avoidance, the proportion of compatible choices was significantly above chance level, suggesting that action selection and initiation costs factor into participants preferences. Interestingly, however, participants' choice tendencies were not predicted by the actual increase in action-initiation costs in the incompatible condition. This indicates that effort-related decisions are not simply based on monitoring performance in the actual task, but they are also influenced by preestablished notions of action-planning costs.

Task-relevance and change detection in action-effect binding

Features of actions are bound to coincidentally occurring stimuli so that re-encountering a stimulus retrieves a previous action episode. One hallmark of the purported mechanism in binding/retrieval tasks is a reliable reaction time advantage for repeating a previous response if tone stimuli repeat rather than alternate across trials. Other measures than reaction times yielded surprisingly mixed results, however. This is particularly true for continuous response features like force or response duration. We therefore conducted two experiments to resolve this disconnect between different measures. Experiment 1 tested for a potentially inflated effect in reaction time data, whereas Experiment 2 took the converse approach of studying conditions that would elicit similarly strong effects on alternative measures. Our results show that confounds in terms of auditory change detection do not inflate reaction time differences, reinforcing an interpretation of these effects as reflecting binding and retrieval. Moreover, strong effects on alternative measures appeared if these features were rendered task-relevant and came with sufficient variability. These observations provide critical evidence for binding and retrieval accounts, especially by showing that these accounts extend from binary decisions to continuous features of an actual motor response.

Perception and action as viewed from the Theory of Event Coding: a multi-lab replication and effect size estimation of common experimental designs

The Theory of Event Coding (TEC) has influenced research on action and perception across the past two decades. It integrates several seminal empirical phenomena and it has continued to stimulate novel experimental approaches on the representational foundations of action control and perceptual experience. Yet, many of the most notable results surrounding TEC originate from an era of psychological research that relied on rather small sample sizes as judged by today's standards. This state hampers future research aiming to build on previous phenomena. We, therefore, provide a multi-lab re-assessment of the following six classical observations: response-effect compatibility, action-induced blindness, response-effect learning, stimulus-response binding, code occupation, and short-term response-effect binding. Our major goal is to provide precise estimates of corresponding effect sizes to facilitate future scientific endeavors. These effect sizes turned out to be considerably smaller than in the original reports, thus allowing for informed decisions on how to address each phenomenon in future work. Of note, the most relevant results of the original observations were consistently obtained in the present experiments as well.

Action representations in prevention behavior: Evidence from motor execution

Human actions sometimes aim at preventing an event from occurring. How these to-be-prevented events are represented, however, is poorly understood. Recent proposals in the literature point to a possible divide between effect-producing, operant actions, and effect-precluding, prevention actions, suggesting that the control of operant actions relies on codes of environment-related effects whereas prevention actions do not. Here we report two experiments on this issue, showing that spatial features (Experiment 1) as well as temporal features (Experiment 2) of to-be-prevented events influence actions in the same way as corresponding features of to-be-produced effects. This implies that selecting and executing prevention actions relies on anticipated environmental changes, comparable to operant actions.

Compatibility effects with touchless gestures

Human actions are suspect to various compatibility phenomena. For example, responding is faster to the side where a stimulus appears than to the opposite side, referred to as stimulus-response (S-R) compatibility. This is even true, if the response is given to a different stimulus feature, while location itself is irrelevant (Simon compatibility). In addition, responses typically produce perceivable effects on the environment. If they do so in a predictable way, responses are faster if they produce a (e.g., spatially) compatible effect on the same side than on the other side. That it, a left response is produced faster if it results predictably in a left effect than in a right effect. This effect is called response-effect (R-E) compatibility. Finally, compatibility could also exist between stimuli and the effects, which is accordingly called stimulus-effect (S-E) compatibility. Such compatibility phenomena are also relevant for applied purposes, be it in laparoscopic surgery or aviation. The present study investigates Simon and R-E compatibility for touchless gesture interactions. In line with a recent study, no effect of R-E compatibility was observed, yet irrelevant stimulus location yielded a large Simon effect. Touchless gestures thus seem to behave differently with regard to compatibility phenomena than interactions via (other) tools such as levers.

GOALIATH: a theory of goal-directed behavior

Commonsense and theorizing about action control agree in assuming that human behavior is (mainly) driven by goals, but no mechanistic theory of what goals are, where they come from, and how they impact action selection is available. Here I develop such a theory that is based on the assumption that GOALs guide Intentional Actions THrough criteria (GOALIATH). The theory is intended to be minimalist and parsimonious with respect to its assumptions, as transparent and mechanistic as possible, and it is based on representational assumptions provided by the Theory of Event Coding (TEC). It holds that goal-directed behavior is guided by selection criteria that activate and create competition between event files that contain action-effect codes matching one or more of the criteria-a competition that eventually settles into a solution favoring the best-matching event file. The criteria are associated with various sources, including biological drives, acquired needs (e.g., of achievement, power, or affiliation), and short-term, sometimes arbitrary, instructed aims. Action selection is, thus, a compromise that tries to satisfy various criteria related to different driving forces, which are also likely to vary in strength over time. Hence, what looks like goal-directed action emerges from, and represents an attempt to satisfy multiple constraints with different origins, purposes, operational characteristics, and timescales-which among other things does not guarantee a high degree of coherence or rationality of the eventual outcome. GOALIATH calls for a radical break with conventional theorizing about the control of goal-directed behavior, as it among other things questions existing cognitive-control theories and dual-route models of action control.

Spatial action-effect binding depends on type of action-effect transformation

Spatial action-effect binding denotes the mutual attraction between the perceived position of an effector (e.g., one's own hand) and a distal object that is controlled by this effector. Such spatial binding can be construed as an implicit measure of object ownership, thus the belonging of a controlled object to the own body. The current study investigated how different transformations of hand movements (body-internal action component) into movements of a visual object (body-external action component) affect spatial action-effect binding, and thus implicit object ownership. In brief, participants had to bring a cursor on the computer screen into a predefined target position by moving their occluded hand on a tablet and had to estimate their final hand position. In Experiment 1, we found a significantly lower drift of the proprioceptive position of the hand towards the visual object when hand movements were transformed into laterally inverted cursor movements, rather than cursor movements in the same direction. Experiment 2 showed that this reduction reflected an elimination of spatial action-effect binding in the inverted condition. The results are discussed with respect to the prerequisites for an experience of ownership over artificial, noncorporeal objects. Our results show that predictability of an object movement alone is not a sufficient condition for ownership because, depending on the type of transformation, integration of the effector and a distal object can be fully abolished even under conditions of full controllability.

Social learning of action-effect associations: Modulation of action control following observation of virtual action's effects

A core assumption of ideomotor theory is that learned bidirectional associations between actions and their effects enable agents to select and initiate actions by anticipating their sensory consequences. Although the acquisition of bidirectional action-effect (A-E) associations built on the experience of one's own movements has received considerable empirical support, the available evidence for A-E learning through the observation of others' actions and their effects remains limited. In two experiments, we tested whether A-E associations could be acquired through social learning in an experimental setup involving observation of virtual actions. In an acquisition phase, participants repeatedly observed finger movements on a screen, and each movement was consistently followed by a specific effect tone. In the subsequent test phase, tones were presented as imperative stimuli in a reaction-time task. In both experiments, reaction times were shorter when tones required the same response with which they had been linked in the preceding observation phase, compared with when they required a different response, revealing the impact of A-E associations acquired through observation. Similar results were obtained whether the movements observed during the acquisition phase were spatially aligned (Experiment 1) or not (Experiment 2) with participants' responses in the test phase, ruling out the possibility that the results merely reflect spatial compatibility effects. Our findings add new evidence for an acquisition of A-E associations through observation. Importantly, we generalize this acquisition process to the observation of virtual actions. These findings further confirm effect-based action control, as proposed by ideomotor theory.

Feature binding contributions to effect monitoring

Monitoring the perceptual effects of body movements is supposed to be a capacity-limited process that can interfere with processing of a concurrent task. Here we studied the contribution of feature binding to such effect monitoring interference. In three experiments, we varied the possibility of feature overlap between responses and effects in a primary task and responses in a secondary task. We show that responses in a secondary task are delayed when they partially, rather than completely, alternate or repeat features of responses/effects of a primary task. Yet, these partial feature repetition/alternation costs are small, and they occur on top of other factors that lengthen the critical effect monitoring process, such as the spatial compatibility of responses and effects in the primary task. The results thus show that feature binding contributes to, but cannot fully account for, delays in a secondary task caused by monitoring effects of a primary task.

Anticipation in sociomotor actions: Similar effects for in- and outgroup interactions

In social interactions, own actions often trigger a particular response from another person. The sociomotor framework proposes that this consistent behavior of others can become incorporated into own action control. In line with this idea, recent studies have shown that own motor actions are facilitated if they are predictably being imitated rather than counterimitated by a social interaction partner. In the present study, we investigated whether this finding is influenced by the relationship between the interacting persons. To that end, we manipulated whether a participant was being imitated and counterimitated by an ingroup or by an outgroup member. In two experiments, we found a beneficial influence of being imitated irrespective of group membership. The results suggest that, while people incorporated their partner's behavior into own action control, this was not further qualified by group membership as a higher-order social variable. This finding points to a universal account of action control for actions with social action effects and actions with inanimate action effects alike.

Flexible weighting of body-related effects in action production

A previous study on ideomotor action control showed that predictable action effects in the agent's environment influenced how an action is carried out. If participants were required to perform a forceful keypress, they exerted more force when these actions would produce a quiet compared to a loud tone, and this observation suggests that anticipated proprioceptive and auditory action effects are integrated with each other during action planning and control. In light of the typically weak influence of body-related effect found in recent work, we aimed to extend this pattern of results to the intra-modal case of integrating proprioceptive/tactile feedback of a movement and following vibro-tactile effects. Our results suggest that the same weighted integration process as for the cross-modal case applies to the intra-modal case. These observations support the idea of a common mechanism which binds all action-related features in an integrated action representation, irrespective of whether these features relate to exafferent or reafferent signals.

Dissociating action-effect activation and effect-based response selection

Anticipated action effects have been shown to govern action selection and initiation, as described in ideomotor theory, and they have also been demonstrated to determine crosstalk between different tasks in multitasking studies. Such effect-based crosstalk was observed not only in a forward manner (with a first task influencing performance in a following second task) but also in a backward manner (the second task influencing the preceding first task), suggesting that action effect codes can become activated prior to a capacity-limited processing stage often denoted as response selection. The process of effect-based response production, by contrast, has been proposed to be capacity-limited. These observations jointly suggest that effect code activation can occur independently of effect-based response production, though this theoretical implication has not been tested directly at present. We tested this hypothesis by employing a dual-task set-up in which we manipulated the ease of effect-based response production (via response-effect compatibility) in an experimental design that allows for observing forward and backward crosstalk. We observed robust crosstalk effects and response-effect compatibility effects alike, but no interaction between both effects. These results indicate that effect activation can occur in parallel for several tasks, independently of effect-based response production, which is confined to one task at a time.

Exploring the representational basis of response-effect compatibility: Evidence from bilingual verbal response-effect mappings

The ideomotor principle states that actions are represented by their anticipated sensory effects. This notion is often tested using the response-effect compatibility (REC) paradigm, where participants' responses are followed either by a compatible or incompatible response effect (e.g., an effect on the right side after a right-hand response is considered R-E compatible due to the spatial overlap, whereas an effect on the left side after the right-hand response is considered incompatible). Shorter reaction times are typically observed in the compatible condition compared to the incompatible condition (i.e., REC effect), suggesting that effect anticipation plays a role in action control. Previous evidence from verbal REC suggested that effect anticipation can be due to conceptual R-E overlap, but there was also phonological overlap (i.e., anticipated reading of a word preceded by the vocal response of saying that very word). To examine the representational basis of REC, in three experiments, we introduced a bilingual R-E mapping to exclude phonological R-E overlap (i.e., in the R-E compatible condition, the translation equivalent of the response word is presented as an effect word in a different language). Our findings show that the REC effect is obtained when presenting the effect word in the same language as the response (i.e., monolingual condition), but the compatibility effect was not found when the semantically same word is presented in a different language, suggesting no conceptually generalized REC in a bilingual setting. (232 words).

When the vibrations allow for anticipating the force to be produced: an extend to Pfister et al. (2014)

According to the ideomotor theory, action selection is done by the mental anticipation of its perceptual consequences. If the distal information processed mainly by vision and hearing are considered essential for the representation of the action, the proximal information processed by the sense of touch and proprioception is of less importance. Recent works seem to show the opposite. Nevertheless, it is necessary to complete these results by offering a situation, more ecological, where response and effect can occur on the same effector. So, the goal of our work was to implement a more relevant spatial correspondence because to touch is not the same action that to hear or to see. To do so, participants pressed a specific key after the presentation of a stimulus. The key vibrated depending on the pressure exerted on it. In a compatible condition, high pressure on a key triggered a high vibration, while in an incompatible condition high pressure triggered a low vibration on the same effectors. As expected, the response times were faster in the compatible condition than the incompatible condition. This means that proximal information participates actively in the selection of action.

Non-action effect binding: A critical re-assessment

Humans typically act to cause effects in their environment, but at times they also voluntarily omit an action to cause a predictable effect. These effects may become bound to the causing non-actions, just as actions and their effects can become associated. In three experiments, we provide a critical re-assessment of previous reports of non-action effect binding. Following this work, participants completed an acquisition phase to associate actions and non-actions with particular effects. In a subsequent test phase, the former effects were presented as stimuli and participants were allowed to choose an action or non-action freely as a response. Binding should lead to more effect-consistent choices than predicted by chance. Previous studies, however, did not control for deliberate strategies of participants that might inflate the consistency bias and, also, did not address overall preferences for either acting or non-acting, which might introduce additional artifacts. We show that these confounds have a strong impact in common experimental designs and introduce ways to mitigate these effects. This improved assessment still corroborated evidence of binding between non-actions and their effects.