Reaching Into the Unknown: Actions, Goal Hierarchies, and Explorative Agency

Never run a changing system: Action-effect contingency shapes prospective agency

Human action control is highly sensitive to action-effect contingencies in the agent's environment. Here we show that the subjective sense of agency (SoA) contributes to this sensitivity as a subjective counterpart to instrumental action decisions. Participants (N = 556) experienced varying reward probabilities and were prompted to give summary evaluations of their SoA after a series of action-effect episodes. Results first revealed a quadratic relation of contingency and SoA, driven by a disproportionally strong impact of perfect action-effect contingencies. In addition to this strong situational determinant of SoA, we observed small but reliable interindividual differences as a function of gender, assertiveness, and neuroticism that applied especially at imperfect action-effect contingencies. Crucially, SoA not only reflected the reward structure of the environment but was also associated with the agent's future action decisions across situational and personal factors. These findings call for a paradigm shift in research on perceived agency, away from the retrospective assessment of single behavioral episodes and towards a prospective view that draws on statistical regularities of an agent's environment.

Motivation(s) from control: response-effect contingency and confirmation of sensorimotor predictions reinforce different levels of selection

Humans and other animals live in dynamic environments. To reliably manipulate the environment and attain their goals they would benefit from a constant modification of motor-responding based on responses' current effect on the current environment. It is argued that this is exactly what is achieved by a mechanism that reinforces responses which have led to accurate sensorimotor predictions. We further show that evaluations of a response's effectiveness can occur simultaneously, driven by at least two different processes, each relying on different statistical properties of the feedback and affecting a different level of responding. Specifically, we show the continuous effect of (a) a sensorimotor process sensitive only to the conditional probability of effects given that the agent acted on the environment (i.e., action-effects) and of (b) a more abstract judgement or inference that is also sensitive to the conditional probabilities of occurrence of feedback given no action by the agent (i.e., inaction-effects). The latter process seems to guide action selection (e.g., should I act?) while the former the manner of the action's execution. This study is the first to show that different evaluation processes of a response's effectiveness influence different levels of responding.

On the course of goal pursuit: The influence of goal progress on explicit judgments of self-agency

The experience of causing our own actions and resulting outcomes (i.e., self-agency) is essential for the regulation of our actions during goal pursuit. In two experiments, participants indicated experienced self-agency over presented outcomes, which varied in distance to their goal in an agency-ambiguous task. In Study 1, progress was manipulated at trial level (i.e., stimuli moved randomly or sequentially towards the goal). In Study 2, progress was constant at trial level (sequential), but varied at task level (i.e., goal discrepancy of the outcomes was random or decreased over trials). Study 1 showed that self-agency gradually increased in the progress condition as unsuccessful outcomes were objectively closer to the goal, while self-agency increased exponentially upon full goal attainment in the absence of progress. The gradual pattern for the progress condition was replicated in Study 2. These studies indicate that explicit judgments of self-agency are more flexible when there is goal progress.

Children's reasoning about the efficiency of others' actions: The development of rational action prediction

The relative efficiency of an action is a central criterion in action control and can be used to predict others' behavior. Yet, it is unclear when the ability to predict on and reason about the efficiency of others' actions develops. In three main and two follow-up studies, 3- to 6-year-old children (n = 242) were confronted with vignettes in which protagonists could take a short (efficient) path or a long path. Children predicted which path the protagonist would take and why the protagonist would take a specific path. The 3-year-olds did not take efficiency into account when making decisions even when there was an explicit goal, the task was simplified and made more salient, and children were questioned after exposure to the agent's action. Four years is a transition age for rational action prediction, and the 5-year-olds reasoned on the efficiency of actions before relying on them to predict others' behavior. Results are discussed within a representational redescription account.

To prevent means to know: Explicit but no implicit agency for prevention behavior

Human agents draw on a variety of explicit and implicit cues to construct a sense of agency for their actions and the effects of these actions on the outside world. Associative mechanisms binding actions to their immediate effects support the evolution of agency for operant actions. However, human agents often also act to prevent a certain event from occurring. Such prevention behavior poses a critical challenge for the sense of agency, as successful prevention inherently revolves around the absence of a perceivable effect. By assessing the psychological microstructure of singular operant and prevention actions we show that this comes with profound consequences: agency for prevention actions is only evident in explicit measures but not in corresponding implicit proxies. These findings attest to an altered action representation in prevention behavior and they support recent proposals to model related processes such as avoidance learning in terms of propositional rather than associative terms.

Connecting action control and agency: Does action-effect binding affect temporal binding?

The sense of agency, i.e., the notion that we, as agents, are in control of our own actions and can affect our environment by acting, is an integral part of human volition. Recent work has attempted to ground agency in basic mechanisms of human action control. Along these lines, action-effect binding has been shown to affect explicit judgments of agency. Here, we investigate if such action-effect bindings are also related to temporal binding which is often used as an implicit measure of agency. In two experiments, we found evidence for the establishment of short-term action-effect bindings as well as temporal binding effects. However, the two phenomena were not associated with each other. This finding suggests that the relation of action control and agency is not a simple one, and it adds to the evidence in favor of a dissociation between subjective agency and perceptual biases such as temporal binding.

Hope, exploration, and equilibrated action schemes

The concepts want, hope, and exploration cannot be organized in relation to a single type of motive (e.g., motive for food). They require, in addition, the motive for acquiring and maintaining a stable scheme that enables reward-directed activity. Facing unpredictability, the animal has to seek not only reward, but also a new equilibrated state within which reward seeking is possible.