Divided Attention and Processes Underlying Sense of Agency

Sense of agency at a gaze-contingent display with jittery temporal delay

Introduction

Inconsistent jittery temporal delays between action and subsequent feedback, prevalent in network-based human-computer interaction (HCI), have been insufficiently explored, particularly regarding their impact on the sense of agency (SoA). This study investigates the SoA in the context of eye-gaze HCI under jittery delay conditions.

Methods

Participants performed a visual search for Chinese characters using a biresolutional gaze-contingent display, which displayed a high-resolution image in the central vision and a low-resolution in the periphery. We manipulated the delay between eye movements and display updates using a truncated normal distribution (μ to μ + 2 σ) with μ ranging from 0 to 400 ms and σ fixed at 50 ms. Playback of recorded gaze data provided a non-controllable condition.

Results

The study revealed that both reported authorship and controllability scores, as well as the fixation count per second, decreased as μ increased, aligning with trends observed under constant delay conditions. The subjective authorship weakened significantly at a μ of 94 ms. Notably, the comparison between jittery and constant delays indicated the minimum value (μ) of the distribution as a critical parameter influencing both authorship perception and visual search time efficiency.

Discussion

This finding underscores the importance of the shortest delay in modulating SoA. Further examining the relative distribution for fixation duration and saccade amplitude suggests an adaptation in action planning and attention distribution in response to delay. By providing a systematic examination of the statistical attributes of jittery delays that most significantly affect SoA, this research offers valuable implications for the design of efficient, delay-tolerant eye-gaze HCI, expanding our understanding of SoA in technologically mediated interactions. Moreover, our findings highlight the significance of considering both constant and variable delay impacts in HCI usability design, marking a novel contribution to the field.

Attention and expectation likely underlie temporal binding measured using the Libet Clock

An interesting finding that has emerged in studies of the sense of agency is that of a perceived compression of the temporal interval between actions and the outcomes they produce. This is generally referred to as temporal binding. Although temporal binding has been studied using various paradigms, possibly the most popular of these is the Libet Clock task. The Libet task is also interesting because it suggests that temporal binding can be decomposed into two components, one purportedly relating to actions and the other relating to outcomes. These are termed action binding and outcome binding, respectively. In this article, I focus specifically on temporal binding revealed using the Libet Clock task and propose the idea that attention underpins the action binding effect, while outcome binding, on the other hand, is driven by the effects of expectation.

Enriched sensory feedback delivered during a voluntary action boosts subjective time compression

Introduction

The subjective experience of time can be influenced by various factors including voluntary actions. In our previous study, we found that the subjective time experience of an action outcome can be compressed when an individual performs a continuous action compared to a single action, suggesting that the sense of agency (SoA), the feeling of control over one's own action outcomes, contributes to the subjective time compression. We hypothesized that enhancing SoA by providing sensory feedback to participants would further compress the subjective time experience.

Methods

To test the hypothesis, we used a temporal reproduction task where participants reproduced the duration of a previously exposed auditory stimulus by performing different voluntary actions: a combination of single actions with single auditory feedback, continuous action with single auditory feedback, or continuous action with multiple auditory feedback.

Results

The results showed that the continuous action conditions, regardless of the type of auditory feedback, led to a compression of the subjective time experience of the reproduced tone, whereas the single action condition did not. Furthermore, a greater degree of subjective time compression during continuous action and a stronger SoA were revealed when enriched with multiple auditory feedback.

Discussion

These results indicate that enriching auditory feedback can increase subjective time compression during voluntary action, which in turn enhances SoA over action outcomes. This suggests the potential for developing new techniques to artificially compress the subjective time experience of daily events.

Task-irrelevant object response to action enhances the sense of agency for controlling the object in automation

The sense of agency (SoA) refers to the experience of controlling our bodies and tools. Recent automated systems require the operators to have less manual control, which decreases the SoA. This study investigated how to increase the SoA when operating automated systems, by focusing on the effect of an object's responses to operators' actions on the SoA. Participants applied brakes to a moving black circle by pressing a key, in order to stop the circle near a goal. Then, they estimated their SoA for stopping the circle. We informed them that there were automatic control trials in which the circle stopped independently of their keypress (86% of the trials). The circle's color briefly changed to white (i.e., flashed) when they pressed the key in a half of the automatic control trials. The SoA was higher with the flash than without it. However, the SoA neither increased when the circle flashed independently of a keypress nor when another object flashed due to a keypress. Furthermore, the keypress contingent object-flash did not influence the SoA when the participants controlled the circle manually. These results indicated that spatiotemporally contingent object responses to actions can enhance the SoA in automatic control situations.

Having a sense of agency can improve memory

In most situations, we are able to tell those outcomes we cause from those we do not. By now, research has provided us with a reasonably good understanding of the cognitive processes that underlie this sense of agency - it is thought to be produced by a comparison between a prediction of the outcome and the actual outcome that occurs. What is less clear is whether having a sense of agency can, itself, influence cognition. In the current study, we examined the possibility that sense of agency can affect memory, and we report evidence that stimuli that one feels a sense of agency over are, in fact, better remembered than counterparts without this. This self-agency effect can be distinguished from previously described control-related memory enhancements and adds to what we know of the cognitive consequences of having a sense of agency.

Measures of agency

The sense of agency is typically defined as the experience of controlling one’s own actions, and through them, changes in the external environment. It is often assumed that this experience is a single, unified construct that can be experimentally manipulated and measured in a variety of ways. In this article, we challenge this assumption. We argue that we should acknowledge four possible agency-related psychological constructs. Having a clear grasp of the possible constructs is important since experimental procedures are only able to target some but not all the possible constructs. The unacknowledged misalignment of the possible constructs of a sense of agency and the experimental procedures is a major theoretical and methodological obstacle to studying the sense of agency. Only if we recognize the nature of this obstacle will we be able to design the experimental paradigms that would enable us to study the responsible computational mechanisms.

The Prospective Sense of Agency is Rooted in Local and Global Properties of Intrinsic Functional Brain Networks

The sense of agency (SoA) refers to a constitutional aspect of the self describing the extent to which individuals feel in control over their actions and consequences thereof. Although the SoA has been associated with mental health and well-being, it is still unknown how interindividual variability in the SoA is embedded in the intrinsic brain organization. We hypothesized that the prospective component of an implicit SoA is associated with brain networks related to SoA and sensorimotor predictions on multiple spatial scales. We replicated previous findings by showing a significant prospective SoA as indicated by intentional binding effects. Then, using task-free fMRI and graph analysis, we analyzed associations between intentional binding effects and the intrinsic brain organization at regional, modular, and whole-brain scales. The results showed that intermodular connections of a frontoparietal module including the premotor cortex, supramarginal gyrus, and dorsal precuneus are associated with individual differences in prospective intentional binding. Notably, prospective intentional binding effects were also related to global brain modularity within a specific structural resolution range. These findings suggest that an implicit SoA generated through sensorimotor predictions relies on the intrinsic organization of the brain connectome on both local and global scales.

Control used and control felt: Two sides of the agency coin

Agency has been defined as the sense of ownership and control of our actions, and the metacognition of agency has now been examined in a number of studies. Here we examined the relations between task demands, the feeling of being in control, and the feeling of using control. As task demands increase, we might feel as if we use a lot of control while feeling little control over the task. It therefore seems possible that the amount of control one feels they have used and how much in control one feels are separable components of the metacognition of control. In two experiments, we manipulated task demands and assessed these two aspects of metacognition. The source of task demand differed for the two experiments. In Experiment 1, we manipulated task demands by varying the sizes of targets in an aiming task. As predicted, we found that reports of control used increased, while reports of control felt decreased, for more difficult aiming conditions. In Experiment 2, we found a similar relation using a different source of demand: response conflict. We connect these reports of control to previous investigations of task demand and agency, as well as prominent conceptions of cognitive control.

Reaching Into the Unknown: Actions, Goal Hierarchies, and Explorative Agency

Action is widely characterized as possessing a teleological dimension. The dominant way of describing goal-directed action and agency is in terms of exploitation, i.e., pursuing pre-specified goals using existing strategies. Recent theoretical developments emphasize the place of exploration, i.e., discovering new goals or acquiring new strategies. The exploitation-exploration distinction poses questions with regard to goals and agency: Should exploration, as some authors have suggested, be regarded as acting without a goal? We argue that recognizing the hierarchical nature of goals is crucial in distinguishing the two kinds of activity, because this recognition prevents the claim that exploration is goal-free, while allowing for a homogeneous account of both exploitative and explorative actions. An action typically causes relatively low-level/proximal (i.e., sensorimotor, immediate) and relatively high-level/distal (i.e., in the environment, at a wider timescale) outcomes. In exploitation, one relies on existing associations between low- and high-level states, whereas in exploration one does not have the ability or intention to control high-level/distal states. We argue that explorative action entails the capacity to exercise control within the low-level/proximal states, which enables the pursuit of indeterminate goals at the higher levels of a goal hierarchy, and the possibility of acquiring new goals and reorganization of goal hierarchies. We consider how the dominant models of agency might accommodate this capacity for explorative action.

Losing Control in Social Situations: How the Presence of Others Affects Neural Processes Related to Sense of Agency

Social contexts substantially influence individual behavior, but little is known about how they affect cognitive processes related to voluntary action. Previously, it has been shown that social context reduces participants' sense of agency over the outcomes of their actions and outcome monitoring. In this fMRI study on human volunteers, we investigated the neural mechanisms by which social context alters sense of agency. Participants made costly actions to stop inflating a balloon before it burst. On "social" trials, another player could act in their stead, but we analyzed only trials in which the other player remained passive. We hypothesized that mentalizing processes during social trials would affect decision-making fluency and lead to a decreased sense of agency. In line with this hypothesis, we found increased activity in the bilateral temporo-parietal junction (TPJ), precuneus, and middle frontal gyrus during social trials compared with nonsocial trials. Activity in the precuneus was, in turn, negatively related to sense of agency at a single-trial level. We further found a double dissociation between TPJ and angular gyrus (AG): activity in the left AG was not sensitive to social context but was negatively related to sense of agency. In contrast, activity in the TPJ was modulated by social context but was not sensitive to sense of agency.

Control Changes the Way We Look at the World

The feeling of control is a fundamental aspect of human experience and accompanies our voluntary actions all the time. However, how the sense of control interacts with wider perception, cognition, and behavior remains poorly understood. This study focused on how controlling an external object influences the allocation of attention. Experiment 1 examined attention to an object that is under a different level of control from the others. Participants searched for a target among multiple distractors on screen. All the distractors were partially under the participant's control (50% control level), and the search target was either under more or less control than the distractors. The results showed that, against this background of partial control, visual attention was attracted to an object only if it was more controlled than other available objects and not if it was less controlled. Experiment 2 examined attention allocation in contexts of either perfect control or no control over most of the objects. Specifically, the distractors were under either perfect (100%) control or no (0%) control, and the search target had one of six levels of control varying from 0% to 100%. When differences in control between the distractors and the target were small, visual attention was now more strongly drawn to search targets that were less controlled than distractors, rather than more controlled, suggesting attention to objects over which one might be losing control. Experiment 3 studied the events of losing or gaining control as opposed to the states of having or not having control. ERP measures showed that P300 amplitude proportionally encoded the magnitude of both increases and decreases in degree of control. However, losing control had more marked effects on P170 and P300 than gaining an equivalent degree of control, indicating high priority for efficiently detecting failures of control. Overall, our results suggest that controlled objects preferentially attract attention in uncontrolled environments. However, once control has been registered, the brain becomes highly sensitive to subsequent loss of control. Our findings point toward careful perceptual monitoring of degree of one's own agentic control over external objects. We suggest that control has intrinsic cognitive value because perceptual systems are organized to detect it and, once it has been acquired, to maintain it.

Attention and the sense of agency: A review and some thoughts on the matter

Much is now known about the sense of agency and how it is produced. What is lacking, though, is an understanding of how it relates to other cognitive domains and operations. Here, the relationship between the sense of agency and attention is explored. A review of the literature suggests that attention is involved in the sense of agency in (at least) two key ways. First, agency processing is dependent on attentional resources. Second, attentional enhancement is necessary for extraneous information to have an influence over the sense of agency. These are discussed and some suggestions for future research are offered.

Goal-Directed Movement Enhances Body Representation Updating

Body representation refers to perception, memory, and cognition related to the body and is updated continuously by sensory input. The present study examined the influence of goals on body representation updating with two experiments of the rubber hand paradigm. In the experiments, participants moved their hidden left hands forward and backward either in response to instruction to touch a virtual object or without any specific goal, while a virtual left hand was presented 250 mm above the real hand and moved in synchrony with the real hand. Participants then provided information concerning the perceived heights of their real left hands and rated their sense of agency and ownership of the virtual hand. Results of Experiment 1 showed that when participants moved their hands with the goal of touching a virtual object and received feedback indicating goal attainment, the perceived positions of their real hands shifted more toward that of the virtual hand relative to that in the condition without a goal, indicating that their body representations underwent greater modification. Furthermore, results of Experiment 2 showed that the effect of goal-directed movement occurred in the active condition, in which participants moved their own hands, but did not occur in the passive condition, in which participants’ hands were moved by the experimenter. Therefore, we concluded that the sense of agency probably contributed to the updating of body representation involving goal-directed movement.