Variability in competitive decision-making speed and quality against exploiting and exploitative opponents

Post-loss speeding or post-win slowing? An empirical note on the interpretation of decision-making time as a function of previous outcome

Differences in response time following previous losses relative to previous wins are robust observations in behavioural science, often attributed to an increased (or decreased) degree of cognitive control exerted after negative feedback, hence, post-loss slowing (or post-loss speeding). This presumes that the locus of this effect resides in the specific modulation of decision time following negative outcomes. Across two experiments, I demonstrate how the use of absolute rather than relative processing speeds, and the sensitivity of processing speeds in response to specific experimental manipulations (Experiment 1: win rate, Experiment 2: feedback), provide clarity as to the relative weighting of post-win and post-loss states in determining these behavioural effects. Both experiments show that the speeding or slowing of decision-time is largely due to the flexibility generated by post-win cognitive states. Given that post-loss speeding may actually represent post-win slowing, conclusions regarding the modulation of decision-making time as a function of previous outcomes need to be more carefully considered.

Autonomous behaviour and the limits of human volition

Humans and some other animals can autonomously generate action choices that contribute to solving complex problems. However, experimental investigations of the cognitive bases of human autonomy are challenging, because experimental paradigms typically constrain behaviour using controlled contexts, and elicit behaviour by external triggers. In contrast, autonomy and freedom imply unconstrained behaviour initiated by endogenous triggers. Here we propose a new theoretical construct of adaptive autonomy, meaning the capacity to make behavioural choices that are free from constraints of both immediate external triggers and of routine response patterns, but nevertheless show appropriate coordination with the environment. Participants (N = 152) played a competitive game in which they had to choose the right time to act, in the face of an opponent who punished (in separate blocks) either choice biases (such as always responding early), sequential patterns of action timing across trials (such as early, late, early, late…), or predictable action-outcome dependence (such as win-stay, lose-shift). Adaptive autonomy was quantified as the ability to maintain performance when each of these influences on action selection was punished. We found that participants could become free from habitual choices regarding when to act and could also become free from sequential action patterns. However, they were not able to free themselves from influences of action-outcome dependence, even when these resulted in poor performance. These results point to a new concept of autonomous behaviour as flexible adaptation of voluntary action choices in a way that avoids stereotypy. In a sequential analysis, we also demonstrated that participants increased their reliance on belief learning in which they attempt to understand the competitor's beliefs and intentions, when transition bias and reinforcement bias were punished. Taken together, our study points to a cognitive mechanism of adaptive autonomy in which competitive interactions with other agents could promote both social cognition and volition in the form of non-stereotyped action choices.

Assessing behavioural profiles following neutral, positive and negative feedback

Previous data suggest zero-value, neutral outcomes (draw) are subjectively assigned negative rather than positive valence. The combined observations of faster rather than slower reaction times, subsequent actions defined by shift rather than stay behaviour, reduced flexibility, and, larger rather than smaller deviations from optimal performance following draws all align with the consequences of explicitly negative outcomes such as losses. We further tested the relationships between neutral, positive and negative outcomes by manipulating value salience and observing their behavioural profiles. Despite speeded reaction times and a non-significant bias towards shift behaviour similar to losses when draws were assigned the value of 0 (Experiment 1), the degree of shift behaviour approached an approximation of optimal performance when the draw value was explicitly positive (+1). This was in contrast to when the draw value was explicitly negative (-1), which led to a significant increase in the degree of shift behaviour (Experiment 2). Similar modifications were absent when the same value manipulations were applied to win or lose trials (Experiment 3). Rather than viewing draws as neutral and valence-free outcomes, the processing cascade generated by draws produces a complex behavioural profile containing elements found in response to both explicitly positive and explicitly negative results.

Post-error Slowing Reflects the Joint Impact of Adaptive and Maladaptive Processes During Decision Making

Errors and their consequences are typically studied by investigating changes in decision speed and accuracy in trials that follow an error, commonly referred to as “post-error adjustments”. Many studies have reported that subjects slow down following an error, a phenomenon called “post-error slowing” (PES). However, the functional significance of PES is still a matter of debate as it is not always adaptive. That is, it is not always associated with a gain in performance and can even occur with a decline in accuracy. Here, we hypothesized that the nature of PES is influenced by one’s speed-accuracy tradeoff policy, which determines the overall level of choice accuracy in the task at hand. To test this hypothesis, we had subjects performing a task in two distinct contexts (separate days), which either promoted speed (hasty context) or cautiousness (cautious context), allowing us to consider post-error adjustments according to whether subjects performed choices with a low or high accuracy level, respectively. Accordingly, our data indicate that post-error adjustments varied according to the context in which subjects performed the task, with PES being solely significant in the hasty context (low accuracy). In addition, we only observed a gain in performance after errors in a specific trial type, suggesting that post-error adjustments depend on a complex combination of processes that affect the speed of ensuing actions as well as the degree to which such PES comes with a gain in performance.

A micro-genesis account of longer-form reinforcement learning in structured and unstructured environments

We explored the possibility that in order for longer-form expressions of reinforcement learning (win-calmness, loss-restlessness) to manifest across tasks, they must first develop because of micro-transactions within tasks. We found no evidence of win-calmness or loss-restlessness when wins could not be maximised (unexploitable opponents), nor when the threat of win minimisation was presented (exploiting opponents), but evidence of win-calmness (but not loss-restlessness) when wins could be maximised (exploitable opponents).