Influences of unconscious priming on voluntary actions: Role of the rostral cingulate zone

Libet's legacy: A primer to the neuroscience of volition

The neuroscience of volition is an emerging subfield of the brain sciences, with hundreds of papers on the role of consciousness in action formation published each year. This makes the state-of-the-art in the discipline poorly accessible to newcomers and difficult to follow even for experts in the field. Here we provide a comprehensive summary of research in this field since its inception that will be useful to both groups. We also discuss important ideas that have received little coverage in the literature so far. We systematically reviewed a set of 2220 publications, with detailed consideration of almost 500 of the most relevant papers. We provide a thorough introduction to the seminal work of Benjamin Libet from the 1960s to 1980s. We also discuss common criticisms of Libet's method, including temporal introspection, the interpretation of the assumed physiological correlates of volition, and various conceptual issues. We conclude with recent advances and potential future directions in the field, highlighting modern methodological approaches to volition, as well as important recent findings.

Enhancing free choice masked priming via switch trials during repeated practice

The masked priming paradigm has been extensively used to investigate the indirect impacts of unconscious stimuli on conscious behaviors, and the congruency effect of priming on free choices has gained increasing attention. Free choices allow participants to voluntarily choose a response from multiple options during each trial. While repeated practice is known to increase priming effects in subliminal visual tasks, whether practice increases the priming effect of free choices in the masked priming paradigm is unclear. And it is also not clear how the proportions of free choice and forced choice trials in one block will affect the free choice masked priming effect. The present study applied repeated practice in the masked priming paradigm and found that after training, the participants were more likely to be influenced by masked primes during free choice, but this training process did not alter the visibility of masked stimuli. In addition, this study revealed that when the proportions of free choice and forced choice trials were equal during the training stage, this enhanced effect by practice was the strongest. These results indicated that practice could enhance masked stimulus processing in free-choice, and that the learning effect may mainly be derived from the early selection and integrated processing of masked stimuli.

Metacognitive Awareness of Difficulty in Action Selection: The Role of the Cingulo-opercular Network

The question whether and how we are able to monitor our own cognitive states (metacognition) has been a matter of debate for decades. Do we have direct access to our cognitive processes, or can we only infer them indirectly based on their consequences? In the current study, we wanted to investigate the brain circuits that underlie the metacognitive experience of fluency in action selection. To manipulate action-selection fluency, we used a subliminal response priming paradigm. On each trial, both male and female human participants additionally engaged in the metacognitive process of rating how hard they felt it was to respond to the target stimulus. Despite having no conscious awareness of the prime, results showed that participants rated incompatible trials (during which subliminal primes interfered with the required response) to be more difficult than compatible trials (where primes facilitated the required response), reflecting metacognitive awareness of difficulty. This increased sense of subjective difficulty was mirrored by increased activity in the rostral cingulate zone and the anterior insula, two regions that are functionally closely connected. Importantly, this reflected activations that were unique to subjective difficulty ratings and were not explained by RTs or prime-response compatibility. We interpret these findings in light of a possible grounding of the metacognitive judgment of fluency in action selection in interoceptive signals resulting from increased effort.

Cost-benefit trade-offs in decision-making and learning

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes.

Value-based decision-making involves trading off the cost associated with an action–such as physical or mental effort–against its expected reward. Although facing conflicts between competing action alternatives is considered aversive and effortful, it remains unclear whether conflict also constitutes a cost in value-based decisions. We tested this hypothesis by combining a classic conflict (flanker) task with a reinforcement-learning task. Results showed that participants learned to maximise their earnings, but were nevertheless biased to follow irrelevant suggestions. Computational model-based analyses showed a greater choice bias with more uncertainty about the best action to make, supporting the hypothesis that the costs linked to conflict management were traded off against expected rewards. We additionally found that learning rates were reduced when following instructions, relative to when choosing freely what to do. Learning was further reduced by conflict between instructions and subjective action values. In short, we found that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and that different types of conflict may have different effects on learning about action outcomes.

'Proactive' use of cue-context congruence for building reinforcement learning's reward function

BACKGROUND

Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model.

RESULTS

In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively.

CONCLUSIONS

Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.