Computational perspectives on forebrain microcircuits implicated in reinforcement learning, action selection, and cognitive control

Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems

When a measure becomes a target, it ceases to be a good measure. For example, when standardized test scores in education become targets, teachers may start "teaching to the test," leading to breakdown of the relationship between the measure - test performance - and the underlying goal - quality education. Similar phenomena have been named and described across a broad range of contexts, such as economics, academia, machine learning, and ecology. Yet it remains unclear whether these phenomena bear only superficial similarities, or if they derive from some fundamental unifying mechanism. Here, we propose such a unifying mechanism, which we label proxy failure. We first review illustrative examples and their labels, such as the "cobra effect," "Goodhart's law," and "Campbell's law." Second, we identify central prerequisites and constraints of proxy failure, noting that it is often only a partial failure or divergence. We argue that whenever incentivization or selection is based on an imperfect proxy measure of the underlying goal, a pressure arises that tends to make the proxy a worse approximation of the goal. Third, we develop this perspective for three concrete contexts, namely neuroscience, economics, and ecology, highlighting similarities and differences. Fourth, we outline consequences of proxy failure, suggesting it is key to understanding the structure and evolution of goal-oriented systems. Our account draws on a broad range of disciplines, but we can only scratch the surface within each. We thus hope the present account elicits a collaborative enterprise, entailing both critical discussion as well as extensions in contexts we have missed.

The Regulation of Task Performance: A Trans-Disciplinary Review

Definitions of meta-cognition typically have two components: (1) knowledge about one's own cognitive functioning; and, (2) control over one's own cognitive activities. Since Flavell and his colleagues provided the empirical foundation on which to build studies of meta-cognition and the autonoetic (self) knowledge required for effective learning, the intervening years have seen the extensive dissemination of theoretical and empirical research on meta-cognition, which now encompasses a variety of issues and domains including educational psychology and neuroscience. Nevertheless, the psychological and neural underpinnings of meta-cognitive predictions and reflections that determine subsequent regulation of task performance remain ill understood. This article provides an outline of meta-cognition in the science of education with evidence drawn from neuroimaging, psycho-physiological, and psychological literature. We will rigorously explore research that addresses the pivotal role of the prefrontal cortex (PFC) in controlling the meta-cognitive processes that underpin the self-regulated learning (SRL) strategies learners employ to regulate task performance. The article delineates what those strategies are, and how the learning environment can facilitate or frustrate strategy use by influencing learners' self-efficacy.

Neurocomputational model of moral behaviour

Our understanding of human morality has dramatically improved in the last decades, thanks to efforts carried out with scientific methods, in addition to the traditional speculative approach. Substantial contributions and relevant empirical data have come from neuroscience, psychology, genetics, comparative ethology, anthropology, and the social sciences. In this fruitful synergy, one useful approach is still missing: computational modeling. More precisely, a neurocomputational model aimed at simulating forms of moral behavior, to our knowledge, has not yet been designed. The purpose of this work is to start filling this gap, proposing MOral Neural Engine (MONE), a model that simulates the emergence of moral cognition. The neural engine in this model is assumed to be based in frontal areas, specifically the orbitofrontal and the ventromedial prefrontal cortex, and in connections to limbic areas involved in emotions and reward, such as the ventral striatum and the amygdala. Moral cognition is probably the result of a collection of several different neural processes, activated depending on the type of moral problem, each associated with a variety of emotions. This model, in its first implementation, deals with only a single moral situation: stealing someone's food, a transgression that typically elicits guilt, learned in the model from the angry facial expressions of the victim.

Computational modeling of stuttering caused by impairments in a basal ganglia thalamo-cortical circuit involved in syllable selection and initiation

Atypical white-matter integrity and elevated dopamine levels have been reported for individuals who stutter. We investigated how such abnormalities may lead to speech dysfluencies due to their effects on a syllable-sequencing circuit that consists of basal ganglia (BG), thalamus, and left ventral premotor cortex (vPMC). "Neurally impaired" versions of the neurocomputational speech production model GODIVA were utilized to test two hypotheses: (1) that white-matter abnormalities disturb the circuit via corticostriatal projections carrying copies of executed motor commands and (2) that dopaminergic abnormalities disturb the circuit via the striatum. Simulation results support both hypotheses: in both scenarios, the neural abnormalities delay readout of the next syllable's motor program, leading to dysfluency. The results also account for brain imaging findings during dysfluent speech. It is concluded that each of the two abnormality types can cause stuttering moments, probably by affecting the same BG-thalamus-vPMC circuit.

The centre of the brain: topographical model of motor, cognitive, affective, and somatosensory functions of the basal ganglia

The basal ganglia have traditionally been viewed as motor processing nuclei; however, functional neuroimaging evidence has implicated these structures in more complex cognitive and affective processes that are fundamental for a range of human activities. Using quantitative meta-analysis methods we assessed the functional subdivisions of basal ganglia nuclei in relation to motor (body and eye movements), cognitive (working-memory and executive), affective (emotion and reward) and somatosensory functions in healthy participants. We document affective processes in the anterior parts of the caudate head with the most overlap within the left hemisphere. Cognitive processes showed the most widespread response, whereas motor processes occupied more central structures. On the basis of these demonstrated functional roles of the basal ganglia, we provide a new comprehensive topographical model of these nuclei and insight into how they are linked to a wide range of behaviors.

Computational models of motivated action selection in corticostriatal circuits

Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.