Rethinking Norm Psychology

Norms permeate human life. Most of people's activities can be characterized by rules about what is appropriate, allowed, required, or forbidden-rules that are crucial in making people hyper-cooperative animals. In this article, I examine the current cognitive-evolutionary account of "norm psychology" and propose an alternative that is better supported by evidence and better placed to promote interdisciplinary dialogue. The incumbent theory focuses on rules and claims that humans genetically inherit cognitive and motivational mechanisms specialized for processing these rules. The cultural-evolutionary alternative defines normativity in relation to behavior-compliance, enforcement, and commentary-and suggests that it depends on implicit and explicit processes. The implicit processes are genetically inherited and domain-general; rather than being specialized for normativity, they do many jobs in many species. The explicit processes are culturally inherited and domain-specific; they are constructed from mentalizing and reasoning by social interaction in childhood. The cultural-evolutionary, or "cognitive gadget," perspective suggests that people alive today-parents, educators, elders, politicians, lawyers-have more responsibility for sustaining normativity than the nativist view implies. People's actions not only shape and transmit the rules, but they also create in each new generation mental processes that can grasp the rules and put them into action.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

The ontogeny of efficient second-order action planning: The developing interplay of controlled and habitual processes in goal-directed actions

The ability to act efficiently plays an important role in everyday human life. The current study investigated efficient motor planning in 2- to 14-year-old children and adults (N = 246) in two different object manipulation tasks that involved everyday objects (a cup and a spoon). Importantly, we manipulated whether or not the efficient controlled grasp was incongruent with the habitual use of the object. We assessed to what extent participants planned their grasping action in an anticipatory controlled manner or relied on the habitual use of an object. We found the ability of efficient movement planning to be correlated between the two conceptually different tasks. Furthermore, the interplay of controlled and habitual processes of action control showed different developmental patterns for the two tasks and does not indicate a simple linear developmental trend. Thus, this study expands our knowledge on the ontogeny of efficient motor planning and highlights the developmental dynamics of the interplay of controlled and habitual processes in goal-directed action control.

Adolescents exhibit reduced Pavlovian biases on instrumental learning

Multiple learning systems allow individuals to flexibly respond to opportunities and challenges present in the environment. An evolutionarily conserved "Pavlovian" learning mechanism couples valence and action, promoting a tendency to approach cues associated with reward and to inhibit action in the face of anticipated punishment. Although this default response system may be adaptive, these hard-wired reactions can hinder the ability to learn flexible "instrumental" actions in pursuit of a goal. Such constraints on behavioral flexibility have been studied extensively in adults. However, the extent to which these valence-specific response tendencies bias instrumental learning across development remains poorly characterized. Here, we show that while Pavlovian response biases constrain flexible action learning in children and adults, these biases are attenuated in adolescents. This adolescent-specific reduction in Pavlovian bias may promote unbiased exploration of approach and avoidance responses, facilitating the discovery of rewarding behavior in the many novel contexts that adolescents encounter.

Reward Devaluation in Autistic Children and Adolescents with Complex Needs: A Feasibility Study

Rewards act as a motivator for positive behavior and learning. Although compounding evidence indicates that reward processing operates differently in autistic individuals who do not have co-occurring learning disabilities, little is known about individuals who have such difficulties or other complex needs. This study aimed first to assess the feasibility of using an adapted reward devaluation paradigm to examine basic reward processes in this underrepresented population, and second to investigate whether autistic children and adolescents with complex needs would show dynamic behavioral changes in response to changes in the motivational value of a reward. Twenty-seven autistic children and adolescents with complex needs and 20 typically developing 5-year-old children took part in the study. Participants were presented with two visual cues on a touchscreen laptop, which triggered the delivery of a video, music, or physical reward. One of the rewards was then presented in abundance to decrease its motivational value. Participants showed decreased interest in the video and music rewards after devaluation. The experimental setup was found to be suitable to test individuals with complex needs, although recommendations are made for the use of physical rewards. The results suggest that autistic participants with complex needs demonstrate goal-directed behavior and that it is feasible to develop experimental paradigms that can shed important light on learning processes that are fundamental to many education and intervention strategies for this population. Autism Res 2020, 13: 1915-1928. © 2020 The Authors. Autism Research published by International Society for Autism Research and Wiley Periodicals LLC. LAY SUMMARY: We adapted an experimental task to conduct research with autistic children and adolescents with complex needs, who remain grossly underrepresented in autism research. We found that once a reward was presented in great quantity, participants were less motivated to obtain it, showing that they adapted their behavior to changes in the value of that reward. This is an important finding to help promote learning and design better interventions for this population.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots

Much current work in robotics focuses on the development of robots capable of autonomous unsupervised learning. An essential prerequisite for such learning to be possible is that the agent should be sensitive to the link between its actions and the consequences of its actions, called sensorimotor contingencies. This sensitivity, and more particularly its role as a key drive of development, has been widely studied by developmental psychologists. However, the results of these studies may not necessarily be accessible or intelligible to roboticians. In this paper, we review the main experimental data demonstrating the role of sensitivity to sensorimotor contingencies in infants' acquisition of four fundamental motor and cognitive abilities: body knowledge, memory, generalization, and goal-directedness. We relate this data from developmental psychology to work in robotics, highlighting the links between these two domains of research. In the last part of the article we present a blueprint architecture demonstrating how exploitation of sensitivity to sensorimotor contingencies, combined with the notion of "goal," allows an agent to develop new sensorimotor skills. This architecture can be used to guide the design of specific computational models, and also to possibly envisage new empirical experiments.

Pediatric OCD in the era of RDoC

The NIMH Research Domain Criteria (RDoC) initiative was established with the goal of developing an alternative research classification to further research efforts in mental health. While RDoC acknowledges that constructs should be considered within a developmental framework, developmental considerations have not yet been well integrated within the existing RDoC matrix. In this paper, we consider RDoC in relation to pediatric OCD, a paradigmatic example of a neuropsychiatric disorder that often has onset in childhood but is also present across the lifespan. We discuss three RDoC subdomains with relevance to OCD as exemplars, providing for each construct a brief review of normative developmental changes, the state of construct-relevant research in pediatric OCD, and challenges and limitations related to developmental considerations within each subdomain. Finally, we conclude with a brief discussion of how RDoC may continue to evolve with regard to developmental considerations in order to further research in pediatric OCD.

Teacher-learner interaction quantifies scaffolding behaviour in imitation learning

Teachers often believe that they take into account learners’ ongoing learning progress in their teaching. Can behavioural data support this belief? To address this question, we investigated the interactive behavioural coordination between teachers and learners during imitation learning to solve a puzzle. The teacher manually demonstrated the puzzle solution to a learner who immediately imitated and learned it. Manual movements of teachers and learners were analysed using a bivariate autoregressive model. To identify bidirectional information exchange and information shared between the two agents, we calculated causality and noise covariance from the model. Information transfer observed from teacher to learner in the lateral component of their motion indicated imitation of the spatial information of the puzzle solution. Information transfer from learner to teacher in the vertical component of their motion indicated the monitoring process through which teachers adjust their timing of demonstration to the learner’s progress. The shared information in the lateral component increased as learning progressed, indicating the knowledge was shared between the two agents. Our findings demonstrated that the teacher interactively engaged in and contingently supported (i.e. scaffolded) imitation. We thus provide a behavioural signature of the teacher’s intention to promote learning indispensable for understanding the nature of teaching.

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Cognitive components underpinning the development of model-based learning

Reinforcement learning theory distinguishes “model-free” learning, which fosters reflexive repetition of previously rewarded actions, from “model-based” learning, which recruits a mental model of the environment to flexibly select goal-directed actions. Whereas model-free learning is evident across development, recruitment of model-based learning appears to increase with age. However, the cognitive processes underlying the development of model-based learning remain poorly characterized. Here, we examined whether age-related differences in cognitive processes underlying the construction and flexible recruitment of mental models predict developmental increases in model-based choice. In a cohort of participants aged 9–25, we examined whether the abilities to infer sequential regularities in the environment (“statistical learning”), maintain information in an active state (“working memory”) and integrate distant concepts to solve problems (“fluid reasoning”) predicted age-related improvements in model-based choice. We found that age-related improvements in statistical learning performance did not mediate the relationship between age and model-based choice. Ceiling performance on our working memory assay prevented examination of its contribution to model-based learning. However, age-related improvements in fluid reasoning statistically mediated the developmental increase in the recruitment of a model-based strategy. These findings suggest that gradual development of fluid reasoning may be a critical component process underlying the emergence of model-based learning.

Control feedback as the motivational force behind habitual behavior

Motivated behavior is considered to be a product of integration of a behavior's subjective benefits and costs. As such, it is unclear what motivates "habitual behavior" which occurs, by definition, after the outcome's value has diminished. One possible answer is that habitual behavior continues to be selected due to its "intrinsic" worth. Such an explanation, however, highlights the need to specify the motivational system for which the behavior has intrinsic worth. Another key question is how does an activity attain such intrinsically rewarding properties. In an attempt to answer both questions, we suggest that habitual behavior is motivated by the influence it brings over the environment-by the control motivation system, including "control feedback." Thus, when referring to intrinsic worth, we refer to a representation of an activity that has been reinforced due to it being effective in controlling the environment, managing to make something happen. As an answer to when does an activity attain such rewarding properties, we propose that this occurs when the estimated instrumental outcome expectancy of an activity is positive, but the precision of this expectancy is low. This lack of precision overcomes the chronic dominance of outcome feedback over control feedback in determining action selection by increasing the relative weight of the control feedback. Such a state of affairs will lead to repeated selection of control relevant behavior and entails insensitivity to outcome devaluation, thereby producing a habit.

From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning

Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior.

Cue-elicited food seeking is eliminated with aversive outcomes following outcome devaluation

In outcome-selective Pavlovian-to-instrumental transfer (PIT), stimuli that are predictive of specific outcomes prime instrumental responses that are associated with these outcomes. Previous human studies yielded mixed evidence in respect to whether the PIT effect is affected by a posttraining devaluation of an outcome, with the PIT effect being preserved after a devaluation of a primary reinforcer (food, drugs) but not following the devaluation of a secondary reinforcer (money). The present research examined whether outcome-selective transfer is eliminated when the devaluation of a primary (liquid) reinforcer is strong and aversive. Experiment 1 confirmed these expectations following a devaluation with bad tasting Tween 20. However, outcome-selective transfer was still observed when the earned (devalued) outcome was not consumed immediately after each test (Experiment 2). These results suggest that the capacity of a Pavlovian cue to motivate a specific response is affected by the incentive value of the shared outcome only when the devaluation yields an aversive outcome that is consumed immediately.

Goal-directed learning and obsessive-compulsive disorder

Obsessive–compulsive disorder (OCD) has become a paradigmatic case of goal-directed dysfunction in psychiatry. In this article, we review the neurobiological evidence, historical and recent, that originally led to this supposition and continues to support a habit hypothesis of OCD. We will then discuss a number of recent studies that have directly tested this hypothesis, using behavioural experiments in patient populations. Based on this research evidence, which suggests that rather than goal-directed avoidance behaviours, compulsions in OCD may derive from manifestations of excessive habit formation, we present the details of a novel account of the functional relationship between these habits and the full symptom profile of the disorder. Borrowing from a cognitive dissonance framework, we propose that the irrational threat beliefs (obsessions) characteristic of OCD may be a consequence, rather than an instigator, of compulsive behaviour in these patients. This lays the foundation for a potential shift in both clinical and neuropsychological conceptualization of OCD and related disorders. This model may also prove relevant to other putative disorders of compulsivity, such as substance dependence, where the experience of ‘wanting’ drugs may be better understood as post hoc rationalizations of otherwise goal-insensitive, stimulus-driven behaviour.

Goal-directed, habitual and Pavlovian prosocial behavior

Although prosocial behaviors have been widely studied across disciplines, the mechanisms underlying them are not fully understood. Evidence from psychology, biology and economics suggests that prosocial behaviors can be driven by a variety of seemingly opposing factors: altruism or egoism, intuition or deliberation, inborn instincts or learned dispositions, and utility derived from actions or their outcomes. Here we propose a framework inspired by research on reinforcement learning and decision making that links these processes and explains characteristics of prosocial behaviors in different contexts. More specifically, we suggest that prosocial behaviors inherit features of up to three decision-making systems employed to choose between self- and other- regarding acts: a goal-directed system that selects actions based on their predicted consequences, a habitual system that selects actions based on their reinforcement history, and a Pavlovian system that emits reflexive responses based on evolutionarily prescribed priors. This framework, initially described in the field of cognitive neuroscience and machine learning, provides insight into the potential neural circuits and computations shaping prosocial behaviors. Furthermore, it identifies specific conditions in which each of these three systems should dominate and promote other- or self- regarding behavior.

Which way to take? Infants select an efficient path to their goal

In two experiments, we examined the development of the ability to select efficient means in order to attain a goal in 1.5- and 2-year-olds (N = 79) using a setup in which two paths led to a goal. One of the paths was shorter, and thus more efficient, than the other path. Experiment 1 showed a strong tendency in both age groups to choose the shorter path. In Experiment 2, the shorter path was initially blocked and became available only after infants repeatedly took the longer path. Children demonstrated increasing use of the more efficient path over time. The results of both experiments point to some abilities of efficient action selection in infants.

Impaired acquisition of goal-directed action in healthy aging

According to dual-system theories, instrumental learning is supported by dissociable goal-directed and habitual systems. Previous investigations of the dual-system balance in healthy aging have yielded mixed results. To further investigate this issue, we compared performance of young (17-24 years) and older (69-84 years) adults on an instrumental learning task. Following the initial learning phase, the behavioral autonomy of the motivational significance of the instrumental outcome was assessed with an outcome-devaluation test and slips-of-action test. The present study provides evidence for a disrupted dual-system balance in healthy aging, as reflected in reduced outcome-induced conflict during acquisition, as well as in impaired performance during the test stage, during which participants had to flexibly adjust their actions to changes in the current desirability of the behavioral outcome. These findings will be discussed in relation to previous aging studies into habitual and goal-directed control, as well as other cognitive impairments, challenges that older adults may face in everyday life, and to the neurobiological basis of the developmental pattern of goal-directed action across the lifespan.

Translational Assessment of Reward and Motivational Deficits in Psychiatric Disorders

Deficits in reward and motivation are common symptoms characterizing several psychiatric and neurological disorders. Such deficits may include anhedonia, defined as loss of pleasure, as well as impairments in anticipatory pleasure, reward valuation, motivation/effort, and reward learning. This chapter describes recent advances in the development of behavioral tasks used to assess different aspects of reward processing in both humans and non-human animals. While earlier tasks were generally developed independently with limited cross-species correspondence, a newer generation of translational tasks has emerged that are theoretically and procedurally analogous across species and allow parallel testing, data analyses, and interpretation between human and rodent behaviors. Such enhanced conformity between cross-species tasks will facilitate investigation of the neurobiological mechanisms underlying discrete reward and motivated behaviors and is expected to improve our understanding and treatment of neuropsychiatric disorders characterized by reward and motivation deficits.

The image-scratch paradigm: a new paradigm for evaluating infants' motivated gaze control

Human infants show spontaneous behaviours such as general movement, goal-directed behaviour, and self-motivated behaviour from a very early age. However, it is unclear how these behaviours are organised throughout development. A major hindrance to empirical investigation is that there is no common paradigm for all ages that can circumvent infants' underdeveloped verbal and motor abilities. Here, we propose a new paradigm, named the image-scratch task, using a gaze-contingent technique that is adaptable to various extents of motor ability. In this task, participants scratch off a black layer on a display to uncover pictures beneath it by using their gaze. We established quantitative criteria for spontaneous eye-movement based on adults' gaze-data and demonstrated that our task is useful for evaluating eye-movements motivated by outcome attractiveness in 8-month-olds. Finally, we discuss the potential of this paradigm for revealing the mechanisms and developmental transitions underlying infants' spontaneous and intentional behaviours.

Directive and incentive functions of affective action consequences: an ideomotor approach

Five experiments examined whether affective consequences become associated with the responses producing them and whether anticipations of positive and negative action outcomes influence action control differently. In a learning phase, one response produced pleasant and another response unpleasant visual effects. In a subsequent test phase, the same actions were carried out in response to a neutral feature of affective stimuli. Results showed that responses were faster when the irrelevant valence of the response cue matched the valence of the response outcome, but only when the responses still produced outcomes. These results suggest that affective action consequences have a directive function in that they facilitate the selection of the associated response over other responses, even when the response outcome is unpleasant (Experiment 4A). Results of another experiment showed that affective action consequences can also have an incentive function in that responses with pleasant outcomes are generally facilitated relative to responses with unpleasant outcomes. However, this motivational effect was seen only in a free-choice test (Experiment 5). The results suggest that behavioral impulses induced by ideomotor processes are constrained by the motivational evaluation of the anticipated action outcome. A model that integrates motivational factors into ideomotor theory is presented.

Functional, structural, and emotional correlates of impaired insight in cocaine addiction

IMPORTANCE

Individuals with cocaine use disorder (CUD) have difficulty monitoring ongoing behavior, possibly stemming from dysfunction of brain regions mediating insight and self-awareness.

OBJECTIVE

To investigate the neural correlates of impaired insight in addiction using a combined functional magnetic resonance imaging and voxel-based morphometry approach.

DESIGN, SETTING, AND PARTICIPANTS

This multimodal imaging study was performed at the Clinical Research Center at Brookhaven National Laboratory. The study included 33 CUD cases and 20 healthy controls.

MAIN OUTCOMES AND MEASURES

Functional magnetic resonance imaging, voxel-based morphometry, Levels of Emotional Awareness Scale, and drug use variables.

RESULTS

Compared with the other 2 study groups, the impaired insight CUD group had lower error-induced rostral anterior cingulate cortex (rACC) activity as associated with more frequent cocaine use, less gray matter within the rACC, and lower Levels of Emotional Awareness Scale scores.

CONCLUSIONS AND RELEVANCE

These results point to rACC functional and structural abnormalities and diminished emotional awareness in a subpopulation of CUD cases characterized by impaired insight. Because the rACC has been implicated in appraising the affective and motivational significance of errors and other types of self-referential processing, functional and structural abnormalities in this region could result in lessened concern (frequently ascribed to minimization and denial) about behavioral outcomes that could potentially culminate in increased drug use. Treatments that target this CUD subgroup could focus on enhancing the salience of errors (eg, lapses).

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

Neurocognitive mechanisms underlying social learning in infancy: infants' neural processing of the effects of others' actions

Social transmission of knowledge is one of the reasons for human evolutionary success, and it has been suggested that already human infants possess eminent social learning abilities. However, nothing is known about the neurocognitive mechanisms that subserve infants' acquisition of novel action knowledge through the observation of other people's actions and their consequences in the physical world. In an electroencephalogram study on social learning in infancy, we demonstrate that 9-month-old infants represent the environmental effects of others' actions in their own motor system, although they never achieved these effects themselves before. The results provide first insights into the neurocognitive basis of human infants' unique ability for social learning of novel action knowledge.

The contributions of value-based decision-making and attentional bias to alcohol-seeking following devaluation

AIMS

To investigate the mediating role of attentional bias for alcohol cues on alcohol-seeking following devaluation of alcohol.

DESIGN

Between subject.

SETTING

Eye-tracking laboratory at the University of Liverpool.

PARTICIPANTS

Student social drinkers (n = 64).

MEASUREMENTS

An operant choice task in which participants chose between simultaneously presented alcohol and non-alcohol drink rewards, while attentional bias for alcohol and non-alcohol drink cues was inferred from eye movements. Participants then consumed 30 mL of an alcoholic beverage, which was either presented alone (no devaluation: n = 32) or had been adulterated to taste unpleasant (devaluation: n = 32). Choice and attentional bias for the alcohol and non-alcohol drink pictures were then measured again.

FINDINGS

Alcohol devaluation reduced behavioural choice for alcohol (F = 32.64, P < 0.001) and attentional bias for the alcohol pictures indexed by dwell time (F = 22.68, P < 0.001), initial fixation (F = 7.08, P = 0.01) and final fixation (F = 22.44, P < 0.001). Mediation analysis revealed that attentional bias partially mediated the effect of devaluation on alcohol choice; however, the proportion of the variance accounted for by attentional bias is low to moderate (~30%).

CONCLUSIONS

Among student social drinkers, attentional bias is only a partial mediator of alcohol choice following devaluation of alcohol. Value-based decision-making may be a more important determinant of drinking behaviour among student social drinkers than attentional bias.

Phasic transition from goal-directed to habitual control over drug-seeking produced by conflicting reinforcer expectancy

The transition from goal-directed to habitual control over drug-seeking has been experimentally demonstrated in animals, but there have been no comparable reports in humans. Following a recent animal design, the current study employed an outcome-devaluation procedure to test whether goal-directed control over tobacco seeking would be abolished by alcohol expectancy. Eighty smokers first learned that two responses earned tobacco or chocolate points, respectively, before tobacco was devalued by health warnings and smoking satiety. Participants were then presented with either a glass of beer/wine or water with instructions that this item could be consumed after the task (alternative reward). Then choice between the tobacco and chocolate response was measured in extinction to assess goal-directed control of tobacco seeking, in a nominal Pavlovian to instrumental transfer (PIT) test to assess stimulus control of tobacco seeking, and in a reacquisition test to assess the impact of direct feedback from the outcomes. The results showed that alcohol expectancy selectively abolished goal-directed control of tobacco seeking but not stimulus control or the impact of feedback from outcomes. These data suggest that 'endogenous' retrieval of low drug value governing goal-directed regulation of drug seeking is disrupted by conflicting appraisal of an alternative reinforcer, promoting habitual control, which may play a role in relapse.

Associative learning mechanisms underpinning the transition from recreational drug use to addiction

Learning theory proposes that drug seeking is a synthesis of multiple controllers. Whereas goal-directed drug seeking is determined by the anticipated incentive value of the drug, habitual drug seeking is elicited by stimuli that have formed a direct association with the response. Moreover, drug-paired stimuli can transfer control over separately trained drug seeking responses by retrieving an expectation of the drug's identity (specific transfer) or incentive value (general transfer). This review covers outcome devaluation and transfer of stimulus-control procedures in humans and animals, which isolate the differential governance of drug seeking by these four controllers following various degrees of contingent and noncontingent drug exposure. The neural mechanisms underpinning these four controllers are also reviewed. These studies suggest that although initial drug seeking is goal-directed, chronic drug exposure confers a progressive loss of control over action selection by specific outcome representations (impaired outcome devaluation and specific transfer), and a concomitant increase in control over action selection by antecedent stimuli (enhanced habit and general transfer). The prefrontal cortex and mediodorsal thalamus may play a role in this drug-induced transition to behavioral autonomy.

Producing and understanding prosocial actions in early childhood

Early prosocial development has become a topic of great interest in recent years as experimental studies have provided evidence that instances of prosocial action can already be found in the second year of life. In this contribution, we review the recent literature on young children's production of prosocial actions, in particular helping, sharing, and comforting, and their understanding of others' prosocial behavior. We summarize the novel insights gained by recent studies and point to directions for future studies. Overall, we suggest that while the field consists of considerable knowledge about the developmental timeline of prosocial behavior, more research is needed to examine and better understand underlying social-cognitive mechanisms.

Choice and goal-directed behavior in preschool children

Preschool children (3-4 years old) were trained to perform two actions to gain different outcomes, in the form of video clips from different cartoons, before one of these outcomes was devalued by noncontingent exposure. The effect of outcome devaluation was subsequently assessed in an extinction test by giving children the opportunity to perform both actions in the absence of any outcomes. When the two actions were trained concurrently, performance during the test was modulated by outcome value and children showed a preference for the action trained with the currently valued outcome. By contrast, when each action was trained separately on different trials, test performance was insensitive to outcome devaluation. These effects of the training schedules are interpreted in terms of dual-process theories of action control.

Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates

Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology, and machine learning to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.

Rational action selection in 1½- to 3-year-olds following an extended training experience

Previous studies failed to find evidence for rational action selection in children under 2 years of age. The current study investigated whether younger children required more training to encode the relevant causal relationships. Children between 1½ and 3 years of age were trained over two sessions to perform actions on a touch-sensitive screen to obtain video clips as outcomes. Subsequently, a visual habituation procedure was employed to devalue one of the training outcomes. As in previous studies, 2- and 3-year-olds chose actions associated with an expected valued outcome significantly more often during a subsequent choice test. Moreover, analysis of children's first responses in the post-devaluation test revealed evidence of rational action selection even in the youngest age group (18-23 months). Consistent with dual-process accounts of action control, the findings support the view that the ability to make rational action choices develops gradually.

Impaired goal-directed behavioural control in human impulsivity

Two dissociable learning processes underlie instrumental behaviour. Whereas goal-directed behaviour is controlled by knowledge of the consequences, habitual behaviour is elicited directly by antecedent Pavlovian stimuli without knowledge of the consequences. Predominance of habitual control is thought to underlie psychopathological conditions associated with corticostriatal abnormalities, such as impulsivity and drug dependence. To explore this claim, smokers were assessed for nicotine dependence, impulsivity, and capacity for goal-directed control over instrumental performance in an outcome devaluation procedure. Reduced goal-directed control was selectively associated with the Motor Impulsivity factor of Barrett's Impulsivity Scale (BIS), which reflects propensity for action without thought. These data support the claim that human impulsivity is marked by impaired use of causal knowledge to make adaptive decisions. The predominance of habit learning may play a role in psychopathological conditions that are associated with trait impulsivity.

How learning to shake a rattle affects 8-month-old infants' perception of the rattle's sound: electrophysiological evidence for action-effect binding in infancy

Bidirectional action-effect associations play a fundamental role in intentional action control and the development of the mirror neuron system. However, it has been questioned if infants are able to acquire bidirectional action-effect associations (i.e., are able to intentionally control their actions). To investigate this, we trained 8-month-old infants for one week to use a novel rattle that produced a specific sound when shaken. Infants were also presented with another sound, which was not related to an action. Thereafter, infants' EEG responses to these two sounds and to an additional, unfamiliar sound were recorded. Infants displayed a stronger mu-desynchronization above cortical motor sites (i.e., motor resonance) when listening to the action-related sound than when hearing other sounds. Our results provide therefore electrophysiological evidence that infants as young as 8 months are able to acquire bidirectional action-effect associations and parallel findings of audiovisual mirror neurons in the monkey brain.

Over-imitation is better explained by norm learning than by distorted causal learning

In over-imitation, children copy even elements of a goal-directed action sequence that appear unnecessary for achieving the goal. We demonstrate in 4-year olds that the unnecessary action is specifically associated with the goal, not generally associated with the apparatus. The unnecessary action is performed flexibly: 4-year olds usually omit it if it has already been performed by an adult. Most 5-year olds do not verbally report the unnecessary action as necessary when achieving the goal, although most of them report an equivalent but necessary action as necessary. Most 5-year olds explain the necessary action in functional terms, but are unsure as to the function of the unnecessary action. These verbal measures do not support the hypothesis that children over-imitate primarily because they encode unnecessary actions as causing the goal even in causally transparent systems. In a causally transparent system, explanations for over-imitation fitting the results are that children are ignorant of the unnecessary action's purpose, and that they learn a prescriptive norm that it should be carried out. In causally opaque systems, however, for children and for adults, any action performed before achieving the goal is likely to be inferred as causally necessary-this is not over-imitation, but ordinary causal learning.

Associative theories of goal-directed behaviour: a case for animal-human translational models

Associative accounts of goal-directed action, developed in the fields of human ideomotor action and that of animal learning, can capture cognitive belief-desire psychology of human decision-making. Whereas outcome-response accounts can account for the fact that the thought of a goal can call to mind the action that has previously procured this goal, response-outcome accounts capture decision-making processes that start out with the consideration of possible response alternatives followed only in the second instance by evaluation of their consequences. We argue that while the outcome-response mechanism plays a crucial role in response priming effects, the response-outcome mechanism is particularly important for action selection on the basis of current needs and desires. We therefore develop an integrative account that encapsulates these two routes of action selection within the framework of the associative-cybernetic model. This model has the additional benefit of providing mechanisms for the incentive modulation of goal-directed action and for the development of behavioural autonomy, and therefore provides a promising account of the multi-faceted process of animal as well as human instrumental decision-making.