Cognitive versus stimulus-response theories of learning

The nature of training in flavor preference learning determines the underlying associative structure

Pairing a palatable flavor (US) with an initially neutral flavor cue (CS) results in an acquired conditioned preference for the latter. Two main associations have been proposed to explain the acquisition of flavor preferences: Flavor-Flavor and Flavor-Nutrient learning. Although the hedonic reaction triggered by US consumption has also been suggested as a possible additional component underlying acquired flavor preference, this issue has received little attention. Here we explored whether the amount of training to the CS-US compound can favor the formation of a Flavor-Hedonic reaction association using rats as subjects and sucrose as the US. We expected that the more exposure to the CS-US compound, the stronger the S-R type association. Since S-R associations are not sensitive to devaluation procedures, we used a Sensory-Specific Satiety procedure to devalue the US after conditioning and then measured preferences for the CS. On Experiment 1 with a short restrictive training (classic procedure), preference for the CS was decreased after devaluation of the US compared to the control condition. On Experiment 2, with short unrestrictive training, preference for the CS was again weakened. Experiment 3 with a long unrestrictive training, rats expressed preference for the CS regardless of the devaluation procedure. These results suggest that, as with an instrumental paradigm, extensive training in flavor preference learning undermines the US devaluation effect.

Topographically selective motor inhibition under threat of pain

ABSTRACT

Pain-related motor adaptations may be enacted predictively at the mere threat of pain, before pain occurrence. Yet, in humans, the neurophysiological mechanisms underlying motor adaptations in anticipation of pain remain poorly understood. We tracked the evolution of changes in corticospinal excitability (CSE) as healthy adults learned to anticipate the occurrence of lateralized, muscle-specific pain to the upper limb. Using a Pavlovian threat conditioning task, different visual stimuli predicted pain to the right or left forearm (experiment 1) or hand (experiment 2). During stimuli presentation before pain occurrence, single-pulse transcranial magnetic stimulation was applied over the left primary motor cortex to probe CSE and elicit motor evoked potentials from target right forearm and hand muscles. The correlation between participants' trait anxiety and CSE was also assessed. Results showed that threat of pain triggered corticospinal inhibition specifically in the limb where pain was expected. In addition, corticospinal inhibition was modulated relative to the threatened muscle, with threat of pain to the forearm inhibiting the forearm and hand muscles, whereas threat of pain to the hand inhibited the hand muscle only. Finally, stronger corticospinal inhibition correlated with greater trait anxiety. These results advance the mechanistic understanding of pain processes showing that pain-related motor adaptations are enacted at the mere threat of pain, as sets of anticipatory, topographically organized motor changes that are associated with the expected pain and are shaped by individual anxiety levels. Including such anticipatory motor changes into models of pain may lead to new treatments for pain-related disorders.

Effect of texture preference on food texture perception: Exploring the role of matching food texture and preference

This study aimed to investigate the influence of smooth texture preference on smoothness perception. An online questionnaire (Study 1, n = 464) and a sensory evaluation test (Study 2, n = 65) were administered to Japanese elderly participants (65-74 years), with common Japanese confectionery (Daifuku) as test foods. Through the online questionnaire, four distinct texture preference groups were formed based on the factors of preference for smoothness and firmness of the inner bean paste layer. Analysis of the food preference scale for imbalanced diet (FPSID) revealed that smooth-texture likers were more likely to be picky eaters than firm-texture likers. Furthermore, high (HiSm) and low smoothness preference groups (LoSm) were selected for the food sensory evaluation test to compare perceived textures (smoothness and firmness). Only the HiSm group exhibited a positive association between perceived smoothness and overall texture liking, perceiving smoothness significantly more intense than the LoSm group in situations where overall texture liking was high. This finding indicates that smooth texture preference does not act independently but rather interacts with food texture matching to affect perception. Our findings suggest that when food texture aligns with individuals' preferences, it elicits hedonic emotions and dynamically enhances food texture perception. This preference-involved perceptual process may contribute to the development of more explicit texture preferences.

Effects of predictive and incentive value manipulation on sign- and goal-tracking behavior

When a neutral stimulus is repeatedly paired with an appetitive reward, two different types of conditioned approach responses may develop: a sign-tracking response directed toward the neutral cue, or a goal-tracking response directed toward the location of impending reward delivery. Sign-tracking responses have been postulated to result from attribution of incentive value to conditioned cues, while goal-tracking reflects the assignment of only predictive value to the cue. We therefore hypothesized that sign-tracking rats would be more sensitive to manipulations of incentive value, while goal-tracking rats would be more responsive to changes in the predictive value of the cue. We tested sign- and goal-tracking before and after devaluation of a food reward using lithium chloride, and tested whether either response could be learned under negative contingency conditions that precluded any serendipitous reinforcement of the behavior that might support instrumental learning. We also tested the effects of blocking the predictive value of a cue using simultaneous presentation of a pre-conditioned cue. We found that sign-tracking was sensitive to outcome devaluation, while goal-tracking was not. We also confirmed that both responses are Pavlovian because they can be learned under negative contingency conditions. Goal-tracking was almost completely blocked by a pre-conditioned cue, while sign-tracking was much less sensitive to such interference. These results indicate that sign- and goal-tracking may follow different rules of reinforcement learning and suggest a need to revise current models of associative learning to account for these differences.

Roles of octopamine neurons in the vertical lobe of the mushroom body for the execution of a conditioned response in cockroaches

Aminergic neurons mediate reward signals in mammals and insects. In crickets, we showed that blockade of synaptic transmission from octopamine neurons (OANs) impairs conditioning of an odor (conditioned stimulus, CS) with water or sucrose (unconditioned stimulus, US) and execution of a conditioned response (CR) to the CS. It has not yet been established, however, whether findings in crickets can be applied to other species of insects. In this study, we investigated the roles of OANs in conditioning of salivation, monitored by activities of salivary neurons, and in execution of the CR in cockroaches (Periplaneta americana). We showed that injection of epinastine (an OA receptor antagonist) into the head hemolymph impaired both conditioning and execution of the CR, in accordance with findings in crickets. Moreover, local injection of epinastine into the vertical lobes of the mushroom body (MB), the center for associative learning and control of the CR, impaired execution of the CR, whereas injection of epinastine into the calyces of the MB or the antennal lobes (primary olfactory centers) did not. We propose that OANs in the MB vertical lobes play critical roles in the execution of the CR in cockroaches. This is analogous to the fact that midbrain dopamine neurons govern execution of learned actions in mammals.

The Role of Habit Formation and Automaticity in Diabetes Self-Management: Current Evidence and Future Applications

PURPOSE OF REVIEW

Diabetes is a chronic condition that requires consistent self-management for optimal health outcomes. People with diabetes are prone to burnout, cognitive burden, and sub-optimal performance of self-management tasks. Interventions that focus on habit formation have the potential to increase engagement by facilitating automaticity of self-management task performance. The purpose of this review is to (1) clarify the conceptualizations of habit formation and behavioral automaticity in the context of health behavior interventions, (2) review the evidence of habit in relation to behaviors relevant to diabetes self-management, and (3) discuss opportunities for incorporating habit formation and automaticity into diabetes self-management interventions.

RECENT FINDINGS

Modern habit research describes a habit as a behavior that results over time from an automatic mental process. Automatic behaviors are experienced as cue-dependent, goal-independent, unconscious, and efficient. Habit formation requires context-dependent repetition to form cue-behavior associations. Results of diabetes habit studies are mixed. Observational studies have shown positive associations between habit strength and target self-management behaviors such as taking medication and monitoring blood glucose, as well as glycemic outcomes such as HbA1c. However, intervention studies conducted in similar populations have not demonstrated a significant benefit of habit-forming interventions compared to controls, possibly due to varying techniques used to promote habit formation. Automaticity of self-management behaviors has the potential to minimize the burden associated with performance of self-management tasks and ultimately improve outcomes for people with diabetes. Future studies should focus on refining interventions focused on context-dependent repetition to promote habit formation and better measurement of habit automaticity in diabetes self-management.

Food Restriction in Anorexia Nervosa in the Light of Modern Learning Theory: A Narrative Review

Improvements in the clinical management of anorexia nervosa (AN) are urgently needed. To do so, the search for innovative approaches continues at laboratory and clinical levels to translate new findings into more effective treatments. In this sense, modern learning theory provides a unifying framework that connects concepts, methodologies and data from preclinical and clinical research to inspire novel interventions in the field of psychopathology in general, and of disordered eating in particular. Indeed, learning is thought to be a crucial factor in the development/regulation of normal and pathological eating behaviour. Thus, the present review not only tries to provide a comprehensive overview of modern learning research in the field of AN, but also follows a transdiagnostic perspective to offer testable explanations for the origin and maintenance of pathological food rejection. This narrative review was informed by a systematic search of research papers in the electronic databases PsycInfo, Scopus and Web of Science following PRISMA methodology. By considering the number and type of associations (Pavlovian, goal-directed or habitual) and the affective nature of conditioning processes (appetitive versus aversive), this approach can explain many features of AN, including why some patients restrict food intake to the point of life-threatening starvation and others restrict calorie intake to lose weight and binge on a regular basis. Nonetheless, it is striking how little impact modern learning theory has had on the current AN research agenda and practice.

Applying adaptive distributed practice to self-managed computer-based anomia treatment: A single-case experimental design

INTRODUCTION

There is a pressing need to improve computer-based treatments for aphasia to increase access to long-term effective evidence-based interventions. The current single case design incorporated two learning principles, adaptive distributed practice and stimuli variability, to promote acquisition, retention, and generalization of words in a self-managed computer-based anomia treatment.

METHODS

Two participants with post-stroke aphasia completed a 12-week adaptive distributed practice naming intervention in a single-case experimental design. Stimuli variability was manipulated in three experimental conditions: high exemplar variability, low exemplar variability, and verbal description prompt balanced across 120 trained words. Outcomes were assessed at 1-week, 1-month, and 3-months post-treatment. Statistical comparisons and effect sizes measured in the number of words acquired, generalized, and retained were estimated using Bayesian generalized mixed-effect models.

RESULTS

Participants showed large and robust acquisition, generalization, and retention effects. Out of 120 trained words, participant 1 acquired ∼77 words (trained picture exemplars) and ∼63 generalization words (untrained picture exemplars of treated words). Similarly, participant 2 acquired ∼57 trained words and ∼48 generalization words. There was no reliable change in untrained control words for either participant. Stimuli variability did not show practically meaningful effects.

CONCLUSIONS

These case studies suggest that adaptive distributed practice is an effective method for re-training more words than typically targeted in anomia treatment research (∼47 words on average per Snell et al., 2010). Generalization across experimental conditions provided evidence for improved lexical access beyond what could be attributed to simple stimulus-response mapping. These effects were obtained using free, open-source flashcard software in a clinically feasible, asynchronous format, thereby minimizing clinical implementation barriers. Larger-scale clinical trials are required to replicate and extend these effects.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Training-Dependent Change in Content of Association in Appetitive Pavlovian Conditioning

In appetitive Pavlovian conditioning, experience with a conditional relationship between a cue [conditioned stimulus (CS)] and a reward [unconditioned stimulus (US)] bestows CS with the ability to promote adaptive behavior patterns. Different features of US (e.g., identity-specific sensory, general motivational) can be encoded by CS based on the nature of the CS-US relationship experienced (e.g., temporal factors such as training amount) and the content of association may determine the influence of CS over behavior (e.g., mediated learning, conditioned reinforcement). The content of association changed with varying conditioning factors, thereby altering behavioral consequences, however, has never been addressed in relevant brain signals evoked by CS. Our previous study found that phospholipase C β1-knockout (PLCβ1-KO) mice display persistent mediated learning over the extended course of odor-sugar conditioning, and that wild-type (WT) mice lose mediated learning sensitivity after extended training. In this study, in order to see whether this behavioral difference between these two genotypes comes from a difference in the course of association content, we examined whether odor CS can evoke the taste sensory representation of an absent sugar US after minimal- and extended training in these mice. In contrast to WT, which lost CS-evoked neural activation (c-Fos expression) in the gustatory cortex after extended training, KO mice displayed persistent association with the sensory feature of sugar, suggesting that sensory encoding is reliably linked to mediated learning sensitivity and there is a training-dependent change in the content of association in WT. PLCβ1 knockdown in the left medial prefrontal cortex (mPFC) resulted in mediated learning sensitivity and CS-evoked gustatory cortical activation after extended training, proposing a molecular component of the neural system underlying this Pavlovian conditioning process. We also discuss how disruption of this process is implicated for hallucination-like behaviors (impaired reality testing).

What Is Learned in Pavlovian Conditioning in Crickets? Revisiting the S-S and S-R Learning Theories

In Pavlovian conditioning in mammals, two theories have been proposed for associations underlying conditioned responses (CRs). One theory, called S-S theory, assumes an association between a conditioned stimulus (CS) and internal representation of an unconditioned stimulus (US), allowing the animal to adjust the CR depending on the current value of the US. The other theory, called S-R theory, assumes an association or connection between the CS center and the CR center, allowing the CS to elicit the CR. Whether these theories account for Pavlovian conditioning in invertebrates has remained unclear. In this article, results of our studies in the cricket Gryllus bimaculatus are reviewed. We showed that after a standard amount of Pavlovian training, crickets exhibited no response to odor CS when water US was devalued by providing it until satiation, whereas after extended training, they exhibited a CR after US devaluation. An increase of behavioral automaticity by extended training has not been reported in Pavlovian conditioning in any other animals, but it has been documented in instrumental conditioning in mammals. Our pharmacological analysis suggested that octopamine neurons mediate US (water) value signals and control execution of the CR after standard training. The control, however, diminishes with extension of training and hence the CR becomes insensitive to the US value. We also found that the nature of the habitual response after extended Pavlovian training in crickets is not the same as that after extended instrumental training in mammals concerning the context specificity. Adaptive significance and evolutionary implications for our findings are discussed.

Appetitive and aversive sensory preconditioning in rats is impaired by disruption of the postrhinal cortex

Episodic memory involves binding stimuli and/or events together in time and place. Furthermore, memories become more complex when new experiences influence the meaning of stimuli within the original memory. Thus collectively, complex episodic memory formation and maintenance involves processes such as encoding, storage, retrieval, updating and reconsolidation, which can be studied using animal models of higher-order conditioning. In the present study aversive and appetitive sensory preconditioning paradigms were used to test the hypothesis that the postrhinal cortex (POR), which is a component of the hippocampal memory system, is involved in higher-order conditioning. Drawing on the known role of the POR in contextual learning, Experiment 1 employed a four-phase sensory preconditioning task that involved fear learning and context discrimination in rats with or without permanent lesions of the POR. In parallel, to examine POR function during higher-order conditioning in the absence of a particular spatial arrangement, Experiments 2 and 3 used a three-phase sensory preconditioning paradigm involving phasic stimuli. In Experiment 2, bilateral lesions of the POR were made and in Experiment 3, a chemogenetic approach was used to temporarily inactivate POR neurons during each phase of the paradigm. Evidence of successful sensory preconditioning was observed in sham rats which, during the critical context discrimination test, demonstrated higher levels of freezing behavior when re-exposed to the paired versus the unpaired context, whereas POR-lesioned rats did not. Data from the appetitive sensory preconditioning paradigm also confirmed the hypothesis in that during the critical auditory discrimination test, sham rats showed greater food cup responding following presentations of the paired compared to the unpaired auditory stimulus, whereas POR-lesioned rats did not. Lastly, in Experiment 3, when the POR was inactivated only during preconditioning or only during conditioning, discrimination during the critical auditory test was impaired. Thus, regardless of whether stimulus-stimulus associations were formed between static or phasic stimuli or whether revaluation of the paired stimulus occurred through association with an aversive or an appetitive unconditioned stimulus, the effects were the same; POR lesions disrupted the ability to use higher-order conditioned stimuli to guide prospective behavior.

A-learning: A new formulation of associative learning theory

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

Development of behavioural automaticity by extended Pavlovian training in an insect

The effect of repetitive training on learned actions has been a major subject in behavioural neuroscience. Many studies of instrumental conditioning in mammals, including humans, suggested that learned actions early in training are goal-driven and controlled by outcome expectancy, but they become more automatic and insensitive to reduction in the value of the outcome after extended training. It was unknown, however, whether the development of value-insensitive behaviour also occurs by extended training of Pavlovian conditioning in any animals. Here we show that crickets Gryllus bimaculatus that had received minimal training to associate an odour with water (unconditioned stimulus, US) did not exhibit conditioned response (CR) to the odour when they were given water until satiation before the test, but those that had received extended training exhibited CR even when they were satiated with water. Further pharmacological experiments suggested that octopamine neurons, the invertebrate counterparts of noradrenaline neurons, mediate US value signals and control execution of CR after minimal training, but the control diminishes with the progress of training and hence the CR becomes insensitive to US devaluation. The results suggest that repetitive sensory experiences can lead to a change from a goal-driven response to a more automatic one in crickets.

Above and beyond the concrete: The diverse representational substrates of the predictive brain

In recent years, scientists have increasingly taken to investigate the predictive nature of cognition. We argue that prediction relies on abstraction, and thus theories of predictive cognition need an explicit theory of abstract representation. We propose such a theory of the abstract representational capacities that allow humans to transcend the "here-and-now." Consistent with the predictive cognition literature, we suggest that the representational substrates of the mind are built as a hierarchy, ranging from the concrete to the abstract; however, we argue that there are qualitative differences between elements along this hierarchy, generating meaningful, often unacknowledged, diversity. Echoing views from philosophy, we suggest that the representational hierarchy can be parsed into: modality-specific representations, instantiated on perceptual similarity; multimodal representations, instantiated primarily on the discovery of spatiotemporal contiguity; and categorical representations, instantiated primarily on social interaction. These elements serve as the building blocks of complex structures discussed in cognitive psychology (e.g., episodes, scripts) and are the inputs for mental representations that behave like functions, typically discussed in linguistics (i.e., predicators). We support our argument for representational diversity by explaining how the elements in our ontology are all required to account for humans' predictive cognition (e.g., in subserving logic-based prediction; in optimizing the trade-off between accurate and detailed predictions) and by examining how the neuroscientific evidence coheres with our account. In doing so, we provide a testable model of the neural bases of conceptual cognition and highlight several important implications to research on self-projection, reinforcement learning, and predictive-processing models of psychopathology.

Failure to demonstrate short-cutting in a replication and extension of Tolman et al.'s spatial learning experiment with humans

Successful demonstrations of novel short-cut taking by animals, including humans, are open to interpretation in terms of learning that is not necessarily spatial. A classic example is that of Tolman, Ritchie, and Kalish (1946) who allowed rats to repeat a sequence of turns through the corridors of a maze to locate a food reward. When the entrance to the corridors was subsequently blocked and alternative corridors were made available, rats successfully selected the corridor leading most directly to the food location. However, the presence of a distinctive light above the goal, in both the training and testing phases, means that approach to the light as a beacon could have been the source of successful short-cutting. We report a replication of the experimental design of Tolman et al. with human participants who explored geometrically equivalent virtual environments. An experimental group, who followed the original procedure in the absence of any distinctive cues proximal to the goal, did not select the corridor which led most directly to the goal. A control group, who experienced a light above the goal during training and testing, were more likely to select a corridor which led in the direction of the goal. A second control group experienced the light above the goal during training, but in the test the location of this cue was shifted by 90° with respect to the start point of exploration. This latter group responded unsystematically in the test, neither selecting a corridor leading to the original goal location, nor one leading directly to the relocated light cue. The results do not support the hypothesis that travelling a multi-path route automatically leads to an integrated cognitive representation of that route. The data offer support for the importance of local cues which can serve as beacons to indicate the location of a goal.

The Role of Habits in Anorexia Nervosa: Where We Are and Where to Go From Here?

PURPOSE OF REVIEW

The persistent maladaptive eating behavior characteristic of anorexia nervosa (AN) can be understood as a learned habit. This review describes the cognitive neuroscience background and the existing data from research in AN.

RECENT FINDINGS

Behavior is habitual after it is frequently repeated and becomes nearly automatic, relatively insensitive to outcome, and mediated by dorsal frontostriatal neural systems. There is evidence for such behavior in AN, in which restrictive intake has been related to dorsal frontostriatal systems. Other neural and neurocognitive data provide mixed findings, some of which suggest disturbances in habit systems in AN. There are compelling behavioral and neural data to suggest that habit systems may underlie the persistence of AN. The habit model needs further research, via more direct behavioral hypothesis testing and probes of the development of habitual behavior. Investigation of the habit-centered model of AN may open avenues for the development of novel treatments.

Proprioceptive stimuli and habit formation: Interresponse time mediated behavior in CD-1 mice

The consolidation of behavioral sequences into relatively ballistic habits is thought to involve the formation of stimulus - response associations. Typically, the stimuli in these associations are assumed to be exteroceptive, i.e., external to the organism. However, responses, themselves, also possess stimulus properties that can mediate behavior. Indeed, these "proprioceptive cues" have long been hypothesized to underlie habit formation (Hull, 1934a, 1934b). One such stimulus involves the time durations between responses - a stimulus termed interresponse time (IRT). We hypothesize that IRTs can come to serve as stimuli that differentially control response elements during habit formation. To examine this hypothesis we report on two experiments that asked whether CD-1 mice utilize IRTs to structure behavior in a two-choice environment. In experiment 1, eight mice were exposed to a free-operant concurrent variable-interval (VI) 30-s VI 60-s reinforcement schedule. We found that switch and stay responses were differentially correlated with IRT durations. In Experiment 2 we directly and differentially reinforced stay/switch responses based on IRT durations in a two-lever procedure. For four of the subjects, the probability of receiving reinforcement after switch responses was proportional to IRT duration. For five of the subjects, these reinforcement probabilities were inversely proportional to IRT duration. Regardless, all of our subjects learned to emit IRT-mediated switching behavior that matched the reinforcement contingencies. Together, Experiments 1 and 2 provide the first evidence of which we are aware that IRTs can come to control sequential choice behavior in mice.

Regulation of habit formation in the dorsal striatum

Habits are an essential and pervasive component of our daily lives that allow us to efficiently perform routine tasks. But their disruption contributes to the symptoms that underlie many psychiatric diseases. Emerging data are revealing the cellular and molecular mechanisms of habit formation in the dorsal striatum. New data suggest that in both the dorsolateral and dorsomedial striatum histone deacetylase (HDAC) activity acts as a critical negative regulator of the transcriptional processes underlying habit formation. In this review, we discuss this recent work and draw conclusions relevant to the treatment of diseases marked by maladaptive habits.

Selective Modulation of Orbitofrontal Network Activity during Negative Occasion Setting

Discrete cues can gain powerful control over behavior to help an animal anticipate and cope with upcoming events. This is important in conditions where understanding the relationship between complex stimuli provides a means to resolving situational ambiguity. However, it is unclear how cortical circuits generate and maintain these signals that conditionally regulate behavior. To address this, we established a Pavlovian serial feature-negative conditioning paradigm, where male mice are trained on a trial in which a conditioned stimulus (CS) is presented alone and followed by reward, or a feature-negative trial in which the CS is preceded by a feature cue indicating there is no reward. Mice learn to respond with anticipatory licking to a solitary CS, but significantly suppress their responding to the same cue during feature-negative trials. We show that the feature cue forms a selective association with its paired CS, because the ability of the feature to transfer its suppressive properties to a separately rewarded cue is limited. Next, to examine the underlying neural dynamics, we conduct recordings in the orbitofrontal cortex (OFC). We find that the feature cue significantly and selectively inhibits CS-evoked activity. Finally, we find that the feature triggers a distinct OFC network state during the delay period between the feature and CS, establishing a potential link between the feature and future events. Together, our findings suggest that OFC dynamics are modulated by the feature cue and its associated conditioned stimulus in a manner consistent with an occasion setting model.SIGNIFICANCE STATEMENT The ability of patterned cues to form an inhibitory relationship with ambiguously rewarded outcomes has been appreciated since early studies on learning and memory. However, it was often assumed that these cues, despite their hierarchical nature, still made direct associative links with neural rewarding events. This model was significantly challenged, largely by the work of Holland and colleagues, who demonstrated that under certain conditions cues can inherit occasion setting properties whereby they modulate the ability of a paired cue to elicit its conditioned response. Here we provide some of the first evidence that the activity of a cortical circuit is selectively modulated by such cues, thereby providing insight into the mechanisms of higher order learning.

The influence of focused-attention meditation states on the cognitive control of sequence learning

Cognitive control processes influence how motor sequence information is utilised and represented. Since cognitive control processes are shared amongst goal-oriented tasks, motor sequence learning and performance might be influenced by preceding cognitive tasks such as focused-attention meditation (FAM). Prior to a serial reaction time task (SRTT), participants completed either a single-session of FAM, a single-session of FAM followed by delay (FAM+) or no meditation (CONTROL). Relative to CONTROL, FAM benefitted performance in early, random-ordered blocks. However, across subsequent sequence learning blocks, FAM+ supported the highest levels of performance improvement resulting in superior performance at the end of the SRTT. Performance following FAM+ demonstrated greater reliance on embedded sequence structures than FAM. These findings illustrate that increased top-down control immediately after FAM biases the implementation of stimulus-based planning. Introduction of a delay following FAM relaxes top-down control allowing for implementation of response-based planning resulting in sequence learning benefits.

The consequence of spatial visual processing dysfunction caused by traumatic brain injury (TBI)

OBJECTIVE

A bi-modal visual processing model is supported by research to affect dysfunction following a traumatic brain injury (TBI). TBI causes dysfunction of visual processing affecting binocularity, spatial orientation, posture and balance. Research demonstrates that prescription of prisms influence the plasticity between spatial visual processing and motor-sensory systems improving visual processing and reducing symptoms following a TBI.

RATIONALE

The rationale demonstrates that visual processing underlies the functional aspects of binocularity, balance and posture. The bi-modal visual process maintains plasticity for efficiency. Compromise causes Post Trauma Vision Syndrome (PTVS) and Visual Midline Shift Syndrome (VMSS). Rehabilitation through use of lenses, prisms and sectoral occlusion has inter-professional implications in rehabilitation affecting the plasticity of the bi-modal visual process, thereby improving binocularity, spatial orientation, posture and balance Main outcomes: This review provides an opportunity to create a new perspective of the consequences of TBI on visual processing and the symptoms that are often caused by trauma. It also serves to provide a perspective of visual processing dysfunction that has potential for developing new approaches of rehabilitation.

CONCLUSIONS

Understanding vision as a bi-modal process facilitates a new perspective of visual processing and the potentials for rehabilitation following a concussion, brain injury or other neurological events.

Memory: Organization and Control

A major goal of memory research is to understand how cognitive processes in memory are supported at the level of brain systems and network representations. Especially promising in this direction are new findings in humans and animals that converge in indicating a key role for the hippocampus in the systematic organization of memories. New findings also indicate that the prefrontal cortex may play an equally important role in the active control of memory organization during both encoding and retrieval. Observations about the dialog between the hippocampus and prefrontal cortex provide new insights into the operation of the larger brain system that serves memory.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

The birth, death and resurrection of avoidance: a reconceptualization of a troubled paradigm

Research on avoidance conditioning began in the late 1930s as a way to use laboratory experiments to better understand uncontrollable fear and anxiety. Avoidance was initially conceived of as a two-factor learning process in which fear is first acquired through Pavlovian aversive conditioning (so-called fear conditioning), and then behaviors that reduce the fear aroused by the Pavlovian conditioned stimulus are reinforced through instrumental conditioning. Over the years, criticisms of both the avoidance paradigm and the two-factor fear theory arose. By the mid-1980s, avoidance had fallen out of favor as an experimental model relevant to fear and anxiety. However, recent progress in understanding the neural basis of Pavlovian conditioning has stimulated a new wave of research on avoidance. This new work has fostered new insights into contributions of not only Pavlovian and instrumental learning but also habit learning, to avoidance, and has suggested that the reinforcing event underlying the instrumental phase should be conceived in terms of cellular and molecular events in specific circuits rather than in terms of vague notions of fear reduction. In our approach, defensive reactions (freezing), actions (avoidance) and habits (habitual avoidance) are viewed as being controlled by unique circuits that operate nonconsciously in the control of behavior, and that are distinct from the circuits that give rise to conscious feelings of fear and anxiety. These refinements, we suggest, overcome older criticisms, justifying the value of the new wave of research on avoidance, and offering a fresh perspective on the clinical implications of this work.

Associative structure of integrated temporal relationships

According to the temporal-coding hypothesis (TCH; Savastano & Miller, Behavioural Processes 44:147-162, 1998), acquired associations include temporal information concerning the interval between the associated elements. Moreover, the TCH posits that subjects can integrate two independently acquired associations that share a common element (e.g., S2-S1 and S1-US), which results in the creation of a third association with its own temporal relationship (S2-US). Some evidence has suggested that such temporal integration occurs at the time of testing (Molet, Miguez, Cham, & Miller, Journal of Experimental Psychology: Animal Behavior Processes 38:369-380, 2012). Here we report two fear-conditioning experiments with rats conducted to identify the associative structure of the integrated temporal relationship. The goal was to distinguish between two possible associative structures that could exist following an initial test on which temporal integration occurs: (1) Conditioned responding to S2 on subsequent tests could be the result of recurring successive activation of two independently learned temporal maps that remain independently stored in memory (i.e., S2-S1 plus S1-US). (2) Temporal integration at the moment of initial testing could result in the formation of a direct S2-US (or S2-response) temporal map. Integration was found to occur at test and to produce a new association that was independent of associations with the common element (S1). However, the associative status of S1 appeared to modulate whether or not the new association with S2 was US-specific (S2-US) or directly activated a fear response (S2-response).

Impaired Reality Testing in Mice Lacking Phospholipase Cβ1: Observed by Persistent Representation-Mediated Taste Aversion

Hallucinations and delusions are the most prominent symptoms of schizophrenia and characterized by impaired reality testing. Representation-mediated taste aversion (RMTA) has been proposed as a potential behavioral assessment of reality testing and has been applied to a neurodevelopmental rat model of schizophrenia. However, the theory underlying this approach has not been generalized yet with any demonstration of impaired reality testing in other animal models of schizophrenia, such as genetically-modified mice. We devised a RMTA procedure for mice that combines a Pavlovian association protocol pairing odor conditioned stimulus (CS) with sugar reward unconditioned stimulus (US), and a conditioned taste aversion (CTA) method. In this RMTA paradigm, we compared performances of wild-type (PLCβ1+/+) mice and phospholipase C β1 knock-out (PLCβ1-/-) mice which are known as one of the genetic models for schizophrenia. With a minimal amount of initial odor-sugar associative training, both PLCβ1+/+ and PLCβ1-/- mice were able to form an aversion to the sugar reward when the odor CS predicting sugar was paired with nausea. With an extended initial training, however, only PLCβ1-/- mice could form a RMTA. This persistent RMTA displayed by PLCβ1-/- mice shows their inability to distinguish real sugar from the CS-evoked representation of sugar at a stage in associative learning where wild-type mice normally could differentiate the two. These results demonstrate an impaired reality testing first observed in a genetic mouse model of schizophrenia, and suggest that RMTA paradigm may, with general applicability, allow diverse biological approaches to impaired reality testing.

The Origins and Organization of Vertebrate Pavlovian Conditioning

Pavlovian conditioning is the process by which we learn relationships between stimuli and thus constitutes a basic building block for how the brain constructs representations of the world. We first review the major concepts of Pavlovian conditioning and point out many of the pervasive misunderstandings about just what conditioning is. This brings us to a modern redefinition of conditioning as the process whereby experience with a conditional relationship between stimuli bestows these stimuli with the ability to promote adaptive behavior patterns that did not occur before the experience. Working from this framework, we provide an in-depth analysis of two examples, fear conditioning and food-based appetitive conditioning, which include a description of the only partially overlapping neural circuitry of each. We also describe how these circuits promote the basic characteristics that define Pavlovian conditioning, such as error-correction-driven regulation of learning.

Investigating habits: strategies, technologies and models

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying action-learning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look “under the hood” of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

Effect of fenfluramine on reinstatement of food seeking in female and male rats: implications for the predictive validity of the reinstatement model

RATIONALE AND OBJECTIVES

Relapse to old unhealthy eating habits while dieting is often provoked by stress or acute exposure to palatable foods. We adapted a rat reinstatement model, which is used to study drug relapse, to study mechanisms of relapse to palatable food seeking induced by food-pellet priming (non-contingent exposure to a small amount of food pellets) or injections of yohimbine (an alpha-2 adrenoceptor antagonist that causes stress-like responses in humans and non-humans). Here, we assessed the predictive validity of the food reinstatement model by studying the effects of fenfluramine, a serotonin releaser with known anorectic effects, on reinstatement of food seeking.

METHODS

We trained food-restricted female and male rats to lever-press for 45-mg food pellets (3-h sessions) and first assessed the effect of fenfluramine (0.75, 1.5, and 3.0 mg/kg, i.p.) on food-reinforced responding. Subsequently, we extinguished the food-reinforced responding and tested the effect of fenfluramine (1.5 and 3.0 mg/kg) on reinstatement of food seeking induced by yohimbine injections (2 mg/kg, i.p.) or pellet priming (four non-contingent pellets).

RESULTS

Fenfluramine decreased yohimbine- and pellet-priming-induced reinstatement. As expected, fenfluramine also decreased food-reinforced responding, but a control condition in which we assessed fenfluramine's effect on high-rate operant responding indicated that the drug's effect on reinstatement was not due to performance deficits.

CONCLUSIONS

The present data support the predictive validity of the food reinstatement model and suggest that this model could be used to identify medications for prevention of relapse induced by stress or acute exposure to palatable food during dietary treatments.

Amphetamine-induced enhancement of responding for conditioned reward in rats: interactions with repeated testing

RATIONALE

The mesolimbic dopamine system underlies the ability of reward-related stimuli to control operant behavior. Previous work has shown that amphetamine potentiates operant responding for conditioned rewards (CRs).

OBJECTIVES

Here, we asked whether the profile of this amphetamine-produced potentiation changes with repeated CR presentation, i.e., as the CR is being extinguished.

METHODS

Amphetamine (0-1.0 mg/kg, i.p.), administered over four daily sessions using a Latin square design, dose-dependently increased lever pressing for a 'lights-off' stimulus previously paired with food in rats.

RESULTS

The amphetamine-produced enhancement of responding for CR was significantly modulated with repeated CR exposure: it was strongest on day 1 and became less pronounced in subsequent sessions whereas the CR effect persisted. In further experiments, rats receiving LiCl devaluation of the primary reward failed to show a significant reduction in the amphetamine-produced enhancement of responding for CR.

CONCLUSIONS

The nature of the dissociable effects of amphetamine on responding for CR versus the CR effect itself remains to be elucidated.

Roles of aminergic neurons in formation and recall of associative memory in crickets

We review recent progress in the study of roles of octopaminergic (OA-ergic) and dopaminergic (DA-ergic) signaling in insect classical conditioning, focusing on our studies on crickets. Studies on olfactory learning in honey bees and fruit-flies have suggested that OA-ergic and DA-ergic neurons convey reinforcing signals of appetitive unconditioned stimulus (US) and aversive US, respectively. Our work suggested that this is applicable to olfactory, visual pattern, and color learning in crickets, indicating that this feature is ubiquitous in learning of various sensory stimuli. We also showed that aversive memory decayed much faster than did appetitive memory, and we proposed that this feature is common in insects and humans. Our study also suggested that activation of OA- or DA-ergic neurons is needed for appetitive or aversive memory recall, respectively. To account for this finding, we proposed a model in which it is assumed that two types of synaptic connections are strengthened by conditioning and are activated during memory recall, one type being connections from neurons representing conditioned stimulus (CS) to neurons inducing conditioned response and the other being connections from neurons representing CS to OA- or DA-ergic neurons representing appetitive or aversive US, respectively. The former is called stimulus–response (S–R) connection and the latter is called stimulus–stimulus (S–S) connection by theorists studying classical conditioning in vertebrates. Results of our studies using a second-order conditioning procedure supported our model. We propose that insect classical conditioning involves the formation of S–S connection and its activation for memory recall, which are often called cognitive processes.

Greater effort boosts the affective taste properties of food

Actions can create preferences, increasing the value ascribed to commodities acquired at greater cost. This behavioural finding has been observed in a variety of species; however, the causal factors underlying the phenomenon are relatively unknown. We sought to develop a behavioural platform to examine the relationship between effort and reinforcer value in mice trained under demanding or lenient schedules of reinforcement to obtain food. In the initial experiment, expenditure of effort enhanced the value of the associated food via relatively lasting changes in its hedonic attributes, promoting an acquired preference for these reinforcers when tested outside of the training environment. Moreover, otherwise neutral cues associated with those reinforcers during training similarly acquired greater reinforcing value, as assessed under conditioned reinforcement. In a separate experiment, expenditure of effort was also capable of enhancing the value of less-preferred low-caloric reinforcers. Analysis of licking microstructure revealed the basis for this increased valuation was, in part, due to increased palatability of the associated reinforcer. This change in the hedonic taste properties of the food can not only serve as a basis for preference, but also guide decision-making and foraging behaviour by coordinating a potentially adaptive repertoire of incentive motivation, goal-directed action and consumption.

Learning: from association to cognition

Since the very earliest experimental investigations of learning, tension has existed between association-based and cognitive theories. Associationism accounts for the phenomena of both conditioning and "higher" forms of learning via concepts such as excitation, inhibition, and reinforcement, whereas cognitive theories assume that learning depends on hypothesis testing, cognitive models, and propositional reasoning. Cognitive theories have received considerable impetus in regard to both human and animal learning from recent research suggesting that the key illustration of cue selection in learning, blocking, often arises from inferential reasoning. At the same time, a dichotomous view that separates noncognitive, unconscious (implicit) learning from cognitive, conscious (explicit) learning has gained favor. This review selectively describes key findings from this research, evaluates evidence for and against associative and cognitive explanatory constructs, and critically examines both the dichotomous view of learning as well as the claim that learning can occur unconsciously.

Structure learning in a sensorimotor association task

Learning is often understood as an organism's gradual acquisition of the association between a given sensory stimulus and the correct motor response. Mathematically, this corresponds to regressing a mapping between the set of observations and the set of actions. Recently, however, it has been shown both in cognitive and motor neuroscience that humans are not only able to learn particular stimulus-response mappings, but are also able to extract abstract structural invariants that facilitate generalization to novel tasks. Here we show how such structure learning can enhance facilitation in a sensorimotor association task performed by human subjects. Using regression and reinforcement learning models we show that the observed facilitation cannot be explained by these basic models of learning stimulus-response associations. We show, however, that the observed data can be explained by a hierarchical Bayesian model that performs structure learning. In line with previous results from cognitive tasks, this suggests that hierarchical Bayesian inference might provide a common framework to explain both the learning of specific stimulus-response associations and the learning of abstract structures that are shared by different task environments.

Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect

Background

In insect classical conditioning, octopamine (the invertebrate counterpart of noradrenaline) or dopamine has been suggested to mediate reinforcing properties of appetitive or aversive unconditioned stimulus, respectively. However, the roles of octopaminergic and dopaminergic neurons in memory recall have remained unclear.

Results

We studied the roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in olfactory and visual conditioning in crickets. We found that pharmacological blockade of octopamine and dopamine receptors impaired aversive memory recall and appetitive memory recall, respectively, thereby suggesting that activation of octopaminergic and dopaminergic neurons and the resulting release of octopamine and dopamine are needed for appetitive and aversive memory recall, respectively. On the basis of this finding, we propose a new model in which it is assumed that two types of synaptic connections are formed by conditioning and are activated during memory recall, one type being connections from neurons representing conditioned stimulus to neurons inducing conditioned response and the other being connections from neurons representing conditioned stimulus to octopaminergic or dopaminergic neurons representing appetitive or aversive unconditioned stimulus, respectively. The former is called 'stimulus-response connection' and the latter is called 'stimulus-stimulus connection' by theorists studying classical conditioning in higher vertebrates. Our model predicts that pharmacological blockade of octopamine or dopamine receptors during the first stage of second-order conditioning does not impair second-order conditioning, because it impairs the formation of the stimulus-response connection but not the stimulus-stimulus connection. The results of our study with a cross-modal second-order conditioning were in full accordance with this prediction.

Conclusion

We suggest that insect classical conditioning involves the formation of two kinds of memory traces, which match to stimulus-stimulus connection and stimulus-response connection. This is the first study to suggest that classical conditioning in insects involves, as does classical conditioning in higher vertebrates, the formation of stimulus-stimulus connection and its activation for memory recall, which are often called cognitive processes.

A cholinergic-dependent role for the entorhinal cortex in trace fear conditioning

Trace conditioning is considered a model of higher cognitive involvement in simple associative tasks. Studies of trace conditioning have shown that cortical areas and the hippocampal formation are required to associate events that occur at different times. However, the mechanisms that bridge the trace interval during the acquisition of trace conditioning remain unknown. In four experiments with fear conditioning in rats, we explored the involvement of the entorhinal cortex (EC) in the acquisition of fear under a trace-30 s protocol. We first determined that pretraining neurotoxic lesions of the EC selectively impaired trace-, but not delay-conditioned fear as evaluated by freezing behavior. A local cholinergic deafferentation of the EC using 192-IgG-saporin did not replicate this deficit, presumably because cholinergic interneurons were spared by the toxin. However, pretraining local blockade of EC muscarinic receptors with the M1 antagonist pirenzepine yielded a specific and dose-dependent deficit in trace-conditioned responses. The same microinjections performed after conditioning were without effect on trace fear responses. These effects of blocking M1 receptors are consistent with the notion that conditioned stimulus (CS)-elicited, acetylcholine-dependent persistent activities in the EC are needed to maintain a representation of a tone CS across the trace interval during the acquisition of trace conditioning. This function of the EC is consistent with recent views of this region as a short-term stimulus buffer.

Minimal acupuncture is not a valid placebo control in randomised controlled trials of acupuncture: a physiologist's perspective

Placebo-control of acupuncture is used to evaluate and distinguish between the specific effects and the non-specific ones. During 'true' acupuncture treatment in general, the needles are inserted into acupoints and stimulated until deqi is evoked. In contrast, during placebo acupuncture, the needles are inserted into non-acupoints and/or superficially (so-called minimal acupuncture). A sham acupuncture needle with a blunt tip may be used in placebo acupuncture. Both minimal acupuncture and the placebo acupuncture with the sham acupuncture needle touching the skin would evoke activity in cutaneous afferent nerves. This afferent nerve activity has pronounced effects on the functional connectivity in the brain resulting in a 'limbic touch response'. Clinical studies showed that both acupuncture and minimal acupuncture procedures induced significant alleviation of migraine and that both procedures were equally effective. In other conditions such as low back pain and knee osteoarthritis, acupuncture was found to be more potent than minimal acupuncture and conventional non-acupuncture treatment. It is probable that the responses to 'true' acupuncture and minimal acupuncture are dependent on the aetiology of the pain. Furthermore, patients and healthy individuals may have different responses. In this paper, we argue that minimal acupuncture is not valid as an inert placebo-control despite its conceptual brilliance.