Effort-Based Reward (EBR) training enhances neurobiological efficiency in a problem-solving task: Insights for depression therapies

Rewarded Maze Training Increases Approach Behavior in Rats Through Neurogenesis-Dependent Growth of Ventral Hippocampus-Prelimbic Circuits

Background

Learning complex navigation routes increases hippocampal volume in humans, but it is not clear whether this growth impacts behaviors outside the learning situation or what cellular mechanisms are involved.

Methods

We trained rats with pharmacogenetic suppression of adult neurogenesis and littermate controls in 3 mazes over 3 weeks and tested novelty approach behavior several days after maze exposure. We then measured hippocampus and prelimbic cortex volumes using magnetic resonance imaging and assessed neuronal and astrocyte morphology. Finally, we investigated the activation and behavioral role of the ventral CA1 (vCA1)-to-prelimbic pathway using immediate-early genes and DREADDs (designer receptors exclusively activated by designer drugs).

Results

Maze training led to volume increase of both the vCA1 region of the hippocampus and the prelimbic region of the neocortex compared with rats that followed fixed paths. Growth was also apparent in individual neurons and astrocytes in these 2 regions, and behavioral testing showed increased novelty approach in maze-trained rats in 2 different tests. Suppressing adult neurogenesis prevented the effects on structure and approach behavior after maze training without affecting maze learning itself. The vCA1 neurons projecting to the prelimbic area were more activated by novelty in maze-trained animals, and suppression of this pathway decreased approach behavior.

Conclusions

Rewarded navigational learning experiences induce volumetric and morphologic growth in the vCA1 and prelimbic cortex and enhance activation of the circuit connecting these 2 regions. Both the structural and behavioral effects of maze training require ongoing adult neurogenesis, suggesting a role for new neurons in experience-driven increases in novelty exploration.

Wild brains: The value of neuroethological approaches in preclinical behavioral neuroscience animal models

For three decades, IBNS has provided a forum for the dissemination of behavioral neuroscience research, broadly defined. Throughout this time, research presented at the annual meetings has reflected representative trends in the field with an emphasis on relevant preclinical animal models. From its inception, IBNS has contributed to my professional development and evolving research interests. Unsurprisingly, throughout the three decades of its existence, IBNS annual programs have reflected research trends that have been thoughtfully evaluated, challenged, and, in some cases, recalibrated. An emphasis in my lab, for example, has slowly navigated toward the inclusion of more diverse species (e.g., nonhuman primate models, wild rats, wild and captive raccoons) assessed in settings that reflect more ethological relevance than typically observed in traditional laboratory settings. Consequently, my research interests are pivoting from laboratory animal model exclusive (L.A.M.E.) endeavors to more natural, diverse, ethoexperimental approaches. As progress toward translational findings for psychiatric and neurological conditions is considered, it is recommended that researchers remain open to nontraditional methodological approaches that incorporate diverse animal models and assessments to inform laboratory-generated findings.

Influence of Effort-based Reward Training on Neuroadaptive Cognitive Responses: Implications for Preclinical Behavioral Approaches for Depressive Symptoms

Despite the presence of multiple pharmacotherapeutic options, incidence rates for depressive disorders continue to rise. Nonpharmacological approaches (e.g., cognitive and behavioral therapies) exhibit encouraging efficacy rates; however, a lack of preclinical models has prevented progress in the identification of relevant neurobiological mechanisms of these approaches. Accordingly, the effort-based reward (EBR) preclinical model exposes rats to response-outcome (R-O) contingencies and provides an opportunity to investigate behavioral clinical approaches. In the current study, male and female rats were assigned to either an EBR contingent- or noncontingent-trained group and exposed to 7 weeks of training. Neuroadaptive cognitive responses were assessed in a cognitive uncertainty task (UT) and an object pattern separation task (OPST). Although no significant effects of EBR were observed in the UT, EBR contingent-trained rats approached the novel panel in the most difficult trial of the OPST faster than the noncontingent-trained group. Additionally, female EBR contingent-trained rats exhibited increased engagement with the novel stimulus panel across all trials. Examination of brain-derived neurotrophic factor (BDNF) in the lateral habenula (LHb), a putative neurobiological target for depressive symptoms, revealed lower BDNF immunoreactivity in EBR contingent-trained rats. Females in both training groups exhibited higher dehydroepiandrosterone/cortisol (DHEA/CORT) ratios, suggesting, along with the increased engagement with novel stimulus panels, that female rats may be more responsive to EBR contingency training than males. Together, these results suggest that EBR contingency training offers promise as a preclinical rat model for behavioral therapeutic interventions for depressive symptoms leading to a clearer understanding of putative neurobiological mechanisms.

Back to nature: herbal treatment, environmental enrichment, and social play can protect against unpredictable chronic stress in Long-Evans rats (Rattus norvegicus)

The importance of integrative biobehavioral responses to complex challenges cannot be overlooked. In this study, the synergetic effects of icariin (a flavonoid present in the plant Epimedium brevicornum), natural enrichment (NaEn), and play behavior were investigated. Rats (n = 60) were assigned to standard housing or NaEn; these two groups were subsequently divided into controls, rats receiving icariin treatments, and rats receiving icariin and allowed to play with an individual from another cage. All rats were exposed to unpredictable mild stressors for 4 weeks. At the end of the treatment, a Forced Swim Task (FST) was conducted to assess emotional regulation during an inescapable acute challenge. Biological samples were collected weekly and before and after the FST to monitor endocrine changes. Corticosterone (CORT), dehydroepiandrosterone (DHEA), and testosterone (T) were assayed. We found that icariin had a significant effect on DHEA/CORT ratios and T levels. NaEn also had a significant effect on both CORT and DHEA, but not on T levels. Play did not appear to be significantly related to the endocrine changes. The strongest positive effects on emotional resilience were observed in NaEn rats that also received icariin. Our results confirmed that using multiple channels to stimulate adaptive responses can be effective in increasing the ability of an organism to face uncertainty. Considering how quickly our life can change due to unpredictable events, our data is instrumental to a better comprehension of the many aspects of integrative biobehavioral responses.

Neuroplasticity Improves Bipolar Disorder: A Review

Bipolar disorder (BD) is known for impairments in neurotrophic and neuroprotective processes, which translate into emotional and cognitive deficits affecting various brain regions. Using its neuroplastic properties, lithium, thus far, is the mood stabilizer used to amend the pathophysiological imbalance in BD. Neuroplasticity has gained massive popularity in the research department in the past decade, yet it lacks direct effort in changing the protocol through which physicians treat BD. Physical activity alongside cognitive therapy is theorized to produce long-term changes in the executive control network due to the assimilation of new neurons, amendment of emotional lability through hippocampal neurogenesis, and strengthening the stability of frontosubcortical and prefrontolimbic brain regions via neurogenesis. This review aims to provide an incentive for utilizing neuroplastic mechanisms concerning impairments dispensed by BD.

Effort-reward balance and work motivation in rats: Effects of context and order of experience

Being motivated means exerting effort toward a goal. The 'law of least work' emphasizes a preference for exerting relatively less effort. The law crosses boundaries among species and between physical and mental work. Organisms should be highly sensitive to shifts in effort-reward balance (ERB) in order to make optimal choices. We used a free operant-foraging task to investigate changes in ERB on choice between options requiring more or less effort. Results showed a consistent preference for the option with less effort and insensitivity to shifts in ERB. A second aim explored the influence of order of experience on effort choice. Choice for the more effortful option significantly increased after experiencing an equal effort-reward relationship during the initial free operant-foraging session. This relative increase in choice for the effortful option persisted even after effort-reward imbalance. The findings highlight the importance of contextual factors such as order of experience when examining the impact of shifting effort-reward associations. Instead of ignoring or reducing order effects, the sequence of experience (e.g. for shifts in ERB) could be manipulated to enhance or reduce value of outcomes or effort itself.

In search of optimal resilience ratios: Differential influences of neurobehavioral factors contributing to stress-resilience spectra

The ability to adapt to stressful circumstances, known as emotional resilience, is a key factor in the maintenance of mental health. Several individual biomarkers of the stress response (e.g., corticosterone) that influence an animal's position along the continuum that ranges from adaptive allostasis to maladaptive allostatic load have been identified. Extending beyond specific biomarkers of stress responses, however, it is also important to consider stress-related responses relative to other relevant responses for a thorough understanding of the underpinnings of adaptive allostasis. In this review, behavioral, neurobiological, developmental and genomic variables are considered in the context of emotional resilience [e.g., explore/exploit behavioral tendencies; DHEA/CORT ratios and relative proportions of protein-coding/nonprotein-coding (transposable) genomic elements]. As complex and multifaceted relationships between pertinent allostasis biomediators are identified, translational applications for optimal resilience are more likely to emerge as effective therapeutic strategies.

Optimizing brain performance: Identifying mechanisms of adaptive neurobiological plasticity

Although neuroscience research has debunked the late 19^th century claims suggesting that large portions of the brain are typically unused, recent evidence indicates that an enhanced understanding of neural plasticity may lead to greater insights related to the functional capacity of brains. Continuous and real-time neural modifications in concert with dynamic environmental contexts provide opportunities for targeted interventions for maintaining healthy brain functions throughout the lifespan. Neural design, however, is far from simplistic, requiring close consideration of context-specific and other relevant variables from both species and individual perspectives to determine the functional gains from increased and decreased markers of neuroplasticity. Caution must be taken in the interpretation of any measurable change in neurobiological responses or behavioral outcomes, as definitions of optimal functions are extremely complex. Even so, current behavioral neuroscience approaches offer unique opportunities to evaluate adaptive functions of various neural responses in an attempt to enhance the functional capacity of neural systems.

Immunomodulatory Effects of Stress and Environmental Enrichment in Long-Evans Rats (Rattus norvegicus)

Stress can influence the secretion of neuroendocrine mediators, thereby exposing immune cells to altered signaling and interactions. Here we investigated the synergetic effect of stress and environmental enrichment on the immune response of Long-Evans rats. Subjects (n = 46) were assigned to 5 treatment groups: acute compared with chronic stress with or without environmental enrichment, plus an unmanipulated control group. Animals also were classified as active, passive, and flexible copers according to back-test assessment. Rats were exposed to enrichment in an open-field containing objects in different areas for 30 min 3 times each week, thus modeling the effects of a temporary increase in environmental stimuli. Animals assigned to chronic stress groups were exposed to predator sound stressors for 30 min daily, whereas animals assigned to acute stress groups were exposed once each week. After 7 wk, a dermal punch biopsy was administered to activate the immune response, after which rats were challenged through a forced swim test. Biologic samples were collected to measure corticosterone, dehydroepiandrosterone (DHEA), oxytocin, testosterone, and the cytokines IL6 and IL10. Rats exposed to chronic stress had lower DHEA:corticosterone ratios, suggesting increased allostatic load. Enrichment exposure modulated these effects, lowering overall corticosterone and testosterone levels and increasing DHEA and oxytocin levels in animals exposed to the predator sound. The immune response was decreased in rats exposed to chronic stress, but the effect of environmental enrichment helped to mitigate the negative influence on cells producing IL6. Combining acute stress and exposure to an enriched environment returned a healthier profile in terms of both immune activation and stress regulation. By using a multidimensional scaling model, we found that a combination of 'good' stress and exposure to brief sessions of enriching stimuli can reliably predict health in Long-Evans rats.

Contingency Training Alters Neurobiological Components of Emotional Resilience in Male and Female Rats

Prior research with a rat model of behavioral therapy [i.e., effort-based reward (EBR) contingency training] suggests that strengthened associations between physical effort and desired outcomes enhance neurobiological indices of resilience. In the current study, male and female Long-Evans rats were exposed to either six weeks of EBR training or noncontingent training prior to 10 days of exposure to chronic unpredictable stress (CUS). Subsequently, all animals were exposed to a problem-solving task and then trained in a spatial learning/foraging task, the Dry Land Maze (DLM). Following habituation training and test trials, rats were assessed in a probe trial that generated a prediction error (cognitive uncertainty). Results indicated that, during CUS exposure, contingency-training enhanced dehydroepiandrosterone/corticosterone ratios (consistent with healthier stress responses), especially in male rats. Additionally, contingency training increased exploratory behaviors in the probe trial as well as differentially influenced on-task problem-solving performance in males and females. Following the probe trial, brains were exposed to histological analyses to determine the effects of sex and contingency training on various neurobiological markers. Contingency training decreased BDNF-immunoreactivity (ir) in the hippocampus CA1 and lateral habenula, implicating differential neuroplasticity responses in the training groups. Further, coordinated fos-ir activation in areas associated with emotional resilience (i.e., motivation-regulation) was observed in contingent-trained animals. In sum, the current findings confirm that behavioral training is associated with neurobiological markers of emotional resilience; however, further assessments are necessary to more accurately determine the therapeutic potential for the EBR contingency training model.

Profiling coping strategies in male and female rats: Potential neurobehavioral markers of increased resilience to depressive symptoms

Coping strategies have been associated with differential stress responsivity, perhaps providing a valuable neurobiological marker for susceptibility to the emergence of depressogenic symptoms or vulnerability to other anxiety-related disorders. Rats profiled with a flexible coping phenotype, for example, exhibit increased neurobiological markers of emotional regulation compared to active and passive copers (Bardi et al., 2012; Lambert et al., 2014). In the current study, responses of male and female rats to prediction errors in a spatial foraging task (dry land maze; DLM) were examined after animals were exposed to chronic unpredictable stress (CUS). Brains were processed following the DLM training/assessment for fos-activation patterns and several measures of neuroplasticity in relevant areas. Behavioral responses observed during both the CUS and DLM phases of testing suggested that males and females employ different means of gathering information such as increased ambulatory exploration in males and rear responses in females. Fecal samples collected during baseline and following CUS swim exposure revealed higher corticosterone (CORT) in active copers, whereas flexible copers had higher dehydroepiandrosterone (DHEA) and DHEA/CORT ratios, both indications of enhanced emotional regulation. Focusing on the neural analysis, flexible copers exhibited fewer fos-immunoreactive cells in the basolateral amygdala and a trend toward lower activation in the insula while encountering the prediction error associated with the DLM probe trial. Coping profiles also differentially influenced markers of neuroplasticity; specifically, flexible copers exhibited higher levels nestin-immunoreactivity (ir). Further, less hippocampal glucocorticoid receptor-ir was observed in the flexible copers than the active and passive copers. In sum, flexible coping rats exhibited evidence of emotional resilience as indicated by several neurobiological measures; however, despite increased rates of depression and related symptoms reported in human females, sex effects weren't as pervasive as coping strategy profiles in the analysis of neurobiological markers employed in the current study.

Paper or Plastic? Exploring the Effects of Natural Enrichment on Behavioural and Neuroendocrine Responses in Long-Evans Rats

Enriched environments are beneficial to neurobiological development; specifically, rodents exposed to complex, rather than standard laboratory, environments exhibit evidence of neuroplasticity and enhanced cognitive performance. In the present study, the nature of elements placed in the complex environment was investigated. Accordingly, rats (n = 8 per group) were housed either in a natural environment characterised by stimuli such as dirt and rocks, an artificial environment characterised by plastic toys and synthetic nesting materials, a natural/artificial environment characterised by a combination of artificial and natural stimuli or a laboratory standard environment characterised by no enrichment stimuli. Following exposure to emotional and cognitive behavioural tasks, including a cricket hunting task, a novel object preference task and a forced swim task, brains were processed for glial fibrillary acidic protein (GFAP)-, neuronal nuclei (NeuN)- and brain-derived neurotrophic factor (BDNF) immunoreactivity. Baseline and stress foecal samples were collected to assess corticosterone (CORT) and dehydroepiandrosterone (DHEA). Natural environment animals exhibited shorter diving latencies and increased diving frequencies in the second forced swimming task, along with higher DHEA/CORT ratios, and higher GFAP immunoreactivity in the hippocampus. The type of environmental enrichment did not influence levels of BDNF immunoreactivity in the CA1, CA3 and dentate gyrus of the hippocampus; however, natural environment animals exhibited higher levels of NeuN immunoreactivity in the retrosplenial cortex, an area involved in spatial memory and other cognitive functions. These results suggest that, in addition to enhancing behavioural and endocrinological variables associated with resilience, exposure to natural stimuli might alter plasticity in brain areas associated with cortical processing and learning.

Brains in the city: Neurobiological effects of urbanization

With a majority of humans now living in cities, strategic research is necessary to elucidate the impact of this evolutionarily unfamiliar habitat on neural functions and well-being. In this review, both rodent and human models are considered in the evaluation of the changing physical and social landscapes associated with urban dwellings. Animal models assessing increased exposure to artificial physical elements characteristic of urban settings, as well as exposure to unnatural sources of light for extended durations, are reviewed. In both cases, increased biomarkers of mental illnesses such as major depression have been observed. Additionally, applied human research emphasizing the emotional impact of environmental threats associated with urban habitats is considered. Subjects evaluated in an inner-city hospital reveal the impact of combined specific genetic vulnerabilities and heightened stress responses in the expression of posttraumatic stress disorder. Finally, algorithm-based models of cities have been developed utilizing population-level analyses to identify risk factors for psychiatric illness. Although complex, the use of multiple research approaches, as described herein, results in an enhanced understanding of urbanization and its far-reaching effects--confirming the importance of continued research directed toward the identification of putative risk factors associated with psychiatric illness in urban settings.

Contingency-based emotional resilience: effort-based reward training and flexible coping lead to adaptive responses to uncertainty in male rats

Emotional resilience enhances an animal's ability to maintain physiological allostasis and adaptive responses in the midst of challenges ranging from cognitive uncertainty to chronic stress. In the current study, neurobiological factors related to strategic responses to uncertainty produced by prediction errors were investigated by initially profiling male rats as passive, active or flexible copers (n = 12 each group) and assigning to either a contingency-trained or non-contingency trained group. Animals were subsequently trained in a spatial learning task so that problem solving strategies in the final probe task, as well-various biomarkers of brain activation and plasticity in brain areas associated with cognition and emotional regulation, could be assessed. Additionally, fecal samples were collected to further determine markers of stress responsivity and emotional resilience. Results indicated that contingency-trained rats exhibited more adaptive responses in the probe trial (e.g., fewer interrupted grooming sequences and more targeted search strategies) than the noncontingent-trained rats; additionally, increased DHEA/CORT ratios were observed in the contingent-trained animals. Diminished activation of the habenula (i.e., fos-immunoreactivity) was correlated with resilience factors such as increased levels of DHEA metabolites during cognitive training. Of the three coping profiles, flexible copers exhibited enhanced neuroplasticity (i.e., increased dentate gyrus doublecortin-immunoreactivity) compared to the more consistently responding active and passive copers. Thus, in the current study, contingency training via effort-based reward (EBR) training, enhanced by a flexible coping style, provided neurobiological resilience and adaptive responses to prediction errors in the final probe trial. These findings have implications for psychiatric illnesses that are influenced by altered stress responses and decision-making abilities (e.g., depression).