Effects of reward size and context on learning in macaque monkeys

Neurons in the monkey frontopolar cortex encode learning stage and goal during a fast learning task

The frontopolar cortex (FPC) is, to date, one of the least understood regions of the prefrontal cortex. The current understanding of its function suggests that it plays a role in the control of exploratory behaviors by coordinating the activities of other prefrontal cortex areas involved in decision-making and exploiting actions based on their outcomes. Based on this hypothesis, FPC would drive fast-learning processes through a valuation of the different alternatives. In our study, we used a modified version of a well-known paradigm, the object-in-place (OIP) task, to test this hypothesis in electrophysiology. This paradigm is designed to maximize learning, enabling monkeys to learn in one trial, which is an ability specifically impaired after a lesion of the FPC. We showed that FPC neurons presented an extremely specific pattern of activity by representing the learning stage, exploration versus exploitation, and the goal of the action. However, our results do not support the hypothesis that neurons in the frontal pole compute an evaluation of different alternatives. Indeed, the position of the chosen target was strongly encoded at its acquisition, but the position of the unchosen target was not. Once learned, this representation was also found at the problem presentation, suggesting a monitoring activity of the synthetic goal preceding its acquisition. Our results highlight important features of FPC neurons in fast-learning processes without confirming their role in the disengagement of cognitive control from the current goals.

Handling and Training of Wild Animals: Evidence and Ethics-Based Approaches and Best Practices in the Modern Zoo

There is an ethical responsibility to provide all animals living in human care with optimal and positive well-being. As animals living in zoos and aquariums frequently interact with their human caregivers as part of their daily care routines, it is both relevant and essential to consider the impact of these interactions on animal well-being. Allowing animals to have choice and control in multiple areas of their lives, such as by providing opportunities for them to voluntarily participate in their own care through, for example, positive reinforcement training, is an essential component of good animal well-being programs. This review aims to describe evidence-based approaches, ethics, and best practices in the handling and training of the many taxa held in zoos and aquariums worldwide, drawing from work in related animal care fields such as laboratories, farms, rescue, and sanctuaries. The importance of ongoing animal well-being assessments is discussed, with a particular focus on the need for continued review and refinement of processes and procedures pertaining to animal training and handling specifically. Review, enquiry, assessment, evaluation, and refinement will aim to dynamically support positive well-being for all animals.

Human interventions in a behavioural experiment for Asian Elephants (Elephas maximus)

Experiments are widely used to investigate the behaviour and cognition of animals. While the automation of experiments to avoid potential experimenter bias is sometimes possible, not all experiments can be conducted without human presence. This is particularly true for large animals in captivity, which are often managed by professional handlers. For the safety of the animals and experimenters, a handler must be present during behavioural studies with certain species. It is not always clear to what extent cues provided by handlers affect the animals, and therefore the experimental results. In this study, we investigate handler interventions during the training process for a behavioural experiment with Asian elephants (Elephas maximus) in Nepal. We show that elephant handlers (mahouts) intervened to guide elephants in performing the learning task using vocal and behavioural cues, despite experimenters requesting minimal intervention. We found that although the frequency of mahout interventions did not decrease as the training progressed, the nature of their interventions changed. We also found more non-verbal than verbal cues across the training. Our results suggest that guidance from handlers may be common in behavioural studies, and continued consideration should be put into experimental design to reduce or account for cues that animals may receive from humans. This study also emphasises the need to take into account the presence of humans in interpreting the results of animal behavioural experiments, which not only presents challenges to behavioural research, but also represents opportunities for further study.

Uncertainty-guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.

Understanding Imitation in Papio papio: The Role of Experience and the Presence of a Conspecific Demonstrator

What factors affect imitation performance? Varying theories of imitation stress the role of experience, but few studies have explicitly tested its role in imitative learning in non-human primates. We tested several predictions regarding the role of experience, conspecific presence, and action compatibility using a stimulus-response compatibility protocol. Nineteen baboons separated into two experimental groups learned to respond by targeting on a touch screen the same stimulus as their neighbor (compatible) or the opposite stimulus (incompatible). They first performed the task with a conspecific demonstrator (social phase) and then a computer demonstrator (ghost phase). After reaching a predetermined success threshold, they were then tested in an opposite compatibility condition (i.e., reversal learning conditions). Seven baboons performed at least two reversals during the social phase, and we found no significant difference between the compatible and incompatible conditions, although we noticed slightly faster response times (RTs) in the compatible condition that disappeared after the first reversal. During the ghost phase, monkeys showed difficulties in learning the incompatible condition, and the compatible condition RTs tended to be slower than during the social phase. Together, these results suggest that (a) there is no strong movement compatibility effect in our task and that (b) the presence of a demonstrator plays a role in eliciting correct responses but is not essential as has been previously shown in human studies.

Influence of Rule- and Reward-based Strategies on Inferences of Serial Order by Monkeys

Knowledge of transitive relationships between items can contribute to learning the order of a set of stimuli from pairwise comparisons. However, cognitive mechanisms of transitive inferences based on rank order remain unclear, as are relative contributions of reward associations and rule-based inference. To explore these issues, we created a conflict between rule- and reward-based learning during a serial ordering task. Rhesus macaques learned two lists, each containing five stimuli that were trained exclusively with adjacent pairs. Selection of the higher-ranked item resulted in rewards. “Small reward” lists yielded two drops of fluid reward, whereas “large reward” lists yielded five drops. Following training of adjacent pairs, monkeys were tested on novels pairs. One item was selected from each list, such that a ranking rule could conflict with preferences for large rewards. Differences between the corresponding reward magnitudes had a strong influence on accuracy, but we also observed a symbolic distance effect. That provided evidence of a rule-based influence on decisions. RT comparisons suggested a conflict between rule- and reward-based processes. We conclude that performance reflects the contributions of two strategies and that a model-based strategy is employed in the face of a strong countervailing reward incentive.

Amygdala electrical-finger-print (AmygEFP) NeuroFeedback guided by individually-tailored Trauma script for post-traumatic stress disorder: Proof-of-concept

•

Randomized clinical trial with a novel self-neuromodulation training in PTSD.

•

Demonstration of feasibility of an fMRI-informed EEG model of Amygdala modulation (AmygEFP).

•

Individually-tailored trauma-related content as the training feedback interface.

•

Results showed reduction of PTSD symptoms following AmygEFP trauma-related feedback training.

Background

Amygdala activity dysregulation plays a central role in post-traumatic stress disorder (PTSD). Hence learning to self-regulate one's amygdala activity may facilitate recovery. PTSD is further characterized by abnormal contextual processing related to the traumatic memory. Therefore, provoking the personal traumatic narrative while training amygdala down-regulation could enhance clinical efficacy. We report the results of a randomized controlled trial (NCT02544971) of a novel self-neuromodulation procedure (i.e. NeuroFeedback) for PTSD, aimed at down-regulating limbic activity while receiving feedback from an auditory script of a personal traumatic narrative. To scale-up applicability, neural activity was probed by an fMRI-informed EEG model of amygdala activity, termed Amygdala Electrical Finger-Print (AmygEFP).

Methods

Fifty-nine adults meeting DSM-5 criteria for PTSD were randomized between three groups: Trauma-script feedback interface (Trauma-NF) or Neutral feedback interface (Neutral-NF), and a control group of No-NF (to control for spontaneous recovery). Before and immediately after 15 NF training sessions patients were blindly assessed for PTSD symptoms and underwent one session of amygdala fMRI-NF for transferability testing. Follow-up clinical assessment was performed at 3- and 6-months following NF treatment.

Results

Patients in both NF groups learned to volitionally down-regulate AmygEFP signal and demonstrated a greater reduction in PTSD symptoms and improved down-regulation of the amygdala during fMRI-NF, compared to the No-NF group. The Trauma-NF group presented the largest immediate clinical improvement.

Conclusions

This proof-of-concept study indicates the feasibility of the AmygEFP-NF process-driven as a scalable intervention for PTSD and illustrates its clinical potential. Further investigation is warranted to elucidate the contribution of AmygEFP-NF beyond exposure and placebo effects.

Macaque monkeys learn and perform a non-match-to-goal task using an automated home cage training procedure

In neurophysiology, nonhuman primates represent an important model for studying the brain. Typically, monkeys are moved from their home cage to an experimental room daily, where they sit in a primate chair and interact with electronic devices. Refining this procedure would make the researchers' work easier and improve the animals' welfare. To address this issue, we used home-cage training to train two macaque monkeys in a non-match-to-goal task, where each trial required a switch from the choice made in the previous trial to obtain a reward. The monkeys were tested in two versions of the task, one in which they acted as the agent in every trial and one in which some trials were completed by a "ghost agent". We evaluated their involvement in terms of their performance and their interaction with the apparatus. Both monkeys were able to maintain a constant involvement in the task with good, stable performance within sessions in both versions of the task. Our study confirms the feasibility of home-cage training and demonstrates that even with challenging tasks, monkeys can complete a large number of trials at a high performance level, which is a prerequisite for electrophysiological studies of monkey behavior.