Probing relationships between reinforcement learning and simple behavioral strategies to understand probabilistic reward learning

Impairments in operant probabilistic reversal learning in BTBR T+tf/J male and female mice

Autism spectrum disorder (ASD) presents with two core symptoms, impairments in social communication and the presence of restricted, repetitive behaviors (RRBs). RRBs are commonly linked to a lack of behavioral flexibility, having a significant negative impact on daily functioning for ASD individuals and their caregivers. Commonly utilized tests of behavioral flexibility employ a traditional deterministic reward approach where choices are either correct or incorrect throughout testing. The incorporation of an 80 %/20 % probabilistic reversal learning paradigm allows for the examination of flexible behavior in the face of variable outcomes, a more ecologically relevant approach. In this task, one specific choice is reinforced on 80 % of trials and the opposite or incorrect choice is reinforced on 20% of trials. Upon successful discrimination learning, the reward contingencies are switched so that the correct choice is now reinforced 20% of trials and the incorrect choice reinforced 80 % of trials, making it the new optimal choice. This translational task has been previously validated in ASD individuals and animal models of ASD, including the BTBR T + tf/J strain. Our lab and others have demonstrated that male BTBR T + tf/J mice have higher expression of lower order RRBs and display deficits in spatial probabilistic reversal learning tasks using a T-maze apparatus. Instead, female BTBR mice do not express the same lower order RRBs and results are mixed on whether females demonstrate similar probabilistic reversal learning deficits in a T-maze. Therefore, the purpose of this study was to assess the validity of using operant chambers to examine BTBR mouse performance on an 80 %/20 % probabilistic reversal learning task and to also examine the sex-specific differences in reversal learning performance in both mouse strains. Results show that BTBR mice, irrespective of sex, were impaired on the reversal learning, requiring more days and trials to reach reversal criterion compared to C57BL/6J mice. These results parallel previous strain findings in the spatial dependent T-maze task in male mice. Further error analysis showed that the impaired behavioral flexibility was due to elevated regressive errors and lose-shift probabilities. BTBR mice have more difficulty maintaining new choice patterns compared to C57BL/6J mice, which supports findings utilizing a spatial T-maze task. Together, these findings further support the use of the BTBR mouse as preclinical models of ASD due to their validity as an ASD model.

Switching costs in stochastic environments drive the emergence of matching behaviour in animal decision-making through the promotion of reward learning strategies

A principle of choice in animal decision-making named probability matching (PM) has long been detected in animals, and can arise from different decision-making strategies. Little is known about how environmental stochasticity may influence the switching time of these different decision-making strategies. Here we address this problem using a combination of behavioral and theoretical approaches, and show, that although a simple Win-Stay-Loss-Shift (WSLS) strategy can generate PM in binary-choice tasks theoretically, budgerigars (Melopsittacus undulates) actually apply a range of sub-tactics more often when they are expected to make more accurate decisions. Surprisingly, budgerigars did not get more rewards than would be predicted when adopting a WSLS strategy, and their decisions also exhibited PM. Instead, budgerigars followed a learning strategy based on reward history, which potentially benefits individuals indirectly from paying lower switching costs. Furthermore, our data suggest that more stochastic environments may promote reward learning through significantly less switching. We suggest that switching costs driven by the stochasticity of an environmental niche can potentially represent an important selection pressure associated with decision-making that may play a key role in driving the evolution of complex cognition in animals.