Reward associations do not explain transitive inference performance in monkeys

Prior experience modifies acquisition trajectories via response-strategy sampling

Few studies have considered how signal detection parameters evolve during acquisition periods. We addressed this gap by training mice with differential prior experience in a conditional discrimination, auditory signal detection task. Naïve mice, mice given separate experience with each of the later correct choice options (Correct Choice Response Transfer, CCRT), and mice experienced in conditional discriminations (Conditional Discrimination Transfer, CDT) were trained to detect the presence or absence of a tone in white noise. We analyzed data assuming a two-period model of acquisition: a pre-solution and solution period (Heinemann EG (1983) in The Presolution period and the detection of statistical associations. In: Quantitative analyses of behavior: discrimination processes, vol. 4, pp. 21-36). Ballinger. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.536.1978andrep=rep1andtype=pdf ). The pre-solution period was characterized by a selective sampling of biased response strategies until adoption of a conditional responding strategy in the solution period. Correspondingly, discriminability remained low until the solution period; criterion took excursions reflecting response-strategy sampling. Prior experience affected the length and composition of the pre-solution period. Whereas CCRT and CDT mice had shorter pre-solution periods than naïve mice, CDT and Naïve mice developed substantial criterion biases and acquired asymptotic discriminability faster than CCRT mice. To explain these data, we propose a learning model in which mice selectively sample and test different response-strategies and corresponding task structures until they exit the pre-solution period. Upon exit, mice adopt the conditional responding strategy and task structure, with action values updated via inference and generalization from the other task structures. Simulations of representative mouse data illustrate the viability of this model.

Resolving the associative learning paradox by category learning in pigeons

A wealth of evidence indicates that humans can engage two types of mechanisms to solve category-learning tasks: declarative mechanisms, which involve forming and testing verbalizable decision rules, and associative mechanisms, which involve gradually linking stimuli to appropriate behavioral responses.1,2,3 In contrast to declarative mechanisms, associative mechanisms have received surprisingly little attention in the broader category-learning literature. Although various forms of associatively driven artificial intelligence (AI) have matched-and even surpassed-humans' performance on several challenging problems,3,4,5,6 associative learning is routinely dismissed as being too simple to power the impressive cognitive achievements of both humans and non-human species.6,7,8,9 Here, we attempt to resolve this paradox by demonstrating that pigeons-which appear to rely solely on associative learning mechanisms in several tasks that promote declarative rule use by humans3,10,11,12-succeed at learning a novel, highly demanding category structure that ought to hinder declarative rule use: the sectioned-rings task. Our findings highlight the power and flexibility that associative mechanisms afford in the realm of category learning.

Hippocampal and medial prefrontal cortices encode structural task representations following progressive and interleaved training schedules

Memory generalisations may be underpinned by either encoding- or retrieval-based generalisation mechanisms and different training schedules may bias some learners to favour one of these mechanisms over the other. We used a transitive inference task to investigate whether generalisation is influenced by progressive vs randomly interleaved training, and overnight consolidation. On consecutive days, participants learnt pairwise discriminations from two transitive hierarchies before being tested during fMRI. Inference performance was consistently better following progressive training, and for pairs further apart in the transitive hierarchy. BOLD pattern similarity correlated with hierarchical distances in the left hippocampus (HIP) and medial prefrontal cortex (MPFC) following both training schedules. These results are consistent with the use of structural representations that directly encode hierarchical relationships between task features. However, such effects were only observed in the MPFC for recently learnt relationships. Furthermore, the MPFC appeared to maintain structural representations in participants who performed at chance on the inference task. We conclude that humans preferentially employ encoding-based mechanisms to store map-like relational codes that can be used for memory generalisation. These codes are expressed in the HIP and MPFC following both progressive and interleaved training but are not sufficient for accurate inference.

Superstitious learning of abstract order from random reinforcement

Humans and other animals often infer spurious associations among unrelated events. However, such superstitious learning is usually accounted for by conditioned associations, raising the question of whether an animal could develop more complex cognitive structures independent of reinforcement. Here, we tasked monkeys with discovering the serial order of two pictorial sets: a "learnable" set in which the stimuli were implicitly ordered and monkeys were rewarded for choosing the higher-rank stimulus and an "unlearnable" set in which stimuli were unordered and feedback was random regardless of the choice. We replicated prior results that monkeys reliably learned the implicit order of the learnable set. Surprisingly, the monkeys behaved as though some ordering also existed in the unlearnable set, showing consistent choice preference that transferred to novel untrained pairs in this set, even under a preference-discouraging reward schedule that gave rewards more frequently to the stimulus that was selected less often. In simulations, a model-free reinforcement learning algorithm (Q-learning) displayed a degree of consistent ordering among the unlearnable set but, unlike the monkeys, failed to do so under the preference-discouraging reward schedule. Our results suggest that monkeys infer abstract structures from objectively random events using heuristics that extend beyond stimulus-outcome conditional learning to more cognitive model-based learning mechanisms.

Flexible auditory training, psychophysics, and enrichment of common marmosets with an automated, touchscreen-based system

Devising new and more efficient protocols to analyze the phenotypes of non-human primates, as well as their complex nervous systems, is rapidly becoming of paramount importance. This is because with genome-editing techniques, recently adopted to non-human primates, new animal models for fundamental and translational research have been established. One aspect in particular, namely cognitive hearing, has been difficult to assess compared to visual cognition. To address this, we devised autonomous, standardized, and unsupervised training and testing of auditory capabilities of common marmosets with a cage-based standalone, wireless system. All marmosets tested voluntarily operated the device on a daily basis and went from naïve to experienced at their own pace and with ease. Through a series of experiments, here we show, that animals autonomously learn to associate sounds with images; to flexibly discriminate sounds, and to detect sounds of varying loudness. The developed platform and training principles combine in-cage training of common marmosets for cognitive and psychoacoustic assessment with an enriched environment that does not rely on dietary restriction or social separation, in compliance with the 3Rs principle.

The authors present a cage-based stand-alone platform for autonomous, standardized, and unsupervised training and testing of visuo-auditory-cued behaviours of common marmosets. The experiments do not require dietary restriction or social separation.

Influence of Rule- and Reward-based Strategies on Inferences of Serial Order by Monkeys

Knowledge of transitive relationships between items can contribute to learning the order of a set of stimuli from pairwise comparisons. However, cognitive mechanisms of transitive inferences based on rank order remain unclear, as are relative contributions of reward associations and rule-based inference. To explore these issues, we created a conflict between rule- and reward-based learning during a serial ordering task. Rhesus macaques learned two lists, each containing five stimuli that were trained exclusively with adjacent pairs. Selection of the higher-ranked item resulted in rewards. “Small reward” lists yielded two drops of fluid reward, whereas “large reward” lists yielded five drops. Following training of adjacent pairs, monkeys were tested on novels pairs. One item was selected from each list, such that a ranking rule could conflict with preferences for large rewards. Differences between the corresponding reward magnitudes had a strong influence on accuracy, but we also observed a symbolic distance effect. That provided evidence of a rule-based influence on decisions. RT comparisons suggested a conflict between rule- and reward-based processes. We conclude that performance reflects the contributions of two strategies and that a model-based strategy is employed in the face of a strong countervailing reward incentive.

Transitive inference after minimal training in rhesus macaques (Macaca mulatta)

Rhesus macaques, when trained for several hundred trials on adjacent items in an ordered list (e.g., A > B, B > C, C > D), are able to make accurate transitive inferences (TI) about previously untrained pairs (e.g., A > C, B > D). How that learning unfolds during training, however, is not well understood. We sought to measure the relationship between the amount of TI training and the resulting response accuracy in 4 rhesus macaques using seven-item lists. The training conditions included the absolute minimal case of presenting each of the six adjacent pairs only once prior to testing. We also tested transfer to nonadjacent pairs with 24 and 114 training trials. Because performance during and after small amounts of training is expected to be near chance levels, we developed a descriptive statistical model to estimate potentially subtle learning effects in the presence of much larger random response variability and systematic bias. These results suggest that subjects learned serial order in an incremental fashion. Thus, rather than performing transitive inference by a logical process, serial learning in rhesus macaques proceeds in a manner more akin to a statistical inference, with an initial uncertainty about list position that gradually becomes more accurate as evidence accumulates. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Positional inference in rhesus macaques

Understanding how organisms make transitive inferences is critical to understanding their general ability to learn serial relationships. In this context, transitive inference (TI) can be understood as a specific heuristic that applies broadly to many different serial learning tasks, which have been the focus of hundreds of studies involving dozens of species. In the present study, monkeys learned the order of 7-item lists of photographic stimuli by trial and error, and were then tested on "derived" lists. These derived test lists combined stimuli from multiple training lists in ambiguous ways, sometimes changing their order relative to training. We found that subjects displayed strong preferences when presented with novel test pairs, even when those pairs were drawn from different training lists. These preferences were helpful when test pairs had an ordering congruent with their ranks during training, but yielded consistently below-chance performance when pairs had an incongruent order relative to training. This behavior can be explained by the joint contributions of transitive inference and another heuristic that we refer to as "positional inference." Positional inferences play a complementary role to transitive inferences in facilitating choices between novel pairs of stimuli. The theoretical framework that best explains both transitive and positional inferences is a spatial model that represents both the position of each stimulus and its uncertainty. A computational implementation of this framework yields accurate predictions about both correct responses and errors on derived lists.

Safety evaluation of a clinical focused ultrasound system for neuronavigation guided blood-brain barrier opening in non-human primates

An emerging approach with potential in improving the treatment of neurodegenerative diseases and brain tumors is the use of focused ultrasound (FUS) to bypass the blood-brain barrier (BBB) in a non-invasive and localized manner. A large body of pre-clinical work has paved the way for the gradual clinical implementation of FUS-induced BBB opening. Even though the safety profile of FUS treatments in rodents has been extensively studied, the histological and behavioral effects of clinically relevant BBB opening in large animals are relatively understudied. Here, we examine the histological and behavioral safety profile following localized BBB opening in non-human primates (NHPs), using a neuronavigation-guided clinical system prototype. We show that FUS treatment triggers a short-lived immune response within the targeted region without exacerbating the touch accuracy or reaction time in visual-motor cognitive tasks. Our experiments were designed using a multiple-case-study approach, in order to maximize the acquired data and support translation of the FUS system into human studies. Four NHPs underwent a single session of FUS-mediated BBB opening in the prefrontal cortex. Two NHPs were treated bilaterally at different pressures, sacrificed on day 2 and 18 post-FUS, respectively, and their brains were histologically processed. In separate experiments, two NHPs that were earlier trained in a behavioral task were exposed to FUS unilaterally, and their performance was tracked for at least 3 weeks after BBB opening. An increased microglia density around blood vessels was detected on day 2, but was resolved by day 18. We also detected signs of enhanced immature neuron presence within areas that underwent BBB opening, compared to regions with an intact BBB, confirming previous rodent studies. Logistic regression analysis showed that the NHP cognitive performance did not deteriorate following BBB opening. These preliminary results demonstrate that neuronavigation-guided FUS with a single-element transducer is a non-invasive method capable of reversibly opening the BBB, without substantial histological or behavioral impact in an animal model closely resembling humans. Future work should confirm the observations of this multiple-case-study work across animals, species and tasks.

Absolute and relative knowledge of ordinal position on implied lists

Does serial learning result in specific associations between pairs of items, or does it result in a cognitive map based on relations of all items? In 2 experiments, we trained human participants to learn various lists of photographic images. We then tested the participants on new lists of photographic images. These new lists were constructed by selecting only 1 image from each list learned during training. In Experiment 1, participants were trained to choose the earlier (experimenter defined) item when presented with adjacent pairs of items on each of 5 different 5-item lists. Participants were then tested on derived lists, in which each item retained its original ordinal position, even though each of the presented pairs was novel. Participants performed above chance on all of the derived lists. In Experiment 2, a different group of participants received the same training as those of Experiment 1, but the ordinal positions of items were systematically changed on each derived list. The response accuracy for Experiment 2 varied inversely with the degree to which an item's original ordinal position was changed. These results can be explained by a model in which participants learned to make both positional inferences about the absolute rank of each stimulus, and transitive inferences about the relative ranks of pairs of stimuli. These inferences enhanced response accuracy when ordinal position was maintained, but not when it was changed. Our results demonstrate quantitatively that, in addition to item-item associations that participants acquire while learning a list of arbitrary items, they form a cognitive map that represents both experienced and inferred relationships. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Associative models fail to characterize transitive inference performance in rhesus monkeys (Macaca mulatta)

It has been suggested that non-verbal transitive inference (if A > B and B > C, then A > C) can be accounted for by associative models. However, little is known about the applicability of such models to primate data. In Experiment 1, we tested the fit of two associative models to primate data from both sequential training, in which the training pairs were presented in a backward order, and simultaneous training, in which all training pairs are presented intermixed from the beginning. We found that the models provided an equally poor fit for both sequential and simultaneous training presentations, contrary to the case with data from pigeons. The models were also unable to predict the robust symbolic distance effects characteristic of primate transitive choices. In Experiment 2, we used the models to fit a list-linking design in which two seven-item transitive lists were first trained independently (A > B…. > F > G and H > I …. > M > N) then combined via a linking pair (G+ H-) into a single, 14-item list. The model produced accurate predictions for between-list pairs, but did not predict transitive responses for within-list pairs from list 2. Overall, our results support research indicating that associative strength does not adequately account for the behavior of primates in transitive inference tasks. The results also suggest that transitive choices may result from different processes, or different weighting of multiple processes, across species.

The science of belief: A progress report

The empirical study of belief is emerging at a rapid clip, uniting work from all corners of cognitive science. Reliance on belief in understanding and predicting behavior is widespread. Examples can be found, inter alia, in the placebo, attribution theory, theory of mind, and comparative psychological literatures. Research on belief also provides evidence for robust generalizations, including about how we fix, store, and change our beliefs. Evidence supports the existence of a Spinozan system of belief fixation: one that is automatic and independent of belief rejection. Independent research supports the existence of a system of fragmented belief storage: one that relies on large numbers of causally isolated, context-sensitive stores of belief in memory. Finally, empirical and observational data support at least two systems of belief change. One system adheres, mostly, to epistemological norms of updating; the other, the psychological immune system, functions to guard our most centrally held beliefs from potential inconsistency with newly formed beliefs. Refining our understanding of these systems can shed light on pressing real-world issues, such as how fake news, propaganda, and brainwashing exploit our psychology of belief, and how best to construct our modern informational world. This article is categorized under: Psychology > Reasoning and Decision Making Philosophy > Knowledge and Belief Philosophy > Foundations of Cognitive Science.

Learned Representation of Implied Serial Order in Posterior Parietal Cortex

Monkeys can learn the implied ranking of pairs of images drawn from an ordered set, despite never seeing all of the images simultaneously and without explicit spatial or temporal cues. We recorded the activity of posterior parietal cortex (including lateral intraparietal area LIP) neurons while monkeys learned 7-item transitive inference (TI) lists with 2 items presented on each trial. Behavior and neuronal activity were significantly influenced by the ordinal relationship of the stimulus pairs, specifically symbolic distance (the difference in rank) and joint rank (the sum of the ranks). Symbolic distance strongly predicted decision accuracy and learning rate. An effect of joint rank on performance was found nested within the symbolic distance effect. Across the population of neurons, there was significant modulation of firing correlated with the relative ranks of the two stimuli presented on each trial. Neurons exhibited selectivity for stimulus rank during learning, but not before or after. The observed behavior is poorly explained by associative or reward mechanisms, and appears more consistent with a mental workspace model in which implied serial order is mapped within a spatial framework. The neural data suggest that posterior parietal cortex supports serial learning by representing information about the ordinal relationship of the stimuli presented during a given trial.