Latent learning, cognitive maps, and curiosity

Olfactory investigation in the home cage

We have developed a behavioral paradigm to study volitional olfactory investigation in mice over several months. We placed odor ports in the wall of a standard cage that administer a neutral odorant stimulus when a mouse pokes its nose inside. Even though animals were fed and watered ad libitum, and sampling from the port elicited no outcome other than the delivery of an odor, mice readily sampled these stimuli hundreds of times per day. This self-paced olfactory investigation persisted for weeks with only modest habituation following the first day of exposure to a given set of odorants. If an unexpected odorant stimulus was administered at the port, the sampling rate increased transiently (in the first 20 min) by an order of magnitude and remained higher than baseline throughout the subsequent day, indicating learned implicit knowledge. Thus, this system may be used to study naturalistic olfactory learning over extended time scales outside of conventional task structures.

A transdisciplinary view on curiosity beyond linguistic humans: animals, infants, and artificial intelligence

Curiosity is a core driver for life-long learning, problem-solving and decision-making. In a broad sense, curiosity is defined as the intrinsically motivated acquisition of novel information. Despite a decades-long history of curiosity research and the earliest human theories arising from studies of laboratory rodents, curiosity has mainly been considered in two camps: 'linguistic human' and 'other'. This is despite psychology being heritable, and there are many continuities in cognitive capacities across the animal kingdom. Boundary-pushing cross-disciplinary debates on curiosity are lacking, and the relative exclusion of pre-linguistic infants and non-human animals has led to a scientific impasse which more broadly impedes the development of artificially intelligent systems modelled on curiosity in natural agents. In this review, we synthesize literature across multiple disciplines that have studied curiosity in non-verbal systems. By highlighting how similar findings have been produced across the separate disciplines of animal behaviour, developmental psychology, neuroscience, and computational cognition, we discuss how this can be used to advance our understanding of curiosity. We propose, for the first time, how features of curiosity could be quantified and therefore studied more operationally across systems: across different species, developmental stages, and natural or artificial agents.

Bias in perceptual learning

Perceptual learning is commonly understood as conferring some benefit to the learner, such as allowing for the extraction of more information from the environment. However, perceptual learning can be biased in several different ways, some of which do not appear to provide such a benefit. Here we outline a systematic framework for thinking about bias in perceptual learning and discuss how several cases fit into this framework. We argue these biases are compatible with an understanding in which perceptual learning is beneficial, but that its benefits are tied to both a person's narrow interests and the training environment or domain, and so if there are changes to either of these, then benefits can turn into liabilities, though these are often temporary. This article is categorized under: Psychology > Learning Philosophy > Value Linguistics > Language Acquisition.

Control over a mixture of policies determines change of mind topology during continuous choice

Behavior is naturally organized into categorically distinct states with corresponding patterns of neural activity; how does the brain control those states? We propose that states are regulated by specific neural processes that implement meta-control that can blend simpler control processes. To test this hypothesis, we recorded from neurons in the dorsal anterior cingulate cortex (dACC) and dorsal premotor cortex (PMd) while macaques performed a continuous pursuit task with two moving prey that followed evasive strategies. We used a novel control theoretic approach to infer subjects' moment-to-moment latent control variables, which in turn dictated their blend of distinct identifiable control processes. We identified low-dimensional subspaces in neuronal responses that reflected the current strategy, the value of the pursued target, and the relative value of the two targets. The top two principal components of activity tracked changes of mind in abstract and change-type-specific formats, respectively. These results indicate that control of behavioral state reflects the interaction of brain processes found in dorsal prefrontal regions that implement a mixture over low-level control policies.

A neural mechanism for conserved value computations integrating information and rewards

Behavioral and economic theory dictate that we decide between options based on their values. However, humans and animals eagerly seek information about uncertain future rewards, even when this does not provide any objective value. This implies that decisions are made by endowing information with subjective value and integrating it with the value of extrinsic rewards, but the mechanism is unknown. Here, we show that human and monkey value judgements obey strikingly conserved computational principles during multi-attribute decisions trading off information and extrinsic reward. We then identify a neural substrate in a highly conserved ancient structure, the lateral habenula (LHb). LHb neurons signal subjective value, integrating information's value with extrinsic rewards, and the LHb predicts and causally influences ongoing decisions. Neurons in key input areas to the LHb largely signal components of these computations, not integrated value signals. Thus, our data uncover neural mechanisms of conserved computations underlying decisions to seek information about the future.

ACC neural ensemble dynamics are structured by strategy prevalence

Medial frontal cortical areas are thought to play a critical role in the brain's ability to flexibly deploy strategies that are effective in complex settings, yet the underlying circuit computations remain unclear. Here, by examining neural ensemble activity in male rats that sample different strategies in a self-guided search for latent task structure, we observe robust tracking during strategy execution of a summary statistic for that strategy in recent behavioral history by the anterior cingulate cortex (ACC), especially by an area homologous to primate area 32D. Using the simplest summary statistic - strategy prevalence in the last 20 choices - we find that its encoding in the ACC during strategy execution is wide-scale, independent of reward delivery, and persists through a substantial ensemble reorganization that accompanies changes in global context. We further demonstrate that the tracking of reward by the ACC ensemble is also strategy-specific, but that reward prevalence is insufficient to explain the observed activity modulation during strategy execution. Our findings argue that ACC ensemble dynamics is structured by a summary statistic of recent behavioral choices, raising the possibility that ACC plays a role in estimating - through statistical learning - which actions promote the occurrence of events in the environment.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

The combined role of visual and olfactory cues in foraging by Cataglyphis ants in laboratory mazes

Foragers use several senses to locate food, and many animals rely on vision and smell. It is beneficial not to rely on a single sense, which might fail under certain conditions. We examined the contribution of vision and smell to foraging and maze exploration under laboratory conditions using Cataglyphis desert ants as a model. Foraging intensity, measured as the number of workers entering the maze and arriving at the target as well as target arrival time, were greater when food, blue light, or both were offered or presented in contrast to a control. Workers trained to forage for a combined food and light cue elevated their foraging intensity with experience. However, foraging intensity was not higher when using both cues simultaneously than in either one of the two alone. Following training, we split between the two cues and moved either the food or the blue light to the opposite maze corner. This manipulation impaired foraging success by either leading to fewer workers arriving at the target cell (when the light stayed and the food was moved) or to more workers arriving at the opposite target cell, empty of food (when the food stayed and the light was moved). This result indicates that ant workers use both senses when foraging for food and readily associate light with food.

Abstract Value Encoding in Neural Populations But Not Single Neurons

An important open question in neuroeconomics is how the brain represents the value of offers in a way that is both abstract (allowing for comparison) and concrete (preserving the details of the factors that influence value). Here, we examine neuronal responses to risky and safe options in five brain regions that putatively encode value in male macaques. Surprisingly, we find no detectable overlap in the neural codes used for risky and safe options, even when the options have identical subjective values (as revealed by preference) in any of the regions. Indeed, responses are weakly correlated and occupy distinct (semi-orthogonal) encoding subspaces. Notably, however, these subspaces are linked through a linear transform of their constituent encodings, a property that allows for comparison of dissimilar option types. This encoding scheme allows these regions to multiplex decision related processes: they can encode the detailed factors that influence offer value (here, risky and safety) but also directly compare dissimilar offer types. Together these results suggest a neuronal basis for the qualitatively different psychological properties of risky and safe options and highlight the power of population geometry to resolve outstanding problems in neural coding.SIGNIFICANCE STATEMENT To make economic choices, we must have some mechanism for comparing dissimilar offers. We propose that the brain uses distinct neural codes for risky and safe offers, but that these codes are linearly transformable. This encoding scheme has the dual advantage of allowing for comparison across offer types while preserving information about offer type, which in turn allows for flexibility in changing circumstances. We show that responses to risky and safe offers exhibit these predicted properties in five different reward-sensitive regions. Together, these results highlight the power of population coding principles for solving representation problems in economic choice.

Imagination as a fundamental function of the hippocampus

Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Macaques preferentially attend to intermediately surprising information

Normative learning theories dictate that we should preferentially attend to informative sources, but only up to the point that our limited learning systems can process their content. Humans, including infants, show this predicted strategic deployment of attention. Here, we demonstrate that rhesus monkeys, much like humans, attend to events of moderate surprisingness over both more and less surprising events. They do this in the absence of any specific goal or contingent reward, indicating that the behavioural pattern is spontaneous. We suggest this U-shaped attentional preference represents an evolutionarily preserved strategy for guiding intelligent organisms toward material that is maximally useful for learning.

An artificial intelligence life cycle: From conception to production

This paper presents the "CDAC AI life cycle," a comprehensive life cycle for the design, development, and deployment of artificial intelligence (AI) systems and solutions. It addresses the void of a practical and inclusive approach that spans beyond the technical constructs to also focus on the challenges of risk analysis of AI adoption, transferability of prebuilt models, increasing importance of ethics and governance, and the composition, skills, and knowledge of an AI team required for successful completion. The life cycle is presented as the progression of an AI solution through its distinct phases-design, develop, and deploy-and 19 constituent stages from conception to production as applicable to any AI initiative. This life cycle addresses several critical gaps in the literature where related work on approaches and methodologies are adapted and not designed specifically for AI. A technical and organizational taxonomy that synthesizes the functional value of AI is a further contribution of this article.

Medial and orbital frontal cortex in decision-making and flexible behavior

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Effects of early life stress during stress hyporesponsive period (SHRP) on anxiety and curiosity in adolescent rats

Repeated exposure to adverse experiences in early life, termed Early Life Stress (ELS), can increase anxiety disorders later in life. Anxiety is directly associated with curiosity, a form of intrinsic drive state associated with increased novelty-seeking behaviour and risk taking for challenging opportunities and could probably modulate learning and memory. In humans, elevated curiosity during adolescence tends to elicit increased exploration, novelty seeking, high risk-taking behaviour and heightened emotionality. Such behaviours are beneficial in maintaining social skills and cognitive functions later in life. We investigated whether ELS-induced anxiety impacts curiosity-like behaviour at adolescence in an animal model. ELS was induced by subjecting Sprague Dawley rat pups to maternal separation and isolation (MS) stress during the stress hyporesponsive period (SHRP) from post-natal days (PND) 4-PND 14. This rat model was tested for anxiety, spontaneous exploratory behaviour and curiosity-like behaviour in a custom-designed arena during adolescence (PND 30-45). ELS-induced changes in the stress were confirmed by corticosterone, while, basal dopamine level was estimated to understand the neurochemical basis of MS stress-induced changes in curiosity. We observed an increase in the levels of anxiety and intrinsic drive state such as curiosity-like behaviour, which was associated with elevated plasma corticosterone and dopamine in MS animals during adolescence suggesting the impact of ELS during SHRP on adolescent behaviour.

The time, the path, its length and strenuousness in maze learning

Previous findings show that rats in a maze tend to choose the shortest path to reach food. But it is not clear whether this choice is based on path length solely, or some other factors. The aim of this experiment was to investigate which factor dominates the behavior in a maze: path (longer and shorter), time (longer and shorter), or effort (more or less strenuous). The experiment involved 40 mice (4 groups), learning a maze with two paths. Each group went through only one of the situations within which we kept one factor constant on two paths while the remaining two factors were varied. Only in the fourth situation all factors were equalized. The results show that there is a statistically significant difference in the maze path preference between four situations. Preference between the paths is such that mice always choose paths requiring less effort.

Curiosity in childhood and adolescence - what can we learn from the brain

•

Curiosity enhances memory via the hippocampus, prefrontal cortex, and ventral striatum.

•

Development of curiosity and its effect on memory in childhood/adolescence not well understood.

•

Maturation of curiosity-promoting brain functions might contribute to increasing benefits of curiosity for learning.

•

Harnessing curiosity in education might need differential approaches across child development.

Accumulating evidence in adults has shown that curiosity and surprise enhance memory via activity in the hippocampus, prefrontal cortex, and dopaminergic areas. Based on findings of how these brain areas and their inter-connections develop during childhood and adolescence, we discuss how the effects of curiosity and surprise on memory may develop during childhood and adolescence. We predict that the maturation of brain areas potentially related to curiosity elicitation (hippocampus, anterior cingulate cortex [ACC], prefrontal cortex) and protracted development of hippocampal-PFC and ACC-PFC connectivity lead to differential effects of curiosity and surprise on memory during childhood and adolescence. Our predictions are centred within the PACE (Prediction-Appraisal-Curiosity-Exploration) Framework which proposes multiple levels of analyses of how curiosity is elicited and enhances memory.