Information-seeking, curiosity, and attention: computational and neural mechanisms

Infants tailor their attention to maximize learning

Infants' remarkable learning abilities allow them to rapidly acquire many complex skills. It has been suggested that infants achieve this learning by optimally allocating their attention to relevant stimuli in the environment, but the underlying mechanisms remain poorly understood. Here, we modeled infants' looking behavior during a learning task through an ideal learner that quantified the informational structure of environmental stimuli. We show that saccadic latencies, looking time, and time spent engaged with a stimulus sequence are explained by the properties of the learning environments, including the level of surprise of the stimulus, overall predictability of the environment, and progress in learning the environmental structure. These findings reveal the factors that shape infants' advanced learning, emphasizing their predisposition to seek out stimuli that maximize learning.

Explaining individual differences in infant visual sensory seeking

Individual differences in infants’ engagement with their environment manifest early in development and are noticed by parents. Three views have been advanced to explain differences in seeking novel stimulation. The optimal stimulation hypothesis suggests that individuals seek further stimulation when they are under‐responsive to current sensory input. The processing speed hypothesis proposes that those capable of processing information faster are driven to seek stimulation more frequently. The information prioritization hypothesis suggests the differences in stimulation seeking index variation in the prioritization of incoming relative to ongoing information processing. Ten‐month‐old infants saw 10 repetitions of a video clip and changes in frontal theta oscillatory amplitude were measured as an index of information processing speed. Stimulus‐locked P1 peak amplitude in response to checkerboards briefly overlaid on the video at random points during its presentation indexed processing of incoming stimulation. Parental report of higher visual seeking did not relate to reduced P1 peak amplitude or to a stronger decrease in frontal theta amplitude with repetition, thus not supporting either the optimal stimulation or the processing speed hypotheses. Higher visual seeking occurred in those infants whose P1 peak amplitude was greater than expected based on their theta amplitude. These findings indicate that visual sensory seeking in infancy is explained by a bias toward novel stimulation, thus supporting the information prioritization hypothesis.

States of curiosity and interest enhance memory differently in adolescents and in children

Curiosity - broadly defined as the desire to acquire new information - enhances learning and memory in adults. In addition, interest in the information (i.e., when the information is processed) can also facilitate later memory. To date, it is not known how states of pre-information curiosity and post-information interest enhance memory in childhood and adolescence. We used a trivia paradigm in which children and adolescents (N = 60, 10-14 years) encoded trivia questions and answers associated with high or low curiosity. States of high pre-answer curiosity enhanced later memory for trivia answers in both children and adolescents. However, higher positive post-answer interest enhanced memory for trivia answers beyond the effects of curiosity more strongly in adolescents than in children. These results suggest that curiosity and interest have positive effects on learning and memory in childhood and adolescence, but might need to be harnessed in differential ways across child development to optimize learning.

The ventral striatum dissociates information expectation, reward anticipation, and reward receipt

Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.

Where Infants Go: Real-Time Dynamics of Locomotor Exploration in Crawling and Walking Infants

Where do infants go? A longstanding assumption is that infants primarily crawl or walk to reach destinations viewed while stationary. However, many bouts of spontaneous locomotion do not end at new people, places, or things. Study 1 showed that half of 10- and 13-month-old crawlers' (N = 29) bouts end at destinations-more than previously found with walkers. Study 2 confirmed that, although infants do not commonly go to destinations, 12-month-old crawlers go to proportionally more destinations than age-matched walkers (N = 16). Head-mounted eye tracking revealed that crawlers and walkers mostly take steps in place while fixating something within reach. When infants do go to a destination, they take straight, short paths to a target fixated while stationary.

The effect of intrinsic and extrinsic motivation on memory formation: insight from behavioral and imaging study

Motivation can be generated intrinsically or extrinsically, and both kinds of motivation show similar facilitatory effects on memory. However, effects of extrinsic and intrinsic motivation on memory formation have not been studied in combination and thus, it is unknown whether they interact and how such interplay is neurally implemented. In the present study, both extrinsic monetary reward and intrinsic curiosity enhanced memory performance, without evidence for an interaction. Functional magnetic resonance imaging revealed that curiosity-driven activity in the ventral striatal reward network appears to work cooperatively with the fronto-parietal attention network, while enhancing memory formation. In contrast, the monetary reward-modulated subsequent memory effect revealed deactivation in parietal midline regions. Thus, curiosity might enhance memory performance by allocation of attentional resources and reward-related processes; while, monetary reward does so by suppression of task-irrelevant processing.

Electronic skins and machine learning for intelligent soft robots

Soft robots have garnered interest for real-world applications because of their intrinsic safety embedded at the material level. These robots use deformable materials capable of shape and behavioral changes and allow conformable physical contact for manipulation. Yet, with the introduction of soft and stretchable materials to robotic systems comes a myriad of challenges for sensor integration, including multimodal sensing capable of stretching, embedment of high-resolution but large-area sensor arrays, and sensor fusion with an increasing volume of data. This Review explores the emerging confluence of e-skins and machine learning, with a focus on how roboticists can combine recent developments from the two fields to build autonomous, deployable soft robots, integrated with capabilities for informative touch and proprioception to stand up to the challenges of real-world environments.

Attention in Psychology, Neuroscience, and Machine Learning

Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.

Network architectures supporting learnability

Human learners acquire complex interconnected networks of relational knowledge. The capacity for such learning naturally depends on two factors: the architecture (or informational structure) of the knowledge network itself and the architecture of the computational unit-the brain-that encodes and processes the information. That is, learning is reliant on integrated network architectures at two levels: the epistemic and the computational, or the conceptual and the neural. Motivated by a wish to understand conventional human knowledge, here, we discuss emerging work assessing network constraints on the learnability of relational knowledge, and theories from statistical physics that instantiate the principles of thermodynamics and information theory to offer an explanatory model for such constraints. We then highlight similarities between those constraints on the learnability of relational networks, at one level, and the physical constraints on the development of interconnected patterns in neural systems, at another level, both leading to hierarchically modular networks. To support our discussion of these similarities, we employ an operational distinction between the modeller (e.g. the human brain), the model (e.g. a single human's knowledge) and the modelled (e.g. the information present in our experiences). We then turn to a philosophical discussion of whether and how we can extend our observations to a claim regarding explanation and mechanism for knowledge acquisition. What relation between hierarchical networks, at the conceptual and neural levels, best facilitate learning? Are the architectures of optimally learnable networks a topological reflection of the architectures of comparably developed neural networks? Finally, we contribute to a unified approach to hierarchies and levels in biological networks by proposing several epistemological norms for analysing the computational brain and social epistemes, and for developing pedagogical principles conducive to curious thought. This article is part of the theme issue 'Unifying the essential concepts of biological networks: biological insights and philosophical foundations'.

Shared striatal activity in decisions to satisfy curiosity and hunger at the risk of electric shocks

Curiosity is often portrayed as a desirable feature of human faculty. However, curiosity may come at a cost that sometimes puts people in harmful situations. Here, using a set of behavioural and neuroimaging experiments with stimuli that strongly trigger curiosity (for example, magic tricks), we examine the psychological and neural mechanisms underlying the motivational effect of curiosity. We consistently demonstrate that across different samples, people are indeed willing to gamble, subjecting themselves to electric shocks to satisfy their curiosity for trivial knowledge that carries no apparent instrumental value. Also, this influence of curiosity shares common neural mechanisms with that of hunger for food. In particular, we show that acceptance (compared to rejection) of curiosity-driven or incentive-driven gambles is accompanied by enhanced activity in the ventral striatum when curiosity or hunger was elicited, which extends into the dorsal striatum when participants made a decision.

Adapt, Explain, Engage—A Study on How Social Robots Can Scaffold Second-language Learning of Children

Social robots are increasingly applied to support children’s learning, but how a robot can foster (or may hinder) learning is still not fully clear. One technique used by teachers is scaffolding, temporarily assisting learners to achieve new skills or levels of understanding they would not reach on their own. We ask if and how a social robot can be utilized to scaffold second-language learning of children at kindergarten age (4--7 years). Specifically, we explore an adapt-and-explain scaffolding strategy in which a robot acts as a peer-like tutor who dynamically adapts its behavior or the learning tasks to the cognitive and affective state of the child, and provides verbal explanations of these adaptations. An evaluation study with 40 children shows that children benefit from the learning adaptation and that the explanations have a positive effect especially for slower learners. Further, in 76% of all cases the robot managed to “re-engage” children who started to disengage from the learning interaction, helping them to achieve an overall higher learning gain. These findings demonstrate that a social robot equipped with suitable scaffolding mechanisms can increase engagement and learning, especially when being adaptive to the individual behavior and states of a child learner.

A curious formulation robot enables the discovery of a novel protocell behavior

We describe a chemical robotic assistant equipped with a curiosity algorithm (CA) that can efficiently explore the states a complex chemical system can exhibit. The CA-robot is designed to explore formulations in an open-ended way with no explicit optimization target. By applying the CA-robot to the study of self-propelling multicomponent oil-in-water protocell droplets, we are able to observe an order of magnitude more variety in droplet behaviors than possible with a random parameter search and given the same budget. We demonstrate that the CA-robot enabled the observation of a sudden and highly specific response of droplets to slight temperature changes. Six modes of self-propelled droplet motion were identified and classified using a time-temperature phase diagram and probed using a variety of techniques including NMR. This work illustrates how CAs can make better use of a limited experimental budget and significantly increase the rate of unpredictable observations, leading to new discoveries with potential applications in formulation chemistry.

Aesthetic appreciation of musical intervals enhances behavioural and neurophysiological indexes of attentional engagement and motor inhibition

From Kant to current perspectives in neuroaesthetics, the experience of beauty has been described as disinterested, i.e. focusing on the stimulus perceptual features while neglecting self-referred concerns. At a neurophysiological level, some indirect evidence suggests that disinterested aesthetic appreciation might be associated with attentional enhancement and inhibition of motor behaviour. To test this hypothesis, we performed three auditory-evoked potential experiments, employing consonant and dissonant two-note musical intervals. Twenty-two volunteers judged the beauty of intervals (Aesthetic Judgement task) or responded to them as fast as possible (Detection task). In a third Go-NoGo task, a different group of twenty-two participants had to refrain from responding when hearing intervals. Individual aesthetic judgements positively correlated with response times in the Detection task, with slower motor responses for more appreciated intervals. Electrophysiological indexes of attentional engagement (N1/P2) and motor inhibition (N2/P3) were enhanced for more appreciated intervals. These findings represent the first experimental evidence confirming the disinterested interest hypothesis and may have important applications in research areas studying the effects of stimulus features on learning and motor behaviour.

How Curiosity Enhances Hippocampus-Dependent Memory: The Prediction, Appraisal, Curiosity, and Exploration (PACE) Framework

Curiosity plays a fundamental role for learning and memory, but the neural mechanisms that stimulate curiosity and its effect on memory are poorly understood. Accumulating evidence suggests that curiosity states are related to modulations in activity in the dopaminergic circuit and that these modulations impact memory encoding and consolidation for both targets of curiosity and incidental information encountered during curiosity states. To account for this evidence, we propose the Prediction, Appraisal, Curiosity, and Exploration (PACE) framework, which attempts to explain curiosity and memory in terms of cognitive processes, neural circuits, behavior, and subjective experience. The PACE framework generates testable predictions that can stimulate future investigation of the mechanisms underlying curiosity-related memory enhancements.

Predictability and Uncertainty in the Pleasure of Music: A Reward for Learning?

Music ranks among the greatest human pleasures. It consistently engages the reward system, and converging evidence implies it exploits predictions to do so. Both prediction confirmations and errors are essential for understanding one's environment, and music offers many of each as it manipulates interacting patterns across multiple timescales. Learning models suggest that a balance of these outcomes (i.e., intermediate complexity) optimizes the reduction of uncertainty to rewarding and pleasurable effect. Yet evidence of a similar pattern in music is mixed, hampered by arbitrary measures of complexity. In the present studies, we applied a well-validated information-theoretic model of auditory expectation to systematically measure two key aspects of musical complexity: predictability (operationalized as information content [IC]), and uncertainty (entropy). In Study 1, we evaluated how these properties affect musical preferences in 43 male and female participants; in Study 2, we replicated Study 1 in an independent sample of 27 people and assessed the contribution of veridical predictability by presenting the same stimuli seven times. Both studies revealed significant quadratic effects of IC and entropy on liking that outperformed linear effects, indicating reliable preferences for music of intermediate complexity. An interaction between IC and entropy further suggested preferences for more predictability during more uncertain contexts, which would facilitate uncertainty reduction. Repeating stimuli decreased liking ratings but did not disrupt the preference for intermediate complexity. Together, these findings support long-hypothesized optimal zones of predictability and uncertainty in musical pleasure with formal modeling, relating the pleasure of music listening to the intrinsic reward of learning.SIGNIFICANCE STATEMENT Abstract pleasures, such as music, claim much of our time, energy, and money despite lacking any clear adaptive benefits like food or shelter. Yet as music manipulates patterns of melody, rhythm, and more, it proficiently exploits our expectations. Given the importance of anticipating and adapting to our ever-changing environments, making and evaluating uncertain predictions can have strong emotional effects. Accordingly, we present evidence that listeners consistently prefer music of intermediate predictive complexity, and that preferences shift toward expected musical outcomes in more uncertain contexts. These results are consistent with theories that emphasize the intrinsic reward of learning, both by updating inaccurate predictions and validating accurate ones, which is optimal in environments that present manageable predictive challenges (i.e., reducible uncertainty).

Motivation to Learn

Learners are more likely to remember what they study if they are motivated to do so. Such motivation can be externally driven by prospective rewards, but also intrinsically driven by curiosity. The present research focused on the role of curiosity during intentional learning. We examined the potential mnemonic benefit of curiosity, whether this benefit is undermined when learners are externally motivated to learn by rewards, and whether it can be attributed to increased study time for information they are more curious about. In two experiments, participants were presented with trivia questions, rated their level of curiosity about each question, and then studied the answers, either with or without a prospect of reward for correct recall on a subsequent test. Study time was either fixed (Experiment 1) or self-paced (Experiment 2). Performance on a memory test 1 week later suggested that curiosity enhanced long-term retention, and that rewards did not undermine the benefit of curiosity. When learning was self-paced, study time increased with curiosity but did not account for the effect of curiosity on memory. The results highlight the important role curiosity plays in learning and suggest that curiosity and rewards can be used together effectively to promote students' learning.

A computational account of threat-related attentional bias

Visual selective attention acts as a filter on perceptual information, facilitating learning and inference about important events in an agent's environment. A role for visual attention in reward-based decisions has previously been demonstrated, but it remains unclear how visual attention is recruited during aversive learning, particularly when learning about multiple stimuli concurrently. This question is of particular importance in psychopathology, where enhanced attention to threat is a putative feature of pathological anxiety. Using an aversive reversal learning task that required subjects to learn, and exploit, predictions about multiple stimuli, we show that the allocation of visual attention is influenced significantly by aversive value but not by uncertainty. Moreover, this relationship is bidirectional in that attention biases value updates for attended stimuli, resulting in heightened value estimates. Our findings have implications for understanding biased attention in psychopathology and support a role for learning in the expression of threat-related attentional biases in anxiety.

Water games by mountain gorillas: implications for behavioral development and flexibility-a case report

Functions of play, which may be performed solo or in a social context, include motor training and behavioral flexibility. Play is often more common in infancy and the juvenile period, although it also occurs in adults of many species. In contrast to social play, few studies have investigated solitary play. Here, we present new empirical data on solitary water play in a subadult and two adult mountain gorillas in Bwindi Impenetrable National Park, Uganda, observed on three different days between January and February 2018. Focal sampling was used to record the behavior of the individuals interacting with water. Movements included vigorous rotation of the arms, splashing the water, tilting the head, making a play face, and sweeping with the hands to create waves on the water surface. One of the episodes represents the first vigorous display of splashing water ever reported for Bwindi gorillas. Our observations highlight three significant components of mountain gorilla development and behavior: play, behavioral flexibility, and exploration.

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a dual-learning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent's chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.

The Neurobiology of Confidence: From Beliefs to Neurons

How confident are you? As humans, aware of our subjective sense of confidence, we can readily answer. Knowing your level of confidence helps to optimize both routine decisions such as whether to go back and check if the front door was locked and momentous ones like finding a partner for life. Yet the inherently subjective nature of confidence has limited investigations by neurobiologists. Here, we provide an overview of recent advances in this field and lay out a conceptual framework that lets us translate psychological questions about subjective confidence into the language of neuroscience. We show how statistical notions of confidence provide a bridge between our subjective sense of confidence and confidence-guided behaviors in nonhuman animals, thus enabling the study of the underlying neurobiology. We discuss confidence as a core cognitive process that enables organisms to optimize behavior such as learning or resource allocation and that serves as the basis of metacognitive reasoning. These approaches place confidence on a solid footing and pave the way for a mechanistic understanding of how the brain implements confidence-based algorithms to guide behavior.

Diverse motives for human curiosity

Curiosity-our desire to know-is a fundamental drive in human behaviour, but its mechanisms are poorly understood. A classical question concerns the curiosity motives. What drives individuals to become curious about some but not other sources of information?¹ Here we show that curiosity about probabilistic events depends on multiple aspects of the distribution of these events. Participants (n = 257) performed a task in which they could demand advance information about only one of two randomly selected monetary prizes that contributed to their income. Individuals differed markedly in the extent to which they requested information as a function of the ex ante uncertainty or ex ante value of an individual prize. This heterogeneity was not captured by theoretical models describing curiosity as a desire to learn about the total rewards of a situation^2,3. Instead, it could be explained by an extended model that allowed for attribute-specific anticipatory utility-the savouring of individual components of the eventual reward-and postulates that this utility increased nonlinearly with the certainty of receiving the reward. Parameter values fitting individual choices were consistent for information about gains or losses, suggesting that attribute-specific anticipatory utility captures fundamental heterogeneity in the determinants of curiosity.

Processing bias: extending sensory drive to include efficacy and efficiency in information processing

Communication signals often comprise an array of colours, lines, spots, notes or odours that are arranged in complex patterns, melodies or blends. Receiver perception is assumed to influence preference and thus the evolution of signal design, but evolutionary biologists still struggle to understand how perception, preference and signal design are mechanistically linked. In parallel, the field of empirical aesthetics aims to understand why people like some designs more than others. The model of processing bias discussed here is rooted in empirical aesthetics, which posits that preferences are influenced by the emotional system as it monitors the dynamics of information processing and that attractive signals have effective designs that maximize information transmission, efficient designs that allow information processing at low metabolic cost, or both. We refer to the causal link between preference and the emotionally rewarding experience of effective and efficient information processing as the processing bias, and we apply it to the evolutionary model of sensory drive. A sensory drive model that incorporates processing bias hypothesizes a causal chain of relationships between the environment, perception, pleasure, preference and ultimately the evolution of signal design, both simple and complex.

The value of uncertainty: An active inference perspective

We discuss how uncertainty underwrites exploration and epistemic foraging from the perspective of active inference: a generic scheme that places pragmatic (utility maximization) and epistemic (uncertainty minimization) imperatives on an equal footing - as primary determinants of proximal behavior. This formulation contextualizes the complementary motivational incentives for reward-related stimuli and environmental uncertainty, offering a normative treatment of their trade-off.

Monkeys are curious about counterfactual outcomes

Many non-human animals show exploratory behaviors. It remains unclear whether any possess human-like curiosity. We previously proposed three criteria for applying the term curiosity to animal behavior: (1) the subject is willing to sacrifice reward to obtain information, (2) the information provides no immediate instrumental or strategic benefit, and (3) the amount the subject is willing to pay depends systematically on the amount of information available. In previous work on information-seeking in animals, information generally predicts upcoming rewards, and animals' decisions may therefore be a byproduct of reinforcement processes. Here we get around this potential confound by taking advantage of macaques' ability to reason counterfactually (that is, about outcomes that could have occurred had the subject chosen differently). Specifically, macaques sacrificed fluid reward to obtain information about counterfactual outcomes. Moreover, their willingness to pay scaled with the information (Shannon entropy) offered by the counterfactual option. These results demonstrate the existence of human-like curiosity in non-human primates according to our criteria, which circumvent several confounds associated with less stringent criteria.

On the automaticity of attentional orienting to threatening stimuli

Attention is biased toward stimuli that have been associated with aversive outcomes in the past. This bias has previously been interpreted as reflecting automatic orienting toward threat signals. However, in many prior studies, either the threatening stimulus provided valuable predictive information, signaling the possibility of an otherwise unavoidable punishment and thereby allowing participants to brace themselves, or the aversive event could be avoided with fast and accurate task performance. Under these conditions, monitoring for threat could be viewed as an adaptive strategy. In the present study, fixating a color stimulus immediately resulted in a shock on some trials, providing a direct incentive not to look at the stimulus. Nevertheless, this contingency resulted in participants fixating the shock-associated stimulus more frequently than a neutral distractor matched for physical salience. Our findings demonstrate that threatening stimuli are automatically attended even when attending such stimuli is actually responsible for triggering the aversive event, providing compelling evidence for automaticity. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Boredom-Driven Curious Learning by Homeo-Heterostatic Value Gradients

This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We offer an instrumental view of action selection, in which an action serves to disclose outcomes that have intrinsic meaningfulness to an agent itself. This motivated two central algorithmic ingredients: devaluation and devaluation progress, both underpin agent's cognition concerning intrinsically generated rewards. The two serve as an instantiation of homeostatic and heterostatic intrinsic motivation. A key insight from our algorithm is that the two seemingly opposite motivations can be reconciled-without which exploration and information-gathering cannot be effectively carried out. We supported this claim with empirical evidence, showing that boredom-enabled agents consistently outperformed other curious or explorative agent variants in model building benchmarks based on self-assisted experience accumulation.

Costly curiosity: People pay a price to resolve an uncertain gamble early

Humans are inherently curious creatures, continuously seeking out information about future outcomes. Such advance information is often valuable, potentially allowing people to select better courses of action. In non-human animals, this drive for information can be so strong that they forego food or water to find out a few seconds earlier whether an uncertain option will provide a reward. Here, we assess whether people will exhibit a similar sub-optimal preference for advance information. Participants played a card-flipping task where they were probabilistically rewarded based on the pattern of 3 cards that were revealed after a 5-s delay. During this delay, participants could instead pay a cost to find out the next card's identity immediately. This choice to find out early did not influence the eventual outcome. Participants preferred to find out early about 80% of the time when the information was free; they were even willing to incur an expense to get advance information about the eventual outcome. The expected magnitude of the outcome, however, had little impact on the likelihood of finding out early. These results suggest that humans, like animals, value non-instrumental information and will pay a price for such information, independent of its utility.

Learning a Set of Interrelated Tasks by Using a Succession of Motor Policies for a Socially Guided Intrinsically Motivated Learner

We aim at a robot capable to learn sequences of actions to achieve a field of complex tasks. In this paper, we are considering the learning of a set of interrelated complex tasks hierarchically organized. To learn this high-dimensional mapping between a continuous high-dimensional space of tasks and an infinite dimensional space of unbounded sequences of actions, we introduce a new framework called "procedures", which enables the autonomous discovery of how to combine previously learned skills in order to learn increasingly complex combinations of motor policies. We propose an active learning algorithmic architecture, capable of organizing its learning process in order to achieve a field of complex tasks by learning sequences of primitive motor policies. Based on heuristics of active imitation learning, goal-babbling and strategic learning using intrinsic motivation, our algorithmic architecture leverages our procedures framework to actively decide during its learning process which outcome to focus on and which exploration strategy to apply. We show on a simulated environment that our new architecture is capable of tackling the learning of complex motor policies by adapting the complexity of its policies to the task at hand. We also show that our "procedures" enable the learning agent to discover the task hierarchy and exploit his experience of previously learned skills to learn new complex tasks.

Computational mechanisms of curiosity and goal-directed exploration

Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. 'Hidden state' exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, 'model parameter' exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of 'Bayes-optimal' behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.

Perceptual Content, Not Physiological Signals, Determines Perceived Duration When Viewing Dynamic, Natural Scenes

The neural basis of time perception remains unknown. A prominent account is the pacemaker-accumulator model, wherein regular ticks of some physiological or neural pacemaker are read out as time. Putative candidates for the pacemaker have been suggested in physiological processes (heartbeat), or dopaminergic mid-brain neurons, whose activity has been associated with spontaneous blinking. However, such proposals have difficulty accounting for observations that time perception varies systematically with perceptual content. We examined physiological influences on human duration estimates for naturalistic videos between 1–64 seconds using cardiac and eye recordings. Duration estimates were biased by the amount of change in scene content. Contrary to previous claims, heart rate, and blinking were not related to duration estimates. Our results support a recent proposal that tracking change in perceptual classification networks provides a basis for human time perception, and suggest that previous assertions of the importance of physiological factors should be tempered.

The principle of consistency and the cause and function of behaviour

At all levels of information processing in the brain, neural and cognitive structures tend towards a state of consistency. When two or more simultaneously active cognitive structures are logically inconsistent, arousal is increased, which activates processes with the expected consequence of increasing consistency and decreasing arousal. Increased arousal is experienced as aversive, while the expected or actual decrease in arousal as a result of increased consistency is experienced as rewarding. Modes of resolution of inconsistency can be divided into purely cognitive solutions, such as changing an attitude or an associated motor plan, and behavioural solutions, such as exploration, aggression, fear, and feeding. Models and theories consistent with the principle of consistency are numerous, have a long and continuing history, and come from many different scientific fields, such as social psychology, perception, neurocognition, learning, motor control, system control, ethology, and stress. The present paper presents a brief overview of relevant information from these fields of research, while focusing mainly on the implications of the principle of consistency for the understanding of the cause and function of behaviour. Based on this overview, it is proposed that all behaviour involving cognitive processing is caused by the activation of inconsistent cognitions and functions to increase perceived consistency.