Computational neuroscience across the lifespan: Promises and pitfalls

A cognitive-computational account of mood swings in adolescence

Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.

Understanding mental health through computers: An introduction to computational psychiatry

Computational psychiatry recently established itself as a new tool in the study of mental disorders and problems. Integration of different levels of analysis is creating computational phenotypes with clinical and research values, and constructing a way to arrive at precision psychiatry are part of this new branch. It conceptualizes the brain as a computational organ that receives from the environment parameters to respond to challenges through calculations and algorithms in continuous feedback and feedforward loops with a permanent degree of uncertainty. Through this conception, one can seize an understanding of the cerebral and mental processes in the form of theories or hypotheses based on data. Using these approximations, a better understanding of the disorder and its different determinant factors facilitates the diagnostics and treatment by having an individual, ecologic, and holistic approach. It is a tool that can be used to homologate and integrate multiple sources of information given by several theoretical models. In conclusion, it helps psychiatry achieve precision and reproducibility, which can help the mental health field achieve significant advancement. This article is a narrative review of the basis of the functioning of computational psychiatry with a critical analysis of its concepts.

Social learning across adolescence: A Bayesian neurocognitive perspective

Adolescence is a period of social re-orientation in which we are generally more prone to peer influence and the updating of our beliefs based on social information, also called social learning, than in any other stage of our life. However, how do we know when to use social information and whose information to use and how does this ability develop across adolescence? Here, we review the social learning literature from a behavioral, neural and computational viewpoint, focusing on the development of brain systems related to executive functioning, value-based decision-making and social cognition. We put forward a Bayesian reinforcement learning framework that incorporates social learning about value associated with particular behavior and uncertainty in our environment and experiences. We discuss how this framework can inform us about developmental changes in social learning, including how the assessment of uncertainty and the ability to adaptively discriminate between information from different social sources change across adolescence. By combining reward-based decision-making in the domains of both informational and normative influence, this framework explains both negative and positive social peer influence in adolescence.

A methodological perspective on learning in the developing brain

The brain undergoes profound development across childhood and adolescence, including continuous changes in brain morphology, connectivity, and functioning that are, in part, dependent on one's experiences. These neurobiological changes are accompanied by significant changes in children's and adolescents' cognitive learning. By drawing from studies in the domains of reading, reinforcement learning, and learning difficulties, we present a brief overview of methodological approaches and research designs that bridge brain- and behavioral research on learning. We argue that ultimately these methods and designs may help to unravel questions such as why learning interventions work, what learning computations change across development, and how learning difficulties are distinct between individuals.

The interpretation of computational model parameters depends on the context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8–30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.

The computational challenge of social learning

The complex reward structure of the social world and the uncertainty endemic to social contexts poses a challenge for modeling. For example, during social interactions, the actions of one person influence the internal states of another. These social dependencies make it difficult to formalize social learning problems in a mathematically tractable way. While it is tempting to dispense with these complexities, they are a defining feature of social life. Because the structure of social interactions challenges the simplifying assumptions often made in models, they make an ideal testbed for computational models of cognition. By adopting a framework that embeds existing social knowledge into the model, we can go beyond explaining behaviors in laboratory tasks to explaining those observed in the wild.

Information about others' choices selectively alters risk tolerance and medial prefrontal cortex activation across adolescence and young adulthood

Adolescence is associated with major changes in the cognitive, emotional and social domains. One domain in which these processes intersect is decision-making. Previous research has shown that individuals’ attitudes towards risk and ambiguity shape their decision-making, and information about others’ choices can influence individuals’ decisions. However, it is currently unknown how information about others’ choices influences risk and ambiguity attitudes separately, and the degree to which others’ choices shape decision-making differentially across development from adolescence to young adulthood. The current study used a computational modeling framework to test how information about others’ choices influences these attitudes. Participants, aged 14–22 years, made a series of risky and ambiguous choices while undergoing fMRI scanning. On some trials, they viewed risky or safe choices of others. Results showed that participants aligned their choices toward the choice preferences of others. Moreover, the tendency to align choices was expressed in changes in risk attitude, but not ambiguity attitude. The change in risk attitude was positively related to neural activation in the medial prefrontal cortex. Results did not show age related differences in behavior and corresponding neural activation, indicating that the manner in which adolescents are influenced by peers is not ubiquitous but rather, is highly context-dependent.

What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience

Reinforcement learning (RL) is a concept that has been invaluable to fields including machine learning, neuroscience, and cognitive science. However, what RL entails differs between fields, leading to difficulties when interpreting and translating findings. After laying out these differences, this paper focuses on cognitive (neuro)science to discuss how we as a field might over-interpret RL modeling results. We too often assume-implicitly-that modeling results generalize between tasks, models, and participant populations, despite negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique (neuro)cognitive processes, a concept we call interpretability, when evidence suggests that they capture different functions across studies and tasks. We conclude that future computational research needs to pay increased attention to implicit assumptions when using RL models, and suggest that a more systematic understanding of contextual factors will help address issues and improve the ability of RL to explain brain and behavior.

Neural tracking of subjective value under riskand ambiguity in adolescence

Although many neuroimaging studies on adolescent risk taking have focused on brain activation during outcome valuation, less attention has been paid to the neural correlates of choice valuation. Subjective choice valuation may be particularly influenced by whether a choice presents risk (known probabilities) or ambiguity (unknown probabilities), which has rarely been studied in developmental samples. Therefore, we examined the neural tracking of subjective value during choice under risk and ambiguity in a large sample of adolescents (N = 188, 12–22 years). Specifically, we investigated which brain regions tracked subjective value coding under risk and ambiguity. A model-based approach to estimate individuals’ risk and ambiguity attitudes showed prominent variation in individuals’ aversions to risk and ambiguity. Furthermore, participants subjectively experienced the ambiguous options as being riskier than the risky options. Subjective value tracking under risk was coded by activation in ventral striatum and superior parietal cortex. Subjective value tracking under ambiguity was coded by dorsolateral prefrontal cortex (PFC) and superior temporal gyrus activation. Finally, overlapping activation in the dorsomedial PFC was observed for subjective value under both conditions. Overall, this is the first study to chart brain activation patterns for subjective choice valuation under risk and ambiguity in an adolescent sample, which shows that the building blocks for risk and ambiguity processing are already present in early adolescence. Finally, we highlight the potential of combining behavioral modeling with fMRI for investigating choice valuation in adolescence, which may ultimately aid in understanding who takes risks and why.

Beyond dichotomies in reinforcement learning

Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.

Applying computational modeling to assess age-, sex-, and strategy-related differences in Spin the Pots, a working memory task for 2- to 4-year-olds

Working memory (WM) develops rapidly during early childhood. In the present study, visual WM (VSM) was measured using the well-established Spin the Pots task (Hughes & Ensor, 2005), a complex non-verbal eight-location object occlusion task. A self-ordered hiding procedure was adopted to allow for an examination of children's strategy use during a VWM task. Participants (N = 640) between the ages of 2 and 4 years were tested under semi-naturalistic conditions, in the home or in a museum. Computational modeling was used to estimate an expected value for the total trials to complete Spin the Pots via a random search and child performance was compared to expected values. Based on this approach, we determined that children who found six stickers retrieved them in significantly fewer trials than the expected value, excluding chance performance and implicating VWM. Results also showed age-related and sex-related changes in VWM. Between 2 and 4 years of age, 4-year-olds performed significantly better than younger children and girls out-performed the boys. Spontaneous use of a color matching hiding strategy was associated with a higher success rate on the task. Implications of these findings for early development of VWM are discussed.

Reinforcement learning across development: What insights can we draw from a decade of research?

The past decade has seen the emergence of the use of reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety of tasks and model variants, have reached convergent conclusions. In this review, we examine whether the tuning of model parameters that govern different aspects of learning and decision-making processes vary consistently as a function of age, and what neurocognitive developmental changes may account for differences in these parameter estimates across development. We explore whether patterns of developmental change in these estimates are better described by differences in the extent to which individuals adapt their learning processes to the statistics of different environments, or by more static learning biases that emerge across varied contexts. We focus specifically on learning rates and inverse temperature parameter estimates, and find evidence that from childhood to adulthood, individuals become better at optimally weighting recent outcomes during learning across diverse contexts and less exploratory in their value-based decision-making. We provide recommendations for how these two possibilities - and potential alternative accounts - can be tested more directly to build a cohesive body of research that yields greater insight into the development of core learning processes.

Adolescent Health Risk Behaviors: Convergent, Discriminant and Predictive Validity of Self-Report and Cognitive Measures

Self-report and cognitive tasks of reward sensitivity and self-regulation have influenced several developmental models that may explain the heightened engagement in risk behaviors during adolescence. Despite some inconsistencies across studies, few studies have explored the convergent, discriminant, and predictive validity of self-report and cognitive measures of these psychological characteristics in adolescence. The present study evaluated the convergent and discriminant validity of self-report and cognitive measures of reward sensitivity and self-regulation among 2017 adolescents (age M = 16.8, SD = 1.1; 56% female; 55% White, 22% Black, 8% Hispanic, 15% other race/ethnic; 49% 10th grade and 51% 12th grade). This study compared the predictive validity of an omnibus measure and specific measures of risk engagement. Convergent and discriminant validity from self-report to cognitive tasks were as predicted, although with weak convergent relationships. As hypothesized, compared to cognitive tasks, self-report measures consistently predicted risky behaviors and explained more variance in the models. These results demonstrate that while cognitive tasks can significantly predict certain risk behaviors, they require increased power to find the very small effects, raising questions about their use as implicit proxies for real world risk behavior.

Social Influence in Adolescent Decision-Making: A Formal Framework

Adolescence is a period of life during which peers play a pivotal role in decision-making. The narrative of social influence during adolescence often revolves around risky and maladaptive decisions, like driving under the influence, and using illegal substances (Steinberg, 2005). However, research has also shown that social influence can lead to increased prosocial behaviors (Van Hoorn et al., 2017) and a reduction in risk-taking (Braams et al., 2019). While many studies support the notion that adolescents are more sensitive to peer influence than children or adults, the developmental processes that underlie this sensitivity remain poorly understood. We argue that one important reason for this lack of understanding is the absence of precisely formulated models. To make a first step toward formal models of social influence during adolescence, we first identify three prominent verbal models of social influence in the literature: (1) social motivation, (2) reward sensitivity, and (3) distraction. We then illustrate how these can be translated into formal models, and how such formal models can inform experimental design and help identify developmental processes. Finally, by applying our formal models to existing datasets, we demonstrate the usefulness of formalization by synthesizing different studies with seemingly disparate results. We conclude with a discussion on how formal modeling can be utilized to better investigate the development of peer influence in adolescence.

Developmental perspectives on risky and impulsive choice

Epidemiological data suggest that risk taking in the real world increases from childhood into adolescence and declines into adulthood. However, developmental patterns of behaviour in laboratory assays of risk taking and impulsive choice are inconsistent. In this article, we review a growing literature using behavioural economic approaches to understand developmental changes in risk taking and impulsivity. We present findings that have begun to elucidate both the cognitive and neural processes that contribute to risky and impulsive choice, as well as how age-related changes in these neurocognitive processes give rise to shifts in choice behaviour. We highlight how variability in task parameters can be used to identify specific aspects of decision contexts that may differentially influence risky and impulsive choice behaviour across development. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

The computational basis of following advice in adolescents

Advice taking helps one to quickly acquire knowledge and make decisions. This age-comparative study (in children [8- to 10-year-olds], adolescents [13- to 15-year-olds], and adults [18- to 22-year-olds]) investigated developmental differences in how advice, experience, and exploration influence learning. The results showed that adolescents were initially easily swayed to follow peer advice but also switched more rapidly to exploring alternatives like children. Whereas adults stayed with the advice over the task, adolescents put more weight on their own experience compared with adults. A social learning model showed that although social influence most strongly affects adolescents' initial expectations (i.e., their priors), adolescents showed higher exploration and discovered the other good option in the current task. Thus, our model resolved the apparently conflicting findings of adolescents being more and less sensitive to peer influence and provides novel insights into the dynamic interaction between social and individual learning.

Developmental Changes in Learning: Computational Mechanisms and Social Influences

Our ability to learn from the outcomes of our actions and to adapt our decisions accordingly changes over the course of the human lifespan. In recent years, there has been an increasing interest in using computational models to understand developmental changes in learning and decision-making. Moreover, extensions of these models are currently applied to study socio-emotional influences on learning in different age groups, a topic that is of great relevance for applications in education and health psychology. In this article, we aim to provide an introduction to basic ideas underlying computational models of reinforcement learning and focus on parameters and model variants that might be of interest to developmental scientists. We then highlight recent attempts to use reinforcement learning models to study the influence of social information on learning across development. The aim of this review is to illustrate how computational models can be applied in developmental science, what they can add to our understanding of developmental mechanisms and how they can be used to bridge the gap between psychological and neurobiological theories of development.