High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

Goals, usefulness and abstraction in value-based choice

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.

Extracting the dynamics of behavior in sensory decision-making experiments

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.

Developmental changes between childhood and adulthood in passive observational and interactive feedback-based categorization rule learning

As children start attending school they are more likely to face situations where they have to autonomously learn about novel object categories (e.g. by reading a picture book with descriptions of novel animals). Such autonomous observational category learning (OCL) gradually complements interactive feedback-based category learning (FBCL), where a child hypothesizes about the nature of a novel object, acts based on his prediction, and then receives feedback indicating the correctness of his prediction. Here we tested OCL and FBCL skills of elementary school children and adults. In both conditions, participants performed complex rule-based categorization tasks that required associating novel objects with novel category-labels. We expected children to perform better in FBCL tasks than in OCL tasks, whereas adults to be skilled in both tasks. As hypothesized, in early-phase learning children performed better in FBCL tasks than in OCL tasks. Unexpectedly, adults performed somewhat better in OCL tasks. Early-phase FBCL performance in the two age groups was matched, but the OCL performance of adults was higher than that of children. In late-phase learning there was only an age group main effect (adults > children). Moreover, performance in post-learning categorization tasks, that did not require label recollection, indicated that in FBCL tasks children were likely to directly learn the associations between an object and a category label, whereas in the OCL tasks they were likely to first learn which feature-dimensions were relevant. These findings shed light on developmental changes in cognitive control and learning mechanisms. Implications for educational settings are discussed.

Personalized learning: From neurogenetics of behaviors to designing optimal language training

Variability in drug responsivity has prompted the development of Personalized Medicine, which has shown great promise in utilizing genotypic information to develop safer and more effective drug regimens for patients. Similarly, individual variability in learning outcomes has puzzled researchers who seek to create optimal learning environments for students. "Personalized Learning" seeks to identify genetic, neural and behavioral predictors of individual differences in learning and aims to use predictors to help create optimal teaching paradigms. Evidence for Personalized Learning can be observed by connecting research in pharmacogenomics, cognitive genetics and behavioral experiments across domains of learning, which provides a framework for conducting empirical studies from the laboratory to the classroom and holds promise for addressing learning effectiveness in the individual learners. Evidence can also be seen in the subdomain of speech learning, thus providing initial support for the applicability of Personalized Learning to language.

Towards the design principles of neural population codes

The ability to record the joint activity of large groups of neurons would allow for direct study of information representation and computation at the level of whole circuits in the brain. The combinatorial space of potential population activity patterns and neural noise imply that it would be impossible to directly map the relations between stimuli and population responses. Understanding of large neural population codes therefore depends on identifying simplifying design principles. We review recent results showing that strongly correlated population codes can be explained using minimal models that rely on low order relations among cells. We discuss the implications for large populations, and how such models allow for mapping the semantic organization of the neural codebook and stimulus space, and decoding.

Reinforcement learning in multidimensional environments relies on attention mechanisms

In recent years, ideas from the computational field of reinforcement learning have revolutionized the study of learning in the brain, famously providing new, precise theories of how dopamine affects learning in the basal ganglia. However, reinforcement learning algorithms are notorious for not scaling well to multidimensional environments, as is required for real-world learning. We hypothesized that the brain naturally reduces the dimensionality of real-world problems to only those dimensions that are relevant to predicting reward, and conducted an experiment to assess by what algorithms and with what neural mechanisms this "representation learning" process is realized in humans. Our results suggest that a bilateral attentional control network comprising the intraparietal sulcus, precuneus, and dorsolateral prefrontal cortex is involved in selecting what dimensions are relevant to the task at hand, effectively updating the task representation through trial and error. In this way, cortical attention mechanisms interact with learning in the basal ganglia to solve the "curse of dimensionality" in reinforcement learning.