Combining computational modeling and neuroimaging to examine multiple category learning systems in the brain

Category learning in a recurrent neural network with reinforcement learning

It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

Neural dynamics underlying the acquisition of distinct auditory category structures

Despite the multidimensional and temporally fleeting nature of auditory signals we quickly learn to assign novel sounds to behaviorally relevant categories. The neural systems underlying the learning and representation of novel auditory categories are far from understood. Current models argue for a rigid specialization of hierarchically organized core regions that are fine-tuned to extracting and mapping relevant auditory dimensions to meaningful categories. Scaffolded within a dual-learning systems approach, we test a competing hypothesis: the spatial and temporal dynamics of emerging auditory-category representations are not driven by the underlying dimensions but are constrained by category structure and learning strategies. To test these competing models, we used functional Magnetic Resonance Imaging (fMRI) to assess representational dynamics during the feedback-based acquisition of novel non-speech auditory categories with identical dimensions but differing category structures: rule-based (RB) categories, hypothesized to involve an explicit sound-to-rule mapping network, and information integration (II) based categories, involving pre-decisional integration of dimensions via a procedural-based sound-to-reward mapping network. Adults were assigned to either the RB (n = 30, 19 females) or II (n = 30, 22 females) learning tasks. Despite similar behavioral learning accuracies, learning strategies derived from computational modeling and involvements of corticostriatal systems during feedback processing differed across tasks. Spatiotemporal multivariate representational similarity analysis revealed an emerging representation within an auditory sensory-motor pathway exclusively for the II learning task, prominently involving the superior temporal gyrus (STG), inferior frontal gyrus (IFG), and posterior precentral gyrus. In contrast, the RB learning task yielded distributed neural representations within regions involved in cognitive-control and attentional processes that emerged at different time points of learning. Our results unequivocally demonstrate that auditory learners' neural systems are highly flexible and show distinct spatial and temporal patterns that are not dimension-specific but reflect underlying category structures and learning strategies.

Mental Effort and Information-Processing Costs Are Inversely Related to Global Brain Free Energy During Visual Categorization

Mental effort is a neurocognitive process that reflects the controlled expenditure of psychological information-processing resources during perception, cognition, and action. There is a practical need to operationalize and measure mental effort in order to minimize detrimental effects of mental fatigue on real-world human performance. Previous research has identified several neurocognitive indices of mental effort, but these indices are indirect measures that are also sensitive to experimental demands or general factors such as sympathetic arousal. The present study investigated a potential direct neurocognitive index of mental effort based in theories where bounded rational decision makers (realized as embodied brains) are modeled as generalized thermodynamic systems. This index is called free energy, an information-theoretic system property of the brain that reflects the difference between the brain's current and predicted states. Theory predicts that task-related differences in a decision makers' free energy are inversely related to information-processing costs related to task decisions. The present study tested this prediction by quantifying global brain free energy from electroencephalographic (EEG) measures of human brain function. EEG signals were recorded while participants engaged in two visual categorization tasks in which categorization decisions resulted from the allocation of different levels of mental information processing resources. A novel method was developed to quantify brain free energy from machine learning classification of EEG trials. Participant information-processing resource costs were estimated via computational analysis of behavior, whereas the subjective expression of mental effort was estimated via participant ratings of mental workload. Following theoretical predictions, task-related differences in brain free energy negatively correlated with increased allocation of information-processing resource costs. These brain free energy differences were smaller for the visual categorization task that required a greater versus lesser allocation of information-processing resources. Ratings of mental workload were positively correlated with information-processing resource costs, and negatively correlated with global brain free energy differences, only for the categorization task requiring the larger amount of information-processing resource costs. These findings support theoretical thermodynamic approaches to decision making and provide the first empirical evidence of a relationship between mental effort, brain free energy, and neurocognitive information-processing.

Abstract Memory Representations in the Ventromedial Prefrontal Cortex and Hippocampus Support Concept Generalization

Memory function involves both the ability to remember details of individual experiences and the ability to link information across events to create new knowledge. Prior research has identified the ventromedial prefrontal cortex (VMPFC) and the hippocampus as important for integrating across events in the service of generalization in episodic memory. The degree to which these memory integration mechanisms contribute to other forms of generalization, such as concept learning, is unclear. The present study used a concept-learning task in humans (both sexes) coupled with model-based fMRI to test whether VMPFC and hippocampus contribute to concept generalization, and whether they do so by maintaining specific category exemplars or abstract category representations. Two formal categorization models were fit to individual subject data: a prototype model that posits abstract category representations and an exemplar model that posits category representations based on individual category members. Latent variables from each of these models were entered into neuroimaging analyses to determine whether VMPFC and the hippocampus track prototype or exemplar information during concept generalization. Behavioral model fits indicated that almost three-quarters of the subjects relied on prototype information when making judgments about new category members. Paralleling prototype dominance in behavior, correlates of the prototype model were identified in VMPFC and the anterior hippocampus with no significant exemplar correlates. These results indicate that the VMPFC and portions of the hippocampus play a broad role in memory generalization and that they do so by representing abstract information integrated from multiple events.SIGNIFICANCE STATEMENT Whether people represent concepts as a set of individual category members or by deriving generalized concept representations abstracted across exemplars has been debated. In episodic memory, generalized memory representations have been shown to arise through integration across events supported by the ventromedial prefrontal cortex (VMPFC) and hippocampus. The current study combined formal categorization models with fMRI data analysis to show that the VMPFC and anterior hippocampus represent abstract prototype information during concept generalization, contributing novel evidence of generalized concept representations in the brain. Results indicate that VMPFC-hippocampal memory integration mechanisms contribute to knowledge generalization across multiple cognitive domains, with the degree of abstraction of memory representations varying along the long axis of the hippocampus.

The effect of subjective awareness measures on performance in artificial grammar learning task

Systematic research into implicit learning requires well-developed awareness-measurement techniques. Recently, trial-by-trial measures have been widely used. However, they can increase complexity of a study because they are an additional experimental variable. We tested the effects of these measures on performance in artificial grammar learning study. Four groups of participants were assigned to different awareness measures conditions: confidence ratings, post-decision wagering, decision strategy attribution or none. Decision-strategy-attribution participants demonstrated better grammar learning and longer response times compared to controls. They also exhibited a conservative bias. Grammaticality by itself was a stronger predictor of strings endorsement in decision-strategy-attribution group compared to other groups. Confidence ratings and post-decision wagering only affected the response times. These results were supported by an additional experiment that used a balanced chunk strength design. We conclude that a decision-strategy-attribution procedure may force participants to adopt an analytical decision-making strategy and rely mostly on conscious knowledge of artificial grammar.

Category learning in Alzheimer's disease and normal cognitive aging depends on initial experience of feature variability

Semantic category learning is dependent upon several factors, including the nature of the learning task, as well as individual differences in the quality and heterogeneity of exemplars that an individual encounters during learning. We trained healthy older adults (n=39) and individuals with a diagnosis of Alzheimer's disease or Mild Cognitive Impairment (n=44) to recognize instances of a fictitious animal, a "crutter". Each stimulus item contained 10 visual features (e.g., color, tail shape) which took one of two values for each feature (e.g., yellow/red, curly/straight tails). Participants were presented with a series of items (learning phase) and were either told the items belonged to a semantic category (explicit condition) or were told to think about the appearance of the items (implicit condition). Half of participants saw learning items with higher similarity to an unseen prototype (high typicality learning set), and thus lower between-item variability in their constituent features; the other half learned from items with lower typicality (low typicality learning set) and higher between-item feature variability. After the learning phase, participants were presented with test items one at a time that varied in the number of typical features from 0 (antitype) to 10 (prototype). We examined between-subjects factors of learning set (lower or higher typicality), instruction type (explicit or implicit), and group (patients vs. elderly control). Learning in controls was aided by higher learning set typicality: while controls in both learning set groups demonstrated significant learning, those exposed to a high-typicality learning set appeared to develop a prototype that helped guide their category membership judgments. Overall, patients demonstrated more difficulty with category learning than elderly controls. Patients exposed to the higher-typicality learning set were sensitive to the typical features of the category and discriminated between the most and least typical test items, although less reliably than controls. In contrast, patients exposed to the low-typicality learning set showed no evidence of learning. Analysis of structural imaging data indicated a positive association between left hippocampal grey matter density in elderly controls but a negative association in the patient group, suggesting differential reliance on hippocampal-mediated learning. Contrary to hypotheses, learning did not differ between explicit and implicit conditions for either group. Results demonstrate that category learning is improved when learning materials are highly similar to the prototype.

Compensatory processing during rule-based category learning in older adults

Healthy older adults typically perform worse than younger adults at rule-based category learning, but better than patients with Alzheimer's or Parkinson's disease. To further investigate aging's effect on rule-based category learning, we monitored event-related potentials (ERPs) while younger and neuropsychologically typical older adults performed a visual category-learning task with a rule-based category structure and trial-by-trial feedback. Using these procedures, we previously identified ERPs sensitive to categorization strategy and accuracy in young participants. In addition, previous studies have demonstrated the importance of neural processing in the prefrontal cortex and the medial temporal lobe for this task. In this study, older adults showed lower accuracy and longer response times than younger adults, but there were two distinct subgroups of older adults. One subgroup showed near-chance performance throughout the procedure, never categorizing accurately. The other subgroup reached asymptotic accuracy that was equivalent to that in younger adults, although they categorized more slowly. These two subgroups were further distinguished via ERPs. Consistent with the compensation theory of cognitive aging, older adults who successfully learned showed larger frontal ERPs when compared with younger adults. Recruitment of prefrontal resources may have improved performance while slowing response times. Additionally, correlations of feedback-locked P300 amplitudes with category-learning accuracy differentiated successful younger and older adults. Overall, the results suggest that the ability to adapt one's behavior in response to feedback during learning varies across older individuals, and that the failure of some to adapt their behavior may reflect inadequate engagement of prefrontal cortex.

The contribution of temporary storage and executive processes to category learning

Three distinctly different working memory processes, temporary storage, mental shifting and inhibition, were proposed to account for individual differences in category learning. A sample of 213 participants completed a classic category learning task and two working memory tasks that were experimentally manipulated for tapping specific working memory processes. Fixed-links models were used to decompose data of the category learning task into two independent components representing basic performance and improvement in performance in category learning. Processes of working memory were also represented by fixed-links models. In a next step the three working memory processes were linked to components of category learning. Results from modeling analyses indicated that temporary storage had a significant effect on basic performance and shifting had a moderate effect on improvement in performance. In contrast, inhibition showed no effect on any component of the category learning task. These results suggest that temporary storage and the shifting process play different roles in the course of acquiring new categories.

Dissociation of category-learning systems via brain potentials

Behavioral, neuropsychological, and neuroimaging evidence has suggested that categories can often be learned via either an explicit rule-based (RB) mechanism critically dependent on medial temporal and prefrontal brain regions, or via an implicit information-integration (II) mechanism relying on the basal ganglia. In this study, participants viewed sine-wave gratings (Gabor patches) that varied on two dimensions and learned to categorize them via trial-by-trial feedback. Two different stimulus distributions were used; one was intended to encourage an explicit RB process and the other an implicit II process. We monitored brain activity with scalp electroencephalography (EEG) while each participant: (1) passively observed stimuli represented of both distributions; (2) categorized stimuli from one distribution, and, 1 week later; (3) categorized stimuli from the other distribution. Categorization accuracy was similar for the two distributions. Subtractions of Event-Related Potentials (ERPs) for correct and incorrect trials were used to identify neural differences in RB and II categorization processes. We identified an occipital brain potential that was differentially modulated by categorization condition accuracy at an early latency (150-250 ms), likely reflecting the degree of holistic processing. A stimulus-locked Late Positive Complex (LPC) associated with explicit memory updating was modulated by accuracy in the RB, but not the II task. Likewise, a feedback-locked P300 ERP associated with expectancy was correlated with performance only in the RB, but not the II condition. These results provide additional evidence for distinct brain mechanisms supporting RB vs. implicit II category learning and use.

Continuous executive function disruption interferes with application of an information integration categorization strategy

Category learning is often characterized as being supported by two separate learning systems. A verbal system learns rule-defined (RD) categories that can be described using a verbal rule and relies on executive functions (EFs) to learn via hypothesis testing. A nonverbal system learns non-rule-defined (NRD) categories that cannot be described by a verbal rule and uses automatic, procedural learning. The verbal system is dominant in that adults tend to use it during initial learning but may switch to the nonverbal system when the verbal system is unsuccessful. The nonverbal system has traditionally been thought to operate independently of EFs, but recent studies suggest that EFs may play a role in the nonverbal system-specifically, to facilitate the transition away from the verbal system. Accordingly, continuously interfering with EFs during the categorization process, so that EFs are never fully available to facilitate the transition, may be more detrimental to the nonverbal system than is temporary EF interference. Participants learned an NRD or an RD category while EFs were untaxed, taxed temporarily, or taxed continuously. When EFs were continuously taxed during NRD categorization, participants were less likely to use a nonverbal categorization strategy than when EFs were temporarily taxed, suggesting that when EFs were unavailable, the transition to the nonverbal system was hindered. For the verbal system, temporary and continuous interference had similar effects on categorization performance and on strategy use, illustrating that EFs play an important but different role in each of the category-learning systems.