Multiple stages of learning in perceptual categorization: evidence and neurocomputational theory

The tail of the caudate is sensitive to both gain and loss feedback during information integration categorization

Although most category learning studies use feedback for training, little attention has been paid to how individuals utilize feedback implemented as gains or losses during categorization. We compared skilled categorization under three different conditions: Gain (earn points for correct answers), Gain and Loss (earn points for correct answers and lose points for wrong answers) and Correct or Wrong (accuracy feedback only). We also manipulated difficulty and point value, with near boundary stimuli having the highest number of points to win or lose, and stimuli far from the boundary having the lowest point value. We found that the tail of the caudate was sensitive to feedback condition, with highest activity when both Gain and Loss feedback were present and least activity when only Gain or accuracy feedback was present. We also found that activity across the caudate was affected by distance from the decision bound, with greatest activity for the near boundary high value stimuli, and lowest for far low value stimuli. Overall these results indicate that the tail of the caudate is sensitive not only to positive rewards but also to loss and punishment, consistent with recent animal research finding tail of the caudate activity in aversive learning.

The role of posterior parietal cortex in detecting changes in feedback contingency

Well-practiced or learned behaviors are extremely resilient. For example, it is extremely difficult for a trained typist to forget how to use a keyboard configuration that they are familiar with. While they can be trained on a new keyboard configuration, the original skill quickly comes back when the old keyboard configuration is used again. This resiliency of learned skills is both a blessing and a curse. It makes useful skills durable, but it also makes maladaptive behaviors difficult to extinguish. Crossley et al. (2013) proposed a computational model and behavioral paradigm aimed at unlearning skills using various feedback contingency manipulations during an extinction phase. They showed that partially-valid feedback during extinction removed evidence for fast reacquisition, which they interpreted as evidence for unlearning. In this article, we replicated the Crossley et al. paradigm using fMRI. Univariate analyses showed differences in BOLD signals between the different experiment phases in the frontoparietal attention network. The superior and inferior parietal lobules (SPL and IPL, respectively) showed the largest cluster differences both between experimental phases and between extinction conditions. In contrast, the prefrontal cortex only showed differences in cluster of activities between extinction conditions. Multivariate pattern analysis was also used with seeds in the SPL and IPL. The results showed that these brain areas were critical in detecting changes in experimental phases. Overall, the fMRI results found mixed evidence for the Crossley et al. model and suggest that while unlearning prevents fast reacquisition, the absence of fast reacquisition does not necessarily implies that unlearning occurred.

Rule and Exemplar-based Transfer in Category Learning

We compared the neural systems involved in transfer to novel stimuli via rule application versus exemplar processing. Participants learned a categorization task involving abstraction of a complex rule and then categorized different types of transfer stimuli without feedback. Rule stimuli used new features and therefore could only be categorized using the rule. Exemplar stimuli included only one of the features necessary to apply the rule and therefore required participants to categorize based on similarity to individual previously learned category members. Consistent and inconsistent stimuli were formed so that both the rule and feature similarity indicated the same category (consistent) or opposite categories (inconsistent). We found that all conditions eliciting rule-based transfer recruited a medial prefrontal-anterior hippocampal network associated with schematic memory. In contrast, exemplar-based transfer recruited areas of the intraparietal sulcus associated with learning and executing stimulus-category mappings along with the posterior hippocampus. These results support theories of categorization that postulate complementary learning and generalization strategies based on schematic and exemplar mechanisms.

Applying Insights on Categorisation, Communication, and Dynamic Decision-Making: A Case Study of a ‘Simple’ Maritime Military Decision

A complete understanding of decision-making in military domains requires gathering insights from several fields of study. To make the task tractable, here we consider a specific example of short-term tactical decisions under uncertainty made by the military at sea. Through this lens, we sketch out relevant literature from three psychological tasks each underpinned by decision-making processes: categorisation, communication and choice. From the literature, we note two general cognitive tendencies that emerge across all three stages: the effect of cognitive load and individual differences. Drawing on these tendencies, we recommend strategies, tools and future research that could improve performance in military domains – but, by extension, would also generalise to other high-stakes contexts. In so doing, we show the extent to which domain general properties of high order cognition are sufficient in explaining behaviours in domain specific contexts.

Linear separability, irrelevant variability, and categorization difficulty

In rule-based (RB) category-learning tasks, the optimal strategy is a simple explicit rule, whereas in information-integration (II) tasks, the optimal strategy is impossible to describe verbally. This study investigates the effects of two different category properties on learning difficulty in category learning tasks-namely, linear separability and variability on stimulus dimensions that are irrelevant to the categorization decision. Previous research had reported that linearly separable II categories are easier to learn than nonlinearly separable categories, but Experiment 1, which compared performance on linearly and nonlinearly separable categories that were equated as closely as possible on all other factors that might affect difficulty, found that linear separability had no effect on learning. Experiments 1 and 2 together also established a novel dissociation between RB and II category learning: increasing variability on irrelevant stimulus dimensions impaired II learning but not RB learning. These results are all predicted by the best available measures of difficulty in RB and II tasks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Category learning in a recurrent neural network with reinforcement learning

It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

Differing effects of gain and loss feedback on rule-based and information-integration category learning

Although most category-learning studies use feedback for training, little attention has been paid to how individuals use feedback value and framing of feedback as gains or losses to support learning. We compared learning of rule-based (RB) and information-integration (II) categories with point-valued feedback in which participants gained or lost higher point values for more difficult category members (those closer to the decision bound). We implemented point-valued feedback in four different conditions: Gain (earn points for correct answers), Loss (lose points for incorrect answers), Gain+Loss (earn points for correct answers and lose points for incorrect answers), and Control (accuracy feedback only without point gain or loss). Participants were trained until they reached criterion. Overall, point-valued feedback led to better learning than control conditions. However, the patterns differed across category-learning tasks. In the II task participants reached learning criterion fastest when they received both Gains and Losses. This is consistent with the reliance of II learning on reinforcement-based mechanisms and research showing common coding of gains and losses in neural regions underlying II learning. In contrast, in the RB task, participants reached criterion most rapidly when they received either Gains or Losses, but not both Gains and Losses together. The relative impairment in the Gain+Loss condition in RB learning may be due to conflicting executive function demands for interpreting and using the separate Gain and Loss information, and is consistent with reliance of RB learning on explicit hypothesis-testing mechanisms.

Left Inferior Frontal Gyrus Integrates Multisensory Information in Category Learning

Humans are able to categorize things they encounter in the world (e.g., a cat) by integrating multisensory information from the auditory and visual modalities with ease and speed. However, how the brain learns multisensory categories remains elusive. The present study used functional magnetic resonance imaging to investigate, for the first time, the neural mechanisms underpinning multisensory information-integration (II) category learning. A sensory-modality-general network, including the left insula, right inferior frontal gyrus (IFG), supplementary motor area, left precentral gyrus, bilateral parietal cortex, and right caudate and globus pallidus, was recruited for II categorization, regardless of whether the information came from a single modality or from multiple modalities. Putamen activity was higher in correct categorization than incorrect categorization. Critically, the left IFG and left body and tail of the caudate were activated in multisensory II categorization but not in unisensory II categorization, which suggests this network plays a specific role in integrating multisensory information during category learning. The present results extend our understanding of the role of the left IFG in multisensory processing from the linguistic domain to a broader role in audiovisual learning.

The role of differential outcomes-based feedback on procedural memory

In categorization tasks, two memory systems may be involved in the learning of categories: one explicit and rule-based system and another implicit and procedure-based system. Learning of rule-based categories relies on some form of explicit reasoning, whereas procedural memory underlies information-integration category-learning tasks, in which performance is maximized only if information of two (or more) dimensions is integrated. The present study aimed at investigating the role of how feedback is administered, whether differential or non-differential, in procedural learning. An information-integration category-learning task was designed, where the to-be-categorized stimuli differed in two dimensions. Participants were randomly assigned to two groups: one group received the reinforcers for correct categorizations differentially, one for each category (the differential outcomes procedure, DOP), and the other group received the reinforcers randomly (the non-differential outcomes procedure, NOP). The participants of the DOP group showed better procedural learning in the categorization task, compared to the NOP group. Moreover, the analysis of learning strategies revealed that more participants developed more optimal strategies in the DOP group than in the NOP group. These results extend the benefits of the differential outcomes-based feedback to non-declarative memory tasks and help better understand the role of feedback in procedural learning.

A difficulty predictor for perceptual category learning

Predicting human performance in perceptual categorization tasks in which category membership is determined by similarity has been historically difficult. This article proposes a novel biologically motivated difficulty measure that can be generalized across stimulus types and category structures. The new measure is compared to 12 previously proposed measures on four extensive data sets that each included multiple conditions that varied in difficulty. The studies were highly diverse and included experiments with both continuous- and binary-valued stimulus dimensions, a variety of different stimulus types, and both linearly and nonlinearly separable categories. Across these four applications, the new measure was the most successful at predicting the observed rank ordering of conditions by difficulty, and it was also the most accurate at predicting the numerical values of the mean error rates in each condition.

Novel representations that support rule-based categorization are acquired on-the-fly during category learning

Humans learn categorization rules that are aligned with separable dimensions through a rule-based learning system, which makes learning faster and easier to generalize than categorization rules that require integration of information from different dimensions. Recent research suggests that learning to categorize objects along a completely novel dimension changes its perceptual representation, making it more separable and discriminable. Here, we asked whether such newly learned dimensions could support rule-based category learning. One group received extensive categorization training and a second group did not receive such training. Later, both groups were trained in a task that made use of the category-relevant dimension, and then tested in an analogical transfer task (Experiment 1) and a button-switch interference task (Experiment 2). We expected that only the group with extensive pre-training (with well-learned dimensional representations) would show evidence of rule-based behavior in these tasks. Surprisingly, both groups performed as expected from rule-based learning. A third experiment tested whether a single session (less than 1 h) of training in a categorization task would facilitate learning in a task requiring executive function. There was a substantial learning advantage for a group with brief pre-training with the relevant dimension. We hypothesize that extensive experience with separable dimensions is not required for rule-based category learning; rather, the rule-based system may learn representations "on the fly" that allow rule application. We discuss what kind of neurocomputational model might explain these data best.

Retinal-specific category learning

Virtually all cognitive theories of category learning (such as prototype theory^1-5 and exemplar theory^6-8) view this important skill as a high-level process that uses abstract representations of objects in the world. Because these representations are removed from visual characteristics of the display, such theories suggest that category learning occurs in higher-level (such as association) areas and therefore should be immune to the visual field dependencies that characterize processing of objects mediated by representations in low-level visual areas. Here we challenge that view by describing a fully controlled demonstration of visual-field dependence in category learning. Eye-tracking was used to control gaze while participants either learned rule-based categories known to recruit prefrontal-based explicit reasoning, or information-integration categories known to depend on basal-ganglia-mediated procedural learning⁹. Results showed that learning was visual-field dependent with information-integration categories, but we found no evidence of visual-field dependence with rule-based categories. A theoretical interpretation of this difference is offered in terms of the underlying neurobiology. Finally, these results are situated within the broad perceptual-learning literature in an attempt to motivate further research on the similarities and differences between category and perceptual learning.

Quantitative modeling of category learning deficits in various patient populations

OBJECTIVE

To provide a select review of our applications of quantitative modeling to highlight the utility of such approaches to better understand the neuropsychological deficits associated with various neurologic and psychiatric diseases.

METHOD

We review our work examining category learning in various patient populations, including individuals with basal ganglia disorders (Huntington's Disease and Parkinson's disease), amnesia and Eating Disorders.

RESULTS

Our review suggests that the use of quantitative models has enabled a better understanding of the learning deficits often observed in these conditions and has allowed us to form novel hypotheses about the neurobiological bases of their deficits.

CONCLUSIONS

We feel that the use of neurobiologically inspired quantitative modeling holds great promise in neuropsychological assessment and that future clinical measures should incorporate the use of such models as part of their standard scoring. Appropriate studies need to be completed, however, to determine whether such modeling techniques adhere to the rigorous psychometric properties necessary for a valid and reliable application in a clinical setting. (PsycINFO Database Record

A neural interpretation of exemplar theory

Exemplar theory assumes that people categorize a novel object by comparing its similarity to the memory representations of all previous exemplars from each relevant category. Exemplar theory has been the most prominent cognitive theory of categorization for more than 30 years. Despite its considerable success in providing good quantitative fits to a wide variety of accuracy data, it has never had a detailed neurobiological interpretation. This article proposes a neural interpretation of exemplar theory in which category learning is mediated by synaptic plasticity at cortical-striatal synapses. In this model, categorization training does not create new memory representations, rather it alters connectivity between striatal neurons and neurons in sensory association cortex. The new model makes identical quantitative predictions as exemplar theory, yet it can account for many empirical phenomena that are either incompatible with or outside the scope of the cognitive version of exemplar theory. (PsycINFO Database Record

Perceptual category learning and visual processing: An exercise in computational cognitive neuroscience

The field of computational cognitive neuroscience (CCN) builds and tests neurobiologically detailed computational models that account for both behavioral and neuroscience data. This article leverages a key advantage of CCN-namely, that it should be possible to interface different CCN models in a plug-and-play fashion-to produce a new and biologically detailed model of perceptual category learning. The new model was created from two existing CCN models: the HMAX model of visual object processing and the COVIS model of category learning. Using bitmap images as inputs and by adjusting only a couple of learning-rate parameters, the new HMAX/COVIS model provides impressively good fits to human category-learning data from two qualitatively different experiments that used different types of category structures and different types of visual stimuli. Overall, the model provides a comprehensive neural and behavioral account of basal ganglia-mediated learning.

Trial-by-trial switching between procedural and declarative categorization systems

Considerable evidence suggests that human category learning recruits multiple memory systems. A popular assumption is that procedural memory is used to form stimulus-to-response mappings, whereas declarative memory is used to form and test explicit rules about category membership. The multiple systems framework has been successful in motivating and accounting for a broad array of empirical observations over the past 20 years. Even so, only a couple of studies have examined how the different categorization systems interact. Both previous studies suggest that switching between explicit and procedural responding is extremely difficult. But they leave unanswered the critical questions of whether trial-by-trial system switching is possible, and if so, whether it is qualitatively different than trial-by-trial switching between two explicit tasks. The experiment described in this article addressed these questions. The results (1) confirm that effective trial-by-trial system switching, although difficult, is possible; (2) suggest that switching between tasks mediated by different memory systems is more difficult than switching between two declarative memory tasks; and (3) point to a serious shortcoming of current category-learning theories.

What is automatized during perceptual categorization?

An experiment is described that tested whether stimulus-response associations or an abstract rule are automatized during extensive practice at perceptual categorization. Twenty-seven participants each completed 12,300 trials of perceptual categorization, either on rule-based (RB) categories that could be learned explicitly or information-integration (II) categories that required procedural learning. Each participant practiced predominantly on a primary category structure, but every third session they switched to a secondary structure that used the same stimuli and responses. Half the stimuli retained their same response on the primary and secondary categories (the congruent stimuli) and half switched responses (the incongruent stimuli). Several results stood out. First, performance on the primary categories met the standard criteria of automaticity by the end of training. Second, for the primary categories in the RB condition, accuracy and response time (RT) were identical on congruent and incongruent stimuli. In contrast, for the primary II categories, accuracy was higher and RT was lower for congruent than for incongruent stimuli. These results are consistent with the hypothesis that rules are automatized in RB tasks, whereas stimulus-response associations are automatized in II tasks. A cognitive neuroscience theory is proposed that accounts for these results.

Categorical evidence, confidence, and urgency during probabilistic categorization

We used a temporally extended categorization task to investigate the neural substrates underlying our ability to integrate information over time and across multiple stimulus features. Using model-based fMRI, we tracked the temporal evolution of two important variables as participants deliberated about impending choices: (1) categorical evidence, and (2) confidence (the total amount of evidence provided by the stimuli, irrespective of the particular category favored). Importantly, in each model, we also included a covariate that allowed us to differentiate signals related to information accumulation from other, evidence-independent functions that increased monotonically with time (such as urgency or cognitive load). We found that somatomotor regions tracked the temporal evolution of categorical evidence, while regions in both medial and lateral prefrontal cortex, inferior parietal cortex, and the striatum tracked decision confidence. As both theory and experimental work suggest that patterns of activity thought to be related to information-accumulation may reflect, in whole or in part, an interaction between sensory evidence and urgency, we additionally investigated whether urgency might modulate the slopes of the two evidence-dependent functions. We found that the slopes of both functions were likely modulated by urgency such that the difference between the high and low evidence states increased as the response deadline loomed.

Generalization in category learning: the roles of representational and decisional uncertainty

Effective generalization in a multiple-category situation involves both assessing potential membership in individual categories and resolving conflict between categories while implementing a decision bound. We separated generalization from decision bound implementation using an information integration task in which category exemplars varied over two incommensurable feature dimensions. Human subjects first learned to categorize stimuli within limited training regions, and then, during fMRI scanning, they also categorized transfer stimuli from new regions of perceptual space. Transfer stimuli differed both in distance from the training region prototype and distance from the decision bound, allowing us to independently assess neural systems sensitive to each. Across all stimulus regions, categorization was associated with activity in the extrastriate visual cortex, basal ganglia, and the bilateral intraparietal sulcus. Categorizing stimuli near the decision bound was associated with recruitment of the frontoinsular cortex and medial frontal cortex, regions often associated with conflict and which commonly coactivate within the salience network. Generalization was measured in terms of greater distance from the decision bound and greater distance from the category prototype (average training region stimulus). Distance from the decision bound was associated with activity in the superior parietal lobe, lingual gyri, and anterior hippocampus, whereas distance from the prototype was associated with left intraparietal sulcus activity. The results are interpreted as supporting the existence of different uncertainty resolution mechanisms for uncertainty about category membership (representational uncertainty) and uncertainty about decision bound (decisional uncertainty).

Answering questions about consciousness by modeling perception as covert behavior

Two main open questions in current consciousness research concern (i) the neural correlates of consciousness (NCC) and (ii) the relationship between neural activity and first-person, subjective experience. Here, possible answers are sketched for both of these, by means of a model-based analysis of what is required for one to admit having a conscious experience. To this end, a model is proposed that allows reasoning, albeit necessarily in a simplistic manner, about all of the so called “easy problems” of consciousness, from discrimination of stimuli to control of behavior and language. First, it is argued that current neuroscientific knowledge supports the view of perception and action selection as two examples of the same basic phenomenon, such that one can meaningfully refer to neuronal activations involved in perception as covert behavior. Building on existing neuroscientific and psychological models, a narrative behavior model is proposed, outlining how the brain selects covert (and sometimes overt) behaviors to construct a complex, multi-level narrative about what it is like to be the individual in question. It is hypothesized that we tend to admit a conscious experience of X if, at the time of judging consciousness, we find ourselves acceptably capable of performing narrative behavior describing X. It is argued that the proposed account reconciles seemingly conflicting empirical results, previously presented as evidence for competing theories of consciousness, and suggests that well-defined, experiment-independent NCCs are unlikely to exist. Finally, an analysis is made of what the modeled narrative behavior machinery is and is not capable of. It is discussed how an organism endowed with such a machinery could, from its first-person perspective, come to adopt notions such as “subjective experience,” and of there being “hard problems,” and “explanatory gaps” to be addressed in order to understand consciousness.