Stimulus modality interacts with category structure in perceptual category learning

Feedback and observational learning differ in effectiveness during category learning in early school aged children and adults

When learning new categories, do children benefit from the same types of training as adults? We compared the effects of feedback-based training with observational training in young adults (ages 18-25) and early school aged children (ages 6-7) across two different multimodal category learning tasks: conjunctive rule based and information integration. We used multimodal stimuli that varied across a visual feature (rotation speed of the "planet" stimulus) and an auditory feature (pitch frequency of a pure tone stimulus). We found an interaction between age and training type for the rule-based category task, such that adults performed better in feedback training than in observational training, whereas training type had no significant effect on children's category learning performance. Overall adults performed better than children in learning both the rule based and information integration category structures. In information integration category learning, feedback versus observational training did not have a significant effect on either adults' or children's category learning. Computational modelling revealed that children defaulted to univariate rules in both tasks. The finding that children do not benefit from feedback training and can learn successfully via observational learning has implications for the design of educational interventions appropriate for children.

Generalisation to novel exemplars of learned shape categories based on visual and auditory spatial cues does not benefit from multisensory information

Although the integration of information across multiple senses can enhance object representations in memory, how multisensory information affects the formation of categories is uncertain. In particular, it is unclear to what extent categories formed from multisensory information benefit object recognition over unisensory inputs. Two experiments investigated the categorisation of novel auditory and visual objects, with categories defined by spatial similarity, and tested generalisation to novel exemplars. Participants learned to categorise exemplars based on visual-only (geometric shape), auditory-only (spatially defined soundscape) or audio-visual spatial cues. Categorisation to learned as well as novel exemplars was then tested under the same sensory learning conditions. For all learning modalities, categorisation generalised to novel exemplars. However, there was no evidence of enhanced categorisation performance for learned multisensory exemplars. At best, bimodal performance approximated that of the most accurate unimodal condition, although this was observed only for a subset of exemplars within a category. These findings provide insight into the perceptual processes involved in the formation of categories and have relevance for understanding the sensory nature of object representations underpinning these categories.

Linear separability, irrelevant variability, and categorization difficulty

In rule-based (RB) category-learning tasks, the optimal strategy is a simple explicit rule, whereas in information-integration (II) tasks, the optimal strategy is impossible to describe verbally. This study investigates the effects of two different category properties on learning difficulty in category learning tasks-namely, linear separability and variability on stimulus dimensions that are irrelevant to the categorization decision. Previous research had reported that linearly separable II categories are easier to learn than nonlinearly separable categories, but Experiment 1, which compared performance on linearly and nonlinearly separable categories that were equated as closely as possible on all other factors that might affect difficulty, found that linear separability had no effect on learning. Experiments 1 and 2 together also established a novel dissociation between RB and II category learning: increasing variability on irrelevant stimulus dimensions impaired II learning but not RB learning. These results are all predicted by the best available measures of difficulty in RB and II tasks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Left Inferior Frontal Gyrus Integrates Multisensory Information in Category Learning

Humans are able to categorize things they encounter in the world (e.g., a cat) by integrating multisensory information from the auditory and visual modalities with ease and speed. However, how the brain learns multisensory categories remains elusive. The present study used functional magnetic resonance imaging to investigate, for the first time, the neural mechanisms underpinning multisensory information-integration (II) category learning. A sensory-modality-general network, including the left insula, right inferior frontal gyrus (IFG), supplementary motor area, left precentral gyrus, bilateral parietal cortex, and right caudate and globus pallidus, was recruited for II categorization, regardless of whether the information came from a single modality or from multiple modalities. Putamen activity was higher in correct categorization than incorrect categorization. Critically, the left IFG and left body and tail of the caudate were activated in multisensory II categorization but not in unisensory II categorization, which suggests this network plays a specific role in integrating multisensory information during category learning. The present results extend our understanding of the role of the left IFG in multisensory processing from the linguistic domain to a broader role in audiovisual learning.

Stimulus modality influences the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning

This study aimed to investigate whether stimulus modality influenced the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning and whether the use of the two strategies was supported by shared or separate neural substrates. To address these issues, we combined behavioral and fNIRS methods in a modified prototype distortion task in which each category member has one rule feature and ten similarity features, and each type of feature can be presented in either the visual modality or the auditory modality. The results in Experiment 1 revealed that the learning effect in the "auditory rule-visual similarity" condition was the highest among all four conditions; further analysis revealed that in the "auditory rule-visual similarity" condition, the number of participants who used the rule-based strategy was more than the number of participants who used the similarity-based strategy, and the learning effect was always much higher for the rule-based strategy than for the similarity-based strategy. The behavioral results in Experiment 2 replicated the main findings in Experiment 1, and the fNIRS results showed that the use of the visual rule-based strategy was mediated by the dorsolateral prefrontal cortex, whereas the use of the auditory similarity-based strategy mainly engaged in the superior temporal gyrus, and the use of the visual similarity-based strategy mainly engaged in the inferior temporal gyrus. The results in Experiment 3 revealed that when the stimuli had only one type of feature, the visual rule rather than the auditory rule was learned more easily. The results provide new evidence that the stimulus modality can influence the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning and that the use of the two types of strategies is supported by separate neural substrates both in the auditory modality and the visual modality.

Procedural-Memory, Working-Memory, and Declarative-Memory Skills Are Each Associated With Dimensional Integration in Sound-Category Learning

This paper investigates relationships between procedural-memory, declarative-memory, and working-memory skills and adult native English speakers' novel sound-category learning. Participants completed a sound-categorization task that required integrating two dimensions: one native (vowel quality), one non-native (pitch). Similar information-integration category structures in the visual and auditory domains have been shown to be best learned implicitly (e.g., Maddox et al., 2006). Thus, we predicted that individuals with greater procedural-memory capacity would better learn sound categories, because procedural memory appears to support implicit learning of new information and integration of dimensions. Seventy undergraduates were tested across two experiments. Procedural memory was assessed using a linguistic adaptation of the serial-reaction-time task (Misyak et al., 2010a,b). Declarative memory was assessed using the logical-memory subtest of the Wechsler Memory Scale-4th edition (WMS-IV; Wechsler, 2009). Working memory was assessed using an auditory version of the reading-span task (Kane et al., 2004). Experiment 1 revealed contributions of only declarative memory to dimensional integration, which might indicate not enough time or motivation to shift over to a procedural/integrative strategy. Experiment 2 gave twice the speech-sound training, distributed over 2 days, and also attempted to train at the category boundary. As predicted, effects of declarative memory were removed and effects of procedural memory emerged, but, unexpectedly, new effects of working memory surfaced. The results may be compatible with a multiple-systems account in which declarative and working memory facilitate transfer of control to the procedural system.

A role for consolidation in cross-modal category learning

The ability to categorize objects and events is a fundamental human skill that depends upon the representation of multimodal conceptual knowledge. This study investigated the acquisition and consolidation of categorical information that required participants to integrate information across visual and auditory dimensions. The impact of wake- and sleep-dependent consolidation was investigated using a paradigm in which training and testing were separated by a delay spanning either an evening of sleep or daytime wakefulness, with a paired-associate episodic memory task used as a measure of classic sleep-dependent consolidation. Participants displayed good evidence of category learning, but did not show any wake- or sleep-dependent changes in memory for category information immediately following the delay. This is in contrast to paired-associate learning, where a sleep-dependent benefit was observed in memory recall. To replicate real-world concept learning, in which knowledge is acquired across multiple distinct episodes, participants were given a second opportunity for category learning following the consolidation delay. Here we found an interaction between consolidation and learning; with greater improvements in category knowledge as a result of the second learning session for those participants who had a sleep-filled delay. These results suggest a role for sleep in the consolidation of recently acquired categorical knowledge; however this benefit does not emerge as an immediate benefit in memory recall, but by enhancing the effectiveness of future learning. This study therefore provides insights into the processes responsible for the formation and development of conceptual representations.

Performance Pressure Enhances Speech Learning

Real-world speech learning often occurs in high pressure situations such as trying to communicate in a foreign country. However, the impact of pressure on speech learning success is largely unexplored. In this study, adult, native speakers of English learned non-native speech categories under pressure or no-pressure conditions. In the pressure conditions, participants were informed that they were paired with a (fictitious) partner, and that each had to independently exceed a performance criterion for both to receive a monetary bonus. They were then informed that their partner had exceeded the bonus and the fate of both bonuses depended upon the participant's performance. Our results demonstrate that pressure significantly enhanced speech learning success. In addition, neurobiologically-inspired computational modeling revealed that the performance advantage was due to faster and more frequent use of procedural learning strategies. These results integrate two well-studied research domains and suggest a facilitatory role of motivational factors in speech learning performance that may not be captured in traditional training paradigms.

A Comparison of the neural correlates that underlie rule-based and information-integration category learning

The influential competition between verbal and implicit systems (COVIS) model proposes that category learning is driven by two competing neural systems-an explicit, verbal, system, and a procedural-based, implicit, system. In the current fMRI study, participants learned either a conjunctive, rule-based (RB), category structure that is believed to engage the explicit system, or an information-integration category structure that is thought to preferentially recruit the implicit system. The RB and information-integration category structures were matched for participant error rate, the number of relevant stimulus dimensions, and category separation. Under these conditions, considerable overlap in brain activation, including the prefrontal cortex, basal ganglia, and the hippocampus, was found between the RB and information-integration category structures. Contrary to the predictions of COVIS, the medial temporal lobes and in particular the hippocampus, key regions for explicit memory, were found to be more active in the information-integration condition than in the RB condition. No regions were more activated in RB than information-integration category learning. The implications of these results for theories of category learning are discussed. Hum Brain Mapp 37:3557-3574, 2016. © 2016 Wiley Periodicals, Inc.

Incidental auditory category learning

Very little is known about how auditory categories are learned incidentally, without instructions to search for category-diagnostic dimensions, overt category decisions, or experimenter-provided feedback. This is an important gap because learning in the natural environment does not arise from explicit feedback and there is evidence that the learning systems engaged by traditional tasks are distinct from those recruited by incidental category learning. We examined incidental auditory category learning with a novel paradigm, the Systematic Multimodal Associations Reaction Time (SMART) task, in which participants rapidly detect and report the appearance of a visual target in 1 of 4 possible screen locations. Although the overt task is rapid visual detection, a brief sequence of sounds precedes each visual target. These sounds are drawn from 1 of 4 distinct sound categories that predict the location of the upcoming visual target. These many-to-one auditory-to-visuomotor correspondences support incidental auditory category learning. Participants incidentally learn categories of complex acoustic exemplars and generalize this learning to novel exemplars and tasks. Further, learning is facilitated when category exemplar variability is more tightly coupled to the visuomotor associations than when the same stimulus variability is experienced across trials. We relate these findings to phonetic category learning.

The Role of Corticostriatal Systems in Speech Category Learning

One of the most difficult category learning problems for humans is learning nonnative speech categories. While feedback-based category training can enhance speech learning, the mechanisms underlying these benefits are unclear. In this functional magnetic resonance imaging study, we investigated neural and computational mechanisms underlying feedback-dependent speech category learning in adults. Positive feedback activated a large corticostriatal network including the dorsolateral prefrontal cortex, inferior parietal lobule, middle temporal gyrus, caudate, putamen, and the ventral striatum. Successful learning was contingent upon the activity of domain-general category learning systems: the fast-learning reflective system, involving the dorsolateral prefrontal cortex that develops and tests explicit rules based on the feedback content, and the slow-learning reflexive system, involving the putamen in which the stimuli are implicitly associated with category responses based on the reward value in feedback. Computational modeling of response strategies revealed significant use of reflective strategies early in training and greater use of reflexive strategies later in training. Reflexive strategy use was associated with increased activation in the putamen. Our results demonstrate a critical role for the reflexive corticostriatal learning system as a function of response strategy and proficiency during speech category learning.

Tests of a Dual-systems Model of Speech Category Learning

In the visual domain, more than two decades of work posits the existence of dual category learning systems. The reflective system uses working memory to develop and test rules for classifying in an explicit fashion. The reflexive system operates by implicitly associating perception with actions that lead to reinforcement. Dual-systems models posit that in learning natural categories, learners initially use the reflective system and with practice, transfer control to the reflexive system. The role of reflective and reflexive systems in second language (L2) speech learning has not been systematically examined. Here monolingual, native speakers of American English were trained to categorize Mandarin tones produced by multiple talkers. Our computational modeling approach demonstrates that learners use reflective and reflexive strategies during tone category learning. Successful learners use talker-dependent, reflective analysis early in training and reflexive strategies by the end of training. Our results demonstrate that dual-learning systems are operative in L2 speech learning. Critically, learner strategies directly relate to individual differences in category learning success.

Cross-modal information integration in category learning

An influential theoretical perspective describes an implicit category-learning system that associates regions of perceptual space with response outputs by integrating information preattentionally and predecisionally across multiple stimulus dimensions. In this study, we tested whether this kind of implicit, information-integration category learning is possible across stimulus dimensions lying in different sensory modalities. Humans learned categories composed of conjoint visual-auditory category exemplars comprising a visual component (rectangles varying in the density of contained lit pixels) and an auditory component (in Exp. 1, auditory sequences varying in duration; in Exp. 2, pure tones varying in pitch). The categories had either a one-dimensional, rule-based solution or a two-dimensional, information-integration solution. Humans could solve the information-integration category tasks by integrating information across two stimulus modalities. The results demonstrated an important cross-modal form of sensory integration in the service of category learning, and they advance the field's knowledge about the sensory organization of systems for categorization.

Prior experience with negative spectral correlations promotes information integration during auditory category learning

Complex sounds vary along a number of acoustic dimensions. These dimensions may exhibit correlations that are familiar to listeners due to their frequent occurrence in natural sounds-namely, speech. However, the precise mechanisms that enable the integration of these dimensions are not well understood. In this study, we examined the categorization of novel auditory stimuli that differed in the correlations of their acoustic dimensions, using decision bound theory. Decision bound theory assumes that stimuli are categorized on the basis of either a single dimension (rule based) or the combination of more than one dimension (information integration) and provides tools for assessing successful integration across multiple acoustic dimensions. In two experiments, we manipulated the stimulus distributions such that in Experiment 1, optimal categorization could be accomplished by either a rule-based or an information integration strategy, while in Experiment 2, optimal categorization was possible only by using an information integration strategy. In both experiments, the pattern of results demonstrated that unidimensional strategies were strongly preferred. Listeners focused on the acoustic dimension most closely related to pitch, suggesting that pitch-based categorization was given preference over timbre-based categorization. Importantly, in Experiment 2, listeners also relied on a two-dimensional information integration strategy, if there was immediate feedback. Furthermore, this strategy was used more often for distributions defined by a negative spectral correlation between stimulus dimensions, as compared with distributions with a positive correlation. These results suggest that prior experience with such correlations might shape short-term auditory category learning.

Dual systems of speech category learning across the lifespan

Although categorization is fundamental to speech processing, little is known about the learning systems that mediate auditory categorization and even less is known about changes across the life span. Vision research supports dual-learning systems that are grounded in neuroscience and are partially dissociable. The reflective, rule-based system is prefrontally mediated and uses working memory and executive attention to develop and test rules for classifying in an explicit fashion. The reflexive, information-integration system is striatally mediated and operates by implicitly associating perception with actions that lead to reinforcement. We examine the extent to which dual-learning systems mediate auditory and speech learning in younger and older adults. We examined auditory category learning when a rule-based strategy (Experiment 1) or information-integration strategy (Experiment 2) was optimal, and found an age-related rule-based deficit, but intact information-integration learning. Experiment 3 examined natural auditory category learning, and found an age-related performance deficit. Computational modeling suggested that this was attributable to older adults' persistent reliance on suboptimal, unidimensional strategies when 2-dimensional strategies were optimal. Working memory capacity was also found to be associated with improved rule-based and natural auditory category learning, but not information-integration category learning. These results suggest that dual-learning systems are operative in speech category learning across the life span, and that performance deficits, when present, are attributable to deficiencies in frontally mediated, rule-based processes.